# HTO

#### High-Tech-Offensive Bayern: Software Toolbox for Embedded Applications

The focus of the project was to realize a visual programming toolbox, applicable in embedded systems. In order to show how easy it is to adopt future machines to the specifications of various customers by using the developed components, an Internet surf terminal was used as a demonstrator.

Within the scope of the project `High-Tech-Offensive Bayern' various reusable software components (modules) were implemented by FORWISS Passau in the area of optical metrology and image processing for the graphical programming system ICONNECT of the company Micro Epsilon. These components are applicable to complex machines that have to fulfill image processing tasks.

In detail modules on the following topics were developed:

- Subpixel image reconstruction
- Prototype fitting
- Stereo vision
- 3D Matching
- Reconstruction of the spacial plane of observed circles
- Plane reconstruction from monocular image sequences

Subpixel Image Reconstruction |

A sequence of digital low resolution images (image 1, image 2, ..) is obtained by taking different pictures of a template image that was slightly displaced in a predefined manner. The reconstructed images (1-3) were calculated from the low resolution images by means of the developed numerical method. In case of the reconstructed images, different reconstruction parameters were applied. So, parts of the information contained in the image sequence can be restored.

Prototype Matching |

The automatic identification and the determination of the position of known objects (prototypes) in digital images is called "prototype matching". Prototypes can be given as grayvalue images, color images or by a CAD layout. It is possible that the objects in the images occur in a translated, rotated or scaled manner. Sometimes the prototypes are perspectively distorted. Also partial occlussions of the objects and changes of lighting variations have to be considered.

There are two possible methods to determine the transformation of the prototype. The first one computes the grayvalue correlation of the specified images. The second one calculates significant characteristics both for the search image and the sample image.

Stereo vision |

Retrieving as much information as possible from two images of one 3D-Scene without any previous knowledge was the main focus of the modules for image processing in the range of stereo vision. Good correspondences (pixel coordinates of the same 3D-point in the two images) have to be found. Based on the determined correspondences, the epipolar geometry can be estimated, giving information on the relative position and orientation of the two cameras.

A projective reconstruction that preserves straight lines and planes, can be computed from the dedicated pixels. In most situations, the determination of the calibration data and the positions of the two cameras is expensive or even impossible (example: moving robots). Then these algorithms offer the facility of orientation and navigation.

3D Matching |

The goal of the modul 3D Matching is to reconstruct the geometric relationship between two sets of clouds of points, that is to calculate a rotation and translation in order to match the clouds of points as best as possible. For this purpose it is necessary to know (at least for some points) the correspondence of points from the first cloud to the points of the second cloud.

Two clouds of points before and after matching.

Reconstruction of the spacial plane of observed circles |

In some applications of image processing the geometry of observed objects is already known, e.g. traffic signs or circles on calibration targets. The image of a circle is an ellipse. From the shape of the ellipse and the intrinsic camera parameters it is possible to calculate the normal vector of the plane of the circle. Indeed, there are two possible normal vectors. If the radius of the real circle is known in addition, the distance of the circle can be determined.

An ellipse in the plane z=1 is shown in black. Rays from the origin through the ellipse define an elliptic cone, four of these are coloured in green and cyan. The two blue circles belong to this cone. The normal vectors of the planes of the circles are shown in red.

Plane Reconstruction |

Many objects in the real world can be modelled with planes, for example streets, houses or walls in rooms. Therefore in many situations in digital image processing it is known, that points lie in a plane, but the exact position of this plane is unknown. The developed algorithm allows to compute the normal vector of a plane out of two images. The distance of this plane cannot be determined, no matter how many points are available.