Integrated 2D/3D Camera

The purpose of the BMBF-project MIDIAS is the development of a new integrated camera system which combines a common gray value camera image (2D image) and a depth image to a single image with depth information for each gray value pixel (3D image). Automatic fusion of relatively coarse depth data and a high-resolution 2D image within a single camera system is a significant progress in camera technology. The integrated 3D camera helps to solve many problems within the domain of automatic understanding of traffic situations.

Fusion of the two sensor sources results in a gain in resolution of the whole camera system. Furthermore an automatic model-independent segmentation of 2D pictures is possible. Hence, the data provided by such a camera is not only of interest in the area of automotive applications, but also for general spatial vision tasks in the field of presence control or gesture recognition.

Partial schedule for FORWISS

The output of the algorithms to be developed by FORWISS is a fused image, where each gray value pixel is extended by depth information. These low level picture processing algorithms should be integrated into the 3D camera. This requires a maximum in robustness for the algorithms, which will be finally implemented into programmable hardware by SIEMENS CT.

For this purpose three partial goals are to be realized:

  1. Calibration of the 2D/3D-sensor system (spatial relation of the low resolution 3D image data to the gray value pixel grid)
  2. Model-independent data fusion of 2D image data and depth data at pixel level
  3. Specification and development of a 3D-image exchange format

Project staff

Project partners