HALCON 在3D视觉及机器人中的应用介绍

Gripping of car door
Pose recognition based on edge and color Information
Which method suits best for determining the pose of the metal clamps?
The following questions make it easier to choose the right method:
Surface-based 3D matching
Multi-view stereo
HALCON supports multi-view stereo
Reconstruct 3D scenes from multiple images
Reconstruct 3D scenes from multiple images
HALCON allows to use the full range of sensors to extract 3D information
Time of flight
Sheet of light Structured light
3D
DFF
Stereo Sheet of light
Photometric stereo
Most 3D methods are based on the same principles as human 3D vision
Binocular stereo offers a flexible solution for 3D surface measurement
Bin picking of metal cylinders
3D sensors
Βιβλιοθήκη Baidu
HALCON supports various 3D sensors that can be used with surface-based 3D matching
Time of flight Laser triangulation (incl. Laser) Fringe projection
Best basis for identification applications
Simply faster development
Powerful 3D vision
3D vision
New 3D features
Multi-view calibration
3D object model
Primitives fitting
These 3D methods are also known, but not very common in machine vision
Depth from … Defocus Motion
Shadows
Texture
Perspective
Moiré
Sheet of light
Sheet of light
Primitive parameters
Sphere
Center Radius
Cylinder
Center Direction Radius
Plane
aX + bY + cZ + d = 0
The XYZ-mapping is needed for triangulation
HALCON allows to use the full range of sensors to extract 3D information
Game industry will provide cheap depth sensors to the vision industry
Image © Microsoft
Surface-based 3D matching is robust
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even at (outside) the image border
Surface-based 3D matching is fast and offers the full pose range
Picking of piled tubes with cylinder fitting
Surface-based matching
HALCON offers 3D to 3D surface-based matching
HALCON offers a new 3D matching technology: Surface-based 3D matching
Pose from Points
Pose from Contours
Matching
HALCON supports descriptor-based matching
What do I want to measure and how accurately?
What are the characteristics of the object?
Best basis for identification applications
Simply faster development
HALCON 10 in a nutshell
Powerful 3D vision
First class matching in any dimension
Faster and smarter
What are the setup requirements?
Pose recognition based on features
Pose recognition of metal parts with only one camera
Pose recognition based on edge information
What do I want to measure and how accurately? What are the characteristics of the object? What are the setup requirements?
3D matching: Model generation
Surface-based 3D matching is robust – even with low resolution
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with low resolution
Engine part
Depth image from TOF camera (176 x 144) Object size about 70 x 45
Surface-based 3D matching is robust – even with low resolution (TOF data)
Localization of a known object and control of the gripping process of the robot
Match 2D representation Refine position for high precision Control robot movement
Pose
Process
There is a variety of methods to measure surfaces in 3D
Stereo
Sheet of Light
Time of Flight
Photometric Stereo
Interferometry
Focus Determination
Before
The XYZ-mapping is needed for triangulation
After
Fit primitives
Segment and fit primitives in 3D data
1. Segmentation
2. Fitting
The XYZ-mapping is needed for triangulation
The surface measurement is performed by recording successive profiles
Structured light “parallelizes” the sheet of light principle
If the geometry is known, the 3D pose can be determined directly
© 2003-2010 MVTec Software GmbH
3D Vision and Applications in Robotics
Dr. Olaf Munkelt Harbin, China, 2011
Overview
3D is required in many application areas in different industries
Surface-based 3D matching using Kinect
Image © Microsoft
Bin picking of smoothly shaped objects using surface-based 3D matching
iStock_000002103108Small We.wmv
The Control Loop
Machine vision application
HALCON
Standard PC Intelligent camera embedded system
result
hole 1 2 3 radius 3.15 mm 3.05 mm 2.79 mm
What do I want to measure and how accurately?
What are the characteristics of the object?
What are the setup requirements?
Two alternative approaches can be selected
xyz_to_object_model_3d
prepare_object_model_3d
segment_object_model_3d
fit_primitives_object_model_3d
Sheet of light combined with the segmentation and fitting of the 3D data allows flexible picking
Reconstruction
Matching
… or both approches can be combined!
HALCON 10 in a nutshell
Powerful 3D vision
First class matching in any dimension
Faster and smarter
3D object model
HALCON offers a new data structure for 3D data
HALCON offers a new data structure for 3D data
Object model 3D Points (X,Y,Z) Point normals Triangles Polygons Primitive parameters xyz-Mapping Matching information