HALCON在D视觉及机器人中的应用介绍
合集下载
相关主题
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
object?
What are the setup requirements?
Pose recognition based on features
Pose recognition of metal parts with only one camera
Pose recognition based on edge information
Sheet of light
The surface measurement is performed by recording successive profiles
Structured light “parallelizes” the sheet of light principle
If the geometry is known, the 3D pose can be determined directly
Best basis for
identification applications
Simply faster development
Powerful 3D vision
3D vision
New 3D features
Multi-view calibration
3D object model
Primitives fitting
HALCON allows to use the full range of sensors to extract 3D information
Time of flight Sheet of light Structured light
DFF Stereo Sheet of light Photometric stereo
Primitive parameters xyz-Mapping
Matching information
Primitive parameters
Sphere
Center Radius
Cylinder
Center Direction Radius
Plane
aX + bY + cZ + d = 0
1. Segmentation 2. Fitting
The XYZ-mapping is needed for triangulation
xyz_to_object_model_3d prepare_object_model_3d
segment_object_model_3d fit_primitives_object_model_3d
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
iStock_000002103108Small We.wmv
The Control Loop
Machine vision application
HALCON
Standard PC Intelligent camera embedded system
result
hole radius
1
3.15 mm
HALCON offers 3D to 3D surface-based matching
HALCON offers a new 3D matching technology: Surface-based 3D matching
Surface-based 3D matching is robust
Reconstruction
Matching
… or both approches can be combined!
HALCON 10 in a nutshell
Powerful 3D vision
First class matching
in any dimension
Faster and smarter
Gripping of car door
Pose recognition based on edge and color Information
Which method suits best for determining the pose of the metal clamps?
The following questions make it easier to choose the right method:
3D
HALCON allows to use the full range of sensors to extract 3D information
Game industry will provide cheap depth sensors to the vision industry
Image © Microsoft
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with low resolution
Sheet of light combined with the segmentation and fitting of the 3D data allows flexible picking
Picking of piled tubes with cylinder fitting
Surface-based matching
Surface-based 3D matching
Multi-view stereo
HALCON supports multi-view stereo
Reconstruct 3D scenes from multiple images
Reconstruct 3D scenes from multiple images
© 2003-2010 MVTec Software GmbH
3D Vision and Applications in Robotics
Dr. Olaf Munkelt Harbin, China, 2011
Overview
3D is required in many application areas in different industries
Image © Microsoft
Bin picking of smoothly shaped objects using surface-based 3D matching
Completeness check with surface-based 3D matching
Bin picking of metal cylinders
What do I want to measure and how
accurately?
What are the characteristics of the
object?
What are the setup requirements?
Two alternative approaches can be selected
3D sensors
HALCON supports various 3D sensors that can be used with surface-based 3D matching
Time of flight
Laser triangulation (incl. Laser) Fringe projection
Pose from Points
Pose from Contours
Matching
HALCON supports descriptor-based matching
What do I want to measure and how
accurately?
What are the characteristics of the
Engine part
Depth image from TOF camera (176 x 144)
Object size about 70 x 45
Surface-based 3D matching is robust – even with low resolution (TOF data)
Surface-based 3D matching is robust – even with low resolution
2
3.05 mm
3
2.79 mm
Pose
Process
There is a variety of methods to measure surfaces in 3D
Stereo
Sheet of Light
Time of Flight
Photometric Stereo
Interferometry
Focus
Match 2D representation
Refine position for high precision
Control robot movement
Most 3D methods are based on the same principles as human 3D vision
Binocular stereo offers a flexible solution for 3D surface measurement
Best basis for
identification applications
Simply faster development
HALCON 10 in a nutshell
Powerful 3D vision
First class matchingarter
What do I want to measure and how
accurately?
What are the characteristics of the
object?
What are the setup requirements?
3D matching: Model generation
Localization of a known object and control of the gripping process of the robot
Surface-based 3D matching is robust – even at (outside) the image border
Surface-based 3D matching is fast and offers the full pose range
Surface-based 3D matching using Kinect
Determination
These 3D methods are also known, but not very common in machine vision
Depth from …
Defocus
Motion
Shadows
Texture
Perspective
Moiré
Sheet of light
3D object model
HALCON offers a new data structure for 3D data
HALCON offers a new data structure for 3D data
Object model 3D Points (X,Y,Z) Point normals Triangles Polygons
The XYZ-mapping is needed for triangulation
Before
The XYZ-mapping is needed for triangulation
After
Fit primitives
Segment and fit primitives in 3D data
What are the setup requirements?
Pose recognition based on features
Pose recognition of metal parts with only one camera
Pose recognition based on edge information
Sheet of light
The surface measurement is performed by recording successive profiles
Structured light “parallelizes” the sheet of light principle
If the geometry is known, the 3D pose can be determined directly
Best basis for
identification applications
Simply faster development
Powerful 3D vision
3D vision
New 3D features
Multi-view calibration
3D object model
Primitives fitting
HALCON allows to use the full range of sensors to extract 3D information
Time of flight Sheet of light Structured light
DFF Stereo Sheet of light Photometric stereo
Primitive parameters xyz-Mapping
Matching information
Primitive parameters
Sphere
Center Radius
Cylinder
Center Direction Radius
Plane
aX + bY + cZ + d = 0
1. Segmentation 2. Fitting
The XYZ-mapping is needed for triangulation
xyz_to_object_model_3d prepare_object_model_3d
segment_object_model_3d fit_primitives_object_model_3d
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
Surface-based 3D matching is robust – even with occlusion
iStock_000002103108Small We.wmv
The Control Loop
Machine vision application
HALCON
Standard PC Intelligent camera embedded system
result
hole radius
1
3.15 mm
HALCON offers 3D to 3D surface-based matching
HALCON offers a new 3D matching technology: Surface-based 3D matching
Surface-based 3D matching is robust
Reconstruction
Matching
… or both approches can be combined!
HALCON 10 in a nutshell
Powerful 3D vision
First class matching
in any dimension
Faster and smarter
Gripping of car door
Pose recognition based on edge and color Information
Which method suits best for determining the pose of the metal clamps?
The following questions make it easier to choose the right method:
3D
HALCON allows to use the full range of sensors to extract 3D information
Game industry will provide cheap depth sensors to the vision industry
Image © Microsoft
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with erroneous data
Surface-based 3D matching is robust – even with low resolution
Sheet of light combined with the segmentation and fitting of the 3D data allows flexible picking
Picking of piled tubes with cylinder fitting
Surface-based matching
Surface-based 3D matching
Multi-view stereo
HALCON supports multi-view stereo
Reconstruct 3D scenes from multiple images
Reconstruct 3D scenes from multiple images
© 2003-2010 MVTec Software GmbH
3D Vision and Applications in Robotics
Dr. Olaf Munkelt Harbin, China, 2011
Overview
3D is required in many application areas in different industries
Image © Microsoft
Bin picking of smoothly shaped objects using surface-based 3D matching
Completeness check with surface-based 3D matching
Bin picking of metal cylinders
What do I want to measure and how
accurately?
What are the characteristics of the
object?
What are the setup requirements?
Two alternative approaches can be selected
3D sensors
HALCON supports various 3D sensors that can be used with surface-based 3D matching
Time of flight
Laser triangulation (incl. Laser) Fringe projection
Pose from Points
Pose from Contours
Matching
HALCON supports descriptor-based matching
What do I want to measure and how
accurately?
What are the characteristics of the
Engine part
Depth image from TOF camera (176 x 144)
Object size about 70 x 45
Surface-based 3D matching is robust – even with low resolution (TOF data)
Surface-based 3D matching is robust – even with low resolution
2
3.05 mm
3
2.79 mm
Pose
Process
There is a variety of methods to measure surfaces in 3D
Stereo
Sheet of Light
Time of Flight
Photometric Stereo
Interferometry
Focus
Match 2D representation
Refine position for high precision
Control robot movement
Most 3D methods are based on the same principles as human 3D vision
Binocular stereo offers a flexible solution for 3D surface measurement
Best basis for
identification applications
Simply faster development
HALCON 10 in a nutshell
Powerful 3D vision
First class matchingarter
What do I want to measure and how
accurately?
What are the characteristics of the
object?
What are the setup requirements?
3D matching: Model generation
Localization of a known object and control of the gripping process of the robot
Surface-based 3D matching is robust – even at (outside) the image border
Surface-based 3D matching is fast and offers the full pose range
Surface-based 3D matching using Kinect
Determination
These 3D methods are also known, but not very common in machine vision
Depth from …
Defocus
Motion
Shadows
Texture
Perspective
Moiré
Sheet of light
3D object model
HALCON offers a new data structure for 3D data
HALCON offers a new data structure for 3D data
Object model 3D Points (X,Y,Z) Point normals Triangles Polygons
The XYZ-mapping is needed for triangulation
Before
The XYZ-mapping is needed for triangulation
After
Fit primitives
Segment and fit primitives in 3D data