RGB - D斯拉姆数据集和基准(RGB-D SLAM Dataset and Benchmark)

RGB - D斯拉姆数据集和基准(RGB-D SLAM Datasetand Benchmark)数据介绍：We provide a large dataset containing RGB-D data and ground-truth data with the goal to establish a novel benchmark for the evaluation of visual odometry and visual SLAM systems. Our dataset contains the color and depth images of a Microsoft Kinect sensor along theground-truth trajectory of the sensor. The data was recorded at full frame rate (30 Hz) and sensor resolution (640×480). The ground-truth trajectory was obtained from a high-accuracy motion-capture system with eight high-speed tracking cameras (100 Hz). Further, we provide the accelerometer data from the Kinect. Finally, we propose an evaluation criterion for measuring the quality of the estimated camera trajectory of visual SLAM systems.关键词：RGB-D,地面实况,基准,测程,轨迹,RGB-D,ground-truth,benchmark,odometry,trajectory,数据格式：IMAGE数据详细介绍：RGB-D SLAM Dataset and BenchmarkContact: Jürgen SturmWe provide a large dataset containing RGB-D data and ground-truth data with the goal to establish a novel benchmark for the evaluation of visual odometry and visual SLAM systems. Our dataset contains the color and depth images of a Microsoft Kinect sensor along the ground-truth trajectory of the sensor. The data was recorded at full frame rate (30 Hz) and sensor resolution (640×480). The ground-truth trajectory was obtained from a high-accuracy motion-capture system with eight high-speed tracking cameras (100 Hz). Further, we provide the accelerometer data from the Kinect. Finally, we propose an evaluation criterion for measuring the quality of the estimated camera trajectory of visual SLAM systems.How can I use the RGB-D Benchmark to evaluate my SLAM system?1. Download one or more of the RGB-D benchmark sequences (fileformats, useful tools)2. Run your favorite visual odometry/visual SLAM algorithm (for example,RGB-D SLAM)3. Save the estimated camera trajectory to a file (file formats, exampletrajectory)4. Evaluate your algorithm by comparing the estimated trajectory with theground truth trajectory. We provide an automated evaluation tool to help you with the evaluation. There is also an online version of the tool. Further remarksJose Luis Blanco has added our dataset to the mobile robot programming toolkit (MRPT) repository. The dataset (including example code and tools) can be downloaded here.∙If you have any questions about the dataset/benchmark/evaluation/file formats, please don't hesitate to contact Jürgen Sturm.∙We are happy to share our data with other researchers. Please refer to the respective publication when using this data.Related publications2011Conference and Workshop PapersReal-Time Visual Odometry from Dense RGB-D Images (F. Steinbruecker, J. Sturm, D. Cremers), In Workshop on Live Dense Reconstruction with Moving Cameras at the Intl. Conf. on Computer Vision (ICCV), 2011. [bib] [pdf] Towards a benchmark for RGB-D SLAM evaluation (J. Sturm, S. Magnenat, N. Engelhard, F. Pomerleau, F. Colas, W. Burgard, D. Cremers, R. Siegwart), In Proc. of the RGB-D Workshop on Advanced Reasoning with Depth Cameras at Robotics: Science and Systems Conf. (RSS), 2011. [bib] [pdf] [pdf]Real-time 3D visual SLAM with a hand-held camera (N. Engelhard, F. Endres, J. Hess, J. Sturm, W. Burgard), In Proc. of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, 2011. [bib] [pdf] [video] [video] [video]File FormatsWe provide the RGB-D datasets from the Kinect in the following format:Color images and depth mapsWe provide the time-stamped color and depth images as a gzipped tar file (TGZ).∙The color images are stored as 640×480 8-bit RGB images in PNG format.∙The depth maps are stored as 640×480 16-bit monochrome images in PNG format.∙The color and depth images are already pre-registered using the OpenNI driver from PrimeSense, i.e., the pixels in the color and depthimages correspond already 1:1.∙The depth images are scaled by a factor of 5000, i.e., a pixel value of 5000 in the depth image corresponds to a distance of 1 meter from thecamera, 10000 to 2 meter distance, etc. A pixel value of 0 meansmissing value/no data.Ground-truth trajectoriesWe provide the groundtruth trajectory as a text file containing the translation and orientation of the camera in a fixed coordinate frame. Note that also our automatic evaluation tool expects both the groundtruth and estimated trajectory to be in this format.∙Each line in the text file contains a single pose.∙The format of each line is 'timestamp tx ty tz qx qy qz qw'∙timestamp (float) gives the number of seconds since the Unix epoch.∙tx ty tz (3 floats) give the position of the optical center of the color camera with respect to the world origin as defined by the motion capture system.∙qx qy qz qw (4 floats) give the orientation of the optical center of the color camera in form of a unit quaternion with respect to the world origin as defined by the motion capture system.∙The file may contain comments that have to start with ”#”.Intrinsic Camera Calibration of the KinectThe Kinect has a factory calibration stored onboard, based on a high level polynomial warping function. The OpenNI driver uses this calibration for undistorting the images, and for registering the depth images (taken by the IR camera) to the RGB images. Therefore, the depth images in our datasets are reprojected into the frame of the color camera, which means that there is a 1:1 correspondence between pixels in the depth map and the color image.The conversion from the 2D images to 3D point clouds works as follows. Note that the focal lengths (fx/fy), the optical center (cx/cy), the distortion parameters (d0-d4) and the depth correction factor are different for each camera. The Python code below illustrates how the 3D point can be computed from the pixel coordinates and the depth value:fx = 525.0 # focal length xfy = 525.0 # focal length ycx = 319.5 # optical center xcy = 239.5 # optical center yds = 1.0 # depth scalingfactor = 5000 # for the 16-bit PNG files# OR: factor = 1 # for the 32-bit float images in the ROS bag filesfor v in range(depth_image.height):for u in range(depth_image.width):Z = (depth_image[v,u] / factor) * ds;X = (u - cx) * Z / fx;Y = (v - cy) * Z / fy;Note that the above script uses the default (uncalibrated) intrinsic parameters. The intrinsic parameters for the Kinects used in the fr1 and fr2 dataset are as follows:Calibration of the color cameraWe computed the intrinsic parameters of the RGB camera from thergbd_dataset_freiburg1/2_rgb_calibration.bag.Camera fx fy cx cy d0 d1 d2 d3 d4(ROS525.0 525.0 319.5 239.5 0.0 0.0 0.0 0.0 0.0 default)Freiburg 1517.3 516.5 318.6 255.3 0.2624 -0.9531 -0.0054 0.0026 1.1633 RGBFreiburg 2520.9 521.0 325.1 249.7 0.2312 -0.7849 -0.0033 -0.0001 0.9172 RGBCalibration of the depth imagesWe verified the depth values by comparing the reported depth values to the depth estimated from the RGB checkerboard. In this experiment, we found that the reported depth values from the Kinect were off by a constant scaling factor, as given in the following table:Camera dsFreiburg 1 Depth 1.035Freiburg 2 Depth 1.031Calibration of the infrared cameraWe also provide the intrinsic parameters for the infraredcamera. Note that the depth images provided in our dataset are alreadypre-registered to the RGB images. Therefore, rectifying the depth images based on the intrinsic parameters is not straight forward.Camera fx fy cx cy d0 d1 d2 d3 d4 Freiburg 1 IR591.1 590.1 331.0 234.0 -0.0410 0.3286 0.0087 0.0051 -0.5643Freiburg 2 IR580.8 581.8 308.8 253.0 -0.2297 1.4766 0.0005 -0.0075 -3.4194 Movies for visual inspectionFor visual inspection of the individual datasets, we also provide movies of the Kinect (RGB and depth) and of an external camcorder. The movie format is mpeg4 stored in an AVI container.Alternate file formatsROS bagFor people using ROS, we also provide ROS bag files that contain the color images, monochrome images, depth images, camera infos, point clouds and transforms – including the groundtruth transformation from the /world frame all in a single file. The bag files (ROS diamondback) contain the following message topics:∙/camera/depth/camera_info (sensor_msgs/CameraInfo) contains the intrinsic camera parameters for the depth/infrared camera, asreported by the OpenNI driver∙/camera/depth/image (sensor_msgs/Image) contains the depth map ∙/camera/rgb/camera_info (sensor_msgs/CameraInfo) contains the intrinsic camera parameters for the RGB camera, as reported by theOpenNI driver∙/camera/rgb/image_color (sensor_msgs/Image) contains the color image from the RGB camera∙/imu (sensor_msgs/Imu), contains the accelerometer data from the Kinect∙/tf (tf/tfMessage), contains:o the ground-truth data from the mocap (/world to /Kinect)o the calibration betwenn mocap and the optical center of the Kinect's color camera (/Kinect to /openni_camera),o and the ROS-specific, internal transformations (/openni_camera to /openni_rgb_frame to /openni_rgb_optical_frame).If you need the point clouds and monochrome images, you can use the adding_point_clouds_to_ros_bag_files script to add them:∙/camera/rgb/image_mono (sensor_msgs/Image) contains the monochrome image from the RGB camera∙/camera/rgb/points (sensor_msgs/PointCloud2) contains the colored point clouds/camera/depth/points (sensor_msgs/PointCloud2) contains the point cloudMobile Robot Programming Toolkit (MRPT)Jose Luis Blanco has added our dataset to the mobile robot programming toolkit (MRPT) repository. The dataset (including example code and tools) can be downloaded hereUseful tools for the RGB-D benchmarkWe provide a set of tools that can be used to pre-process the datasets and to evaluate the SLAM/tracking results. The scripts can be downloaded here.To checkout the repository using subversion, runsvn checkouthttps://svncvpr.in.tum.de/cvpr-ros-pkg/trunk/rgbd_benchmark/rgbd_benchmar k_toolsAssociating color and depth imagesThe Kinect provides the color and depth images in an un-synchronized way. This means that the set of time stamps from the color images do not intersect with those of the depth images. Therefore, we need some way of associating color images to depth images.For this purpose, you can use the ''associate.py'' script. It reads the time stamps from the rgb.txt file and the depth.txt file, and joins them by finding the best matches.usage: associate.py [-h] [--first_only] [--offset OFFSET][--max_difference MAX_DIFFERENCE]first_file second_fileThis script takes two data files with timestamps and associates them positional arguments:first_file first text file (format: timestamp data)second_file second text file (format: timestamp data)optional arguments:-h, --help show this help message and exit--first_only only output associated lines from first file--offset OFFSET time offset added to the timestamps of the second file(default: 0.0)--max_difference MAX_DIFFERENCEmaximally allowed time difference for matching entries(default: 0.02)EvaluationAfter estimating the camera trajectory of the Kinect and saving it to a file, we need to evaluate the error in the estimated trajectory by comparing it with the ground-truth. There are different error metrics. Two prominent methods is the absolute trajectory error (ATE) and the relative pose error (RPE). The ATE is well-suited for measuring the performance of visual SLAM systems. In contrast, the RPE is well-suited for measuring the drift of a visual odometry system, for example the drift per second.For both metrics, we provide automated evaluation scripts that can be downloaded here. Note that there is also an online version available on our website. Both trajectories have to be stored in a text file (format: 'timestamp tx ty tz qx qy qz qw', more information). For comparison, we offer a set of trajectories from RGBD-SLAM.Absolute Trajectory Error (ATE)The absolute trajectory error directly measures the difference between points of the true and the estimated trajectory. As a pre-processing step, we associate the estimated poses with ground truth poses using the timestamps. Based on this association, we align the true and the estimated trajectory using singular value decomposition. Finally, we compute the difference between each pair of poses, and output the mean/median/standard deviation of these differences. Optionally, the script can plot both trajectories to a png or pdf file. usage: evaluate_ate.py [-h] [--offset OFFSET] [--scale SCALE][--max_difference MAX_DIFFERENCE] [--save SAVE][--save_associations SAVE_ASSOCIATIONS][--plot PLOT][--verbose]first_file second_fileThis script computes the absolute trajectory error from the ground truthtrajectory and the estimated trajectory.positional arguments:first_file first text file (format: timestamp tx ty tz qx qy qzqw)second_file second text file (format: timestamp tx ty tz qx qy qzqw)optional arguments:-h, --help show this help message and exit--offset OFFSET time offset added to the timestamps of the second file(default: 0.0)--scale SCALE scaling factor for the second trajectory (default:1.0)--max_difference MAX_DIFFERENCEmaximally allowed time difference for matching entries(default: 0.02)--save SAVE save aligned second trajectory to disk (format: stamp2x2 y2 z2)--save_associations SAVE_ASSOCIATIONSsave associated first and aligned second trajectory todisk (format: stamp1 x1 y1 z1 stamp2 x2 y2 z2)--plot PLOT plot the first and the aligned second trajectory to animage (format: png)--verbose print all evaluation data (otherwise, only the RMSEabsolute translational error in meters after alignmentwill be printed)Relative Pose Error (RPE)For computing the relative pose error, we provide a script ''evaluate_rpe.py''. This script computes the error in the relative motion between pairs of timestamps. By default, the script computes the error between all pairs of timestamps in the estimated trajectory file. As the number of timestamp pairs in the estimated trajectory is quadratic in the length of the trajectory, it can make sense to downsample this set to a fixed number (–max_pairs). Alternatively, one can choose to use a fixed window size (–fixed_delta). In this case, each pose in the estimated trajectory is associated with a later pose according to the window size (–delta) and unit (–delta_unit). This evaluation technique is useful for estimating the drift.usage: evaluate_rpe.py [-h] [--max_pairs MAX_PAIRS] [--fixed_delta][--delta DELTA] [--delta_unit DELTA_UNIT][--offset OFFSET] [--scale SCALE] [--save SAVE][--plot PLOT] [--verbose]groundtruth_file estimated_fileThis script computes the relative pose error from the ground truth trajectory and the estimated trajectory.positional arguments:groundtruth_file ground-truth trajectory file (format: "timestamp tx tytz qx qy qz qw")estimated_file estimated trajectory file (format: "timestamp tx ty tzqx qy qz qw")optional arguments:-h, --help show this help message and exit--max_pairs MAX_PAIRSmaximum number of pose comparisons (default: 10000,set to zero to disable downsampling)--fixed_delta only consider pose pairs that have a distance of deltadelta_unit (e.g., for evaluating the drift persecond/meter/radian)--delta DELTA delta for evaluation (default: 1.0)--delta_unit DELTA_UNITunit of delta (options: 's' for seconds, 'm' formeters, 'rad' for radians, 'f' for frames; default:'s')--offset OFFSET time offset between ground-truth and estimatedtrajectory (default: 0.0)--scale SCALE scaling factor for the estimated trajectory (default:1.0)--save SAVE text file to which the evaluation will be saved(format: stamp_est0 stamp_est1 stamp_gt0stamp_gt1trans_error rot_error)--plot PLOT plot the result to a file (requires --fixed_delta,output format: png)--verbose print all evaluation data (otherwise, only the meantranslational error measured in meters will beprinted)Generating a point cloud from imagesThe depth images are already registered to the color images, so the pixels in the depth image already correspond one-to-one to the pixels in the color image. Therefore, generating colored point clouds is straight-forward. An example script is available in ''generate_pointcloud.py'', that takes a color image and a depth map as input, and generates a point cloud file in the PLY format. This format can be read by many 3D modelling programs, for example meshlab. You can download meshlab for Windows, Mac and Linux.usage: generate_pointcloud.py [-h] rgb_file depth_file ply_fileThis script reads a registered pair of color and depth images and generates a colored 3D point cloud in the PLY format.positional arguments:rgb_file input color image (format:png)depth_file input depth image (format: png)ply_file output PLY file (format: ply)optional arguments:-h, --help show this help message and exitAdding point clouds to ROS bag filesOn the download page, we already provide ROS bag files with added point clouds for the datasets for visual inspection in RVIZ. Because of the large size of the resulting files, we downsampled these bag files to 2 Hz. In case that you want to generate ROS bag files that contain the point clouds for all images (at 30 Hz), you can use the ''add_pointclouds_to_bagfile.py'' script.usage: add_pointclouds_to_bagfile.py [-h] [--start START][--duration DURATION] [--nth NTH][--skip SKIP] [--compress]inputbag [outputbag]This scripts reads a bag file containing RGBD data, adds the corresponding PointCloud2 messages, and saves it again into a bag file. Optional argumentsallow to select only a portion of the original bag file.positional arguments:inputbag input bag fileoutputbag output bag fileoptional arguments:-h, --help show this help message and exit--start START skip the first N seconds of input bag file (default:0.0)--duration DURATION only process N seconds of input bag file (default: off)--nth NTH only process every N-th frame of input bag file(default: 15)--skip SKIP skip N blocks in the beginning (default: 1)--compress compress output bag fileVisualizing the datasets in RVIZRVIZ is the standard visualization tool in ROS. It can be easily adapted to display many different messages. In particular, it can be used to display the point clouds from a ROS bag file. For this, run (in three different consoles) roscorerosrun rviz rvizrosbag play rgbd_dataset_freiburg1_xyz-2hz-with-pointclouds.bagIf this is the first launch, you will have to enable the built-in displays (Menu –> Plugins –> Check “Loaded” for the builtin plugins). In the displays tab, set the “fixed frame” to ”/world”. Click on “Add”, and select the PointCloud2 display, and set topic to ”/camera/rgb/points”. To show the colors, change “color transformer” to “RGB8” in the point cloud display and the “style”to “points”. If you want, you can set the decay time to a suitable value, for example 5 seconds, to accumulate the points in the viewer as they come in. The result should then look as follows:数据预览：点此下载完整数据集。