Overview

CloningDCB is a dataset of synthetic driving sequences generated using the CARLA simulator. The driving tasks are performed by 40 individuals on both dynamic and static driving platforms. Driving is not random but based on orchestrated scenarios.

Each sequence includes RGB images accompanied by standard ground truth data (depth, optical flow, semantic and instance segmentation), ego-vehicle information, and, most notably, eye-tracking and EEG recordings.

Best guidance for training

CloningDCB dataset helps to reduce the amount of driving hours required for a sensorimotor model lo "learn" what to consider in a situation to make a decision.

Open for research and commercial purposes

CloningDCB may be used for research and commercial purposes. It is released publicly under the Creative Commons Attribution-Commercial-ShareAlike 4.0 license. For detailed information, please check our terms of use.

Ground-truth annotations

CloningDCB comes with photorealistic color images, per-pixel semantic segmentation, depth, instance panoptic segmentation, optical flow and CANBUS information. To see some examples of per-pixel ground-truth, please check our examples of annotations.

Diversity of scenarios

CloningDCB features more than 40 hours of curated driving scenarios and free driving performed by human drivers in simulators of two of the most recognised research centers in Spain.

5 weather variations for every driving scenario are also provided.

Annotations (Ground-Truth)

CloningDCB brings per-pixel ground-truth semantic segmentation, scene depth, instance panoptic segmentation, optical flow and real eye tracking. Check some of our examples:

Semantic Segmentation
Semantic Segmentation
Depth
Optical Flow
Eye Tracking
Panoptic Instance Segmentation

Download

The CloningDCB dataset is provided free of charge to academic and non-academic entities to support research, scientific publication, teaching, but also for commercial purposes.

Remember to check our terms of use. If you find the CloningDCB Dataset useful for your work, we kindly ask you to cite our white paper.

Data Size Details Links
RGB Images 21 GB The images have a resolution of 2048 * 1024 and are encoded in png format. [Mirror 1] [Mirror 2]
Semantic Segmentation 386 MB The data is stored in grayscale format using the 19 cityscapes ids for training. [Mirror 1] [Mirror 2]
Semantic Segmentation (Color) 392 MB The data is stored in RGB using Cityscape's color convention. [Mirror 1] [Mirror 2]
Depth 59 GB The depth is saved in EXR format with 32 bits (float) per channel. The measurement units are 1e5 meters. [Mirror 1] [Mirror 2]
Panoptic Instance Segmentation 102 MB These labels are provided only for dynamic classes (vehicles, riders and pedestrians). [Mirror 1] [Mirror 2]
Bounding Boxes 6 MB These labels are provided only for dynamic classes (vehicles, riders and pedestrians). [Mirror 1] [Mirror 2]
Camera metadata 1 KB Camera parameter information [Mirror 1]

How to use it

  • RGB:

    Contains RGB images with a resolution of 960x540 in JPG format.

  • SS:

    Contains the pixel-level semantic segmentation labels in color, in PNG format. We follow the 19 training classes defined on Cityscapes.

  • OF:

    Contains the optical flow of the image, with respect to the previous one in PNG format.

  • IS:

    Contains the instance segmentation of the dynamic objects of the image in PNG format. Each instance is codified using the RGB channels, where RG corresponds to the instance number and B to the class ID. Dynamic objects are Person, Rider, Car, Truck, Bus, Train, Motorcycle and Bicycle.

  • CB:

    Contains the CANBUS information of the road at every timestamp, at a frequency of 25 Hertz. We provide the annotations in a json file with the next structure:

    • speed: the vehicle's real speed as modulus of the speed vector in that moment.
    • acceleration: the vehicle's real acceleration in that moment, considering the current throttle and brake .
    • brake: the negative acceleration over the car caused by the brake pedal
    • throttle: the positive acceleration over the car caused by the throttle pedal
    • steer: the angle of the steering wheel.
    • ego position: current X, Y, Z coordinates of the vehicle in the world
    • gyroscope: current rotations in all 3 axis (X, Y, X) of the vehicle
    • accelerometer: current accelerations over the 3 axis (X, Y, Z) of the vehicle in that moment
    • hand_brake: flag indicating if the hand brake is active
    • rear_gear: flag indicating if the rear gear is selected
    • blinker: -1: left, 1: right, 0 otherwise
    • direction: The current command to execute. 1- "turn left", 2- "turn right", 3- "go straight", 4- "follow the road", 5- "change lane on left", 6- "change lane on right"
    • tag: identificator of current scenario or if the maneuver is valid or not

  • Depth:

    Contains the depth map of the image as png codified in CARLA format. read this for more info

Terms of Use

The CloningDCB Dataset is provided by the Computer Vision Center (UAB).

CloningDCB may be used for research and commercial purposes, and it is subject to the Creative Commons Attribution-Commercial-ShareAlike 4.0. A summary of the CC-BY-SA 4.0 licensing terms can be found here.

Due to constraints from our asset providers for CloningDCB, we prohibit the use of generative AI technologies for reverse engineering any assets or creating content for stock media platforms based on the CloningDCB dataset.

While we strive to generate precise data, all information is presented 'as is' without any express or implied warranties. We explicitly disclaim all representations and warranties regarding the validity, scope, accuracy, completeness, safety, or utility of the licensed content, including any implied warranties of merchantability, fitness for a particular purpose, or otherwise.

Privacy Policy

CloningDCB.org uses Google Analytics to collect data. We need this data to understand how you use our website so we can improve its design and functionality.

With your consent, Google Analytics will process and collect your personal data (cookies and IP address) to give us valuable information. Google Analytics will transfer your data to the United States and store it for 2 months. To learn more about Google's data transfer policies, click here.

Standard contractual clauses are legal clauses written by the European Commission. They are part of a contract between Google Ireland Ltd. and Google LLC, and Google LLC must follow them. Standard contractual clauses tell Google LLC what it can and cannot do with your data

You have certain rights over your data: for example, you can require us to delete them or to provide you with a copy. We take responsibility for the processing of your data. We are available to answer any question and handle any request from you. Please use contact@cloningdcb.org if you we want to get in touch with us.

Principal Investigators

Name & Surname Center
Antonio M. López CVC & UAB
Aura Hernández CVC & UAB

Team

Surname Name Center
Abad Rubén CVC & UAB
Borràs Agnés CVC
Cano Pau CVC & UAB
Contreras Ainoa CVC & UAB
Gil Débora CVC & UAB
Levy Alexandre F. CVC
Porres Diego CVC
Sánchez Carles CVC
Sánchez Gemma CVC & UAB
Serrat Joan CVC & UAB
Villalonga Gabriel CVC & UAB
Xiao Yi CVC & UAB

Contact

If you have any questions about the dataset, please feel free to contact us using this email: contact@cloningdcb.org

Acknowledgements

  • Funded by:

    • Project TED2021-132802BI00 funded by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR

  • Personal acknowledgments:

    • Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program. CVC's authors acknowledge the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities.

  • Developed by: