LeRobot Integration

rosbag_to_lerobot tool

rosbag_to_lerobot is used to convert ROS 2 rosbag2 data collected by robots into LeRobot V3 datasets. The tool parses robot state, action, and camera topics based on YAML configurations for different models, and performs time alignment, resampling, and dataset generation.

Features

• Input formats: A rosbag2 directory, metadata.yaml, an .mcap or .db3 data file in the same directory, or a .tar, .tar.gz, or .tgz archive containing one rosbag2 dataset.

• Supported robot types: Conversion configurations are currently provided for Quanta X1, Quanta X2, and Desktop.

• Topic configuration: YAML files define the state, action, and camera data sources for each robot type. Do not hard-code topics in the conversion code.

• Time synchronization: The converter calculates the common time range across all valid data streams, then aligns and resamples states, actions, and images at the target FPS.

• Episodes: The tool can convert a single rosbag or organize batch data and platform task data into multiple episodes.

• Output format: Low-dimensional state and action data is stored in Parquet files, camera data is stored in MP4 files, and the metadata required by LeRobot V3 is generated.

Current delivery status

tools/rosbag_to_lerobot/ has been implemented in the SDK internal codebase and passed basic functional verification, but the publicly exported package of SDK v1.1.2 does not yet include this tool.

Therefore, the following directory structure and commands are provided to illustrate the interface and workflow of the current conversion tool. They can only be executed directly after obtaining an SDK version containing rosbag_to_lerobot or an officially released toolkit.

Tool directory

tools/rosbag_to_lerobot/
├── scripts/
│   ├── convert_rosbag_to_lerobot.py
│   ├── convert_rosbag_to_lerobot_v3_batch.py
│   └── convert_platform_task_to_lerobot_v3.py
├── config/
│   ├── quanta_x1/
│   ├── quanta_x2/
│   └── desktop/
├── requirements.txt
├── README.md
└── README_CN.md

Dependency installation

The current tool pins LeRobot 0.4.2. After the tool is officially released, install dependencies from the bundled requirements.txt whenever possible. Download the LeRobot v0.4.2 installation package and install it as follows:

python -m pip install ./lerobot-0.4.2-py3-none-any.whl

python -m pip install -r tools/rosbag_to_lerobot/requirements.txt

Input data

A standard rosbag2 directory typically contains:

my_rosbag/
├── metadata.yaml
└── xxx.mcap

Or:

my_rosbag/
├── metadata.yaml
└── xxx.db3

metadata.yaml is used to describe the rosbag2 dataset, and .mcap / .db3 store actual ROS 2 message data.

The tool also supports .tar, .tar.gz, and .tgz archives containing a complete rosbag2 dataset.

Quick Start

After obtaining the SDK containing the tool, execute the following in the tools/rosbag_to_lerobot directory:

python scripts/convert_rosbag_to_lerobot.py     --bag-path /path/to/rosbag2     --output-dir /path/to/lerobot_data     --repo-id "my_org/my_dataset"     --config config/quanta_x1/lerobot_v3_16d.yaml

Parameters

For other optional parameters, refer to the output of executing --help with the current tool.

Robot Configurations

The current tool provides the following model configurations:

• Quanta X1: Left and right robot-arm end-effector poses, left and right grippers, head state, three camera streams, and other data.

• Quanta X2: Wrist poses, gripper states, and camera data configured for the corresponding gripper type.

• Desktop: Left and right robot arms, grippers, and camera data.

Refer to the YAML file for the corresponding robot model for the specific topic names, data types, and field mappings.

Quanta X1 data structure

The current LeRobot V3 configuration for Quanta X1 uses a 16-dimensional state/action representation, mainly including:

• Left arm end-effector position and orientation;

• Left gripper status;

• Right arm end-effector position and orientation;

• Right gripper status;

• Head pitch and yaw.

Field names, order, and units are subject to the corresponding YAML configuration and the metadata of the converted dataset.

Time synchronization and action construction

Different robot data sources may have different sampling frequencies. The conversion tool first calculates the common time interval of valid data streams, and then resamples each data source according to the target FPS, so that images, state, action, and timestamp in the same LeRobot frame are aligned to a unified timeline.

The default configuration can use the state at the next timestep to construct the current action:

action[t] = state[t + 1]

Actions can also be constructed from independent action data sources based on the model configuration. Refer to the action configuration in the corresponding YAML file for the actual usage.

Output directory structure

The converted LeRobot V3 dataset typically contains:

lerobot_data/
├── meta/
│   ├── info.json
│   ├── stats.json
│   ├── tasks.parquet
│   └── episodes/
│       └── chunk-000/
│           └── file-000.parquet
├── data/
│   └── chunk-000/
│       └── file-000.parquet
└── videos/
    └── observation.images.<camera_name>/
        └── chunk-000/
            └── file-000.mp4

• data/: Low-dimensional time-series data such as states, actions, and timestamps.

• videos/: Camera videos.

• meta/info.json: Information such as dataset features, FPS, and data version.

• meta/stats.json: Data statistics.

• meta/tasks.parquet: Task information.

• meta/episodes/: Episode metadata.

Batch conversion

When you need to batch process multiple rosbag data files, you can use:

scripts/convert_rosbag_to_lerobot_v3_batch.py

Batch conversion is suitable for organizing multiple collection results into multiple episodes.

Post-Conversion Checklist

After completing the conversion, it is recommended to check at least the following:

1. Whether meta, data, and videos in the output directory are complete;

2. Whether the number of episodes and frame counts meet expectations;

3. Whether camera videos can be played normally;

4. Whether the shape, order, and units of state and action are consistent with the robot model configuration;

5. Whether images, state, action, and timestamps are correctly aligned;

6. Whether there are obvious frame drops, abnormal timestamps, or inconsistent data lengths;

7. Whether task information corresponds to the actual episode.

Differences from SDK DataCollector JSON conversion

The SDK also officially supports the DataCollector → collected_data JSON → tools/convert_to_lerobot.py conversion workflow.

This workflow and the rosbag2 → rosbag_to_lerobot → LeRobot V3 workflow introduced in this section are two different data pipelines.

The input directories, conversion scripts, configuration methods, and command-line arguments of the two workflows must not be mixed. For the conversion method of SDK DataCollector JSON data, please refer to the corresponding description in "Data Collection Example".