Technical Details

Simple to start. Precise once you're in it.

Our data collection stack is built around letting you feel as comfortable controlling the robot as possible.

INPUTQuest controllers
→
CONTROLROS 2 arm & camera nodes
→
CAPTURECollection GUI
→
STORAGEPer-run .mcap rosbags
→
CONVERTLeRobot dataset
→
SHAREHugging Face
ARMS

xArm-series robotic arms, controlled in Cartesian space

CAMERAS

Intel RealSense, RGB + aligned depth per arm and top-down

INPUT

Meta Quest Controllers with custom button mapping

Quest controllers, headset off

Teleoperation runs on standard Quest controllers — but you don't wear the headset. Set it down where it can see the controllers, and it becomes a tracking reference rather than something strapped to your face for hours of collection.

Cartesian Online Trajectory Planning

Arms are controlled in Cartesian space using dynamic online planning — a following mode built by UFactory for sensor-driven and reactive tasks, where new target coordinates stream in continuously rather than being interpolated ahead of time as a fixed trajectory. This creates fast and responsive arm movements, allowing for precise manipulation even in dynamic environments.

One interface for the whole session

Everything an operator needs — arm state, recording controls, labeling, camera feeds, and system health — lives in a single collection GUI.

collection_gui — session 0042

ARM STATUS

Left armREADY
Right armREADY
Clear errors

QUEST STATUS

HeadsetREADY
Left controllerREADY
Right controllerREADY

DATA LABELING

pickplacefoldfailure+ add

QUEST CONTROLLERS

Scale (L / R)0.7× / 1.2×
Active StateActive / Non-Active

CAMERA FEEDS

DepthActive
DepthActive
DepthActive
DepthActive

DATA RECORDING

Current rosbagrun_017.mcap
StartR Save↵ Skip⌫ E-stopS

Built for efficiency, not just capture

Collection is optimized to make the most of your system's throughput:

Published Topics

A representative topic set from a two-arm rig, grouped by what they carry. Quest controller input topics live under the /quest/ namespace.

Commanded and achieved poses (target_frame, actual_pose) are both expressed in each arm's base frame.

CAMERAS — per camera (left, right, top, base)

/camera_*/color/image_rawsensor_msgs/Image · RGB
/camera_*/aligned_depth_to_color/image_rawsensor_msgs/Image · depth, aligned
/camera_*/color/camera_infosensor_msgs/CameraInfo
/camera_*/aligned_depth_to_color/camera_infosensor_msgs/CameraInfo
/camera_*/color/metadatacamera driver metadata msg
/camera_*/extrinsics/depth_to_colorcamera driver extrinsics msg

ROBOT STATE

/joint_statessensor_msgs/JointState · pos · vel · effort, 100 Hz
/tf, /tf_statictf2_msgs/TFMessage
/robot_descriptionstd_msgs/String · URDF

QUEST INPUT

/quest_left_hand_pose, /quest_right_hand_posegeometry_msgs/PoseStamped
/quest_left_hand_inputs, /quest_right_hand_inputscustom msg · buttons & triggers

ARM & GRIPPER CONTROL

/{l,r}_arm_ctrl/target_framegeometry_msgs/PoseStamped · commanded EE traj, deadman-gated
/{l,r}_arm_ctrl/actual_posegeometry_msgs/PoseStamped · achieved TCP pose, base frame, 10 Hz
/{l,r}_arm_ctrl/gripper_cmdcustom msg · commanded
/{l,r}_arm_ctrl/gripper_statuscustom msg · 50 Hz

CONTROL & SCALING STATE

/{left,right}_control_modestd_msgs/Int8 · 0 idle · 1 active · 2 locked
/{l,r}_arm_ctrl/mirror_enabledstd_msgs/Bool · false normal · true mirrored
/quest_{left,right}_scalingstd_msgs/Float32 · e.g. 0.7 or 1.7
/{left,right}_arm/statusstd_msgs/Int8 · 0 motion · 1 idle · 4 stopped, 2 Hz

Mirroring flips an arm's input mapping so the controller normally driving the left arm can drive the right instead (and vice versa) — useful for switching which side of the robot you're working on.

Runs on ROS 2 Humble

The stack targets native Ubuntu 22.04, or Ubuntu 24.04 via Docker if that's what your workstation runs.

Coming soon

Data collection
Hardware