Rainbow Robotics RB-Y1 teleoperation: the leader arm, the VR option and the recording pipeline you still have to build

The Rainbow Robotics RB-Y1 is a humanoid upper body on a wheeled base: two 7-DoF arms with ±0.05 mm repeatability, a 6-DoF torso (Rainbow calls it a single leg) that bends and reaches, a 2-DoF head and a base that drives at 1.5 m/s. It is designed for learning from demonstration, and Rainbow sells an official 14-DoF leader arm and a VR teleop option for exactly that. What Rainbow does not ship is the last step: a camera rig and a recording pipeline into a standard dataset format. That gap, more than the hardware, is what a team buying an RB-Y1 for data has to plan for.

Three ways to drive an RB-Y1

Why a leader arm rather than VR for most RB-Y1 data: a joint-matched leader gives the operator all seven joints of each arm at once, including the elbow posture that a hand-tracked pose leaves undefined on a 7-DoF arm. Demonstrations come out smoother and more consistent between operators, which is what imitation learning wants. VR earns its place when the operator also needs to look around through the head cameras or drive the base during the task.

The SDK and the control loop

The rby1-sdk is Apache 2.0, in Python (`pip install rby1-sdk`) and C++. Its command builders cover joint position, joint velocity, joint and Cartesian impedance, whole-body and optimal control, SE2 velocity for the base, gravity compensation and stop. For teleoperation, the streaming classes (`RobotCommandStreamHandler` in both languages, and a C++ real-time control builder) are what you use: they accept a continuous stream of targets instead of point-to-point moves. The Python examples include command streaming, Cartesian command streaming, real-time control and a leader-arm teleop example with a monitor.

The dual-PC design matters for remote work. The Robot PC handles motion and safety; the User PC runs ROS 2, the SDK and your code. Keep the leader-arm reader, the retargeting and the safety limits on or next to the User PC, and put only the operator's view and commands across the network. The robot holds posture with joint brakes when power is cut, and Rainbow's docs start with E-stop basics; both belong in the operator's training.

The missing piece: cameras and recording

Two facts shape every RB-Y1 data program. First, Rainbow's spec sheet says a vision sensor bracket is provided but "the vision camera is not provided": you choose and mount the cameras. Second, the SDK's record and replay examples (`21_record.py` and `22_replay.py`) save robot state and actions but not camera frames. When a user asked for teleop recording with camera observations in issue #96, the reply pointed to those scripts and said camera examples are not officially provided, because camera platforms vary too much.

There is also no RB-Y1 driver among LeRobot's built-in robots as of October 2026, while Reachy 2 and the Unitree G1 have one. So the work is yours or your vendor's:

Our data collection guide covers episode structure, formats and the quality checks that catch bad episodes early.

Config A or Config M, and rehearsing in simulation

Rainbow sells the RB-Y1 in two configurations. Config A has two wheels, a 1,250 Wh battery and weighs 131 kg; Config M has four wheels, a 2,500 Wh battery and weighs 170 kg. The upper body is the same, so the choice is about the base and how long a shift runs between charges. Rainbow does not publish runtimes, so measure them on your unit during the first week and build the charging plan from that number.

Rainbow publishes URDF, MJCF and USD models, and its developer docs have MuJoCo and Isaac Sim sections. Use them to rehearse before an operator touches the robot: check the leader-arm joint mapping, the velocity limits and the stop path in simulation, then repeat the same checks on the real robot at low speed.

Torso, base and whole-body data

The RB-Y1 has no balance problem, but it does have a whole-body question. The 6-DoF torso changes where the arms are, and the base changes where everything is. The leader arm drives the arms only, so torso and base motion come from another input or stay fixed. For tabletop work, set the torso to a fixed pose per task and park the base. For mobile manipulation, record the torso joints and the base velocity command as part of the action, and drive the base smoothly: base jerks move the cameras and make arm trajectories look noisier than they are.

Grippers or hands

Rainbow's optional two-finger gripper has a 100 mm stroke, 30 N typical and 150 N stall grip force, and a 0.62 s close time, and the leader arm's trigger maps straight to it. Rainbow also mentions dexterous-hand options without public details. For most bimanual datasets the gripper is the better start: one trigger per hand, consistent between operators, and easy for a policy to learn. A dexterous hand needs a different input than a trigger, which on the RB-Y1 means custom work. Whichever you choose, record the gripper command and the measured gripper position as separate fields, so a policy can tell the difference between asking for a close and achieving one, which matters most on objects that slip.

Latency

Leader-arm teleop is joint-to-joint, so the operator feels any delay as the robot lagging the arm in their hands. Below about 50 to 100 ms round trip that lag is hard to notice, and fine bimanual work with 3 kg arms is realistic. Between 100 and 200 ms operators slow down, which is acceptable for most pick, place and hand-over tasks. Above that, use local smoothing and velocity limits on the User PC and keep the operator on coarse motion and supervision. Our latency guide covers budgets by task.

What a good RB-Y1 operator does

RB-Y1 or Reachy 2?

Both are wheeled bimanual humanoids with 7-DoF arms. The Pollen Robotics Reachy 2 has a built-in LeRobot driver, so it gets to a dataset with less integration work. The RB-Y1 is heavier and stiffer, with ±0.05 mm repeatability, a 6-DoF torso, force-torque sensing and an official joint-matched leader arm, so it suits contact-rich and precise bimanual tasks once the recording pipeline is built.

Takeover and fallback

When an RB-Y1 runs a policy, the leader arm is a natural takeover device: the operator engages the follow button, finishes the step, releases, and the correction is recorded as its own episode. Our human-in-the-loop guide covers escalation design.

How RoboPilots runs RB-Y1 teleoperation

The robot stays at your site. We agree the input with you (leader arm, VR or both), help close the recording gap with a camera layout and a wrapper that writes your format, connect operators through the User PC on your network, and train a dedicated group of operators on your unit and your protocol. Several humanoid makers have sent us robots to test our operating protocols, and each new platform starts from that base.

As of October 2026, foreign-produced advanced robotic devices are on the FCC Covered List under FCC public notice DA 26-786 (July 28, 2026), unless the Department of War grants them a Conditional Approval. The FCC definition covers mobile robots, including humanoids, quadrupeds and autonomous mobile robots, that weigh over 4.4 lb with any base or dock and have a sensor, network connectivity and control software, when they do not qualify as US-made under the federal Buy American test, whatever the nationality of the maker. Covered models cannot receive new FCC equipment authorizations, which most devices need before they can be imported or sold in the US, while the FCC says models it authorized earlier can still be imported, sold and used; US buyers should confirm the current status with the seller.