Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

Earlier this month I placed 🥈 2nd in the LeHome Challenge at ICRA 2026 — and 🥇 1st of 62 teams in the simulation round before that. This paper explains my solution.

The task: teach a cheap two-armed robot to fold different garments — long tops, short tops, long pants, and shorts — both in simulation and on a real robot. The robot only sees three cameras and never gets told which garment it's folding, so it has to figure that out on its own.

Here's the short version of how it works 👇

🧠 The policy is its own value function. From the same forward pass that picks the next action chunk, cheap heads predict success probability, task completion %, garment type, and future keypoint distances + a Q-residual. Those become the advantage signal for RL - no separate critic.

🔁 A fully asynchronous RL loop coordinated only through the HF Hub: 1 trainer (H200) ships a fresh checkpoint ~every 40 min while N rollout workers (and a human doing teleop / DAgger corrections) collect data in parallel. Nobody waits, it uses the off-policy nature of the loop to the fullest.

📈 Binary success is too sparse, so I densify it by combining multiple approaches simultaneously — from objective keypoint distances, the success-probability value baseline, completion %, and relative success rate of different garments.

🎛️ The RL combines AWR (sample good actions more often) + RECAP (feed advantage as a conditioning input, then ask for good actions only at inference, with CFG). I also tune the inference knobs — execution length, playback speed, inpainting overlap, CFG scale, best-of-N — with a per-parameter Thompson-sampling bandit folded into rollout collection.

📊 Round 1: 1st of 62, 79.6% success (+6.1 over 2nd, top score on 3 of 4 garments). Round 2: 2nd place, with only ~1 week and no access to the eval robot — so the pipeline was sim → my robot → their robot, leaning on heavy augmentation to make the policy more robust.

💡 Biggest win left on the table: I ran full RL in sim and BC + human corrections on real separately. They're very complementary — combining them should push success much higher.