We present GR-RL, a robotic learning framework that turns a generalist vision-language-action (VLA) policy into a highly capable specialist for long-horizon dexterous manipulation. The optimality of human demonstrations is a core assumption in existing VLA policies. However, we claim that in training models for highly dexterous and precise manipulation tasks, human demonstrations may be noisy and sub-optimal.

GR-RL proposes a multi-stage training pipeline that filters, augments, and reinforces the demonstrations by reinforcement learning. First, GR-RL learns a vision-language-conditioned task progress, filters the demonstration trajectories, and only keeps the transitions that contribute positively to the progress. Specifically, we show that by directly applying offline RL with sparse reward, the resulting Q-values can be treated as a robust progress function. Next, we devise a series of simple yet effective augmentation tricks that greatly improve the performance of GR-RL. Lastly, to better align the VLA policy with its deployment behaviors for high-precision control, we perform online RL by learning a latent space noise predictor.

With this pipeline, GR-RL is—to our knowledge—the first learning-based policy that can autonomously lace up a shoe by threading shoelaces through multiple eyelets with an 83.3% success rate, a task requiring long-horizon reasoning, millimeter-level precision, and compliant soft-body interaction. We hope GR-RL provides a step toward enabling generalist robot foundation models to specialize into reliable real-world experts.

Through a multi-stage training pipeline, we achieved substantial performance improvements. The baseline model trained with behavior cloning achieved a success rate of 45.7%. After implementing task-progress-based data filtering, the success rate increased to 61.6%. Further enhancement through data augmentation techniques boosted the performance to 72.7%.

Subsequent online reinforcement learning fine-tuning initially showed transient performance fluctuation due to policy distribution shift, followed by rapid recovery and sustained growth. Using the enhanced offline‐learned model as the starting point for online reinforcement learning, GR-RL’s success rate ultimately rose to 83.3% after roughly 150 real-world closed-loop exploration and correction trajectories.

Detailed stage analysis revealed that data filtering and online RL substantially improved reliability during the critical eyelet-threading phase, while data augmentation provided consistent performance gains across all operational stages.

GR-RL demonstrates robust behavior across various cases. The model automatically retries when the shoelace accidentally drops or when it misses the eyelet. In cases where the two ends of the shoelace are crossed and the correct end is underneath, the model can identify it and pull it out.

GR- RL can also actively manipulate the scene to make the task easier: