Lumine: Building Generalist Agents in 3D Open Worlds

Mondstadt Region: Lumine successfully completes a full five-hour, three-act main storyline playthrough of Mondstadt. Even though its reasoning ability is trained only on the content of the first act, Lumine autonomously carries out all story progressions.

Liyue Region: In the completely out-of-distribution region of Liyue, Lumine demonstrates impressive generalization, successfully reaching and completing the main storyline in Liyue Harbor, and visiting the Adeptus dwelling deep within the mountains.

Beyond Genshin: Lumine exhibits exceptional cross-game generalization ability. Without any additional fine-tuning and modification, Lumine directly completes the entire chapter of Herta Space Station in Honkai: Star Rail within 7 hours; similarly, it successfully finishes approximately 100 minutes of main storyline content in Wuthering Waves.

A Scalable Recipe for Building General-Purpose Agents

Lumine adopts a human-like interaction paradigm that unifies perception, reasoning, and action in an end-to-end manner, powered by a vision-language model. It processes raw pixels at 5 Hz to produce precise 30 Hz keyboard–mouse actions and adaptively invokes reasoning only when necessary.

Lumine makes full use of the collected raw gameplay data, following a curriculum that builds skills incrementally:

• 1731 hours of human gameplay for pre-training to master action primitives;
• 200 hours of instruction following data to ground control in language;
• 15 hours of reasoning data to enable adaptive thinking.

The resulting model can not only complete hours-long missions but also follow diverse instructions to accomplish a broad spectrum of tasks in both 3D open-world exploration and 2D GUI manipulation across collection, combat, puzzle-solving, and NPC interaction. Genshin's diverse gameplay offers the perfect setting to showcase these abilities across different types of activities.