ModelBaseSFTRL
9B9.21%10.54%38.33%
35B-A3B7.79%23.38%41.04%
Player-1: Learning to Play
from Synthetic Experience
Beyond watching demonstrations, with learning by playing.
1 National University of Singapore2 ARC Lab, Tencent3 Nanyang Technological University
See Player-1 in the Wild
1
FROM COMPUTER-USE AGENT TO GAME CONTROL
A few extensions let computer-use agents control games.
01 / ACTION SPACE
Extending the computer-use agent for games
From cursor positioning to camera control.
Left - Computer-use control:
Use absolute coordinates to move items in the inventory.
Right - Game control:
Relative mouse input turns the view to follow a flying bat.
2
FROM CONTROLS TO DEMONSTRATIONS
Curated gameplay demonstrations provide the starting point.
02 / SFT DATA
Demonstrations for supervised initialization
Curated gameplay data for aligned supervision.
The collection comprises 6,044 trajectories from 172 games: 5,360 gameplay trajectories and 684 in-game UI trajectories. Recorded controls are mapped to the shared game control action space. Task instructions, reasoning, and keyboard-mouse actions make each step a supervised learning example.
3
FROM DEMONSTRATIONS TO PLAY
Player-1-Gym enables learning by playing with verifiable rewards.
03 / PLAYER-1-GYM
Explore Player-1-Gym environments
Shared control demands with diverse game mechanics.
Each environment accepts keyboard and mouse actions and computes rewards directly from game state. Randomizing appearance, camera, scene, and control broadens the agent’s training experience.
Environment randomization
Let the learnt experience diverse.
- Appearance
- Colors, lighting, assets
- Camera
- Projection and viewpoint
- Scene
- Layouts, spawns, routes
- Control
- Look, move, drive, steer gains
2D Collect & Dodge
2D Platforming
3D Lane Driving
3D Chasing
LEARNING DYNAMICS
Reinforcement learning across sixteen worlds
4
FROM EXPERIENCE TO HELD-OUT ENVIRONMENTS
Experience gained through play transfers to held-out environments.
04 / BENCHMARK & RESULTS
Experience that transfers
MINECRAFT · MINEDOJO-300 · PLAYER-1-9B
task success after RL
Player-1-9B: 38.33% task success after RL.
Player-1 learns through interaction in synthetic worlds, then transfers to games outside the RL training suite: From 10.54% after SFT to 38.33% after RL, a +27.79 percentage-point gain on the 300 tasks in MineDojo.
GameWAM, the strongest evaluated external baseline, reaches 12.63% under the paper's evaluation protocol.
PERFORMANCE
SFT improves, RL boosts
ModelBaseSFTRL
9B0.29750.43610.6922
35B-A3B0.39120.56440.7236
ModelBaseSFTRL
9B29.22%31.43%34.45%
35B-A3B33.49%34.60%38.79%
TRAJECTORIES
Recorded gameplay in Minecraft, ViZDoom, and GameWorld
| Task / map | Steps | Outcome | Trajectory |
|---|---|---|---|
| Loading trajectories… | |||
Cite this work
BibTeX
@misc{ouyang2026player1,
title = {Player-1: Learning to Play from Synthetic Experience},
author = {Ouyang, Mingyu and Wang, Yiran and Hu, Siyuan and Yin, Xingyilang and Sun, Huiqiang and Wang, Guangzhi and Pu, Junfu and Cheng, Cheng and Chen, Honghao and Shan, Ying and Shou, Mike Zheng and Ng, Hwee Tou},
year = {2026}
}
