How we test
The rules every run follows
An agent that plays a game can look like a cheat or a bot. These rules are what make it research instead, and they apply to every run reported on this site.
What the agent may touch
Game window
A private single-player world on one PC.
pixels
Local agent
Reads the screen and decides, on the same PC.
key presses
Keyboard and mouse
The same input a person uses, back into the game.
Never touched
- Game memory
- Game files
- Code injection
- Other players
Runs happen in a private world on one PC. The agent sees the screen and sends keyboard and mouse input. It does not read the game's memory, inject code or change game files, and it is never used with or against other players.
How a trial is run
01
Declare
The task, the pass rule and the stop conditions are written down first. Rules are not adjusted after seeing the outcome.
02
Arm
A single key halts all agent input, and it is armed before any input is sent. The agent also refuses to act if the game is not the window in front.
03
Run
The founder watches live runs and can stop them at any moment. During a run, only local code acts. A cloud AI assistant may choose goals between runs, never inside one.
04
Record
Each decision tick is recorded with what the agent saw and what it pressed. Frames are kept so the record can be checked against the picture.
05
Report
The result is written up with its limits and its failures, under one of two labels.
How results are reported
Every claim is one of two kinds
- Measured
- A number read from the log.
- Inferred
- A conclusion drawn from looking at frames.
Every entry carries one of two labels
- Development note
- Prototype work.
- Scored result
- A trial run against rules fixed beforehand.
Prototype work is never presented as a validated capability.
- Limits stated
- Every entry says what was not shown: one place, one creature, one attempt, when that is all it was.
- Failures stay in
- Failed attempts, wrong labels and operator mistakes are part of the record.
- Who did what
- Results name whether a local rule, an AI assistant or a person made each choice.
How the AI collaborators are kept honest
When more than one AI system reviews the work, each states its position separately before they compare. Agreement is only reported where each one actually reached it.