Purewhiter / mobilegym

What it does

• Fully programmable state. Capture, configure, diff and restore the entire environment as a single JSON blob. Initial state is *exactly* identical across all models and trials.
• Deterministic judges. Every task ships with a programmatic check function. No VLM judging required, no string-similarity guesswork. Sub-millisecond verdicts at million-judgement scale.
• Full-environment state comparison. Detect *unexpected side effects* (an accidentally-followed user, an inadvertently-sent message) that real-device pipelines structurally cannot see.
• Brutally lightweight. ≈400 MB RAM + ≈50 MB disk per instance. 256 parallel instances on a single server use <10 % CPU. A full 256-task evaluation finishes in ~6 minutes.
• Modular by design. New apps drop in through a manifest contract — no edits to the OS or benchmark layers. Same for new tasks, agents, judges and reward functions.