game-automation

1# Game automation: the agent's hands and eyes
2 
3The real game is the oracle. The loop is launch → navigate → set up the scene → act → **screenshot and read
4logs** → decide. Build it once per game, then every change you make can be checked in a minute.
5 
6## Windows (native or from WSL): `um win`
7```bash
8um win setup                                   # once: PowerShell tools + an ffmpeg with gfxcapture, picks NVENC/AMF/QSV/x264
9um win ps                                      # windowed processes: pid, name, title
10um win launch --steam 105600                   # or: um win launch "C:\Games\Foo\Foo.exe" -- -windowed
11um win shot --exe Terraria.exe shot.png --scale 0.33   # full frame + a 1/3 copy that's cheap to look at
12um win drive --proc Terraria "focus" "click 640 360" "key 0x1B" "type hello" "hold 0x44 1500"
13um win drive --proc Terraria "hold 0x20 150" "sleep 200" "hold 0x20 150"   # sleep <ms> pauses between steps
14um win drive --proc Terraria idle              # seconds since the user last touched mouse/keyboard (does not wait)
15um win kill <pid>                              # exact PID only
16um win reg get "HKCU\Software\..."             # registry (reg set backs the key up first)
17```
18From Python (for longer scripts), `from um.win import Drive, shot, Recorder`, then `d = Drive("AoE2DE_s")`,
19`d.focus()`, `d.click(x, y)`, `d.key("0x0D")`.
20 
21**WinDrive commands** (`um/ps1/WinDrive.ps1`; stdin protocol, one reply per line). Coordinates are in
22the game window's **client area**.
23…