Game tests

esx probe runs a test copy of Fallout 4 and checks your mod in the game. An optional F4SE plugin, the probe, runs in the game process and obeys requests from esx. A test reads the ammunition count, the animation events, the clips that play, the graph files, the camera and the player model. It can take a screenshot. It sends no key and no mouse input, and it opens no window.

This works on Linux with Proton, for Fallout 4 runtime 1.11.240. Windows is not supported yet.

What you supply

Part	Where you get it
F4SE for your runtime	The F4SE site links to the download. Runtime 1.11.240 needs build 0.7.9.
Address Library for F4SE Plugins	The file for your runtime, for example version-1-11-240-0.bin.
The probe DLL	Build it from the probe folder of the esx repository. probe/README.md has the steps.
gamescope	Your distribution has a package. It gives the headless display.

esx downloads nothing. No release of the DLL exists yet, so the install needs --allow-unverified-dll.

Install

esx probe doctor
esx probe install --f4se ~/Downloads/f4se_0_07_09.7z \
                  --address-library ~/Downloads/AddressLibrary.zip \
                  --dll probe/build/esx-probe.dll --allow-unverified-dll

doctor writes nothing. It shows the runtime, the F4SE build that the runtime needs and each missing part.

install does not change your installed game. Each file goes into a test folder that esx owns, in ~/.local/share/esx/probe. The test folder has a link for each file of the game, then F4SE, the Address Library file and the probe. The game runs there in a Proton prefix of its own, with its own INI files, plugin list and saves. esx-probe-manifest.json in the test folder lists each installed file with its SHA-256. esx probe uninstall removes them.

If Steam has no single Proton version selected for the game, add --proton DIR to start and run.

Run one session by hand

esx probe start --mod build/Data --cell QASmoke --label "my test"
esx probe call game.state --session s-20261004-012143-64f1
esx probe call item.give --args '{"form": "MyGun", "equip": true}' --session s-…
esx probe call camera.set --args '{"view": "third"}' --session s-…
esx probe call screenshot.take --args '{"name": "third"}' --session s-…
esx probe call ammo.get --session s-…
esx probe stop --session s-…

start copies the mod into the test folder, starts the game and returns when the game is ready. Its result has the session ID. --save FILE.fos starts from a save in place of a cell. call sends one request. esx probe ops lists the requests. stop quits the game and removes the mod from the test folder.

Only one game runs at a time. A second start fails with GAME_BUSY and shows the label of the owner.

Write a test plan

A plan is a TOML file. esx probe run checks the file, starts a session, runs each step, stops the session and prints one report.

plan = 1
name = "counted reload"

[profile]
mod = ["../build/Data"]
cell = "QASmoke"

[[step]]
id = "give-weapon"
setup = true
do = "item.give"
args = { form = "MyGun", equip = true, mods = ["MyGunLongBarrel"] }
expect = { equipped = true }

[[step]]
id = "give-ammo"
setup = true
do = "ammo.set"
args = { loaded = 6, reserve = 200 }

[[step]]
id = "draw"
setup = true
do = "weapon.draw"
args = { drawn = true }
expect = { drawn = true }

[[step]]
id = "mod-graph"
setup = true
do = "anim.state"
expect = { "graphs.player_1st.behavior_files" = { contains = "MyGunBehavior.hkb" } }

[[group]]
id = "reloads"
for_each = [
  { loaded = 5, clip = "WPNReload1" },
  { loaded = 0, clip = "WPNReload" },
]

  [[group.step]]
  id = "set-count"
  do = "ammo.set"
  args = { loaded = "${loaded}" }
  save = { total_before = "total" }

  [[group.step]]
  id = "reload"
  do = "action.run"
  args = { action = "ActionReload" }
  timeout_ms = 12000
  wait = [{ clip_start = "${clip}", owner = "player_1st" }, { event = "reloadComplete" }]

  [[group.step]]
  id = "magazine-full"
  do = "ammo.get"
  expect = { loaded = 6, total = "${total_before}" }

esx probe run tests/reload.toml --check          # check the file only
esx probe run tests/reload.toml -o build/test-report

The steps run first, in file order. Then each group runs one time for each for_each entry.

Key of a step	Meaning
do, args	The request and its arguments.
expect	Checks on the response. A literal checks equality. A table uses one of eq, ne, lt, le, gt, ge, contains, matches, one_of.
wait	Conditions that must occur after the request: event, clip_start, clip_end or graph_active, with an optional owner.
save	Stores a response value in a variable. Use it later as "${name}".
setup	A failure of this step stops the plan with result error.
screenshot	true takes a screenshot after the step. The report folder gets the PNG file.
timeout_ms	The time limit of the step with its waits.

A wait on clip_start proves which clip played. A refill of the magazine does not prove it: the vanilla reload refills too. A check of behavior_files in anim.state proves that the weapon uses the graph of your mod.

Ops

Subject	Ops
Game	game.state, game.coc, game.load, game.save, game.quit, env.check
Console and message boxes	console.run, ui.message.get, ui.message.answer
Items	form.find, item.give, weapon.get, weapon.draw, ammo.get, ammo.set
Actions	action.run, anim.send_event
Animation	anim.wait, anim.trace, anim.clips, anim.state, anim.vars.set
View	camera.get, camera.set, player.model, screenshot.take
Logs	papyrus.log

esx probe ops lists each op with its arguments.

Read the result

Result	Exit code	Meaning
pass	0	Each check and each wait passed.
fail	7	A check or a wait failed. error.details has the complete report.
error	5	A setup step failed, or the game ended (GAME_CRASHED) or stopped (GAME_HUNG).

A wait that fails lists what it saw instead:

{"clip_start": "WPNReload4", "owner": "player_1st", "ok": false, "error": "TIMEOUT",
 "seen": {"clips": ["WPNReload3", "WPNRunForwardReady"], "events": ["reloadStateEnter", "ReloadComplete"]}}

The report folder has report.json, events.ndjson with each animation event and clip of the run, the probe log, the F4SE log and the Papyrus log. The report has the SHA-256 of each mod file, so you can prove which build the test used.

Facts about the game

• Draw the weapon before a reload. ActionReload is refused while the weapon is holstered.
• An editor ID of your mod works as a form: esx reads it from the plugin. For a record of the base game, use Fallout4.esm:LOCALID.
• The owner of a first-person clip is player_1st. Third person is player_3rd.
• With a mod active, the game asks a question before each load of a save. game.load answers it.
• The item count of the ammunition includes the loaded rounds.

esx help-topic game-test has the complete reference: each key of the plan, each error reason and the safety rules.

Limits

• The probe cannot judge the look of the game. A screenshot gives the image; a wrong pose, a clipped hand and a missing texture still need a person to look at it.
• The probe gives the animation name of a clip as the graph has it, not the file that the engine loaded. The local time of a clip and the states of a state machine are not available.
• Held input, for example a sprint reload, is not done.