How it works¶

The loop¶

sequenceDiagram
    participant T as Test step
    participant J as jevtest
    participant A as On-device agent
    participant L as Lockfile
    participant M as Jev (TypeSafe)
    T->>J: do: Sign in with "${EMAIL}"
    J->>L: saved steps for this do:?
    alt saved (every later run)
        loop each saved step
            J->>A: wait until its element is on screen
            J->>A: tap / type / swipe
        end
    else first run
        loop until Jev says done
            J->>A: wait until the screen stopped moving
            A-->>J: accessibility tree (ms)
            J->>M: screen as text + choice questions
            M-->>J: chosen option + probabilities
            J->>A: tap / type / swipe
        end
        J->>L: save the steps it took
    end
    J->>T: ✓ or ✗ with the reason

Read the screen. A small agent on the device (an instrumentation APK on Android, an XCUITest runner on iOS) stays running for the whole run and returns the accessibility tree in milliseconds (measured medians: 7 ms on an Android emulator, 55 ms on a Pixel 4a, 39 ms on an iOS simulator, 170 ms on an iPhone over USB).
Describe it as text. Each element becomes a short line: {"id": "e4", "type": "button", "text": "Sign in", "position": "top-center"}. When the app reports text the screen doesn't show, such as a web page's buttons below the screen, working out a do: also gets that text as off_screen, so Jev knows to scroll to it rather than tap the nearest thing on screen.
Ask Jev to choose. One request, several questions: what's the next action (tap, type, scroll, back, done, impossible, …), on which element, and which quoted value to type. Jev answers each by choosing one of the options, with probabilities.
Act through the agent or adb, and save the step: its action and its element, by kind and name (and which one, if several have that name).
Repeat until Jev answers done or impossible. A goal gets at most 10 actions, or its max_actions.

Later runs don't ask Jev at all: they repeat the saved steps, each waiting until its element is on screen, like a tap: does.

expect: checks are one yes/no question each, passed when Jev finds the statement more likely true than false (yes-probability above 0.5, or the confidence setting). see: and not_see: never ask Jev.

Finding what a step names¶

tap: Save, see: Saved and scroll_to: Item 30 are matched in code, exactly: an element matches when its whole text, a part of its text (an iOS label or value, an Android text or description), its hint or its id is the target, ignoring case (details). There's no "contains" anywhere, so a step never lands on Unsaved changes or Save draft, and see: "Taps: 2" never passes on Taps: 20.

A step never guesses. If nothing says the target, it waits for it, and fails at its timeout with the texts that come close. To describe something instead of naming it (an icon, a field with no label), write a do: step.

Jev is involved in an exact step only when several elements say the target exactly (two Delete buttons): Jev chooses among those only, must confirm its choice, and the output says so.

Jev¶

Jev is TypeSafe's decision model, reached through TypeSafe's API. It reads text and answers by choosing from options it's given: it never writes free text. That's why jevtest can trust it with a test: everything it does is one of a fixed list of actions on one of the elements actually on screen, and everything typed comes from your test file.

jevtest uses no other model. Each release of jevtest is built and tested against one pinned Jev version (jev-1.13.0), printed at the start of every run, so a new Jev can't change behaviour underneath you. A test file's model must be a pinned version too: an alias such as jev-latest moves when a new Jev ships. A new Jev version comes with a new jevtest release.

The lockfile¶

Jev's probabilities wobble slightly between identical calls, so a close decision can come out differently. Measured on this project: across 20 identical repeats, 4 of 97 action decisions changed at least once. Yes/no checks never changed.

So jevtest keeps a lockfile next to the test file (tests.yaml → tests.lock.json) with:

The steps each do: took the first time, worked out by Jev. Every later run repeats exactly those steps, with no network call, and they don't depend on the screen looking exactly as it did: each one only needs its element on screen. A scroll's length depends on the screen, so after a saved scroll jevtest scrolls on the same way until the next step's element is on screen, clear of the screen's edges, and at the end of the content without it, back the other way. It scrolls no further than that, to stay as close as it can to the screens the steps were recorded on.
Jev's answer to every other question (an expect:, which of several exact matches a step means), keyed by a hash of the exact model, screen and question: the same screen and question always get the same answer. A recorded answer is never applied to a screen it wasn't recorded on.

Saved steps are kept per platform (Android, iOS), since the same goal takes different steps on each. Devices of one platform share them, so they must show the app the same way: two Android versions can name the same web field differently (see Troubleshooting). For devices that don't, use a test file per device, so each has its own lockfile.

If the app changes (a button is renamed), a saved step's element never shows up: --lock record works that do: out again from where it got to, and --lock frozen fails and says so.

Commit the lockfile. --lock says how each run uses it, and --prune-lock removes what a run no longer used.

Waiting: wait until, or fail¶

Every wait in jevtest is the same thing: wait until a condition is true, checking it every interval (0.25 s), for at most the step's timeout (10 s). If it isn't true by then, the step fails and says what it waited for:

✗ tap: Save — Waited 10s until an element says 'Save' on screen and stopped moving; close but not exact: 'Save draft'

Step	Waits until
`tap:`, `type: … into:`, `clear:`, `swipe: … target:`	an element says the target and has stopped moving: it's in the same place at two checks in a row (on Android, and drawn the same, except a text field, whose cursor blinks), so a tap never lands where a sliding page's button was a moment ago (and it isn't under the keyboard)
`type:` without `into:`	the keyboard is up
`see:` / `not_see:`	the text is on screen / gone
`expect:`	Jev judges it true of a screen that stopped moving (asked once per such screen)
a saved `do:` step	its element is on screen and has stopped moving
working a `do:` out with Jev	the screen stopped moving
`scroll_to:`, before its first look and after each scroll	the screen stopped moving
`screenshot:`	the screen stopped moving
`hide_keyboard`, `rotate:`	the keyboard is gone / the screen has turned (3 s)

Stopped moving means the screen reads the same at two checks in a row, and on Android its text is also drawn the same. The tree alone can't show every movement: Android reports a system dialog fading in at its final place at once, and one sliding up at its first place for about half a second before its final one (both measured), so only its pixels show it moving. Only text counts: a spinner or a blinking cursor never stops, but the screen they're on has. On iOS an element's frame moves with its animation, so the tree says it all.

An action doesn't wait afterwards: the next step waits for what it needs. The on-device agents never wait either; they answer each read at once.

Nothing assumed¶

jevtest never fills a gap for you:

No default device, lock mode or results folder: the test file and the command say everything. Settings have defaults that suit most apps, and limits that stop a test from hiding a broken app.
A misspelled step, a value of the wrong type or an option on the wrong action is an error, never a guess. - bakc doesn't become a goal for Jev; wait: "2" isn't quietly read as 2.
The whole test file is checked before any device is touched, and every problem is reported at once.
A device name must match exactly one device.
A secret set differently in .env and the environment is an error, not a choice.

Never changing the app or the device¶

jevtest tests the app as users get it. It doesn't turn off animations, speed anything up, grant permissions, or change device settings on its own. It installs your build and its own agent, and changes only what a step asks for (rotate:, dark_mode:, network:, autofill:, location:), and it puts those back at the end of the run (all but an Android emulator's location, which the emulator can't report).