Skip to content

Architecture

jevtest follows one rule: source-code dependencies point inward, toward the part that changes least. The things that change most often (Android and iOS tooling, the model's HTTP API, file formats, the terminal) sit on the outside. What jevtest is sits in the middle, untouched by any of them.

flowchart LR
    CLI["cli<br/>arguments · wiring · console"] --> APP["application<br/>runner · brain · planning"]
    CLI --> ADP["adapters<br/>devices · Jev + lockfile · test files · reports"]
    APP --> DOM["domain<br/>steps · screen · results · failures · ports · settings"]
    ADP --> DOM

The rule is checked on every commit by import-linter: the build fails if any import points the wrong way.

The layers

Layer What it holds May import Must never
domain What jevtest is: steps and checks, the screen, decisions, results, failures, settings and their limits, and the ports (interfaces) for devices, the decision model and time the standard library's pure parts only import any other layer, I/O, or a third-party package
application How tests run: the test runner, the brain that asks the model questions, sharding tests across devices domain touch a device, the network, a file or the terminal except through a port
adapters Everything outside: Android and iOS devices, the Jev client and lockfile, YAML test files and .env, JUnit and JSON reports domain import application or cli; let a tool's exception escape
cli The command line: arguments, the console output, and the composition root, the one place that picks real implementations for the ports everything hold test logic

What crosses between them

Each layer hands the next one plain, immutable domain types:

  • The loader (adapter) turns YAML into a Suite of typed Steps: one class per action (Touch, TypeText, ScrollTo, …), so a step can only carry what its action needs.
  • A device (adapter) turns the agent's XML or JSON into a frozen Screen of Elements. The wire format never leaves the adapter.
  • The brain (application) asks typed Choice and YesNo questions. The Jev adapter turns them into Jev's JSON, checks every answer against the options offered, and returns Picked and Probability answers.
  • The runner (application) returns RunResult, TestResult, StepResult and CheckResult records, and reports progress through the RunListener port. The console (cli) renders them; the report adapters write them as JUnit and JSON, and the notes beside each screenshot as text, JSON and a page.

Errors

Adapters catch the exceptions of the tools they wrap (subprocess, HTTP, YAML) and raise one of jevtest's own failures instead: TestFileError, DeviceError, ModelError, SetupRefused, or StepFailed. Nothing outside an adapter ever handles a tool's exception. Every message says what to fix. Before any test runs, any of them is printed as error: … and the run exits 2. During a test, a StepFailed, DeviceError or ModelError fails that test and the run goes on; a SetupRefused (a key or model Jev refuses) stops the run with exit 2, since every later question would be refused too.

The ports

class Device(Protocol):        # a phone, emulator or simulator with the app installed
    def screen(self) -> Screen: ...
    def tap(self, x: int, y: int) -> None: ...
    ...

class DecisionModel(Protocol): # Jev behind its lockfile
    def ask(self, state: State, questions: Mapping[str, Question]) -> Mapping[str, Answer]: ...

class Clock(Protocol): ...     # time, so tests run instantly
class RunListener(Protocol): ...  # told what happens as tests run
class ScreenNotesWriter(Protocol): ...  # saves what was on a screen beside its screenshot

The application depends only on these. jevtest.cli.main wires real implementations to them, and the tests wire fakes. That is why every layer is tested without the layer below it: the runner against a FakeDevice and a FakeModel, the device adapters against recorded agent output, the Jev adapter against a fake HTTP opener.

Checks that keep it this way

Check Command Enforced
Dependencies point inward lint-imports CI, pre-commit
Types, two checkers, no Any mypy (strict) and pyright (strict) CI, pre-commit
Every ruff rule, a docstring on every public object, complexity at most 10 ruff check . CI, pre-commit
Tests: 100% line and branch coverage, random order, Hypothesis properties pytest --cov CI, pre-commit
Docs build, no broken links or anchors mkdocs build --strict CI

All of these are in the contributing guide, with the security and packaging checks.

Where things live

Module What it does
domain/steps.py Each action is a type that says what it is (app_may_leave); waits and SETTING_SCOPES say where each setting applies. The runner and the loader read these instead of keeping lists of types.
domain/settings.py The settings, their defaults and their limits.
adapters/testfile/steps.py ACTIONS: the one table of how each action key reads in YAML.
adapters/devices/android_screen.py, ios_screen.py The agents' screen dumps into a Screen: pure functions, tested on real captures.
adapters/shapes.py, devices/tool_output.py Parsed YAML, JSON and plists, checked into typed values; a surprise is jevtest's own error, never a KeyError.