Python API¶

jevtest is a command-line tool, and the command line is its supported interface. The Python modules below are documented for contributors and for anyone embedding jevtest; they follow the architecture.

Domain¶

steps ¶

Tests and their steps: one action, then the checks that must hold after it.

Each kind of action is its own type, so a step can only carry the values that kind needs.

SWIPE_DISTANCE `module-attribute` ¶

SWIPE_DISTANCE = (70.0, 60.0)

How far a swipe's finger moves unless a step says, in percent of what it swipes: left or right, and up or down.

KEYS `module-attribute` ¶

KEYS = (
    "backspace",
    "delete",
    "enter",
    "escape",
    "return",
    "space",
    "tab",
)

The keys key: presses on every platform.

ANDROID_KEYS `module-attribute` ¶

ANDROID_KEYS = (
    "app_switch",
    "back",
    "dpad_down",
    "dpad_left",
    "dpad_right",
    "dpad_up",
    "home",
    "menu",
    "move_end",
    "move_home",
    "power",
    "search",
    "volume_down",
    "volume_up",
)

The keys key: presses on Android only (as does a key code number).

Action `module-attribute` ¶

Action = (
    Do
    | Use
    | Touch
    | Clear
    | TypeText
    | Scroll
    | Swipe
    | ScrollTo
    | Key
    | Wait
    | Background
    | Rotate
    | Location
    | OpenUrl
    | DarkMode
    | Grant
    | Network
    | AutofillOff
    | Screenshot
    | Launch
    | Stop
    | Restart
    | ClearData
    | Reinstall
    | Back
    | Home
    | HideKeyboard
)

Everything a step can do.

Check `module-attribute` ¶

Check = Expect | See | NotSee

Something that must be true after a step's action.

SETTING_SCOPES `module-attribute` ¶

SETTING_SCOPES: Mapping[str, Scope] = {
    "timeout": Scope(
        "a step that waits (for its element, its checks, or a do:/scroll_to: condition)",
        lambda action, checks: (
            bool(checks)
            or action is not None
            and waits(action)
        ),
    ),
    "interval": Scope(
        "a step that waits (for its element, its checks, or a do:/scroll_to: condition)",
        lambda action, checks: (
            bool(checks)
            or action is not None
            and waits(action)
        ),
    ),
    "max_actions": Scope(
        "a do: step",
        lambda action, _: isinstance(action, Do),
    ),
    "max_scrolls": Scope(
        "a scroll_to: step",
        lambda action, _: isinstance(action, ScrollTo),
    ),
    "confidence": Scope(
        "a step with an expect: check",
        lambda _, checks: any(
            isinstance(check, Expect) for check in checks
        ),
    ),
}

Each setting a step can change for itself, and the steps it means something for.

Do `dataclass` ¶

Bases: _Action

Reach a plain-English goal: Jev picks the actions. Quoted values are what it may type.

Use `dataclass` ¶

Bases: _Action

Run another test's steps here.

Touch `dataclass` ¶

Bases: _Action

Tap, double-tap or long-press the element a target names.

Clear `dataclass` ¶

Bases: _Action

Erase a text field.

TypeText `dataclass` ¶

Bases: _Action

Type text, into the named field or (without into) the focused one. Typed exactly as written.

Scroll `dataclass` ¶

Bases: _Action

Scroll the content.

Attributes:

Name	Type	Description
`along`	`str \| None`	Scroll what this text is in (a carousel, say), dragging along its line; None: the page.

Swipe `dataclass` ¶

Bases: _Action

Swipe across the screen, or on the element a target names.

Attributes:

Name	Type	Description
`distance`	`float \| None`	How far the finger moves, in percent of what it swipes across; None: `SWIPE_DISTANCE`'s.

ScrollTo `dataclass` ¶

Bases: _Action

Scroll until the text is on screen.

Attributes:

Name	Type	Description
`along`	`str \| None`	Scroll what this text is in (a carousel, say), dragging along its line; None: the page.

Key `dataclass` ¶

Bases: _Action

Press a named key (enter, delete, ...), or on Android a key code given as a number.

Wait `dataclass` ¶

Bases: _Action

Wait a fixed time.

Background `dataclass` ¶

Bases: _Action

Send the app to the background for a while, then bring it back.

Rotate `dataclass` ¶

Bases: _Action

Rotate the device.

Location `dataclass` ¶

Bases: _Action

Set the GPS location.

OpenUrl `dataclass` ¶

Bases: _Action

Open a deep link or URL. The link may open in another app.

DarkMode `dataclass` ¶

Bases: _Action

Switch dark appearance on or off.

Grant `dataclass` ¶

Bases: _Action

Grant the app runtime permissions.

Each platform names permissions its own way (android.permission.CAMERA, camera), so a file that runs on both gives the names per platform. names pairs a platform with its names; None stands for every platform.

names_on ¶

names_on(platform: Platform) -> tuple[str, ...] | None

The permissions' names on platform: its own, else those for every platform; None when it has none.

Network `dataclass` ¶

Bases: _Action

Switch Wi-Fi and mobile data on or off.

AutofillOff `dataclass` ¶

Bases: _Action

Turn off the autofill service: no password manager offers to save or fill what a test types.

Screenshot `dataclass` ¶

Bases: _Action

Save a screenshot.

Launch `dataclass` ¶

Bases: _Action

Launch the app.

Stop `dataclass` ¶

Bases: _Action

Stop the app.

Restart `dataclass` ¶

Bases: _Action

Stop and launch the app.

ClearData `dataclass` ¶

Bases: _Action

Stop the app and clear its data.

Reinstall `dataclass` ¶

Bases: _Action

Uninstall and install the build again.

Back `dataclass` ¶

Bases: _Action

Go back.

Home `dataclass` ¶

Bases: _Action

Go to the home screen.

HideKeyboard `dataclass` ¶

Bases: _Action

Close the on-screen keyboard.

Expect `dataclass` ¶

Jev judges the statement true of the screen.

See `dataclass` ¶

The text is on screen.

NotSee `dataclass` ¶

The text is not on screen.

Step `dataclass` ¶

One action (or none), then the checks that must hold after it.

Attributes:

Name	Type	Description
`action`	`Action \| None`	What to do, or None for a step that only checks.
`checks`	`tuple[Check, ...]`	What must be true afterwards, in order.
`settings`	`Settings`	The settings this step runs with: the file's, with any the step sets for itself.
`label`	`str`	The action as written in the test file, on one line, e.g. `tap: Save`; empty for a step that only checks.

Test `dataclass` ¶

A named list of steps.

Attributes:

Name	Type	Description
`name`	`str`	Unique across a test file and everything it includes.
`fresh`	`bool`	Start from a clean install (True) or carry on from where the previous test left the app (False).
`steps`	`tuple[Step, ...]`	The steps, in order.

Suite `dataclass` ¶

One test file, ready to run.

Attributes:

Name	Type	Description
`path`	`Path`	The test file.
`apps`	`Mapping[Platform, Path]`	The build for each platform, in the order the file lists them.
`devices`	`Mapping[Platform, tuple[str, ...]]`	The devices for each platform; tests are split across several.
`tests`	`tuple[Test, ...]`	The tests to run, in order.
`library`	`Mapping[str, Test]`	Every test `use:` can name: this file's and those of the files it includes.
`variables`	`Mapping[str, str]`	The `${NAME}` values the file uses.
`includes`	`tuple[Path, ...]`	The library files it includes, directly or not.
`settings`	`Settings`	The file's settings. Each step carries its own copy, with the step's own changes.

steps ¶

steps() -> Iterator[tuple[str, Step]]

Every step the tests take, with the test that has it: each test's steps once, however many use it.

Scope `dataclass` ¶

Which steps a setting means something for.

Attributes:

Name	Type	Description
`where`	`str`	The steps, in words, for the error when a setting is on any other step.
`applies`	`Callable[[Action \| None, Sequence[Check]], bool]`	Whether the setting means something for a step with this action and these checks.

waits ¶

waits(action: Action) -> bool

Whether the action waits until something is true.

That's its element being on screen (tap:, type: into, ...), or its own condition (type: the keyboard up, do: before each move, scroll_to: after each scroll, screenshot: the screen stopped moving).

screen ¶

The screen of the app under test, as a flat list of elements.

Bounds `module-attribute` ¶

Bounds = tuple[int, int, int, int]

An element's rectangle: left, top, right, bottom, in the device's own units.

Point `module-attribute` ¶

Point = tuple[int, int]

A point on the screen, in the device's own units.

STRAIGHT_QUOTES `module-attribute` ¶

STRAIGHT_QUOTES = str.maketrans(
    {"‘": "'", "’": "'", "“": '"', "”": '"'}
)

Curly quotes as the straight ones a keyboard types. The permission prompts write them curly (measured: Android's “Don’t allow”, iOS's “Don’t Allow”); a test file says tap: Don't allow.

FILLS `module-attribute` ¶

FILLS = 0.9

The share of the screen an element covers to be taken for a barrier behind a dialog (Screen._clear_touch).

EDGE `module-attribute` ¶

EDGE = 0.08

The share of the screen's height, at its top and at its bottom, that phones keep for their own gestures.

MIN_LANE `module-attribute` ¶

MIN_LANE = 20

The fewest points or pixels across that a drag along something can use (less, and the page is dragged instead), and that a strip of a barrier needs to take a tap clear of its dialog.

SIDE_EDGE `module-attribute` ¶

SIDE_EDGE = 0.15

The share of the screen's width, at each side, where a swipe inward never starts: Android takes one as back (measured: from 78 of 1080 pixels at the default back sensitivity; twice that is kept clear, for a higher one), and iOS one from the left.

NEAR_LIMIT `module-attribute` ¶

NEAR_LIMIT = 5

How many near matches an error lists.

Element `dataclass` ¶

One thing on the screen: a button, a text field, a label, a switch, ...

Attributes:

Name	Type	Description
`kind`	`str`	What it is: button, text_field, password_field, text, switch, checkbox, image, cell, ...
`text`	`str`	What it shows, as one line: its label, content description or value, or several of them together.
`parts`	`tuple[str, ...]`	The separate texts `text` is made of, when it's made of more than one: an iOS label and value (`Email: a@b.c`), an Android text and content description (`Go (Go now)`), the lines of a text with several (Flutter's tab `Form` / `Tab 2 of 3`). Each can be matched on its own.
`hint`	`str`	Placeholder or hint text.
`resource_id`	`str`	The Android resource id or iOS accessibility identifier.
`bounds`	`Bounds`	Where it is on the screen.
`enabled`	`bool`	Whether it responds to input.
`editable`	`bool`	Whether it takes typed text; an iOS picker wheel takes a value to turn to.
`clickable`	`bool`	Whether it responds to a tap.
`scrollable`	`bool`	Whether its content scrolls.
`focused`	`bool`	Whether it has input focus.
`checked`	`bool \| None`	On or off for a switch or checkbox; None for anything that can't be checked.
`selected`	`bool`	Whether it is selected.
`value`	`str`	A text field's current contents (what clearing it must delete).
`position`	`float \| None`	Where a slider's thumb is, from 0 (its start) to 1 (its end); None when not known.
`adjustable`	`bool`	Whether it's moved by a drag along it, like a slider: iOS says so of sliders, steppers, picker wheels and page controls, and of a Flutter slider, which it otherwise reports as plain text.
`id`	`str`	Its id on this screen (e1, e2, ...), assigned by `Screen`.
`touch`	`Point \| None`	Where a finger touches it, when not its middle (`Screen` sets it for a barrier behind a dialog).

slider `property` ¶

slider: bool

Whether a left or right swipe on it takes a thumb to that end: a slider, or what iOS says is adjustable.

center `property` ¶

center: Point

The middle of the element: where a tap lands, unless it has a touch point (see tap_point).

tap_point `property` ¶

tap_point: Point

Where a finger touches it: its touch point if it has one, else its middle.

end `property` ¶

end: Point

A point just inside the right edge: tapping there puts the text cursor after the text.

says ¶

says(target: str) -> bool

Whether the element's text, one of its parts, its hint or its id is exactly target (folded).

Case is ignored because platforms render the same text in different case (Android shows many buttons in capitals; the permission prompt says "Don't allow" on Android and "Don't Allow" on iOS), and curly quotes match straight ones, which is all a keyboard types. Nothing else is ignored: the whole text must be the target, never a part of a longer text. Spaces and line breaks between the target's words count as one space, as they do in the screen's text when it's read.

names ¶

names() -> tuple[str, ...]

Every text a test can name the element by, in order: text, parts, hint, id.

label ¶

label() -> str

A short description for logs and for Jev, e.g. button 'Sign in'.

Screen `dataclass` ¶

Everything on the screen at one moment.

Attributes:

Name	Type	Description
`width`	`int`	Screen width, in the device's own units.
`height`	`int`	Screen height, in the device's own units.
`elements`	`tuple[Element, ...]`	The elements, in reading order. Each gets an id (e1, e2, ...) here.
`keyboard_visible`	`bool`	Whether the on-screen keyboard is up.
`keyboard_top`	`int`	Where the keyboard starts (0 when there is none).
`system_bars`	`tuple[Bounds, ...]`	Where the system draws over the app, like Android's status bar: a touch there is the phone's.
`scrollers`	`tuple[Bounds, ...]`	Where content scrolls (a list, a carousel) that isn't an element of its own: an iOS scroll view with no label.
`bars`	`tuple[Bounds, ...]`	The app's own bars at the top and bottom (navigation and tab bars), which don't scroll.
`offscreen`	`tuple[str, ...]`	Texts the app reports but doesn't show, such as a web page's below the screen: not elements, as nothing can be done with them where they are, only for the error when a step looks for one.

content_height `property` ¶

content_height: int

The height not covered by the keyboard: where page gestures belong.

page `property` ¶

page: Bounds

Where a drag across the page goes: across the biggest thing that scrolls, clear of the keyboard and bars.

Across the screen less the keyboard and the app's top and bottom bars when nothing says it scrolls. A drag that starts on a bar moves nothing (measured: a pull from a UIKit navigation bar with a search field didn't refresh the list below it, and a scroll from 80% of a landscape screen started on a Compose navigation bar), and neither does one that starts on the keyboard.

editable `property` ¶

editable: tuple[Element, ...]

The elements that take typed text.

under_keyboard ¶

under_keyboard(el: Element) -> bool

Whether the keyboard is over the point where a tap on the element lands: touching it would hit a key.

swiped ¶

swiped(
    el: Element, direction: Direction
) -> tuple[Bounds, Bounds]

Where a swipe on el goes: what it goes across, and where the finger goes down.

A finger swipes a row, a pager or a carousel, not just the text on it, which can be far narrower: a pager turns only after half its width (measured with Compose). So a swipe goes along el's own line, across the smallest list row (a cell) or scrolling container holding it. In a row the finger goes down anywhere on the row; in a scrolling container, on el itself, since that's the item it moves (an Android list row that doesn't fill the list is swiped only from on it). A slider, or an element in neither, is swiped itself.

under_system_bar ¶

under_system_bar(el: Element) -> bool

Whether a system bar is over the point where a tap on the element lands: the phone would take the touch.

clear_of_edges ¶

clear_of_edges(el: Element) -> bool

Whether the element's middle is clear of the top and bottom EDGE of the screen.

Of the part the keyboard leaves, when it's up. Phones keep those edges for their own gestures, like the home swipe.

in_lane ¶

in_lane(
    el: Element, lane: Bounds, direction: Direction
) -> bool

Whether the element's middle is clear of the screen's edges and inside the lane being scrolled along.

Clear of the edges as clear_of_edges; inside the lane in the direction of the scroll. A carousel reports an item peeking in at its end with its middle past the carousel's own end (measured on an iPhone: a SwiftUI chip at x 354-402 in a carousel ending at 370), where a tap on it misses. Only in the direction of the scroll: a lane up and down is only as wide as what it was found by.

takes_keys ¶

takes_keys(el: Element) -> bool

Whether typing now goes into the element without a tap: it has focus and the keyboard is up.

Tapping it anyway would move the text cursor, and with the keyboard over it, hit a key.

by_id ¶

by_id(element_id: str) -> Element

The element with this id.

Raises:

Type	Description
`KeyError`	No element has that id.

shows ¶

shows(text: str) -> bool

Whether an element says exactly text (see Element.says). Never a part of a longer text.

off_screen ¶

off_screen(text: str) -> bool

Whether the app reports text somewhere the screen doesn't show (folded, as Element.says).

near ¶

near(text: str) -> tuple[str, ...]

What the screen says that text may have meant, for the error when it isn't there. Never matched.

region ¶

region(el: Element) -> str

Where an element is, in words (top-left ... bottom-right): Jev reads words better than numbers.

folded ¶

folded(text: str) -> str

text as matching compares it: case ignored, each letter in one encoding (NFC), and quotes straight.

So an é typed as an e and an accent matches an é stored as one character, and Don't matches Don’t.

near_names ¶

near_names(
    target: str, elements: Sequence[Element]
) -> tuple[str, ...]

Longer names on these elements that contain target, ignoring case.

Only for error messages, so the user can fix the test file: a near match is never matched.

decisions ¶

What Jev decided to do next while reaching a do: goal.

Each move is its own type, so a move that acts on an element always has one. describe() is also what Jev is shown as the actions already taken, so its wording is part of every recorded decision: change it and recorded lockfiles no longer match.

ElementMove `module-attribute` ¶

ElementMove = (
    TouchElement | SwipeElement | TypeInto | ClearField
)

A move on one element of the screen.

PageMove `module-attribute` ¶

PageMove = (
    WaitForScreen
    | GoBack
    | PressEnter
    | CloseKeyboard
    | ScrollPage
)

A move on the screen as a whole.

Move `module-attribute` ¶

Move = Finished | Impossible | ElementMove | PageMove

Every move Jev can pick: Finished and Impossible end the goal, the rest are made on the device.

STEP_ACTIONS `module-attribute` ¶

STEP_ACTIONS = frozenset(
    {
        "tap",
        "double_tap",
        "long_press",
        "swipe_left",
        "swipe_right",
        "type",
        "clear",
        "scroll_up",
        "scroll_down",
        "scroll_left",
        "scroll_right",
        "back",
        "press_enter",
        "hide_keyboard",
    }
)

The actions a saved step can be. type, clear, taps and swipes have a target; type also has text.

Finished `dataclass` ¶

The goal is reached.

describe ¶

describe() -> str

done.

Impossible `dataclass` ¶

The goal can't be reached from this screen.

describe ¶

describe() -> str

impossible.

WaitForScreen `dataclass` ¶

The screen is still loading: wait for it to change.

describe ¶

describe() -> str

wait.

GoBack `dataclass` ¶

Go back, or dismiss a dialog or menu.

describe ¶

describe() -> str

back.

PressEnter `dataclass` ¶

Press Enter to submit what was typed.

describe ¶

describe() -> str

press enter.

CloseKeyboard `dataclass` ¶

Close the on-screen keyboard.

describe ¶

describe() -> str

hide keyboard.

ScrollPage `dataclass` ¶

Scroll the page to reveal more content.

describe ¶

describe() -> str

E.g. scroll down.

TouchElement `dataclass` ¶

Tap, double-tap or long-press an element.

describe ¶

describe() -> str

E.g. tap button 'Sign in'.

SwipeElement `dataclass` ¶

Swipe on one element, e.g. to delete a list row.

describe ¶

describe() -> str

E.g. swipe_left cell 'Item 3'.

TypeInto `dataclass` ¶

Type one of the goal's quoted values into a field.

describe ¶

describe() -> str

E.g. type "${EMAIL}" into text_field 'Email'.

ClearField `dataclass` ¶

Erase a text field.

describe ¶

describe() -> str

E.g. clear text_field 'Email'.

Decision `dataclass` ¶

A move, with how sure Jev was of it.

Attributes:

Name	Type	Description
`move`	`Move`	What to do.
`confidence`	`float`	Jev's confidence in the chosen action (0 to 1).
`probabilities`	`Mapping[str, float]`	Jev's probability for each action it was offered.

Target `dataclass` ¶

Which element a saved step acts on: its kind and name, and which of the elements with both it is.

Attributes:

Name	Type	Description
`kind`	`str`	The element's kind (`button`, `text_field`, ...).
`name`	`str`	What it says: its text, else its hint, else its id; empty for an element with none of them (a web field on some Android versions). A `${NAME}` value in it is written as its name.
`nth`	`int`	Which of the elements with this kind and name, counting from the top of the screen (1 = first).
`count`	`int`	How many elements had this kind and name when the step was saved.

describe ¶

describe() -> str

E.g. button 'Sign in', the 2nd of 3 button 'Delete', the 1st of 2 text_field (no name).

SavedStep `dataclass` ¶

One step a do: goal took, saved in the lockfile so later runs repeat it without Jev.

Attributes:

Name	Type	Description
`action`	`str`	One of `STEP_ACTIONS`.
`target`	`Target \| None`	The element it acts on, for taps, swipes, `type` and `clear`; None for the others.
`text`	`str \| None`	For `type`: the value as the goal writes it, a `${NAME}` as its name.

describe ¶

describe() -> str

E.g. tap button 'Sign in', type "${EMAIL}" into text_field 'Email', scroll down.

model ¶

The questions jevtest asks a decision model, and the answers it gets back.

A decision model (Jev) never writes free text: every question offers options and it picks one (Choice), or it gives the probability that something is true (YesNo). These types are what the application layer speaks; the adapter turns them into the model's wire format.

Question `module-attribute` ¶

Question = Choice | YesNo

A question for the model.

Answer `module-attribute` ¶

Answer = Picked | Probability

An answer from the model.

State `module-attribute` ¶

State = Mapping[str, object]

What the model is shown alongside the questions: the screen, and whatever else the question needs.

Choice `dataclass` ¶

Pick one of options (option id -> description).

YesNo `dataclass` ¶

The probability that the instructions' question is true.

Picked `dataclass` ¶

The answer to a Choice: one of its option ids.

Probability `dataclass` ¶

The answer to a YesNo: the probability of yes (0 to 1).

ModelCall `dataclass` ¶

One request to the model, for reports and -v output.

Attributes:

Name	Type	Description
`state`	`State`	What the model was shown.
`questions`	`Mapping[str, Question]`	What it was asked.
`answers`	`Mapping[str, Answer]`	What it answered.
`recorded`	`bool`	Whether the answers came from the lockfile rather than the model.
`ms`	`int`	How long the model took (0 when recorded).
`cost`	`float \| None`	What the request cost in US dollars (0 when recorded; None when the price isn't known).
`served_by`	`str \| None`	The model version that answered.

results ¶

What happened when tests ran: plain records, built by the runner, read by reports and the console.

CheckResult `dataclass` ¶

How one check went.

Attributes:

Name	Type	Description
`check`	`Check`	The check.
`status`	`Status`	Pass or fail.
`detail`	`str \| None`	What was found (e.g. Jev's probability), or why it failed.
`model_calls`	`tuple[ModelCall, ...]`	The model requests the check made.

StepResult `dataclass` ¶

How one step went.

Attributes:

Name	Type	Description
`step`	`Step`	The step.
`status`	`Status`	Pass or fail.
`seconds`	`float`	How long it took.
`detail`	`str \| None`	What the action did (e.g. which element it tapped), or why it failed.
`decisions`	`tuple[Decision, ...]`	For a `do:` step worked out by Jev, the moves it chose, in order.
`model_calls`	`tuple[ModelCall, ...]`	The model requests the action made (the checks keep their own).
`checks`	`tuple[CheckResult, ...]`	The checks that ran, in order; the first failure stops them.
`steps`	`tuple[StepResult, ...]`	For a `use:` step, the used test's step results.
`screenshot`	`str \| None`	The screenshot taken when this step failed the test.
`ran`	`tuple[str, ...]`	For a `do:` step that repeated its saved steps, each one as it ran.

failure `property` ¶

failure: str | None

Plain-English reason for the step's first failure, or None if it passed.

TestResult `dataclass` ¶

How one test went.

Attributes:

Name	Type	Description
`name`	`str`	The test's name.
`status`	`Status`	Pass or fail.
`seconds`	`float`	How long it took.
`steps`	`tuple[StepResult, ...]`	The step results, up to and including the first failure.
`start_failure`	`str \| None`	Why the app could not be started, if it couldn't.
`screenshot`	`str \| None`	The screenshot taken when the app could not be started.

failure `property` ¶

failure: str | None

Plain-English reason the test failed, or None if it passed.

RunResult `dataclass` ¶

The results of every test a device ran.

passed `property` ¶

passed: int

How many tests passed.

failed `property` ¶

failed: int

How many tests failed.

seconds `property` ¶

seconds: float

How long the tests took, added up.

failures ¶

The things that can go wrong, as jevtest's own exception types.

Adapters catch the exceptions of the tools they wrap (subprocess, HTTP, YAML, ...) and raise one of these instead, so nothing outside an adapter ever handles a tool's exception. Every message says what to fix.

JevtestError ¶

Bases: Exception

Base class: something jevtest could not do. The message says why and what to fix.

TestFileError ¶

Bases: JevtestError

A test file, a file it includes, its .env, or the command line is wrong.

DeviceError ¶

Bases: JevtestError

The device, its tools, or jevtest's on-device agent couldn't do something.

ModelError ¶

Bases: JevtestError

The decision model (Jev) couldn't be asked, or answered something unusable.

NotRecorded ¶

Bases: ModelError

--lock frozen: the screen and question aren't in the lockfile.

The screen may still be changing, so a step that waits looks again until its timeout.

StepFailed ¶

Bases: JevtestError

A step or check didn't get the result the test expects: the test fails, the run goes on.

SetupRefused ¶

Bases: JevtestError

Jev refused the run's setup: its API key, or its model.

Every question would be refused the same way, so the run stops rather than fail each test in turn.

ports ¶

The interfaces jevtest's core needs from the outside world.

The application layer depends only on these. Adapters implement them; the command line wires implementations to them in one place (jevtest.cli.main), which is also what lets tests pass fakes instead.

Device ¶

Bases: Protocol

A phone, emulator or simulator with the app under test installed on it.

Every method raises DeviceError when the device can't do it.

install ¶

install(app: Path) -> str

Install the build and return its package or bundle id.

prepare_for_test ¶

prepare_for_test() -> None

Before each test: check the device can be tested, and start again what a failed test lost.

Raises DeviceError if it can't be tested now (e.g. it's locked). A device starts again what it needs that a failed test lost (such as Android's agent).

launch ¶

launch() -> None

Launch the app and wait until it is in the foreground.

resume ¶

resume() -> None

Bring the app back to the foreground without restarting it.

stop ¶

stop() -> None

Stop the app.

clear_data ¶

clear_data() -> None

Clear the app's data, as if newly installed.

reinstall ¶

reinstall() -> None

Uninstall and install the build again.

app_state ¶

app_state() -> AppState

Where the app is. Cheap: it doesn't read the screen.

screen ¶

screen() -> Screen

What's on the screen now.

screenshot ¶

screenshot(path: Path) -> None

Save a PNG of the screen.

looks ¶

looks(elements: Sequence[Element]) -> str

A fingerprint of how the elements are drawn now, where their bounds alone can't show them moving; else "".

tap ¶

tap(x: int, y: int) -> None

Tap a point.

double_tap ¶

double_tap(x: int, y: int) -> None

Double-tap a point.

long_press ¶

long_press(x: int, y: int) -> None

Press and hold a point.

swipe ¶

swipe(
    direction: Direction,
    element: Element | None = None,
    screen: Screen | None = None,
    distance: float | None = None,
) -> None

Swipe on an element, or across the part of the screen the keyboard doesn't cover.

distance is how far the finger moves, in percent of that; None: SWIPE_DISTANCE's.

scroll ¶

scroll(
    direction: Direction,
    screen: Screen | None = None,
    lane: Bounds | None = None,
) -> None

Scroll the content so more of it in direction comes into view: the page's, or lane's.

lane is where a finger drags along a carousel, say (Screen.swiped).

type_text ¶

type_text(text: str, at: Point | None = None) -> None

Type into the focused field, or first focus the field at at.

choose ¶

choose(picker: Element, value: str) -> None

Turn the picker wheel picker to value, as a finger spinning it would.

clear_text ¶

clear_text(element: Element) -> None

Erase a text field.

key ¶

key(name: str) -> None

Press a named key.

back ¶

back() -> None

Go back.

home ¶

home() -> None

Go to the home screen.

hide_keyboard ¶

hide_keyboard() -> None

Close the on-screen keyboard.

rotate ¶

rotate(orientation: Orientation) -> None

Rotate the device. Put back when the device is closed.

set_location ¶

set_location(latitude: float, longitude: float) -> None

Set the GPS location.

open_url ¶

open_url(url: str) -> None

Open a deep link or URL.

dark_mode ¶

dark_mode(*, on: bool) -> None

Switch dark appearance on or off. Put back when the device is closed.

grant ¶

grant(permissions: Sequence[str]) -> None

Grant the app runtime permissions, by this platform's names for them.

network ¶

network(*, on: bool) -> None

Switch Wi-Fi and mobile data on or off. Put back when the device is closed.

autofill_off ¶

autofill_off() -> None

Turn off the autofill service, a password manager offering to save what was typed. Put back when closed.

restore ¶

restore() -> None

Put back what steps changed (orientation, appearance, network, autofill, location).

Called before each fresh test, so one test's changes never leak into the next.

close ¶

close() -> None

Put back what steps changed and release everything the device started.

DecisionModel ¶

Bases: Protocol

The model that makes jevtest's judgement calls (Jev), behind its lockfile.

calls `property` ¶

calls: Sequence[ModelCall]

Every request made so far, in order.

replays_only `property` ¶

replays_only: bool

Whether only recorded answers and saved steps may be used (--lock frozen): nothing new is asked.

ask ¶

ask(
    state: State, questions: Mapping[str, Question]
) -> Mapping[str, Answer]

Ask several questions about one state; get one answer per question.

Raises:

Type	Description
`ModelError`	The model couldn't be asked, or answered outside the options.
`NotRecorded`	The answer isn't recorded, and only recorded answers may be used.

saved_steps ¶

saved_steps(key: str) -> tuple[SavedStep, ...] | None

The steps saved for the do: goal key, to repeat; None when it's to be worked out with Jev.

Raises:

Type	Description
`NotRecorded`	None are saved, and only saved steps may be used.

save_steps ¶

save_steps(key: str, steps: Sequence[SavedStep]) -> None

Save the steps the do: goal key took, for later runs to repeat.

ScreenNotesWriter ¶

Bases: Protocol

Saves what was on a screen beside its screenshot, for people and agents writing tests.

write ¶

write(screenshot: Path, notes: ScreenNotes) -> None

Save notes beside screenshot (same name, other extensions).

Clock ¶

Bases: Protocol

Time, so tests can use a fake one.

now ¶

now() -> float

Seconds on a monotonic clock.

sleep ¶

sleep(seconds: float) -> None

Wait.

RunListener ¶

Bases: Protocol

Told what happens as tests run, so it can be shown as it happens.

test_started ¶

test_started(test: Test) -> None

A test is starting.

start_failed ¶

start_failed(reason: str) -> None

The app could not be started for the test that just started.

use_started ¶

use_started(name: str, depth: int) -> None

A use: step is running another test's steps.

step_done ¶

step_done(result: StepResult, depth: int) -> None

A step's action finished (its checks follow).

check_done ¶

check_done(result: CheckResult, depth: int) -> None

A check finished.

test_done ¶

test_done(result: TestResult) -> None

A test finished.

settings ¶

Settings: what a test file may tune, the defaults that suit most apps, and the limits that keep a test honest.

A test file needs no settings at all. Its settings: block changes a default for every step in the file, and a step can change timeout, interval, max_actions, max_scrolls or confidence for itself. Every value must be within LIMITS: a limit is where a setting stops tuning a test and starts hiding a problem in the app.

JEV_VERSION `module-attribute` ¶

JEV_VERSION = re.compile('jev-\\d+\\.\\d+\\.\\d+')

jevtest uses Jev only, pinned to a version (jev-1.13.0): an alias such as jev-latest moves when a new Jev ships, so recorded decisions would no longer match what a live run asks.

LIMITS `module-attribute` ¶

LIMITS: Mapping[str, tuple[float, float]] = {
    "timeout": (1, 300),
    "interval": (0.05, 2),
    "max_actions": (1, 50),
    "max_scrolls": (1, 500),
    "confidence": (0.5, 0.99),
}

The lowest and highest value of each numeric setting, inclusive.

WHOLE `module-attribute` ¶

WHOLE = frozenset({'max_actions', 'max_scrolls'})

Settings that count something, so take whole numbers.

STEP_SETTINGS `module-attribute` ¶

STEP_SETTINGS = frozenset(LIMITS)

Settings a step can change for itself. model is for the whole file: one file, one Jev.

REMOVED `module-attribute` ¶

REMOVED: Mapping[str, str] = {
    "settle": "`settle` is gone (jevtest 0.9): each step waits until what it needs is on screen; remove it"
}

Settings older versions took, and what to do instead.

DEFAULTS `module-attribute` ¶

DEFAULTS = Settings()

The settings of a file without a settings: block.

Settings `dataclass` ¶

How steps wait, how far they go, and how sure Jev must be.

Attributes:

Name	Type	Description
`model`	`str`	The Jev version that decides and judges. Part of every lockfile key, so changing it re-asks Jev.
`timeout`	`float`	Seconds a step waits until what it needs is true (its element is on screen, its check holds).
`interval`	`float`	Seconds between two checks of what a step waits for.
`max_actions`	`int`	Actions a `do:` goal may take.
`max_scrolls`	`int`	Scrolls a `scroll_to:` may make. It also stops at the end of the content.
`confidence`	`float`	An `expect:` passes when Jev's probability that the statement is true is above this.

changed ¶

changed(values: Mapping[str, float]) -> Settings

These settings with some numeric ones changed (by name); a count is kept a whole number.

kinds ¶

Small closed sets of values, as enums instead of loose strings.

Platform ¶

Bases: StrEnum

A mobile platform jevtest can test on.

AppState ¶

Bases: StrEnum

Where the app under test is, as the device reports it.

Direction ¶

Bases: StrEnum

A direction to scroll or swipe.

opposite `property` ¶

opposite: Direction

The other way along the same line.

Orientation ¶

Bases: StrEnum

A device orientation.

Gesture ¶

Bases: StrEnum

A touch on one point of the screen.

Status ¶

Bases: StrEnum

The outcome of a test, step or check.

variables ¶

${NAME} values: written as names in test files, filled in only where the app needs them.

VARIABLE `module-attribute` ¶

VARIABLE = re.compile('\\$\\{([A-Za-z_][A-Za-z0-9_]*)\\}')

A ${NAME} reference.

names_in ¶

names_in(text: str) -> list[str]

The variable names text refers to, in order.

fill ¶

fill(text: str, variables: Mapping[str, str]) -> str

text with each ${NAME} replaced by its value.

Used only for what the app sees (typed text, compared text, searched-for elements, URLs). Logs, reports and everything sent to Jev keep the names (see hide), so secrets stay out of them.

Raises:

Type	Description
`KeyError`	`text` names a variable that `variables` doesn't have.

hide ¶

hide(text: str, variables: Mapping[str, str]) -> str

text with every variable's value replaced by its ${NAME}: the reverse of fill.

Used on everything read from the screen before it leaves the device's side: what Jev is sent, and what jevtest prints. An app that shows a value (a signed-in user's email) keeps it out of Jev's requests, logs and lockfile keys. Longer values are replaced first, so a value inside another is never left half hidden. Both are compared in one Unicode encoding (NFC): an é shown as an e and an accent is still the value's é. A value is also hidden as jevtest reads screen text (one_line): a b typed into a field reads back as a b.

words ¶

Words as jevtest writes and compares them: numbers, and text in one Unicode encoding.

nfc ¶

nfc(text: str) -> str

text with each letter in one encoding (NFC): an é written as an e and an accent becomes the one-character é.

Apps and keyboards write both; compared this way, they are the same text.

number_text ¶

number_text(value: float) -> str

A number as a person wrote it, never rounded: 2 for 2.0, 151.2093 for 151.2093.

one_line ¶

one_line(text: str) -> str

text as jevtest reads it from a screen.

Each run of spaces, tabs and line breaks becomes one space, with none at the ends.

lines ¶

lines(text: str) -> tuple[str, ...]

Each line of text that says something, as one_line reads it; none when it has only one.

An app that joins several texts into one element separates them with line breaks: Flutter reports a tab as Form and Tab 2 of 3, one line each.

plural ¶

plural(n: int, noun: str, many: str | None = None) -> str

1 file, 2 files; many for a noun that doesn't just add an s (entry, entries).

ordinal ¶

ordinal(n: int) -> str

1st, 2nd, 3rd, 4th, ... 11th, 21st.

Application¶

runner ¶

Running tests: each step's action on the device, then its checks, until the first failure.

The runner reaches the device, the model and time only through the domain's ports, reports progress to a RunListener as it goes, and returns plain result records. It prints nothing and writes no files except the screenshots it is asked to take.

STUCK `module-attribute` ¶

STUCK = 2

Times in a row a do: move may leave the screen as it was before the goal fails: the move does nothing.

END_OF_CONTENT `module-attribute` ¶

END_OF_CONTENT = 2

Scrolls in a row that must move nothing before scroll_to: calls it the end. One isn't enough: a real phone's web view sometimes ignores a single scroll.

KEYBOARD_UP `module-attribute` ¶

KEYBOARD_UP = "; the keyboard is up, and the app may not show it while it is (in landscape, an Android keyboard leaves the app a strip): close it first with a `hide_keyboard` step"

Added when what a step looks for isn't on screen and the keyboard is up.

OFF_SCREEN `module-attribute` ¶

OFF_SCREEN = "; the app has it off screen: bring it on screen first, e.g. with `scroll_to:`"

Added when what a step looks for isn't on screen but the app reports it where the screen doesn't show it.

LEFT_CONFIRM `module-attribute` ¶

LEFT_CONFIRM = 4

Checks, LEFT_INTERVAL apart, that another app is still on top before the app counts as having left. A touch in Android 15's gesture strip puts the home screen on top for a moment while the phone decides whether it's a swipe home (measured: 20-65 ms after a tap, then the app again).

LEFT_INTERVAL `module-attribute` ¶

LEFT_INTERVAL = 0.25

Seconds between the checks of LEFT_CONFIRM.

PICKER `module-attribute` ¶

PICKER = 'picker'

A picker wheel: typing into one turns it to that value (Device.choose).

Stillness ¶

Whether something has stopped moving: it looks the same at two checks in a row.

still ¶

still(now: object) -> bool

Whether now is what the previous check saw; now is what the next check compares with.

forget ¶

forget() -> None

Start again: the next check has nothing to compare with.

TestRunner ¶

Runs a suite's tests on one device.

Every wait is one wait_until: a condition on the screen, checked every interval seconds, for at most timeout seconds; if it isn't met by then, the step fails and says what it waited for.

Parameters:

Name	Type	Description	Default
`suite`	`Suite`	The test file: its tests, the tests `use:` can name, and its `${NAME}` values.	required
`device`	`Device`	The device, with the app installed.	required
`brain`	`Brain`	jevtest's judgement, backed by the decision model.	required
`screenshots`	`Path`	Where failure screenshots (and `screenshot:` steps) are saved.	required
`platform`	`Platform`	The device's platform (`android`, `ios`): part of the key a `do:` goal's steps are saved under, since the same goal takes different steps on each.	required
`clock`	`Clock`	Time.	required
`listener`	`RunListener`	Told about each test, step and check as it finishes.	required
`notes`	`ScreenNotesWriter`	Saves what was on the screen beside each screenshot, as steps name it.	required

run ¶

run() -> RunResult

Run every test in the suite, in order.

run_test ¶

run_test(test: Test) -> TestResult

Start the app, then run the test's steps until one fails.

brain ¶

The questions jevtest asks its decision model, and what it does with the answers.

The model only picks from options it is given, so every question is a Choice over things jevtest can actually do on the current screen, or a YesNo. Text to type always comes from the test file, never from the model.

The wording of the questions and of the screen description is part of every recorded decision: change it and recorded lockfiles no longer match.

Located `dataclass` ¶

The element a target names, and how it was found.

Attributes:

Name	Type	Description
`element`	`Element`	The element.
`chosen`	`str \| None`	None when its text is the target; otherwise how Jev chose it, for the step's output.

describe ¶

describe() -> str

E.g. button 'Save', or button 'Gear' (chosen by Jev).

Brain ¶

jevtest's judgement: the next move toward a goal, which element a target means, and checks.

Parameters:

Name	Type	Description	Default
`model`	`DecisionModel`	Where the questions go (Jev behind its lockfile, or a fake).	required

next_action ¶

next_action(
    goal: str, screen: Screen, actions_taken: Sequence[str]
) -> Decision

One model request: the next move toward goal from screen.

The target, field and value are asked in the same request (they're only used when the chosen action needs them), so each move costs one round trip.

locate ¶

locate(
    target: str,
    screen: Screen,
    candidates: Sequence[Element] | None = None,
) -> Located | None

The element the test file names, or None if no element on the screen says it.

Only an exact match counts: an element whose whole text, one of its parts, its hint or its id is the target, ignoring case (Element.says). If several match, Jev chooses among those only (and must confirm its choice). A target no element says is never guessed: describe it in a do: step instead.

check ¶

check(statement: str, screen: Screen) -> float

The model's probability that statement is true of the screen.

quoted_values ¶

quoted_values(goal: str) -> list[str]

The values a goal puts in quotes: the only text the model may have typed.

describe ¶

describe(screen: Screen) -> list[dict[str, object]]

The screen as the model reads it: one short record per element, positions in words.

planning ¶

Splitting a file's tests across several devices.

shard ¶

shard(
    tests: Sequence[Test], devices: int
) -> list[tuple[Test, ...]]

Deal the tests out to devices devices, in order, one group per device in turn.

A fresh: false test goes where the test before it went, since it carries on from where that one left the app. A device can end up with nothing to run.

Python API¶

Domain¶

steps ¶

SWIPE_DISTANCE module-attribute ¶

KEYS module-attribute ¶

ANDROID_KEYS module-attribute ¶

Action module-attribute ¶

Check module-attribute ¶

SETTING_SCOPES module-attribute ¶

Do dataclass ¶

Use dataclass ¶

Touch dataclass ¶

Clear dataclass ¶

TypeText dataclass ¶

Scroll dataclass ¶

Swipe dataclass ¶

ScrollTo dataclass ¶

Key dataclass ¶

Wait dataclass ¶

Background dataclass ¶

Rotate dataclass ¶

Location dataclass ¶

OpenUrl dataclass ¶

DarkMode dataclass ¶

Grant dataclass ¶

names_on ¶

Network dataclass ¶

AutofillOff dataclass ¶

Screenshot dataclass ¶

Launch dataclass ¶

Stop dataclass ¶

Restart dataclass ¶

ClearData dataclass ¶

Reinstall dataclass ¶

Back dataclass ¶

Home dataclass ¶

HideKeyboard dataclass ¶

Expect dataclass ¶

See dataclass ¶

NotSee dataclass ¶

Step dataclass ¶

Test dataclass ¶

Suite dataclass ¶

steps ¶

Scope dataclass ¶

waits ¶

screen ¶

Bounds module-attribute ¶

Point module-attribute ¶

STRAIGHT_QUOTES module-attribute ¶

FILLS module-attribute ¶

EDGE module-attribute ¶

MIN_LANE module-attribute ¶

SIDE_EDGE module-attribute ¶

NEAR_LIMIT module-attribute ¶

Element dataclass ¶

slider property ¶

center property ¶

tap_point property ¶

end property ¶

says ¶

names ¶

label ¶

Screen dataclass ¶

content_height property ¶

page property ¶

editable property ¶

under_keyboard ¶

swiped ¶

under_system_bar ¶

clear_of_edges ¶

in_lane ¶

takes_keys ¶

by_id ¶

shows ¶

off_screen ¶

near ¶

region ¶

folded ¶

near_names ¶

SWIPE_DISTANCE `module-attribute` ¶

KEYS `module-attribute` ¶

ANDROID_KEYS `module-attribute` ¶

Action `module-attribute` ¶

Check `module-attribute` ¶

SETTING_SCOPES `module-attribute` ¶

Do `dataclass` ¶

Use `dataclass` ¶

Touch `dataclass` ¶

Clear `dataclass` ¶

TypeText `dataclass` ¶

Scroll `dataclass` ¶

Swipe `dataclass` ¶

ScrollTo `dataclass` ¶

Key `dataclass` ¶

Wait `dataclass` ¶

Background `dataclass` ¶

Rotate `dataclass` ¶

Location `dataclass` ¶

OpenUrl `dataclass` ¶

DarkMode `dataclass` ¶

Grant `dataclass` ¶

Network `dataclass` ¶

AutofillOff `dataclass` ¶

Screenshot `dataclass` ¶

Launch `dataclass` ¶

Stop `dataclass` ¶

Restart `dataclass` ¶

ClearData `dataclass` ¶

Reinstall `dataclass` ¶

Back `dataclass` ¶

Home `dataclass` ¶

HideKeyboard `dataclass` ¶

Expect `dataclass` ¶

See `dataclass` ¶

NotSee `dataclass` ¶

Step `dataclass` ¶

Test `dataclass` ¶

Suite `dataclass` ¶

Scope `dataclass` ¶

Bounds `module-attribute` ¶

Point `module-attribute` ¶

STRAIGHT_QUOTES `module-attribute` ¶

FILLS `module-attribute` ¶

EDGE `module-attribute` ¶

MIN_LANE `module-attribute` ¶

SIDE_EDGE `module-attribute` ¶

NEAR_LIMIT `module-attribute` ¶

Element `dataclass` ¶

slider `property` ¶

center `property` ¶

tap_point `property` ¶

end `property` ¶

Screen `dataclass` ¶

content_height `property` ¶

page `property` ¶

editable `property` ¶

ElementMove `module-attribute` ¶

PageMove `module-attribute` ¶

Move `module-attribute` ¶

STEP_ACTIONS `module-attribute` ¶

Finished `dataclass` ¶

Impossible `dataclass` ¶

WaitForScreen `dataclass` ¶

GoBack `dataclass` ¶

PressEnter `dataclass` ¶

CloseKeyboard `dataclass` ¶

ScrollPage `dataclass` ¶

TouchElement `dataclass` ¶

SwipeElement `dataclass` ¶

TypeInto `dataclass` ¶

ClearField `dataclass` ¶

Decision `dataclass` ¶

Target `dataclass` ¶

SavedStep `dataclass` ¶

Question `module-attribute` ¶

Answer `module-attribute` ¶

State `module-attribute` ¶

Choice `dataclass` ¶

YesNo `dataclass` ¶