· 5 min read

Maestro in GitHub Actions: Smoke Tests on Every PR

How I run Maestro flows in GitHub Actions on an Android emulator, with tags for a fast smoke suite on pull requests and a full suite every night.

Maestro tests that only run on a laptop stop protecting you after a few weeks. Someone forgets to run them before a merge, a flow breaks, and nobody notices until a release. Running Maestro in GitHub Actions fixes that, but the obvious version, "run every flow on every pull request", is slow enough that people start to ignore it. This post shows the setup I use instead: a short smoke suite on every PR, the full suite every night, and test reports and failure screenshots saved where you can find them.

Split the suite with tags

The first decision is not about CI at all. It is which flows must run on every change. I keep that list short: the app starts, the main screen loads, and the one or two paths that would make the app useless if they broke. Everything else waits for the nightly run.

Maestro supports this with tags. You put them in the header of a flow, above the --- line:

appId: com.example.app
tags:
  - smoke
---
- launchApp:
    clearState: true
- assertVisible:
    id: "home_screen"

Then you pick flows from the command line:

maestro test .maestro --include-tags=smoke
maestro test .maestro --exclude-tags=wip

I use only three tags:

One rule I hold myself to: wip is not a place to hide a flaky flow. If a flow fails at random, it usually waits for time instead of state. I covered how to fix that in waiting for app state, not for time. A flow you tag away stops testing anything.

The GitHub Actions workflow

Here is the workflow. It builds a Flutter APK, starts an Android emulator, runs Maestro and uploads the results. The same steps work for a native Android app if you swap the build command.

name: e2e-android

on:
  pull_request:
  schedule:
    - cron: "0 2 * * *"
  workflow_dispatch:

jobs:
  maestro:
    runs-on: ubuntu-latest
    timeout-minutes: 45
    env:
      E2E_SUITE: ${{ github.event_name == 'pull_request' && 'smoke' || 'full' }}
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "17"

      - uses: subosito/flutter-action@v2
        with:
          channel: stable
          cache: true

      - name: Build APK
        run: flutter build apk --debug

      - name: Enable KVM
        run: |
          echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
          sudo udevadm control --reload-rules
          sudo udevadm trigger --name-match=kvm

      - name: Install Maestro
        run: |
          curl -fsSL "https://get.maestro.mobile.dev" | bash
          echo "$HOME/.maestro/bin" >> "$GITHUB_PATH"

      - name: Run Maestro on emulator
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 34
          arch: x86_64
          target: google_apis
          script: ./scripts/e2e.sh

      - name: Upload Maestro results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: maestro-results-${{ env.E2E_SUITE }}
          path: build/maestro

A few lines need a word of explanation.

The suite is picked by the event

E2E_SUITE is smoke for pull requests and full for the nightly schedule and manual runs. The workflow file stays the same for both. Only the set of flows changes.

KVM makes the emulator usable

Without hardware acceleration, an x86_64 emulator on a Linux runner is very slow. The three udevadm lines give the runner user access to /dev/kvm. This is the step the android-emulator-runner README recommends for Ubuntu runners.

The script lives in a file

The emulator action runs an inline script line by line, not as one shell script. Variables and if blocks that span lines do not behave the way you expect. So I keep the logic in scripts/e2e.sh and call that one file. Remember to make it executable with chmod +x before you commit it.

The test script

#!/usr/bin/env bash
set -euo pipefail

mkdir -p build/maestro
adb install -r build/app/outputs/flutter-apk/app-debug.apk

args=(--exclude-tags=wip)
if [ "${E2E_SUITE:-smoke}" = "smoke" ]; then
  args+=(--include-tags=smoke)
fi

maestro test .maestro \
  "${args[@]}" \
  --format junit \
  --output build/maestro/report.xml \
  --test-output-dir build/maestro

Two details matter here.

First, --include-tags is added only for the smoke suite. The full suite excludes wip and runs everything else. When you combine both flags, Maestro selects the included flows first and then removes the excluded ones.

Second, the output goes to one folder. --format junit with --output writes a JUnit XML report. --test-output-dir is where Maestro saves the screenshot of the failing step, the view hierarchy at that moment, and a commands.json log for each flow. The Maestro docs on reports and artifacts list everything that ends up there. Note that the JUnit file is not written into the output directory on its own; that is why I point --output at the same folder.

Because the upload step has if: always(), the folder is uploaded even when Maestro fails the job. That is exactly when you need it.

Reading a failure without re-running it

A red CI job is only useful if you can see why it is red. With this setup, a failure gives you three things:

  1. The job log, with the Maestro command that failed.
  2. A screenshot of the screen at the failing step.
  3. The view hierarchy at that step, so you can check whether the element was there with a different id.

The third one saves the most time. Often the screenshot shows the right screen, but the flow still fails. The hierarchy then shows that the widget lost its semantics identifier, or that a label changed. If your Flutter widgets do not expose stable ids, start there: I wrote about Semantics identifiers for Maestro.

My order of work on a red smoke run is simple. Download the artifact, open the screenshot, open the hierarchy, then decide: is the app broken, or is the flow? Only then do I run it locally.

What about iOS?

Maestro also runs on the iOS Simulator, but that needs a macOS runner. On GitHub Actions, macOS minutes cost more than Linux minutes, and the simulator setup takes longer. So I keep pull requests on Android only and run iOS flows on a slower schedule or before a release. Most flow logic is shared between platforms in a Flutter app, so the Android smoke suite catches most regressions early. Platform-specific screens, like permission dialogs, still need the iOS run.

If you build and ship the iOS app with Xcode Cloud, the build side is a separate topic. I covered it in Flutter on Xcode Cloud with ci_post_clone.

Takeaways