Opening deep links on iOS simulator extremely flaky on GitHub Actions

[winghouchan]

Is there an existing issue for this?

• I have searched the existing issues and didn't find mine.

Steps to reproduce

Using the reproduction repository

See GitHub Action runs in reproduction repository. Note, some runs have had multiple attempts. Maestro logs/screenshots are saved as artifacts labelled with the Maestro, iOS, and Xcode versions. The artifacts will be retained for 30 days from when the flow was run (which at the time of writing, is the same as when this issue was created). If you need the artifacts and they have expired, drop me a message and I'll trigger re-runs.

Each workflow run executes a test which opens a deep link and asserts the correct screen is opened. The variable between each run is the Maestro version: 1.39.0, 1.39.13, 1.40.0, and 1.41.0 have been tested. iOS versions (18.0–18.5) and Xcode versions (16.3, 16.4) are controlled variables for all runs.

Self-reproduction

1. Have a flow which opens a deep link. Ensure the app has not been opened before. The reproduction has the following flow:

   name: Open deep link
   appId: com.winghouchan.maestroiosdeeplinkmcve
   tags:
     - subflow
   ---
   - openLink:
       link: com.winghouchan.maestroiosdeeplinkmcve://foo
   
   - runFlow:
       when:
         visible: ".*Open in.*"
       commands:
         - tapOn:
             text: Open
   
   - assertVisible: /foo

2. Run the tests on GitHub Actions (environment details listed in a later section or see the linked workflow). The reproduction has this workflow: https://github.com/winghouchan/maestro-ios-deep-link-mcve/blob/7fcf4add9080643c65cb9ab79e41f3f2eaf1ab8a/.github/workflows/reproduction.yaml.

Actual results

For Maestro versions 1.40.0 and 1.41.0, the assertion for the expected screen fails the majority of the time as the app doesn't seem to open after the "Open" button in the "Open in..." dialog is pressed.

For Maestro versions 1.39.13 and 1.39.0 (probably versions <=1.39.13 can be included), all steps run as expected.

Expected results

The app is opened at the correct screen by the runner and the assertion passes.

About app

The MCVE app is a React Native (0.79.5) / Expo (53.0.20) app. Source available in the app directory of the reproduction repository. Binary available in the build directory of the reproduction repository.

About environment

Environment is a GitHub-hosted macOS 15 runner. Full details here.

Summary:

• macOS version: 15.5
• Processor architecture: arm64
• Java version: 21
• Xcode version(s): 16.3, 16.4

Test flow passes as expected when run locally on macOS 15.5 (arm64), Java 21, Xcode 16.4.

Logs

See GitHub Action run artifacts for logs for all runs. A representative log has been attached.

maestro.log

Maestro version

• 1.40.0
• 1.41.0
• 2.0.x

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

No response

[linear]

MAE-241 Opening deep links on iOS simulator extremely flaky on GitHub Actions

[harkairt]

Related #940

[howwellapp]

I have the same problem. Maestro 1.39.5, iPhone 16 Pro Max - 18.6.

Deep link dialog opens:

My test has:

      - runFlow:
          when:
            visible: Open
          commands:
            - tapOn: Open

Roughly 80% of the time, the tap works as expected (app opens).

Roughly 20% of the time, the app does not open. I do see the tap in the video -- you can see the button get darker:

And maestro log has tap:

MaestroCommand(runFlowCommand=RunFlowCommand(commands=[MaestroCommand(runFlowCommand=RunFlowCommand(commands=[MaestroCommand(tapOnElementCommand=TapOnElementCommand(selector=ElementSelector(textRegex=Open, idRegex=null, size=null, below=null, above=null, leftOf=null, rightOf=null, containsChild=null, containsDescendants=null, traits=null, index=null, enabled=null, optional=false, selected=null, checked=null, focused=null, childOf=null), retryIfNoChange=true, waitUntilVisible=false, longPress=false, repeat=null, waitToSettleTimeoutMs=null, label=null, optional=false))], condition=Condition(platform=null, visible=ElementSelector(textRegex=Open, idRegex=null, size=null, below=null, above=null, leftOf=null, rightOf=null, containsChild=null, containsDescendants=null, traits=null, index=null, enabled=null, optional=false, selected=null, checked=null, focused=null, childOf=null), notVisible=null, scriptCondition=null, label=null), sourceDescription=null, config=null, label=null, optional=false))], condition=Condition(platform=IOS, visible=null, notVisible=null, scriptCondition=${IOS_UPDATE_ID}, label=null), sourceDescription=null, config=null, label=null, optional=false))], condition=null, sourceDescription=helper/openApp.yaml, config=MaestroConfig(appId=com.howwellapp.howwellapp.dev, name=null, tags=[], ext={}, onFlowStart=null, onFlowComplete=null), label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE), action=null)
23:13:11.069 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Tap on "Open" metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(tapOnElementCommand=TapOnElementCommand(selector=ElementSelector(textRegex=Open, idRegex=null, size=null, below=null, above=null, leftOf=null, rightOf=null, containsChild=null, containsDescendants=null, traits=null, index=null, enabled=null, optional=false, selected=null, checked=null, focused=null, childOf=null), retryIfNoChange=true, waitUntilVisible=false, longPress=false, repeat=null, waitToSettleTimeoutMs=null, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE), action=TapPoint(point=Point(x=287, y=533)))
23:13:11.070 [ INFO] maestro.Maestro.tap-BUbHBYE: Tapping on element:  UiElement(treeNode=TreeNode(attributes={accessibilityText=Open, title=, value=, text=, hintText=, resource-id=, bounds=[220,511][355,555], enabled=true, focused=false, selected=false, checked=false}, children=[], clickable=null, enabled=true, focused=false, checked=false, selected=false), bounds=Bounds(x=220, y=511, width=135, height=44))
23:13:11.070 [ INFO] ios.IOSDevice.invoke-F9mIFpk: Waiting for animation to end with timeout 3000
23:13:11.468 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Screen diff request finished with isScreenStatic = true
23:13:11.469 [ INFO] ios.IOSDevice.invoke: screen static = true
23:13:11.470 [ INFO] maestro.Maestro.tap-BUbHBYE: Refreshed element
23:13:11.471 [ INFO] maestro.Maestro.screenshotBasedTap-hbl3e4M: Tapping at (287, 533) using screenshot based logic for wait
23:13:12.008 [ INFO] ios.IOSDevice.invoke-F9mIFpk: Waiting for animation to end with timeout 3000
23:13:12.411 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Screen diff request finished with isScreenStatic = true
23:13:12.412 [ INFO] ios.IOSDevice.invoke: screen static = true
23:13:12.412 [ INFO] maestro.Maestro.screenshotBasedTap-hbl3e4M: Something has changed in the UI judging by view hierarchy. Proceed.
23:13:12.412 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Tap on "Open" COMPLETED
23:13:12.412 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run flow when "Open" is visible COMPLETED

However, app does not open.

[winghouchan]

Note

This is still reproducible in Maestro 2.0.x

[HughDillon27]

Same problem here, I have consistently seen this command cause our test run to completely hang and timeout, the test with the openLink command in it completes with a failure, but subsequent tests are not started

[vitalyiegorov]

Same here, spent a week on this

vitalyiegorov/suuudokuuu#102

[Fishbowler]

Seems to be lots of folks with issues in GHA, but not in other platforms. We need someone who knows GHA who cares about this issue enough to go digging and work out why performance is a pig for iOS :(