Skip to content

Commit 77d6f41

Browse files
committed
docs(analyze): Adds new --analyze flag docs and assets
1 parent b1ce863 commit 77d6f41

File tree

2 files changed

+77
-0
lines changed

2 files changed

+77
-0
lines changed

.gitbook/assets/analyze-report.png

393 KB
Loading

cli/test-suites-and-reports.md

+77
Original file line numberDiff line numberDiff line change
@@ -112,3 +112,80 @@ Note that your Flows should **not** depend on device state and should be treated
112112
#### Configuring part of the Flows to run sequentially
113113

114114
For instance, if you have three Flows, `flowA`, `flowB`, and `flowC`, but you want to run only `flowA` and `flowB` sequentially, don't add `flowC` and `flowD` to the list. Maestro will run these Flows in non-deterministic ordering **after** the Flow sequence has finished executing.
115+
116+
#### Analyze
117+
118+
{% hint style="warning" %}
119+
120+
This is an **experimental** feature powered by LLM technology. All feedback is
121+
welcome.
122+
123+
{% endhint %}
124+
125+
Maestro introduces a new feature that leverages AI to analyze your end-to-end (E2E) mobile tests and provide actionable insights based on your test logs, commands, and screenshots captured during your test runs. The AI-powered analysis identifies potential issues in your app's functionality, UI, and internationalization, helping you improve app quality efficiently.
126+
127+
**Login Requirement:** Before you use the AI analysis feature, ensure you are logged into Maestro. Run the following command:
128+
129+
```bash
130+
maestro login
131+
```
132+
133+
> Quick Note: You don't need to be a paying customer to try out this feature. Users on the 7-day trial can also leverage this capability.
134+
135+
**Analyzing your tests**: To analyze your test flows with AI, use the --analyze flag with the maestro test command:
136+
137+
```bash
138+
maestro test flow-file.yaml --analyze
139+
```
140+
141+
{% hint style="info" %}
142+
143+
While we aim for precision, please note that this is a beta release, and the results should be validated before making critical decision
144+
145+
{% endhint %}
146+
147+
This will enable AI analysis and provide a detailed report based on your test artifacts:
148+
149+
##### Examples
150+
151+
Successful Analysis
152+
153+
Command:
154+
155+
```bash
156+
maestro test flow-file.yaml --analyze
157+
```
158+
159+
Output:
160+
161+
```bash
162+
🔎 Analyzing Flow(s)...
163+
164+
To view the report, open the following link in your browser:
165+
file:///path/to/your/insights-report.html
166+
167+
Analyze support is in Beta. We would appreciate your feedback in our Slack channel: #community-chat
168+
169+
```
170+
171+
<figure><img src="../.gitbook/assets/analyze-report.png" alt=""><figcaption></figcaption></figure>
172+
173+
No Issues Found
174+
175+
If no issues are detected, you will see a message like this:
176+
177+
```bash
178+
Hey, we analyzed your flow for spelling, grammar, and internationalization issues, and good news 🙌 we didn't find any issues!
179+
```
180+
181+
##### Disabling Notifications
182+
183+
To disable the AI analysis notification, set the `MAESTRO_CLI_ANALYSIS_NOTIFICATION_DISABLED` environment variable to true before running Maestro:
184+
185+
```bash
186+
export MAESTRO_CLI_ANALYSIS_NOTIFICATION_DISABLED=true
187+
```
188+
189+
##### Feedback
190+
191+
The Analyze feature is currently in Beta. Share your feedback and suggestions in our Slack channel:[#community-chat](https://mobile-dev-inc.slack.com/archives/C083YB8N42G)

0 commit comments

Comments
 (0)