You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(ai): Adds --analyze flag docs for TestCommand and update Ai Configurations (#110)
* docs(analyze): Adds new --analyze flag docs and assets
* docs(analyze): Removes unnecessary note
* Update cli/test-suites-and-reports.md
* docs(ai): Adjuts ai configuration to robin cloud
Support for more models and providers is tracked [in this issue](https://github.com/mobile-dev-inc/maestro/issues/1957).
31
20
32
21
### API key
33
22
34
-
To use this command, an API key for the LLM service is required. To set it,
35
-
export the `MAESTRO_CLI_AI_KEY` env var.
23
+
{% hint style="success" %}
24
+
To use these features, you must have an account on [Robin](https://www.robintest.com), the official Maestro cloud platform. Make sure to [create an account](https://signin.robintest.com/sign-up) and [**retrieve your API key**](MAESTRO_CLOUD_API_KEY) to set it up correctly by following this link:
To use this command, an API key for the LLM is required. With the latest changes, we have migrated hard-coded API calls and prompts to our cloud backend, allowing us to enhance these features dynamically without requiring new CLI releases. To set it,
Copy file name to clipboardExpand all lines: cli/test-suites-and-reports.md
+75
Original file line number
Diff line number
Diff line change
@@ -112,3 +112,78 @@ Note that your Flows should **not** depend on device state and should be treated
112
112
#### Configuring part of the Flows to run sequentially
113
113
114
114
For instance, if you have three Flows, `flowA`, `flowB`, and `flowC`, but you want to run only `flowA` and `flowB` sequentially, don't add `flowC` and `flowD` to the list. Maestro will run these Flows in non-deterministic ordering **after** the Flow sequence has finished executing.
115
+
116
+
#### Analyze
117
+
118
+
{% hint style="warning" %}
119
+
120
+
This is an **experimental** feature powered by LLM technology. All feedback is
121
+
welcome.
122
+
123
+
{% endhint %}
124
+
125
+
Maestro introduces a new feature that leverages AI to analyze your end-to-end (E2E) mobile tests and provide actionable insights based on your test logs, commands, and screenshots captured during your test runs. The AI-powered analysis identifies potential issues in your app's functionality, UI, and internationalization, helping you improve app quality efficiently.
126
+
127
+
**Login Requirement:** Before you use the AI analysis feature, ensure you are logged into Maestro. Run the following command:
128
+
129
+
```bash
130
+
maestro login
131
+
```
132
+
133
+
**Analyzing your tests**: To analyze your test flows with AI, use the --analyze flag with the maestro test command:
134
+
135
+
```bash
136
+
maestro test flow-file.yaml --analyze
137
+
```
138
+
139
+
{% hint style="info" %}
140
+
141
+
While we aim for precision, please note that this is a beta release, and the results should be validated before making critical decision
142
+
143
+
{% endhint %}
144
+
145
+
This will enable AI analysis and provide a detailed report based on your test artifacts:
146
+
147
+
##### Examples
148
+
149
+
Successful Analysis
150
+
151
+
Command:
152
+
153
+
```bash
154
+
maestro test flow-file.yaml --analyze
155
+
```
156
+
157
+
Output:
158
+
159
+
```bash
160
+
🔎 Analyzing Flow(s)...
161
+
162
+
To view the report, open the following link in your browser:
163
+
file:///path/to/your/insights-report.html
164
+
165
+
Analyze support is in Beta. We would appreciate your feedback in our Slack channel: #community-chat
The Analyze feature is currently in Beta. Share your feedback and suggestions in our Slack channel:[#community-chat](https://mobile-dev-inc.slack.com/archives/C083YB8N42G)
0 commit comments