Skip to content

Commit 449f6f2

Browse files
bjuricbjuric
committed
Results annotation
1 parent 85b286e commit 449f6f2

File tree

2 files changed

+113
-20
lines changed

2 files changed

+113
-20
lines changed

‎docs/05-annotations.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ Use annotations in [meta](/docs/meta) and avoid them in features where possible.
3535
| [`@Parallel`](/docs/parallel#parallel-scenario-outline-examples) | Examples | [v3.69.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.69.0) | Executes expanded examples in outlines in parallel |
3636
| [`@Timeout`](/docs/locator-level-timeouts#timeout-annotations) | Step | [v3.73.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.73.0) | Specifies a timeout period on a [wait](/docs/dsl#sleeps-and-waits), [until/while](/docs/dsl#untilwhile), [for each](/docs/dsl#foreach), [assertion](/docs/dsl#assertions) or [locator](/docs/dsl#element-locators) DSL step. Examples: `@Timeout('10s')`, `@Timeout('2m30s')`, `@Timeout('2m')` |
3737
| [`@Delay`](/docs/locator-level-timeouts#delay-annotation) | Step | [v3.73.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.73.0) | Specifies a delay interval on a [wait](/docs/dsl#sleeps-and-waits), [until/while](/docs/dsl#untilwhile) DSL step. Examples: `@Delay('2s')`, `@Delay('1s500ms')`, `@Delay('1m')` |
38+
| [`@Results`](/docs/reports/csv#results-annotation) | Feature<br/>Scenario<br/>StepDef<br/>Scenario Outline<br/>Examples | [v3.77.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.77.0) | Generates CSV results for a gherkin node |
3839
| `@Trim` | Step | [v3.62.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.62.0) | Trims strings when performing comparison operations |
3940
| `@IgnoreCase` | Step | [v3.62.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v3.62.0) | Ignores case when performing comparison operations |
4041
| `@Ignore` | Feature<br/>Scenario<br/>Rule<br/>Background<br/>Examples | [v1.0.0](https://github.com/gwen-interpreter/gwen-web/releases/tag/v1.0.0) | Ignores and skips over a Gherkin block/node to avoid evaluation |

‎docs/12-reports/03-csv.mdx

Lines changed: 112 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: CSV Reports
33
toc_min_heading_level: 2
4-
toc_max_heading_level: 2
4+
toc_max_heading_level: 3
55
---
66

77
import Link from '@docusaurus/Link';
@@ -15,7 +15,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
1515
1616
Gwen can also generate CSV reports for you.
1717

18-
## Configure CSV files
18+
## Configuration
1919

2020
By default, Gwen will generate the following CSV files under `reports/results` subdirectory in your configured [`output`](/docs/settings#gwen-outDir) directory, but you can completely reconfigure this to suit your own requirements:
2121
- Feature level results:
@@ -27,11 +27,12 @@ By default, Gwen will generate the following CSV files under `reports/results` s
2727
- scenario-results-FAILED.csv - A log of all failed scenario results including the failed reason
2828
- scenario-results-ALL.csv - A log of all scenario results (passed or failed) including the failed reason (for failed)
2929

30-
### Configuration
30+
### Results files
31+
32+
CSV files can be configured within the `gwen.reports.results.files` element in your [settings file](/docs/settings#settings-files).
3133

32-
Files can be configured within the `gwen.reports.results.files` element in your settings file.
3334
```json
34-
<name> {
35+
<fileId> {
3536
file = "<filename>"
3637
scope = "[scope]"
3738
status = "[status]"
@@ -47,7 +48,7 @@ Files can be configured within the `gwen.reports.results.files` element in your
4748
Where:
4849

4950
<ul>
50-
<li><code>&lt;name&gt;</code> - (Mandatory) Is an arbitatry name for the file element</li>
51+
<li><code>&lt;fileId&gt;</code> - (Mandatory) Is an arbitatry Id for the file element</li>
5152
<li>
5253
<code>&lt;filename&gt;</code> - (Mandatory) - Is the name or path to the target CSV file
5354
<ul>
@@ -64,7 +65,7 @@ Where:
6465
<li><code>Scenario: &lt;name&gt;</code> - To log the results of a specifically named scenario only to the CSV file as it completes, where <code>&lt;name&gt;</code> is the name of the scenario to log the results for</li>
6566
<li><code>StepDef</code> - To log the results of all <Link to="/docs/meta#stepdefs">StepDefs</Link> to the CSV file as they complete</li>
6667
<li><code>StepDef: &lt;name&gt;</code> - To log the results of a specifically named StepDef only to the CSV file as it completes, where <code>&lt;name&gt;</code> is the name of the StepDef to log the results for</li>
67-
<li>Or blank to not log any results automatically (for when you want explicitly manage CSV logging yourself)</li>
68+
<li>Or blank to not log any results automatically (for when you want explicitly manage CSV logging yourself through a [`@Results`](#results-annotation) annotation</li>
6869
</ul>
6970
</li>
7071
<li>
@@ -78,7 +79,9 @@ Where:
7879
<li><code>fields</code> - (Mandatory) - Contains an array of fields as described below</li>
7980
</ul>
8081

81-
Fields are configured as arrays witihin the `fields` attribute and have the following structure:
82+
### Results fields
83+
84+
CSV fields are configured as arrays witihin the `fields` attribute and have the following structure:
8285

8386
```json
8487
{ name = "<fieldName>", ref = [reference], optional = [optionality] }
@@ -616,6 +619,102 @@ In this configuration, fields are replicated across file definitions (note: fiel
616619

617620
</Tabs>
618621

622+
## Results Annotation
623+
624+
Used to log results to a file at a specific Feature, Scenario, StepDef, Scenaro Outline, or Examples node level.
625+
626+
### Syntax:
627+
628+
```gherkin
629+
@Results('<fileId>')
630+
```
631+
632+
Where:
633+
634+
<ul>
635+
<li>
636+
<code>&lt;fileId&gt;</code> - Is a configured file Id under the `gwen.reports.results.files` settings element.
637+
</li>
638+
</ul>
639+
640+
### Example:
641+
642+
The configuration for `my.csv` might be defiend as follows in your [settings file](/docs/settings#settings-files). Note that the `scope` attribute is deliberately absent here, since we will using the `Results` annotation to specify where the scope will be.
643+
644+
<i>gwen.conf</i>
645+
```json
646+
gwen {
647+
..
648+
reports {
649+
results {
650+
files {
651+
my {
652+
csv {
653+
file = "file.csv"
654+
fields = [
655+
{ field = "status", ref = "gwen.scenario.eval.status.keyword.upperCased" }
656+
{ field = "name", ref = "my name" }
657+
{ field = "job", ref = "my job" }
658+
{ field = "duration", ref = "gwen.scenario.eval.duration" }
659+
]
660+
}
661+
}
662+
}
663+
}
664+
}
665+
..
666+
}
667+
```
668+
669+
Specifying the `@Results('my.csv')` annoation on a scenario, would then log to results to the configured file when the scenario completes.
670+
The scope of the `my.csv` file would then be bound to this Scenario.
671+
672+
<i>my.feature</i>
673+
```gherkin {3}
674+
Feature: My feature
675+
676+
@Results('my.csv')
677+
Scenario: My scenario
678+
Given my name is "Gwen"
679+
And my job is "Automation"
680+
..
681+
```
682+
683+
And the generated results file would then contain:
684+
685+
<i>file.csv</i>
686+
```csv
687+
status,name,job,duration
688+
PASSED,Gwen,Automation,1ms
689+
```
690+
691+
When used on a Scenario Outline, each Examples scenario would be logged.
692+
693+
<i>my.feature</i>
694+
```gherkin {3}
695+
Feature: My feature
696+
697+
@Results('my.csv')
698+
Scenario Outline: My scenario
699+
Given my name is <name>
700+
And my job is <job>
701+
Examples:
702+
| name | job |
703+
| Gwen | Automation |
704+
| Stacey | Agent |
705+
..
706+
```
707+
708+
The generated results file would then contain:
709+
710+
<i>file.csv</i>
711+
```csv
712+
status,name,job,duration
713+
PASSED,Gwen,Automation,1ms
714+
PASSED,Gwen,Automation,2ms
715+
..
716+
```
717+
619718
## Logging Results
620719

621720
With the above settings in place, CSV files containing evaluated results will automatiacally be logged when you launch Gwen to execute with the `-f results` [CLI](/docs/cli) option.
@@ -712,13 +811,6 @@ pnpm -b f html,results gwen/features/todo
712811
</TabItem>
713812

714813
</Tabs>
715-
716-
### Output
717-
718-
- The CSV results logged to:
719-
- `output/reports/results/`
720-
- The HTML report will be generated at
721-
- `output/reports/index.html`.
722814

723815
</TabItem>
724816

@@ -782,13 +874,13 @@ gwen -b f html,results features\todo
782874

783875
</Tabs>
784876

785-
### Output
877+
</TabItem>
878+
879+
</Tabs>
880+
881+
<b>Output</b>
786882

787883
- The CSV results logged to:
788884
- `output/reports/results/`
789885
- The HTML report will be generated at
790886
- `output/reports/index.html`.
791-
792-
</TabItem>
793-
794-
</Tabs>

0 commit comments

Comments
 (0)