Skip to content

Commit 9aa93a7

Browse files
gbraadgbraad
authored
Split development and testing
1 parent 1100d6e commit 9aa93a7

File tree

4 files changed

+107
-101
lines changed

4 files changed

+107
-101
lines changed

‎content/Developing.md

Lines changed: 0 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -84,103 +84,3 @@ If you need to update mocks use:
8484
```bash
8585
$ make generate_mocks
8686
```
87-
88-
89-
## Running e2e tests
90-
91-
We have automated e2e tests which keep CRC in shape.
92-
93-
### Introduction
94-
95-
End-to-end (e2e) tests borrow code from [Clicumber](http://github.com/crc-org/clicumber) package to provide basic functionality for testing CLI binaries. This facilitates running commands in a persistent shell instance (`bash`, `tcsh`, `zsh`, Command Prompt, or PowerShell), assert its outputs (standard output, standard error, or exit code), check configuration files, and so on. The general functionality of Clicumber is then extended by CRC specific test code to cover the whole functionality of CRC.
96-
97-
### How to run
98-
99-
First, one needs to set the following flags in `Makefile`, under `e2e` target:
100-
101-
- `--pull-secret-file`
102-
absolute path to your OpenShift pull secret.
103-
- `--bundle-location`
104-
if bundle is embedded, this flag should be set to `--bundle-location=embedded` or not passed at all
105-
if bundle is not embedded, then absolute path to the bundle should be passed
106-
- `--crc-binary`
107-
if `crc` binary resides in `$GOPATH/bin`, then this flag does not need to be passed
108-
otherwise absolute path to the `crc` binary should be passed.
109-
110-
To start e2e tests, run:
111-
112-
```bash
113-
$ make e2e
114-
```
115-
116-
#### How to run only a subset of all e2e tests
117-
118-
119-
Implicitly, all e2e tests for your operating system are executed. If you want to run only tests from one feature file, you have to override `GODOG_OPTS` environment variable. For example:
120-
121-
```bash
122-
make e2e GODOG_OPTS="--godog.tags='@basic && @windows'" BUNDLE_LOCATION=<bundle location> PULL_SECRET_FILE=<pull secret path>
123-
```
124-
125-
Please notice `@basic && @windows`, where `@basic` tag stands for `basic.feature` file and `@windows` tag for e2e tests designed for Windows.
126-
127-
### How to test cert rotation
128-
129-
On linux platform first stop the network time sync using:
130-
131-
```bash
132-
$ sudo timedatectl set-ntp off
133-
```
134-
135-
Set the time 2 month ahead:
136-
137-
```bash
138-
$ sudo date -s '2 month'
139-
```
140-
141-
Start the crc with `CRC_DEBUG_ENABLE_STOP_NTP=true` set:
142-
143-
```bash
144-
$ CRC_DEBUG_ENABLE_STOP_NTP=true crc start
145-
```
146-
147-
### Logs
148-
149-
Test logs can be found in `test/e2e/out/test-results`.
150-
151-
152-
## Releasing on GitHub
153-
154-
### Releasing using the github actions workflow
155-
156-
The GitHub Actions workflow [`Publish release on github`](https://github.com/crc-org/crc/actions/workflows/release.yml) creates a draft release and provides a template with all the component versions and the git change log.
157-
158-
To start the workflow go to the [workflow page](https://github.com/crc-org/crc/actions/workflows/release.yml) and click on the `Run Workflow` button, make sure to choose the appropriate tag for the release.
159-
160-
Once the draft release is available, edit it to include the notable changes for the release and press publish to make it public.
161-
162-
163-
### Releasing using the `gh-release.sh` script
164-
165-
In the CRC repository, we have a script [`gh-release.sh`](https://github.com/crc-org/crc/blob/main/gh-release.sh) which uses the [`gh`](https://cli.github.com) tool, make sure it is installed.
166-
167-
Create a markdown file containing a list of the notable changes named `notable_changes.txt` in the same directory as the script.
168-
169-
An example `notable_changes.txt`:
170-
```bash
171-
$ cat notable_changes.txt
172-
- Fixes a bug where `oc` binary was not extracted from bundle when using microshift preset [#3581](https://github.com/crc-org/crc/issues/3581)
173-
- Adds 'microshift' as a possible value to the help string of the 'preset' config option [#3576](https://github.com/crc-org/crc/issues/3576)
174-
```
175-
176-
Then run the script from the release tag and follow the prompts, it’ll ask for confirmation before pushing the draft release to GitHub.
177-
178-
> [!NOTE]
179-
> The script will exit with error if it doesn’t find a tag starting with `v` in the current git HEAD.
180-
181-
```bash
182-
$ git checkout v2.18.0
183-
$ ./gh-release.sh
184-
```
185-
186-
Verify the draft release on the releases page and if everything looks good press publish to make the release public.

‎content/Releasing.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
Releasing on GitHub
2+
===================
3+
4+
5+
## Releasing using the github actions workflow
6+
7+
The GitHub Actions workflow [`Publish release on github`](https://github.com/crc-org/crc/actions/workflows/release.yml) creates a draft release and provides a template with all the component versions and the git change log.
8+
9+
To start the workflow go to the [workflow page](https://github.com/crc-org/crc/actions/workflows/release.yml) and click on the `Run Workflow` button, make sure to choose the appropriate tag for the release.
10+
11+
Once the draft release is available, edit it to include the notable changes for the release and press publish to make it public.
12+
13+
14+
## Releasing using the `gh-release.sh` script
15+
16+
In the CRC repository, we have a script [`gh-release.sh`](https://github.com/crc-org/crc/blob/main/gh-release.sh) which uses the [`gh`](https://cli.github.com) tool, make sure it is installed.
17+
18+
Create a markdown file containing a list of the notable changes named `notable_changes.txt` in the same directory as the script.
19+
20+
An example `notable_changes.txt`:
21+
```bash
22+
$ cat notable_changes.txt
23+
- Fixes a bug where `oc` binary was not extracted from bundle when using microshift preset [#3581](https://github.com/crc-org/crc/issues/3581)
24+
- Adds 'microshift' as a possible value to the help string of the 'preset' config option [#3576](https://github.com/crc-org/crc/issues/3576)
25+
```
26+
27+
Then run the script from the release tag and follow the prompts, it’ll ask for confirmation before pushing the draft release to GitHub.
28+
29+
> [!NOTE]
30+
> The script will exit with error if it doesn’t find a tag starting with `v` in the current git HEAD.
31+
32+
```bash
33+
$ git checkout v2.18.0
34+
$ ./gh-release.sh
35+
```
36+
37+
Verify the draft release on the releases page and if everything looks good press publish to make the release public.

‎content/SUMMARY.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,10 @@ CRC Engineering docs
55
[Disclaimer](README.md)
66

77
## Development guide
8-
- [Developing](Developing.md)
8+
- [Development]()
9+
- [Developing](Developing.md)
10+
- [Testing](Testing.md)
11+
- [Releasing](Releasing.md)
912
- [Debugging](Debugging-guide.md)
1013
- [Installation]()
1114
- [macOS code signing](macOS-code-signing.md)

‎content/Testing.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
Testing
2+
=======
3+
4+
5+
## Running e2e tests
6+
7+
We have automated e2e tests which ensure the quality of our deliverable; CRC, and system bundle.
8+
9+
10+
### Introduction
11+
12+
End-to-end (e2e) tests borrow code from [Clicumber](http://github.com/crc-org/clicumber) package to provide basic functionality for testing CLI binaries. This facilitates running commands in a persistent shell instance (`bash`, `tcsh`, `zsh`, Command Prompt, or PowerShell), assert its outputs (standard output, standard error, or exit code), check configuration files, and so on. The general functionality of Clicumber is then extended by CRC specific test code to cover the whole functionality of CRC.
13+
14+
### How to run
15+
16+
First, one needs to set the following flags in `Makefile`, under `e2e` target:
17+
18+
- `--pull-secret-file`
19+
absolute path to your OpenShift pull secret.
20+
- `--bundle-location`
21+
if bundle is embedded, this flag should be set to `--bundle-location=embedded` or not passed at all
22+
if bundle is not embedded, then absolute path to the bundle should be passed
23+
- `--crc-binary`
24+
if `crc` binary resides in `$GOPATH/bin`, then this flag does not need to be passed
25+
otherwise absolute path to the `crc` binary should be passed.
26+
27+
To start e2e tests, run:
28+
29+
```bash
30+
$ make e2e
31+
```
32+
33+
#### How to run only a subset of all e2e tests
34+
35+
36+
Implicitly, all e2e tests for your operating system are executed. If you want to run only tests from one feature file, you have to override `GODOG_OPTS` environment variable. For example:
37+
38+
```bash
39+
make e2e GODOG_OPTS="--godog.tags='@basic && @windows'" BUNDLE_LOCATION=<bundle location> PULL_SECRET_FILE=<pull secret path>
40+
```
41+
42+
Please notice `@basic && @windows`, where `@basic` tag stands for `basic.feature` file and `@windows` tag for e2e tests designed for Windows.
43+
44+
### How to test cert rotation
45+
46+
On linux platform first stop the network time sync using:
47+
48+
```bash
49+
$ sudo timedatectl set-ntp off
50+
```
51+
52+
Set the time 2 month ahead:
53+
54+
```bash
55+
$ sudo date -s '2 month'
56+
```
57+
58+
Start the crc with `CRC_DEBUG_ENABLE_STOP_NTP=true` set:
59+
60+
```bash
61+
$ CRC_DEBUG_ENABLE_STOP_NTP=true crc start
62+
```
63+
64+
### Logs
65+
66+
Test logs can be found in `test/e2e/out/test-results`.

0 commit comments

Comments
 (0)