|
| 1 | +## Data Explorer End-To-End Tests |
| 2 | + |
| 3 | +### Test Information |
| 4 | + |
| 5 | +All tests are stored in `/explorer/e2e`. Playwright will run tests in any file with the suffix `*.spec.ts`, so long as these files are stored in `/explorer/e2e`. |
| 6 | +Tests for a specific configuration are stored as `config_name/config_name-test_name.spec.ts`. While these are the tests |
| 7 | +that Playwright actually runs, most test code is stored in `e2e/testFunctions.ts`. These functions contain the |
| 8 | +code that runs for all tests, except for tests that are not reused between configurations and that only run in one tab. This allows tests to be repeated |
| 9 | +for different tabs in different configurations without reusing code, and means that information that tests |
| 10 | +depend on can be stored in one place to make adjusting tests to changes in user facing content is straightforward. |
| 11 | +Config specific constants used for tests are kept in the `config_name/config_name-tabs.ts`, and use interfaces and |
| 12 | +custom types in `testInterfaces.ts`. |
| 13 | + |
| 14 | +### Running tests |
| 15 | + |
| 16 | +#### Running Locally |
| 17 | + |
| 18 | +To run all tests locally, run `npm run test:anvil-cmg` or `npm run test:anvil-catalog`. If there is no server running |
| 19 | +on `localhost:3000`, this will create a dev build of the correct configuration and run the tests on Chromium, Firefox, and Webkit (Safari). The tests may be flaky |
| 20 | +if run on the `dev:config` version instead of a dev build, as the site will run too slow and they may time out. Traces |
| 21 | +and screenshots from any tests that fail will be output to `explorer/playwright-report`. To manually run an individual test file, |
| 22 | +run `npx playwright test e2e/path-to/filename.spec.ts -c <config_file>`. The available config files are `playwright_anvil.config.ts` |
| 23 | +and `playwright_anvil-catalog.config.ts`. To run an individual test, add the argument `-g <test_name>`. More information |
| 24 | +about command line options for the test can be found in [Playwright's Documentation](https://playwright.dev/docs/test-cli). |
| 25 | +To debug or write tests, it can be useful to use Playwright's UI mode with `npx playwright test --ui`, which allows you |
| 26 | +to easily run individual tests, and view the actions Playwright takes step by step in and their result. |
| 27 | + |
| 28 | +#### Running in GitHub |
| 29 | + |
| 30 | +All tests are run automatically when a pull request is made, using GitHub Actions. These actions create a development build |
| 31 | +for each configuration, then run the test. When a test passes, it will become visible on GitHub. If one fails, screenshots |
| 32 | +and traces can be downloaded from GitHub. To view step by step what happened in the test, visit `trace.playwright.dev` |
| 33 | +in a web browser and upload the `trace.zip` file. This web app, which runs entirely in browser, allows you to step |
| 34 | +through the actions taken as part of the test and view the impact on the web page. |
| 35 | + |
| 36 | +### Current tests |
| 37 | + |
| 38 | +#### AnVIL-CMG |
| 39 | + |
| 40 | +- Filters (`anvil-filters.spec.ts`) |
| 41 | + - Check that all filters specified in `e2e/anvil/anvil-tabs.ts` are present, and that clicking them opens a menu with checkboxes |
| 42 | + - This filter runs on all tabs in `anviltabs.ts` |
| 43 | + - Check that checking up to the first five entries in the first filter on the datasets tab works and that it does not remove all elements from the list of tabs |
| 44 | + - Check that selecting a filter causes the selected checkbox entries to remain selected across all tabs |
| 45 | + - Currently uses the 4th filter and starts on the "files" tab |
| 46 | + - Check that the counts next to the filter checkbox match the number of entries once the filter is selected |
| 47 | + - Checks an arbitrary list of 6 filters on the datasets and activities tab |
| 48 | + - Check that the blue filter buttons appear with the correct text when the filter is selected, and that they cause the filter to become deselected when clicked |
| 49 | + - Checks an arbitrary list of 3 filters on the "files" and "biosamples" tabs |
| 50 | + - Check that the clear all button deselects all filters, after an arbitrary list is selected |
| 51 | + - Uses an arbitrary list of 3 filters and runs on the "files tab" |
| 52 | +- Pagination (`anvil-pagination.spec.ts` and `anvil-pagination-content.spec.ts`) |
| 53 | + - Check that, on the first page, the back button is disabled and the forward button is enabled |
| 54 | + - Uses the "Donors" tab only |
| 55 | + - Check that paginating forward on the donors tab keeps the currently displayed page number correct, and that on the last page the back button is enabled and the front page is enabled |
| 56 | + - Uses the "Donors" tab only |
| 57 | + - NOTE: this test may be problematic because it assumes there is relatively short number of pages on the "Donors" tab. This could potentially be resolved by adding filters or by limiting the number of tests run |
| 58 | + - Check that paginating forwards by up to 5 pages changes the content on the first row of the table, and that paginating backwards causes that text to remain the same |
| 59 | + - Uses the "BioSamples" tab only |
| 60 | +- Sort (`anvil-sort.spec.ts`) |
| 61 | + - Check that clicking each table header (the sort button) does not cause the first and last rows of the table to become empty |
| 62 | + - Runs on all headers (so all sort buttons) on all tabs |
| 63 | + - Does not check that any actual sorting occurs, only that the sort buttons exist and do not break the site |
| 64 | + - This limitation is difficult to avoid without making the test dependent on the site data |
| 65 | +- Navigation |
| 66 | + - Check that tabs appear as selected properly when tabs are accessed |
| 67 | + - Runs on all tabs |
| 68 | + - `anvil-urls.spec.ts` |
| 69 | + - Check that selecting a different tab changes the current url and that each tab has the correct data headers |
| 70 | + - Runs from "Datasets" to "Activities", "Activities" to "Datasets", "Datasets" to "Files", "Datasets" to "Donors", "Datasets" to "Biosamples" |
| 71 | + - `anvil-tabs-buttons.spec.ts` |
| 72 | + - All tests rely on correct lists of tabs, columns, and filters in `testFunctions.ts` |
| 73 | + |
| 74 | +#### AnVIL-Catalog |
| 75 | + |
| 76 | +- Check that tabs appear as selected properly when tabs are accessed (`anvilcatalog-url.spec.ts`) |
| 77 | + - Runs on all tabs |
| 78 | +- Check that selecting a different tab changes the current url and that each tab has the correct data headers |
| 79 | + - Runs from "Studies" to "Consortia", from "Workspaces" to "Studies", and from "Consortia" to "Workspaces" |
| 80 | + - `anvilcatalog-tabs-buttons.spec.ts` |
| 81 | +- Sort (`anvil-sort.spec.ts`) |
| 82 | + - Check that clicking each table header (the sort button) does not cause the first and last rows of the table to become empty |
| 83 | + - Runs on all headers (so all sort buttons) on all tabs |
| 84 | +- Edit Columns Button (`anvilcatalog-select-tabs.spec.ts`) |
| 85 | + - Check that the checkboxes in the "Edit Columns" button are activated/deselected and deactivated/selected where proper |
| 86 | + - Runs on all tabs |
| 87 | + - Check that selecting all checkboxes in the Edit Columns menu adds the correct headets to the table |
| 88 | + - Only runs on the "Consortia" tab (other tabs do not have editable columns) |
| 89 | + |
| 90 | +### Candidate Additional Tests (anvil-cmg): |
| 91 | + |
| 92 | +- File download |
| 93 | +- File copy button |
| 94 | +- Top Counts (# activities, files, biosamples) |
| 95 | +- Sign in (probably smoke test / existence only for ease) |
| 96 | +- "Contact Us" form |
| 97 | +- Edit Columns Button (already on AnVIL-Catalog) |
| 98 | +- Datasets backpages |
| 99 | + - Check that "Dataset details" match relevant fields in the table |
| 100 | + - Check access control works properly |
| 101 | + - Smoke test for "Request link" button |
| 102 | + - Check boxes |
| 103 | + - Press "Request link" button |
| 104 | + - Press resulting button to go to Terra |
| 105 | + - Check that url begins "anvil.terra.bio" |
| 106 | +- Smoke test buttons at bottom of screen (logos, help, privacy) |
| 107 | +- Smoke test "Help & Documentation" buttons |
0 commit comments