Skip to content

Commit

Permalink
Merge branch 'main' into patch-3
Browse files Browse the repository at this point in the history
  • Loading branch information
rick-bt authored Jan 25, 2024
2 parents ce5679e + b95ece4 commit 8b9bd02
Show file tree
Hide file tree
Showing 8 changed files with 95 additions and 351 deletions.
45 changes: 27 additions & 18 deletions docs/basics/acct-team-mgmt/concurrency-limits.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,22 @@
---
id: concurrency-limits
title: Concurrency Limits and Team Accounts
sidebar_label: Concurrency Limits and Team Accounts
sidebar_label: Concurrency Limits and Allocations for Teams
---

import useBaseUrl from '@docusaurus/useBaseUrl';

<p><span className="sauceGreen">Enterprise Plans only</span></p>

If your organization has multiple teams sharing a Sauce Labs account, it's important to monitor your concurrency limits--the number of virtual machines that you can run tests on simultaneously. You'll need to ensure that your organization's concurrency is distributed among your accounts.
If your organization has multiple teams sharing a Sauce Labs account, you can use Team Limits to ensure that your organization's concurrency is distributed among different teams.

If you are an org admin, you can view your organization's concurrency limits:
If you are an org admin, you can view and control your organization's concurrency limits:

1. In Sauce Labs, click **ACCOUNT** and then click **Team Management**.

<img src={useBaseUrl('img/team-mgmt/team-mgmt-nav.png')} alt="Team management navigation" width="300"/>

2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center. You can also view the **PEAK VM CONCURRENCY** and **VM TESTS BY DAY** for the organization.

<img src={useBaseUrl('img/team-mgmt/ccy-limits-org-mgmt.png')} alt="Org management concurrency limits" width="600"/>
2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center.

3. To view the limits for another data center, in the upper-right corner of the window, click the **DATA CENTER** dropdown and select the relevant data center.

Expand All @@ -28,11 +26,16 @@ If you are an org admin, you can view your organization's concurrency limits:

<img src={useBaseUrl('img/team-mgmt/ccy-limits-teams.png')} alt="Team concurrency limits" width="600"/>

For more information about usage, see [Viewing and Exporting Usage Data](/basics/acct-team-mgmt/viewing-exporting-usage-data/).
For more usage information, see [Viewing and Exporting Usage Data](/basics/acct-team-mgmt/viewing-exporting-usage-data/).

5. To allocate concurrency to teams, click on the team name to open the Teams tab.

6. Under the Teams tab, Enter **Team VM Concurrency** and click on Update.

## How Concurrency Limits Work

Through Sauce Labs, organization admins set the total concurrency of their organization's account. When setting the concurrency limit for a team, be mindful that team members share the concurrency limit of the team. If a team exceeds its concurrency limit, any additional tests will be queued (see [Queueing Tests](#queueing-tests) for more information).
## How Concurrency Allocation Works

Through Sauce Labs, organization admins set the total concurrency of their organization's account. When setting the concurrency limit for a team, be mindful that team members share the concurrency limit of the team. If a team exceeds its concurrency limit, any additional tests will be queued. Be informed that queuing will be disabled from March 1st, 2024.

#### Example - Org Limits

Expand All @@ -42,15 +45,7 @@ When Team A is using all 250 concurrency slots, that only leaves 250 of the orga

#### Example - Team Limits

An org admin allocates 100 VMs to Team A, which has five members. Since the members of a team share the team's VM limit, the number of concurrent tests being run by the five team members cannot exceed 100. However, concurrency limits cannot be set at the user level, so User A could be running 20 tests, while user B is running 50 tests, and users C, D, and E are running 10 tests each. Or User A could be running 100 tests while Users B, C, D, and E are not running any tests.

## Queueing Tests

As tests complete, queued tests are allocated to concurrency slots in the order they were queued.

If you run more tests at once than your concurrency allows (e.g., you have a concurrency of 100 but send 200 tests at once), the extra tests will be queued and run as the first tests finish. A test is only allowed to remain in the queue for 10 minutes. If it has not been moved out of the queue before 10 minutes elapse, or if too many tests are already queued, the test is removed from the queue and not run at all.

Queuing is intended to deal with small mistakes in concurrency management. Sauce Labs does not recommend queuing tests intentionally.
An organization admin allocates 100 VMs to Team A, which has five members. Since the members of a team share the team's VM limit, the number of concurrent tests being run by the five team members cannot exceed 100. However, concurrency limits cannot be set at the user level, so User A could be running 20 tests, while user B is running 50 tests, and users C, D, and E are running 10 tests each. Or User A could be running 100 tests while Users B, C, D, and E are not running any tests.

## Checking Concurrency

Expand All @@ -61,3 +56,17 @@ curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \
--request GET 'https://api.us-west-1.saucelabs.com/rest/v1.2/users/<username>/concurrency' \
--header 'Content-Type: application/json' \ | json_pp
```

:::caution

Queuing will be disabled on March 1st, 2024. To better manage your usage, check the [Usage Analytics](https://docs.saucelabs.com/dev/api/usage/) that Sauce Labs provides for you

:::

## Queueing Tests

As tests are complete, queued tests are allocated to concurrency slots in the order they were queued.

If you run more tests at once than your concurrency allows (e.g., you have a concurrency of 100 but send 200 tests at once), the extra tests will be queued and run as the first tests finish. A test is only allowed to remain in the queue for 10 minutes. If it has not been moved out of the queue before 10 minutes elapse, or if too many tests are already queued, the test is removed from the queue and not run at all.

Queuing is intended to deal with small mistakes in concurrency management. Sauce Labs does not recommend queuing tests intentionally.
18 changes: 18 additions & 0 deletions docs/dev/cli/saucectl/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,11 @@ Execute tests according to the environment, framework, and test suite specificat
<td><span className="t-cli">-h</span></td>
<td>Usage information for the <code>run</code> command.</td>
</tr>
<tr>
<td><span className="t-cli"><a href="#--help">--live-logs</a></span></td>
<td></td>
<td>Tail the live log output from a running Sauce Orchestrate container.</td>
</tr>
<tr>
<td><span className="t-cli"><a href="#--no-color">--no-color</a></span></td>
<td></td>
Expand Down Expand Up @@ -366,6 +371,19 @@ Usage information for the `run` command.

</div>

### <span className="cli">--live-logs</span>

<div className="cli-desc">
<p><small>| OPTIONAL | BOOLEAN | <span className="sauceGreen">Sauce Orchestrate Only</span></small></p>

Tail the live log output from a running Sauce Orchestrate container.

```bash
saucectl run --live-logs
```

</div>

### <span className="cli">--no-color</span>

<div className="cli-desc">
Expand Down
2 changes: 1 addition & 1 deletion docs/visual-testing/integrations/storybook.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,4 +176,4 @@ module.exports = {

```
If you wanna configure your own devices please follow the configuration steps inside the [playwright docs](https://playwright.dev/docs/emulation#devices).
If you'd like to configure your own devices, please follow the configuration steps inside the [playwright docs](https://playwright.dev/docs/emulation#devices).
28 changes: 16 additions & 12 deletions docs/web-apps/automated-testing/cypress.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,18 @@ Sauce Labs supports the following test configurations for Cypress:
<th>Supported Browsers</th>
<th>End of Life</th>
</tr>
<tbody>
<tr>
<td rowspan='2'>13.6.3</td>
<td rowspan='2'>20</td>
<td><b>macOS:</b> 11.00, 12, 13</td>
<td rowspan='2'>Chrome, Firefox, Microsoft Edge, Webkit (Experimental)</td>
<td rowspan='2'>January 22, 2025</td>
</tr>
<tr>
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
<tbody>
<tr>
<td rowspan='2'>13.6.0</td>
Expand Down Expand Up @@ -130,18 +142,6 @@ Sauce Labs supports the following test configurations for Cypress:
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
<tbody>
<tr>
<td rowspan='2'>12.3.0</td>
<td rowspan='2'>16</td>
<td><b>macOS:</b> 11.00, 12, 13</td>
<td rowspan='2'>Chrome, Firefox, Microsoft Edge, Webkit (Experimental)</td>
<td rowspan='2'>Jan 15, 2024</td>
</tr>
<tr>
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
</table>

## How to Get Started
Expand Down Expand Up @@ -180,6 +180,10 @@ Cypress does not currently work with Firefox 101 on Windows.
Cypress does not currently work with Firefox 105.
See https://github.com/cypress-io/cypress/issues/23897 for more information.

### Microsoft Edge 120

Cypress does not currently work with Microsoft Edge 120.

### Webkit

- Cypress only supports launching Webkit with a fixed resolution of 1280x720.
Expand Down
24 changes: 12 additions & 12 deletions docs/web-apps/automated-testing/playwright.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,18 @@ Sauce Labs supports the following test configurations for Playwright:
<th>Supported Browsers</th>
<th>End of Life</th>
</tr>
<tbody>
<tr>
<td rowspan='2'>1.41.0</td>
<td rowspan='2'>20</td>
<td><b>macOS:</b> 11.00, 12, 13</td>
<td rowspan='2'>Chromium, Chrome, Firefox, Webkit</td>
<td rowspan='2'>January 22, 2025</td>
</tr>
<tr>
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
<tbody>
<tr>
<td rowspan='2'>1.40.1</td>
Expand Down Expand Up @@ -131,18 +143,6 @@ Sauce Labs supports the following test configurations for Playwright:
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
<tbody>
<tr>
<td rowspan='2'>1.29.2</td>
<td rowspan='2'>16</td>
<td><b>macOS:</b> 11.00, 12, 13</td>
<td rowspan='2'>Chromium, Chrome, Firefox, Webkit</td>
<td rowspan='2'>Jan 15, 2024</td>
</tr>
<tr>
<td><b>Windows:</b> 10, 11</td>
</tr>
</tbody>
</table>

## How to Get Started
Expand Down
Loading

0 comments on commit 8b9bd02

Please sign in to comment.