You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This article is a comparison of the AVA, Jest, Mocha, and `mocha-parallel-tests` JavaScript test runners. [Permalink](https://github.com/scraggo/comparing-javascript-test-runners/blob/master/README.md)
7
+
This article is a comparison of the AVA, Jest, and Mocha JavaScript test runners. [Permalink](https://github.com/scraggo/comparing-javascript-test-runners/blob/master/README.md)
8
8
9
9
To generate the speed metrics in the article, I created a node application (part of this repo) that runs tests in all the frameworks listed above. [See the documentation here](https://github.com/scraggo/comparing-javascript-test-runners/blob/master/docs/test-runner.md).
10
10
@@ -29,7 +29,7 @@ To generate the speed metrics in the article, I created a node application (part
29
29
-[AVA](#ava)
30
30
-[Jest](#jest)
31
31
-[Mocha](#mocha)
32
-
-[mocha-parallel-tests](#mocha-parallel-tests)
32
+
-[mocha-parallel-tests](#mocha-parallel-tests)
33
33
-[Popularity and Community Comparison](#popularity-and-community-comparison)
34
34
-[Speed Comparison](#speed-comparison)
35
35
-[What do "serial" and "parallel" mean?](#what-do-serial-and-parallel-mean)
@@ -57,7 +57,6 @@ In recent years, JavaScript has become a more robust language thanks to the stea
57
57
- AVA - <https://github.com/avajs/ava>
58
58
- Jest - <https://jestjs.io/>
59
59
- Mocha - <https://mochajs.org/>
60
-
- and an interesting wrapper called mocha-parallel-tests <https://github.com/mocha-parallel/mocha-parallel-tests>
61
60
62
61
### Goals
63
62
@@ -192,10 +191,14 @@ Being the most established of the testing frameworks, Mocha enjoys a solid place
192
191
>
193
192
> The [SuperAgent request library](https://visionmedia.github.io/superagent/) test documentation was generated with [Mocha's "doc" reporter](https://mochajs.org/#doc)
194
193
195
-
### mocha-parallel-tests
194
+
_Update [8.0.0 / 2020-06-10](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#800--2020-06-10): Mocha 8 has built-in support for running tests in parallel!_
⚠️ Important note: As of [Mocha 8x](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#800--2020-06-10), there's built-in support for running tests in parallel. In case you can't currently upgrade to mocha 8x, `mocha-parallel-tests` is a viable choice!
201
+
199
202
`mocha-parallel-tests` is not a testing framework. It's a wrapper over Mocha designed to significantly speed it up. It's new in 2019 and has a small team. I'll go into detail on why I'm including it here (and what "parallel" means) in the "speed" portion of this article. From the readme:
200
203
201
204
> `mocha-parallel-tests` is a test runner for tests written with mocha testing framework which allows you to run them in parallel.
@@ -217,7 +220,6 @@ Now that we know a bit about each framework, lets look at some of their populari
217
220
218
221
Overall, we can see that _all_ the frameworks are rising in popularity. To me, this indicates that more people are writing JavaScript applications and testing them - which is quite exciting. The fact that none of them are on a downward trend makes all of them viable in this category.
219
222
220
-
221
223
|| Weekly Downloads \*| Last Publish | Publishes in 1 Year | Contributors |
@@ -270,18 +272,26 @@ For all of the frameworks with parallel capabilities, only separate test files a
270
272
271
273
To generate the speed metrics in the article, I created a node application that runs tests in all the frameworks listed above. [The documentation for it](https://github.com/scraggo/comparing-javascript-test-runners/blob/master/docs/test-runner.md) explains how I wrote and ran the tests. The aim was to simulate a "true" test run in a significantly sized enterprise codebase. Here are the results (`node` version 12):
272
274
273
-
|| Speed | Type |
274
-
| -------------------- | ----- | -------- |
275
-
| mocha-parallel-tests | 7.3s | parallel |
276
-
| AVA | 9.6s | parallel |
277
-
| Jest | 12.5s | parallel |
278
-
| Mocha | 16.2s | serial |
275
+
|| Speed | Type |
276
+
| ---------------------- | ----- | -------- |
277
+
| mocha 8.x `--parallel`| 5.3s | parallel |
278
+
| mocha-parallel-tests | 7.3s | parallel |
279
+
| AVA | 9.6s | parallel |
280
+
| Jest | 12.5s | parallel |
281
+
| Mocha | 16.2s | serial |
279
282
280
283
A caveat with all benchmarking tests: the hardware environment (the make, model, RAM, processes running, etc) will affect measured results. For this reason, we'll only be considering the speeds relative to each other.
281
284
282
-
🥇`mocha-parallel-tests` is the clear winner in this run (and most runs). 🥈AVA is close behind (and actually ran faster than `mocha-parallel-tests` in a few of the runs.) 🥉Jest is also fast, but seems to have a bit more overhead than the other two.
285
+
🥇Mocha 8.x with `--parallel` enabled is the fastest of the frameworks in this run (and most runs.) (Mocha versions prior to 8.x can't enable this option.)
286
+
287
+
-[Mocha's docs on the new flag](https://mochajs.org/#-parallel-p)
288
+
-[This is a great resource on getting up and running with this new flag.](https://developer.ibm.com/technologies/node-js/articles/parallel-tests-mocha-v8/)
289
+
290
+
🥈`mocha-parallel-tests` and AVA are close behind (AVA actually ran faster than `mocha-parallel-tests` in a few of the runs.)
291
+
292
+
🥉Jest is also fast, but seems to have a bit more overhead than the other two.
283
293
284
-
Mocha lags far behind the parallel runners - which is to be expected because it runs tests in serial. If speed is your most important criteria (and its drawbacks are not an issue), you'll see a 200-1000% increase in test speed using `mocha-parallel-tests` instead (depending on your machine, `node` version, and the tests themselves).
294
+
Mocha in serial mode lags far behind the parallel runners - which is to be expected because it tests must take "real" time to execute, one after the other. If speed is your most important criteria (and its drawbacks are not an issue), you'll see a 200-1000% increase in test speed using `mocha-parallel-tests` instead (depending on your machine, `node` version, and the tests themselves).
285
295
286
296
### Ease of Use Comparison
287
297
@@ -476,13 +486,13 @@ AVA
476
486
477
487
### Full Comparison (with "Nice to Haves")
478
488
479
-
Let's recap our findings and fill in some gaps with our "nice to haves." (MPT = `mocha-parallel-tests`)
489
+
Let's recap our findings and fill in some gaps with our "nice to haves." (M8 = Mocha 8.x with `--parallel` enabled. MPT = `mocha-parallel-tests`)
| Popularity | Jest - 1st place. Mocha - 2nd place. AVA - 3rd place. MPT - 4th place. |
484
494
| Maintenance | Mocha - 1st place. AVA - 2nd place. Jest - 3rd place. MPT - 4th place. |
485
-
| Speed | MPT - 1st place. AVA - 2nd place. Jest - 3rd place. Mocha - 4th place.|
495
+
| Speed |M8/MPT - 1st place. AVA - 2nd place. Jest - 3rd place. Mocha - 4th place. |
486
496
| Amount of necessary configuration/dependencies | Jest - 1st place. AVA - 2nd place. Mocha - 3rd place. MPT - 4th place. |
487
497
| Organization strategies: `describe` and `it` blocks | Mocha and Jest use these, AVA does not (omits all test globals) |
488
498
| Running the tests | Jest - 1st place. Mocha - 2nd place. MPT - 3rd place. AVA - 4th place. |
@@ -499,7 +509,7 @@ Let's recap our findings and fill in some gaps with our "nice to haves." (MPT =
499
509
500
510
As you can see, all the frameworks are incredibly robust for most testing needs. However, if you picked one at random, it might not work for a specific use case. It's not an easy choice, but here's how I'd break it down:
501
511
502
-
- 🏅Mocha is recommended if you want your tests to run in any environment. It's incredibly community-supported and is extend-able with your favorite 3rd-party packages. Using `mocha-parallel-tests` would give you a speed advantage.
512
+
- 🏅Mocha is recommended if you want your tests to run in any environment. It's incredibly community-supported and is extend-able with your favorite 3rd-party packages. Using `mocha-parallel-tests`(or 8.x with `--parallel` enabled) would give you a speed advantage.
503
513
- 🏅Jest is recommended if you want to get tests up and running quickly. It has everything built in and requires very little configuration. The command line and GUI experience is unmatched. Finally, it's the most popular and makes an excellent pair with React.
504
514
- 🏅AVA is recommended if you want a minimalist framework with no globals. AVA is fast, easy to configure, and you get ES-Next transpilation out of the box. You don't want hierarchical `describe` blocks and you want to support a smaller project.
505
515
@@ -526,6 +536,8 @@ More on Mocha:
526
536
527
537
-<https://devdocs.io/mocha/>
528
538
-[The Ultimate Unit Testing Cheat-sheet For Mocha, Chai and Sinon](https://gist.github.com/yoavniran/1e3b0162e1545055429e)
539
+
-[Mocha's docs on the new --parallel flag](https://mochajs.org/#-parallel-p)
540
+
-[This is a great resource on getting up and running with --parallel](https://developer.ibm.com/technologies/node-js/articles/parallel-tests-mocha-v8/)
0 commit comments