Notes and FAQ

Notes

For accurate results, you should generate those images via CodeceptJS. For example, by using the built-in method I.saveScreenshot(), or the method I.takeScreenshot() provided by this helper.

The reason is, that every browser renders the image slightly different. Only when you generate both images (expected and actual image) using the same browser/settings, there will be no differences.

We've even seen differences between images taken by the same test setup by only changing the browser from default to headless mode.

FAQ

What's the difference between tolerance and threshold?

• tolerance defines the number of pixels that are allowed to be different between both images. When the relative count of different pixels is below the tolerance, the images are considered equal. Tolerance simply counts every different pixel, regardless of how different that pixel is.
• threshold is used by the pixelmatch library to determine, which pixels are actually different. By raising the threshold, you will get a lower count of different pixels. Threshold inspects the color-difference between two pixels to determine if they are different.

If tests are too sensitive, you can either increase tolerance (to allow a higher number of different pixels), or increase the threshold (to reduce the number of different pixels).