You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Automated Visual Regression Testing can help you detect when the layout of a web page has changed. Instead of comparing pixels from screenshots, layout differences can be detected by comparing HTML tags and attributes with a baseline. If a change is detected, it could mean that something broke, the web page was redesigned, or dynamic content changed.
6
6
@@ -10,35 +10,42 @@ Automated Visual Regression Testing can help you detect when the layout of a web
10
10
11
11
To handle automated visual testing, SeleniumBase uses the ``self.check_window()`` method, which can set visual baselines for comparison and then compare the latest versions of web pages to the existing baseline.
12
12
13
-
The first time a test calls ``self.check_window()`` with a unique "name" parameter, the visual baseline is set, which means a folder is created with the following files:
14
-
* page_url.txt -> The URL of the current window
15
-
* screenshot.png -> A screenshot of the current window
16
-
* tags_level1.txt -> HTML tags from the window
17
-
* tags_level2.txt -> HTML tags + attribute names
18
-
* tags_level3.txt -> HTML tags + attribute names+values
13
+
The first time a test calls ``self.check_window()`` with a unique ``name`` parameter, the visual baseline is set, which means a folder is created with the following files:
14
+
15
+
<li><b>page_url.txt</b> -> The URL of the current window</li>
16
+
<li><b>screenshot.png</b> -> A screenshot of the current window</li>
17
+
<li><b>tags_level1.txt</b> -> HTML tags from the window</li>
18
+
<li><b>tags_level2.txt</b> -> HTML tags + attribute names</li>
19
+
<li><b>tags_level3.txt</b> -> HTML tags + attribute names+values</li>
19
20
20
21
After the first time ``self.check_window()`` is called, later calls will compare the HTML tags and attributes of the latest window to the ones from the first call (<i>or to the ones from the call when the baseline was last reset</i>).
21
22
22
23
Here's an example call:
23
-
```
24
+
25
+
```python
24
26
self.check_window(name="first_test)", level=3)
25
27
```
28
+
26
29
On the first run (<i>or if the baseline is being set/reset</i>) the "level" doesn't matter because that's only used for comparing the current layout to the existing baseline.
27
30
28
31
Here's how the level system works:
29
-
* level=0 ->
30
-
DRY RUN ONLY - Will perform a comparison to the baseline, and print out any differences that are found, but won't fail the test even if differences exist.
31
-
* level=1 ->
32
-
HTML tags are compared to tags_level1.txt
33
-
* level=2 ->
34
-
HTML tags and attribute names are compared to tags_level2.txt
35
-
* level=3 ->
36
-
HTML tags and attribute names+values are compared to tags_level3.txt
32
+
33
+
<li><b>level=0</b> ->
34
+
DRY RUN ONLY - Will perform a comparison to the baseline, and print out any differences that are found, but won't fail the test even if differences exist.</li>
35
+
<li><b>level=1</b> ->
36
+
HTML tags are compared to tags_level1.txt</li>
37
+
<li><b>level=2</b> ->
38
+
HTML tags and attribute names are compared to tags_level2.txt</li>
39
+
<li><b>level=3</b> ->
40
+
HTML tags and attribute names+values are compared to tags_level3.txt</li>
37
41
38
42
As shown, Level-3 is the most strict, Level-1 is the least strict. If the comparisons from the latest window to the existing baseline don't match, the current test will fail, except for Level-0 checks, which print Level-3 results without failing the test.
39
43
40
44
You can reset the visual baseline on the command line by adding the following parameter at runtime:
41
-
``--visual_baseline``
45
+
46
+
```bash
47
+
--visual_baseline
48
+
```
42
49
43
50
As long as ``--visual_baseline`` is used on the command line while running tests, the ``self.check_window()`` method cannot fail because it will rebuild the visual baseline rather than comparing the html tags of the latest run to the existing baseline. If there are any expected layout changes to a website that you're testing, you'll need to reset the baseline to prevent unnecessary failures.
44
51
@@ -49,6 +56,7 @@ If you want to use ``self.check_window()`` to compare a web page to a later vers
49
56
Automated Visual Testing with ``self.check_window()`` is not very effective for websites that have dynamic content because that changes the layout and structure of web pages. For those pages, you're much better off using regular SeleniumBase functional testing, unless you can remove the dynamic content before performing the comparison, (such as by using ``self.ad_block()`` to remove dynamic ad content on a web page).
50
57
51
58
Example usage of ``self.check_window()`` with different levels:
59
+
52
60
```python
53
61
self.check_window(name="testing", level=0)
54
62
self.check_window(name="xkcd_home", level=1)
@@ -61,6 +69,7 @@ Example usage of ``self.check_window()`` with different levels:
61
69
```
62
70
63
71
Here's an example where clicking a button makes a hidden element visible:
72
+
64
73
```python
65
74
from seleniumbase import BaseCase
66
75
@@ -76,7 +85,9 @@ class VisualLayoutTest(BaseCase):
76
85
self.click("button")
77
86
self.check_window(name="helloworld", level=3)
78
87
```
88
+
79
89
Here's the output of that: (<i>Text changes do not impact visual comparisons</i>)
90
+
80
91
```
81
92
AssertionError:
82
93
First differing element 39:
@@ -92,6 +103,7 @@ First differing element 39:
92
103
```
93
104
94
105
Here's an example where a button is removed from a web page:
106
+
95
107
```python
96
108
from seleniumbase import BaseCase
97
109
@@ -105,7 +117,9 @@ class VisualLayoutTest(BaseCase):
105
117
self.remove_element('a.donate-button')
106
118
self.check_window(name="python_home", level=3)
107
119
```
120
+
108
121
Here's the output of that:
122
+
109
123
```
110
124
AssertionError:
111
125
First differing element 33:
@@ -123,6 +137,7 @@ First differing element 33:
123
137
```
124
138
125
139
Here's an example where a web site logo is resized:
140
+
126
141
```python
127
142
from seleniumbase import BaseCase
128
143
@@ -137,7 +152,9 @@ class VisualLayoutTest(BaseCase):
To run the example (from [examples/visual_testing/](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/visual_testing/)) with a pytest HTML Report, use:
161
-
```
178
+
179
+
```bash
162
180
pytest test_layout_fail.py --html=report.html
163
181
```
182
+
164
183
Here's what the pytest HTML Report looks like:<br />
Copy file name to clipboardExpand all lines: mkdocs.yml
+11-9Lines changed: 11 additions & 9 deletions
Original file line number
Diff line number
Diff line change
@@ -2,15 +2,15 @@
2
2
site_name: SeleniumBase
3
3
site_url: https://seleniumbase.io
4
4
site_author: Michael Mintz
5
-
site_description: A fast & reliable web automation framework for end-to-end testing with Python, pytest and WebDriver. Includes tools for reporting, visual testing, RPA, and more.
5
+
site_description: A complete framework for end-to-end testing with Python, pytest, and WebDriver.
0 commit comments