deepshallow.txt

Deep and Shallow

Generally speaking, a script can fall into broad categories; Deep and Shallow.

A Deep scripts, not surprisingly are ones that execute their purpose with a deep level of inquisition. It is not enough to check that the message on the screen was correct or that the page flow works as expected. Instead, the script will check the database for correctness, services were updated correctly, etc.

Of course each of these types of Deep checks are separate scripts keeping them in the spirit of 'And' scripts.

A Shallow script by comparison only does a quick skim across the surface of the application to ensure success. They will still have failing criteria in them, but they are far less rigorous than Deep ones. A Shallow script for instance might only check that message displayed the user is as expected.

Both are important, but a mature automation setup will overwhelmingly be Deep ones. This makes sense as one of the reasons for creating automation is to get a deep understanding of its current state of its functional characteristics. Ironically though, Deep tests are not the ones that should be constructed first. Shallow tests are almost always faster to create and will therefore provide information quicker. For this reason alone, Shallow tests should be considered before Deep ones.

Shallow scripts also tend to form the backbone of an application's suite of 'smoke' checks. Once those succeed then the Deep ones are performed.

The notion of Deep and Shallow starts to get muddied when looking at the construction of a Deep script. A Deep script goes deep when investigating its primary purpose but goes shallow on everything leading up to that point.

Consider a Deep script that checks the status of a newly created mortgage application. Logging into the system, entering the information and doing the submission would be done in a Shallow manner with very little checking while the actual status checking would be done Deeply. Not only is this faster, but it helps prevent scripts from becoming 'And' ones.

The risk of losing information in the Shallow areas is mitigated by scripts that Deeply check what this particular script treated Shallowly.

Too often new automation teams do too much of one or the other type of script. Good automation has an intentional mix of the two styles.