Testing
This page documents our current approach to testing the GeoGardenClub app.
+The current goal of testing in GeoGardenClub is to prevent catastrophic regression. In other words, we want our tests to ensure that changes to the code do not result in an app where important features no longer work. This means that our test suite should ensure that:
+-
+
- All commonly accessed screens display without error. (The tests might not check screens that are displayed "rarely", such as those resulting from anomalous conditions like network instability.) +
- CRUD operations on entities can be performed successfully when available. +
- Buttons on all commonly accessed screens, when tapped, do not generate an error, and the resulting screen is checked to see that at least some of the intended results are displayed. +
Run the tests​
To run the test suite, invoke ./run_tests.sh. It should produce output similar to the following:
~/GitHub/geogardenclub/ggc_app git:[issue-235]
./run_tests.sh
+ flutter test integration_test/app_test.dart --coverage
00:15 +0: loading /Users/philipjohnson/GitHub/geogardenclub/ggc_app/integration_test/app_test.dart Ru00:41 +0: loading /Users/philipjohnson/GitHub/geogardenclub/ggc_app/integration_test/app_test.dart
00:48 +0: loading /Users/philipjohnson/GitHub/geogardenclub/ggc_app/integration_test/app_test.dart 6.6s
Xcode build done. 33.2s
00:54 +0: GGC Integration Test (All) Fixture 1 Tests
Testing admin feature
Testing badge feature
Testing chapter feature
Testing chat feature
Testing crop feature
Testing garden feature
Testing gardener feature
Testing geobot feature
Testing home feature
Testing observation feature
Testing outcome feature
Testing planting feature
Testing settings feature
Testing task feature
Testing variety feature
02:00 +1: All tests passed!
+ genhtml -q coverage/lcov.info -o coverage/html
Overall coverage rate:
source files: 472
lines.......: 35.0% (4464 of 12740 lines)
functions...: no data found
Message summary:
no messages were reported
Run the tests<
Always watch the iOS simulator!​
-warning
If testing with the iOS simulator, the testing process will occasionally (and unpredictably) pause waiting for you to click on a button to allow pasting:
For this reason, it's important to always watch the simulator at least until the tests start, because you might need to click a button to let the tests proceed.
This is a security feature in the iOS operating system. I do not know of a way to disable it.
Always monitor the iOS simulator!​
+warning
If testing with the iOS simulator, the testing process will occasionally (and unpredictably) pause waiting for you to click on a button to allow pasting:
For this reason, it's important to always monitor the simulator at least until the tests start, because you might need to click a button to let the tests proceed. Otherwise the test process will hang indefinitely.
This is a security feature in the iOS operating system. There is apparently no way to disable it at the current time.
About app_test.dart​
To further understand the test process, it's helpful to review the code that is run by the ./run_tests.sh command:
// integration_test/app_test.dart
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
group('GGC Integration Test (All)', () {
patrolWidgetTest('Fixture 1 Tests', (PatrolTester $) async {
await Firebase.initializeApp();
setFirebaseUiIsTestMode(true);
FirebaseAuth mockAuth = MockFirebaseAuth();
String email = 'jennacorindeane@gmail.com';
mockAuth.createUserWithEmailAndPassword(email: email, password: '');
TestFixture testFixture = await TestFixture.getInstance(testFixture1Path);
await $.pumpWidgetAndSettle(ProviderScope(
overrides: [
firebaseAuthProvider.overrideWithValue(mockAuth),
badgesProvider.overrideWith((_) => testFixture.getBadgesStream()),
badgeDatabaseProvider.overrideWith((_) => testFixture.getBadgeDatabase()),
badgeInstancesProvider.overrideWith((_) => testFixture.getBadgeInstancesStream()),
badgeInstanceDatabaseProvider.overrideWith((_) => testFixture.getBadgeInstanceDatabase()),
bedsProvider.overrideWith((_) => testFixture.getBedsStream()),
bedDatabaseProvider.overrideWith((ref) => testFixture.getBedDatabase()),
chaptersProvider.overrideWith((_) => testFixture.getChaptersStream()),
chapterDatabaseProvider.overrideWith((_) => testFixture.getChapterDatabase()),
chatRoomDatabaseProvider.overrideWith((_) => testFixture.getChatRoomDatabase()),
chatUserDatabaseProvider.overrideWith((_) => testFixture.getChatUserDatabase()),
cropsProvider.overrideWith((_) => testFixture.getCropsStream()),
cropDatabaseProvider.overrideWith((_) => testFixture.getCropDatabase()),
editorsProvider.overrideWith((_) => testFixture.getEditorsStream()),
editorDatabaseProvider.overrideWith((_) => testFixture.getEditorDatabase()),
familiesProvider.overrideWith((_) => testFixture.getFamiliesStream()),
familyDatabaseProvider.overrideWith((_) => testFixture.getFamilyDatabase()),
gardensProvider.overrideWith((_) => testFixture.getGardensStream()),
gardenDatabaseProvider.overrideWith((_) => testFixture.getGardenDatabase()),
gardenersProvider.overrideWith((_) => testFixture.getGardenersStream()),
gardenerDatabaseProvider.overrideWith((_) => testFixture.getGardenerDatabase()),
observationsProvider.overrideWith((_) => testFixture.getObservationsStream()),
observationDatabaseProvider.overrideWith((_) => testFixture.getObservationDatabase()),
outcomesProvider.overrideWith((_) => testFixture.getOutcomesStream()),
outcomeDatabaseProvider.overrideWith((_) => testFixture.getOutcomeDatabase()),
plantingsProvider.overrideWith((_) => testFixture.getPlantingsStream()),
plantingDatabaseProvider.overrideWith((_) => testFixture.getPlantingDatabase()),
rolesProvider.overrideWith((_) => testFixture.getRolesStream()),
roleDatabaseProvider.overrideWith((_) => testFixture.getRoleDatabase()),
tagsProvider.overrideWith((_) => testFixture.getTagsStream()),
tagDatabaseProvider.overrideWith((_) => testFixture.getTagDatabase()),
tasksProvider.overrideWith((_) => testFixture.getTasksStream()),
taskDatabaseProvider.overrideWith((_) => testFixture.getTaskDatabase()),
usersProvider.overrideWith((_) => testFixture.getUsersStream()),
userDatabaseProvider.overrideWith((_) => testFixture.getUserDatabase()),
varietiesProvider.overrideWith((_) => testFixture.getVarietiesStream()),
varietyDatabaseProvider.overrideWith((_) => testFixture.getVarietyDatabase()),
],
child: const MyApp(),
));
expect($(HomeScreen).visible, equals(true), reason: 'Login fails');
await checkIntegrity($, reason: 'startup');
await testAdmin($);
await checkIntegrity($, reason: 'admin feature');
await testBadge($);
await checkIntegrity($, reason: 'badge feature');
await testChapter($);
await checkIntegrity($, reason: 'chapter feature');
await testChat($);
await checkIntegrity($, reason: 'chat feature');
await testCrop($);
await checkIntegrity($, reason: 'crop feature');
await testGarden($);
await checkIntegrity($, reason: 'garden feature');
await testGardener($);
await checkIntegrity($, reason: 'gardener feature');
await testGeoBot($);
await checkIntegrity($, reason: 'geobot feature');
await testHome($);
await checkIntegrity($, reason: 'home feature');
await testObservation($);
await checkIntegrity($, reason: 'observation feature');
await testOutcome($);
await checkIntegrity($, reason: 'outcome feature');
await testPlanting($);
await checkIntegrity($, reason: 'planting feature');
await testSettings($);
await checkIntegrity($, reason: 'settings feature');
await testTask($);
await checkIntegrity($, reason: 'task feature');
await testVariety($);
await checkIntegrity($, reason: 'variety feature');
});
});
}
TestFi
get<Entity>Stream() - returns a Stream of the List of the entities from the test fixture.get<Entity>Database() - returns The Fixture<Entity>Database from the test fixture.
get<Entity>Stream() - returns a Stream of the List of the entities from the test fixture.get<Entity>Database() - returns The Fixture<Entity>Database from the test fixture.