Go Mutation Testing Experiment: String Utilities

This project demonstrates how mutation testing can drive the evolution of your test suite—from achieving 100% code coverage to ensuring that tests truly validate correct behavior under subtle edge conditions. We implemented several string manipulation functions and initially wrote tests that achieved full coverage. By running mutation testing with GoMutesting, we identified weaknesses in our tests and enhanced them accordingly.

Project Structure

.
├── go.mod
├── main.go
└── pkg
    └── stringsutil
         ├── stringsutil.go
         └── stringsutil_test.go

• go.mod: Module declaration.
• main.go: A simple program to demonstrate the functions.
• pkg/stringsutil/stringsutil.go: Contains the implementation of our string utilities.
• pkg/stringsutil/stringsutil_test.go: Contains unit tests for the string utilities.

String Utility Functions

The stringsutil package provides the following functions:

• ReverseString: Reverses the input string while properly handling Unicode characters.
• IsPalindrome: Checks whether a given string reads the same forwards and backwards.
• TrimWhitespace: Removes leading and trailing whitespace (including spaces, tabs, newlines, and carriage returns) from a string.
• FormatString: Formats a string based on a format specifier and corresponding arguments.

Evolution of the Experiment

Initial Test Suite

We began by writing a comprehensive set of unit tests for our string manipulation functions. These tests achieved 100% code coverage, meaning every line of code was executed during testing. However, high coverage alone did not guarantee that every behavioral nuance was validated.

Running Mutation Testing

Using GoMutesting, we introduced small, systematic changes (mutants) into our code to check if our tests could catch these modifications. The results revealed:

• Equivalent Mutants:
  A change in ReverseString (altering the loop condition from i < j to i <= j) did not affect output because swapping the middle element with itself is harmless.
• Non-Equivalent Mutants:
  Several mutations in the TrimWhitespace function—such as changes in loop conditions and boundary checks—survived. These mutations exposed gaps in our tests for edge cases (e.g., single whitespace, carriage returns, newlines).

Enhancing the Test Suite

Guided by the surviving mutants, we added new edge case tests, including:

• Testing a single whitespace character and a single non-whitespace character.
• Verifying strings with leading or trailing carriage returns, newlines, and tabs.
• Mixed scenarios (e.g., a newline at the beginning of a string).

These enhancements improved our mutation score from around 85% to over 91%, meaning that our tests now caught almost all non-equivalent mutations.

What Mutation Testing Empowered Us To Do

• Identify Weak Spots:
  It revealed that 100% code coverage did not guarantee robust behavioral validation.
• Focus Test Improvements:
  The surviving mutants pinpointed specific edge cases where our tests were lacking, driving targeted enhancements.
• Increase Confidence:
  With improved tests, we are more confident that our code behaves correctly in all scenarios, including subtle boundary conditions.
• Drive Continuous Improvement:
  Mutation testing provided a feedback loop that encouraged us to refine our test suite continuously.

How to Run

Running Unit Tests

To run the tests and check coverage, execute:

go test -v -cover ./...

Running Mutation Testing

First, install GoMutesting if you haven't already:

go install github.com/avito-tech/go-mutesting/cmd/go-mutesting@latest

Then, run mutation testing on the stringsutil package:

go-mutesting ./pkg/stringsutil

The output will show detailed results for each mutant and a final mutation score.

Conclusion

This experiment illustrates that achieving high code coverage is only the first step. Mutation testing empowered us to evolve our test suite by identifying subtle, uncovered edge cases and driving targeted improvements. The result is a more robust and trustworthy codebase.

Enjoy experimenting further and leveraging mutation testing to continuously improve your tests!