Feature/never enough tests stress suite #14021
Closed
+4,243
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This contribution adds a comprehensive stress testing suite for pytest that pushes the boundaries of pytest's capabilities and validates its behavior under extreme conditions. Features: - 1,660+ test cases covering edge cases and stress scenarios - Parametrization explosion testing (1,000 tests from single function) - Cross-language integration tests (Python ↔ C++ via subprocess) - Deep fixture chain validation (5+ levels) - Chaos testing with randomization and edge case detection - Performance benchmarking tools - Parallel execution validation (pytest-xdist) Test Categories: - Parametrization stress tests (100, 400, 1000 cases) - String edge cases (null bytes, Unicode, huge strings) - Numeric boundary tests (overflow, underflow, precision) - C++ boundary conditions (buffer sizes, memory allocation) - Async fixture handling - Fixture dependency patterns (deep chains, diamond dependencies) - Collection stress tests Validated Against: - pytest 9.1.0.dev107+g8fb7815f1 - Python 3.12.3 - Successfully executed 1,626 tests in 17.82s with 4 parallel workers Benefits: - Validates pytest handles extreme parametrization efficiently - Tests cross-language subprocess integration patterns - Identifies boundary condition bugs (found and fixed C++ buffer bug) - Provides regression testing for performance at scale - Demonstrates best practices for large test suites This suite found a real boundary condition bug in the C++ components during development, demonstrating the value of chaos testing methodology.
- Add FORK_AND_CONTRIBUTE.md: Complete 11-step guide for forking and submitting PRs - Add PULL_REQUEST_TEMPLATE.md: Standardized PR template with checklist - Add PRE_CONTRIBUTION_CHECKLIST.md: Comprehensive validation checklist - Update README.md: Add Quick Contribution Setup section These files provide clear guidance for contributors on how to: - Fork and clone the pytest repository - Set up the development environment - Create and submit pull requests - Validate their work before submission
for more information, see https://pre-commit.ci
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.