Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lower the time tests take to run #1567

Open
another-rex opened this issue Feb 4, 2025 · 1 comment
Open

Lower the time tests take to run #1567

another-rex opened this issue Feb 4, 2025 · 1 comment

Comments

@another-rex
Copy link
Collaborator

Our go tests are starting to take quite a long time now, we should look into performance improvements or gating tests behind flags so they are not always ran. (though caching should already be doing this)

Here are the tests that are currently taking more than 2 seconds, sorted in ascending order:

TestRun_MavenTransitive/scans_transitive_dependencies_by_specifying_pom.xml,2.04
TestRun_MavenTransitive/scans_transitive_dependencies_for_pom.xml_by_default,2.35
TestRun/config_file_can_be_broad,2.38
TestRun_Licenses/No_vulnerabilities_with_license_summary_in_markdown,2.42
TestRun_MavenTransitive/scans_dependencies_from_multiple_registries,2.52
TestRun/PURL_SBOM_case_sensitivity_(local),2.52
TestRun_MavenTransitive/resolve_transitive_dependencies_with_native_data_source,2.6
TestRun_Licenses/Some_packages_with_ignored_licenses,2.7
Test_runGovulncheck,2.9
TestRun/requirements.txt_can_have_all_kinds_of_names,2.97
TestRun_LocalDatabases/only_the_files_in_the_given_directories_are_checked_by_default_(no_recursion),3.39
TestRun_LocalDatabases/.gitignored_files,3.91
TestRun_LocalDatabases/all_supported_lockfiles_in_the_directory_should_be_checked#01,3.97
TestRun_LocalDatabases/nested_directories_are_checked_when_`--recursive`_is_passed,4.53
TestRun_Fix/fix_non-interactive_json_override_pom.xml,4.54
TestRun_Fix/fix_non-interactive_override_pom.xml,4.67
TestPrintCycloneDX14Results_WithVulnerabilities,4.87
TestPrintCycloneDX15Results_WithLicenseViolations,4.95
TestPrintOutputResults_WithVulnerabilities,4.96
TestFromCSVFile,5.05
TestRun_LocalDatabases/ignoring_.gitignore,5.08
TestRunCallAnalysis/Run_with_govulncheck,5.42
TestPrintTableResults_NoTerminalWidth_WithVulnerabilities,6.05
TestCVE202230633,6.12
TestRun/folder_of_supported_sbom_with_vulns,7.27
TestRun_Fix/fix_non-interactive_relax_package.json,8.87
TestRun_LocalDatabases/all_supported_lockfiles_in_the_directory_should_be_checked,9.46
TestRun_Fix/fix_non-interactive_json_relax_package.json,9.61
TestRun_Fix/fix_non-interactive_json_in-place_package-lock.json,10.18
TestRun_Fix/fix_non-interactive_in-place_package-lock.json,11.06
TestRun_LocalDatabases/one_specific_supported_sbom_with_vulns,22.01
@G-Rath
Copy link
Collaborator

G-Rath commented Feb 5, 2025

(though caching should already be doing this)

Sadly caching doesn't tend to help because our top-most packages by definition depend on everything meaning 1. they're generally the slowest of tests as they're the most e2e-y, and 2. pretty much all changes to the codebase invalid their test cache

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants