Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running multiple importer jobs at the same time #1207

Open
Tracked by #198
ctron opened this issue Jan 28, 2025 · 0 comments
Open
Tracked by #198

Allow running multiple importer jobs at the same time #1207

ctron opened this issue Jan 28, 2025 · 0 comments
Assignees

Comments

@ctron
Copy link
Contributor

ctron commented Jan 28, 2025

As this issue already has links established to relevant sub-issues, let's re-purpose this issue to be the User Story that defines the expected functionality for the first downstream release.

As a TPA user I want to have 4 importers preconfigured; Github Security Advisory data (enabled), Vulnerability (CVE) data (enabled), Red Hat SBOM (disabled) and Red Hat Vex (disabled) and able to execute in parallel so that I can efficiently populate the application with data from external sources.

Acceptance criteria:

GHSA Importers enabled by default to load data in temporal reverse order (latest first) based on GHSA git repository folders structure
CVE Importer enabled by default to load data in temporal reverse order (latest first) based on CVE git repository folders structure
RH SBOM Importer by default disabled
RH VEX Importer by default disabled
Importers run in parallel

The corresponding downstream issue is TC-2218

The original issue description is below, and this still adds value.
A single instance of the importer service should be able to run multiple importer runs at the same time.

Also, multiple instances of the importer service should be able to distribute importer runs between them.

I think it is worth solving those two problems at the same time, since the might be related and might use a common foundation of sharing the load.

@ctron ctron added this to Trustify Jan 28, 2025
@ctron ctron mentioned this issue Dec 12, 2024
5 tasks
@jcrossley3 jcrossley3 moved this to In progress in Trustify Feb 11, 2025
jcrossley3 added a commit to trustification/trustify-helm-charts that referenced this issue Feb 11, 2025
jcrossley3 added a commit to trustification/trustify-helm-charts that referenced this issue Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In progress
Development

No branches or pull requests

2 participants