Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Syntactic indexing: create job scheduling worker #59795

Closed
keynmol opened this issue Jan 24, 2024 · 0 comments
Closed

Syntactic indexing: create job scheduling worker #59795

keynmol opened this issue Jan 24, 2024 · 0 comments
Assignees
Labels
graph/backend Related to Go code in the backend team/graph Graph Team (previously Code Intel/Language Tools/Language Platform)

Comments

@keynmol
Copy link
Contributor

keynmol commented Jan 24, 2024

Depends on #59794

As part of this we should make some of the existing codeintel scheduling workers more generic: https://github.com/sourcegraph/sourcegraph/blob/main/cmd/worker/shared/main.go#L106-L118

The main purpose of this worker is to periodically schedule the actual indexing jobs for repositories and revisions that are not up-to-date (where "up-to-date" determination logic can be reused from the auto-indexing code).

As part of this issue, we should investigate how feasible it is to reuse the existing indexing policies, with the addition of Syntactic option - this way we don't duplicate configuration, and improvements to scheduling/repo matching mechanism will benefit both precise and syntactic.

A working end-to-end POC for this can be found in this branch: https://github.com/sourcegraph/sourcegraph/compare/main...syntactic-indexing-e2e#diff-1f01a15243f70514364308363d577121f701bfb20ca222a40f64d4e4735074de

@varungandhi-src varungandhi-src added team/graph Graph Team (previously Code Intel/Language Tools/Language Platform) graph/backend Related to Go code in the backend labels Jan 25, 2024
@keynmol keynmol self-assigned this Feb 1, 2024
@keynmol keynmol changed the title Batch indexing: create job scheduling worker Syntactic indexing: create job scheduling worker Feb 7, 2024
@mmanela mmanela added the Migrated label May 6, 2024 — with Linear
@eseliger eseliger removed the Migrated label May 21, 2024
@linear linear bot closed this as completed May 31, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
graph/backend Related to Go code in the backend team/graph Graph Team (previously Code Intel/Language Tools/Language Platform)
Projects
None yet
Development

No branches or pull requests

4 participants