Skip to content

Growth in New Dependents Metrics #2528

@ryscheng

Description

@ryscheng

What is it?

Total number of projects adding or updating the tool/package over the last 6 months

Ultimately in order to do this, we need ADD_DEPENDENCY events for repo => packages in the event table.
However at the present, we only have the ability to get the current SBOM for repos.

That leads 2 options:

  1. Low robustness, high performance: dlt connector to get regular snapshots of the SBOM. If we ever experience a failure/outage, we might just be missing data points in a way we'll never be able to recover from.
    https://docs.github.com/en/rest/dependency-graph/sboms?apiVersion=2022-11-28

  2. High robustness, low performance: Use git commit history as an approximate time stamp so that we can use this API to get add/remove dependency events. We might need to crawl entire git commit histories, which will be slow and terrible..
    https://docs.github.com/en/rest/dependency-graph/dependency-review?apiVersion=2022-11-28

But even in this API, you don't get a timestamp of when things are added or removed, just a summary of the diff. Not ideal. Here's a crazy proposal, doable, but not ideal:

For every repository, we get the commit history (hopefully time stamped)
Then for every repo, for every 1 {week/month/year}? period, we call the dependency diff API to get a list of things added/removed in a period of time.
This requires (n*m) GitHub API calls, where n is the number of repos, and m is the number of commits. Idk if we can even make that work given our current quotas, worth a shot but high execution risk.

Metadata

Metadata

Assignees

Labels

Projects

Status

Up Next

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions