Skip to content

Commit

Permalink
fix(offline): report all ecosystems without local databases in one si…
Browse files Browse the repository at this point in the history 
…ngle line
  • Loading branch information
@G-Rath
G-Rath committed Sep 25, 2024
1 parent 331e9b6 commit edb00fb
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 52 deletions.
52 changes: 2 additions & 50 deletions cmd/osv-scanner/__snapshots__/main_test.snap
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2108,31 +2108,7 @@ Scanned <rootdir>/fixtures/locks-many/yarn.lock file and found 1 package
---

[TestRun_LocalDatabases_AlwaysOffline/#00 - 2]
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for RubyGems ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Packagist ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for npm ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for npm ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not find local databases for ecosystems: Alpine, Packagist, PyPI, RubyGems, npm

---

Expand All @@ -2153,31 +2129,7 @@ Scanned <rootdir>/fixtures/locks-many/yarn.lock file and found 1 package
---

[TestRun_LocalDatabases_AlwaysOffline/#00 - 4]
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for PyPI ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for RubyGems ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Alpine ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for Packagist ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for npm ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not load db for npm ecosystem: unable to fetch OSV database: no offline version of the OSV database is available
could not find local databases for ecosystems: Alpine, Packagist, PyPI, RubyGems, npm

---

Expand Down
21 changes: 19 additions & 2 deletions internal/local/check.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import (
"fmt"
"os"
"path"
"slices"
"strings"

"github.com/google/osv-scanner/pkg/lockfile"
"github.com/google/osv-scanner/pkg/models"
Expand Down Expand Up @@ -116,6 +118,9 @@ func MakeRequest(r reporter.Reporter, query osv.BatchedQuery, offline bool, loca
return db, nil
}

// slice to track ecosystems that did not have an offline database available
var missingDbs []string

for _, query := range query.Queries {
pkg, err := toPackageDetails(query)

Expand Down Expand Up @@ -143,8 +148,13 @@ func MakeRequest(r reporter.Reporter, query osv.BatchedQuery, offline bool, loca
db, err := loadDBFromCache(pkg.Ecosystem)

if err != nil {
// currently, this will actually only error if the PURL cannot be parses
r.Errorf("could not load db for %s ecosystem: %v\n", pkg.Ecosystem, err)
if errors.Is(err, ErrOfflineDatabaseNotFound) {
missingDbs = append(missingDbs, string(pkg.Ecosystem))
} else {
// the most likely error at this point is that the PURL could not be parsed
r.Errorf("could not load db for %s ecosystem: %v\n", pkg.Ecosystem, err)
}

results = append(results, osv.Response{Vulns: []models.Vulnerability{}})

continue
Expand All @@ -153,5 +163,12 @@ func MakeRequest(r reporter.Reporter, query osv.BatchedQuery, offline bool, loca
results = append(results, osv.Response{Vulns: db.VulnerabilitiesAffectingPackage(pkg)})
}

if len(missingDbs) > 0 {
missingDbs = slices.Compact(missingDbs)
slices.Sort(missingDbs)

r.Errorf("could not find local databases for ecosystems: %s\n", strings.Join(missingDbs, ", "))
}

return &osv.HydratedBatchedResponse{Results: results}, nil
}

0 comments on commit edb00fb

Please sign in to comment.