Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf- add correct indexes for api/v2/purl?q= #1292

Merged
merged 1 commit into from
Feb 12, 2025

Conversation

JimFuller-RedHat
Copy link
Collaborator

@JimFuller-RedHat JimFuller-RedHat commented Feb 12, 2025

This PR ensures that any searches of api/v2/purl?q= takes advantage of newly defined GIN indexes which is confirmed by running explain on test query

explain analyze SELECT 
    qp.id, 
    qp.versioned_purl_id, 
    qp.qualifiers, 
    qp.purl 
FROM 
    qualified_purl qp
WHERE 
    (qp.qualifiers ->> 'arch' ILIKE 'test1') OR 
    (qp.qualifiers ->> 'distro' ILIKE 'test2') OR
    (qp.qualifiers ->> 'repository_url' ILIKE 'test3') OR 
    (qp.purl ->> 'ty' ILIKE 'test4') OR 
    (qp.purl ->> 'namespace' ILIKE 'test5') OR 
    (qp.purl ->> 'name' ILIKE 'test6') OR 
    (qp.purl ->> 'version' ILIKE 'test7');

which results in query plan using indexes.

 Bitmap Heap Scan on qualified_purl qp  (cost=123.77..248.84 rows=110 width=200) (actual time=49.323..49.326 rows=0 lo
ops=1)
   Recheck Cond: (((qualifiers ->> 'arch'::text) ~~* 'test1'::text) OR ((qualifiers ->> 'distro'::text) ~~* 'test2'::t
ext) OR ((qualifiers ->> 'repository_url'::text) ~~* 'test3'::text) OR ((purl ->> 'ty'::text) ~~* 'test4'::text) OR ((purl ->> 'namespace'::text) ~~* 'test5'::text) OR ((purl ->> 'name'::text) ~~* 'test6'::text) OR ((purl ->> 'version'::text) ~~* 'test7'::text))
   ->  BitmapOr  (cost=123.77..123.77 rows=110 width=0) (actual time=49.318..49.321 rows=0 loops=1)
         ->  Bitmap Index Scan on qualifiedpurlqualifierarchjsonginidx  (cost=0.00..53.80 rows=1 width=0) (actual time=0.080..0.080 rows=0 loops=1)
               Index Cond: ((qualifiers ->> 'arch'::text) ~~* 'test1'::text)
         ->  Bitmap Index Scan on qualifiedpurlqualifierdistrojsonginidx  (cost=0.00..28.83 rows=1 width=0) (actual time=0.065..0.065 rows=0 loops=1)
               Index Cond: ((qualifiers ->> 'distro'::text) ~~* 'test2'::text)
         ->  Bitmap Index Scan on qualifiedpurlqualifierrepositoryurljsonginidx  (cost=0.00..34.60 rows=1 width=0) (actual time=0.131..0.131 rows=0 loops=1)
               Index Cond: ((qualifiers ->> 'repository_url'::text) ~~* 'test3'::text)
         ->  Bitmap Index Scan on qualifiedpurltypejsongistidx  (cost=0.00..1.39 rows=1 width=0) (actual time=0.006..0.006 rows=0 loops=1)
               Index Cond: ((purl ->> 'ty'::text) ~~* 'test4'::text)
         ->  Bitmap Index Scan on qualifiedpurlnamespacejsongistidx  (cost=0.00..1.53 rows=19 width=0) (actual time=18.622..18.623 rows=0 loops=1)
               Index Cond: ((purl ->> 'namespace'::text) ~~* 'test5'::text)
         ->  Bitmap Index Scan on qualifiedpurlnamejsongistidx  (cost=0.00..1.74 rows=47 width=0) (actual time=11.122..11.122 rows=0 loops=1)
               Index Cond: ((purl ->> 'name'::text) ~~* 'test6'::text)
         ->  Bitmap Index Scan on qualifiedpurlversionjsongistidx  (cost=0.00..1.69 rows=41 width=0) (actual time=19.290..19.290 rows=0 loops=1)
               Index Cond: ((purl ->> 'version'::text) ~~* 'test7'::text)

Note- will probably drop already existing GIST indexes once we determine not needed in graph analysis (part of other ongoing multi sbom work).

Copy link
Contributor

@ctron ctron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM: We just be need to careful to not have GIN indices and the tonic crate come too close.

@JimFuller-RedHat JimFuller-RedHat added this pull request to the merge queue Feb 12, 2025
Merged via the queue into main with commit 5016973 Feb 12, 2025
3 checks passed
@JimFuller-RedHat JimFuller-RedHat deleted the search-package-perf branch February 12, 2025 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants