Do not include any Quarkiverse guides on the initial guides page

[marko-bekhta]

This one is more to start the discussion on what we would want to do about the Quarkiverse guides once we get them searchable by the search service (quarkusio/search.quarkus.io#260 (comment)).

Once the change is deployed to production, the search service will return results found over the entire Quarkiverse and not just the guides that we currently explicitly list on the https://quarkus.io/guides/. Hence, do we want to keep the explicit list visible on the initial page, or remove them, or do something else?

cc: @yrodiere

[github-actions]

🙈 The PR is closed and the preview is expired.

[yrodiere]

Once the change is deployed to production, the search service will return results found over the entire Quarkiverse and not just the guides that we currently explicitly list on the https://quarkus.io/guides/. Hence, do we want to keep the explicit list visible on the initial page, or remove them, or do something else?

I'd remove them.

@gastaldi , @gsmet , agreed?

[yrodiere]

Once the change is deployed to production, the search service will return results found over the entire Quarkiverse and not just the guides that we currently explicitly list on the https://quarkus.io/guides/.

Note relevance/scoring is not great at the moment (including in the preview you're seeing for this PR), but there's a PR to search.quarkus.io to improve that: quarkusio/search.quarkus.io#402

[holly-cummins]

Once the change is deployed to production, the search service will return results found over the entire Quarkiverse and not just the guides that we currently explicitly list on the https://quarkus.io/guides/. Hence, do we want to keep the explicit list visible on the initial page, or remove them, or do something else?

I'd remove them.

@gastaldi , @gsmet , agreed?

I'd vote for "include them, but with a different card design so users can see it's a different source." I'm not sure users necessarily want to have to guess whether a guide would be quarkus-hosted or quarkiverse-hosted before they do a search for docs. If a user goes to quarkus.io and searches for (say) OpenFaces, is it helping them to screen out results that match?

If we were worried about clutter, we could apply a higher score threshold to the quarkiverse guides, so that we only show them if something is really relevant.

[marko-bekhta]

"with a different card design so users can see it's a different source."

that's probably more about this one: quarkusio/search.quarkus.io#401

and to clarify ... the Quarkivese guides will be included in the search results; this PR is about including (or not) any quarkiverse guides on the guides landing page (the one user sees without doing any search: https://quarkus.io/guides/)

[holly-cummins]

and to clarify ... the Quarkivese guides will be included in the search results; this PR is about including (or not) any quarkiverse guides on the guides landing page (the one user sees without doing any search: https://quarkus.io/guides/)

Ah, got it - thanks for clarifying, @marko-bekhta!

[yrodiere]

and to clarify ... the Quarkivese guides will be included in the search results; this PR is about including (or not) any quarkiverse guides on the guides landing page (the one user sees without doing any search: https://quarkus.io/guides/)

Interestingly... they don't? At least not in this PR's preview: https://quarkus-site-pr-2219-preview.surge.sh/guides/#q=hibernate+search+extra

Previews seem to be using the prod, search.quarkus.io, which is not what I remembered...

[maxandersen]

Interesting.

How would versioning work here ? Currently the guides page Is versioned after Quarkus core platform rather than Quarkus platform which every user app is using.

But each quarkiverse is on its own version thing.

I reckon this will be more a global Search not bound to versions or is it only sesrching the very latest ?

[marko-bekhta]

yeah ... search will still filter the quarkus guides by the selected version, and as for the quarkiverse guides -- we are only indexing the "latest" (dev/main) versions, so only those will be included in search results.

We've started with indexing all versions of the quarkiverse guides but couldn't think of a good way to do version filtering as the extension versioning is different 😕. And if we do not filter by version, there will be a lot of duplicated guides for different versions 🙈

see also (quarkusio/search.quarkus.io#260 (comment))

[yrodiere]

We've started with indexing all versions of the quarkiverse guides but couldn't think of a good way to do version filtering as the extension versioning is different 😕. And if we do not filter by version, there will be a lot of duplicated guides for different versions 🙈

see also (quarkusio/search.quarkus.io#260 (comment))

This.

We could consider filtering Quarkiverse guides per Quarkus version, that shouldn't be too hard, but the data to do this simply isn't there.

We would need Quarkiverse extensions to provide some machine-readable metadata mapping Quarkiverse extension version to Quarkus version. I doubt we can hope having this for every extension, but we don't need to: each extension that provides this metadata could be indexed/filtered as necessary, while others would remained unfiltered (only dev is indexed, and returned for all Quarkus versions -- i.e. the behavior we have on staging).

[gastaldi]

We would need Quarkiverse extensions to provide some machine-readable metadata mapping Quarkiverse extension version to Quarkus version.

The Quarkus core version used to build the extension is available in the extension's YAML metadata and can be exposed somehow to accommodate the indexing needs

[gsmet]

@yrodiere so I wouldn't have a problem keeping them if they were in a separate section and also considered per guide, not per page (or you could end up with 10 pages for a given extension and 5 for another).

I understand the rationale of indexing pages and pointing to pages, but it doesn't work very well for the home page.

But given the state of it, I would only display them when searching for now. And take a step back once we can get some cycles to think about the home page using faceting.

[yrodiere]

We would need Quarkiverse extensions to provide some machine-readable metadata mapping Quarkiverse extension version to Quarkus version.

The Quarkus core version used to build the extension is available in the extension's YAML metadata and can be exposed somehow to accommodate the indexing needs

That's better than nothing but unfortunately not enough, as extensions can only be built with one version of Quarkus, but still often work (or at least are intended to work) with multiple versions of Quarkus.

If my Quarkiverse extension has a branch named 1.0 built with Quarkus 2.15, a branch 2.0 built with Quarkus 3.2, and a branch 2.1 built with Quarkus 3.8... which branch should I assume works with Quarkus 3.5?
Can I assume the extension works with Quarkus latest (3.18), or should I assume it doesn't, and remove it from search results for latest? This would force the maintainer to re-publish their extension on every new Quarkus release on pain of being removed from the website -- even though they know it works, thanks to the SNAPSHOTS CI...

What I'm getting at is: the information we have right now is not enough to make informed choices, so I'd better we don't try -- until we have more information.

[yrodiere]

@yrodiere so I wouldn't have a problem keeping them if they were in a separate section and also considered per guide, not per page (or you could end up with 10 pages for a given extension and 5 for another).

I understand the rationale of indexing pages and pointing to pages, but it doesn't work very well for the home page.

Not sure which home page we're talking about... each Quarkiverse extension's documentation home page?

But given the state of it, I would only display them when searching for now.

I'll take it as an approval; @marko-bekhta is this still a draft or can we go ahead?

And take a step back once we can get some cycles to think about the home page using faceting.

+1, though I'd appreciate an issue describing the problem -- I'm a bit lost, sorry :/

[gastaldi]

@yrodiere in quarkusio/quarkus#41908 we introduced a requires-quarkus-core specifying a version range with which the extension is compatible. Although we do not enforce them at the moment, it could be a useful data for this purpose

[yrodiere]

@yrodiere in quarkusio/quarkus#41908 we introduced a requires-quarkus-core specifying a version range with which the extension is compatible. Although we do not enforce them at the moment, it could be a useful data for this purpose

Definitely! Once there's a few Quarkiverse extensions using it, and assuming the information is available easily on quarkiverse.io, we can look into what I suggested above:

each extension that provides this metadata could be indexed/filtered as necessary, while others would remained unfiltered (only dev is indexed, and returned for all Quarkus versions -- i.e. the behavior we have on staging).

[marko-bekhta]

I understand the rationale of indexing pages and pointing to pages, but it doesn't work very well for the home page.

maybe we can generate quarkiverse.yaml as part of the https://github.com/quarkiverse/quarkiverse-docs build, I think we should be able to get the title/url/summary and if we put these under a different type e.g.:

types:
  quarkiverse-reference:
  - title: Neo4j Client
    url: https://quarkiverse.github.io/quarkiverse-docs/quarkus-neo4j/dev/index.html
    summary: Neo4j is a graph database management system developed by Neo4j, Inc. Neo4j is a native graph database focused not only on the data itself, but especially on the relations between data. Neo4j stores data as a property graph, which consists of vertices or nodes as we call them, connected with edges or relationships. Both of them can have properties.
    origin: quarkiverse-hub
  - title: Amazon DynamoDB
  .........

then we could display them all at the bottom of the /guides page (when there's no search query).

I'll take it as an approval; @marko-bekhta is this still a draft or can we go ahead?

rebased 🙈 and moved from draft 😃

[gastaldi]

maybe we can generate quarkiverse.yaml as part of the https://github.com/quarkiverse/quarkiverse-docs build, I think we should be able to get the title/url/summary and if we put these under a different type

That'a a good idea, and totally doable

[maxandersen]

Also remember an alternative (or possible additional) way of presenting this search is to do it per quarkus platform - because here we know the platform and we know the latest version of any given extension what they work on.

Thus we could present it for any version of a quarkus platform (io.quarkus.platform not the core io.quarkus) we can know what is the right set of docs to index/search.

[maxandersen]

Also remember an alternative (or possible additional) way of presenting this search is to do it per quarkus platform - because here we know the platform and we know the latest version of any given extension what they work on.

Thus we could present it for any version of a quarkus platform (io.quarkus.platform not the core io.quarkus) we can know what is the right set of docs to index/search.

btw. @gastaldi et.al. is saying the same above - I'm just saying we could use io.quarkus.platform rather than core.io quarkus - then it could be applied to RHBQ and other platforms too + I would show the latest released version doc rather than "latest dev".

[yrodiere]

Unless I'm badly misinformed, most Quarkiverse extensions are not part of any platform, so I'm not sure this solves much with the problem we're currently interested in... ?

It could still be a valueable new way of presenting things, of course.

[yrodiere]

I would show the latest released version doc rather than "latest dev".

dev is actually just the documentation of the main branch. For many extensions, that dev is actually the documentation of the latest released version. Some (many?) extensions that don't do branches even have only dev to show.

[yrodiere]

If anyone has a suggestion for immediate improvements or objects to merging this until we take a few more weeks to evolve the Quarkiverse metadata, let them talk now or forever be silent. Okay, not forever, they can open issues.

I'll merge tomorrow.

[yrodiere]

Merged. @marko-bekhta I think we can go ahead and deploy search.quarkus.io to production? And maybe merge the quarkiverse.io changes afterwards? Can you please take care of all that and make sure everything works as expected?

[marko-bekhta]

yes, sure 😃🏃🏻‍♂️