WIP: feat(search): support code search by zoekt

[adlternative]

WIP: support zoekt code search

Try to support #33702

[wxiaoguang]

There are already so many search engines builtin into Gitea. Many of them have various bugs.

So the questions are:

1. Will more search engines be added into Gitea to make Gitea have plenty of builtin search engines?
2. Will the search engines become unmaintained and the bugs will never be fixed?

[hiifong]

To be honest I prefer this zoekt search engine compared to the existing search engine

[lunny]

maybe this can replace bleve but we need some comparsion tests.

[wxiaoguang]

To be honest I prefer this zoekt search engine compared to the existing search engine

That's understandable. So a few months later, another one feels "yoekt" is better, then introduce "yoekt", then a few months later, someone feels "xoekt" is better, then introduce "xoekt", and then "woekt", "voekt", "uoekt" ... "coekt", "boekt", "aoekt". Then Gitea contains all search engines on the internet.

---

I do not mean objection to introduce improvements. But actually it needs to:

1. Clarify the existing problems & fix existing problems.
2. Remove unnecessary search engine before introducing new ones.

So a clear roadmap about the "search engine plan" is necessary.

[adlternative]

There are already so many search engines builtin into Gitea. Many of them have various bugs.

So the questions are:

1. Will more search engines be added into Gitea to make Gitea have plenty of builtin search engines?

In my opinion, supporting multiple search engines is a good thing, as users may have different needs. Even GitLab now supports both ES and Zoekt search engines. see https://docs.gitlab.com/user/search

2. Will the search engines become unmaintained and the bugs will never be fixed?

I'm not too worried about this; Gitea should have good community maintenance. It might be because the code search functionality is not exposed by default, so many bugs haven't been discovered.

[wxiaoguang]

In my opinion, supporting multiple search engines is a good thing, as users may have different needs. Even GitLab now supports both ES and Zoekt search engines. see https://docs.gitlab.com/user/search
I'm not too worried about this; Gitea should have good community maintenance. It might be because the code search functionality is not exposed by default, so many bugs haven't been discovered.

Well, do you know how many search engines are in Gitea now? And what longstanding bugs do they have? https://github.com/go-gitea/gitea/issues?q=is%3Aissue%20state%3Aopen%20code%20search

And some bugs didn't get fixed in months, for example: "Search Functionality Issues with Bleve Engine #31565", I don't see "good community maintenance"

[adlternative]

To be honest I prefer this zoekt search engine compared to the existing search engine

That's understandable. So a few months later, another one feels "yoekt" is better, then introduce "yoekt", then a few months later, someone feels "xoekt" is better, then introduce "xoekt", and then "woekt", "voekt", "uoekt" ... "coekt", "boekt", "aoekt". Then Gitea contains all search engines on the internet.

you don't need to worry about this: zoekt is a popular code search engine, currently used by code platforms like Gerrit, Sourcegraph, and GitLab, wrote by Gerrit author, and maintained by Sourcegraph. Zoekt has advantages that traditional search engines (like ES) do not possess: support for regex matching, substring search, etc. I don't think any new open-source code search engines will be able to replace it in the short term.

I do not mean objection to introduce improvements. But actually it needs to:

1. Clarify the existing problems & fix existing problems.
2. Remove unnecessary search engine before introducing new ones.

So a clear roadmap about the "search engine plan" is necessary.

You are right, where should the roadmap be written? I don't have experience with this. I will supplement its documentation when the zoekt functionality is more complete

[wxiaoguang]

I don't think any new open-source code search engines will be able to replace it in the short term.

Yep, if zoekt wins, we need to drop some others.

[adlternative]

In my opinion, supporting multiple search engines is a good thing, as users may have different needs. Even GitLab now supports both ES and Zoekt search engines. see https://docs.gitlab.com/user/search
I'm not too worried about this; Gitea should have good community maintenance. It might be because the code search functionality is not exposed by default, so many bugs haven't been discovered.

Well, do you know how many search engines are in Gitea now? And what longstanding bugs do they have? https://github.com/go-gitea/gitea/issues?q=is%3Aissue%20state%3Aopen%20code%20search

And some bugs didn't get fixed in months, for example: "Search Functionality Issues with Bleve Engine #31565", I don't see "good community maintenance"

Sure, it's regrettable that this part of the content is unmaintained. However, for the zoekt code search, I can commit to maintaining it thoroughly.

[adlternative]

I don't think any new open-source code search engines will be able to replace it in the short term.

Yep, if zoekt wins, we need to drop some others.

Yeah, I hope this can be divided into at least two steps:

1. Support zoekt
2. Deprecate other search engines

Zoekt may also have some issues, as GitLab has not completely deprecated ES and fully switched to Zoekt...

[wxiaoguang]

To make the code clear, we need to refactor the related code first: Refactor issue & code search #33860

Each "indexer" should provide the "search modes" they support by themselves. And we need to remove the "fuzzy" search for code.