Temporarily removing https://search.ononoki.org

[unixfox]

Hello @ononoki1,

I'm informing you that we temporarily removed your instance from the public list on https://searx.space

We believe that you are caching the requests from the server check.searx.space in order to be always at the top of the list. Please stop doing that because:

• It's bad for privacy to cache the requests.
• The metrics related to your instance on https://searx.space are incorrect because by default your instance have a much higher response time than shown on https://searx.space.
• It's not fair for the other instances to manipulate the ranking.

Once you have removed the cache we can add back your instance.

[ononoki1]

Actually I'm caching all requests from VPS IP, which can prevent bots from affecting the search quality of searxng while keeping normal users not affected. There are a lot of bots using VPS IP that search some simple words like weather or time every 5 to 15 mins. They use well-designed HTTP headers so filtron cannot block them. And such behaviour would lead to rate limit by Google or other search engines, which downgraded normal user's experience. To minimize such effect, simply caching would be great.

But still, as you requested, I have disabled the cache.

BTW, I didn't intend to manipulate, because it's very easy to find that the result must be cached since my server in US is even faster than other servers in EU, which is impossible in normal case.

[unixfox]

It's great that you care about the user experience of your instance!

If you are being rate limited by Google I suggest you to set use_mobile_ui to true in the settings.yml, and you won't have any rate limit errors anymore.

About the other engines, it's unfortunately the issue with running a public instance. You won't be able to block all the bots, unless you start to degrading the user experience by having a web firewall like a browser verification page, but I don't recommend doing that.

You could also use the new limiter plugin in searx in order to block more efficiently the bots: https://docs.searxng.org/src/searx.plugins.limiter.html#limiter-plugin. Or you can use proxies in order to circumvent the rate limit of the engines.

Feel free to talk about the ways to limit the bot usage on our matrix or IRC channel: https://github.com/searxng/searxng#contact

BTW, I didn't intend to manipulate, because it's very easy to find that the result must be cached since my server in US is even faster than other servers in EU, which is impossible in normal case.

searx.space rank based on the internal response time of the instance, not by the response time between the searx.space server and your server. This allows to have a fair ranking even for the instances outside of europe.

---

I'll back by your instance.

[ononoki1]

If you are being rate limited by Google I suggest you to set use_mobile_ui to true in the settings.yml, and you won't have any rate limit errors anymore.

This is amazing! Thank you so much!

[tiekoetter]

searx.space rank based on the internal response time of the instance, not by the response time between the searx.space server and your server. This allows to have a fair ranking even for the instances outside of europe.

It definitely does depend on the response time between the searx.space checker and the instance. We talked about this in the first meeting because it is very "eurocentric" as @ononoki1 pointed out.

[ononoki1]

Besides, concerning response time check, I suggest that more modern client should be used. For example, the client should support http3 (or at least http2) to better simulate real user's experience. My instance supports http3 (see here) while many others don't. By doing so we can promote the usage of http3, which is good for users.

[unixfox]

httpx, the HTTP client used by SearX and SearXNG doesn't support HTTP3 yet: encode/httpx#275