Statistics

[sebersole]

We should take this opportunity to evaluate the statistics we want to collect and expose to include

1. existing ones that are good as-is
2. existing ones that need tweaking
3. existing ones that need removing
4. new ones

At the moment this discussion is just a holder. I still need to go through them and categorize

[gavinking]

Would be interesting to hear from @Sanne and others involved in benchmarking whether there are any statistics we produce that they find especially useful. (On the other hand, as usual, benchmarking is as usual a sort-of unrealistic case.)

[Sanne]

Good idea.

Among stats which I find extremely useful are the one relating to 2LC: hits, misses, etc.. as they help asses if they are being configured effectively.

Of the problematic ones, the "Query Statistic" is one that would need some changes: it's impossible to track this data accurately while also using a bounded map to avoid storing each and every query string; there are some open issues relating to this. IMO we should remove this, and have perhaps a hint or something similar so t hat for a specific query stats can get enabled - but it has to be explicit so that people can mark specific queries they want stats for.

In my experience talking with users (and reading forums), the one reporting the "slowest queries" is very popular, but I believe should be structured in a different way.

More in general though, when I do performance work one of the first things I do is to disable stats. Being able to flag particularly slow queries and checking 2LC setup are useful diagnostic tools which I enable when suspecting that something is possibly wrong in those areas - so this leads me to think that it would also be nice to not have a global "all stats on" vs "all stats off" but there could be use of more fine grained settings.

Metrics and custom Flight Recorder events are likley to make this all obsolete though; perhaps let's not spend too much time on this beyond removing the not-really-useful or tricky ones, then invest in modern metrics.