Change confidenceLevel from boolean to Enum

[abhinavmuk04]

Description

Change confidenceLevel from boolean to Enum

Motivation and Context

This will help us with future changes where we want to consider choosing the 'FACT' CBO over regular HBO

Test Plan

Reran existing tests where previosuly 'isConfident' was used with new 'confidenceLevel' functions

Contributor checklist

• Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

== NO RELEASE NOTE ==

[jaystarshot]

Why not keep it a fraction and add a method to get high low etc?
This can be used for eg-
Table scan - confidence 1 ,
filter on top decreases confidence by 0.05
so filter-tablescan can have confidence 0.95

[rschlussel]

Why not keep it a fraction and add a method to get high low etc? This can be used for eg- Table scan - confidence 1 , filter on top decreases confidence by 0.05 so filter-tablescan can have confidence 0.95

So that approach is nice because it allows for appreciating that compounding errors makes things worse, but my concern is that it's way more precision than we actually know and more than we really have a use for. For example, even though a filter on top of a join on top of a filter has a problem of compounding error, a filter on top of a tablescan is also basically a guess if there are no histograms, or even with histograms if your filter is a more complex function. Similarly if you have filters on multiple columns and you don't know the extent to which they are correlated, it's all a guess. I'm not sure it's worth teasing out the relative confidence levels of all these things because the benefit isn't clear to me.

And for similar reasons it would also be hard to keep all the relative confidences consistent (like is x actually less confident than y or did the particular paths the confidence levels came through derive their computation in different ways, were implemented at different times,etc., but really we'd think y is less confident than x or they're about the same)

I think until there's a clear use case for a continuous range of confidence levels, a few discrete options is simpler and does what we need.

[rschlussel]

there are many other stats rules that need to be updated. check for all classes that extend simple stats rule or all usages of PlanNodeStatsEstimate.Builder

[rschlussel]

minor comment, but looks good. Thanks!