Feature Request / Improvement
approx_set(x) produces a HyperLogLog sketch at a fixed precision of 4096 buckets, which corresponds to a standard error of approximately 1.6% (1.04 / sqrt(4096)). For workloads that store these sketches and later combine them with merge() and read them with cardinality(), ~1.6% is often too coarse, and there is currently no way to tune it.
approx_distinct(x, maxStandardError) already lets users trade memory for accuracy, but the equivalent knob is missing on approx_set, even though both are backed by the same HyperLogLog implementation. Other engines expose this: for reference, Amazon Redshift's approximate distinct counting uses a tunable HyperLogLog precision and documents an average relative (standard) error range of roughly 0.01–0.6%.
Proposal
Add a two-argument form approx_set(x, maxStandardError) that controls the sketch precision, mirroring approx_distinct's maxStandardError (permitted range [0.0040625, 0.26], using the same standard-error-to-buckets mapping). The single-argument approx_set(x) remains unchanged for backward compatibility with existing stored sketches.
Caveat
HyperLogLog sketches can only be merged when they share the same precision (bucket count). Sketches produced with different maxStandardError values — or mixing the single- and two-argument forms when they resolve to different bucket counts — cannot be merged. This should be documented, and attempting such a merge should fail as a user error rather than an internal error.
Feature Request / Improvement
approx_set(x)produces aHyperLogLogsketch at a fixed precision of 4096 buckets, which corresponds to a standard error of approximately 1.6% (1.04 / sqrt(4096)). For workloads that store these sketches and later combine them withmerge()and read them withcardinality(), ~1.6% is often too coarse, and there is currently no way to tune it.approx_distinct(x, maxStandardError)already lets users trade memory for accuracy, but the equivalent knob is missing onapprox_set, even though both are backed by the sameHyperLogLogimplementation. Other engines expose this: for reference, Amazon Redshift's approximate distinct counting uses a tunable HyperLogLog precision and documents an average relative (standard) error range of roughly 0.01–0.6%.Proposal
Add a two-argument form
approx_set(x, maxStandardError)that controls the sketch precision, mirroringapprox_distinct'smaxStandardError(permitted range[0.0040625, 0.26], using the same standard-error-to-buckets mapping). The single-argumentapprox_set(x)remains unchanged for backward compatibility with existing stored sketches.Caveat
HyperLogLogsketches can only be merged when they share the same precision (bucket count). Sketches produced with differentmaxStandardErrorvalues — or mixing the single- and two-argument forms when they resolve to different bucket counts — cannot be merged. This should be documented, and attempting such a merge should fail as a user error rather than an internal error.