Skip to content

Conversation

@simonswine
Copy link
Contributor

@simonswine simonswine commented Jan 9, 2026

This PR implements a complete heatmap query API for visualizing profile data distribution over time and value dimensions, with support for both individual profiles and span-level data.

Key Changes:

  • Heatmap Query Infrastructure: Adds new SelectHeatmap endpoint with proto definitions (HeatmapQuery, HeatmapReport, HeatmapSeries), query backend implementation for INDIVIDUAL and SPAN query types, and data aggregation pipeline with builder/merger components for processing parquet data into time/value buckets.
  • Exemplar Support: Each Y-axis bucket includes optional exemplar metadata (profile ID, span ID, timestamp, labels) representing the highest-value profile in that cell, enabling drill-down from aggregated visualizations to individual profiles with little-endian span ID encoding for query consistency.
  • Series Limiting & Validation: Implements TopSeries to return top-N series by total value, with input validation for step, limit, and time range parameters.

@simonswine simonswine force-pushed the 20260105_add-heatmap branch from ba47839 to eb96f68 Compare January 12, 2026 13:18
@simonswine simonswine marked this pull request as ready for review January 16, 2026 11:35
@simonswine simonswine changed the title WIP: Allow retrieving heatmap from spans/individual profiles Allow retrieving heatmap from spans/individual profiles Jan 16, 2026
Copy link
Contributor

@aleks-p aleks-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

}

// createYAxisBuckets creates evenly spaced Y-axis buckets
func createYAxisBuckets(minValue, maxValue uint64, numBuckets int) []yBucket {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use []int64 for Y buckets like for time buckets or does tracking min and max make some things easier?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mistake was made in the Exemplar proto API, not too sure this should either be int64 or straight a double: int64 is used in parquet and that is using it because of pprof. Our API Repsonse is using doubles (so is SelectSeries.

I would argue those values should be an int64. Will provide a PR in main after this merges

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will provide a PR in main first, before fixing it here

Copy link
Contributor

@marcsanmi marcsanmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! 👏


// findTimeBucket finds the index of the time bucket for a given timestamp
func findTimeBucket(timestamp int64, buckets []int64) int {
for i, bucket := range buckets {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a binary search be faster for the already sorted buckets?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends on the amount of points, it should really be calcuated

}

// This attribute table is used to store the attribute values for the heatmap. It might be reused for other queries at a later time.
type attributeTable struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a specific attributetable model in my PR. If it makes sense, we might want to reuse it once merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes let's do that whatever merges first

@simonswine simonswine merged commit db2dc63 into main Jan 22, 2026
22 of 23 checks passed
@simonswine simonswine deleted the 20260105_add-heatmap branch January 22, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants