[ENH] optimize metric state updates to prevent O(N^2) reallocation

[andersendsa]

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Initialize states with empty lists instead of dummy tensors for reduction='none'. Append to list during update and only do a single torch.cat inside compute. This dramatically speeds up long evaluation loops for multi-horizon metrics.

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

Any other comments?

PR checklist

• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
• Added/modified tests
• Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
  To run hooks independent of commit, execute pre-commit run --all-files

[codecov]

Codecov Report

❌ Patch coverage is 65.00000% with 7 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@5a35991). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
..._forecasting/metrics/base_metrics/_base_metrics.py	65.00%	7 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2371   +/-   ##
=======================================
  Coverage        ?   87.43%           
=======================================
  Files           ?      175           
  Lines           ?    10176           
  Branches        ?        0           
=======================================
  Hits            ?     8897           
  Misses          ?     1279           
  Partials        ?        0           

Flag	Coverage Δ
cpu	87.43% <65.00%> (?)
pytest	87.43% <65.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[phoeenniixx]

Thanks! Can you please share an example how this helps?
Here you check if losses is a list, when does that happen?

[andersendsa]

Thanks! Can you please share an example how this helps? Here you check if losses is a list, when does that happen?

When reduction="none" is used for a metric, the metric state needs to accumulate predictions and targets across multiple batches in order to evaluate them at the end.

Previously, losses and lengths were initialized as tensors. On every single forward pass (the update() call), the code ran: self.losses = torch.cat([self.losses, losses], dim=0)

Using torch.cat() in a loop requires allocating a brand new block of memory, copying the accumulated tensor so far, copying the new tensor, and freeing the old tensor. As the tensor gets larger over thousands of batches, this operation gets slower and slower (O(N^2) time complexity).

In this change, I replaced the initial tensor with an empty list []. In update(), we simply append to the list: self.losses.append(losses)

List appends are O(1). Then, at the very end when compute() is called, we do a single torch.cat() operation on the whole list, dropping the time complexity down to O(N). When I wrote a quick test loop simulating 1,000 batches, the time to execute dropped from over 6 seconds to roughly 0.01 seconds.

As for checking if isinstance(self.losses, list): By default, we initialized the state as []. So during standard update() calls, it's a list. However, because this metric inherits from TorchMetrics, some backend processes like syncing metrics across multiple GPUs (dist_reduce_fx=cat) may automatically concatenate our list state back into a single tensor before compute() is called. To handle cases where a user runs in distributed mode or manually modifies the state, checking if isinstance(self.losses, list): allows it to safely append to a list, or fallback gracefully to concatenating tensors.