[ENH] Integrate Autoformer model into V2 architecture

[harshsomankar123-tech]

Description

This PR integrates the Autoformer model (based on the NeurIPS 2021 paper) into the new PyTorch-Forecasting V2 architecture. Autoformer implements a series-wise decomposition connection to progressively decompose time series into trend and seasonal components, and utilizes an Auto-Correlation mechanism to replace standard point-wise self-attention for long-term forecasting.

Reference Issues/PRs

fix: #2004

What does this implement/fix? Explain your changes.

This PR introduces the Autoformer architecture to the V2 models ecosystem. Specifically, it implements:

1. Autoformer Layer Submodules (pytorch_forecasting/models/autoformer/sub_modules.py):

   • AutoCorrelation: Fast Fourier Transforms (FFT) for periodicity discovery and time-delay aggregation.
   • AutoCorrelationLayer: Projection layer wrapper.
   • series_decomp & moving_avg: Custom trend-seasonal time series decomposition.
   • Encoder & Decoder: Progressive series-decomposition architecture.
   • Embeddings: DataEmbedding_wo_pos, TokenEmbedding, and temporal embeddings.
   • my_Layernorm: Specialized LayerNorm for seasonal components.

2. Autoformer V2 Model (pytorch_forecasting/models/autoformer/_autoformer_v2.py):

   • Inherits from TslibBaseModel to handle target values and continuous covariates in the V2 input dictionary.
   • Supports both standard regression losses and QuantileLoss predictions.
   • Implements channel dimension alignment to resolve potential broadcasting mismatches between target dimensions, covariates, and quantile channels during decoding.

3. Wrapper Package & Registry (pytorch_forecasting/models/autoformer/_autoformer_pkg_v2.py):

   • Specifies metadata tags (exogenous, multivariate, and pred_int) and parameter grids for dynamic discovery by the package registry.

What should a reviewer concentrate their feedback on?

Reviewers should primarily focus on:

• Channel Dimension Alignment: Specifically within _autoformer_v2.py, verifying the logic used to prevent broadcasting mismatches between targets, covariates, and quantile channels.
• Quantile Loss Integration: Ensuring the continuous covariates and target values are correctly handled alongside QuantileLoss.
• V2 Input Dictionary Parsing: Confirming that TslibBaseModel handles the dictionary routing to the encoder/decoder appropriately.

Did you add any tests for the change?

Yes, a comprehensive test suite was added in tests/test_models/test_autoformer_v2.py.

• Custom Unit Tests: 7/7 passed. Verifies model initialization, forward pass dimensions, and quantile predictions.
• Automated Framework & Registry Tests: 18/18 passed. Verifies checkpointing, serialization/reloading, prediction modes, and datamodule collation compatibility.

Any other comments?

All tests were verified locally using .venv/bin/pytest tests/test_models/test_autoformer_v2.py and .venv/bin/pytest -k Autoformer.

PR checklist

• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
• Added/modified tests
• Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
  To run hooks independent of commit, execute pre-commit run --all-files

[harshsomankar123-tech]

@phoeenniixx @PranavBhatP @fkiraly Could you please take a look when you have some time?
Thanks!

[codecov]

Codecov Report

❌ Patch coverage is 87.37374% with 50 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@7bcf66c). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...forecasting/layers/_attention/_auto_correlation.py	80.80%	19 Missing ⚠️
...asting/layers/_embeddings/_autoformer_embedding.py	78.26%	15 Missing ⚠️
...ch_forecasting/models/autoformer/_autoformer_v2.py	89.47%	10 Missing ⚠️
...orecasting/layers/_encoders/_autoformer_encoder.py	86.36%	6 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2303   +/-   ##
=======================================
  Coverage        ?   87.50%           
=======================================
  Files           ?      184           
  Lines           ?    10557           
  Branches        ?        0           
=======================================
  Hits            ?     9238           
  Misses          ?     1319           
  Partials        ?        0           

Flag	Coverage Δ
cpu	87.50% <87.37%> (?)
pytest	87.50% <87.37%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[phoeenniixx]

Thanks!
I have added some suggestions based on a shallow review. Will do a more deeper review shortly

[phoeenniixx]

Thanks!
Added a few comments, please have a look at them,
I think we should try to de duplicate the tests here and only tests those things which are not being tested by the test framework.
See the tests for the test framework here: https://github.com/sktime/pytorch-forecasting/blob/main/pytorch_forecasting/tests/test_all_v2/test_all_estimators_v2.py

[harshsomankar123-tech]

@phoeenniixx @fkiraly Please Take a Look
Thanks!