Skip to content

[ENH] Add train/test split strategy for v2 datamodules - #2375

Open
andersendsa wants to merge 3 commits into
sktime:mainfrom
andersendsa:train-val-test-split
Open

[ENH] Add train/test split strategy for v2 datamodules#2375
andersendsa wants to merge 3 commits into
sktime:mainfrom
andersendsa:train-val-test-split

Conversation

@andersendsa

@andersendsa andersendsa commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Closes:#1974

What does this implement/fix? Explain your changes.

Added train_val_test_split_strategy to TslibDataModule and EncoderDecoderTimeSeriesDataModule allowing users to split their data set randomly (default) or sequentially. Fixed an associated test that checks for multivariate targets being list of tensors.

PR checklist

  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
  • Added/modified tests
  • Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
    To run hooks independent of commit, execute pre-commit run --all-files

@andersendsa andersendsa changed the title [ENH] dd train/test split strategy for v2 datamodules [ENH] Add train/test split strategy for v2 datamodules Aug 8, 2026
@andersendsa
andersendsa marked this pull request as draft August 8, 2026 09:38
@andersendsa
andersendsa marked this pull request as ready for review August 8, 2026 09:50
@phoeenniixx

Copy link
Copy Markdown
Member

Thanks!
What startegies are you implementing here? can you please specify? I can see we are just taking the indices in the sequential manner?
You can look at #2087 which adds temporal split - to make sure we dont duplicate the work

@andersendsa

andersendsa commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks! What startegies are you implementing here? can you please specify? I can see we are just taking the indices in the sequential manner? You can look at #2087 which adds temporal split - to make sure we dont duplicate the work

Hi @phoeenniixx
The strategies I have implemented are random and sequential.

random : This is the default strategy (and the existing behavior), which shuffles the indices of all time series before applying the split ratios using torch.randperm(total_series).

sequential: This is a new strategy that generates a sequence of indices without randomizing using torch.arange(total_series). By keeping the time series in their original order, it guarantees that train datasets get the first N series, validation datasets get the next M series, and test datasets get the final remainder.

@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 27.27273% with 16 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@033169a). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...h_forecasting/data/tests/test_tslib_data_module.py 0.00% 10 Missing ⚠️
...g/data/data_module/_encoder_decoder_data_module.py 50.00% 3 Missing ⚠️
...forecasting/data/data_module/_tslib_data_module.py 50.00% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2375   +/-   ##
=======================================
  Coverage        ?   87.33%           
=======================================
  Files           ?      175           
  Lines           ?    10185           
  Branches        ?        0           
=======================================
  Hits            ?     8895           
  Misses          ?     1290           
  Partials        ?        0           
Flag Coverage Δ
cpu 87.33% <27.27%> (?)
pytest 87.33% <27.27%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants