Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] fix gradient_clip value in tutorials to ensure reproducible outputs similar to the committed cell output #1750

Merged
merged 2 commits into from
Feb 17, 2025

Conversation

gbilleyPeco
Copy link
Contributor

@gbilleyPeco gbilleyPeco commented Jan 8, 2025

Update gradient_clip=0.01 to gradient_clip=0.1. Fixes #1769.

Description

I've been following along with the Interpretable forecasting with N-Beats tutorial found here https://pytorch-forecasting.readthedocs.io/en/latest/tutorials/ar.html, copy-pasting the code exactly, and I am getting different results compared to what is shown on the documentation page.

The first discrepancy between my results and the tutorial is when using lightning.pytorch.tuner.Tuner to find the optimal learning rate. The suggested learning rate and resulting plot are different.

Perhaps more of a problem, however, is that my model fails to learn (training loss doesn't decrease), even when increasing the epochs. When I run the code from the Evaluate Results section, all of the resulting plots show a flat predicted curve.

I believe I've found the problem. Testing on my side confirms.

Code cells [6] and [14] have the parameter gradient_clip=0.01, which is the source of the problem. The N-HiTS tutorial uses gradient_clip=0.1, so I tried that out, which fixes the problem. The outputs don't match the tutorial outputs exactly, but they're much closer than what gradient_clip=0.01 yields.

Checklist

  • Linked issues (if existing)
  • Amended changelog for large changes (and added myself there as contributor)
  • Added/modified tests
  • Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
    To run hooks independent of commit, execute pre-commit run --all-files

Make sure to have fun coding!

Update gradient_clip=0.01 to gradient_clip=0.1.
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@yarnabrina
Copy link
Member

Hi @gbilleyPeco , thanks for the PR. Is there any specific objective reason to prefer 0.1 over 0.01, except may be the fact 0.01 is too small? Can you please update the goal of the PR in the description section?

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Would you mind copying the description from discord why we are doing this in the first post in this PR, or an issue that you reference?

@Sohaib-Ahmed21
Copy link
Contributor

@gbilleyPeco thanks for pointing. Was also working around to get correct predictions and it worked.

@fkiraly fkiraly added the documentation Improvements or additions to documentation label Feb 6, 2025
Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable with the explanation.

@fkiraly fkiraly changed the title Fix gradient_clip value. [DOC] fix gradient_clip value in tutorials to ensure reproducible outputs similar to the committed cell output Feb 17, 2025
@fkiraly fkiraly merged commit 654b94d into sktime:main Feb 17, 2025
34 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Locally run tutorials do not show same results as web tutorials
4 participants