mx: add ceil and RNE rounding modes to the cast from fp32 to e8m0 #1643
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1643
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 0220b19 with merge base 4d1c774 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
vkuzo
added a commit
that referenced
this pull request
Jan 30, 2025
Summary: Why we want this: in the newly released cuBLAS 12.8 documentation, the RNE rounding mode is used for the cast to e8m0. We want to properly emulate this cast. This is a copy-pasta of #516 with the modifications being keeping FLOOR the default more, and removing e3m0, credit to NicoleMayer for the original code. I don't have a way to check bitwise equivalency with the most recent cuBLAS version yet, but will come back and add tests when I do. Test Plan: ``` pytest test/prototype/mx_formats/ -s -x ``` Reviewers: Subscribers: Tasks: Tags: ghstack-comment-id: 2613597310 ghstack-source-id: 267dc2222b6469f2e86b5bd928554ae44018b9dc ghstack-comment-id: 2623497069 Pull Request resolved: #1643
facebook-github-bot
added
the
CLA Signed
vkuzo
added
the
topic: improvement
drisspg
approved these changes
Jan 30, 2025
vkuzo
added a commit
that referenced
this pull request
Jan 30, 2025
Summary: Why we want this: in the newly released cuBLAS 12.8 documentation, the RNE rounding mode is used for the cast to e8m0. We want to properly emulate this cast. This is a copy-pasta of #516 with the modifications being keeping FLOOR the default more, and removing e3m0, credit to NicoleMayer for the original code. I don't have a way to check bitwise equivalency with the most recent cuBLAS version yet, but will come back and add tests when I do. Test Plan: ``` pytest test/prototype/mx_formats/ -s -x ``` Reviewers: Subscribers: Tasks: Tags: ghstack-comment-id: 2613597310 ghstack-source-id: 0999376e5c37815c28483adaaef451b01e75c330 ghstack-comment-id: 2623497069 Pull Request resolved: #1643
vkuzo
added a commit
that referenced
this pull request
Jan 30, 2025
Summary: Why we want this: in the newly released cuBLAS 12.8 documentation, the RNE rounding mode is used for the cast to e8m0. We want to properly emulate this cast. This is a copy-pasta of #516 with the modifications being keeping FLOOR the default more, and removing e3m0, credit to NicoleMayer for the original code. I don't have a way to check bitwise equivalency with the most recent cuBLAS version yet, but will come back and add tests when I do. Test Plan: ``` pytest test/prototype/mx_formats/ -s -x ``` Reviewers: Subscribers: Tasks: Tags: ghstack-comment-id: 2613597310 ghstack-source-id: a160d4c9320301e0d3d62128ad386b7e0c985d59 ghstack-comment-id: 2623497069 Pull Request resolved: #1643
vkuzo
added a commit
that referenced
this pull request
Feb 10, 2025
Summary: Why we want this: in the newly released cuBLAS 12.8 documentation, the RNE rounding mode is used for the cast to e8m0. We want to properly emulate this cast. This is a copy-pasta of #516 with the modifications being keeping FLOOR the default more, and removing e3m0, credit to NicoleMayer for the original code. I don't have a way to check bitwise equivalency with the most recent cuBLAS version yet, but will come back and add tests when I do. Test Plan: ``` pytest test/prototype/mx_formats/ -s -x ``` Reviewers: Subscribers: Tasks: Tags: ghstack-comment-id: 2613597310 ghstack-source-id: e52136a40ad0b4fdd00aa089c3f38141a21d0f56 ghstack-comment-id: 2623497069 Pull Request resolved: #1643
vkuzo
added a commit
that referenced
this pull request
Feb 10, 2025
Summary: Why we want this: in the newly released cuBLAS 12.8 documentation, the RNE rounding mode is used for the cast to e8m0. We want to properly emulate this cast. This is a copy-pasta of #516 with the modifications being keeping FLOOR the default more, and removing e3m0, credit to NicoleMayer for the original code. I don't have a way to check bitwise equivalency with the most recent cuBLAS version yet, but will come back and add tests when I do. Test Plan: ``` pytest test/prototype/mx_formats/ -s -x ``` Reviewers: Subscribers: Tasks: Tags: ghstack-comment-id: 2613597310 ghstack-source-id: c607b6cc89726159f370dcb3d03e107ace1c80f4 ghstack-comment-id: 2623497069 Pull Request resolved: #1643
This was referenced Feb 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Why we want this: the MX spec describes floor rounding, which isn't always the most accurate. It is good to have more options to experiment with.
This is a copy-pasta of #516 with the
modifications being keeping FLOOR the default more, and removing e3m0, credit
to NicoleMayer for the original code. I don't have a way to check bitwise
equivalency with the most recent cuBLAS version yet, but will come back
and add tests when I do.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags: