-
Notifications
You must be signed in to change notification settings - Fork 1.3k
fix: round to correct precision when step uses exponential notation #8290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
expect(roundToStepPrecision(5, 10)).toBe(5); | ||
}); | ||
it('should round to the correct decimal places for steps with decimals', () => { | ||
expect(roundToStepPrecision(1.24, 0.1)).toBe(1.24); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe I'm over thinking this, but shouldn't this be
expect(roundToStepPrecision(1.24, 0.1)).toBe(1.24); | |
expect(roundToStepPrecision(1.24, 0.1)).toBe(1.2); |
I realize it's the existing behaviour, I'll have to look back to remember why
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for looking into this! I was wondering the same thing while looking at the original implementation 🤔
let precision = pointIndex >= 0 ? stepString.length - pointIndex : 0;
I also thought precision should maybe be stepString.length - pointIndex - 1
, but the original behavior seemed correct, since all the existing test cases in NumberField.test and number.test passed. Also, this #8274 (comment) suggests that for an exponential notation like 1e-7, the precision should be 8. So, I implemented the existing behavior for exponential notation as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hi @scrgl thanks for pointing out that incorrect implementation! 872c527 fixed the issue mentioned in #8290 (comment) and added more tests.
I opted not to use split
as suggested in the snippet because it creates additional intermediate arrays every time snapValueToStep
is called. Since this function may be invoked multiple times by hooks and components, those arrays could quickly add up. Also, since step
values without explicit exponential notation are likely more common in user land, normalizing everything to exponential format upfront would add extra overhead for those cases.
Hi @lucasweng, thank you for taking the time to look into this! I believe it would be helpful to add a few more test cases to cover edge scenarios. Specifically, since JavaScript automatically converts very small numbers into exponential notation, it would be great to include a case like 0.0000000231 as a step value. This gets transformed into "2.31e-8", and your current implementation may not handle it correctly—it identifies the dot and treats the value as if it were in standard decimal format. To simplify things and make the logic more robust, I’d suggest normalizing all numbers to exponential notation upfront. That way, you only need to handle one consistent format. Here's a snippet that might help—though please double-check it against your requirements and test cases:
|
Closes #8274
✅ Pull Request Checklist:
📝 Test Instructions:
roundToStepPrecision
.🧢 Your Project: