Skip to content
This repository was archived by the owner on Apr 18, 2025. It is now read-only.

Add full blob encoding and consistency check #1379

Merged
merged 10 commits into from
Jul 25, 2024
Merged

Add full blob encoding and consistency check #1379

merged 10 commits into from
Jul 25, 2024

Conversation

darth-cy
Copy link

@darth-cy darth-cy commented Jul 22, 2024 •
edited
Loading

Description

  • To make sure the full blob test is valid when encoding config changes, an assertion guard is added to make sure full blob test produces a blob of appropriate len.
  • An additional batch data processing fn is added to segment raw batch bytes with metadata

roynalnaruto
roynalnaruto reviewed Jul 22, 2024
View reviewed changes
Copy link

@roynalnaruto roynalnaruto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, added a nit-pick suggestion

@darth-cy darth-cy marked this pull request as draft July 23, 2024 04:09
@darth-cy darth-cy marked this pull request as ready for review July 24, 2024 23:18
@darth-cy darth-cy changed the title Add assertion for full blob encoding Add full blob encoding and consistency check Jul 24, 2024
@roynalnaruto roynalnaruto self-requested a review July 24, 2024 23:33
roynalnaruto
roynalnaruto reviewed Jul 25, 2024
View reviewed changes
Copy link

@roynalnaruto roynalnaruto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make use of constants and some nit-picks

@roynalnaruto roynalnaruto self-requested a review July 25, 2024 07:49
roynalnaruto
roynalnaruto approved these changes Jul 25, 2024
View reviewed changes
Copy link

@roynalnaruto roynalnaruto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@roynalnaruto roynalnaruto merged commit 54125eb into develop Jul 25, 2024
17 checks passed
@roynalnaruto roynalnaruto deleted the test/full_blob_guard branch July 25, 2024 16:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@roynalnaruto roynalnaruto roynalnaruto approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@darth-cy @roynalnaruto