Skip to content

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #3035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #3035

wants to merge 31 commits into from

Conversation

@rw1nkler
Copy link
Contributor

@rw1nkler rw1nkler commented Sep 10, 2025 •
edited
Loading

Added functionality:

  • multiple testcases covering the decoding of frames with "compressed" blocks of different type
  • cocotb testing utilities for common operations on XLS channels
  • collection of PnR benchmarks
  • ZSTD frame generator
  • tests of axi ram writer/reader procs as well as memreader & memwriter

Supersedes #2717

@rw1nkler rw1nkler mentioned this pull request Sep 10, 2025
Closed
@dplassgit dplassgit requested a review from proppy September 10, 2025 18:55
@rw1nkler
Copy link
Contributor Author

rw1nkler commented Sep 16, 2025

@proppy Please let us know if there’s anything else we can do to move this forward

proppy
proppy reviewed Sep 16, 2025
View reviewed changes
dependency_support/com_github_alexforencich_verilog_axi/BUILD.bazel
@@ -0,0 +1,15 @@
# Copyright 2025 The XLS Authors
Copy link
Member

@proppy proppy Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is a precedent for our project to import SystemVerilog third_party dependency, I'll have to figure out how to make this work.

Copy link
Member

@proppy proppy Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we discussed, it might be worth exploring landing the verilog dependency in a separate PR (and merge first the cocotb tests that don't depend on it)

rw1nkler and others added 26 commits September 19, 2025 15:38
@rw1nkler
Add a separate test case for each test file
2dca4ce 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @rw1nkler
Add "pytype binary, library" hint
53571ca
@koblonczek @rw1nkler
Add cocotb tests
af3b51a 
Add cocotb testing utilities:
- XLSStruct for easier handling and serializing/deserializing XLS structs
- XLSChannel that serves as a dummy receiving channel
- XLSMonitor that monitors transactions on an XLS channel
- XLSDriver that can send data on an XLS channel
- LatencyScoreboard that can measure latency between corresponding transactions on input and output buses
- File-backed AXI memory python model

Add cocotb tests for:
- modules/zstd/memory/MemReader
- modules/zstd/memory/AxiWriter
- modules/zstd/memory/MemWriter

Co-authred-by: Pawel Czarnecki <[email protected]>
Co-authred-by: Robert Winkler <[email protected]>
Co-authred-by: Michal Czyz <[email protected]>
Signed-off-by: Michal Czyz <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @rw1nkler
xls/modules/zstd: expose fifo verilog module
11a248d 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/cocotb: Add ZSTD frame generator library
e0856b5 
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
@lpawelcz @rw1nkler
dependency_support: Add zstandard python library
a0a309f 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd: Add verilog simulation of the ZstdDecoder
bffd3a3 
Signed-off-by: Pawel Czarnecki <[email protected]>
@koblonczek @rw1nkler
Use frame generator in tests
de73006 
Co-authored-by: Robert Winkler <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec: Write decoded data to the memory
5b6d804 
ZstdDecoder:
  * Cocotb tests:
    * Move third-party verilog modules (AXI Interconnect) to external directory
    * Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read
      and Write transactions
    * Add reference memory and fill it with expected data for comparison
      against testbench memory at the end of the decoding

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec: Remove stream-based output interface
03aef2a 
Adjust cocotb test to:
* Removal of Repacketizer proc
* Removal of stream-based output channels from
  * SequenceExecutor
  * ZstdDecoder

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec_cocotb_test: Improve Verilog simulation
9adb781 
* Decode multiple ZSTD frames in a single cocotb testbench
* Add one cocotb testbench per type of the ZSTD frames:
  * Frames with RAW blocks only
  * Frames with RLE blocks only
  * Frames with Compressed blocks only (disabled)
  * Frames with mixed blocks (disabled)

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/memory/mem_writer: Reduce the amount of random test cases
eaeb6ed 
Caused by timeouts after rebase - looks like regression in the
performance of codegen

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/memory: Move verilog sources to rtl subdirectory
fa09c1f 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
xls/modules/zstd/external: Add docstring and license for third party …
e3e64bd 
…sources

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd: Adjust cocotb test to removal of RESET CSR
bcaf1c5 
Signed-off-by: Pawel Czarnecki <[email protected]>
@wsipak @lpawelcz @rw1nkler
Style changes
7153a6a 
Improve formatting, wording and fix lint issues

Co-authored-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Wojciech Sipak <[email protected]>
@rw1nkler
Extend cocotb tests of ZSTD decoder
aa24e64 
Co-authored-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Krzysztof Oblonczek <[email protected]>
Co-authored-by: Wojciech Sipak <[email protected]>
Co-authored-by: Dominik Lau <[email protected]>
Co-authored-by: Szymon Gizler <[email protected]>

Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Collect P&R benchmarks
be24b01 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Add fixed version of RamDemux together with a test
5e90234 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @rw1nkler
Document decoder testing
49116aa
@rw1nkler
dependency_support: Add verilog_axi package
8cc9a8e 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Use bazel package with verilog_axi
e27ac0b 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Downgrade dependencies
3392904 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Run tests selectively in GitHub CI
e860acf 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Remove zstandard python library
978cb54 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: update readme
aac3756 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Add cocotb check for verifying huffman codes
a914314 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: cocotb: drop unused testcase
1a5df3b 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: disable huffman code builder PnR
0393f5b 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: fix formatting
86d2c87 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Update python lock file
acf0ffd 
Signed-off-by: Robert Winkler <[email protected]>
@proppy
Copy link
Member

proppy commented Oct 10, 2025

Marking this as draft until it's ready to review again.

@proppy proppy marked this pull request as draft October 10, 2025 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@proppy proppy proppy left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@rw1nkler @proppy @sgizler @koblonczek @lpawelcz @wsipak