Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update adapters and converters #440

Merged
merged 191 commits into from
Mar 31, 2025
Merged
Changes from 1 commit
Commits
Show all changes
191 commits
Select commit Hold shift + click to select a range
fbbfc57
Fix weight converters and return their corresponding v5 weight descr
thodkatz Aug 15, 2024
a37b568
Create an interface for weight conversion
thodkatz Aug 15, 2024
db891eb
fix import_callable annotation
FynnBe Dec 2, 2024
a391d94
improve error traceback for single weights format attempt
FynnBe Dec 2, 2024
4103b51
add load_state
FynnBe Dec 2, 2024
7ec7afb
update ONNXModelAdapter
FynnBe Dec 2, 2024
ed8f1db
update TorchscriptModelAdapter typing
FynnBe Dec 2, 2024
9ae626d
update unzipping in tensorflow model adapter
FynnBe Dec 3, 2024
fceed3c
add upper bounds to dependencies
FynnBe Dec 3, 2024
77e1e84
update dev envs
FynnBe Dec 3, 2024
a131369
WIP setup run expensive tests
FynnBe Dec 3, 2024
0888c52
WIP resource tests
FynnBe Dec 3, 2024
40dfe25
expose sha256 arg
FynnBe Dec 3, 2024
bb539d4
update torchscript adapter
FynnBe Dec 4, 2024
fedd43c
bump spec lib version
FynnBe Dec 4, 2024
b7d5f98
Merge remote-tracking branch 'thodkatz/weight-converters' into fix_to…
FynnBe Dec 4, 2024
7f6fdf1
WIP refactor backend libs
FynnBe Dec 5, 2024
7fea808
add summary_path arg
FynnBe Dec 5, 2024
4564f7c
update annotation
FynnBe Dec 5, 2024
00e6ba1
fix tf seeding
FynnBe Dec 5, 2024
5d1e2ce
expose test_description_in_conda_env
FynnBe Dec 5, 2024
df36d15
docstring formatting
FynnBe Dec 5, 2024
376507f
absorb test_description_in_conda_env into test_description
FynnBe Dec 6, 2024
a8a50ec
Merge branch 'install_conda_envs' into fix_torch_load
FynnBe Dec 6, 2024
9690574
all model adapters in backends
FynnBe Dec 6, 2024
f52a894
sort tests
FynnBe Dec 6, 2024
523c54b
add create_model_adapter
FynnBe Dec 9, 2024
d438a12
pin pyright
FynnBe Dec 9, 2024
f9a1a67
continue refactor of weight converters and backends
FynnBe Dec 9, 2024
80f9ed0
update test_weight_converters.py
FynnBe Dec 9, 2024
dad8186
add test_bioimageio_collection.py
FynnBe Dec 9, 2024
e835432
add onnx as dev dep
FynnBe Dec 9, 2024
459696d
add get_pre_and_postprocessing
FynnBe Dec 10, 2024
169cf17
use dim instead of deprecated dims arg name
FynnBe Dec 10, 2024
7d8e7fc
update tests
FynnBe Dec 11, 2024
8b2727e
add todo
FynnBe Dec 11, 2024
02252ac
udpate pytorch_to_onnx converter
FynnBe Dec 11, 2024
ab8616f
expose determinism to cli test command
FynnBe Dec 18, 2024
f11b428
WIP unify model adapters
FynnBe Dec 18, 2024
e5bbe7a
fix TorchscriptModelAdapter
FynnBe Dec 19, 2024
3720e85
update predict_sample_without_blocking
FynnBe Dec 19, 2024
76c27e9
ensure batch and channel axes have standardized id
FynnBe Dec 19, 2024
4cbfc5a
support validation context 'raise_errors'
FynnBe Dec 20, 2024
447409f
fix ONNXModelAdapter
FynnBe Dec 20, 2024
3b514f8
_get_axis_type ->_guess_axis_type
FynnBe Dec 20, 2024
b264331
fix get_axes_infos
FynnBe Dec 20, 2024
84f24fe
bump pyright version
FynnBe Dec 20, 2024
27ea9aa
add test cases
FynnBe Dec 20, 2024
de759d5
fix pip install with no-deps
FynnBe Jan 7, 2025
d5abcfc
don't ignore model.v0_4.ScaleLinearKwargs with axes not implemented
FynnBe Jan 10, 2025
c64ff1b
fix keras and tf backends
FynnBe Jan 13, 2025
852e8b1
use AxisBase instead
FynnBe Jan 14, 2025
4299a8c
remove TooFewDevicesException exception
FynnBe Jan 15, 2025
3009f5b
improve test_mean_var_std_calculator
FynnBe Jan 15, 2025
b6f84f9
annotate relative and absolute tolerance
FynnBe Jan 24, 2025
b09d389
add check_reproducibility
FynnBe Jan 24, 2025
e35735d
add imagecodecs dependency
FynnBe Jan 24, 2025
34d5478
improve increase_available_weight_formats
FynnBe Jan 24, 2025
cfa3713
make docstring test os independent
FynnBe Jan 27, 2025
b4e76d1
return Tensor instead of np.ndarray
FynnBe Jan 27, 2025
43438d9
append to conv report
FynnBe Jan 27, 2025
bf7494e
avoid loguru logger in conftest.py to avoid error messages by pytest-…
FynnBe Jan 27, 2025
c99c89a
add todo and docstrings
FynnBe Feb 3, 2025
8c61d69
update typing
FynnBe Feb 3, 2025
666e80c
fix stats for absence of batch axis
FynnBe Feb 3, 2025
0d01271
split test_mean_var_std_calculator
FynnBe Feb 3, 2025
489d04e
default_ns -> blocksize_paramter and update docstrings
FynnBe Feb 5, 2025
f970c6b
deprecate tolerance arguments
FynnBe Feb 5, 2025
c775546
sort Tensor methods and add argmax() and item()
FynnBe Feb 5, 2025
05af735
remove decimal argument
FynnBe Feb 5, 2025
0d97bf3
update conda test build
FynnBe Feb 6, 2025
d13067e
remove decimal cli arg
FynnBe Feb 6, 2025
d3599f3
fix mismatched elements per million
FynnBe Feb 6, 2025
c171ab0
fix axis id conversion
FynnBe Feb 7, 2025
8d76a81
add UpdateFormatCmd
FynnBe Feb 7, 2025
cb47733
fix load_torch_model
FynnBe Feb 7, 2025
6a6d6a2
add logging to increase_available_weight_formats
FynnBe Feb 7, 2025
ee310fa
bump polite-pig
FynnBe Feb 7, 2025
19e2092
do not use pytest-xdist for cleaner test logs
FynnBe Feb 7, 2025
6ec6451
use summary.display() for nicer test logs
FynnBe Feb 7, 2025
0d113c2
avoid unnecessary imports in enable_determinism
FynnBe Feb 7, 2025
01c0fbd
add default args to enable_determinism
FynnBe Feb 7, 2025
e5f0efd
rename UpdateFormatCmd path arg to output
FynnBe Feb 7, 2025
961503d
enable_determinsm for all tests
FynnBe Feb 7, 2025
9f0cf83
use common EXAMPLE_DESCRIPTIONS URL
FynnBe Feb 7, 2025
01eb0fd
rename IncreaseWeightFormatsCmd arg path to output
FynnBe Feb 7, 2025
cf6f9ee
improve weights converters
FynnBe Feb 20, 2025
e6bd24b
Separates pipeline build logic per version
Tomaz-Vieira Feb 20, 2025
73f4dba
Factors out common code from pipeline setup funcs
Tomaz-Vieira Feb 20, 2025
8919900
Apllies black
Tomaz-Vieira Feb 20, 2025
aa22fef
make loading pytorch weights safe
FynnBe Feb 21, 2025
adbb976
accept single tensor source
FynnBe Feb 21, 2025
2d0d189
make sure to perform io checks
FynnBe Feb 21, 2025
f6fff61
better logging in CLI
FynnBe Feb 24, 2025
d3787a5
image io with PermissiveFileSource
FynnBe Feb 24, 2025
f01ee3d
fix predict_many and improve save_sample
FynnBe Feb 24, 2025
bd784e0
remove markdown dep (needed in spec)
FynnBe Feb 24, 2025
03d4940
bump pyright
FynnBe Feb 24, 2025
fc3a03f
Merge remote-tracking branch 'Tomaz-Vieira/fix_v4_pre_and_post_proces…
FynnBe Feb 24, 2025
89d7466
replace get_proc_class with get_proc
FynnBe Feb 24, 2025
67fa607
use overload for 'latest'
FynnBe Feb 25, 2025
667eeac
ensure 0.4 models get float32 inputa
FynnBe Feb 25, 2025
ddf86fc
log block shape
FynnBe Feb 25, 2025
3298944
add shape attribute
FynnBe Feb 25, 2025
70fb627
add TODO exclude default values
FynnBe Feb 26, 2025
503c04b
add exclude_defaults arg
FynnBe Feb 26, 2025
ed6d158
add update-hashes command
FynnBe Feb 27, 2025
b652258
update unet2d_diff_output_shape example and add cli tests
FynnBe Feb 28, 2025
4affce2
log output path
FynnBe Mar 3, 2025
e97686c
unify cmd naming pattern
FynnBe Mar 3, 2025
a2fefeb
limit converters to latest format
FynnBe Mar 3, 2025
5185936
update tests
FynnBe Mar 3, 2025
c310947
alaways zip converted tf weights
FynnBe Mar 3, 2025
5563e92
add epsilon
FynnBe Mar 3, 2025
fe15700
move test_model imports
FynnBe Mar 3, 2025
165e9e8
add segment-anything for testing
FynnBe Mar 4, 2025
a436beb
ignore bioimageio_unzipped_tf_weights/
FynnBe Mar 4, 2025
3865b39
use remembered file_name for prettier context
FynnBe Mar 4, 2025
4d4f3d3
skip known failure
FynnBe Mar 4, 2025
060603d
default to an absolute_tolerance for legacy models too
FynnBe Mar 4, 2025
570a7ff
ignore dogfood/
FynnBe Mar 4, 2025
55ef47e
add timm for model testing
FynnBe Mar 4, 2025
36a0812
return tested model descr
FynnBe Mar 4, 2025
b74686e
check legacy test kwargs for weight format specific tolerance
FynnBe Mar 10, 2025
c88c73e
mark known invalid resource descriptions
FynnBe Mar 13, 2025
db0f0da
load from zip for images/tensors
FynnBe Mar 13, 2025
9ad111b
improve update commands
FynnBe Mar 13, 2025
24797e6
show ruff fixes during precommit
FynnBe Mar 13, 2025
97929a6
add cellpose for model testing
FynnBe Mar 13, 2025
ed3e38d
bump spec version
FynnBe Mar 13, 2025
af396aa
various fixes
FynnBe Mar 17, 2025
8a06149
import architecture without explicitly extracting the package
FynnBe Mar 17, 2025
bacfd76
fix type annotation
FynnBe Mar 18, 2025
b847ced
improve onnx converters
FynnBe Mar 18, 2025
aa1079b
fix UpdateCmdBase.run
FynnBe Mar 18, 2025
a1cd381
add monai for model testing
FynnBe Mar 19, 2025
02883c1
update dev env
FynnBe Mar 19, 2025
312440a
always use AxisId that now includes axis id normalization
FynnBe Mar 20, 2025
e16a9ca
only run parameterized tests if tests passed so far
FynnBe Mar 22, 2025
014bc92
rename CLI argument weight_format -> weight-format (also accept weigh…
FynnBe Mar 22, 2025
0c35b86
WIP add stop_early arg
FynnBe Mar 23, 2025
d1a27cd
improve error messages
FynnBe Mar 24, 2025
f22438f
call validate_sha256 directly
FynnBe Mar 25, 2025
545ac54
improve test error handling
FynnBe Mar 25, 2025
a255f25
ValidationSummary.log -> save
FynnBe Mar 25, 2025
5402b6a
CLI render -> display
FynnBe Mar 25, 2025
c0ff94b
fix traceback for custom module from zipfile
FynnBe Mar 26, 2025
bf5b247
adapt for new status valid-format
FynnBe Mar 26, 2025
f10fcc4
WIP fix tests
FynnBe Mar 26, 2025
dbefc67
use get_validation_context
FynnBe Mar 26, 2025
1f741d9
add underscore weight-format aliases
FynnBe Mar 26, 2025
d8eb5dc
add source default
FynnBe Mar 26, 2025
25ff832
add sections to console output for update commands
FynnBe Mar 26, 2025
339d216
set default exclude_defaults=True for update-format
FynnBe Mar 26, 2025
ecd2aab
WIP fix tests
FynnBe Mar 26, 2025
3b23c7c
bump spec version
FynnBe Mar 26, 2025
300af37
fix tests
FynnBe Mar 27, 2025
7c5b528
update dev envs
FynnBe Mar 27, 2025
5d68384
update build workflow
FynnBe Mar 27, 2025
0464851
merge conda and main test jobs
FynnBe Mar 27, 2025
621b2db
merge conda and main test jobs
FynnBe Mar 27, 2025
6411e6f
remove tf env (tf is in env-full)
FynnBe Mar 27, 2025
e48fbda
search for env-file
FynnBe Mar 27, 2025
58dcc92
fix env-py38.yaml
FynnBe Mar 27, 2025
f050601
add tests.__init__.py for tests.utils
FynnBe Mar 27, 2025
e8286e2
only save bioimageio_cache once
FynnBe Mar 27, 2025
f68edf5
puplulate cache before testing
FynnBe Mar 27, 2025
6e2cd00
actually write to the cache!
FynnBe Mar 27, 2025
f1bf29b
fix populating cache
FynnBe Mar 27, 2025
a7dc02d
point pyright to pyproject.toml
FynnBe Mar 27, 2025
e427e40
use cache/restore for cache look-up
FynnBe Mar 27, 2025
5fefb84
fix env cache key
FynnBe Mar 27, 2025
1fdb9e8
pyright fixes
FynnBe Mar 27, 2025
953c393
add bioimageio_cache
FynnBe Mar 27, 2025
6582ca0
limit pytest testfolders
FynnBe Mar 27, 2025
659c395
save env cache earlier
FynnBe Mar 27, 2025
f3af766
ignore docstring tests in backends for their optional dependencies
FynnBe Mar 28, 2025
f21ebed
fix step names
FynnBe Mar 28, 2025
b21deb1
ignore docstring tests in weight_converters for their optional depend…
FynnBe Mar 28, 2025
fec762e
try to ignore more explicitly
FynnBe Mar 28, 2025
677298c
add conda list step
FynnBe Mar 28, 2025
fab4c7f
use long options instead
FynnBe Mar 28, 2025
33fb00c
pin xarray
FynnBe Mar 28, 2025
5bb0d6f
Merge remote-tracking branch 'origin/main' into fix_torch_load
FynnBe Mar 28, 2025
820e3c2
improve caching
FynnBe Mar 28, 2025
51f163b
xarray 2023 compatibility
FynnBe Mar 28, 2025
6f78383
test_zero_mean_unit_variance_fixed
FynnBe Mar 28, 2025
e2596d1
name pyright step
FynnBe Mar 28, 2025
6a248ca
remove default for source: CliPositionalArg
FynnBe Mar 28, 2025
2ef774f
install uncached pip deps
FynnBe Mar 28, 2025
7e9888a
display validation summary on test failure
FynnBe Mar 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add summary_path arg
FynnBe committed Dec 5, 2024
commit 7fea808f0570cfec4b74f36efb2835ab18bc3e9c
4 changes: 4 additions & 0 deletions bioimageio/core/cli.py
Original file line number Diff line number Diff line change
@@ -133,13 +133,17 @@ class TestCmd(CmdBase, WithSource):
decimal: int = 4
"""Precision for numerical comparisons"""

summary_path: Optional[Path] = None
"""Path to save validation summary as JSON file."""

def run(self):
sys.exit(
test(
self.descr,
weight_format=self.weight_format,
devices=self.devices,
decimal=self.decimal,
summary_path=self.summary_path,
)
)

6 changes: 6 additions & 0 deletions bioimageio/core/commands.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""These functions implement the logic of the bioimageio command line interface
defined in `bioimageio.core.cli`."""

import json
from pathlib import Path
from typing import Optional, Sequence, Union

@@ -26,6 +27,7 @@ def test(
weight_format: WeightFormatArgAll = "all",
devices: Optional[Union[str, Sequence[str]]] = None,
decimal: int = 4,
summary_path: Optional[Path] = None,
) -> int:
"""test a bioimageio resource
@@ -35,6 +37,7 @@ def test(
weight_format: (model only) The weight format to use
devices: Device(s) to use for testing
decimal: Precision for numerical comparisons
summary_path: Path to save validation summary as JSON file.
"""
if isinstance(descr, InvalidDescr):
descr.validation_summary.display()
@@ -47,6 +50,9 @@ def test(
decimal=decimal,
)
summary.display()
if summary_path is not None:
_ = summary_path.write_text(summary.model_dump_json(indent=4))

return 0 if summary.status == "passed" else 1