Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] PathProperty -p repartion for openvino backbone and layer #2937

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

piotrrak
Copy link

@piotrrak piotrrak commented Feb 11, 2025

DR Description

This PR is preparation for OpenVINO backend.

  • adds PathProperty that represents filesystem path
  • adds ModelPathProperty that facilitates checking if path points to file with given set of the file extensions.

The ModelPathProperty generalizes nntrainer::PropsTflModelPath and will be used by OpenVINO model path property.
Since OpenVINO has multiple frontends there is need to filter paths with multiple possible extensions.
In addition this allows to switch Property<std::string> to PathProperty and use c++17 std::filesystem library facilities for properties that are paths.

OpenVINO frontends are:

  • TensorflowLite
  • PyTorch
  • PaddlePaddle
  • ONNX
  • JAX

Self evaluation:

  1. Build test: [ ]Passed [ ]Failed [*]Skipped
  2. Run test: [ ]Passed [ ]Failed [*]Skipped

How to evaluate:

  1. Describe how to evaluate in order to be reproduced by reviewer(s). TODO

Add signed-off message automatically by running $git commit -s ... command.

$ git push origin <your_branch_name>


It is common to have an configuration property that includes paths,
introduce new property kind; Its implementation is based
on portable <filesystem> library.
This frees us from path parsing, handing in cross-platform manner.
Also it provides basic file operations, like creating directoris.

Signed-off-by: Piotr Rak <[email protected]>
Specialized helper for file model path.
Its main purpouse is to ensure file path meets criterions of
given type model files.

It covers PropsTflModelPath (file readability and correct file extension),
while extending it to set of possible file extensions.

Signed-off-by: Piotr Rak <[email protected]>
piotrrak and others added 7 commits February 12, 2025 01:42
This allows to build test-suite w/o tensorflow-lite.
Probably can be dropped when we'll stop using python-env provided
OpenVINO.
Adds sanity tests for path property.
This avoids opening file stream.

Signed-off-by: Piotr Rak <[email protected]>
That should be cheap enough using stat() or alike underneath.

Signed-off-by: Piotr Rak <[email protected]>
This had potential to overflow for files larger than 4GB (2^32)B
This days it is bit small.

This change is NOT the real fix of this potential issue, but makes it
(sample_size * RawFileDataProducer::pixel_size) times less likely.

The return type of this function still holds just 2^32-1.

Signed-off-by: Piotr Rak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants