Skip to content

Commit ce4ec62

Browse files
pmaipmai
committed
Initial refactoring to use sane OSITrace reader
Signed-off-by: Pierre R. Mai <[email protected]>
1 parent 7fd9311 commit ce4ec62

File tree

9 files changed

+63
-638
lines changed

9 files changed

+63
-638
lines changed

Diff for: ‎.github/workflows/ci.yml

+2-26
Original file line numberDiff line numberDiff line change
@@ -48,42 +48,20 @@ jobs:
4848
- name: Checkout
4949
uses: actions/checkout@v4
5050
with:
51+
submodules: recursive
5152
lfs: true
5253

5354
- name: Set up Python ${{ matrix.python-version }}
5455
uses: actions/setup-python@v5
5556
with:
5657
python-version: ${{ matrix.python-version }}
5758

58-
- name: Cache Dependencies
59-
id: cache-depends
60-
uses: actions/cache@v3
61-
with:
62-
path: protobuf-3.20.1
63-
key: ${{ runner.os }}-v2-depends
64-
65-
- name: Download ProtoBuf
66-
if: steps.cache-depends.outputs.cache-hit != 'true'
67-
run: curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.20.1/protobuf-all-3.20.1.tar.gz && tar xzvf protobuf-all-3.20.1.tar.gz
68-
69-
- name: Build ProtoBuf
70-
if: steps.cache-depends.outputs.cache-hit != 'true'
71-
working-directory: protobuf-3.20.1
72-
run: ./configure DIST_LANG=cpp --prefix=/usr && make
73-
74-
- name: Install ProtoBuf
75-
working-directory: protobuf-3.20.1
76-
run: sudo make install && sudo ldconfig
77-
78-
- name: Install Open Simulation Interface
79-
shell: bash
59+
- name: Set up Virtual Environment
8060
run: |
81-
git submodule update --init
8261
python -m venv .venv
8362
source .venv/bin/activate
8463
python -m pip install --upgrade pip
8564
pip install -r requirements_develop.txt
86-
cd open-simulation-interface && pip install . && cd ..
8765
8866
- name: Generate parsed rules
8967
run: |
@@ -100,7 +78,5 @@ jobs:
10078
run: |
10179
source .venv/bin/activate
10280
pip install .
103-
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.txt -r rules
104-
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.txt -r rules --parallel
10581
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.osi -r rules
10682
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.osi -r rules --parallel

Diff for: ‎README.md

+7-7
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The full documentation on the validator and customization of the rules is availa
1111
## Usage
1212

1313
```bash
14-
usage: osivalidator [-h] [--data DATA] [--rules RULES] [--type {SensorView,GroundTruth,SensorData}] [--output OUTPUT] [--timesteps TIMESTEPS] [--debug] [--verbose] [--parallel] [--format {separated,None}]
14+
usage: osivalidator [-h] --data DATA [--rules RULES] [--type {SensorView,GroundTruth,SensorData}] [--output OUTPUT] [--timesteps TIMESTEPS] [--debug] [--verbose] [--parallel] [--format {None}]
1515
[--blast BLAST] [--buffer BUFFER]
1616

1717
Validate data defined at the input
@@ -29,13 +29,13 @@ optional arguments:
2929
Number of timesteps to analyze. If -1, all.
3030
--debug Set the debug mode to ON.
3131
--verbose, -v Set the verbose mode to ON.
32-
--parallel, -p Set parallel mode to ON.
33-
--format {separated,None}, -f {separated,None}
34-
Set the format type of the trace.
32+
--parallel, -p (Ignored) Set parallel mode to ON.
33+
--format {None}, -f {None}
34+
(Ignored) Set the format type of the trace.
3535
--blast BLAST, -bl BLAST
36-
Set the in-memory storage count of OSI messages during validation.
36+
Set the maximum in-memory storage count of OSI messages during validation.
3737
--buffer BUFFER, -bu BUFFER
38-
Set the buffer size to retrieve OSI messages from trace file. Set it to 0 if you do not want to use buffering at all.
38+
(Ignored) Set the buffer size to retrieve OSI messages from trace file. Set it to 0 if you do not want to use buffering at all.
3939
```
4040
4141
## Installation
@@ -80,5 +80,5 @@ $ source .venv/Scripts/activate
8080
## Example command
8181
8282
```bash
83-
$ osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.txt --rules rules/
83+
$ osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.osi --rules rules/
8484
```

Diff for: ‎data/20210818T150542Z_sv_312_50_one_moving_object.txt

-6.04 KB
Binary file not shown.

Diff for: ‎doc/usage.adoc

+15-34
Original file line numberDiff line numberDiff line change
@@ -7,65 +7,46 @@ terminal which has the following usage:
77

88
[source,bash]
99
----
10-
usage: osivalidator [-h] [--data DATA] [--rules RULES] [--type {SensorView,GroundTruth,SensorData}] [--output OUTPUT] [--timesteps TIMESTEPS] [--debug] [--verbose] [--parallel] [--format {separated,None}]
10+
usage: osivalidator [-h] --data DATA [--rules RULES] [--type {SensorView,GroundTruth,SensorData}] [--output OUTPUT] [--timesteps TIMESTEPS] [--debug] [--verbose] [--parallel] [--format {None}]
1111
[--blast BLAST] [--buffer BUFFER]
1212
1313
Validate data defined at the input
1414
15+
mandatory arguments:
16+
--data DATA Path to the file with OSI-serialized data.
17+
1518
optional arguments:
1619
-h, --help show this help message and exit
17-
--data DATA Path to the file with OSI-serialized data.
1820
--rules RULES, -r RULES
19-
Directory with text files containig rules.
21+
Directory with text files containig rules.
2022
--type {SensorView,GroundTruth,SensorData}, -t {SensorView,GroundTruth,SensorData}
21-
Name of the type used to serialize data.
23+
Name of the type used to serialize data.
2224
--output OUTPUT, -o OUTPUT
23-
Output folder of the log files.
25+
Output folder of the log files.
2426
--timesteps TIMESTEPS
25-
Number of timesteps to analyze. If -1, all.
27+
Number of timesteps to analyze. If -1, all.
2628
--debug Set the debug mode to ON.
2729
--verbose, -v Set the verbose mode to ON.
28-
--parallel, -p Set parallel mode to ON.
29-
--format {separated,None}, -f {separated,None}
30-
Set the format type of the trace.
30+
--parallel, -p (Ignored) Set parallel mode to ON.
31+
--format {None}, -f {None}
32+
(Ignored) Set the format type of the trace.
3133
--blast BLAST, -bl BLAST
32-
Set the in-memory storage count of OSI messages during validation.
34+
Set the maximum in-memory storage count of OSI messages during validation.
3335
--buffer BUFFER, -bu BUFFER
34-
Set the buffer size to retrieve OSI messages from trace file. Set it to 0 if you do not want to use buffering at all.
36+
(Ignored) Set the buffer size to retrieve OSI messages from trace file. Set it to 0 if you do not want to use buffering at all.
3537
----
3638

3739
To run the validation first you need an OSI trace file which consists of
3840
multiple OSI messages. In the directory `+data+` of the repository we
3941
already provide an example trace file which is called
40-
`+20210818T150542Z_sv_312_50_one_moving_object.txt+`. Use the
41-
https://github.com/OpenSimulationInterface/open-simulation-interface/blob/master/format/txt2osi.py[txt2osi.py]
42-
of the OSI repo in the format directory to convert from `+*.txt+` to
43-
`+*.osi+` files. See usage below:
44-
45-
[source,bash]
46-
----
47-
usage: txt2osi converter [-h] [--file FILE]
48-
[--type {SensorView,GroundTruth,SensorData}]
49-
[--output OUTPUT] [--compress]
50-
51-
Convert txt trace file to osi trace files.
52-
53-
optional arguments:
54-
-h, --help show this help message and exit
55-
--file FILE, -f FILE Path to the file with serialized data.
56-
--type {SensorView,GroundTruth,SensorData}, -t {SensorView,GroundTruth,SensorData}
57-
Name of the type used to serialize data.
58-
--output OUTPUT, -o OUTPUT
59-
Output name of the file.
60-
----
42+
`+20210818T150542Z_sv_312_50_one_moving_object.osi+`.
6143

6244
To validate the trace files you simply call `+osivalidator+` and provide
6345
the path to the trace:
6446

6547
[source,bash]
6648
----
6749
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.osi
68-
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.txt
6950
----
7051

7152
You can also validate the traces in parallel to increase the speed of
@@ -84,7 +65,7 @@ need to generate them and then specify them:
8465
[source,bash]
8566
----
8667
python rules2yml.py # Generates the rule directory
87-
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.txt --rules rules/ -p
68+
osivalidator --data data/20210818T150542Z_sv_312_50_one_moving_object.osi --rules rules -p
8869
----
8970

9071
After successfully running the validation the following output is

0 commit comments

Comments
 (0)