Update all entries to support of `entry_creator`

[maria-18-git]

All axs entries should use entry_creator for object creation.
Now different types of save are used for it.
For example, set_path(entry_name setting) is used in case of record_entry.

First entries without dependencies on others should be updated and then it should be tested.
Also we need to check whether we can move pipeline parameter , e.g.

    "pipeline": [ "^^", "execute", [[
        [ "run" ],
        [],
        [ "get", "stored_newborn_entry" ]
    ]] ],

to entry_creator or not.

[maria-18-git]

Updated loadgen_wheel_recipe

on chai:

 axs byquery package_wheel,package_name=mlperf_loadgen
['^', 'byname', 'loadgen_package_wheel_for_python3.9']

Structure of created dicrectory:

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ tree -a `axs byquery package_wheel,package_name=mlperf_loadgen , get_path`
/home/maria/work_collection/loadgen_package_wheel_for_python3.9
├── data_axs.json
├── mlperf_loadgen-4.0-cp39-cp39-linux_x86_64.whl
└── mlperf_loadgen.egg-info
    ├── dependency_links.txt
    ├── PKG-INFO
    ├── SOURCES.txt
    └── top_level.txt

1 directory, 6 files

axs byquery python_package,package_name=mlperf_loadgen,desired_python_version===3.9
...
        "/home/maria/bin/python3" -m pip install /home/maria/work_collection/loadgen_package_wheel_for_python3.9/mlperf_loadgen-4.0-cp39-cp39-linux_x86_64.whl --prefix=/home/maria/work_collection/mlperf_loadgen_package_for_python3.9/install --ignore-installed
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /home/maria/work_collection/loadgen_package_wheel_for_python3.9/mlperf_loadgen-4.0-cp39-cp39-linux_x86_64.whl                                                                         DEPRECATION: devscripts 2.22.1ubuntu1 has a non-standard version number. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of devscripts or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063
Installing collected packages: mlperf-loadgen
Successfully installed mlperf-loadgen-4.0

[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: python3 -m pip install --upgrade pip
INFO:root:Matched Rule #1/2 produced an entry, which matches the original query.

['^', 'byname', 'mlperf_loadgen_package_for_python3.9']

Structure of directory:

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ tree -a /home/maria/work_collection/mlperf_loadgen_package_for_python3.9/                                              /home/maria/work_collection/mlperf_loadgen_package_for_python3.9/
├── data_axs.json
└── install
    └── lib
        └── python3.9
            └── site-packages
                ├── mlperf_loadgen-4.0.dist-info
                │   ├── direct_url.json
                │   ├── INSTALLER
                │   ├── METADATA
                │   ├── RECORD
                │   ├── REQUESTED
                │   ├── top_level.txt
                │   └── WHEEL
                └── mlperf_loadgen.cpython-39-x86_64-linux-gnu.so

5 directories, 9 files

[maria-18-git]

Check correct work with mlperf_loadgen_package_for_python3.9:

• bert

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ axs byquery loadgen_output,task=bert,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,execution_device=cpu
['^', 'byname', 'generated_by_bert_using_onnxrt_loadgen_on_get_14f91579f37040deafe3c11f1dcbdbca']

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ export ACCURACY_OUTPUT=`axs byquery loadgen_output,task=bert,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,execution_device=cpu , get accuracy_dict` 

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ echo "Accuracy: $ACCURACY_OUTPUT"
Accuracy: {'exact_match': 85.0, 'f1': 85.0}

[maria-18-git]

• Object detection(ssd_resnet34)

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ axs byquery loadgen_output,task=object_detection,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=ssd_resnet34,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu
['^', 'byname', 'generated_by_object_detection_using_onnxrt_loadgen_on_get_1ee92880af2a4eaa82f117dc1c19effe']

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ export ACCURACY_OUTPUT=`axs byquery loadgen_output,task=object_detection,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=ssd_resnet34,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu , get accuracy` 

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$  echo "Accuracy: $ACCURACY_OUTPUT"
Accuracy: 22.852

maria@chai ~/work_collection/axs2mlperf/loadgen_wheel_recipe (master *=)$ echo `axs func round $ACCURACY_OUTPUT 0`
23.0

[maria-18-git]

• Object detection(retinanet, openimages)

maria@chai ~/work_collection/axs2mlperf (master *=)$ axs byquery loadgen_output,task=object_detection,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=retinanet_openimages,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu
['^', 'byname', 'generated_by_object_detection_using_onnxrt_loadgen_on_get_3e468e916d9c446e980202a79ede34af']

maria@chai ~/work_collection/axs2mlperf (master *=)$ export ACCURACY_OUTPUT=`axs byquery loadgen_output,task=object_detection,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=retinanet_openimages,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu , get accuracy` 

maria@chai ~/work_collection/axs2mlperf (master *=)$ echo "Accuracy: $ACCURACY_OUTPUT"
Accuracy: 52.98

[maria-18-git]

• image classification (onnxrt, resnet50)

maria@chai ~/work_collection/axs2mlperf (master *=)$ axs byquery loadgen_output,task=image_classification,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=resnet50,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu
['^', 'byname', 'generated_by_image_classification_using_onnxrt_loadgen_on_get_2f186ef2c94d44a8a92dfb544c7f04f1']

maria@chai ~/work_collection/axs2mlperf (master *=)$ export ACCURACY_OUTPUT=`axs byquery loadgen_output,task=image_classification,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=resnet50,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu , get accuracy `  

maria@chai ~/work_collection/axs2mlperf (master *=)$  echo "Accuracy: $ACCURACY_OUTPUT"
Accuracy: 85.0

[maria-18-git]

• image classification (pytorch, resnet50)

maria@chai ~/work_collection/axs2mlperf (master *=)$ axs byquery loadgen_output,task=image_classification,framework=pytorch,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=resnet50,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu
['^', 'byname', 'generated_by_image_classification_using_torch_loadgen_on_get_2e326755c3b54c468b072d331639386d']

maria@chai ~/work_collection/axs2mlperf (master *=)$ export ACCURACY_OUTPUT=`axs byquery loadgen_output,task=image_classification,framework=pytorch,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,model_name=resnet50,loadgen_dataset_size=20,loadgen_buffer_size=100,execution_device=cpu , get accuracy` 

maria@chai ~/work_collection/axs2mlperf (master *=)$ echo "Accuracy: $ACCURACY_OUTPUT"
Accuracy: 75.0

[maria-18-git]

Install mlperf_loadgen_package_for_python3.9 automatically

• Remove packages:

 axs byquery package_wheel,package_name=mlperf_loadgen --- , remove

axs byquery python_package,package_name=mlperf_loadgen,desired_python_version===3.9 --- , remove

• Run experiments:
  bert:

maria@chai ~/work_collection/axs2mlperf (master *=)$ axs byquery loadgen_output,task=bert,framework=onnxrt,loadgen_scenario=Offline,loadgen_mode=AccuracyOnly,execution_device=cpu
...
INFO:root:[work_collection] byquery(package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9) did not find anything, but there are tags: {'package_wheel'} , trying to find a producer...
INFO:root:[work_collection] A total of 1 matched rules found.

INFO:root:Matched Rule #1/1: ['package_wheel', 'package_name=mlperf_loadgen'] from Entry 'loadgen_wheel_recipe'...                                                                               INFO:root:Pipeline: [['get', 'pipeline']], Cumulative params: {'__query': 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9', 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['package_wheel'], '__cumulative_param_names': ['__query', 'package_name', 'desired_python_version', 'tags']}
INFO:root:[work_collection] byquery(package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9) did not find anything, but there are tags: {'package_wheel'} , trying to find a producer...                                                                                                                                                                                           INFO:root:[work_collection] A total of 1 matched rules found.

INFO:root:Matched Rule #1/1: ['package_wheel', 'package_name=mlperf_loadgen'] from Entry 'loadgen_wheel_recipe'...                                                                               INFO:root:Pipeline: [['get', 'pipeline']], Cumulative params: {'__query': 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9', 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['package_wheel'], '__cumulative_param_names': ['__query', 'package_name', 'desired_python_version', 'tags']}
WARNING:root:[loadgen_wheel_recipe -> loadgen_wheel_recipe] parameter 'pipeline' is contained here, but BLOCKED by this entry -- all blockers: {'loadgen_wheel_recipe'}
INFO:root:Matched Rule #1/1 didn't produce a result, 0 more matched rules to try...
                                                                                                                                                                                                 ------------------------------------------------------------------------------------------------------------------------                                                                         While computing nested_calls in ['^^', 'execute', [[['get_kernel'], ['byquery', ['^^', 'substitute', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===#{desired_python_version}#']], ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]]]] the following exception was raised: In pipeline [                                                            ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
        ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]                                                                                                        ] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                                                                                     ========================================================================================================================                                                                         ------------------------------------------------------------------------------------------------------------------------
While computing nested_calls in [[['get', 'collection_object'], ['attached_entry', [['^^', 'get', 'newborn_entry_name'], {'tags': ['^^', 'get', ['tags', ['^^', 'get', 'newborn_entry_tags']]]}]], ['plant', ['_parent_entries', ['^^', 'parent_list_from_names']]], ['plant', ['^^', 'slice', ['^^', 'get', ['__cumulative_param_names', []]], {'plantable': True, 'skip_missing': True}]], ['plant', ['^^', 'slice', ['^^', 'get', 'newborn_entry_param_names'], {'plantable': True, 'skip_missing': True}]], ['plant', ['^^', 'substitute', [['^^', 'get', 'newborn_entry_param_listmap']]]]]] the following exception was raised: In pipeline [
        ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                            ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]                                                                                                        ] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9'] 
...
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================
------------------------------------------------------------------------------------------------------------------------
While computing nested_calls in {'__query': ['python_package', 'package_name=mlperf_loadgen', ['desired_python_version', '3.9']], 'installable': ['^^', 'execute', [[['get_kernel'], ['byquery', ['^^', 'substitute', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===#{desired_python_version}#']], ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]]]], 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['python_package'], '__cumulative_param_names': ['__query', 'installable', 'package_name', 'desired_python_version', 'tags']} the following exception was raised: In pipeline [
        ['get_kernel']
        ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                            ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================
------------------------------------------------------------------------------------------------------------------------
While computing nested_calls in [['^^', 'python_sync_pip_package', [['^^', 'get', 'tensorflow_query']]], ['^^', 'python_sync_pip_package', [['^^', 'get', 'onnxruntime_query']]], ['^^', 'python_sync_pip_package', [['^^', 'get', 'loadgen_query']]]] the following exception was raised: In pipeline [
        ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                            ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================
------------------------------------------------------------------------------------------------------------------------
While computing nested_calls in ['^^', 'ext_use_python_deps'] the following exception was raised: In pipeline [                                                                                          ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
        ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================
------------------------------------------------------------------------------------------------------------------------
While computing nested_calls in ['^^', 'execute', [[['run'], [], ['get', 'output_entry']]]] the following exception was raised: In pipeline [                                                            ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
        ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================
ERROR:root:RuntimeError: In pipeline [
        ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                            ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
None

[maria-18-git]

axs byquery package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9 doesn't work correctly with entry_creator.

[maria-18-git]

If
axs byquery package_wheel,package_name=mlperf_loadgen --- , remove
then

axs byquery python_package,package_name=mlperf_loadgen,desired_python_version===3.9
INFO:root:[work_collection] byquery(python_package,package_name=mlperf_loadgen,desired_python_version===3.9) did not find anything, but there are tags: {'python_package'} , trying to find a producer...
INFO:root:[work_collection] A total of 2 matched rules found.                                                                                                                                    
INFO:root:Matched Rule #1/2: ['python_package', 'package_name=mlperf_loadgen', 'desired_python_version.'] from Entry 'loadgen_wheel_recipe'...                                                   INFO:root:Pipeline: [['get_kernel'], ['byname', 'pip'], ['install']], Cumulative params: {'__query': 'python_package,package_name=mlperf_loadgen,desired_python_version===3.9', 'installable': ['^^', 'execute', [[['get_kernel'], ['byquery', ['^^', 'substitute', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===#{desired_python_version}#']], ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]]]], 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['python_package'], '__cumulative_param_names': ['__query', 'installable', 'package_name', 'desired_python_version', 'tags']}
INFO:root:[work_collection] byquery(package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9) did not find anything, but there are tags: {'package_wheel'} , trying to find a producer...
INFO:root:[work_collection] A total of 1 matched rules found.

INFO:root:Matched Rule #1/1: ['package_wheel', 'package_name=mlperf_loadgen'] from Entry 'loadgen_wheel_recipe'...                                                                               INFO:root:Pipeline: [['get', 'pipeline']], Cumulative params: {'__query': 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9', 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['package_wheel'], '__cumulative_param_names': ['__query', 'package_name', 'desired_python_version', 'tags']}                                                      INFO:root:[work_collection] byquery(package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9) did not find anything, but there are tags: {'package_wheel'} , trying to find a producer...
INFO:root:[work_collection] A total of 1 matched rules found.                                                                                                                                    
INFO:root:Matched Rule #1/1: ['package_wheel', 'package_name=mlperf_loadgen'] from Entry 'loadgen_wheel_recipe'...                                                                               INFO:root:Pipeline: [['get', 'pipeline']], Cumulative params: {'__query': 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9', 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['package_wheel'], '__cumulative_param_names': ['__query', 'package_name', 'desired_python_version', 'tags']}                                                      WARNING:root:[loadgen_wheel_recipe -> loadgen_wheel_recipe] parameter 'pipeline' is contained here, but BLOCKED by this entry -- all blockers: {'loadgen_wheel_recipe'}                          INFO:root:Matched Rule #1/1 didn't produce a result, 0 more matched rules to try...

------------------------------------------------------------------------------------------------------------------------                                                                         While computing nested_calls in ['^^', 'execute', [[['get_kernel'], ['byquery', ['^^', 'substitute', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===#{desired_python_version}#']], ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]]]] the following exception was raised: In pipeline [
        ['get_kernel']                                                                                                                                                                                   ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
        ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
...
While computing nested_calls in {'__query': 'python_package,package_name=mlperf_loadgen,desired_python_version===3.9', 'installable': ['^^', 'execute', [[['get_kernel'], ['byquery', ['^^', 'substitute', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===#{desired_python_version}#']], ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]]]], 'package_name': 'mlperf_loadgen', 'desired_python_version': '3.9', 'tags': ['python_package'], '__cumulative_param_names': ['__query', 'installable', 'package_name', 'desired_python_version', 'tags']} the following exception was raised: In pipeline [
        ['get_kernel']
        ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']                                                                                                            ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]
] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
========================================================================================================================                                                                         ERROR:root:RuntimeError: In pipeline [
        ['get_kernel']
        ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
        ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}]                                                                                                        ] step ['find_file', 'mlperf_loadgen.*\\.whl', {'return_full': True, 'abs_paths': True}] cannot be executed on value (None) produced by ['byquery', 'package_wheel,package_name=mlperf_loadgen,desired_python_version===3.9']
None       

It is the same issue.

[maria-18-git]

It is an issue with entry_creator in loadgen_wheel_recipe/data_axs.json. We don't use entry_creator many times.