Skip to content

Commit 73d9d83

Browse files
jeanbezzhangwei217245houjunwangvsagithub-actions[bot]
authored
Synchronize stable with develop branch (#172)
* fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * Merge tools/llsm_importer (#81) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format --------- Co-authored-by: Houjun Tang <[email protected]> * added a tutorial for llsm_importer * added a tutorial for llsm_importer * Adding tutorial for llsm_importer tool. (#84) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer --------- Co-authored-by: Houjun Tang <[email protected]> * make sure the line feed is included for string attribute * update timing for overall completion time * Update .gitlab-ci.yml removing Cori Remove Cori CI in advance of Cori's decommission by the end of the month. * Remove unnecessary fflush call Signed-off-by: Chen Wang <[email protected]> * update formatting * LLSM_importer Tutorial and Timing for job completion time. (#86) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting --------- Co-authored-by: Houjun Tang <[email protected]> * Fix Issue #85, server segfault when another client application with different number of ranks connects to it * Committing clang-format changes * update metrics * Update .gitlab-ci.yml * update VPIC output timing precision (#88) * update VPIC output timing precision * update timing to make consistent * llsm_importer (#1) formatter on llsm_importer * Tiff Parallel Reader sync to latest version (#89) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * llsm_importer (#1) formatter on llsm_importer --------- Co-authored-by: Houjun Tang <[email protected]> * Update .gitlab-ci.yml * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * Data type for new kvtag (including refactoring and serde framework) (#90) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * llsm_importer (#1) formatter on llsm_importer * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake --------- Co-authored-by: Houjun Tang <[email protected]> * LLSM Importer update: new job script + new data type update on kvtags (#92) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * update metrics * forcibly enable openmp * adding C flags from the mex compiler * Update .gitlab-ci.yml * updated code * clang format * llsm_importer (#1) formatter on llsm_importer * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * update * print numWorkers * update scripts * update script * update script * formatting * update llsm_tools.c * remove unnecessary hash table init --------- Co-authored-by: Houjun Tang <[email protected]> Co-authored-by: Jean Luca Bez <[email protected]> * Update .gitlab-ci.yml * fix warnings, commenting off 'find_path' and 'find_library' for Mercury in src/commons/CMakeLists.txt (#93) * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * update metrics * forcibly enable openmp * adding C flags from the mex compiler * Update .gitlab-ci.yml * updated code * clang format * llsm_importer (#1) formatter on llsm_importer * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * update * print numWorkers * update scripts * update script * update script * formatting * update llsm_tools.c * remove unnecessary hash table init * update script * fix some warnings * fix some warnings * update * update * fix warning * update * update * update * update * update * update * update * update * update * fix warnings * fix warnings * fix warnings * fix warnings * fix warnings * fix warnings * fix warnings * update * update * update * server address and file paths using 1024, TMP_DIR path using 1024/2, NAME_MAX for appname and objname takes 1024/2, HOSTNAME takes 1024/8, NA_INFO_STRING takes 1024/4 * update * update * update * update --------- Co-authored-by: Houjun Tang <[email protected]> Co-authored-by: Jean Luca Bez <[email protected]> * Update clang-format-fix.yml * Update clang-format-fix.yml * Increase the default server cache size to 32GB and flush frequency to 30s * Committing clang-format changes * add FindMERCURY.cmake * update commons/CMakeLists.txt * Fix unnecessary memory allocation (#103) * Fix an issue with opening a deleted container, added test (#101) * Fix an issue with opening a deleted container, added test * Refactor the query aggregation process * Fix container tag delete error (#102) * Fix container tag delete error * Committing clang-format changes * Update tag delete function * Refactor metdata lookup process for tag deletion * Committing clang-format changes * Formatting and comment * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Fix the server cache issue when cache becomes full and needs flush (#113) * Fix the server cache issue when cache becomes full and need flush * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Fix a wrong option description. (#115) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * update option message --------- Co-authored-by: Houjun Tang <[email protected]> * Install header files needed by PDCpy (#114) * Support unlimited object dimension size (#117) * Support unlimited object dimension szie * Add function description for PDC_SIZE_UNLIMITED * Fix obj_round_robin_io_all test code * Committing clang-format changes * More header files fix for PDCpy * Fix cmake path (#121) * Fix cmake path * Fix cmake path * Kvtag query (#122) * Add a new collective kvtag query api to return full/aggregated results to all clients * Committing clang-format changes * Add test code * Committing clang-format changes * Add an optimization when multiple clients issue different queries * Add test program * Fix free issue * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Rebase develop to stable (#125) * updates in documentation * update docs * trigger update * trigger update * remove extension * include docs build dependencies * update file * update metrics * Update .gitlab-ci.yml * Update .gitlab-ci.yml * update logo * Update .gitlab-ci.yml * adding recovered documentation * update text * fix typo * update CSS * increase content width * remove build docs * Refer documentation to readthedocs website (#95) * Update README.md * Update getting_started.rst * Update getting_started.rst * include updated text * fix links * fix typo * Update README.md * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * update code * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * clang format * clang format * clang-format-10 * change file name * update llsm importer * update llsm importer * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update tag names * update tag names * update query startingpos * update job scripts * fix iteration count in final report * update job scripts and benckmark program * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * Remove unnecessary fflush call Signed-off-by: Chen Wang <[email protected]> * Fix Issue #85, server segfault when another client application with different number of ranks connects to it * Committing clang-format changes * update VPIC output timing precision (#88) * update VPIC output timing precision * update timing to make consistent * llsm_importer (#1) formatter on llsm_importer * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * add FindMERCURY.cmake * LLSM Importer update: new job script + new data type update on kvtags (#92) * remove unnecessary install block from CMakeLists.txt * update output * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribute value * change the max length of attribute value * llsm tiff import test * llsm tiff import test * llsm tiff import test * llsm tiff import test * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update code * update cmake and llsm_importer * update cmake and llsm_importer * close if in cmake * cmake fix tiff * cmake policy to suppress warning * add pdc include dir * update code * update code * update code * update code * update code * update code * update array generating method * update array generating method * update array generating method * update array generating method * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * update CMakeLists * fix return type * fix return type * add timing * add timing * fix output * llsm tiff importer 1st version: read csv and import tiff files to PDC, adding metadata available in CSV files and TIFF loader * fix vairable name * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * fix cmake * add scripts * add scripts * add scripts * debugging for nonMPI program * debugging for nonMPI program * debugging for nonMPI program * clang format, without PDC, everything works perfectly. program fails at PDC init stage where PDCprop_create(PDC_CONT_CREATE, pdc) is being created * enable MPI * enable MPI * enlarge BCase size * enlarge BCase size * enlarge BCase size * resolve bcast count * llsm data path in script * llsm data path in script * update csv reader * update csv reader * update csv reader * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * update pdc * enlarge max write * update pdc * update pdc * update pdc * update pdc * update pdc_import.c * update pdc_import.c * update pdc_export.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update pdc_import.c * update tools/cmake * clang format * clang format * added a tutorial for llsm_importer * added a tutorial for llsm_importer * make sure the line feed is included for string attribute * update timing for overall completion time * update formatting * update metrics * forcibly enable openmp * adding C flags from the mex compiler * Update .gitlab-ci.yml * updated code * clang format * llsm_importer (#1) formatter on llsm_importer * add type for kvtag structure (#2) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * Feature/metadata type (#3) * upate metadata type system * update serde framework to coupe with the new data type system * replace unnecessary data types * adding type for pdc_kvtag_t, all occurances are fixed * update new commons CMake for publishing commons * commons compilation passed * compiled * remove unnecessary header files from installation * resolve conflict * add important files * clang formatting * update cmake * update * print numWorkers * update scripts * update script * update script * formatting * update llsm_tools.c * remove unnecessary hash table init --------- Co-authored-by: Houjun Tang <[email protected]> Co-authored-by: Jean Luca Bez <[email protected]> * fix warnings, commenting off 'find_path' and 'find_library' for Mercury in src/commons/CMakeLists.txt (#93) * Revert "update output" This reverts commit fe1f8b44995bc0dabd3b957e1032c2da26f56fdd. * build kvtag_add_get_scale * comment off free * update code * 1. kvtag_scale_add_get added \n 2. uint64_t support for obj/tag/query count \n 3. moving work assigning block downwards right before creating objects \n 4. everything is tested working * do while loop added, tested with 1m object and works * 1m objects test works, 10m object test fail as the original also fails * add new executable to test set * enlarge PDC_SERVER_ID_INTERVAL * update code * update console args * add p search test * add console arg for changing number of attributes per object * free allocated memory * fix query count issue * fix attr length definition * code refactored * code refactored * code refactored * code refactored * code refactored * code refactored * fix data type * fix data type * fix data type * add client side statistics * add client side statistics * fix format * clang formatter * update CMake * update CMake * update CMake * free allocated memory properly * clang format * clang format * clang-format-10 * change file name * address review comments * update llsm importer * update llsm importer * update server checkpoint intervals * update gitignore * adding job scripts * adding one debugging msg * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update container creation to collective mode for debugging purpose * update output for uint64_t * add scripts * update output for uint64_t * update output for uint64_t * update output for uint64_t * update scripts * update scripts * delete debugging message * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * make Cmake to publish scripts directory * update tag names * update tag names * update query startingpos * update query startingpos * update job scripts * add progressive timing for kvtag_add_get_scale * fix iteration count in final report * update job scripts and benckmark program * update message format * update message format * update message format * update message format * clang format * update job scripts * comment off object/container close procedure in benchmark to save node hours * change the max number of object to 1M * change the max length of attribut…
1 parent f2011d5 commit 73d9d83

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+2344
-728
lines changed

.devcontainer/devcontainer.Dockerfile

Lines changed: 4 additions & 91 deletions
Original file line numberDiff line numberDiff line change
@@ -1,95 +1,8 @@
11
# Note: Run `docker build -f .devcontainer/Dockerfile -t pdc:latest .` from the root directory of the repository to build the docker image.
22

33
# Use Ubuntu Jammy (latest LTS) as the base image
4-
FROM ubuntu:jammy
5-
6-
7-
8-
# Install necessary tools, MPICH, UUID library and developer files
9-
RUN apt-get update && apt-get install -y \
10-
build-essential \
11-
git \
12-
mpich \
13-
libmpich-dev \
14-
uuid \
15-
uuid-dev \
16-
autoconf \
17-
libtool \
18-
cmake \
19-
cmake-curses-gui \
20-
wget \
21-
axel \
22-
curl \
23-
vim \
24-
nano \
25-
gdb \
26-
cgdb \
27-
curl \
28-
valgrind
29-
30-
# Set WORK_SPACE environment variable and create necessary directories
31-
RUN mkdir -p /workspaces
32-
ENV WORK_SPACE=/workspaces
33-
34-
35-
# Clone the repositories
36-
WORKDIR $WORK_SPACE/source
37-
RUN git clone https://github.com/ofiwg/libfabric.git && \
38-
git clone https://github.com/mercury-hpc/mercury.git --recursive
39-
40-
COPY ./ ${WORK_SPACE}/source/pdc
41-
42-
ENV LIBFABRIC_SRC_DIR=$WORK_SPACE/source/libfabric
43-
ENV MERCURY_SRC_DIR=$WORK_SPACE/source/mercury
44-
ENV PDC_SRC_DIR=$WORK_SPACE/source/pdc
45-
ENV LIBFABRIC_DIR=$WORK_SPACE/install/libfabric
46-
ENV MERCURY_DIR=$WORK_SPACE/install/mercury
47-
ENV PDC_DIR=$WORK_SPACE/install/pdc
48-
49-
RUN mkdir -p $LIBFABRIC_SRC_DIR && \
50-
mkdir -p $MERCURY_SRC_DIR && \
51-
mkdir -p $LIBFABRIC_DIR && \
52-
mkdir -p $MERCURY_DIR && \
53-
mkdir -p $PDC_DIR
54-
55-
56-
# Save the environment variables to a file
57-
RUN echo "export LIBFABRIC_SRC_DIR=$WORK_SPACE/source/libfabric" > $WORK_SPACE/pdc_env.sh && \
58-
echo "export MERCURY_SRC_DIR=$WORK_SPACE/source/mercury" >> $WORK_SPACE/pdc_env.sh && \
59-
echo "export PDC_SRC_DIR=$WORK_SPACE/source/pdc" >> $WORK_SPACE/pdc_env.sh && \
60-
echo "export LIBFABRIC_DIR=$WORK_SPACE/install/libfabric" >> $WORK_SPACE/pdc_env.sh && \
61-
echo "export MERCURY_DIR=$WORK_SPACE/install/mercury" >> $WORK_SPACE/pdc_env.sh && \
62-
echo "export PDC_DIR=$WORK_SPACE/install/pdc" >> $WORK_SPACE/pdc_env.sh
63-
64-
65-
# Build and install libfabric
66-
WORKDIR $LIBFABRIC_SRC_DIR
67-
RUN git checkout v1.18.0 && \
68-
./autogen.sh && \
69-
./configure --prefix=$LIBFABRIC_DIR CC=mpicc CFLAG="-O2" && \
70-
make clean && \
71-
make -j && make install && \
72-
make check
73-
74-
ENV LD_LIBRARY_PATH="$LIBFABRIC_DIR/lib:$LD_LIBRARY_PATH"
75-
ENV PATH="$LIBFABRIC_DIR/include:$LIBFABRIC_DIR/lib:$PATH"
76-
RUN echo 'export LD_LIBRARY_PATH=$LIBFABRIC_DIR/lib:$LD_LIBRARY_PATH' >> $WORK_SPACE/pdc_env.sh && \
77-
echo 'export PATH=$LIBFABRIC_DIR/include:$LIBFABRIC_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh
78-
79-
80-
# Build and install Mercury
81-
WORKDIR $MERCURY_SRC_DIR
82-
ENV MERCURY_CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=$MERCURY_DIR -DCMAKE_C_COMPILER=mpicc -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=ON -DNA_USE_OFI=ON -DNA_USE_SM=OFF -DNA_OFI_TESTING_PROTOCOL=tcp "
83-
RUN git checkout v2.2.0 \
84-
mkdir -p build
85-
WORKDIR ${MERCURY_SRC_DIR}/build
86-
RUN cmake $MERCURY_CMAKE_FLAGS ../ && \
87-
make -j && make install && \
88-
ctest
89-
90-
# Set the environment variables
91-
ENV LD_LIBRARY_PATH="$MERCURY_DIR/lib:$LD_LIBRARY_PATH"
92-
ENV PATH="$MERCURY_DIR/include:$MERCURY_DIR/lib:$PATH"
93-
RUN echo 'export LD_LIBRARY_PATH=$MERCURY_DIR/lib:$LD_LIBRARY_PATH' >> $WORK_SPACE/pdc_env.sh \
94-
echo 'export PATH=$MERCURY_DIR/include:$MERCURY_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh
4+
# FROM ubuntu:jammy
5+
FROM hpcio/pdc-dev-base:latest
956

7+
RUN rm -rf $PDC_SRC_DIR && \
8+
rm -rf $PDC_DIR

.devcontainer/devcontainer.json

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,5 +5,63 @@
55
3000
66
],
77
"postCreateCommand": ".devcontainer/post-create.sh",
8-
"postStartCommand": ".devcontainer/post-start.sh"
8+
"postStartCommand": ".devcontainer/post-start.sh",
9+
"postAttachCommand": ".devcontainer/post-attach.sh",
10+
"customizations": {
11+
"vscode": {
12+
"extensions": [
13+
"1YiB.rust-bundle",
14+
"batisteo.vscode-django",
15+
"ChrisChinchilla.vscode-pandoc",
16+
"DamianKoper.gdb-debug",
17+
"donjayamanne.python-environment-manager",
18+
"donjayamanne.python-extension-pack",
19+
"dustypomerleau.rust-syntax",
20+
"eamodio.gitlens",
21+
"formulahendry.code-runner",
22+
"GitHub.copilot",
23+
"GitHub.copilot-chat",
24+
"JScearcy.rust-doc-viewer",
25+
"julialang.language-julia",
26+
"KevinRose.vsc-python-indent",
27+
"lextudio.iis",
28+
"lextudio.restructuredtext",
29+
"lextudio.restructuredtext-pack",
30+
"ms-azuretools.vscode-docker",
31+
"ms-python.python",
32+
"ms-python.vscode-pylance",
33+
"ms-toolsai.jupyter",
34+
"ms-toolsai.jupyter-keymap",
35+
"ms-toolsai.jupyter-renderers",
36+
"ms-toolsai.vscode-jupyter-cell-tags",
37+
"ms-toolsai.vscode-jupyter-slideshow",
38+
"ms-vscode.cmake-tools",
39+
"ms-vscode.cpptools",
40+
"ms-vscode.cpptools-extension-pack",
41+
"ms-vscode.cpptools-themes",
42+
"njpwerner.autodocstring",
43+
"PolyMeilex.rust-targets",
44+
"rogalmic.bash-debug",
45+
"rust-lang.rust-analyzer",
46+
"serayuzgur.crates",
47+
"shakram02.bash-beautify",
48+
"shd101wyy.markdown-preview-enhanced",
49+
"Swellaby.rust-pack",
50+
"tamasfe.even-better-toml",
51+
"trond-snekvik.simple-rst",
52+
"twxs.cmake",
53+
"VisualStudioExptTeam.intellicode-api-usage-examples",
54+
"VisualStudioExptTeam.vscodeintellicode",
55+
"wholroyd.jinja",
56+
"xaver.clang-format",
57+
"yzane.markdown-pdf",
58+
"yzhang.markdown-all-in-one",
59+
"ZhangYue.rust-mod-generator"
60+
],
61+
"settings": {
62+
"C_Cpp.clang_format_path": "/home/project/software/clang-format-lint-action/clang-format/clang-format10",
63+
"terminal.integrated.scrollback": 10000
64+
}
65+
}
66+
}
967
}

.devcontainer/post-attach.sh

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash
2+
3+
4+
WORK_SPACE_INITIALIZED_FILE=/workspaces/.workspace_initialized
5+
6+
if ! [ -f $WORK_SPACE_INITIALIZED_FILE ]; then
7+
/bin/bash /workspaces/pdc/.devcontainer/post-start.sh
8+
watch -t -n 5 'echo "Press Ctrl+C when there is no building processes."; echo "Number of initial PDC building processes:"; ps -ef | grep make | grep -v -c grep'
9+
else
10+
echo "Welcome Back!"
11+
fi
12+
13+
/bin/bash

.devcontainer/post-create.sh

100644100755
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,41 @@
11
#!/bin/bash
2+
3+
4+
WORK_SPACE_INITIALIZED_FILE=/workspaces/.workspace_initialized
5+
6+
if ! [ -f $WORK_SPACE_INITIALIZED_FILE ]; then
7+
touch $WORK_SPACE_INITIALIZED_FILE
8+
echo "First time to create workspace, start to install PDC"
9+
else
10+
echo "Workspace already initialized, skip the installation"
11+
exit 0
12+
fi
13+
14+
rm -rf $PDC_SRC_DIR
15+
rm -rf $PDC_DIR
16+
17+
18+
ln -s /workspaces/pdc $(dirname $PDC_SRC_DIR)
19+
20+
mkdir -p /workspaces/source
21+
ln -s $PDC_SRC_DIR /workspaces/source/pdc
22+
23+
mkdir -p /workspaces/install/pdc
24+
ln -s /workspaces/install/pdc $(dirname $PDC_SRC_DIR)
25+
26+
# Build and install PDC
27+
export PDC_CMAKE_FLAGS="-DBUILD_MPI_TESTING=ON -DBUILD_SHARED_LIBS=ON -DBUILD_TOOLS=OFF -DBUILD_TESTING=ON -DCMAKE_INSTALL_PREFIX=$PDC_DIR -DPDC_ENABLE_MPI=ON -DMERCURY_DIR=$MERCURY_DIR -DCMAKE_C_COMPILER=mpicc -DMPI_RUN_CMD=mpiexec "
28+
29+
cd $PDC_SRC_DIR
30+
rm -rf build && mkdir -p build
31+
32+
33+
cd ${PDC_SRC_DIR}/build
34+
cmake $PDC_CMAKE_FLAGS ../ 2>&1 > ./cmake_config.log || echo "ignoring cmake config error and proceed"
35+
make -j && make install
36+
37+
# Set the environment variables
38+
export LD_LIBRARY_PATH="$PDC_DIR/lib:$LD_LIBRARY_PATH"
39+
export PATH="$PDC_DIR/include:$PDC_DIR/lib:$PATH"
40+
echo 'export LD_LIBRARY_PATH=$PDC_DIR/lib:$LD_LIBRARY_PATH' >> $WORK_SPACE/pdc_env.sh
41+
echo 'export PATH=$PDC_DIR/include:$PDC_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh

.devcontainer/post-start.sh

100644100755
Lines changed: 3 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,6 @@
11
#!/bin/bash
22

3+
nohup /bin/bash /workspaces/pdc/.devcontainer/post-create.sh 2>&1 > /workspaces/pdc_install.out &
34

4-
ln -s /workspaces/pdc /home/codespace/source/pdc
5-
mkdir -p /workspaces/install
6-
mkdir -p /workspaces/source
7-
ln -s $PDC_SRC_DIR /workspaces/source/pdc
8-
ln -s $PDC_DIR /workspaces/install/pdc
9-
10-
export PDC_SRC_DIR=/workspaces/source/pdc
11-
12-
# Build and install PDC
13-
export PDC_CMAKE_FLAGS="-DBUILD_MPI_TESTING=ON -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=ON -DCMAKE_INSTALL_PREFIX=$PDC_DIR -DPDC_ENABLE_MPI=ON -DMERCURY_DIR=$MERCURY_DIR -DCMAKE_C_COMPILER=mpicc -DMPI_RUN_CMD=mpiexec "
14-
15-
cd $PDC_SRC_DIR
16-
rm -rf build && mkdir -p build
17-
18-
19-
cd ${PDC_SRC_DIR}/build
20-
cmake $PDC_CMAKE_FLAGS ../ 2>&1 > ./cmake_config.log || echo "ignoring cmake config error and proceed"
21-
make -j && make install
22-
23-
# Set the environment variables
24-
export LD_LIBRARY_PATH="$PDC_DIR/lib:$LD_LIBRARY_PATH"
25-
export PATH="$PDC_DIR/include:$PDC_DIR/lib:$PATH"
26-
echo 'export LD_LIBRARY_PATH=$PDC_DIR/lib:$LD_LIBRARY_PATH' >> $WORK_SPACE/pdc_env.sh
27-
echo 'export PATH=$PDC_DIR/include:$PDC_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh
28-
29-
30-
cd $PDC_SRC_DIR/build
31-
# ctest
5+
echo "Wait for 10 seconds for the building processes to start."
6+
sleep 10s

.docker/dev.Dockerfile

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Note: Run `docker build -f .docker/Dockerfile -t pdc:latest .` from the root directory of the repository to build the docker image.
2+
3+
# Use Ubuntu Jammy (latest LTS) as the base image
4+
FROM zhangwei217245/pdc_dev_base:latest
5+
6+
# Build and install PDC
7+
ENV PDC_CMAKE_FLAGS="-DBUILD_MPI_TESTING=ON -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=ON -DCMAKE_INSTALL_PREFIX=$PDC_DIR -DPDC_ENABLE_MPI=ON -DMERCURY_DIR=$MERCURY_DIR -DCMAKE_C_COMPILER=mpicc -DMPI_RUN_CMD=mpiexec "
8+
9+
10+
WORKDIR $PDC_SRC_DIR
11+
RUN rm -rf build && \
12+
mkdir -p build
13+
14+
# COPY ../ ${PDC_SRC_DIR}
15+
# RUN ls -l $PDC_SRC_DIR
16+
17+
WORKDIR ${PDC_SRC_DIR}/build
18+
RUN cmake $PDC_CMAKE_FLAGS ../ 2>&1 > ./cmake_config.log || echo "ignoring cmake config error and proceed" && \
19+
make -j && make install
20+
21+
# Set the environment variables
22+
ENV LD_LIBRARY_PATH="$PDC_DIR/lib:$LD_LIBRARY_PATH"
23+
ENV PATH="$PDC_DIR/include:$PDC_DIR/lib:$PATH"
24+
RUN echo 'export LD_LIBRARY_PATH=$PDC_DIR/lib:$LD_LIBRARY_PATH' >> $WORK_SPACE/pdc_env.sh && \
25+
echo 'export PATH=$PDC_DIR/include:$PDC_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh
26+
27+
28+
# WORKDIR $PDC_SRC_DIR/build
29+
# RUN ctest

.docker/dev.Dockerfile.dockerignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Exclude files and directories from the Docker build context
2+
!/.git/

0 commit comments

Comments
 (0)