Mpi omp support (#346)

* #3 Introduce fixtures for Fortran compiler and tool_box.

* #3 Remove explicit compiler information from MpCommonArgs (since it's already part of the config toolbox).

* #3 Added linker as tool.

* #3 Added test for linking shared libraries.

* #3 Pass compiler flags to the linker if a compiler was specified.

* #3 Remove unused function.

* #3 Removed more unused code.

* #3 Automatically add a linker for each compiler.

* #3 Fixed typo.

* #3 Support vendor for compiler and linker.

* #3 Make linker having a vendor, too.

* #3 Add set_default_vendor method to tool repository.

* Ignore build directory for git.

* Updated test.

* # Fix some mypy errors and warnings.

* Avoid using get() for singleton, instead use __new__ which makes mypy happier.

* Changed the transformation_script parameter of function psyclone to accept a function that can return file-specific transformation scripts

* Make mypy happy by using patch.object.

* Remove more comments and confusion about mypy :)

* Try to make mypy happy on older python versions.

* Make flake8 happy.

* Sort imported name alphabetically.

* Try to fix failing hash test (and add some additional improvements in the test).

* Removed fpath= for input transformation_script function to pass mypy test for Python 3.7; Moved transformation_script_hash test to unit test from system test

* Fix mypy typing check errors for psyclone unit test

* Fix config typing issue with mypy in psyclone unit test

* Fix flake8 issues; Revert Config mypy typing fix

* Add comment to ignore typing check for fpath parameter of input transformation_script function

* Fix assert check after transformation_script function is changed from being called twice to once

* Filter out 'no transformation script' warning for psyclone system test

* Replace 'ignore' typing of fpath of transformation_script with removing keyword argument

* #3 Support proper tests to check if tools are available.

* 1. Updated transformation_script description; 2. Modified mock_transformation_script; 3.Removed redundant _analysis_for_prebuilds

* Updated lfric/atm.py and lfric/gungho.py examples to pass in transformation_script functions

* Added description for the psyclone step to instructions on writing a config

* #3 Added git as a tool.

* #3 Fix incorrect | usage in typing.

* #3 Added unit tests for git.

* #3 Renamed git.py to versioning.py, to avoid name clash with the corresponding test_git.py tests.

* #3 Converted svn and fcm to tools.

* #3 Fixed missing whitespace.

* Modified the documentation for writing a config with PSyclone

* Add config as a parameter for run_psyclone for the transformation_script to use;Updated the related functions and tests;
Changed the logic of the transformation_script examples

* #3 Replaced ar with tool object.

* #3 Added tests for ar.py.

* #3 Removed debug output.

* #3 Converted PSyclone to be a tool.

* #3 Removed debug print, fixed python 3.7 typing information.

* #3 Updated comments.

* Modified the get_optimisation_script function examples and updated the doc formatting

* #3 Add Rsync tool.

* #3 Removed now unused function.

* #3 Added test for rsync.

* #3 Fixed all mypy warnings about functions not checked.

* #3 Replace all mock-tests to use subprocess so the name of the executable is tested as well.

* #3 Remove duplicated flags.

* #3 Fixed changed order of linking.

* #3 Removed run_command function.

* #3 Fixed 3.8 typing error.

* #3 Fixed unused imports.

* #3 Move flags checksum into Flags, and remove now unused tools.py file.

* #3 Renamed newtools to tools.

* #3 Made custom function for all git functions called (instead of just calling run).

* #3 Updated and fixed comments.

* #3 Fixed errors in comments.

* Fixed minor errors in documentation.

* #3 Make it easier to create wrapper around standard compiler.

* #3 Added documentation for all tool related classes and their usage.

* #3 Added MISC category.

* Addressed reviewer's comments.

* Updated cli to properly use ToolBox etc, removing hard-coded gnu command linker option.

* Fixed mypy failures, including changes to import statement to avoid cyclic imports :(.

* #3 Fix circular import.

* Added #TODO so that this can be removed once fparser supports sentinels.

* Fix typing problems by ignoring fparser.

* Replaced more string names for artefacts with enums.

* Removed EXECUTABLES from constants.

* Moved Artefact class out of ArtefactStore and renamed it to ArtefactSet.

* Moved OBJECT_FILES from constants into ArtefactSet.

* Moved OBJECT_ARCHIVES from constants to ArtefactSet.

* Moved PRAGMAD_C from constants to ArtefactSet.

* Turned 'all_source' into an enum.

* Allow integer as revision.

* Fixed flake8 error.

* Removed specific functions to add/get fortran source files etc.

* Removed non-existing and unneccessary collections.

* Try to fix all run_configs.

* Fixed rebase issues.

* Added replace functionality to ArtefactStore, updated test_artefacts to cover all lines in that file.

* Started to replace artefacts when files are pre-processed.

* Removed linker argument from linking step in all examples.

* Try to get jules to link.

* Fixed build_jules.

* Fixed other issues raised in reviews.

* Try to get jules to link.

* Fixed other issues raised in reviews.

* Simplify handling of X90 files by replacing the X90 with x90, meaning only one artefact set is involved when running PSyclone.

* Make OBJECT_ARCHIVES also a dict, migrate more code to replace/add files to the default build artefact collections.

* Fixed some examples.

* Fix flake8 error.

* Fixed failing tests.

* Support empty comments.

* Fix preprocessor to not unnecessary remove and add files that are already in the output directory.

* Allow find_soure_files to be called more than once by adding files (not replacing artefact).

* Updated lfric_common so that files created by configurator are written in build (not source).

* Use c_build_files instead of pragmad_c.

* Removed unnecessary str.

* Documented the new artefact set handling.

* Fixed typo.

* Make the PSyclone API configurable.

* Fixed formatting of documentation, properly used ArtefactSet names.

* Support .f and .F Fortran files.

* Removed setter for tool.is_available, which was only used for testing.

* #3 Fix documentation and coding style issues from review.

* Renamed Categories into Category.

* Minor coding style cleanup.

* Removed more unnecessary ().

* Re-added (invalid) grab_pre_build call.

* Fixed typo.

* Renamed set_default_vendor to set_default_compiler_suite.

* Renamed VendorTool to CompilerSuiteTool.

* Also accept a Path as exec_name specification for a tool.

* Move the check_available function into the base class.

* Fixed some types and documentation.

* Fix typing error.

* Added explanation for meta-compiler.

* Improved error handling and documentation.

* Replace mpiifort with mpifort to be a tiny bit more portable.

* Use classes to group tests for git/svn/fcm together.

* Fixed issue in get_transformation script, and moved script into lfric_common to remove code duplication.

* Code improvement as suggested by review.

* Fixed run config

* Added reference to ticket.

* Updated type information.

* More typing fixes.

* Fixed typing warnings.

* As requested by reviewer removed is_working_copy functionality.

* Issue a warning (which can be silenced) when a tool in a toolbox is replaced.

* Fixed flake8.

* Fixed flake8.

* Fixed failing test.

* Addressed issues raised in review.

* Removed now unnecessary operations.

* Updated some type information.

* Fixed all references to APIs to be consistent with PSyclone 2.5.

* Added api to the checksum computation.

* Fixed type information.

* Added test to verify that changing the api changes the checksum.

* Make compiler version a tuple of integers

* Update some tests to use tuple versions

* Explicitly test handling of bad version format

* Fix formatting

* Tidying up

* Make compiler raise an error for any invalid version string

Assume these compilers don't need to be hashed.
Saves dealing with empty tuples.

* Check compiler version string for compiler name

* Fix formatting

* Add compiler.get_version_string() method

Includes other cleanup from PR comments

* Add mpi and openmp settings to BuildConfig, made compiler MPI aware.

* Looks like the circular dependency has been fixed.

* Revert "Looks like the circular dependency has been fixed." ...
while it works with the tests, a real application still triggered it.

This reverts commit 150dc37.

* Don't even try to find a C compiler if no C files are to be compiled.

* Updated gitignore to ignore (recently renamed) documentation.

* Fixed failing test.

* Return from compile Fortran early if there are no files to compiles. Fixed coding style.

* Add MPI enables wrapper for intel and gnu compiler.

* Fixed test.

* Automatically add openmp flag to compiler and linker based on BuildConfig.

* Removed enforcement of keyword parameters, which is not supported in python 3.7.

* Fixed failing test.

* Support more than one tool of a given suite by sorting them.

* Use different version checkout for each compiler vendor with mixins

* Refactoring, remove unittest compiler class

* Fix some mypy errors

* Use 'Union' type hint to fix build checks

* Return run_version_command to base Compiler class

Provides default version command that can be overridden for other compilers.
Also fix some incorrect tests
Other tidying

* Add a missing type hint

* Remove inheritance from mixins and use protocol

* Simplify compiler inheritance

Mixins have static methods with unique names,
overrides only happen in concrete classes

* Simplify usage of compiler-specific parsing mixins.

* Test for missing mixin.

* Fixed test.

* Added more tests for invalid version numbers.

* Added more test cases for invalid version number, improved regex to work as expected.

* Fixed typo in test.

* Fixed test.

* Split tests into smaller individual ones, fixed missing asssert in test.

* Parameterised compiler version tests to also test wrapper.

* Added missing MPI parameter when getting the compiler.

* Fixed comments.

* Order parameters to be in same order for various compiler classes.

* Remove stray character

* Made mpi and openmp default to False in the BuildConfig constructor.

* Removed white space.

* Support compilers that do not support OpenMP.

* Added documentation for openmp parameter.

---------

Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: jasonjunweilyu <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>