diff --git a/.gitignore b/.gitignore index f19a515..bd8e414 100644 --- a/.gitignore +++ b/.gitignore @@ -3,7 +3,7 @@ __pycache__ /archive .vscode -.pytensils +.venv .pytest_cache .coverage testing.py \ No newline at end of file diff --git a/pytensils/__init__.py b/pytensils/__init__.py index cf99000..6ccf26b 100644 --- a/pytensils/__init__.py +++ b/pytensils/__init__.py @@ -1,3 +1,10 @@ +""" pytensils + +A Python package that provides general utility functions for managing +configuration, user-logging, directories and data-types as well as +a basic run-time profiler. +""" + import pytensils.config as config import pytensils.logging as logging import pytensils.utils as utils diff --git a/pytensils/config.py b/pytensils/config.py index 67ed385..cf8a35a 100644 --- a/pytensils/config.py +++ b/pytensils/config.py @@ -1,14 +1,6 @@ -""" -Information ---------------------------------------------------------------------- -Name : config.py -Location : ~/ - -Description ---------------------------------------------------------------------- -Contains the `class` methods for managing configuration. -""" +""" Configuration management """ +from __future__ import annotations import os import json import copy @@ -21,6 +13,23 @@ class Handler(): + """ A `class` that represents a configuration-handler. + + Parameters + ---------- + path : `str` + Directory path to the folder that contains the `file_name` of the + '.json' config-file. + file_name : `str` + File name of the '.json' config-file. + create: `bool` + `True` or `False`, creates an empty log-file, `file_name` + within `path` when `True`. + Logging: `pytensils.logging.Handler` + An instance of the `pytensils.logging.Handler` class that allows + for native 'pretty' user-logging of all `ValidationError` + exceptions. + """ def __init__( self, @@ -123,7 +132,8 @@ def __init__( ) def read(self) -> dict: - """ Reads a '.json' config-file and returns the content as a `dict`. + """ Reads a '.json' config-file, updates the config data + and returns the content as a `dict`. """ with open( os.path.join( @@ -164,6 +174,9 @@ def read(self) -> dict: ) ) + # Update the config data + self.data = copy.deepcopy(dict_object) + return dict_object except json.decoder.JSONDecodeError: @@ -201,8 +214,7 @@ def read(self) -> dict: ) def write(self): - """ Writes a '.json' config-file. - """ + """ Writes a '.json' config-file. """ with open( os.path.join( self.path, @@ -218,17 +230,27 @@ def write(self): def validate( self, - dtypes: dict + dtypes: Union[dict, Handler], ) -> bool: """ Validates the config-file data against the dtypes in `dtypes`. Returns `True` when validation completes successfully. Parameters ---------- - dtypes : `dict` + dtypes : Union[`dict`, `pytensils.config.Handler`] Dictionary object that contains the expected configuration value dtypes. """ + assert type(dtypes) in [dict, Handler], ( + ''.join([ + '{dtypes} must be either a `dict` or an instance of', + ' `pytensils.config.Handler`.' + ]) + ) + + # Parse the config data as a dictionary + if isinstance(dtypes, Handler): + dtypes = dtypes.to_dict() # Validate instance self._validate_instance(dict_object=dtypes, parameter='dtypes') @@ -251,8 +273,7 @@ def validate( return True def to_dict(self) -> dict: - """ Returns a dictionary object of the config-file data. - """ + """ Returns a dictionary object of the config-file data. """ return copy.deepcopy(self.data) def from_dict( diff --git a/pytensils/errors.py b/pytensils/errors.py index ad4e6bf..cfafde8 100644 --- a/pytensils/errors.py +++ b/pytensils/errors.py @@ -1,16 +1,8 @@ -""" -Information ---------------------------------------------------------------------- -Name : errors.py -Location : ~/ -Description ---------------------------------------------------------------------- -Contains the `class` exceptions for all native `pytensils` errors and -all error accessor functions. -""" +""" errors """ class config: + """ A `class` that contains all config-related errors. """ # Exceptions class OSError(OSError): diff --git a/pytensils/logging.py b/pytensils/logging.py index 0b421eb..08029ed 100644 --- a/pytensils/logging.py +++ b/pytensils/logging.py @@ -1,13 +1,4 @@ -""" -Information ---------------------------------------------------------------------- -Name : logging.py -Location : ~/ - -Description ---------------------------------------------------------------------- -Contains the `class` methods for 'pretty' user-logging. -""" +""" Pretty user-logging """ import os import textwrap @@ -31,16 +22,45 @@ # Setup CPython logging pytensils = logging.getLogger('pytensils') pytensils.setLevel(level=logging.DEBUG) + +# Prevent propagation to the root logger to avoid duplicate messages +pytensils.propagate = False + +# Setup a CPython logging stream handler debugger = logging.StreamHandler() debugger.setLevel(level=logging.DEBUG) debugger.setFormatter(fmt=logging.Formatter('[DEBUG] %(message)s')) -pytensils.addHandler(hdlr=debugger) + +# Add the handler if it does not currently exist +if not any(isinstance(h, logging.StreamHandler) for h in pytensils.handlers): + pytensils.addHandler(hdlr=debugger) # Setup tabulate tabulate.PRESERVE_WHITESPACE = True class Handler(): + """ A `class` that represents a logging-handler. + + Parameters + ---------- + path : `str` + Directory path to the folder that contains the `file_name` of the + log-file. + file_name : `str` + File name of the log-file. + description : `str` + Information about the executed Python job run. + metadata : `dict` + Environment parameters to display as metadata about the executed + Python job run. + create: `bool` + `True` or `False`, creates an empty log-file, `file_name` + within `path` when `True`. + debug_console: `bool` + `True` or `False`, outputs the logging content to the console + output when `True` using `logging.debug()`. + """ def __init__( self, @@ -51,7 +71,7 @@ def __init__( create: bool = True, debug_console: bool = False ): - """ Initializes an instance of the logger-handler class. + """ Initializes an instance of the logging-handler class. Parameters ---------- diff --git a/pytensils/profiler.py b/pytensils/profiler.py index bbb70d6..40965b3 100644 --- a/pytensils/profiler.py +++ b/pytensils/profiler.py @@ -1,13 +1,4 @@ -""" -Information ---------------------------------------------------------------------- -Name : profiler.py -Location : ~/ - -Description ---------------------------------------------------------------------- -Contains utility functions for profiling functions. -""" +""" Run-time profiling """ import time import datetime as dt @@ -31,7 +22,7 @@ def wrapper(*args, **kwargs): result = func(*args, **kwargs) t2 = time.time() td = dt.timedelta(seconds=(t2-t1)) - print("\n[INFO] Function {%s()} executed in %s hh:mm:ss" % ( + print("\n[INFO] Function {%s()} executed in %s hh:mm:ss." % ( func.__name__, td ) diff --git a/pytensils/utils.py b/pytensils/utils.py index 8b497b3..2eff889 100644 --- a/pytensils/utils.py +++ b/pytensils/utils.py @@ -1,13 +1,4 @@ -""" -Information ---------------------------------------------------------------------- -Name : utils.py -Location : ~/ - -Description ---------------------------------------------------------------------- -Contains utility functions for managing directories and data-types. -""" +""" Directory and data-type utilities """ import os import ast @@ -55,6 +46,7 @@ def generate_output_directory( def as_type( value: str, return_dtype: Literal[ + 'none', 'str', 'int', 'float', @@ -88,6 +80,12 @@ def as_type( """ try: + if ( + return_dtype.strip().upper() == 'NONE' + and not ast.literal_eval(value) + ): + return None + if return_dtype.strip().upper() == 'STR': return str(value) diff --git a/setup.cfg b/setup.cfg index b8496e3..cee1b6c 100644 --- a/setup.cfg +++ b/setup.cfg @@ -4,7 +4,6 @@ description = A Python package that provides general utility functions for manag long_description = file: README.md long_description_content_type = text/markdown author = Tom Eleff -author_email = eleffthomas@gmail.com url = https://github.com/thomaseleff/pytensils project_urls = Issues = https://github.com/thomaseleff/pytensils/issues diff --git a/tests/resources/closes-on-exception-filenotfounderror.log b/tests/resources/closes-on-exception-filenotfounderror.log index 12cfc95..037db88 100644 --- a/tests/resources/closes-on-exception-filenotfounderror.log +++ b/tests/resources/closes-on-exception-filenotfounderror.log @@ -3,5 +3,5 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - - Start time : 2024-07-08 18:33:26 + + Start time : 2024-12-06 18:04:46 diff --git a/tests/resources/closes-on-exception-notimplementederror.log b/tests/resources/closes-on-exception-notimplementederror.log index 0fab4e1..bdb89de 100644 --- a/tests/resources/closes-on-exception-notimplementederror.log +++ b/tests/resources/closes-on-exception-notimplementederror.log @@ -4,7 +4,7 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:26 + Start time : 2024-12-06 18:04:46 -------------------------------------------------------------------------- @@ -27,9 +27,9 @@ -------- Run-time performance summary. - - Start time : 18:33:26.617300 - End time : 18:33:26.893199 - Run time : 00:00:00.275899 + Start time : 18:04:46.649282 + End time : 18:04:46.847472 + Run time : 00:00:00.198190 + -------------------------------------------------------------------------- diff --git a/tests/resources/closes-on-exception-oserror.log b/tests/resources/closes-on-exception-oserror.log index 12cfc95..49c74eb 100644 --- a/tests/resources/closes-on-exception-oserror.log +++ b/tests/resources/closes-on-exception-oserror.log @@ -4,4 +4,5 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:26 + Start time : 2024-12-06 18:04:46 + diff --git a/tests/resources/closes-on-exception-success.log b/tests/resources/closes-on-exception-success.log index 7679932..4490bf0 100644 --- a/tests/resources/closes-on-exception-success.log +++ b/tests/resources/closes-on-exception-success.log @@ -4,7 +4,7 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:25 + Start time : 2024-12-06 18:04:45 -------------------------------------------------------------------------- @@ -13,8 +13,8 @@ Run-time performance summary. - Start time : 18:33:25.566032 - End time : 18:33:25.737897 - Run time : 00:00:00.171865 + Start time : 18:04:45.812073 + End time : 18:04:45.939506 + Run time : 00:00:00.127433 -------------------------------------------------------------------------- diff --git a/tests/resources/closes-on-exception-typeerror.log b/tests/resources/closes-on-exception-typeerror.log index 12cfc95..72fd1b4 100644 --- a/tests/resources/closes-on-exception-typeerror.log +++ b/tests/resources/closes-on-exception-typeerror.log @@ -4,4 +4,4 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:26 + Start time : 2024-12-06 18:04:46 \ No newline at end of file diff --git a/tests/resources/closes-on-exception-validationerror.log b/tests/resources/closes-on-exception-validationerror.log index 12cfc95..f345a54 100644 --- a/tests/resources/closes-on-exception-validationerror.log +++ b/tests/resources/closes-on-exception-validationerror.log @@ -4,4 +4,4 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:26 + Start time : 2024-12-06 18:04:46 diff --git a/tests/resources/closes-on-exception.log b/tests/resources/closes-on-exception.log index b35cc44..b8e4096 100644 --- a/tests/resources/closes-on-exception.log +++ b/tests/resources/closes-on-exception.log @@ -4,7 +4,7 @@ Generates close-on-exception content for `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:25 + Start time : 2024-12-06 18:04:46 -------------------------------------------------------------------------- @@ -16,7 +16,7 @@ >>> ZeroDivisionError: division by zero Filename : T:\Documents\Projects\[...]sils\tests\test_logging.py - Line Number : Line 329 + Line Number : Line 361 Function : divide_by_zero() Exception : ZeroDivisionError @@ -27,8 +27,8 @@ Run-time performance summary. - Start time : 18:33:25.823342 - End time : 18:33:26.211961 - Run time : 00:00:00.388619 + Start time : 18:04:46.014435 + End time : 18:04:46.316471 + Run time : 00:00:00.302036 -------------------------------------------------------------------------- diff --git a/tests/resources/example.log b/tests/resources/example.log index 4288f21..4e504c9 100644 --- a/tests/resources/example.log +++ b/tests/resources/example.log @@ -4,7 +4,9 @@ Generates example user-log content for all `pytenstils.logging` functionality. - Start time : 2024-07-08 18:33:24 + + Start time : 2024-12-06 18:04:44 + Extra parameter (1) : 1 Extra parameter (2) : 2 @@ -74,8 +76,8 @@ Run-time performance summary. - Start time : 18:33:24.702949 - End time : 18:33:24.981427 - Run time : 00:00:00.278478 + Start time : 18:04:44.965261 + End time : 18:04:45.237671 + Run time : 00:00:00.272410 -------------------------------------------------------------------------- diff --git a/tests/resources/test_config.log b/tests/resources/test_config.log index 4bd7a25..392c3dd 100644 --- a/tests/resources/test_config.log +++ b/tests/resources/test_config.log @@ -3,8 +3,8 @@ Environment information summary. - Start time : 2024-07-08 18:33:24 - + Start time : 2024-12-06 18:04:44 + -------------------------------------------------------------------------- Configuration validation @@ -31,8 +31,8 @@ Run-time performance summary. - Start time : 18:33:24.471101 - End time : 18:33:24.680127 - Run time : 00:00:00.209026 + Start time : 18:04:44.750909 + End time : 18:04:44.938004 + Run time : 00:00:00.187095 -------------------------------------------------------------------------- diff --git a/tests/test_config.py b/tests/test_config.py index 82b4ce6..4595581 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -202,7 +202,7 @@ def test_from_dict_success( } -def test_validate_success( +def test_validate_from_dict_success( CONFIG_FIXTURE: config.Handler, DTYPES_FIXTURE: config.Handler ): @@ -212,6 +212,25 @@ def test_validate_success( assert not CONFIG_FIXTURE.validation_errors +def test_validate_from_config_handler_success( + CONFIG_FIXTURE: config.Handler, + DTYPES_FIXTURE: config.Handler +): + assert CONFIG_FIXTURE.validate( + dtypes=DTYPES_FIXTURE + ) + assert not CONFIG_FIXTURE.validation_errors + + +def test_validate_assertionerror( + CONFIG_FIXTURE: config.Handler +): + with pytest.raises(AssertionError): + CONFIG_FIXTURE.validate( + dtypes='invalid-type' + ) + + def test_validate_instance_validationerror(CONFIG_FIXTURE: config.Handler): with pytest.raises(errors.config.ValidationError): CONFIG_FIXTURE._validate_instance( diff --git a/tests/test_logging.py b/tests/test_logging.py index b88db94..5572f03 100644 --- a/tests/test_logging.py +++ b/tests/test_logging.py @@ -10,6 +10,7 @@ """ import os +from io import StringIO import pandas as pd import logging as clogging import pytest @@ -228,8 +229,7 @@ def test_write_typeerror(): ) -def test_debug_console(caplog: pytest.LogCaptureFixture): - caplog.set_level(clogging.DEBUG) +def test_debug_console(): # Initialize logging Logging = logging.Handler( @@ -241,19 +241,51 @@ def test_debug_console(caplog: pytest.LogCaptureFixture): debug_console=True ) - # Create debug console output - Logging.write(content='Log-content-for-the-output-console') - - # Cleanup - os.remove( - os.path.join( - os.path.dirname(__file__), - 'resources', - 'python.log' - ) + # Ensure that a StreamHandler is attached + handler = next( + ( + h for h in logging.pytensils.handlers if isinstance( + h, + clogging.StreamHandler + ) + ), + None ) + assert handler is not None + + # Replace the StreamHandler with StringIO + log_output = StringIO() + original_stream = handler.stream + handler.setStream(log_output) + + try: + + # Loging + test_message = "Log-content-for-the-output-console" + Logging.write(test_message) + + # Flush the handler and retrieve the log output + handler.flush() + log_output.seek(0) + captured_logs = log_output.read() + + # Ensure the log message is present in the captured output + assert test_message in captured_logs + assert "DEBUG" in captured_logs - assert 'Log-content-for-the-output-console' in caplog.text + finally: + + # Restore the StreamHandler + handler.setStream(original_stream) + + # Cleanup + os.remove( + os.path.join( + os.path.dirname(__file__), + 'resources', + 'python.log' + ) + ) def test_pretty_dict_valueerror(): diff --git a/tests/test_utils.py b/tests/test_utils.py index 4cb2704..951e7c6 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -36,6 +36,10 @@ def test_generate_output_directory_oserror(): ) +def test_as_type_success_none(): + assert not utils.as_type(value='None', return_dtype='none') + + def test_as_type_success_str(): assert utils.as_type(value='ABC') == 'ABC'