Skip to content
Energy Data Pipeline Test

incorporated more logging into aws.s3.py module #15

Sign in to view logs

incorporated more logging into aws.s3.py module

incorporated more logging into aws.s3.py module #15

Workflow file for this run

.github/workflows/test.yaml at d89b9e8
name: Energy Data Pipeline Test
on:
push:
branches: [ "dev" ]
pull_request:
branches: [ "uat" ]
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.12
uses: actions/setup-python@v3
with:
python-version: "3.12"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Create .env file
run: |
echo ACCESS_SECRET=${{ secrets.ACCESS_SECRET }} >> .env
echo ACCESS_KEY=${{ secrets.ACCESS_KEY }} >> .env
- name: Format Scripts in test and src
run: |
black src
black tests
- name: Format SQL Scripts with sqlfluff
run: |
echo "sqlfluff fix ."
- name: Lint SQL Scripts with sqlfluff
run: |
echo "sqlfluff lint ."
- name: Lint Python scripts with Ruff
run: |
python -m ruff check src/ tests/
- name: Run Async ETL file
run: |
python src/async_etl.py
- name: Run Sync ETL file
run: |
python src/sync_etl.py
- name: Test with pytest
run: |
pytest tests/