Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CLI to use Typer #180

Draft
wants to merge 15 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export TOWER_ACCESS_TOKEN=$(op read "op://Employee/Enterprise Staging Platform/credential" --no-newline)
export TOWER_WORKSPACE_ID=$(op read "op://Employee/Enterprise Staging Platform/workspace_id" --no-newline)
export TOWER_API_ENDPOINT=$(op read "op://Employee/Enterprise Staging Platform/hostname" --no-newline)
11 changes: 11 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ conda env create -f environment.yml
conda activate seqerakit
```

Or [install tw-cli from GitHub](https://github.com/seqeralabs/tower-cli?tab=readme-ov-file#getting-started).

You can then install the local repository for development using the `pip -e` command which will install it in place without copying it to your `PYTHONPATH`. Using `--no-deps` will ignore dependencies which have already been installed via Conda. This assumes the current working directory is a clone of this repository.

```console
Expand All @@ -37,6 +39,15 @@ pip install -e . --no-deps

You can then develop the code before committing changes and opening a pull request.

#### uv workflow

1. [Install uv](https://docs.astral.sh/uv/getting-started/installation/)
2. `uv run seqerakit -h`

##### Publishing

1. [uv publish](https://docs.astral.sh/uv/guides/publish/)

### pre-commit

We use [pre-commit](https://pre-commit.com/) which runs [Black](https://github.com/psf/black) and [Ruff](https://github.com/astral-sh/ruff) to ensure code consistency during development. Install pre-commit and configure with `pre-commit install` which will now run before every commit ensuring code consistency.
Expand Down
24 changes: 9 additions & 15 deletions .github/workflows/e2e-testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,15 @@ jobs:
SENTIEON_LICENSE_BASE64: ${{ secrets.SENTIEON_LICENSE_BASE64 }}

steps:
- name: Check out source-code repository
uses: actions/checkout@v3

- name: Setup Python
uses: actions/setup-python@v2
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v4
with:
python-version: '3.12'

- name: Install pip & seqerakit
run: |
pip install -e .
enable-cache: true
cache-dependency-glob: 'uv.lock'

- name: Install Tower CLI v0.9.1
- name: Install Tower CLI
run: |
wget https://github.com/seqeralabs/tower-cli/releases/download/v0.9.1/tw-linux-x86_64 \
wget https://github.com/seqeralabs/tower-cli/releases/download/v0.10.3/tw-linux-x86_64 \
&& chmod +x tw-linux-x86_64 \
&& sudo mv tw-linux-x86_64 /usr/local/bin/tw

Expand All @@ -64,7 +58,7 @@ jobs:
echo $GOOGLE_KEY | base64 -d > $temp_file
export GOOGLE_KEY=$temp_file

seqerakit examples/yaml/e2e/*.yml --delete || true
uv run seqerakit examples/yaml/e2e/*.yml --delete || true

- name: dryrun
run: |
Expand All @@ -80,7 +74,7 @@ jobs:
echo $GOOGLE_KEY | base64 -d > $temp_file
export GOOGLE_KEY=$temp_file

seqerakit examples/yaml/e2e/*.yml
uv run seqerakit examples/yaml/e2e/*.yml

- name: teardown
if: ( success() || failure() ) && ( github.event_name != 'workflow_dispatch' || inputs.clearup )
Expand All @@ -89,4 +83,4 @@ jobs:
echo $GOOGLE_KEY | base64 -d > $temp_file
export GOOGLE_KEY=$temp_file

seqerakit examples/yaml/e2e/*.yml --delete
uv runseqerakit examples/yaml/e2e/*.yml --delete
23 changes: 8 additions & 15 deletions .github/workflows/python-testing.yml
Original file line number Diff line number Diff line change
@@ -1,29 +1,22 @@
name: Lint-and-test
name: Lint and Test

on: [push]

jobs:
lint:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: psf/black@stable
- uses: chartboost/ruff-action@v1
- uses: actions/setup-python@v4
- uses: pre-commit/action@v3.0.1
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- uses: astral-sh/setup-uv@v4
with:
python-version: 3.8
cache: 'pip' # caching pip dependencies
cache-dependency-path: setup.py
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
pip install pytest-mock
pip install -e .
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: Run tests
run: |
python -m pytest --import-mode=append tests/
uv run pytest
24 changes: 9 additions & 15 deletions .github/workflows/teardown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,23 +26,17 @@ jobs:
SENTIEON_LICENSE_BASE64: ${{ secrets.SENTIEON_LICENSE_BASE64 }}

steps:
- name: Check out source-code repository
uses: actions/checkout@v3

- name: Setup conda
uses: conda-incubator/setup-miniconda@v3
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v4
with:
auto-update-conda: true
environment-file: environment.yml
python-version: '3.12'
mamba-version: '*'
channels: conda-forge,bioconda
activate-environment: seqerakit
use-mamba: true
enable-cache: true
cache-dependency-glob: 'uv.lock'

- name: Install pip & seqerakit
- name: Install Tower CLI v0.9.1
run: |
pip install -e .
wget https://github.com/seqeralabs/tower-cli/releases/download/v0.10.3/tw-linux-x86_64 \
&& chmod +x tw-linux-x86_64 \
&& sudo mv tw-linux-x86_64 /usr/local/bin/tw

- name: teardown
run: |
Expand All @@ -51,4 +45,4 @@ jobs:
echo $GOOGLE_KEY | base64 -d > $temp_file
export GOOGLE_KEY=$temp_file

seqerakit --delete examples/yaml/seqerakit-e2e.yml
uv run seqerakit --delete examples/yaml/seqerakit-e2e.yml
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ repos:
rev: 'v2.7.1'
hooks:
- id: prettier
- repo: https://github.com/psf/black
rev: 24.10.0
hooks:
- id: black
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.272
rev: v0.8.3
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.5.9
hooks:
- id: uv-lock
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
2 changes: 1 addition & 1 deletion examples/yaml/datasets/rnaseq_samples.csv
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ WT_REP2,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357072_1.fastq.
RAP1_UNINDUCED_REP1,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357073_1.fastq.gz,,auto
RAP1_UNINDUCED_REP2,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357074_1.fastq.gz,,reverse
RAP1_UNINDUCED_REP2,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357075_1.fastq.gz,,reverse
RAP1_IAA_30M_REP1,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357076_1.fastq.gz,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357076_2.fastq.gz,reverse
RAP1_IAA_30M_REP1,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357076_1.fastq.gz,s3://ngi-igenomes/test-data/rnaseq/minimal/GSE110004/SRR6357076_2.fastq.gz,reverse
2 changes: 1 addition & 1 deletion examples/yaml/pipelines/nextflow.config
Original file line number Diff line number Diff line change
@@ -1 +1 @@
process.maxRetries = 1
process.maxRetries = 1
2 changes: 1 addition & 1 deletion examples/yaml/pipelines/nf_core_rnaseq_params.yml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
outdir: 'az://seqeralabs-showcase/nf-core-rnaseq/results'
outdir: 'az://seqeralabs-showcase/nf-core-rnaseq/results'
Original file line number Diff line number Diff line change
@@ -1 +1 @@
outdir: 'gs://seqeralabs-showcase-eu-north-1/nf-core-viralrecon/illumina/results'
outdir: 'gs://seqeralabs-showcase-eu-north-1/nf-core-viralrecon/illumina/results'
56 changes: 56 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
[project]
name = "seqerakit"
version = "0.4.9"
description = "Automate creation of Seqera Platform resources"
readme = "README.md"
requires-python = ">=3.8,<4"
keywords = [
"nextflow",
"bioinformatics",
"workflow",
"pipeline",
"seqera-platform",
"seqera",
]
authors = [
{name = "Esha Joshi", email = "[email protected]"},
{name = "Adam Talbot", email = "[email protected]"},
{name = "Harshil Patel", email = "[email protected]"},
]
# TODO
maintainers = [
{name = "Esha Joshi", email = "[email protected]"},
]
# TODO https://pypi.org/classifiers/
classifiers = [
"License :: OSI Approved :: Apache Software License"
]
dependencies = [
"pyyaml>=6.0.0",
"typer>=0.15.1",
]
[project.urls]
Homepage = "https://seqera.io/"
Documentation = "https://docs.seqera.io/platform/24.2/seqerakit/installation"
Repository = "https://github.com/seqeralabs/seqera-kit"
Issues = "https://github.com/seqeralabs/seqera-kit/issues"
Changelog = "https://github.com/seqeralabs/seqera-kit/releases"

[project.scripts]
seqerakit = "seqerakit.cli:run"

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
packages = ["seqerakit"]
zip-safe = false
include-package-data = true

[dependency-groups]
dev = [
"pre-commit>=3.5.0",
"pytest-mock>=3.14.0",
"pytest>=8.3.4",
]
Loading
Loading