Skip to content

Commit

Permalink
feat: github actions pipeline, pre-commit (#2)
Browse files Browse the repository at this point in the history
  • Loading branch information
TheRealSeber authored Nov 28, 2023
1 parent 66a03ae commit 8f5e347
Show file tree
Hide file tree
Showing 5 changed files with 49 additions and 5 deletions.
18 changes: 18 additions & 0 deletions .github/workflows/pre-commit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Pre-commit linter

on: push

jobs:
run-linters:
runs-on: ubuntu-latest
steps:
- name: Check out Git repository
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Set up pre-commit Cache
uses: pre-commit/[email protected]
26 changes: 26 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-ast
- id: check-yaml
- id: check-added-large-files
- id: end-of-file-fixer
- id: trailing-whitespace

- repo: https://github.com/asottile/reorder-python-imports
rev: v3.12.0
hooks:
- id: reorder-python-imports

- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
args: [--max-line-length=88]

- repo: https://github.com/psf/black
rev: 23.9.1
hooks:
- id: black
language_version: python3.11
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
### Otodom / Pracuj tasks repository
This repository are going to contain early stages of the pracuj / otodom scrapers. Here we will create fundamentals for the future one big project and the done things are going to be merged into one.

The tasks you can find in the corresponding directories.
The tasks you can find in the corresponding directories.
4 changes: 2 additions & 2 deletions otodom/task_1/task_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ If something is missing you can leave the value as an empty string.
The Bot should be able to iterate through all the listings pages. The listings should be again collected and the duplicates should be removed.
### Task 2

Create a **settings.json** file. It should contain things which are going to define what bot is going to scrap. An example may look like:
Create a **settings.json** file. It should contain things which are going to define what bot is going to scrap. An example may look like:
```json
{
"base_url": "str",
Expand All @@ -40,4 +40,4 @@ Create a **settings.json** file. It should contain things which are going to def
```
and so on. Anything what may be usefull **please try to include**. Dependingly on the data the URL should be somehow generated. Look into Url how the Url is changed accordingly to what search parameters you applied on the site.

**Solutions** you can create in the **pracuj/task1/<your_name>** file and then make create a pull request.
**Solutions** you can create in the **pracuj/task1/<your_name>** file and then make create a pull request.
4 changes: 2 additions & 2 deletions pracuj/task_1/task_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ If something is missing you can leave the value as an empty string.
The Bot should be able to iterate through all the listings pages. The listings should be again collected and the duplicates should be removed.
### Task 2

Create a **settings.json** file. It should contain things which are going to define what bot is going to scrap. An example may look like:
Create a **settings.json** file. It should contain things which are going to define what bot is going to scrap. An example may look like:
```json
{
"base_url": "str",
Expand All @@ -33,4 +33,4 @@ Create a **settings.json** file. It should contain things which are going to def
```
and so on. Anything what may be usefull **please try to include**. Start with the most important things. Dependingly on the data the URL should be somehow generated. Look into Url how the Url is changed accordingly to what search parameters you applied on the site.

**Solutions** you can create in the **pracuj/task1/<your_name>** file and then make create a pull request.
**Solutions** you can create in the **pracuj/task1/<your_name>** file and then make create a pull request.

0 comments on commit 8f5e347

Please sign in to comment.