Skip to content

Commit

Permalink
docs(data/README.md): rephrased documentation for data directory
Browse files Browse the repository at this point in the history
  • Loading branch information
mr.hasan committed Feb 23, 2025
1 parent e20f8f4 commit e7e8573
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,12 @@ This directory contains pre-collected data used for testing and development purp

## Structure

- `raw/`: Contains raw data files as collected from the source.
- `processed/`: Contains processed data files that have been cleaned or transformed for use in the project.
- `raw/`: Contains raw data files as collected from the source. There are two folders here
- `metafile/`: The schema for this `csv` file collection is filepath, start_date, end_date. Where the filepath
is the location of the logs for the failed tasks. start_date and end_date are also the metainformation of the failed tasks.
- `logfile/`: The actual log files contents. As airflow stores logs in configured file location, we've collect log from those files.
- `processed/`: Contains processed data files that have been cleaned or transformed for use in the project. Here resides the files after bieng
extracted and transformed for categorization.

## Usage

Expand Down

0 comments on commit e7e8573

Please sign in to comment.