Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate the transform and load step into /analytics #2482

Closed
5 tasks done
widal001 opened this issue Oct 15, 2024 · 4 comments · Fixed by #2617
Closed
5 tasks done

Migrate the transform and load step into /analytics #2482

widal001 opened this issue Oct 15, 2024 · 4 comments · Fixed by #2617
Assignees
@DavidDudas-Intuitial

Comments

@widal001
Copy link
Collaborator

widal001 commented Oct 15, 2024 •
edited by DavidDudas-Intuitial
Loading

Summary

Migrate the transform and load step prototyped in the sandbox repo into the /analytics codebase.

Acceptance criteria

  • The prototype code has been loaded into /analytics/src/ (alongside existing code in the analytics package)
  • The transform and load step can be triggered using a command line entry point for the analytics package (e.g. poetry run analytics etl...) see existing CLI guide for reference
  • A second CLI entry point is created to create/update the target schema in that local db file
  • All of the code quality checks set up for the analytics package are passing.
  • The documentation for these entry points has been added to the CLI guide
@DavidDudas-Intuitial
Copy link
Collaborator

Related: #2470

Closed
@DavidDudas-Intuitial DavidDudas-Intuitial moved this from In Progress to In Review in Simpler Grants Product & Delivery Oct 18, 2024
Merged
7 tasks
@DavidDudas-Intuitial
Copy link
Collaborator

PR: #2617

@widal001
Copy link
Collaborator Author

widal001 commented Nov 5, 2024

Beep boop: Automatically setting the point and sprint values for this issue in project HHS/13 because they were unset when the issue was closed.

1 similar comment
@widal001
Copy link
Collaborator Author

widal001 commented Nov 5, 2024

Beep boop: Automatically setting the point and sprint values for this issue in project HHS/13 because they were unset when the issue was closed.

@DavidDudas-Intuitial DavidDudas-Intuitial mentioned this issue Nov 5, 2024
Merged
DavidDudas-Intuitial added a commit that referenced this issue Nov 5, 2024
@DavidDudas-Intuitial
[Issue 2665] Minor fix to Makefile and CLI to integrate GitHub export…
e513579 
… and import capabilities (#2744)

## Summary
Partially Fixes #2665

### Time to review: __1 min__

## Changes proposed
> What was added, updated, or removed in this PR.

Minor change to Makefile and CLI to integrate the export work done in
#2481 with the import work done in #2482.

## Context for reviewers
> Testing instructions, background context, more in-depth details of the
implementation, and anything else you'd like to call out or ask
reviewers. Explain how the changes were verified.

Issue #2481 added capability to export data from GitHub to a flat file
that can be ingested by the new import capabilities added by issue
#2482. However, in testing, I just noticed a filename mismatch in
`Makefile` that precludes e2e integration. This PR fixes the filename
mismatch.

## Additional information
> Screenshots, GIF demos, code examples or output to help show the
changes working as expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DavidDudas-Intuitial DavidDudas-Intuitial

Labels
None yet
Projects
Simpler Grants Product & Delivery
Status: Done
Simpler.Grants.gov Product Backlog
Status: Done
Milestone
No milestone
2 participants
@widal001 @DavidDudas-Intuitial