Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logic to support context-specific modules #534

Closed
wants to merge 7 commits into from

Conversation

jshoughtaling
Copy link
Contributor

@jshoughtaling jshoughtaling commented Mar 14, 2024

This PR proposes a new architecture that allows users to (1) apply a targeted subset of checks based on a particular context of interest (e.g. ICU, pregnancy, oncology, etc.), and (2) add modules that include multiple complex checks at any level (TABLE, FIELD, CONCEPT) without needing to add multiple columns per check to the orchestration csv files. I've also incorporated a first demo module (ICU) that can be selected by passing a single argument (contextModule = ICU) to the main executeDqChecks function.

It also adds documentation about how to create and insert new modules, either for local use or for dissemination within focused consortia.

The code changes address the following issues:

Possible elements to discuss:

  • Ability to pull checks/modules from separate repositories
  • Ability to incorporate pre-defined concept sets (e.g. TermHub) into modules
  • Separation of module logic (sql) and orchestration (csv) into sub directories or other locations

High-level view of modular structure:

DQDModules drawio (1)

katy-sadowski and others added 7 commits February 21, 2024 21:13
Develop
Add missing changes to NEWS
@jshoughtaling jshoughtaling changed the base branch from main to develop March 14, 2024 13:05
@jshoughtaling jshoughtaling changed the base branch from develop to main March 14, 2024 13:06
Copy link

codecov bot commented Mar 14, 2024

Codecov Report

Attention: Patch coverage is 23.94366% with 54 lines in your changes are missing coverage. Please review.

Project coverage is 81.60%. Comparing base (6ef7ee2) to head (ec4bd24).

Files Patch % Lines
R/sqlOnly.R 16.66% 45 Missing ⚠️
R/runCheck.R 38.46% 8 Missing ⚠️
R/readThresholdFile.R 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #534      +/-   ##
==========================================
- Coverage   85.03%   81.60%   -3.43%     
==========================================
  Files          16       16              
  Lines         909      957      +48     
==========================================
+ Hits          773      781       +8     
- Misses        136      176      +40     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@katy-sadowski
Copy link
Collaborator

katy-sadowski commented Mar 24, 2024

Hi @jshoughtaling thanks again for working on this! One thing I'd like to add which I don't think was mentioned in our call the other week - one requirement I think should be imposed for a custom-checks solution is that users should be able to add their own checks without modifying the package itself. So the custom SQL queries should live in a directory outside DQD which DQD can be pointed at as the location for the custom SQL.

Let me know what you think about trying to incorporate this here :)

Also - you'll want to point this PR at develop instead of main, to save yourself merge conflict grief later on ;) thanks.

@jshoughtaling jshoughtaling changed the base branch from main to develop March 24, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants