Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dataflows] An ability to add goodtables checks to the validate processor #192

Closed
2 tasks
cschloer opened this issue Jun 22, 2020 · 3 comments
Closed
2 tasks

Comments

@cschloer
Copy link
Contributor

datahq/dataflows#142

  • Declare which version of Python you are using (python --version)
  • Declare which operating system you are using
@adyork
Copy link

adyork commented Aug 17, 2020

We talked about having a test that would check values in a sci_name column for abbreviated taxon names like first letter genus, then period then species name like G. morhua. We would suggest using the full genus like Gadus morhua.

We would want to flag any sci_names that match ^\w\.

Note that these are actually good and we don't want to flag them:
Gadus sp.
Gadus spp.
https://regex101.com/r/8eFGXw/1/

@roll
Copy link
Member

roll commented Aug 17, 2020

Closing it for now, as we decided to separate dataflows/goodtables logic

@roll roll closed this as completed Aug 17, 2020
@adyork
Copy link

adyork commented Aug 17, 2020

I had started a python notebook in Colab that installed the dataflows commit we wanted to test (from datahq/dataflows#146) and loaded some test data for this issue from our frictionless-usecases repo that has "bad" names we want to check. Didn't get to testing validate_metadata which is probably good because it isn't being further developed.

I'm linking here in case we want to modify this to do goodtables testing or whatever implementation. The link to the data, and basic flow is there. https://gist.github.com/adyork/9ae791ebee7b0b651be034ec1b033c18#file-test-field-name-validation-ipynb

load('https://github.com/BCODMO/frictionless-usecases/raw/master/usecases/818993_seabirdCTD/orig/head/FK190211_CTD004_01032019.csv', format='csv', ),
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants