Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Corrupt files #8

Open
oscarbatori opened this issue Jun 29, 2020 · 0 comments
Open

Corrupt files #8

oscarbatori opened this issue Jun 29, 2020 · 0 comments

Comments

@oscarbatori
Copy link
Contributor

The following files have seemingly random column names, or at least ones that deviate heavily from the standard:

2014/20140826__az__primary__greenlee__precinct.csv
2014/20140826__az__primary__coconino__precinct.csv
2014/20140826__az__primary__gila__precinct.csv
2014/20140826__az__primary__yuma__precinct.csv
2014/20140826__az__primary__pinal__precinct.csv
2014/20140826__az__primary__maricopa__precinct.csv
2014/20141104__az__general__yuma__precinct.csv
2014/20140826__az__primary__mohave__precinct.csv
2014/20140826__az__primary__apache__precinct.csv
2014/20141104__az__general__gila__precinct.csv
2014/20141104__az__general__pinal__precinct.csv
2014/20141104__az__general__santa_cruz__precinct.csv
2014/20141104__az__general__coconino__precinct.csv
2014/20141104__az__general__greenlee__precinct.csv
2014/20141104__az__general__pima__precinct.csv
2014/20141104__az__general__maricopa__precinct.csv
2014/20140826__az__primary__pima__precinct.csv
2014/20141104__az__general__la_paz__precinct.csv
2014/20141104__az__general__mohave__precinct.csv
2014/20141104__az__general__apache__precinct.csv
2016/counties/20160830__az__primary__graham__precinct.csv
2016/counties/20160830__az__primary__santa_cruz__precinct.csv
2016/counties/20160830__az__primary__la_paz__precinct.csv
2016/counties/20160322__az__primary__president__la_paz__precinct.csv
2016/counties/20160322__az__primary__president__apache__precinct.csv
2016/counties/20160830__az__primary__apache__precinct.csv
2016/counties/20160322__az__primary__president__cochise__precinct.csv
2016/counties/20160830__az__primary__mohave__precinct.csv
2016/counties/20160322__az__primary__president__mohave__precinct.csv
2016/counties/20160830__az__primary__yuma__precinct.csv
2016/counties/20160322__az__primary__president__pinal__precinct.csv
2016/counties/20160322__az__primary__president__gila__precinct.csv
2016/counties/20160830__az__primary__gila__precinct.csv
2016/counties/20160322__az__primary__president__yuma__precinct.csv
2016/counties/20160830__az__primary__navajo__precinct.csv
2016/counties/20160830__az__primary__yavapai__precinct.csv
2016/counties/20160322__az__primary__president__coconino__precinct.csv
2016/counties/20160830__az__primary__pima__precinct.csv
2016/counties/20160322__az__primary__president__greenlee__precinct.csv
2016/counties/20160830__az__primary__pinal__precinct.csv
2016/counties/20160322__az__primary__president__maricopa__precinct.csv
2016/counties/20160322__az__primary__president__pima__precinct.csv
2016/counties/20160322__az__primary__president__santa_cruz__precinct.csv
2016/counties/20160830__az__primary__maricopa__precinct.csv
2016/counties/20160830__az__primary__cochise__precinct.csv
2016/counties/20160830__az__primary__coconino__precinct.csv
2016/counties/20160830__az__primary__greenlee__precinct.csv

Here is a dump out of the schema:

/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__greenlee__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__coconino__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__gila__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__yuma__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__pinal__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'early_votes',
       'polling_place_votes', 'late_early_votes', 'provisional_votes',
       'party_name', 'contest_name', 'choice_name', 'precinct_designation',
       'precinct_designation.1', 'precinct_name', 'votes_allowed',
       'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__maricopa__precinct.csv
Index(['precinct_name', 'choice_name', 'party_id', 'candidate_id',
       'contest_type', 'contest', 'contest_order_id', 'choice_order',
       'contest_name', 'vote_total', 'precinct_id', 'precinct_order',
       'votes_allowed', 'processed_done', 'processed_started', 'contest_total',
       'write_in', 'undervote', 'overvote'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__yuma__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__mohave__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__apache__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__gila__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__pinal__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'early_votes',
       'polling_place_votes', 'late_early_votes', 'provisional_votes',
       'party_name', 'contest_name', 'choice_name', 'precinct_designation',
       'precinct_designation.1', 'precinct_name', 'votes_allowed',
       'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__santa_cruz__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'party', 'vote_type_id', 'vote_type', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__coconino__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__greenlee__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__pima__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__maricopa__precinct.csv
Index(['precinct_name', 'choice_name', 'party_id', 'candidate_id',
       'contest_type', 'contest', 'contest_order_id', 'choice_order',
       'contest_name', 'vote_total', 'precinct_id', 'precinct_order',
       'votes_allowed', 'processed_done', 'processed_started', 'contest_total',
       'write_in', 'undervote', 'overvote'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20140826__az__primary__pima__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__la_paz__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__mohave__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count', 'Unnamed: 7'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2014/20141104__az__general__apache__precinct.csv
Index(['precinct_id', 'precinct_name', 'race_id', 'race', 'candidate_id',
       'candidate', 'count'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__graham__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__santa_cruz__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'polling_place_votes_ds200',
       'early_votes_ds200', 'party_name', 'contest_name', 'choice_name',
       'precinct_name', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__la_paz__precinct.csv
Index(['Unnamed: 0', 'precinct_id', 'precinct_name', 'Unnamed: 3', 'contest',
       'contest_name', 'Unnamed: 6', 'Unnamed: 7', 'Unnamed: 8', 'Unnamed: 9',
       'Unnamed: 10', 'Unnamed: 11', 'Unnamed: 12', 'choice', 'choice_name',
       'Unnamed: 15', 'Unnamed: 16', 'party', 'candidate_party',
       'vote_type_id', 'vote_type', 'Unnamed: 21', 'vote_total'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__la_paz__precinct.csv
Index(['contest', 'precinct_name', 'contest_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__apache__precinct.csv
Index(['contest', 'precinct_name', 'contest_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__apache__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'polling_place_votes_ds200',
       'early_votes_ds200', 'party_name', 'contest_name', 'choice_name',
       'precinct_designation', 'precinct_name', 'subjurisdiction',
       'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__cochise__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__mohave__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__mohave__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__yuma__precinct.csv
/Users/oscarbatori/anaconda3/envs/doltpy-dev/lib/python3.8/site-packages/IPython/core/interactiveshell.py:3062: DtypeWarning: Columns (2) have mixed types.Specify dtype option on import or set low_memory=False.
  has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
Index(['Unnamed: 0', 'precinct_id', 'precinct_name', 'Unnamed: 3', 'contest',
       'contest_name', 'Unnamed: 6', 'Unnamed: 7', 'Unnamed: 8', 'Unnamed: 9',
       'Unnamed: 10', 'Unnamed: 11', 'Unnamed: 12', 'choice', 'choice_name',
       'Unnamed: 15', 'Unnamed: 16', 'party', 'candidate_party',
       'vote_type_id', 'vote_type', 'Unnamed: 21', 'vote_total'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__pinal__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'early_votes',
       'polling_place_votes', 'late_early_votes', 'provisional_votes',
       'party_name', 'contest_name', 'choice_name', 'precinct_designation',
       'precinct_designation.1', 'precinct_name', 'votes_allowed',
       'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__gila__precinct.csv
Index(['contest', 'subjurisdiction', 'precinct_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__gila__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__yuma__precinct.csv
Index(['contest', 'precinct_name', 'contest_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__navajo__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'late_early_votes', 'provisional_votes', 'party_name',
       'contest_name', 'choice_name', 'precinct_designation', 'precinct_name',
       'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__yavapai__precinct.csv
Index(['record_type', 'precinct_id', 'precinct_name', 'contest',
       'vote_for_value', 'contest_order_id', 'contest_name', 'candidate_name',
       'party_name', 'candidate_id', 'vote_total', 'vote_type', 'done'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__coconino__precinct.csv
Index(['contest', 'subjurisdiction', 'contest_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__pima__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__greenlee__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__pinal__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'early_votes',
       'polling_place_votes', 'late_early_votes', 'provisional_votes',
       'party_name', 'contest_name', 'choice_name', 'precinct_designation',
       'precinct_designation.1', 'precinct_name', 'votes_allowed',
       'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__maricopa__precinct.csv
Index(['precinct_name', 'choice_name', 'party_id', 'candidate_id',
       'contest_type', 'contest', 'contest_order_id', 'choice_order',
       'contest_name', 'vote_total', 'precinct_id', 'precinct_order',
       'votes_allowed', 'processed_done', 'processed_started', 'contest_total',
       'write_in', 'undervote', 'overvote'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__pima__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160322__az__primary__president__santa_cruz__precinct.csv
Index(['contest', 'precinct_name', 'contest_id', 'contest_name', 'choice',
       'choice_name', 'number'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__maricopa__precinct.csv
Index(['precinct_name', 'choice_name', 'party_id', 'candidate_id',
       'contest_type', 'contest', 'contest_order_id', 'choice_order',
       'contest_name', 'vote_total', 'precinct_id', 'precinct_order',
       'votes_allowed', 'processed_done', 'processed_started', 'contest_total',
       'write_in', 'undervote', 'overvote'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__cochise__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__coconino__precinct.csv
Index(['Unnamed: 0', 'precinct_id', 'precinct_name', 'Unnamed: 3', 'contest',
       'contest_name', 'Unnamed: 6', 'Unnamed: 7', 'Unnamed: 8', 'Unnamed: 9',
       'Unnamed: 10', 'Unnamed: 11', 'Unnamed: 12', 'choice', 'choice_name',
       'Unnamed: 15', 'Unnamed: 16', 'party', 'candidate_party',
       'vote_type_id', 'vote_type', 'Unnamed: 21', 'vote_total'],
      dtype='object')
/Users/oscarbatori/Documents/open-elections/openelections-data-az/2016/counties/20160830__az__primary__greenlee__precinct.csv
Index(['contest', 'choice', 'precinct_id', 'vote_total', 'polling_place_votes',
       'early_votes', 'provisional_votes', 'party_name', 'contest_name',
       'choice_name', 'precinct_designation', 'precinct_name',
       'subjurisdiction', 'votes_allowed', 'referendum_flag'],
      dtype='object')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant