Skip to content
This repository has been archived by the owner on Sep 20, 2021. It is now read-only.

Bibcheck: Adds a temporary plugin #14

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Bibcheck: Adds a temporary plugin #14

wants to merge 2 commits into from

Conversation

GiorgosPa
Copy link
Contributor

  • Corrects the authors from some old atlas papers

@jalavik jalavik changed the title Bibcheck: Adds a temporary plugin WIP Bibcheck: Adds a temporary plugin May 5, 2014
@jalavik
Copy link
Contributor

jalavik commented Aug 8, 2014

ping @jmartinm we should review this.

@jalavik jalavik changed the title WIP Bibcheck: Adds a temporary plugin Bibcheck: Adds a temporary plugin Sep 19, 2014
@kaplun kaplun force-pushed the master branch 2 times, most recently from 8771cdc to 8c30f9d Compare September 24, 2014 09:23
@jalavik
Copy link
Contributor

jalavik commented Nov 11, 2014

@GiorgosPa Can you rebase this and remove/comment out the added entries in rules.cfg (to avoid activating the rules on PROD). Then I will merge this.

kaplun
kaplun reviewed Nov 12, 2014
View reviewed changes
bibcheck/plugins/atlas_authors.py Outdated
authors = {}
kb_name = 'atlas_authors'
description = 'Names of authors from some atlas papers'
if not CFG_ATLAS_AUTHORS_FILE and not kb_exists(kb_name):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From here...

@kaplun
Copy link
Contributor

kaplun commented Nov 12, 2014

Sorry to join late to the party 😄. I haven't seen this PR... I think it would be nice to rename the atlas check to something more generic (and actually could be almost something to include into Invenio). In the end this is going to be a typical problem not just related with ATLAS but also other use cases (plus there is no atlas-specific filter in the configuration).

E.g. could be named author_typos.py or something similar, and also the various CFG_ATLAS thingy could be renamed to remove the word "ATLAS".

@jalavik
Copy link
Contributor

jalavik commented Nov 12, 2014

Yeah, @GiorgosPa and I had a quick chat about this yesterday and we agreed to kill the KB and config variable parts actually. Instead turn it into a batch rule check_records(), that receives the source file for the mappings as parameter, then build the mapping once in memory and then replace all the values needed over each record - without the need of using a knowledgebase.

@dset0x This is a use case to think about actually for the new matcher checker, regarding "setting up" a rule (pre-loading of stuff etc.) once and run it on all things..

@jalavik
Copy link
Contributor

jalavik commented Nov 12, 2014

That said.. for the generic case, maybe a stable author typos mapping in a KB would make sense, but indeed the check should not worry about creating and filling the KB then. Just check if it is there or not.

@GiorgosPa GiorgosPa force-pushed the bibcheck branch 6 times, most recently from 7b513f4 to 6e7d239 Compare November 12, 2014 15:10
@GiorgosPa
Copy link
Contributor Author

I updated the PR so that no configuration variable or database knowledge base is used. I also think that the dictionary building will happen only once what do you think @kaplun ?

@kaplun
Copy link
Contributor

kaplun commented Nov 13, 2014

Thanks @GiorgosPa, so just one small note for me, but beside that, I think the plugin will now no longer have performance issues 👍

@GiorgosPa
Copy link
Contributor Author

Updated.

@jalavik
Copy link
Contributor

jalavik commented Dec 9, 2014

@kaplun Do you think this one is okay to merge?

jalavik
jalavik reviewed Dec 9, 2014
View reviewed changes
bibcheck/bibcheck_inspire_plugins_units_test.py Outdated
@@ -0,0 +1,149 @@
# -*- coding: utf-8 -*-
##
## This file is part of Invenio.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to INSPIRE

jalavik
jalavik reviewed Dec 9, 2014
View reviewed changes
bibcheck/plugins/journal_names.py Outdated
@@ -0,0 +1,38 @@
# -*- coding: utf-8 -*-
## This file is part of Invenio.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

INSPIRE

@GiorgosPa
Bibcheck: Adds a plugin to correct author names
917ae0c 
* Corrects the authors using a knwoledge base file with observed errors.

Signed-off-by: Georgios Papoutsakis <[email protected]>
@GiorgosPa GiorgosPa force-pushed the bibcheck branch 2 times, most recently from 1dd2d7d to 2ee0c67 Compare December 12, 2014 09:52
@GiorgosPa
BibCheck: adds a new plugin and unit tests
f358896 
* Adds journal_names plugin.

* Adds unit tests for the new plugin and
  atlas_authors plugin.

* Fixes pep8 issues on atlas_authors.

Signed-off by: Georgios Papoutsakis <[email protected]>
@kaplun kaplun force-pushed the master branch 2 times, most recently from 978e635 to c40e554 Compare September 8, 2015 07:56
@kaplun kaplun force-pushed the master branch 5 times, most recently from 4ea2c64 to 988b237 Compare September 23, 2015 11:07
ksachs pushed a commit to ksachs/inspire that referenced this pull request Apr 13, 2016
@jalavik
Merge pull request inspirehep#14 from bittirousku/phenixharvest
b522878 
spiders: new PHENIX spider
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@jmartinm jmartinm

Labels
ready
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@GiorgosPa @jalavik @kaplun @jmartinm