Skip to content

Commit 4a5c15b

Browse files
updated README
1 parent e9a2219 commit 4a5c15b

File tree

1 file changed

+12
-121
lines changed

1 file changed

+12
-121
lines changed

README.rst

+12-121
Original file line numberDiff line numberDiff line change
@@ -1,130 +1,26 @@
11
Reference Coreference Scorer
22
============================
33

4-
Description
4+
DESCRIPTION
55
-----------
66

7-
This is the official implementation of the revised coreference scorer used for CoNLL-2011/2012 shared tasks on coreference resolution.
8-
It addresses issues that prevented the consistent scoring of predicted mentions in the past.
7+
This is the official implementation of the revised coreference scorer
8+
used for CoNLL-2011/2012 shared tasks on coreference resolution. It
9+
addresses issues that prevented the consistent scoring of predicted
10+
mentions in the past.
911

1012

1113
VERSION
1214
-------
1315

14-
The current stable version is v8.01
15-
16-
Citation
17-
--------
18-
19-
We would appreciate if you cite the paper when you use this scorer as some of us are academics or wanting to be academics, and citations matter.
20-
21-
::
22-
23-
@InProceedings{pradhan-EtAl:2014:P14-2,
24-
author = {Pradhan, Sameer and Luo, Xiaoqiang and Recasens, Marta and Hovy, Eduard and Ng, Vincent and Strube, Michael},
25-
title = {Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation},
26-
booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
27-
month = {June},
28-
year = {2014},
29-
address = {Baltimore, Maryland},
30-
publisher = {Association for Computational Linguistics},
31-
pages = {30--35},
32-
url = {http://www.aclweb.org/anthology/P14-2006}
33-
}
34-
35-
36-
Usage
37-
-----
38-
39-
40-
::
41-
42-
perl scorer.pl <metric> <key> <response> [<document-id>]
43-
44-
45-
<metric>: the metric desired to score the results. one of the following values:
46-
47-
muc: MUCScorer (Vilain et al, 1995)
48-
bcub: B-Cubed (Bagga and Baldwin, 1998)
49-
ceafm: CEAF (Luo et al., 2005) using mention-based similarity
50-
ceafe: CEAF (Luo et al., 2005) using entity-based similarity
51-
blanc: BLANC (Luo et al., 2014) BLANC metric for gold and predicted mentions
52-
all: uses all the metrics to score
53-
54-
<key>: file with expected coreference chains in CoNLL-2011/2012 format
55-
56-
<response>: file with output of coreference system (CoNLL-2011/2012 format)
57-
58-
<document-id>: optional. The name of the document to score. If name is not
59-
given, all the documents in the dataset will be scored. If given
60-
name is "none" then all the documents are scored but only total
61-
results are shown.
62-
63-
64-
OUTPUT
65-
------
66-
67-
The score subroutine returns an array with four values in this order:
68-
69-
Coreference Score
70-
~~~~~~~~~~~~~~~~~
71-
72-
Recall = recall_numerator / recall_denominator
73-
Precision = precision_numerator / precision_denominator
74-
F1 = 2 * Recall * Precision / (Recall + Precision)
75-
76-
Also recall, precision and F1 are printed in the standard output when variable ``$VERBOSE`` is not null.
77-
78-
79-
Identification of mentions
80-
~~~~~~~~~~~~~~~~~~~~~~~~~~
81-
82-
A score for identification of mentions (recall, precision and F1) is
83-
also included. Mentions from system response are compared with key
84-
mentions. This version performs strict mention matching as was used in
85-
the CoNLL-2011 and 2012 shared tasks.
86-
87-
AUTHORS
88-
-------
89-
90-
Emili Sapena, Universitat Politècnica de Catalunya, http://www.lsi.upc.edu/~esapena, esapena <at> lsi.upc.edu
91-
Sameer Pradhan, http://cemantix.org, pradhan <at> cemantix.org
92-
Sebastian Martschat, sebastian.martschat <at> h-its.org
93-
Xiaoqiang Luo, xql <at> google.com
94-
95-
96-
LICENSE
97-
-------
98-
This program is free software; you can redistribute it and/or modify it
99-
under the terms of the GNU General Public License as published by the
100-
Free Software Foundation; either version 2 of the License, or (at your
101-
option) any later version. This program is distributed in the hope that
102-
it will be useful, but WITHOUT ANY WARRANTY; without even the implied
103-
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
104-
GNU General Public License for more details.
105-
106-
You should have received a copy of the GNU General Public License along
107-
with this program; if not, write to the Free Software Foundation, Inc.,
108-
59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
109-
110-
COPYRIGHT
111-
---------
112-
113-
2009-2011, Emili Sapena esapena <at> lsi.upc.edu
114-
2011- Sameer Pradhan pradhan <at> cemantix.org
115-
116-
Reference Coreference Scorer
117-
============================
118-
119-
Description
120-
-----------
121-
122-
This is the official implementation of the revised coreference scorer as described in the following paper:
16+
The current stable (official) version for scoring predicted mentions is **v8.01**
12317

124-
Citation
18+
CITATION
12519
--------
12620

127-
We would appreciate if you cite the paper when you use this scorer as some of us are academics or wanting to be academics, and citations matter.
21+
We would appreciate if you cite the paper when you use this scorer as
22+
some of us are academics or wanting to be academics, and citations
23+
matter.
12824

12925
::
13026

@@ -141,12 +37,7 @@ We would appreciate if you cite the paper when you use this scorer as some of us
14137
}
14238

14339

144-
VERSION
145-
-------
146-
147-
The current stable (official) version for scoring predicted mentions is **v8.01**
148-
149-
Usage
40+
USAGE
15041
-----
15142

15243
::
@@ -190,7 +81,7 @@ Coreference Score
19081
These values are to standard output when variable ``$VERBOSE`` is not null.
19182

19283

193-
Identification of mentions
84+
Identification of Mentions
19485
~~~~~~~~~~~~~~~~~~~~~~~~~~
19586

19687
A score for identification of mentions (recall, precision and F1) is

0 commit comments

Comments
 (0)