Skip to content

Commit e14f795

Browse files
committed
Update licensing, minor instruction adjustment
1 parent 654133c commit e14f795

File tree

2 files changed

+14
-4
lines changed

2 files changed

+14
-4
lines changed

README.md

+11-4
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
# [InFact: A Strong Baseline for Automated Fact-Checking](https://aclanthology.org/2024.fever-1.12/)
1+
# InFact: A Strong Baseline for Automated Fact-Checking
2+
[📄 Paper](https://aclanthology.org/2024.fever-1.12/)
23

34
This is the implementation of **Information-Retrieving Fact-Checker (InFact)**, the text-only claim verification system that won the [AVeriTeC fact-checking challenge](https://fever.ai/task.html), see also the [AVeriTeC Leaderboard](https://eval.ai/web/challenges/challenge-page/2285/leaderboard/5655).
45

@@ -7,7 +8,7 @@ This is the implementation of **Information-Retrieving Fact-Checker (InFact)**,
78
89
<img src="resources%2FConcept%20Figure.png" width="500">
910

10-
InFact is a text-only, 6-stage LLM pipeline for fact-checking with external evidence retrieval through Google Search. For evaluation on benchmarks that provide a knowledge store (like on AVeriTeC or FEVER), it supports the retrieval of relevant evidence by leveraging an embedding model to perform semantic search.
11+
InFact is a text-only, 6-stage LLM pipeline for fact-checking with external evidence retrieval through Google Search. For evaluation on benchmarks that provide a knowledge store (like AVeriTeC or FEVER), it supports the retrieval of relevant evidence by leveraging an embedding model to perform semantic search.
1112

1213
> [!NOTE]
1314
> The execution of the embedding model and the Llama LLMs requires GPUs.
@@ -40,6 +41,7 @@ You can also find the Docker image [here](https://hub.docker.com/r/tudamailab/in
4041
pip install -r requirements.txt
4142
```
4243

44+
4345
## Configuration & Building
4446
### 1. Configure API Keys
4547
To this end, run the script `scripts/setup.py` with
@@ -52,7 +54,8 @@ python -m scripts.setup
5254
Alternatively, you may manually insert all necessary API keys into the file `config/api_keys.yaml`.
5355

5456
### 2. Optional: Configure Datasets & Results Paths
55-
In `config/globals.py`, insert the paths to the respective input/output directories.
57+
In `config/globals.py`, insert the paths to the respective input/output directories. If you want to evaluate on AVeriTeC, you'll need to download the benchmark separately [here](https://huggingface.co/chenxwh/AVeriTeC/tree/main/data).
58+
5659
5760
### 3. Optional: Build AVeriTeC Knowledge Base (KB)
5861
Simply run the script `scripts/averitec/build.py`with
@@ -72,11 +75,15 @@ With `scripts/run.py`, you can fact-check your own claims. The command to run it
7275
```bash
7376
python -m scripts.run
7477
```
75-
If you provided a Serper API key, this will use Google Search to verify your claim.
78+
If you configured a Serper API key, InFact will automatically use Google Search to verify your claim, otherwise it falls back to DuckDuckGo (which has strong rate limits).
7679
7780
### Reproduce InFact Paper Results
7881
To reproduce InFact's result on AVeriTeC using its default hyperparameters, run
7982
```bash
8083
python -m scripts.averitec.evaluate
8184
```
8285
The number of workers executing the evaluation is based on the number of GPUs in your machine.
86+
87+
88+
## License
89+
This repository and all its contents are licensed under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0).

infact/tools/search/query_serper.py

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# Copyright 2024 Google LLC
2+
# Major modifications applied by Technical University of Darmstadt, FG Multimodal Grounded Learning.
3+
14
"""Class for querying the Google Serper API."""
25

36
import random

0 commit comments

Comments
 (0)