LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content

Welcome to our GitHub repository! This repository is based on the ideas introduced in

Shabtay, Nimrod, Felipe Maia Polo, Sivan Doveh, Wei Lin, M. Jehanzeb Mirza, Leshem Chosen, Mikhail Yurochkin et al. "LiveXiv--A Multi-Modal Live Benchmark Based on Arxiv Papers Content." arXiv preprint arXiv:2410.10783 (2024).

Installation

To use the code in this repository, clone the repo and create a conda environment using:

conda env create --file=environment.yaml
conda activate sloth

Data

Our data can be found on HuggingFace.

Efficient evaluation

Quick start

If you are interested in checking how our efficient eval method works in practice, please check this notebook.

Reproducing results from the paper

Please check our notebooks.

Citing

If you find LiveXiv useful for your research and applications, please cite using this BibTeX:

@misc{shabtay2024livexivmultimodallive,
      title={LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content}, 
      author={Nimrod Shabtay and Felipe Maia Polo and Sivan Doveh and Wei Lin and M. Jehanzeb Mirza and Leshem Chosen and Mikhail Yurochkin and Yuekai Sun and Assaf Arbelle and Leonid Karlinsky and Raja Giryes},
      year={2024},
      eprint={2410.10783},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2410.10783}, 
}