Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
raphaelsty authored Aug 5, 2023
1 parent ef2a069 commit d0e34ca
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This repository aims to replicate the findings of the SparseEmbed paper, focusin

The `SparsEmbed` model is compatible with any MLM based model using the class `AutoModelForMaskedLM` from HuggingFace.

### Differences from the original paper
### Differences with the original paper

1. **Loss Function:** We did not yet implement the distillation loss used in the paper. We have initially opted for a cosine loss like the one used in SentenceTransformer library. This decision was made to fine-tune the model from scratch, avoiding the use of a cross-encoder as a teacher. The distillation loss should be available soon.

Expand Down

0 comments on commit d0e34ca

Please sign in to comment.