What needs to be done before finalfusion 1?

[danieldk]

I think that for downstream users it would be great if we had a first release. Due to semver, once we release 1.0.0, we cannot break APIs anymore in 1.x.y. Basically, once someone puts

finalfusion = "1"

in their Cargo.toml, it should always work for 1.x.y. Of course, we can extend the API.

This issue is for discussing what still needs to be done before 1.0.0. What sore thumbs are sticking out. Also, we should probably look if there are any public APIs that need to be private or crate-private.

[danieldk]

Just realized that there is one big blocker: we expose ndarray types in interfaces and ndarray is not 1 yet.

If we'd go to version 1 now, we would be stuck with ndarray 0.12.

[sebpuetz]

Spontaneously adding a chunk for word frequencies comes to mind, I believe we have discussed this somewhere in the past but I forgot what the outcome was.

That would require changes to Embeddings::new which is part of the public API unless we'd want to expose some set_frequencies method post 1.0.0.

Another thing that comes to mind is that we could discuss batched embeddings lookup, taking &[&str] and returning Array2<f32>, although I'm not sure if such a method would be too niche. This probably wouldn't touch any existing API, unless we'd want an abstraction for embedding() too handle both cases.

Eliminating dead buckets would be another change where I'm unsure how it touches existing APIs. This would probably implemented by adding a new vocabulary type which in turn would probably add a new variant to the public VocabWrap enum.

There might be more ideas lurking around, I'll add those here later.

[sebpuetz]

Judgin by rust-ndarray/ndarray#670 the next release will be 0.13 with no signs of a 1.x release any time soon.

[danieldk]

One other thing that I still want to do that also extends enums is the mmap'ed variant of quantized embedding matrices. I will put this high on my todo list, because it would be nice to have this in sticker.