Concepts refacto & new generation specific splitter

[AntoninPoche]

This PR should absolutely be reviewed commit by commit.

It touches many files due to renaming and some file moves. Also, it has a lot of lines because the notebooks are rerunning. But it is not that complex.

Description

This PR was initiated by the observation that ModelWithSplitPoint was too complicated (with several split points, many granularities, and aggregation strategies). But in practice, either for classification or generation, we always use the same settings. It was initiated for classification in #150.

Hence, the goal of this PR is to remove dead code, simplify API, make computations more efficient

The main modifications are:

• Limit ModelWithSplitPoints to a single split point and remove the split_point parameter from any other object.
• get_activations() does not return a dict[torch.Tensor] anymore but a tuple[torch.Tensor, torch.tensor | None]. With a single split point, no need for a dictionary of activations. Only a tuple (activations, predictions) where the predictions can be None. This removes the need for the get_split_activations()
• Abstract splitting into a BaseSplitter and renaming every model_with_split_points attribute to splitter.
• Optimize how nnsight is used, also require nnsight>=0.7.0,<0.8.0
• Introduce a new SplitterForGeneration which forces the granularity to tokens, thus with no aggregations. Though there is still a parameter to include special tokens (but not padding).
• Adapt documentation to splitters' abstraction.
• Split the interpreto.model_wrapping module in interpreto.attributions.inference_wrappers, interpreto.concepts.splitters and move llm_interface.py to interpreto.commons. With new splitters and inference wrappers, the module was messy.
• Add inputs_to_activations to splitters.
• Rename encode_activations and decode_concepts to activations_to_concepts and concepts_to_activations.
• Add a device property and a to() method to concept explainers and link them directly to the concept_model device. The splitters are already user exposed, but users had no unified way to move the concept_model device. This also removes device management in interpretations.
• Adapt generation concept notebooks to modifications and rerun to validate validity.
• Fix the vendoring script.

Breaking changes

• ModelWithSplitPoints(split_points=[3]) is deprecated, not breaking. Use ModelWithSplitPoints(split_point=3) instead.
• AnythingElse.split_point or AnythingElse(split_point=...) will raise an error. The splitters are the only ones managing the split point.
• get_activations() output is not a dict but a tuple now.
• splitters do not have a get_split_activations() method anymore. It is not necessary; activations are the first element of the tuple.

# Old and broken
activations_dict = splitter.get_activations(inputs)
activations = splitter.get_split_activationns

# New
activations, _  = splitter.get_activations(inputs)

• from interpreto.model_wrapping import ModelWithSplitPoints is not the correct path anymore. One should use from interpreto import ModelWithSplitPoints which was already the recommended way.
• Similarly, from interpreto.model_wrapping.llm_interface import OpenAILLM is not the correct path anymore. One should use from interpreto.commons.llm_interface import OpenAILLM.
• encode_activations() and decode_concepts() methods from concept explainers have been renamed to activations_to_concepts() and concepts_to_activations(). But these are mainly internal to the library.
• Interpretation methods do not have a concept_model_device anymore. But it should not change much, since the concept model is usually already on the device.

# Old and broken
interpretation_method = TopKInputs(concept_explainer, concept_model_device="cuda")

# New
concept_explainer.to("cuda")  # only moves the `.concept_model`, not the `.splitter`
interpretation_method = TopKInputs(concept_explainer)

Type of Change

• 📚 Examples / docs / tutorials / dependencies update
• 🥂 Improvement (non-breaking change which improves an existing feature)
• 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• I've read the CODE_OF_CONDUCT.md document.
• I've read the CONTRIBUTING.md guide.
• I've successfully run the style checks using make lint.
• I've written tests for all new methods and classes that I created and successfully ran make test.
• I've written the docstring in Google format for all the methods and classes that I used.

[fanny-jourdan]

Great job, Antonin! The changes were necessary and well done ;)
I've added a few comments to the code (and some questions to make sure I understood it correctly), and most importantly, some things to fix in the notebook that aren't running properly. I'll leave it to you to take care of these small issues, and then you can merge!

[fanny-jourdan]

I reran all the notebooks with your changes, and everything is working fine now. :)