Model monitoring #5202
Description
Model monitoring involves continuously monitoring the performance of predictions made by models to identify potential problems such as anomalous predictions, label shift, or concept drift.
We aim to implement model monitoring strategies, not only to help the users tracking the performance of their deployed models, but also to collect new data samples prioritizing the ones that show low confidence, out-of-distribution features or other anomalies.