Practical Guide to Applied Conformal Prediction in Python

Data Science, while inherently intricate, is fundamentally the art of uncovering patterns and insights from information. As the field evolves, the ability to quantify and interpret the uncertainty of predictions becomes increasingly critical, especially in probabilistic machine learning models.

This repository is dedicated to advancing my own understanding and application of prediction uncertainty through the lens of conformal prediction. Built upon the principles and exercises outlined in the highly regarded book “Practical Guide to Applied Conformal Prediction” by Valery Manokhin (2023), it provides a structured exploration of this essential framework.

The primary goal of this repository is to serve as a practical and comprehensive resource for my own seeking to deepen the knowledge in this area. By offering detailed explanations, solutions, and insights derived from the book, it aims to empower practitioners and learners alike to confidently apply conformal prediction techniques in real-world scenarios.

Personal Motivation for Advancing in Conformal Prediction

1. Harnessing the Power of Conformal Prediction for Uncertainty Management: Conformal Prediction represents a transformative approach in addressing uncertainty within machine learning models. Its ability to provide reliable and interpretable prediction intervals makes it a pivotal tool in the evolving landscape of data science.

2. Enhancing Validation in Conventional Machine Learning Models: A main motivation is to strengthen the validation processes of conventional machine learning models. By integrating conformal prediction techniques, it becomes possible to assess predictions with greater accuracy and robustness, ensuring dependable results.

3. Building Trust in Predictive Models: Confidence in predictive models is critical for their application in real-world scenarios. Conformal prediction offers a framework that not only bolsters my model's reliability but also empowers decision-making with clear, quantifiable metrics of uncertainty.

Content Review:

Part 1: Introduction

Section 1: Introduction Conformal Prediction

Section 2: Overview of Conformal Prediction

Part 2: Conformal Prediction Framework

Section 3: Fundamentals of Conformal Prediction

Section 4: Validity & Efficiency of Conformal Prediction

Section 5: Types of Conformal Prediction

Part 3: Applications of Conformal Prediction

Section 6: Conformal Predictions for Classification

Section 7: Conformal Prediction for Regression

Section 8: Conformal Prediction for Time Series & Forecasting

When installing and using neuralprophet, especially in environments with custom Python or CUDA configurations, several common issues may occur. Below are the problems encountered and how we resolved them:

a) Cause: torchvision was compiled against a different version of torch or a mismatched CUDA version.

Solution: Reinstall torchvision from the exact same CUDA version index as your installed torch. For example, with torch==2.6.0+cu124:

b) Cause: Either plotly is not installed or the installed version is incompatible or not properly loaded in the current kernel.

Solution: Install a known compatible version and restart the runtime:

c) Recommended Versions for Smooth Setup

pip install neuralprophet==0.9.0
pip install torch==2.6.0+cu124
pip install torchvision==0.17.0+cu124
pip install plotly==5.14.1
ipywidgets
--index-url https://download.pytorch.org/whl/cu124

Section 9: Conformal Prediction for Computer Vision

Section 10: Conformal Prediction for Natural Language Processing

Part 4: Advanced Topics

Section 11: Handing Imbalanced Data

Section 12: Multi-Class Conformal Prediction

Usage and Contribution

This repository is open to everyone interested in enhancing their understanding of conformal prediction and its applications. It is designed to complement your learning journey and provide practical insights, but it should not be considered a replacement for the book “Practical Guide to Applied Conformal Prediction” by Valery Manokhin (2023).

Contributions to this repository are highly encouraged! If you have ideas for improvement, additional exercises, or solutions to share, feel free to submit a pull request. Together, we can foster a collaborative environment where knowledge is both shared and expanded.

Disclaimer:

This repository is intended solely for educational purposes and adheres strictly to the principles of fair use. It does not seek to infringe upon any copyrights held by the author or publisher of “Practical Guide to Applied Conformal Prediction” by Valery Manokhin (2023). All materials and exercises provided here are meant to supplement the learning experience and respect the intellectual property of their original creators.

Citation:

Manokhin, V. (2023). Practical Guide to Applied Conformal Prediction in Python. Packt Publishing. ISBN: 978-1-80512-276-0.

Mandatory Libraries

install OpenML
pip install nonconformist
pip install openml
pip install dtype_diet
install XGBoost
pip install xgboost
pip install --upgrade xgboost
install CatBoost
pip install catboost
pip install ipywidgets

Virtual Environment

pip install --upgrade pip
python3 -m pip install virtualenv
python3 -m venv env
source env/bin/activate
source env/bin/deactivate
pip3 install -r requirements.txt

Github Environment

Performed from Terminal Console

1. git init
2. git remote add origin ["copy here ssh or https"]
3. git remote -v
4. git add -A
5. git add .
6. git commit -m "insert here your commit"
7. git status
8. git push origin master

Additional GitHub Commands

if you already created your repository, then:

1. git remote add origin ["copy here ssh or https"] 
2. same procedure applied above
3. Note: if you already got your ReadMe.md & License.md then,
firstly request your git pull origin master. THIS IS ALWAYS A RECOMMENDED PRACTICE.
4. git push origin master