Pinned Loading

1. common-crawl-filtering common-crawl-filtering Public

   This project provides a robust, scalable pipeline for processing thousands of WET raw web data from Common Crawl into a high-quality dataset, and includes a from-scratch transformer implementation …

   Jupyter Notebook

2. llm-alignment-reasoning-RL llm-alignment-reasoning-RL Public

   This repository implements a production-grade evaluation and supervised fine-tuning (SFT) pipeline for measuring and improving Qwen 2.5 0.5B zero-shot performance on the MATH dataset.

   Python 1

3. systems-transformer-optimizations systems-transformer-optimizations Public

   This project implements systems-level optimizations for transformer training, including custom Triton kernels, PyTorch distributed training, optimizer state sharding, and memory/latency benchmarkin…

   Python

4. tokenizer_leakage tokenizer_leakage Public

   This repository serves to document my empirical studies on tokenizer data leakage and how it affects training and downstream tasks.

   Jupyter Notebook

5. Momentum-Experimentation Momentum-Experimentation Public

   This work explores the concept of momentum in gradient descent optimization. It provides a detailed mathematical foundation for understanding how momentum accelerates convergence in gradient-based …

   Jupyter Notebook

6. Architectures-From-Scratch Architectures-From-Scratch Public

   A curated collection of deep learning architectures I have implemented from scratch to clearly understand the design choices and the different inductive biases.