This repository contains all my Python code and Jupyter notebooks for the projects I have built around understanding Large Language Models (LLMs) and transformer mechanisms.
These are personal projects I worked on to deepen my understanding of how LLMs work under the hood - covering areas like tokenization, embeddings, attention mechanisms, model evaluation, and more. Each project is a self-contained experiment involving data analysis, visualization, and hands-on implementation using PyTorch, NumPy, and related ML libraries.
- Text Tokenization - tokenization schemes, compression, cross-language comparisons
- Embeddings - cosine similarity, semantic axes, analogy vectors, graph-based similarity
- Model Output & Logits - softmax distributions, probabilistic token selection, perplexity
- Attention Mechanisms - multi-head attention, normalization, numerical simulations
- Model Evaluation - LLM loss functions, HellaSwag evaluation, bias measurement
- Regressions & Predictions - linear/logistic regressions on token-level data
Each folder corresponds to a numbered project (e.g., project_01_three_tokenization_schemes/) and contains:
- A Jupyter notebook (
.ipynb) with full implementation and explanations - Any supporting data or utility scripts
- Python 3.x
- PyTorch
- NumPy, Pandas
- Matplotlib, Seaborn
- Scikit-learn
- HuggingFace Transformers
These projects reflect my personal exploration and experimentation. Results, observations, and additional notes are documented inline within each notebook.