Notes on Information Theory and Learning Algorithms

Notes summarizing some of the core information and coding theories, some practical learning algorithms and their mathematical foundations. With an emphasis on mathematical derivations and intuitions.

Project Structure

Notes-on-InformationTheory-and-LearningAlgorithms/
├── Information Theory/
│   ├── information_theory_notes.pdf
│   ├── information_theory_notes.tex
│   └── images/
├── Coding Theory/
│   ├── coding_notes.pdf
│   ├── coding_notes.tex
│   └── images/
├── Learning Algorithms/
│   ├── learning_notes.pdf
│   ├── learning_notes.tex
│   └── images/
└── README.md

Reference

1. Dobilas, Saul. “VAE: Variational Autoencoders - How to Employ Neural Networks to Generate New Images | Towards Data Science.” 18 Apr. 2022.
2. MacKay, David J C. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 25 Sept. 2003.
3. Murphy, Kevin P. Probabilistic Machine Learning : An Introduction. Cambridge, MIT Press, 2022.
4. Psota, Eric Thomas, and Lance Pérez. “The Manifestation of Stopping Sets and Absorbing Sets as Deviations on the Computation Trees of LDPC Codes.” Journal of Electrical and Computer Engineering, vol. 2010, no. 18, 4 Aug. 2010.
5. Ryan, William E, and Shu Lin. Channel Codes: Classical and Modern. Cambridge University Press, 2009.