Implementations and experiments based on Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto.
This repository contains practical implementations of several foundational reinforcement learning problems, with a focus on understanding the underlying algorithms through code and experimentation.
Reinforcement Learning (RL) is a branch of machine learning in which an agent learns how to make decisions by interacting with an environment and receiving rewards or penalties.
This repository explores several classical reinforcement learning problems introduced in Sutton & Barto.
The current projects focus on:
- Blackjack
- Gambler's Problem
- Multi-Armed Bandits
- Exploration vs. Exploitation
The implementations are inspired by:
Reinforcement Learning: An Introduction — 2nd Edition Richard S. Sutton and Andrew G. Barto
The book is available online through the authors' official reinforcement learning resources.
An implementation of the classic Blackjack reinforcement learning problem.
The problem demonstrates how an agent can estimate the value of states through interaction with an environment.
Key concepts include:
- State representation
- Rewards
- Value estimation
- Policy evaluation
- Monte Carlo methods
The Gambler's Problem is a classical reinforcement learning example used to demonstrate Dynamic Programming and Value Iteration.
The agent attempts to maximize the probability of reaching a target capital through a sequence of bets.
Key concepts include:
- Markov Decision Processes
- Value functions
- Bellman equations
- Dynamic Programming
- Value Iteration
- Policy extraction
The Multi-Armed Bandit problem explores the fundamental trade-off between:
- Exploration — trying different actions to discover their rewards
- Exploitation — selecting the action currently believed to provide the highest reward
The project demonstrates the basic principles behind action selection and learning from rewards.
A MATLAB implementation of the Multi-Armed Bandit experiment is also included for comparison and experimentation.
This provides an opportunity to explore the same reinforcement learning concepts using a different programming environment.
The projects in this repository cover several foundational reinforcement learning concepts:
Agent
│
▼
Environment
│
├── State
├── Action
└── Reward
│
▼
Learning
│
▼
Improved Policy
Important topics include:
- Markov Decision Processes
- State and action spaces
- Rewards
- Value functions
- Policies
- Dynamic Programming
- Monte Carlo methods
- Exploration vs. exploitation
- Multi-Armed Bandits
RL_Projects/
│
├── Black_Jack/
│
├── Gambler_Problem/
│
├── N_Armed_Bandit/
│
├── N_Armed_Bandit_Code_in_matlab/
│
└── README.md
Each directory contains the implementation and supporting files for its corresponding reinforcement learning experiment.
- Python
- MATLAB
- Reinforcement Learning
- Machine Learning
- Numerical Computing
- Data Visualization
The main purpose of this repository is to develop a practical understanding of reinforcement learning fundamentals through implementation.
The projects provide hands-on experience with:
- Modeling reinforcement learning environments
- Defining states and actions
- Designing reward structures
- Estimating value functions
- Implementing classical RL algorithms
- Understanding exploration and exploitation
- Translating mathematical concepts into working code
This repository represents practical experimentation with foundational reinforcement learning algorithms and problems.
The implementations are intended to complement the theoretical concepts presented in Sutton & Barto's Reinforcement Learning: An Introduction.
Potential improvements for this repository include:
- Add detailed mathematical explanations for each implementation
- Add algorithm pseudocode
- Standardize project structure
- Add visualizations of learning progress
- Add experiment configuration
- Add reproducible random seeds
- Add automated tests
- Compare different exploration strategies
- Add performance metrics
- Add Jupyter Notebook demonstrations
- Add more Sutton & Barto examples
Ali Valizadeh
Python Developer · Django · AI, NLP & Automation · University Instructor
GitHub: