🧠 Classical Neural Networks

Implementation and visualization of several classical neural network models and learning algorithms using Python.

This repository explores fundamental neural network concepts through practical implementations of McCulloch-Pitts neurons, Perceptron, ADALine, and Hebbian Learning.

The project is primarily educational and focuses on understanding how early neural network models work through implementation and experimentation.

---

📌 Overview

Before modern deep learning architectures, several foundational neural network models established the basic concepts of artificial neurons and machine learning.

This repository provides implementations of four important models:

• McCulloch-Pitts Neuron
• Perceptron
• ADALine
• Hebbian Learning

Each implementation provides a practical way to study the underlying learning mechanism and behavior of these classical models.

---

🧠 Models

1. McCulloch-Pitts Neuron

The McCulloch-Pitts neuron is one of the earliest mathematical models of an artificial neuron.

It uses binary inputs and a threshold-based activation mechanism to produce an output.

Inputs
  │
  ▼
Weighted Sum
  │
  ▼
Threshold
  │
  ▼
Binary Output

---

2. Perceptron

The Perceptron is a fundamental supervised learning algorithm for binary classification.

The model calculates a weighted sum of its inputs and applies an activation function to determine the output.

Input Features
      │
      ▼
Weighted Sum
      │
      ▼
Activation Function
      │
      ▼
Prediction
      │
      ▼
Weight Update

The implementation demonstrates how the model learns by adjusting its weights based on classification errors.

---

3. ADALine

ADALine (Adaptive Linear Neuron) is a classical neural network model that uses a linear activation function during training.

Unlike the Perceptron, ADALine updates its weights based on the difference between the predicted continuous output and the target value.

The project includes data, images, implementation code, and visualization related to the ADALine experiment.

---

4. Hebbian Learning

Hebbian Learning is an unsupervised learning principle based on the idea that connections between simultaneously active neurons become stronger.

A simplified representation is:

Neural Activity
      │
      ▼
Co-activation
      │
      ▼
Weight Adjustment

The repository includes a practical implementation of the Hebbian learning rule.

---

📂 Project Structure

Neural-Networks/
│
├── ADALine/
│   ├── data/
│   ├── img/
│   ├── ADALine.png
│   └── ADALine.py
│
├── Hebb/
│   ├── data/
│   ├── img/
│   └── hebb.py
│
├── McCulloch-Pitts/
│   └── McCulloch-Pitts.py
│
├── Perceptron/
│   ├── data/
│   ├── img/
│   └── perceptron.py
│
└── README.md

---

🛠️ Technologies

• Python
• NumPy
• Matplotlib
• Classical Machine Learning
• Artificial Neural Networks
• Data Visualization

---

🎯 Learning Objectives

This repository was developed to explore the fundamentals of artificial neural networks and classical machine learning algorithms.

The main learning objectives include:

• Understanding the artificial neuron concept
• Understanding threshold-based activation
• Implementing supervised learning
• Implementing unsupervised learning
• Understanding weight updates
• Exploring classification with a Perceptron
• Understanding linear learning with ADALine
• Studying Hebbian learning
• Visualizing model behavior

---

🔬 Classical Neural Network Evolution

The models in this repository represent important steps in the development of artificial neural networks:

McCulloch-Pitts
       │
       ▼
   Perceptron
       │
       ▼
    ADALine
       │
       ▼
Modern Neural Networks
       │
       ▼
   Deep Learning

Hebbian Learning represents a complementary learning principle based on neuron co-activation rather than traditional supervised error correction.

---

📊 Model Comparison

Model	Learning Type	Main Concept
McCulloch-Pitts	Rule / Threshold	Artificial neuron
Perceptron	Supervised	Binary classification
ADALine	Supervised	Linear learning
Hebbian	Unsupervised	Weight adaptation

---

▶️ Running the Projects

Each model is implemented independently inside its own directory.

For example:

python Perceptron/perceptron.py

python ADALine/ADALine.py

python Hebb/hebb.py

python McCulloch-Pitts/McCulloch-Pitts.py

The exact runtime requirements may vary depending on the implementation and installed Python libraries.

---

📈 Visualization

Several experiments include image and visualization files to help demonstrate the behavior and results of the implemented models.

The img/ directories contain visual materials associated with the corresponding experiments.

---

🎓 Academic Context

This repository represents practical experimentation with foundational concepts in:

• Artificial Neural Networks
• Machine Learning
• Computational Intelligence
• Pattern Recognition

The implementations are intended to complement the theoretical study of classical neural network models.

---

🚀 Possible Future Improvements

Potential improvements for this repository include:

• Add detailed mathematical explanations for each model
• Standardize the implementations using a common interface
• Add automated tests
• Add reproducible datasets
• Add experiment configuration
• Improve result visualization
• Add training metrics and learning curves
• Compare the models on common datasets
• Add notebooks demonstrating each algorithm interactively
• Add modern neural network implementations for comparison

---

👨‍💻 Author

Ali Valizadeh

Python Developer · Django · AI, NLP & Automation · University Instructor

GitHub:

https://github.com/AliValizade