ML-From-Scratch: Machine Learning Models and Algorithms in NumPy

Summary

ML-From-Scratch is a comprehensive GitHub repository offering bare-bones NumPy implementations of fundamental machine learning models and algorithms. It emphasizes accessibility, making complex concepts easier to understand for learners and practitioners. This project covers a wide range of topics, from linear regression to deep learning and reinforcement learning, all implemented from scratch.

Use at your own risk

OSRepos shares public repositories for knowledge and discovery only. Any installation, execution, configuration, or use of code from these repositories is the user's own responsibility. Always review the repository, source code, dependencies, licenses, and security implications before running or installing anything. OSRepos is not responsible for issues, damages, or losses resulting from third-party repositories.

Introduction

The ML-From-Scratch repository by eriklindernoren is an exceptional resource for anyone looking to understand the core mechanics of machine learning. It provides Python implementations of fundamental machine learning models and algorithms, built entirely from scratch using NumPy. The primary goal of this project is not optimization, but rather to present the inner workings of these algorithms in a transparent and accessible way, making it an invaluable educational tool. From supervised and unsupervised learning to deep learning and reinforcement learning, this repository offers a hands-on approach to mastering machine learning concepts.

Installation

To get started with ML-From-Scratch, follow these simple steps:

$ git clone https://github.com/eriklindernoren/ML-From-Scratch
$ cd ML-From-Scratch
$ python setup.py install

Examples

The repository includes numerous examples demonstrating the practical application of the implemented algorithms. Here are a few highlights:

Polynomial Regression

Explore how a regularized polynomial regression model fits temperature data.

$ python mlfromscratch/examples/polynomial_regression.py

Polynomial Regression Example

Classification With CNN

See a Convolutional Neural Network (CNN) in action, classifying the digit dataset.

$ python mlfromscratch/examples/convolutional_neural_network.py

+---------+
| ConvNet |
+---------+
Input Shape: (1, 8, 8)
+----------------------+------------+--------------+
| Layer Type           | Parameters | Output Shape |
+----------------------+------------+--------------+
| Conv2D               | 160        | (16, 8, 8)   |
| Activation (ReLU)    | 0          | (16, 8, 8)   |
| Dropout              | 0          | (16, 8, 8)   |
| BatchNormalization   | 2048       | (16, 8, 8)   |
| Conv2D               | 4640       | (32, 8, 8)   |
| Activation (ReLU)    | 0          | (32, 8, 8)   |
| Dropout              | 0          | (32, 8, 8)   |
| BatchNormalization   | 4096       | (32, 8, 8)   |
| Flatten              | 0          | (2048,)      |
| Dense                | 524544     | (256,)       |
| Activation (ReLU)    | 0          | (256,)       |
| Dropout              | 0          | (256,)       |
| BatchNormalization   | 512        | (256,)       |
| Dense                | 2570       | (10,)        |
| Activation (Softmax) | 0          | (10,)        |
+----------------------+------------+--------------+
Total Parameters: 538570

Training: 100% [------------------------------------------------------------------------] Time: 0:01:55
Accuracy: 0.987465181058

CNN Classification Example

Generating Handwritten Digits with GANs

Witness a Generative Adversarial Network (GAN) learning to generate handwritten digits.

$ python mlfromscratch/unsupervised_learning/generative_adversarial_network.py

GAN Handwritten Digit Generation Example

Deep Reinforcement Learning

Observe a Deep Q-Network solving the CartPole-v1 environment from OpenAI gym.

$ python mlfromscratch/examples/deep_q_network.py

Deep Q-Network CartPole Example

Why Use ML-From-Scratch?

This repository is ideal for:

Deepening Understanding: By implementing algorithms from scratch, it offers unparalleled insight into their mathematical foundations and operational mechanisms.
Educational Purposes: It serves as an excellent learning resource for students and self-learners in machine learning, data science, and artificial intelligence.
NumPy Proficiency: It's a great way to improve your NumPy skills by seeing how complex algorithms are built using its core functionalities.
Comprehensive Coverage: It spans a broad spectrum of machine learning paradigms, including supervised, unsupervised, reinforcement, and deep learning.