Reimplementation of the scalar-value backpropagation library, karpathy/micrograd, but in Ruby and WITH TESTS!
Follow along with karpathy's video here: https://www.youtube.com/watch?v=VMj-3S1tku0
Here's an example of a backprop on a single perceptron found in /examples/perceptron_auto_backprop.rb:
Modern neural networks rely on automatic differentiation.
Frameworks hide the details, but under the hood the idea is straightforward:
- Build a computational graph while executing math operations
- Track dependencies between values
- Run backpropagation to compute gradients
micrograd_rb shows exactly how this works with a tiny codebase.
No magic.
Just graphs and the chain rule.
Minimal scalar-based autograd engine
Computational graph construction
Reverse-mode backpropagation
Common operations:
- addition
- multiplication
- tanh activation
Graph visualization examples
Full test coverage
Small codebase (~100 lines core logic)
graph LR
O["O | grad:1.0000 | data:0.7071"]
Otanh(("tanh"))
Otanh --> O
N["N | grad:0.5000 | data:0.8814"]
Nplus(("+"))
Nplus --> N
X1W1X2W2["X1W1X2W2 | grad:0.5000 | data:-6.0000"]
X1W1X2W2plus(("+"))
X1W1X2W2plus --> X1W1X2W2
X1W1["X1W1 | grad:0.5000 | data:-6.0000"]
X1W1mul(("*"))
X1W1mul --> X1W1
X1["X1 | grad:-1.5000 | data:2.0000"]
W1["W1 | grad:1.0000 | data:-3.0000"]
X2W2["X2W2 | grad:0.5000 | data:0.0000"]
X2W2mul(("*"))
X2W2mul --> X2W2
X2["X2 | grad:0.5000 | data:0.0000"]
W2["W2 | grad:0.0000 | data:1.0000"]
B["B | grad:0.5000 | data:6.8814"]
N --> Otanh
X1W1X2W2 --> Nplus
X1W1 --> X1W1X2W2plus
X1 --> X1W1mul
W1 --> X1W1mul
X2W2 --> X1W1X2W2plus
X2 --> X2W2mul
W2 --> X2W2mul
B --> Nplus
Clone the repository:
git clone https://github.com/kocsisbalazs/Ruby-Micrograd
cd micrograd_rb