mini-torch is a minimal deep learning framework built from scratch in Python. It implements reverse-mode automatic differentiation, a dynamic computation graph, and a basic neural network training pipeline.
Every computation builds a graph of Tensor objects. Each Tensor stores its data, gradient, and a local backward function. During backpropagation, gradients flow backward through this graph using the chain rule.
Tensor is the fundamental data structure. It represents both values and nodes in the computation graph.
class Tensor:
def __init__(self, data):
self.data = np.array(data, dtype=float)
self.grad = np.zeros_like(self.data)
self._backward = lambda: None
self.children = ()
Every operation between Tensors creates a new Tensor and defines how gradients should flow back to its parents.
mini-torch uses reverse-mode automatic differentiation. During the forward pass, the computation graph is built dynamically. During the backward pass, the graph is traversed in reverse topological order.
loss.backward()
This computes gradients for every Tensor that contributed to the loss.
Neural networks are built by composing Tensors. Parameters are ordinary Tensors that are updated by an optimizer.
x = x @ W + b
x = x.relu()
x = x @ W2 + b2
There is no special parameter type. Any Tensor can be optimized.
mini-torch includes stochastic gradient descent with momentum. The optimizer updates parameters using their gradients after backpropagation.
optimizer.zero_grad()
loss.backward()
optimizer.step()
mini-torch prioritizes clarity over performance. It intentionally avoids advanced features like broadcasting in backward passes or GPU acceleration.
The goal is to make the mechanics of modern deep learning frameworks explicit and understandable.