Theorist

Summary

Theorist is a lightweight, zero-dependency Python library for hyperparameter optimization based on a prediction-error loop. Instead of searching blindly or modeling only what works, it forces the optimizer to make a prediction before each experiment and uses the resulting prediction error as the learning signal. This builds a predictive understanding that transfers across tasks and adapts quickly to changing conditions.

Key Points

• Single-file Python implementation with no external dependencies.
• Operates on a predict → run → surprise → learn → transfer loop.
• Cross-domain transfer: knowledge gained from one optimization task accelerates a different task.
• Drop-in replacement for random search or grid search with better sample efficiency.
• Persists experience in a Brain object (~/.theorist/); subsequent tasks start “smarter”.

Concepts

• Prediction-error loop: The core mechanism that forces a guess before each trial, measures the deviation (surprise), and updates its internal model accordingly.
• Brain: The persistent knowledge store that retains learned patterns across experiments and domains.
• Domain: A label that allows the Brain to transfer insights between related but distinct tasks (e.g., training vs. inference).
• compare utility: A helper to run head-to-head comparisons against random search on the same objective.

Details

Theorist implements a first-principles approach to optimization. Most tools (e.g., Optuna, Ray Tune, Bayesian optimization) model what works by fitting a surface over past results. Theorist instead models why it works by maintaining an internal predictive model. Before each evaluation it outputs a prediction for the metric; after observing the true value, it computes the prediction error and updates its model.