The First Law of Complexodynamics

The First Law of Complexodynamics is Scott Aaronson’s blog-post exploration of a puzzle you can see in your coffee cup: physical systems start simple, become intricate and interesting, then fade into featureless equilibrium — complexity rises, peaks, and falls, even though entropy increases the whole time.

This is the accessible entry point to a two-page story. The formal version, with an actual theorem, is The Coffee Automaton. The measurement machinery comes from Kolmogorov Complexity.

Why Students Should Care

It sharpens a distinction students constantly blur: entropy is not complexity. Maximum-entropy states are boring, not intricate.
Defining “interesting structure” rigorously is surprisingly hard — and the candidate definitions here (complextropy, sophistication) connect directly to compression and MDL.
The rise-then-fall arc is a suggestive lens for thinking about learning dynamics in neural networks.

The Puzzle

The Second Law of Thermodynamics says entropy never decreases:

S(t_2) \geq S(t_1) \quad \text{for } t_2 > t_1

If “more entropy” meant “more complex,” the universe would just get steadily more complex forever. But look around: complexity behaves differently. A freshly shuffled deck isn’t complex — it’s random. A factory-ordered deck isn’t complex — it’s trivially simple. Complexity peaks somewhere in between order and randomness. Why? And can we define “complexity” so that statement is even meaningful?

The Coffee Example

Consider cream being poured into coffee:

Time	State	Entropy	Complexity
t=0	Separated layers	Low	Low
t=mid	Swirling patterns	Medium	High
t=∞	Uniform mixture	High	Low

The intricate swirls at mid-mixing are more “complex” than either extreme: the initial state is trivially describable, and the final state is just noise.

Defining Complexity

Here is the hard part. Plain Kolmogorov complexity fails: it assigns its highest value to the fully mixed random state, which is exactly the state we want to call boring.

Aaronson proposes complextropy: roughly, the length of the shortest efficient program that outputs a set (or distribution) within which the observed state looks random. For a string $x$ :

\text{Complextropy}(x) = \min_{S: x \in S} K(S)

where $K(S)$ is the Kolmogorov complexity of the set $S$ , and $x$ must look random within $S$ .

The idea in words: describe the pattern, and let the noise be noise. You pay only for the structure. Ordered states need a tiny pattern description; random states fit the trivial pattern “any string of this length”; only the in-between swirls force a long pattern description.

Interactive Demo

Watch complexity rise and fall as a system evolves:

Complexodynamics

t = 0

Low Entropy (Ordered)High Entropy (Random)

Entropy

1.000

Complexity

0.102

The Key Insight

Entropy increases monotonically. Complexity peaks at intermediate times— neither fully ordered nor fully random states are complex.

Why Complexity Peaks

At $t=0$ : simple description (“all black on left, all white on right”)

At $t=\text{mid}$ : complex description (must specify the intricate mixing patterns)

At $t=\infty$ : simple description (“random noise” or “uniform distribution”)

Both endpoints are cheap to describe — for opposite reasons. Only the middle is expensive.

The Sophistication Connection

Kolmogorov’s own notion of sophistication formalizes the same instinct:

\text{Soph}(x) = \min \{K(S) : x \in S, K(x|S) \geq \log|S| - O(1)\}

You do not need to memorize the condition. It says: find the simplest set that contains $x$ as a typical (incompressible-within-the-set) member; the complexity of that set is the sophistication of $x$ . It is the “structure only, noise excluded” measure — the Coffee Automaton paper later uses it to prove the rise-and-fall rigorously.

Implications for AI

Deep learning training may follow a similar arc:

Early training: simple patterns (high loss, low complexity)
Mid training: complex intermediate features
Late training: simplified, generalizable representations

This is an analogy rather than a theorem — but it is one reason this blog post appears on ML reading lists.

Common Confusion

Entropy vs. complexity: entropy measures disorder and only goes up; complexity (complextropy/sophistication) measures structure and peaks in the middle. The uniform mixture has maximum entropy and near-zero complexity.
Random is not complex: a random string is incompressible (high Kolmogorov complexity) but has no structure worth describing (low sophistication). The two measures deliberately disagree here.
“First Law” is a playful name: this is a blog post proposing a conjecture and a definition, not an established law of physics.
This page vs. Coffee Automaton: this post asks the question and proposes complextropy; the paper builds a concrete model and proves the rise-and-fall theorem.

Where To Go Next

Read The Coffee Automaton for the formal model and the main theorem.
Read Kolmogorov Complexity for shortest-description reasoning from the ground up.
Read MDL Tutorial for the practical “structure vs. noise” trade-off in model selection.
Read MDL Weights for description-length ideas applied to neural networks.

Key Resource

The First Law of Complexodynamics — Scott Aaronson
https://scottaaronson.blog/?p=762