Sample size and variance

After this lesson you can estimate how much evidence a trading claim needs — and you'll know why almost every claim you'll ever hear falls short of it.

The school's charter

The schools before this one handed you a method. This one is about checking things: your own results and your own backtests, and anything being sold to you by someone who wants money for it. The tools are a handful of statistical ideas taught with trading numbers, and we start with the one School VII kept promising — how big a sample has to be before it means anything.

The coin-flip yardstick

Take a system that truly wins 55% of the time, which is a good real-world number, and run stretches of it. Over 10 trades, anywhere from 2 to 9 wins turns up routinely, so a 10-trade result is compatible with nearly any underlying truth. Over 100 trades the observed rate mostly lands between 45% and 65%. That's better, and it's still wide enough that a genuinely losing system at 48% and your good one overlap heavily. The fog only thins properly once you're into the several hundreds.

The rule behind those numbers is that statistical noise shrinks with the square root of the sample, so four times the trades buys only twice the precision. Precision is expensive, and that costs you in both directions. Your own 30-trade capstone campaign, journaled to perfection, still supports only soft conclusions; School V's "change one thing" rule was calibrated to exactly this. Going the other way, a service advertising 20 documented wins is showing you a sample a coin could plausibly produce.

Expectancy needs even more care than win rate, because R-multiples have outliers. One +8R trade in a set of 30 shifts the average by more than a quarter-R, and course 4 comes back to that. There's also a subtlety that matters more in trading than in most places. Trades close together in time share a regime (School IV), so 60 trades from one good quarter carry far less independent evidence than 60 spread across two years. Count in market conditions covered rather than in trades. A system that has only traded one regime hasn't yet been tested against the others, and School VI, course 6 told you when you find out.

Living with it

At retail trade frequencies you'll spend your whole career under some amount of statistical fog, never fully certain the edge is real. That's uncomfortable, and it's also the ordinary condition of the job. What this school builds over its six courses is a way of working inside it: stack different kinds of evidence instead of waiting for any one kind to become conclusive. A mechanism that makes sense (School III's standard), honest backtests across regimes (course 2), live paper results that match them (course 3), and process discipline that rules out the self-inflicted explanations (School VII). No one of those settles the question on its own, but together they support the working confidence a career runs on. The risk rules from Schools II and VI were sized for the same fog: you cap the bad year because you can never be sure there isn't one coming.

Check yourself

1. A friend shows 8 wins in 10 trades. What can you conclude? (Almost nothing — a mediocre system produces that stretch routinely. Ask for hundreds, across conditions.)
2. Why do 60 trades from one quarter overstate their own evidence? (They share one regime — the effective sample of market conditions is nearly one.)
3. Why does the fog justify the risk rules rather than excuse abandoning them? (Permanent uncertainty means the bad stretch can always be ahead — so every dial stays sized for it.)

The idea this lesson installs

Count evidence in regimes covered, not trades taken.

Next: Course 2 — "Backtests that lie," featuring a confession of ours.