Every experiment, and how each one is scored
Retail trading runs on claims that are almost never checked: strategies with impressive win rates, screenshots of the good days, backtests fitted to the past. So we take a claim, write the rule down with a date before we look, grade it forward on public data, and publish the verdict — including when the answer is embarrassing.
Rule frozen before the fact • Graded forward, winners and losers • Recomputable from public data • Published either way
Verdict calendar
Every date below was fixed before the data it judges existed. None moves because of what the data shows.
| 2026-11-02 | Verdict date for 02, 03 and 06–09, corrected as one family. Read: 03, 07 and 09 inconclusive (no edge detected); 02, 06 and 08 had not met their sample floors and lock automatically when they do |
| ~2027-01 | Experiment 05, first read (attribution, not a tradeable edge) |
| on/after 2027-01-11 | Experiment 12, the AI-judge study, one look |
| ~2027-09 | Experiment 04, the turn-of-month effect |
What a verdict is allowed to say — the five outcomes
“Failed” is three different findings wearing one word, and the difference decides what you should do next. A claim that was tested properly and lost is not the same as one the data was never able to answer. These are the only five things we let ourselves conclude, and every one of them gets published.
One rule holds all five together: the label is chosen by the interval we published in advance, not by how the number reads afterwards. Worked example, including the arithmetic: Gate 1. The CORRECTION tag further down is not a sixth verdict — it marks a field note about a mistake in our own apparatus, which is a different kind of finding from a verdict on a claim.
Our own low-float momentum screen
We started with ourselves. We built a screener for low-float stocks gapping up on heavy volume, pre-registered the rule we believed in, and ran it forward for seven weeks. It lost — significantly, and in the opposite direction to the one we predicted. Here is the whole thing, including the number that flipped sign as the sample grew.
EXPERIMENT 02RUNNINGThe 2-period RSI "high win rate" mean-reversion trade
One of the most widely published retail strategies of the last twenty years: buy a large-cap when the 2-day RSI collapses while it's still above its 200-day average. It is usually sold on its win rate. We are testing whether a high win rate survives contact with expectancy — on liquid names, where trading costs can't be blamed for the answer.
EXPERIMENT 03INCONCLUSIVEThe MACD bullish crossover
Arguably the most widely taught indicator signal in retail trading — it ships on every charting platform and appears in essentially every beginner course. Which makes it the least likely thing in the world to still contain an edge, and exactly the sort of universally-believed claim nobody bothers to check. Registered prior: no edge, roughly 1 in 6 that it clears.
EXPERIMENT 04RUNNINGThe turn-of-month effect
A 40-year-old claim from the academic calendar-anomaly literature, still sold today in seasonality newsletters: returns cluster in the last few and first few trading days of the month. It was the strongest survivor of our 25-test calendar sweep — in which, corrected for multiple testing, nothing survived. So it gets the only test that counts: forward, window frozen in advance, on QQQ.
EXPERIMENT 05RUNNINGOvernight vs intraday — the market that only pays while it's closed
Over 27 years, essentially all of QQQ's return came between the close and the next open; the hours the market was actually open made nothing. Firms sold this as "night effect" ETFs, and the ETFs died. We registered it forward as an attribution claim — where returns happen, explicitly not a tradeable edge — with each day's intraday leg as its own control.
EXPERIMENT 06RUNNINGThe Supertrend flip
The most heavily marketed single indicator in retail video content, tested exactly as it is sold: default settings (10, 3), no trend filter. On forty of the most liquid stocks on earth, a signal everyone can see should already be arbitraged flat. Registered prior: no edge, roughly 1 in 6 that it clears.
EXPERIMENT 07INCONCLUSIVEThe moving-average pullback
Buy a stock in a long-term uptrend when it dips to its 20-day average. Probably the most widely taught swing entry in existence — and the most coherent mechanism in this batch, which earns it the highest prior: still no edge expected, roughly 1 in 5.
EXPERIMENT 08RUNNINGBollinger Band mean reversion
Buy the close below the lower band while the stock is above its 200-day. Sold, like Experiment 02, on its win rate — and win-rate-flattering by construction: many small reverts, occasional large losses. Tested on expectancy instead. Registered prior: roughly 1 in 6.
EXPERIMENT 09INCONCLUSIVEVolatility contraction (NR7) in an uptrend
That a quiet day precedes a big day is documented. That the big day goes up is the part that is sold — and the part with no documented edge. One disclosed deviation: we enter at the next open, not on an intraday breakout. Registered prior: roughly 1 in 6.
EXPERIMENT 12RUNNINGCan an AI judge forecast a drawdown better than four numbers?
AI that reads the filings and tells you what a stock will do is now heavily sold to retail traders. Two pinned AI judges — a filing reader and a general model from a different company — graded forward on one question: do they forecast a 20% drawdown better than the plain score they both see? Registered prior for the real test: about 1 in 4 per judge.
Field notes — the mistakes, written up
Errors we found in our own record. Both of these made our numbers look better than they were, and both are written up in full.
FIELD NOTEMETHODThe number nobody reports: how many strategies did you test before this one?
Mine 200 zero-edge strategies over 30 trades and the median best "win rate" is 73%. A hit rate you are shown cannot be read without two numbers you are almost never shown — how many candidates were searched, and how many trades each one got. We ran the arithmetic, checked our own first instinct (it was wrong), and surveyed four backtesting tools for either number.
FIELD NOTECORRECTIONWe logged 128 picks after the opening bell
The whole record rests on picks being written down before the market opens. On seven occasions our scheduler drifted past the bell and logged anyway, and we didn't notice for six weeks — an outside reader found it in our public CSV. What broke, what it did to the numbers, why we won't claim it flattered us, and the gate that makes it impossible now.
FIELD NOTEThe bug that makes your backtest look good — and why you can't see it
Historical prices are silently rewritten every time a company splits. Mix a price you stored with one you download later and you invent returns that never happened — and in cheap stocks the error is always in your favour. We found it in our own code, where it had been inflating our numbers for weeks. How it works, why it survives review, and the six checks that catch it.
How an experiment works here
- The claim gets written down first. Rule, universe, entry, exit, cost assumption, and the exact bar it has to clear — all frozen with a date, in a public file, before any of the data it will be judged on exists.
- It runs forward only. No backtest ever counts as evidence. Only outcomes that happen after the registration date are scored.
- It is graded against a control. "It made money" is not a result. The question is always whether it beat the honest alternative — doing the simple thing instead.
- The statistics are done properly. Confidence intervals, a correction for the fact that repeated bets on the same names are not independent, and an explicit count of how many results you'd expect to look good by chance alone.
- The verdict is published either way. That is the whole point. A method we hoped would work and didn't is a more useful result than one more success story.
What should we test next?
- If there's a strategy, indicator or claim you keep seeing sold — and you've never seen anyone check it honestly — tell us and it goes in the public queue. We pick from it.
- Two rules: it has to be specific enough to write down as a rule before the fact, and we test techniques, not people.
We are not selling signals, and nothing here is advice. The output of this project is verdicts on public claims — including our own, which is where we started and where the record is worst.
Get every verdict by email
Verdicts land on dates fixed in advance; each experiment page shows its date. We'll email you each one, free, the day it publishes — and nothing else. No signals, no picks, no offers. Verdicts stay free and public for everyone either way; this just means you don't have to remember to check back.
One confirmation email first — you're not on the list until you click it. Privacy
Rules are frozen in HYPOTHESES.md; every verdict is recorded in the audit log; the raw data is downloadable from the track record. Method and standards: the method · principles.
Educational and informational only — not investment advice, and not a broker-dealer.
Evidence Validation · Score & metrics · Trust & Data
Tools Calculator · Vetting Guide · 7-Check Scorecard
Legal Disclaimer · Privacy · Terms