How to vet any stock-alert service: the 7 checks most services fail
Most stock-alert services show you a wall of winning trades and ask you to trust it. Here are seven checks that separate a service telling the truth from one selling you a screenshot — every one you can apply yourself, for free, before you pay a cent.
Why the wall of green screenshots means nothing
A screenshot proves one thing: that a trade once looked good on someone's screen. It doesn't prove the trade was taken, held, exited where they say, or that the ten losers around it were shown to you. Three tricks do almost all the damage:
Survivorship. Post the winners, quietly delete the losers. The record looks 90% accurate because the other 90% was never published.
Hindsight (look-ahead) rules. "Our system would have caught this" is a claim made after the move happened. A rule invented to fit past winners always looks brilliant on the data it was built from and falls apart on anything new.
Un-realizable fills. "Sold at +40%" — at a price that printed for two seconds on a stock you couldn't actually get filled on. The advertised return and the return you'd have banked are different numbers.
Regulators have noticed. A December 2025 SEC exam alert flagged cherry-picked and misleading performance among the most common deficiencies it finds, and FINRA has issued six-figure fines over misleading influencer promotion. But enforcement moves slowly and mostly touches registered firms — so for now, protecting yourself comes down to knowing what to look for.
The 7 checks
1 · Were the picks published before the outcome was known?
The one that matters most. A real record timestamps every call before the market can prove it right or wrong. If the "record" is a summary posted after the fact, it's marketing, not evidence. Look for a public, dated log you can scroll back through — not a highlights reel.
✓ Green flag: every pick has a timestamp earlier than its result.
✕ Red flag: results described in the past tense with no pre-dated entry.
2 · Are the losers shown right next to the winners?
An honest service publishes the full distribution — every pick, graded, winners and losers alike. If you can only find wins, you're not looking at a track record; you're looking at the survivors.
✓ Green flag: a complete log where the worst calls sit next to the best.
✕ Red flag: "featured trades," testimonials, and no way to see the ones that failed.
3 · Is a "win" defined as a return you could actually get?
Ask how a win is scored. The honest definition is a realizable result: enter at a price you could actually get (the next open, not the pre-market spike), exit at a real price, and subtract trading costs and slippage. The dishonest definition is "the stock touched +20% at some point" — which you could almost never have captured.
✓ Green flag: wins are open-to-close or exit-rule based, net of costs.
✕ Red flag: wins are "ever touched +X%" — an untradeable high-water mark.
4 · Were the rules fixed in advance — or fitted to the past?
Any strategy can be made to look perfect on the data it was designed against. The only honest test is out-of-sample: register the rule with a date, then judge it only on picks that came after. If a service can't tell you when a rule was frozen, assume it was reverse-engineered from its own winners.
✓ Green flag: rules are dated and scored only on later picks.
✕ Red flag: "backtested" results with no forward, out-of-sample record.
5 · Is there enough of a sample to mean anything?
Ten winning trades prove nothing — a coin can flip heads ten times. You want a record with a real number of picks behind it — our own rule of thumb is at least a couple hundred graded picks (n > 200) — before you trust a win rate or an average return.
✓ Green flag: a track record with hundreds of graded picks.
✕ Red flag: a dazzling percentage built on a dozen trades.
6 · Is the record immutable — or could it be quietly edited?
Ask: could they delete an embarrassing pick tomorrow? If the log lives somewhere they fully control and can rewrite, the past is negotiable. The strongest form is an append-only log — new picks can be added, but nothing already logged can be changed or removed.
✓ Green flag: an append-only, never-edited log; grading done by an automated job, not by hand. Strongest form: a published hash chain, so any edit to history is provable by a stranger — not just promised.
✕ Red flag: a curated page that can be reshaped at will.
7 · Can a total stranger verify it without trusting anyone?
The final test wraps up the other six: could someone who has never heard of this service, and has no reason to believe it, confirm every claim from public data? If verification requires taking their word for anything, it isn't verification.
✓ Green flag: raw data published; numbers re-derivable by anyone.
✕ Red flag: "trust our verified results" with nothing to check.
Three more checks for AI trading agents
A new kind of record is being sold now: an AI model that reads the news and the numbers, decides, and builds a track record. The seven checks above still apply, and the best AI-agent platforms already pass several of them (pre-registration, forward-only, a log they can't edit). These three are where the AI version of the record most often comes up short.
8 · If many strategies or models are running, does the record correct for it?
Run fifty strategies across five AI models and a few will look excellent by luck alone. Showing you the best one, or letting you pick the winner off a leaderboard, is survivorship with extra steps. An honest platform says how many were tried and adjusts for it, or shows the whole field so you can see where the winner sits.
✓ Green flag: the number of strategies tried is published, and results are corrected for it (or every strategy is shown, ranked).
✕ Red flag: a leaderboard of top performers with no count of how many were run. Why the count matters →
9 · Is there a confidence interval, or just a Sharpe ratio?
A Sharpe ratio, win rate or return over a few months is one draw from a very noisy process. Over 90 trading days, an annualized Sharpe ratio has a standard error of roughly 1.7 or more, so a 90-day Sharpe of 1.4 is statistically compatible with zero. A number without its uncertainty can't separate skill from luck, however long the waiting period before it was shown.
✓ Green flag: every headline number comes with an interval, and the record says plainly when the interval still includes zero.
✕ Red flag: a precise-looking Sharpe or return at 60 or 90 days, with no interval.
10 · Is the AI's explanation being sold as proof?
Seeing the model's reasoning for each trade is useful for auditing what it did. It is not evidence that the reasoning caused the result. Research on large language models finds their stated reasons are often not the real basis for their outputs, and their stated confidence runs high. The question that matters is whether the model's calls beat the simple rule it was layered on, forward, with an interval.
✓ Green flag: the AI is graded against the same strategy run without it (the plain rule), and its stated confidence is checked against how often it was right.
✕ Red flag: "read the model's reasoning" offered in place of a comparison against the simpler alternative.
family_verdict.py): three experiments locked, none survived it, and the rest are corrected the same way when they lock. #9: yes — every verdict carries a 95% interval built on the right independent unit: the company for the stock-picking tests, the monthly cycle or the week for the calendar tests, never the single trade. #10: we don’t publish an AI’s calls yet. When we do, it will be graded against the plain score it sees and checked for calibration, exactly as described above, before any of its output is shown as more than a test.
The 60-second version
- Picks timestamped before the result?
- Losers shown next to winners?
- "Win" = a return you could actually get, net of costs?
- Rules frozen with a date and judged only on later picks?
- Hundreds of picks, not a dozen?
- Log is append-only and machine-graded?
- A stranger could verify it all from public data?
If it’s an AI trading agent, add:
- Corrected for how many strategies or models were run?
- An interval, not just a Sharpe?
- AI graded against the plain rule, not on its explanations?
Seven yeses: rare, and worth your attention. Any hard no on #1, #2, or #7: keep your money.
Run these seven against a service and get the verdict → Two minutes, no account, nothing leaves your browser.
Why we wrote this — and where we stand on it
We run ThePickLog, a research project built around one uncomfortable rule: every pick is logged before the open and graded automatically five trading days later — winners and losers alike — in a log we can't edit. We hold ourselves to all seven checks above, publicly, including the ones that make us look bad. We also publish our own failures of that discipline: in August 2026 we found our scanner had logged 128 picks after the opening bell and wrote up the correction. Our own honest answer right now is that bought-and-held, the picks as a group have lost money — and we publish that, because a record you can only trust when it's flattering isn't a record at all.
We think the whole category should be held to this standard, so we wrote it down as an open one. If you run an alert service and you're one of the honest ones tired of being lumped in with the screenshot factories, the standard is here — the same yardstick we grade ourselves against.
Either way: run the seven checks on us too. That's the point.
Educational / informational only. Not investment advice, not a recommendation to buy or sell any security. Low-priced and low-float stocks are highly volatile and carry a substantial risk of loss. Past performance does not predict future results.
About
Evidence Validation · Score & metrics · Trust & Data
Tools Calculator · Vetting Guide · 7-Check Scorecard
Legal Disclaimer · Privacy · Terms