alert backtest — fires, forward moves and false-signal rate, no look-ahead

Alert backtest: what price did after each fire

Take a price or indicator recipe — "RSI crosses below 35", "price crosses up its 20-day average" — and AtlasVector replays it bar by bar over history. A "fire" is counted once, when the condition flips from false to true — the same way a live alert pings you once, not on every bar the condition stays true. The forward move is then measured over the next 1 / 5 / 21 bars: hit rate, average forward move, and how often the fire reverses (the false-signal rate). Nothing that happened after a fire is used to decide the fire. The replay runs on real price history only.

AAPL⚑ price crosses down its 20-day average

⚑ The recipe: price crosses down its 20-day average — the price breaks below its 20-day moving average — a momentum-rolling-over / breakdown trigger. expects price to fall

TOO FEW FIRES · triggered 9× over 260 bars

5-bar hit rate 33% [12–65%]avg forward move +1.00%median +1.09%edge vs baseline +0.32%false-signal rate 89% [56–98%]

Fires9

Hit rate33%[12–65%]

Avg move (5b)+1.00%

False signals89%[56–98%]

What happened after each fire, by horizon

HorizonResolved firesHit rateAvg moveMedianFalse signal

1-bar956% [27–81%]-0.60%-0.01%44% [19–73%]

5-bar · headline933% [12–65%]+1.00%+1.09%89% [56–98%]

21-bar850% [22–78%]-0.22%-0.88%100% [68–100%]

A fire is only scored once the later bar exists; outcomes that have not happened yet are dropped, never guessed. A hit rate near 50% is indistinguishable from chance; meaningfully above it, in the direction the recipe expects, is the start of a measurable edge.

Would arming this alert have been worth it? We replay the exact recipe over real daily bars with NO look-ahead, de-dupe fires on the false→true edge, then measure the forward move over 1/5/21 bars after each fire — hit rate, average move, edge vs an unconditional baseline, and the whipsaw (head-fake) rate — with a desk read. Your decisions, your risk.

Retail Pro · $99 a month⚑ Arm this recipe as a live alert

The exact recipe replayed above — price crosses down its 20-day average — arming this recipe as a live alert on your names, checked on every bar, and kept under your name. That is the seat’s. The backtest is complete without it.

Start checkout →

This bearish alert fired 9×; over 5 bars it went your way 33% of the time (avg +1.00%), head-faking 89% of the time. The 95% interval on that hit rate is 12–65% — 9 resolved fires is under the 20-observation floor this product publishes a rate on, so no direction is claimed.

How often does it fire?

Your alert fired 9 times over 260 bars.

We replay your exact condition bar by bar. A "fire" is counted once at the moment the condition flips from false to true — the same way a live alert de-dupes so a condition that stays true for ten bars pings you once, not ten times. Every rate below is a share of those fires, so the fire count IS the sample size: 20 resolved fires is the floor this product publishes a rate on, the same floor its graded call record uses.

→ Only 9 resolved fires — under the 20-observation floor, so the rates below are shown with their intervals and claim no direction.

Did the move go your way? (5-bar hit rate)

Over the next 5 bars after each fire, price moved in the expected (down) direction 33% of the time (95% interval 12–65%), averaging +1.00% (median +1.09%).

Hit rate is the share of fires where the forward return's SIGN matched what the trigger implies — a "bearish" alert wants price to fall. A coin flip is 50%, and the point estimate alone cannot tell you which side of it you are on: the 95% Wilson interval is the range of true hit rates this many fires is compatible with, so an edge is only claimed when the WHOLE interval clears 50%. Mean can be skewed by one big move, so we show the median too.

→ 9 resolved fires is under the 20-observation floor this product publishes a rate on, so no direction is claimed.

How often does it head-fake? (false-signal rate)

89% of fires went against you at some point within 5 bars before the horizon was up (95% interval 56–98%).

False-signal rate measures the "it triggered, then immediately reversed" experience — the share of fires where, at any bar inside the horizon, price moved adversely to your intent. It is a rate off the same handful of fires, so it carries the same 95% interval and the same rule: "whippy" is only asserted when the whole band sits above 50%.

→ 9 resolved fires is under the 20-observation floor this product publishes a rate on, so no whipsaw call is claimed.

Glossary

Fire: One moment the alert condition flips from false to true; counted once per flip, like a de-duped live alert.
Hit rate: Share of fires whose forward-return sign matched the alert's directional intent. 50% is a coin flip.
Forward return: close[t+h] / close[t] − 1 — the percentage price change over the next h bars after a fire.
False-signal rate: Share of fires that moved adversely to your intent at any bar within the horizon (whipsaw).
Horizon: How many bars forward we measure the outcome (here 1, 5, and 21 bars).
95% interval: The Wilson score band around a rate: the range of true rates this many fires is compatible with. A direction is claimed only when the whole band clears 50% and the sample clears 20 resolved fires.