Methodology

The unit of measurement is a surface: a grid of questions of the form “starting from today’s close, did the market reach a level x% away within n days?” On the NASDAQ workspace the grid runs from −20% to +20% in 1% steps, and from 1 to 30 days ahead.

A level counts as reached if any later high (for a level above) or low (for a level below) touches it, even if the price comes back before the close. Today’s own bar never counts, and only dates with a full window of future data are used. The outcome depends only on prices: a condition never changes an outcome, it only decides which dates are averaged together.

Each condition — a moving-average ratio, an oscillator, a bond yield — is split into ranges by the quantiles of its own history. By default there are ten, each holding about a tenth of the dates. Dates without a value, such as an indicator’s warm-up period, belong to no bin. Lots of tied values can merge edges and leave fewer bins than you asked for.

For each bin, AlphaVerify computes the share of its dates that reached each level within each horizon. Cells with less than 30 dates are left blank, not estimated.

The baseline is the same surface computed over every date. Subtracting it cell by cell gives the bin’s shift: positive (red in the figures) where the level was reached more often inside the bin than over all dates, negative (blue) where less often. Every synthetic market subtracts its own baseline, never the observed one.

A condition that predicts direction should make one side more likely and the mirror side less likely. The score therefore compares each level with its mirror image at the same horizon — +4% with −4% after 10 days, for example — and adds up the absolute differences across the grid. If both sides shift by the same amount, the pair adds nothing.

Each pair is weighted by its distance: a ±2% pair counts twice as much as a ±1% pair. A cell counts only where at least two bins of the same condition have enough data to compare, so a condition cannot score on cells where it does not split the dates.

A synthetic market should look like the real one day by day, but without any memory. AlphaVerify describes each daily bar by four parts — the gap from the previous close, the body from open to close, and the upper and lower wicks — measured in log terms. It estimates their average and how they move together, then draws every synthetic day independently from that distribution and chains the days into prices. Each synthetic market is as long as the real history, and a fixed seed makes the ensemble reproducible.

Within a day, the four parts keep their observed relationships. Across days there are none: no autocorrelation, no volatility clustering, no regimes. In a market like that no condition can predict the next day, so whatever score it earns is down to chance.

Conditions computed from prices, such as a moving average, are recomputed on every synthetic market, with that market’s own bin edges. Conditions taken from outside data — bond yields, the VIX, the dollar index, the calendar — cannot be rebuilt from synthetic prices, so their observed values and edges are reused. Volume is copied from the real history.

Each bin is scored on the real history and on each of the 1,000 synthetic markets. Its p-value is its rank among all 1,001 scores:

The bin clears when p is below 0.05 — equivalently, when its score beats the synthetic 95th percentile. The real history counts as one of the draws, so p is never zero; the smallest possible value is 1/1,001. Ties count against the bin.

On some synthetic markets a bin may have too little data to score, and scores zero. Those runs stay in the count, which makes p a bit smaller than it would be with only the scorable runs. The specification shows the alternative and when it would change a decision; every result records how many synthetic runs were scorable so the gap can be checked.

Selection doesn’t add a new test. It keeps every bin that cleared and only those — no top-k cut, no quota per condition, no minimum effect size — ranked by p-value alone. The order is evidence against chance, not size of effect.

Two numbers give the selection some context without changing it: how many bins would clear by chance if none had any effect (5% of those tested), and a Benjamini–Hochberg q-value for each bin, roughly the share of bins at least this strong that would be false discoveries. With 400 bins tested, even the strongest possible single result (p = 1/1,001) has a q-value of about 0.40.

The last stage asks what the kept conditions say together about the latest bar. A kept bin is active if the latest value falls inside it; from each family of related conditions (one indicator at several periods, say), only the best-ranked active bin is used.

The used conditions are combined by naive Bayes: each multiplies the baseline odds of reaching a level by how much it changes those odds on its own. Rates get smoothed a little so no odds come out infinite. The combined estimate is then compared with the rate history actually shows on dates when all the used conditions held at once.

Naive Bayes is exact only when conditions are independent given the outcome. Conditions that measure similar things count the same evidence more than once and push the estimate further from the baseline than the evidence supports; the one-per-family rule removes only the most obvious overlap.

• A memoryless null. Real markets have memory; the null has none. A condition that tracks persistent volatility can beat it without predicting anything beyond “volatile periods stay volatile.” Clearing means beats a memoryless market, not carries a usable edge. The null also has no liquidity, execution or trading costs.
• Imperfect synthetic bars. Drawing from a normal distribution can occasionally produce an impossible bar, such as a high below the body; the reconstruction does not repair them.
• No multiple-testing correction in the decision rule. A bin clears at a raw p-value below 0.05. Benjamini–Hochberg q-values are reported, but all conditions share the same synthetic markets and the null’s calibration is not established, so they give context rather than a guarantee.
• Resolution. With 1,000 synthetic markets, p-values move in steps of 0.001.
• Combination. The combined forecast assumes conditions are independent given the outcome; overlapping conditions overstate it.
• Scope. Findings hold for the data and period of the workspace that produced them. AlphaVerify is research software, not investment advice.

A null with memory, a block bootstrap for instance, would deal with the first limitation, but it could also keep the very relationship being tested. The specification sets out what any replacement would have to state: what it removes, what it keeps, and how its calibration is checked.