Vectorised backtesting: signal to position to P&L

A backtest is four arrays — signal, position, return, cost — and the entire difficulty is in the alignment between them. Written vectorised, the lag is a visible line of code that a reviewer can check; written as a loop, it is an assumption nobody can see.

Stage	Operation	The trap
Signal	Cross-sectional z-score per date	Full-sample normalisation
Position	Scale to a risk target, then lag	Forgetting the lag
Gross P&L	Position times forward return	Using the same-period return
Turnover	Absolute change in position	Ignoring it entirely
Costs	Turnover times a cost model	A fixed cost per share
Net P&L	Gross minus costs	Reporting only the gross

Table 5.1 · The pipeline. Every row has a trap and every trap inflates the result. A backtest that has not been checked against all six is optimistic by default, not by accident.

import numpy as np
import pandas as pd

def backtest(signal: pd.DataFrame, returns: pd.DataFrame,
             cost_bps: float = 5.0, target_vol: float = 0.10) -> pd.Series:
    """signal and returns are (dates x assets), aligned and already lagged-safe."""
    # 1. Cross-sectional z-score, within each date only.
    z = signal.sub(signal.mean(axis=1), axis=0).div(signal.std(axis=1), axis=0)

    # 2. Dollar-neutral weights summing to one unit of gross exposure.
    w = z.div(z.abs().sum(axis=1), axis=0)

    # 3. Act tomorrow on what you knew today.
    pos = w.shift(1)

    gross = (pos * returns).sum(axis=1)
    turnover = pos.diff().abs().sum(axis=1)
    cost = turnover * cost_bps / 1e4

    net = gross - cost
    return net * (target_vol / (net.std() * np.sqrt(252)))

Listing 5.2 · The whole backtest. The shift(1) on line 3 is the entire difference between a backtest and a fantasy. Keeping it on its own line, with a comment, is deliberate — it is the line a reviewer should be able to find in two seconds. Time O(dates x assets) · Space O(dates x assets).

How much to lag. One bar is the minimum and rarely the truth. A signal computed from the close cannot be traded at that close, so the honest lag is to the next open at least — and if the data arrives with a publication delay, the lag is that delay rather than one bar. The right question is not "how many periods should I shift" but "when could I first have acted on this", and the answer comes from the data’s release schedule, not from the frequency of the index.

Statistic	Formula	The error people make
Annualised return	Mean daily $\times 252$	Mixing arithmetic and geometric
Annualised volatility	Daily std $\times \sqrt{252}$	Multiplying by 252
Sharpe ratio	Annualised excess return over volatility	Forgetting the risk-free rate
Max drawdown	Largest peak-to-trough on the cumulative curve	Computing it on returns rather than levels
Turnover	Sum of absolute position changes	Counting one side only

Table 5.3 · The statistics, computed correctly. The volatility row is the one that appears in real code. Annualising a standard deviation by multiplying rather than taking the square root inflates it by a factor of nearly sixteen, and the resulting Sharpe is comically small rather than obviously wrong.

Vectorised backtesting: signal to position to P&L

After this lesson you should be able to

The rest of this lesson is in Premium