A Sydney-based quant fund ran a VaR model calibrated on 2015–2019 data. March 2020 arrived. The model predicted a 1-day 99% VaR of $480,000. Actual loss: $3.1M — 6.5x the estimate. The model wasn't wrong because of bad luck. It was wrong because nobody had stress-tested it against a fat-tailed volatility regime. The firm's risk engine failed silently, and capital evaporated before anyone noticed.
Model risk is the risk of adverse outcomes from decisions based on incorrect or misused models. The U.S. Federal Reserve's SR 11-7 guidance defines it formally: model risk arises from both model error and inappropriate use. APRA CPG 220 extends this to Australian ADIs, requiring documented model inventories, independent validation, and board-level oversight. Most quant shops satisfy none of these three requirements adequately.
Consider what model error actually costs at scale. A risk model with a 5% annual miscalibration on a $10M book produces $500,000 of unrecognised exposure. Compounded across three desks and two years, that's $3M of shadow risk sitting outside your limits framework. Basel III's FRTB rules address this by mandating P&L attribution tests and backtesting exceptions — if a model produces more than 12 backtesting exceptions in 250 trading days, it moves to the punitive Standardised Approach, materially increasing capital requirements.
A governance-gated model lifecycle has four non-negotiable stages: build, independent validate, approve, and monitor. Build documentation must specify assumptions, data inputs, and known limitations — not after the fact, but before deployment. Independent validation means a separate quant team runs out-of-sample tests, sensitivity analyses, and stress scenarios including at minimum one historical crisis period. Approval requires sign-off from a risk committee with documented escalation thresholds. Ongoing monitoring tracks exception rates, input data quality, and model drift quarterly. One practical framework traders study is Value at Risk methodology alongside its well-documented limitations. The formal regulatory architecture is detailed in Basel III capital framework documentation. For Australian firms, APRA's prudential obligations interact directly with how model risk is formally defined and managed across the institution.
A risk engine that has never failed a validation test has never been properly tested. Build the adversarial process into the governance structure — not as a compliance checkbox, but as the mechanism that keeps the model honest.
The model that kills a fund is rarely the one you knew was broken — it is the one everyone assumed was fine.
This content is for educational purposes only and does not constitute financial product advice. Past performance is not indicative of future results. Profit Logic Ltd (ACN 688 669 936) accepts no responsibility for errors or omissions in this content or anywhere on this website. Always seek advice from a licensed financial adviser before making investment decisions.