Monograph · matching commit on main

Notes on a two-tape market

What this repository computes, in the notation a microstructure text would use — and only the objects that exist in the C++ engine, the simulator, or the MLX trainer.

This is not a survey of prediction markets. It is a companion to csrc/include/cmf/engine.hpp, cmf/simulator.py, cmf/model.py, cmf/ppo.py, and cmf/policy.py. If a symbol does not appear in those files, it is not claimed here.

Front matter

Two venues print related prices. Call the liquid futures tape the fast stream \(X\) and the 15-minute Polymarket CLOB the slow stream \(Y\). The model’s job is to form a probability that the binary settles up, \(\hat p_t = \mathbb{P}(S_T > S_0 \mid \mathcal{F}_t)\), and to buy a token only when that probability clears the displayed offer.

Convention

Time is discrete with step \(\Delta t = 1\) second in the simulator. A 15-minute market has \(T = 900\) ticks. The agent sees the last \(H = 64\) ticks. Bold letters are vectors. Code names sit in mono after each definition.

1. The lag thesis

Write \(S_t\) for the Binance futures mid and \(Q_t\) for the Polymarket up-token mid. The modelling assumption — planted in the simulator, hoped for in production — is that there exists a delay \(\tau \ge 0\) such that

\[ Q_t \approx \Pi\!\left(S_{t-\tau}\right) + \eta_t, \]

where \(\Pi\) maps a futures path into a binary price in \((0,1)\) and \(\eta_t\) is inventory and reprint noise. In the simulator \(\tau \sim \mathrm{Unif}\{4,\ldots,14\}\) seconds (lag_min, lag_max).

LACUNA concatenated five 18-dimensional snapshots. This repository keeps the two tapes separate and lets each attend to the other. The economically interesting direction is slow attending to fast: the CLOB reading a tape that has already moved.

2. Binary contracts and share PnL

A 15-minute up token pays \(1\) if \(S_T > S_0\) and \(0\) otherwise. The down token pays the complement. With cash \(D\) (dollars) spent at entry price \(p^{\mathrm{in}}\),

\[ n = \frac{D}{p^{\mathrm{in}}}, \qquad \mathrm{PnL} = \bigl(p^{\mathrm{out}} - p^{\mathrm{in}}\bigr)\, n. \]

That is the Phase-4 LACUNA identity, implemented in LagMarket._close. Training then reports the dimensionless return \(\mathrm{PnL}/D\).

Fills

The book is not a mid. Opening and closing use opposite sides:

\[ \begin{aligned} p^{\mathrm{in}}_{\mathrm{UP}} &= a_t, & p^{\mathrm{out}}_{\mathrm{UP}} &= b_t, \\ p^{\mathrm{in}}_{\mathrm{DOWN}} &= 1 - b_t, & p^{\mathrm{out}}_{\mathrm{DOWN}} &= 1 - a_t. \end{aligned} \]

cmf/simulator.py · _open_price, _close_price

At expiry the settlement price is the binary payoff, not the last mid: \(p^{\mathrm{out}}_{\mathrm{UP}} = \mathbf{1}_{S_T > S_0}\).

Remark

Buying at the mid and selling at the mid is how a paper engine prints fantasy. A one-tick round trip here pays the spread, and the tests require \(\mathrm{PnL} < 0\) on open-then-close.

3. Microstructure (C++ engine)

The hot path lives in cmf::FusionEngine. Features are squashed with \(\tanh\) so every coordinate is roughly in \([-1,1]\).

3.1 Mid, microprice, imbalance

\[ \begin{aligned} m &= \tfrac12(b^{(1)} + a^{(1)}), \\ m^{\star} &= \frac{a^{(1)} q^{b,1} + b^{(1)} q^{a,1}}{q^{b,1}+q^{a,1}}, \\ \mathrm{I}_L &= \frac{\sum_{\ell=1}^{L} q^{b,\ell} - \sum_{\ell=1}^{L} q^{a,\ell}} {\sum_{\ell=1}^{L} q^{b,\ell} + \sum_{\ell=1}^{L} q^{a,\ell}}. \end{aligned} \]

cmf::Book::mid, microprice, imbalance · types.hpp

3.2 Order-flow imbalance

After Cont, Kukanov and Stoikov (2014). For one side and one level, if the price rises the arriving size is signed as an addition; if it falls, the previous size is removed; if it is unchanged, the size delta is used. Bid and ask contributions are added at \(L=1\) and \(L=5\): \(\mathrm{OFI}_1\), \(\mathrm{OFI}_5\).

FusionEngine::ofi_side

3.3 Cumulative volume delta

\[ \mathrm{CVD}_t = \sum_{u \le t} \varepsilon_u\, p_u q_u, \qquad \varepsilon_u \in \{+1,-1\}. \]

Acceleration is the last increment, scaled by an EMA of notional. A trade is flagged large when its notional exceeds three times that EMA.

3.4 Kyle’s \(\lambda\)

Over a window of the last \(W \le 16\) ticks, a one-factor regression of mid returns on signed notional:

\[ \hat\lambda = \frac{\sum_i r_i v_i}{\sum_i v_i^2}. \]

This is Kyle (1985) in the cheapest discrete form: price impact per unit signed volume. The feature stored is \(\tanh(50\hat\lambda)\).

3.5 Hawkes intensity

Two self-exciting counters, buy and sell, with decay \(\beta = 1.6\) and jump \(\alpha = 0.55\):

\[ \lambda^{\pm}_{t} = e^{-\beta\Delta t}\,\lambda^{\pm}_{t-} + \alpha\,\mathbf{1}_{\{\varepsilon_t = \pm 1\}}. \]

Flow imbalance is \((\lambda^{+} - \lambda^{-})/(\lambda^{+} + \lambda^{-} + 1)\). The simulator uses the same family with \(\beta = 1.4\), \(\alpha = 0.7\).

3.6 VPIN

Volume is packed into buckets of target size \(8\) times a typical notional. Bucket \(j\) contributes \(|V^b_j - V^s_j| / V_j\). VPIN is the mean of the last \(16\) such imbalances (Easley, López de Prado, O’Hara).

3.7 Realized volatility and book slope

\[ \sigma_n = \sqrt{\frac{1}{n}\sum_{i=1}^{n} r_i^2}, \qquad \mathrm{slope} = \frac{\sum_\ell |p_\ell - m|\, q_\ell}{\sum_\ell |p_\ell - m|^2}. \]

Vol expansion is \(\sigma_5 / \sigma_{20} - 1\). The stored slope feature is bid slope minus ask slope.

4. Lead–lag

Hayashi and Yoshida (2005) estimate covariance of asynchronous returns without interpolation. The engine uses a discrete cousin: Pearson correlation of fast returns \(\{r^X_i\}\) with slow returns shifted by \(k \in \{0,1,2,4,8,16\}\) ticks, on the last \(48\) overlapping pairs.

\[ \rho(k) = \mathrm{Corr}\bigl(r^X_{i},\, r^Y_{i+k}\bigr). \]

Lead strength is \(\max_k \rho(k) - \rho(0)\). Estimated lag is \(\arg\max_k \rho(k) / 16\). Both are clamped to \([-1,1]\).

FusionEngine::fill_lag, lagged_corr

Definition — lead strength

If the slow tape is a delayed copy of the fast tape, \(\rho(k)\) peaks at \(k \approx \tau\), not at \(0\). The excess correlation is the scalar the network is allowed to see without discovering delay from scratch.

5. The simulator

Futures follow a jump-diffusion with Hawkes-signed flow. With \(Z_t \sim \mathcal{N}(0,1)\) and occasional Gaussian jumps \(J_t\),

\[ S_{t} = S_{t-1} \exp\!\Bigl(\bigl(\mu - \tfrac12\sigma^2\bigr)\Delta t + \sigma\sqrt{\Delta t}\, Z_t + J_t\Bigr) \exp(\varepsilon_t \cdot 0.08\sigma). \]

The unlagged fair for the binary is a logistic of the normalized move from the window open:

\[ \nu = \mathrm{sd}(\Delta S / S)\sqrt{900}, \qquad M_t = \frac{S_t - S_0}{S_0\,\nu}, \qquad f_t = \bigl(1 + e^{-3.4 M_t}\bigr)^{-1}. \]

The CLOB does not print \(f_t\). It prints a sticky, lagged quote

\[ \tilde f_t = f_{t-\tau} + I_t, \]

where \(I_t\) is an AR(1) inventory term (\(0.92, 0.08\)) and the quote only updates if \(|\tilde f_t - Q_{t-}| > 0.012\) or a Poisson clock with probability \(0.08\) fires. Spread widens with quote velocity.

Resolution is exactly \(\mathbf{1}_{S_{900} > S_0}\), independent of \(Q_t\). That is why a calibrated \(\hat p_t\) can beat the book: \(f_t\) already knows more than \(Q_t\).

LagMarket._simulate_paths

6. Dual-stream network

Fast tokens \(x_{1:H}\in\mathbb{R}^{H\times 24}\), slow tokens \(y_{1:H}\in\mathbb{R}^{H\times 16}\), position \(u\in\mathbb{R}^{8}\), lag features \(\ell\in\mathbb{R}^{8}\). Width \(d=384\), \(8\) heads, \(7\) fine blocks plus \(2\) coarse blocks on a \(4\)-tick pool. Per-tick feature gates, QK-norm, ALiBi, pad masks, LayerScale, and FiLM from \([u;\ell]\) (time-to-expiry, inventory, Hayashi–Yoshida).

6.1 RMSNorm, RoPE, SwiGLU

\[ \mathrm{RMS}(z) = \frac{z}{\sqrt{\tfrac{1}{d}\lVert z\rVert_2^2 + \varepsilon}} \odot g, \]

\[ \mathrm{RoPE}(q,k)\ \text{with base } 10^4 \quad\text{(Su et al., RoFormer)}, \]

\[ \mathrm{SwiGLU}(z) = W_3\bigl(\mathrm{SiLU}(W_1 z)\odot W_2 z\bigr), \quad h = \tfrac83 d\ \text{rounded to a multiple of }8. \]

6.2 Attention and fusion

\[ \mathrm{Attn}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_h}}\right)V, \quad d_h = d/4. \]

Each block does self-attention on each stream, then cross-attention both ways, then SwiGLU. Queries and keys are RMS-normalized before RoPE. The lag channel is slow queries, fast keys:

\[ y \leftarrow y + \gamma\odot\mathrm{Attn}(y, x, x). \]

A \(4\)-tick mean-pool path runs two extra blocks and adds back. Each stream is pooled by a learned query mixed with the last valid tick, then gated:

\[ \begin{aligned} z_c &= \mathrm{MLP}([u;\ell]), \\ g &= \sigma\bigl(W_g[z_x; z_y; z_c]\bigr), \\ z &= \mathrm{RMS}\bigl(g \odot z_x + (1-g)\odot z_y + z_c\bigr). \end{aligned} \]

Heads: policy logits in \(\mathbb{R}^{3}\), value scalar, temperature-scaled expiry logit, next-return, lag, and a heteroscedastic \(\log\sigma^2\).

cmf/model.py · DualStreamBlock, FusionModel.encode

7. Training

7.1 Supervised pretrain

On a frame the losses are

\[ \begin{aligned} \mathcal{L}_{\mathrm{bce}} &= \mathrm{BCEWithLogits}(\hat\ell_t, \mathbf{1}_{S_T > S_0}), \\ \mathcal{L}_{\mathrm{ret}} &= \bigl(\hat r_{t\to t+5} - r_{t\to t+5}\bigr)^2, \\ \mathcal{L}_{\tau} &= \bigl(\hat\tau_t - \tau/16\bigr)^2, \\ \mathcal{L}_{\mathrm{bc}} &= -\log \pi_\theta(a^{\star}_t \mid s_t), \end{aligned} \]

plus a Brier term on \(\sigma(\hat\ell_t)\), a heteroscedastic \(\mathrm{e}^{-\log\sigma^2}\mathcal{L}_{\mathrm{bce}}+\log\sigma^2\), and a soft book-utility \(-\mathbb{E}[w^{\uparrow}(y-a)+w^{\downarrow}((1-y)-(1-b))]\) with \(w=\sigma((\mathrm{edge}-0.045)/0.015)\). Hold frames are downsampled so behavioural cloning does not collapse to \(\mathrm{HOLD}\).

7.2 GAE and PPO

\[ \delta_t = r_t + \gamma V(s_{t+1})(1-d_t) - V(s_t), \]

\[ \hat A_t = \sum_{l\ge 0} (\gamma\lambda)^l \delta_{t+l}(1-d_{t+l}), \qquad \hat R_t = \hat A_t + V(s_t). \]

Defaults: \(\gamma = 0.97\), \(\lambda = 0.95\), clip \(\varepsilon = 0.2\). The clipped surrogate is the usual PPO objective (Schulman et al. 2017). Value is clipped; entropy of the 3-way actor is regularized; the same auxiliary heads stay on with weight \(0.35\); a \(0.45\) clone term toward the oracle remains so PPO cannot erase pretrain.

Fusion actions during rollout are not \(\arg\max\) of the actor. They are the Chapter 8 rule applied to the expiry head. The actor is only a regularizer.

cmf/ppo.py · gae, FusionTrainer._loss, Pretrainer._loss

8. The decision rule

Let \(\hat\ell_t\) be the expiry logit and \(\hat p_t = \sigma(\hat\ell_t)\). Against displayed bid and ask,

\[ \begin{aligned} e^{\uparrow}_t &= \hat p_t - a_t, \\ e^{\downarrow}_t &= (1-\hat p_t) - (1-b_t). \end{aligned} \]

The hurdle widens with the uncertainty head \(u_t\in(0,1)\) and as time-to-expiry \(t_e\) collapses (the lag is worth less near settlement):

\[ \theta_t = 0.045\,(1+1.6 u_t)\,(1+0.85(1-t_e)). \]

\[ a_t = \begin{cases} \mathrm{BUY} & e^{\uparrow}_t > \theta_t, \\ \mathrm{SELL} & e^{\downarrow}_t > \theta_t, \\ \mathrm{HOLD} & \text{otherwise.} \end{cases} \]

In a position the flip threshold is \(0.10\,(1+0.8 u_t)\). The oracle still uses a flat \(4.5\) cent hurdle on true \(f_t\).

cmf/policy.py · decide_from_prob

8.1 Ensemble (desk)

The live desk does not trust one head. Every second it computes a digital-option prior, the fusion \(\hat p\), and a lag tilt, then blends

\[ p = 0.45\,\Phi(d_2) + 0.35\,\hat p_{\mathrm{fusion}} + 0.20\,\tilde p_{\mathrm{lag}}, \quad d_2 = \frac{\ln(S_t/S_0)-\tfrac12\sigma^2\tau}{\sigma\sqrt{\tau}}. \]

A trade fires only if at least two heads agree with the blend, or if \(a_{\mathrm{UP}} + a_{\mathrm{DOWN}} < 1\) (complement arb).

cmf/quant.py · ensemble_signal · cmf/ingest.py

What this is not

Clearing a 4.5 cent edge on a planted delay is not a live Sharpe. On 64 held-out episodes CMF-2 made \(+1.01\) units of stake (Sharpe \(2.27\)) against the unlagged oracle at \(+1.72\).

Notation index

Symbol	Meaning	Code
\(S_t\)	Binance futures mid	`bn_path`
\(Q_t, b_t, a_t\)	Polymarket mid, bid, ask	`mid, bid, ask`
\(\tau\)	Planted lag (seconds)	`true_lag`
\(f_t\)	Unlagged logistic fair	`fair`
\(\hat p_t\)	Model \(\mathbb{P}(\mathrm{UP})\)	`sigmoid(p_up)`
\(n, D\)	Shares, dollars staked	`shares, size`
\(m^\star\)	Microprice	`Book::microprice`
\(\mathrm{I}_L\)	Depth imbalance, \(L\) levels	`imbalance`
\(\mathrm{OFI}_L\)	Cont–Kukanov–Stoikov OFI	`ofi_l1, ofi_l5`
\(\mathrm{CVD}\)	Cumulative signed notional	`cvd`
\(\hat\lambda\)	Kyle impact	`kyle_lambda`
\(\lambda^\pm\)	Hawkes buy/sell intensity	`hawkes_buy/sell`
VPIN	Bucketed informed-flow proxy	`vpin`
\(\rho(k)\)	Lagged Pearson of returns	`lagged_corr`
\(H, d\)	History 64, width 384	`history, dim`
\(\gamma, \lambda_{\mathrm{GAE}}\)	0.97, 0.95	`gamma, gae_lambda`
\(e^{\uparrow}, e^{\downarrow}\)	Edges vs offer	`up_edge, down_edge`

References

Cont, Kukanov, Stoikov (2014). The price impact of order book events. Kyle (1985). Continuous auctions and insider trading. Easley, López de Prado, O’Hara (2012). Flow toxicity and VPIN. Hawkes (1971). Spectra of some self-exciting point processes. Hayashi, Yoshida (2005). On covariance estimation of non-synchronously observed diffusion processes. Schulman et al. (2017). Proximal policy optimization algorithms. Su et al. (2021). RoFormer. Shazeer (2020). GLU variants improve transformer. Zhang, Sennrich (2019). Root mean square layer normalization. Press et al. (2021). Train short, test long: ALiBi. Lim et al. (2021). Temporal fusion transformers. Henry et al. (2020). Query-key normalization for transformers. Gneiting, Raftery (2007). Strictly proper scoring rules. HumanPlane (2025). LACUNA / cross-market-state-fusion.

Companion to shubhxho/polymarket-model. Serve with uv run python -m http.server --directory docs 4173.