Customer Case Study · Fintech · India
1,500 instruments, every stop-loss on time
TradeChef, an India-based options trading and strategy platform, runs three heavy workloads on one stack: a live tick-to-order chain trading Nifty and Sensex options, a nightly bhavcopy ingestion pipeline, and multi-year backtesting over billions of rows. Its short-premium strategies survive on one thing above all: protective exits that fire the moment mark-to-market breaches a threshold. Opstral's Managed Service Ops put the whole chain under one latency budget, detects trouble at the leading edge, names the guilty hop in seconds, and prices every minute of slowness in rupees.
- ~3,000/speak ticks across ~1,500 Nifty and Sensex option instruments, aggregated from multiple feed providers
- 150 mstick-to-order latency budget, decomposed hop by hop and enforced
- 8B+rows of tick and enrichment data in ClickHouse, metadata in PostgreSQL
- 3–4 minearly warning: leading indicators fire before the latency SLO breaches
- 90 secfrom first signal to guilty-hop isolation, previously 30–45 minutes of cross-team triage
- ₹13.7mlargest single-strategy margin book whose protective exits the chain defends
The Challenges
TradeChef's strategies sell option premium: ThetaPlay and SellSpread open multi-leg positions on five-minute cycles and rely on MTM-triggered protective exits to cap losses. The exit path runs through six hops: exchange feed, tick gateway, enrichment against ClickHouse and PostgreSQL, event queue, CEP strategy engine, order service, exchange gateway. When any hop slows during a fast market move, the stop-loss executes late on a leveraged book. The entry can wait five minutes; the exit cannot wait five seconds.
- The exit is latency-critical
Short-premium books with up to ₹13.7m margin depend on MTM-triggered exits. A slow chain does not just delay data, it delays the stop-loss, and a fast move prices that delay immediately.
- Six hops, one budget, no owner
Each service had its own latency alarm; nobody owned the end-to-end 150 ms tick-to-order budget, so every degradation became an argument about whose hop it was.
- Alarms fired after the money
p99 latency alerts are lagging indicators. By the time they fired, exits had already gone out late. The signals that lead a breach, queue lag, ack times, cache hit-rate, were not first-class.
- Provider, exchange, or us?
Ticks arrive from multiple feed providers and orders leave through the exchange gateway. When something slowed, triage burned time on the question that changes everything: their side, which we fail over or protect against, or our side, which we fix?
- Three workloads, one database
The nightly bhavcopy load and multi-year backtests share ClickHouse and PostgreSQL with the live path. Batch health was invisible to the trading team until it surfaced as morning slowness.
- Slowness had no price
Nobody could say what a minute of chain lag cost in rupees, so infrastructure priorities were argued with adjectives instead of numbers.
The Solution
Managed Service Ops onboarded all three pipelines as business processes with operating windows, decomposed the tick-to-order budget across the chain, wired leading indicators to every hop, and mapped each service to the strategy books riding on it, so a slowdown surfaces as rupees at risk, a named guilty hop, and a governed action, not as a wall of latency graphs.
- OnboardThree pipelines, three operating windows
Live chain is mission-critical 09:15–15:30 IST; the bhavcopy load owns its evening window; backtests are welcome off-hours and flagged instantly if they intrude on market hours.
- Budget150 ms, decomposed and enforced
Feed gateway 10 ms, enrichment 25 ms, queue 15 ms, CEP 60 ms, order service 25 ms, exchange ack 15 ms. Every hop carries its own budget and its own leading indicator.
- DetectLeading indicators, not post-mortems
Consumer lag and queue depth, order-ack p99, enrichment cache hit-rate, ClickHouse parts and merge backlog, PostgreSQL pool saturation, and tick sequence gaps fire minutes before the SLO breaches.
- QuantifyBlast radius in rupees
Each service maps to the strategy books on it. Impact is computed from open MTM exposure and slippage per second of delayed exit, plus missed five-minute entry cycles, a number, not an adjective.
- ActMOPs with trading-grade governance
Scaling CEP consumers and prioritizing exit orders over entries run auto-safe. Pausing new entry cycles requires approval. A strategy kill-switch never fires without a human. Entries can wait; exits go first.
- VerifySherlock closes it, not hope
An incident ends only when tick-to-order p99 holds under 150 ms across a full verification window. Recovery is proven with mechanical checks, never assumed from a quiet dashboard.
Governed autonomy, trading edition: nothing touches live orders without an approval gate; auto-safe scope is limited to scale-out and queue prioritization; exchange-side slowness triggers protection, never blame; and every action, human or automated, lands in the audit trail with rollback attached.
The Dependency Chain, Live
This is TradeChef's estate as Managed Service Ops models it. Click any component to see its role, its share of the 150 ms budget, the leading indicators watching it, and the MOPs standing by. External systems, the exchange feed and the exchange order gateway, are shown dashed: they are probed and disambiguated, not blamed.
Simulate a Slowdown
Pick a component and a duration, and see what Managed Service Ops computes: the cascade through the chain, the strategy books whose exits are at risk, the exposure in rupees without intervention, and what the governed 90-second response protects. The same engine drives the live product; these figures are illustrative.
The Impact
The trading team moved from arguing about whose hop was slow to reading a single screen that names the hop, prices the risk, and shows the fix already in flight, with recovery proven before the incident closes.
| Dimension | Before | After · Managed Service Ops |
|---|---|---|
| Detection | p99 alarms after exits had already gone out late | 3–4 minutes of lead time from queue lag, ack p99, cache hit-rate and sequence-gap indicators |
| Isolation | 30–45 minutes of cross-team triage across queue, CEP, OMS and DB owners | 90 seconds to a named guilty hop, with exchange-vs-internal disambiguated by probe |
| Protective exits | At the mercy of whichever hop was slow that day | Exit orders prioritized by MOP; entry cycles pause under approval; zero late exits since go-live |
| Batch pipelines | Bhavcopy and backtest health invisible until morning slowness | Completeness KPIs verified before market open; off-hours workloads fenced by operating windows |
| Cost of slowness | Unpriced, argued with adjectives | Rupees per minute, per strategy book, computed from open exposure and slippage curves |
| Closure | Assumed when graphs went quiet | Sherlock-verified: p99 must hold under budget across a full window before the incident closes |
In premium selling, the entry can wait five minutes; the exit cannot wait five seconds. This is the first system that treats our latency budget as a business number, with a rupee value attached to every hop, and proves recovery instead of assuming it.
Put your revenue-critical chain under one latency budget
Per-hop budgets, leading-indicator detection, blast radius in currency, governed MOPs and verified recovery, deployed inside your environment, for any pipeline where seconds are money.
The governed response, step by step
Click a node for detail · dashed = external dependency · the lower lane runs off-hours and shares the databases with the live chain
Every node carries its latency budget, leading indicators and pre-approved MOPs. The two databases are shared by all three workloads, which is exactly why they are modeled together.
- Feed Providersmulti-vendorExternal dependency
The ~1,500 instruments arrive from multiple feed providers, no single vendor carries them all. Each provider is probed separately, so one lagging vendor is failed over, not debated.
Watched- per-provider probe RTT
- per-provider tick-rate divergence
- cross-provider sequence reconciliation
- Rebalance instruments to the healthy provider (auto-safe)
- Protective pause if all feeds stale (approval)
- Tick Gateway10 ms budget
Aggregates and normalizes up to ~3,000 ticks/sec from multiple providers into one stream for ~1,500 instruments.
Watched- ingress lag
- decode errors
- sequence gaps
- Scale gateway pods (auto-safe)
- Failover to standby gateway (auto-safe)
- Enrichment25 ms budget
Joins each tick with instrument metadata (PostgreSQL) and history (ClickHouse).
Watched- cache hit-rate
- CH query p95
- PG pool saturation
- Cache-serve stale metadata (auto-safe, degraded)
- Bump PG pool (auto-safe)
- Event Queue15 ms budget
Kafka stream between enrichment and the strategy engine. Lag here is the earliest cascade signal.
Watched- consumer lag
- queue depth
- partition skew
- Scale consumers (auto-safe)
- Exit-priority ordering (auto-safe)
- CEP Strategy Engine60 ms budget
Runs ThetaPlay, ThetaArbitrage and SellSpread: 5-minute entry cycles, MTM-triggered protective exits.
Watched- cycle evaluation time
- event backlog
- CPU/mem pressure
- Scale CEP workers (auto-safe)
- Pause new entry cycles (approval)
- Strategy kill-switch (human only)
- Order Service25 ms budget
Places entry and exit orders. Exits outrank entries by policy.
Watched- order-ack p99
- reject rate
- retry depth
- Exit orders jump the queue (auto-safe)
- Reroute to secondary gateway session (approval)
- ExchangeOMS gwExternal dependency
Exchange order gateway. External: ack probes decide "exchange slow" vs "our OMS slow".
Watched- ack p99 by session
- throttle/reject codes
- Protective pause if exchange-side (approval)
- PostgreSQLmetadataShared
Instrument and strategy metadata. Shared by live, bhavcopy and backtest.
Watched- pool saturation
- replication lag
- lock waits
- Pool resize (auto-safe)
- Failover read replica (approval)
- ClickHouse8B+ rowsShared
Tick history and enrichment store. The shared resource all three workloads meet on.
Watched- insert latency
- parts count
- merge backlog
- Throttle merges in market hours (auto-safe)
- Evict off-window batch queries (approval)
- Bhavcopy Loaderevening windowOff-hours
Nightly EOD load. If it silently under-fills, tomorrow’s enrichment misfires: completeness is verified before open.
Watched- files received per exchange
- rows loaded
- T+0 completion by 20:00
- Retry failed segment (auto-safe)
- Alert before-open if incomplete (auto)
- Backtest Engineoff-hours onlyOff-hours
3–4 year strategy simulations over ClickHouse. Fenced to off-hours; flagged the moment it intrudes.
Watched- window intrusion
- CH resource share
- run overrun
- Preempt at market open (auto-safe)
- Reschedule (auto)
TradeChef is named with permission. Tick rates, latency budgets, margin figures and rupee exposures on this page are representative and illustrative of the deployment class; the interactive simulation uses the same computation model as the product with illustrative inputs. Exact results vary by instrument mix, market conditions and rollout scope.