Data plane

Abstract

POST /run takes its bars in the request. That is fine for a one-shot evaluate, but a live chart re-sends the entire history on every tick and asks the exchange for bars the server already has.

The data plane moves bars out of the request body and into an engine-owned store:

• /data/series* — a SQLite-backed OHLCV store keyed by a canonical series key. Upload closed bars, fetch history from CCXT, read bounds, read the newest window.
• /data/session* — a live session bound to a stored series, advanced from the engine feed instead of the client payload.

Bars are keyed by exchange|market|SYMBOL|interval — for example binance|spot|BTC/USDT|1h. Upserts are keyed by unix-second time and the later write wins, so a forming bar is re-written in place rather than duplicated.

Implemented in backend/datastore/series.py and backend/api/data_series.py, without Flask or CCXT in the store itself.

Conceptual model

Series key

Every route takes the same four fields. market is optional and defaults to spot.

Responses echo the normalized key text, e.g. binance|spot|BTC/USDT|1h.

Interface surface

GET /data/series

Stored bounds for one series. Unknown series report count 0 rather than 404.

{ "status": "success", "key": "binance|spot|BTC/USDT|1h",
  "count": 30, "oldest": 1700000000, "newest": 1700104400, "updated_at": 1791968289 }

GET /data/series/bars

Newest stored bars, ascending. limit defaults to the store window default and is capped at the store maximum.

{ "status": "success", "key": "binance|spot|BTC/USDT|1h", "count": 2,
  "bars": [ { "time": 1700100800, "open": 128.0, "high": 129.0,
              "low": 127.0, "close": 128.5, "volume": 10.0 }, … ] }

POST /data/series/provide

Upload closed bars. At most 5,000 per request; over that is 413 BATCH_TOO_LARGE — split the upload into chunks.

{ "status": "success", "key": "binance|spot|BTC/USDT|1h",
  "added": 30, "updated": 0, "count": 30, "newest": 1700104400 }

POST /data/series/fetch

Walk CCXT fetch_ohlcv pages forward from now - history_bars and persist. history_bars defaults to 5,000 and caps at 100,000. Upstream failure is 502 UPSTREAM; missing ccxt is 503 CCXT_UNAVAILABLE.

{ "status": "success", "key": "binance|spot|BTC/USDT|1h",
  "fetched": 5000, "count": 5000, "oldest": …, "newest": … }

POST /data/session

Build a live interpret session over the stored window and bind it to the series key. Accepts the same script/inputs/realtime fields as live sessions plus exchange, symbol, interval, market, and history_bars.

Returns the /session envelope plus session_id, cache, and data:

{ "status": "success", "session_id": "49564cd995514ef48254eb9dd5322e90",
  "count": 20, "cache": "hit",
  "data": { "key": "binance|spot|BTC/USDT|1h", "count": 30, "newest": 1700104400 } }

When the store holds fewer than history_bars bars it returns 409 DATA_MISS with have and need rather than silently evaluating a short history:

{ "status": "miss", "code": "DATA_MISS", "have": 0, "need": 50 }

POST /data/session/<session_id>/tick

Takes { exchange, symbol } only — the series binding is already established. It fetches candles from two bars before the stored newest (so a corrected or still-forming bar is picked up), persists closed bars, re-ticks the forming bar with update_last_bar, and appends newer bars with append_bars.

The response adds appended, forming, and unchanged:

{ "status": "success", "session_id": "…", "count": 31,
  "appended": 1, "forming": true, "unchanged": false }

409 SESSION_KEY_MISMATCH guards a session bound to another series. Upstream failure is 502 UPSTREAM and keeps the session.

Storage

The store writes through a single connection, so the data plane has the same single-worker constraint as live sessions (GUNICORN_WORKERS=1). Serving it from more than one worker without a shared volume will silently split history.

Failure modes

Internals (map)

See also

• Live sessions
• Auth and keys
• Observability