PYNE CONSOLIDATED DOCUMENTATION — LLM CONTEXT PACK
Generated: 2026-07-28
Source: docs/pyne
Public: https://hoox.sh/pyne/docs
Repository: https://github.com/jango-blockchained/pynescript

FILE: docs/pyne/api/app-lifecycle.mdx
App Lifecycle
Abstract
backend/app.py constructs a single global Flask app, applies security-minded defaults (body size, CORS allowlist), registers free and Pro routes, and exposes a module main for local development. There is no multi-app factory pattern today — import app for WSGI (gunicorn) or run the module for the built-in server.
Conceptual model
Interface surface
Process entry
``bash
make run
equivalent
python -m backend.app
`
Environment:
| Variable | Default | Role |
| --- | --- | --- |
| HOST | ... | Bind address (localhost-first for dev) |
| PORT | | Listen port |
| ALLOWEDORIGINS | https://pynescript.ai, https://app.pynescript.ai, localhost regex | CORS origins (comma-separated; regex allowed) |
| APIKEYSTORE | /root/pynescript/data/apikeys.json | JSON key store path (file-backed default) |
| ADMINTOKEN | unset | Documented for admin createkey (see auth) |
debug=False always on the dev runner.
Health
GET / → JSON:
`json
{
"status": "healthy",
"service": "pynescript-pro-api",
"version": "..",
"timestamp": ,
"endpoints": { "…": "…" }
}
`
Hard limits
MAXCONTENTLENGTH = — reject oversized bodies before JSON parse fills memory (audit -- / S).
CORS methods: GET, POST only.
Allow headers: Content-Type, Authorization, X-Admin-Token.
supportscredentials=False.
Localhost regex used in the default origin list:
`text
^https?://(?:localhost|\.\.\.)(?::\d+)?$
`
Same-origin / no Origin header (curl, server-to-server) remains usable under flask-cors behavior.
Blueprints
`python
app.registerblueprint(previewbp) urlprefix=/preview
app.registerblueprint(backtestbp) urlprefix=/backtest
`
Defined in backend/api/preview.py.
Error handlers
| Code | Body code | Message |
| --- | --- | --- |
| | NOTFOUND | Endpoint {path} not found. |
| | INTERNALERROR | Internal server error. |
Both return JSON with status: "error".
Internals
| Path | Role |
| --- | --- |
| backend/app.py | App object, free routes, auth routes |
| backend/api/preview.py | Pro blueprints |
| backend/middleware/schemas.py | Request validation for /run and auth |
| backend/requirements.txt | Flask, flask-cors, numpy, matplotlib, … |
Recommended production: gunicorn/uvicorn-class WSGI with multiple workers; for multi-worker key consistency prefer SQLite/Redis stores over the default in-memory/file singleton assumptions — see auth.
Invariants and edge cases
. Import side effects: creating app configures CORS immediately from env.
. Body too large → Flask before route logic (not custom JSON).
. Unknown fields on schema-validated routes → UNKNOWNFIELDS (/run, auth); preview routes currently use looser getjson() (schemas exist but are not always applied in handlers).
. Dev bind default is loopback — set HOST=... only when intentional.
Worked example — smoke test
`bash
curl -s http://...:/ | jq .status
"healthy"
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Browser CORS errors | Origin not in ALLOWEDORIGINS |
| on large OHLCV | Exceeded MB — downsample or paginate |
| Import errors for matplotlib | Missing backend deps — pip install -r backend/requirements.txt` |
See also
Auth and keys
Run endpoint
Docker / CI
---
FILE: docs/pyne/api/auth-and-keys.mdx
Auth and Keys
Abstract
Pro routes authenticate with a raw API key presented as Authorization: Bearer …, Authorization: ApiKey …, or ?apikey=. Keys are minted as pyn + url-safe secret, stored with a short keyid and SHA- hash metadata, and tracked against monthly-ish call limits by tier. Free /run does not require a key.
Conceptual model
Interface surface
Tiers (TIERLIMITS)
| Tier | callslimit |
| --- | --- |
| free | (treated as unlimited in isratelimited — limit means “no cap”) |
| hobby | |
| pro | |
| team | |
| enterprise | inf |
callsremaining() returns inf when limit is .
Endpoints
POST /auth/createkey
Body schema CREATEKEYSCHEMA: optional tier (default "hobby").
Response:
``json
{
"status": "success",
"apikey": "pyn…",
"keyid": "hexchars",
"tier": "hobby",
"message": "Store this API key securely. It will not be shown again."
}
`
Decorated with @requireadmintoken. Implementation note: requireadmintoken currently sets g.isadmin = True unconditionally (placeholder); production deployments should enforce ADMINTOKEN / role checks before exposing this route. App docstring describes X-Admin-Token + ADMINTOKEN env as the intended design.
GET /auth/usage
@requireapikey — returns keyid, tier, and usage block (callsused, callslimit, callsremaining, timestamps).
POST /auth/validate
Body: { "apikey": "…" } — validates without incrementing usage. Returns tier info, active, ratelimited.
Decorators
| Decorator | Behavior |
| --- | --- |
| requireapikey | Resolve key → / or set g.apikey |
| trackusage | Wraps requireapikey; increments calls when response status < (tuple responses) or always for bare Response |
| requireadmintoken | Placeholder admin gate for createkey |
Client presentation
`bash
curl -H "Authorization: Bearer $APIKEY" \
-H "Content-Type: application/json" \
-d @body.json \
http://...:/preview/chart
`
Internals
APIKey dataclass
Fields: keyid, keyhash, tier, callsused, callslimit, createdat, lastused.
Helpers: isactive() (always True today), isratelimited(), gettierinfo(), incrementcalls().
Stores
| Module | Use |
| --- | --- |
| middleware/auth.py → APIKeyStore | Default JSON file store (APIKEYSTORE path) |
| middleware/keystoresqlite.py | Hash-only rows, WAL, multi-worker friendly |
| middleware/keystoreredis.py | Redis-backed alternative |
Default getkeystore() returns the JSON-file APIKeyStore singleton. JSON store currently keeps raw keys as dict keys in the file (dev convenience); SQLite module documents hash-only storage for safer leaks.
Key minting:
`text
rawkey = "pyn" + secrets.tokenurlsafe()
keyid = sha(rawkey).hexdigest()[:]
keyhash = sha(rawkey).hexdigest()
`
Schemas
CREATEKEYSCHEMA, VALIDATEKEYSCHEMA in middleware/schemas.py — strict types, reject unknown fields.
Invariants and edge cases
. /run is unauthenticated — rate-limit at reverse proxy if exposed publicly.
. Limit means unlimited, not “zero calls” — free tier limit value is with that semantics.
. Usage increments only after handler returns — failed auth never bills; handler xx under trackusage skips increment when returned as (response, status).
. Revocation exists on the store API (revokekey); no public HTTP revoke route in app.py.
. Admin decorator gap — do not assume createkey is locked in all deployments until the TODO is closed.
Worked example
`bash
create (when admin gate is open / local tests)
curl -s -X POST http://...:/auth/createkey \
-H 'Content-Type: application/json' \
-d '{"tier":"hobby"}'
validate
curl -s -X POST http://...:/auth/validate \
-H 'Content-Type: application/json' \
-d '{"apikey":"pyn…"}'
`
Failure modes
| Code | Meaning |
| --- | --- |
| UNAUTHORIZED | Missing/invalid key |
| RATELIMITED | callsused >= callslimit (finite limits) |
| INVALIDTIER | createkey with unknown tier string |
| INVALID_KEY` | validate endpoint miss |
See also
App lifecycle
Preview
Security (DevOps)
---
FILE: docs/pyne/api/contract.mdx
Evaluate Contract
Abstract
The evaluate contract is the JSON dialect that lets AXIS, HOOX, CLI tools, and edge workers call any PYNE-compatible host interchangeably. Flask POST /run is the reference implementation; Python pyne-worker and TypeScript pine-worker aim for the same fields even when transport (Worker fetch, queue, etc.) differs.
This page freezes the semantic envelope — not transport auth headers.
Conceptual model
Invariant: given the same script + OHLCV + mode, hosts should agree on plot series and strategy events within known parity limits (see numerical validation and pine-worker testing).
Interface surface
Request (single evaluate)
| Field | Type | Required | Description |
| --- | --- | --- | --- |
| script | string | yes | Full Pine source |
| data | Bar[] | yes | Chronological OHLCV |
| symbol | string | no | Default CHART / host-specific |
| mode | "interpret" \| "compile" | no | Default interpret |
| datasource | string | no | Optional external history provider id |
| dataoptions | object | no | Provider configuration |
Bar
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| time | number | recommended | Unix ms preferred |
| open | number | yes | |
| high | number | yes | |
| low | number | yes | |
| close | number | yes | |
| volume | number | no | Default host-dependent |
| bid / ask | number | no | Runtime bid/ask hooks |
Response (success)
| Field | Type | Description |
| --- | --- | --- |
| status | "success" | Flask sets this; workers may omit and use HTTP only — prefer including it |
| plots | (number\|null)[] | Primary series (plot ) |
| series | Record | Named multi-series |
| plotmeta | Record | Display metadata |
| events | StrategyEvent[] | Strategy / trade-like events |
| drawings | Drawing[] | line/label/box export |
| count | number | Bars evaluated |
| scriptid | string | Stable hash prefix of source |
| runid | string | Per-invocation id |
| mode | string | Echo effective mode |
| datasource | string | Flask echo of resolved source label |
PlotMeta (Flask)
``json
{ "title": "C", "color": "F", "linewidth": , "index": }
`
StrategyEvent (minimum)
Hosts should preserve:
`json
{
"type": "string",
"barindex": ,
"scriptid": "…",
"runid": "…"
}
`
Additional fields follow evaluator todict() / TS port parity — treat unknown keys as forward-compatible.
Response (failure)
Flask /run:
`json
{
"status": "error",
"code": "EXECUTIONERROR",
"message": "Runtime Error at bar …"
}
`
Workers may use equivalent { error: string } or HTTP xx/xx with message body. Clients should accept:
. Envelope status === "error" with message, or
. Presence of top-level error string (raw Runtime.run shape).
Batch evaluate (Flask extension)
POST /run/batch:
Request: scripts: (string | {id, script})[] (max ) + shared data + same optional fields as single run.
Response:
`json
{
"status": "success" | "partial",
"results": [ / per-script success or error object with id / ],
"count": ,
"ok": ,
"datasource": "chart"
}
`
Workers may implement batch as N single evaluates; AXIS should not assume batch exists on every host.
Internals — reference mapping
| Contract field | Flask source |
| --- | --- |
| plots / series / plotmeta | Runtime.run packing loop |
| events | evaluator.strategystate.drainevents() → todict() |
| drawings | DrawingRegistry.exportforapi |
| scriptid | sha(source)[:] |
| runid | Runtime.runid |
Schema enforcement on Flask free tier: RUNSCHEMA / RUNBATCHSCHEMA in backend/middleware/schemas.py (strict, reject extras).
Tests: tests/testbackend.py (testrunsuccess, batch cases).
Divergences to remember
| Surface | Divergence |
| --- | --- |
| Preview / backtest | Columnar data dict, not Bar[]; not the evaluate contract |
| Compile mode | Extra keys objectmode, generatedcode may appear |
| Auth | Flask free /run open; workers may require HOOX mTLS / API gateways |
| Error codes | Flask uses code enums; workers should map to stable strings when possible |
Invariants and edge cases
. Bar order is chronological ascending — hosts do not sort for you.
. na / missing appear as JSON null in series arrays.
. Duplicate plot titles get suffixes (, …) on Flask; clients should key on returned map keys, not assumed titles.
. Idempotent scriptid for identical source text across hosts.
. runid is not stable across retries — use for log correlation only.
. Parity is best-effort on floating edges; pin fixtures when testing hosts against each other.
Worked example — host-agnostic client sketch
`typescript
type EvaluateRequest = {
script: string;
data: Array;
symbol?: string;
mode?: "interpret" | "compile";
};
type EvaluateSuccess = {
plots: Array;
series: Record>;
events: Array>;
drawings: Array>;
count: number;
scriptid: string;
runid: string;
mode: string;
};
`
POST that body to Flask /run or the worker’s evaluate route; branch on status / error.
Failure modes
| Client bug | Symptom |
| --- | --- |
| Sending columnar preview data to /run | Schema data must be a list |
| Assuming batch on worker | / unknown field |
| Ignoring series and only reading plots` | Multi-plot scripts look single-series |
See also
Run endpoint
Runtime bridge
pine-worker
Pro API usage (end user)
Events
---
FILE: docs/pyne/api/endpoints/backtest.mdx
Backtest Endpoint
Abstract
POST /backtest/quick is a Pro, usage-tracked route that runs a simplified strategy simulation over columnar OHLCV (or generated mock bars), returns trade lists + summary metrics, and embeds a base equity-curve PNG. It is optimized for speed/demo UX; it is not a full fidelity substitute for event-aware Runtime strategy evaluation on /run.
Conceptual model
Blueprint: backtestbp, prefix /backtest.
Interface surface
Request
``json
{
"script": "//@version=\nstrategy(\"s\")…",
"data": {
"open": [], "high": [], "low": [], "close": [], "volume": []
},
"initialcapital": .,
"mockdata": false,
"mockbars":
}
`
| Field | Default | Notes |
| --- | --- | --- |
| script | required | Empty → NOSCRIPT |
| data | {} | Columnar; may be omitted if mockdata |
| initialcapital | | Starting equity |
| mockdata | false | Force synthetic bars |
| mockbars | | Length of mock series |
If neither usable close nor mockdata → NODATA.
Success
`json
{
"status": "success",
"result": {
"equitycurve": [, …],
"trades": [
{
"entrytime": ,
"entryprice": ,
"exittime": ,
"exitprice": ,
"direction": "long",
"pnl": ,
"pnlpct": ,
"size":
}
],
"summary": {
"totalpnl": ,
"totalpnlpct": ,
"sharperatio": ,
"maxdrawdown": ,
"maxdrawdownpct": ,
"winrate": ,
"profitfactor": ,
"totaltrades": ,
"winningtrades": ,
"losingtrades": ,
"avgwin": ,
"avgloss":
},
"equitychart": ""
},
"tierinfo": {},
"meta": {
"bars": ,
"initialcapital": ,
"completedat":
}
}
`
Errors
| code | HTTP | When |
| --- | --- | --- |
| NOSCRIPT | | Empty script |
| NODATA | | No data and not mock |
| BACKTESTERROR | | Exception in simulation |
| UNAUTHORIZED / RATELIMITED | / | Auth |
Internals
runquickbacktest → runbacktest in backend/services/backtest.py.
MVP simulation characteristics:
. Optionally parse(script) (errors soft-ignored for MVP path).
. Precompute long/short entry signals from dual SMA cross ( vs ) starting at bar .
. Exit heuristics via RSI-like average and opposite signals / end of series.
. Equity curve + trade PnL with optional commission/slippage parameters on the lower-level API.
. Metrics: Sharpe (√ scaling), max drawdown, win rate, profit factor.
. Chart via renderequitycurve.
generatemockohlcv(nbars) produces synthetic columns for demos.
| Path | Role |
| --- | --- |
| backend/api/preview.py | quickbacktest route |
| backend/services/backtest.py | Simulation + metrics |
| backend/services/chartrenderer.py | Equity PNG |
Invariants and edge cases
. Script content is lightly used in the MVP sim — do not treat results as broker-accurate for arbitrary strategies. Prefer /run events for engine-faithful strategy traces.
. Columnar data, same family as preview, not /run bar lists.
. Mock path ignores incomplete user data when mockdata or empty close.
. One usage increment per successful HTTP response under trackusage.
Worked example
`bash
curl -s http://...:/backtest/quick \
-H "Authorization: Bearer $APIKEY" \
-H 'Content-Type: application/json' \
-d '{
"script": "//@version=\nstrategy(\"demo\")\n// body optional for MVP",
"mockdata": true,
"mockbars": ,
"initialcapital":
}' | jq '.result.summary'
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Implausible trades vs script logic | MVP signal model, not full evaluator |
| Empty trade list | No crosses in series / short history |
| Missing equitychart | Renderer exception swallowed → empty string |
See also
Runtime bridge — faithful strategy events via /run`
Chart renderer
Strategy builtins
---
FILE: docs/pyne/api/endpoints/preview.mdx
Preview Endpoints
Abstract
Preview routes render PNG thumbnails (base) for desk UIs and docs. They require an API key via @trackusage, accept OHLCV-shaped objects (column arrays), and do not run the full Runtime bar loop for /preview/chart (data is plotted directly). /preview/indicator evaluates a small set of closed-form ta. expressions in pure Python for the series, then plots.
Conceptual model
Blueprint: previewbp with urlprefix="/preview" in backend/api/preview.py.
Interface surface
Auth
Both routes: @trackusage → must pass requireapikey and consume one call on success.
POST /preview/chart
Body (documented; schemas exist as PREVIEWCHARTSCHEMA)
``json
{
"script": "optional string",
"data": {
"open": [], "high": [], "low": [], "close": [], "volume": []
},
"options": {
"type": "line",
"color": "F",
"showvolume": false,
"width": ,
"height":
}
}
`
| Option | Default | Clamp |
| --- | --- | --- |
| type | "line" | "ohlcv" selects candlestick renderer |
| color | F | line chart only |
| width | | max |
| height | | max |
| showvolume | false | twin-axis volume on line chart |
Success
`json
{
"status": "success",
"chart": "",
"meta": {
"type": "line",
"bars": ,
"lastvalue": ,
"firstvalue": ,
"change": ,
"changepct": ,
"renderedat":
},
"tierinfo": {}
}
`
Errors
| code | When |
| --- | --- |
| NODATA | Missing data |
| NOCLOSEDATA | Empty close array |
| RENDERERROR | matplotlib / renderer exception |
Note: script is accepted for API symmetry but not executed on this path today — the chart is pure data visualization.
POST /preview/indicator
Body
`json
{
"expression": "ta.sma(close, )",
"data": { "close": [/ … /] },
"options": { "color": "FF", "width": , "height": }
}
`
Supported expression prefixes in computeindicator:
| Expression | Series |
| --- | --- |
| ta.sma(…, period) | SMA of close |
| ta.ema(…, period) | EMA |
| ta.rsi(…, period) | RSI |
| ta.macd(…) | MACD histogram (//) |
| other | falls back to raw close |
Parse failures in the helper fall back to plotting close.
Internals
| Path | Role |
| --- | --- |
| backend/api/preview.py | Route handlers + indicator math |
| backend/services/chartrenderer.py | PNG encode |
| backend/middleware/auth.py | trackusage |
Schemas PREVIEW in schemas.py document intended validation; current handlers use request.getjson() or {} directly.
Invariants and edge cases
. Data shape differs from /run: preview uses columnar dicts; /run uses list of bar dicts.
. Not a full Pine interpreter for indicator expressions — string prefix parsing only.
. Width/height clamps protect worker memory/CPU.
. Successful responses include tierinfo from the authenticated key.
Worked example
`bash
curl -s http://...:/preview/chart \
-H "Authorization: Bearer $APIKEY" \
-H 'Content-Type: application/json' \
-d '{
"data": {"close": [,,,,], "volume": [,,,,]},
"options": {"type": "line", "showvolume": true}
}' | jq '.meta'
``
Failure modes
| Symptom | Cause |
| --- | --- |
| | Missing key |
| | Tier limit |
| Empty-looking PNG | Renderer empty-chart path when all values null |
See also
Chart renderer
Auth and keys
Run
---
FILE: docs/pyne/api/endpoints/run.mdx
POST /run and /run/batch
Abstract
/run is the primary free evaluate entry: accept Pine source + bar list, optionally wire request. data sources, execute via Runtime, and return plots/series/events/drawings. /run/batch runs up to eight scripts on the same OHLCV (AXIS multi-indicator), isolating per-script errors so one failure does not discard siblings.
Conceptual model
Interface surface
POST /run
Request (RUNSCHEMA)
| Field | Type | Required | Default | Notes |
| --- | --- | --- | --- | --- |
| script | string | yes | | Pine source |
| data | list | yes | | OHLCV bar objects |
| symbol | string | no | "CHART" | Syminfo / request wiring |
| datasource | string | no | "" | mock / ccxt / yahoo / … |
| dataoptions | object | no | {} | Provider options |
| mode | string | no | "interpret" | "interpret" \| "compile" |
Unknown fields → UNKNOWNFIELDS . Empty script/data still checked after schema with NOSCRIPT / NODATA.
Bar shape (runtime expectation):
``json
{ "time": , "open": , "high": , "low": , "close": , "volume": }
`
Optional bid / ask per bar update runtime bid/ask state.
Success response
`json
{
"status": "success",
"plots": [],
"series": {},
"plotmeta": {},
"events": [],
"drawings": [],
"scriptid": "hex",
"runid": "hex",
"count": ,
"mode": "interpret",
"datasource": "chart"
}
`
| Field | Meaning |
| --- | --- |
| plots | Primary series (first plot) — backward compatible list |
| series | Named multi-plot map title → values per bar |
| plotmeta | Per-title color, linewidth, index |
| events | Strategy events with scriptid / runid stamps |
| drawings | Exported line/label/box registry for AXIS |
| scriptid | sha(source)[:] |
| runid | Per-Runtime instance id |
Errors
| HTTP | code | When |
| --- | --- | --- |
| | MISSINGFIELD / INVALIDFIELD / UNKNOWNFIELDS / INVALIDBODY | Schema |
| | NOSCRIPT / NODATA | Empty after defaults |
| | DATASOURCEERROR | resolverequestsources failed |
| | EXECUTIONERROR | Runtime returned error key |
POST /run/batch
Request (RUNBATCHSCHEMA)
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| scripts | list | yes | strings or {id, script} objects |
| data | list | yes | Shared OHLCV |
| symbol / datasource / dataoptions / mode | optional | same as /run |
Hard cap: RUNBATCHMAXSCRIPTS = → TOOMANYSCRIPTS.
Response
`json
{
"status": "success" | "partial",
"results": [ { "id": "…", "status": "success|error", … } ],
"count": ,
"ok": ,
"datasource": "chart"
}
`
HTTP for partial success (per-script errors inside results). Envelope validation failures still .
Internals
`python
runtime = Runtime(symbol=str(symbol))
result = runtime.run(script, ohlcv, datafeed=…, dataprovider=…, mode=…)
`
See runtime bridge for bar-loop details and compile mode.
Paths: backend/app.py (runpinescript, runpinescriptbatch), backend/middleware/schemas.py.
Invariants and edge cases
. No API key on these routes — protect at the edge if needed.
. Batch isolates exceptions per script (try/except around runtime.run).
. Whitespace-only datasource coerced to None → chart default.
. Compile mode requires numba; errors surface as EXECUTIONERROR message from runtime.
. Schema rejects extras — clients must not send AXIS-only fields without updating schema.
Worked example
`bash
curl -s http://...:/run \
-H 'Content-Type: application/json' \
-d '{
"script": "//@version=\nindicator(\"t\")\nplot(close)",
"data": [
{"time": , "open": , "high": , "low": ., "close": ., "volume": },
{"time": , "open": ., "high": ., "low": ., "close": ., "volume": }
],
"symbol": "TEST:DEMO"
}'
`
Failure modes
| Symptom | Cause |
| --- | --- |
| UNKNOWNFIELDS | Typo’d property (e.g. ohlcv instead of data) |
| Parse errors in message | Invalid Pine — same engine as CLI |
| Batch status: partial | At least one script failed; inspect results[i].message` |
See also
Contract
Runtime bridge
App lifecycle
---
FILE: docs/pyne/api/index.mdx
Pro API
The PYNE Pro API is a Flask HTTP service that runs Pine scripts over caller-supplied OHLCV, returns plots / series / strategy events / drawings, and gates thumbnail + backtest routes behind API keys. It is intentionally not the language server: different auth, scaling, and failure domains.
Abstract
Where the LSP answers static editor questions, the Pro API answers dynamic evaluate questions:
``text
POST JSON { script, data: OHLCV[], … }
→ backend.runtime.Runtime.run
→ { plots, series, plotmeta, events, drawings, scriptid, runid, … }
`
Free-tier evaluate (POST /run, POST /run/batch) validates bodies with hand-rolled schemas, enforces a MB body cap, and restricts CORS. Pro routes under /preview/ and /backtest/ use trackusage (Bearer / ApiKey / query key). Chart PNGs are matplotlib Agg renders encoded as base.
The same evaluate contract is mirrored by edge workers (pyne-worker / pine-worker) so AXIS and HOOX can swap hosts without redesigning payloads — see contract.
Conceptual model
Interface surface
| Method | Path | Auth | Role |
| --- | --- | --- | --- |
| GET | / | none | Health + endpoint map |
| POST | /run | none (free) | Single-script evaluate |
| POST | /run/batch | none (free) | ≤ scripts, shared OHLCV |
| POST | /preview/chart | API key + usage | Line / OHLCV PNG |
| POST | /preview/indicator | API key + usage | Expression series PNG |
| POST | /backtest/quick | API key + usage | Quick strategy metrics + equity PNG |
| POST | /auth/createkey | admin decorator | Mint pyn… key |
| GET | /auth/usage | API key | Usage counters |
| POST | /auth/validate | none (body key) | Validate without consuming quota |
Local runner: make run → python -m backend.app (default ...:).
Tracks in this tab
. App lifecycle — Flask app, CORS, limits, blueprints
. Auth and keys — tiers, stores, decorators
. Run · Preview · Backtest
. Chart renderer
. Runtime bridge — Runtime / series / evaluator glue
. Contract — shared evaluate schema across Flask and workers
Internals (map)
| Path | Role |
| --- | --- |
| backend/app.py | App factory surface, /run, /auth, CORS |
| backend/api/preview.py | Preview + backtest blueprints |
| backend/middleware/ | Auth, schemas, optional Redis/SQLite stores |
| backend/runtime.py | Bar-loop evaluate façade |
| backend/evaluator.py | CustomEvaluator bridge |
| backend/series.py | PineSeries history deque |
| backend/services/` | Charts + backtest simulation |
See also
Evaluate scripts (end user)
Pro API usage guide
Runtime hub
LSP hub — editor path, not HTTP
---
FILE: docs/pyne/api/runtime-bridge.mdx
Runtime Bridge
Abstract
backend.runtime.Runtime is the HTTP stack’s evaluate façade. It owns symbol metadata namespaces (syminfo, timeframe, barstate, …), walks OHLCV bars, updates PineSeries for OHLC, visits a shared CustomEvaluator on the parsed AST each bar, drains strategy events, and packages multi-plot series for AXIS. Compile mode optionally swaps the AST walker for a Numba-backed subset engine.
This is the bridge between Flask handlers and the language core — not a second semantics.
Conceptual model
Interface surface
Construction
``python
Runtime(symbol: str = "AAPL", runid: str | None = None)
`
Sets syminfo.tickerid / name / prefix (prefix from EXCHANGE:SYMBOL form).
Generates runid as uuid().hex[:] when omitted.
Helpers: configurefootprint, updatebidask.
run(sourcecode, ohlcvdata, datafeed=None, dataprovider=None, mode="interpret")
| Arg | Role |
| --- | --- |
| sourcecode | Pine text |
| ohlcvdata | list[dict] with open/high/low/close/time[/volume/bid/ask] |
| datafeed / dataprovider | request. wiring; auto-resolved from chart bars when unset |
| mode | "interpret" (default) or "compile" |
Interpret path (default)
. Resolve request sources (non-fatal on failure).
. parse(sourcecode, mode="exec") — on failure { "error": "Parse Error: …" }.
. Init PineSeries for OHLC + context dict (timeframe daily defaults, barstate, chart colors).
. Construct CustomEvaluator(context=…, datafeed=…, dataprovider=…), reset var declarations + drawing registry.
. Per bar:
Update series + calendar fields from timestamp
Update barstate flags (isfirst / islast / history confirmed)
processpendingorders when available
evaluator.visit(tree) — on failure { "error": "Runtime Error at bar …" }
Lock pine defs after first bar (pinedefslocked) to avoid multi-dispatch blow-ups
Drain strategy events; stamp scriptid / runid
Capture plot outputs
. Build series / plotmeta from all plot indices (disambiguate duplicate titles with suffixes).
. Export drawings via DrawingRegistry.exportforapi(bartimes).
Return keys: plots, series, plotmeta, events, drawings, count, scriptid, runid, mode.
Compile path
. Import pynescript.compiler.engine — missing → error string.
. Require hasnumba().
. compilescript + compiled.run(opens, highs, lows, closes, volumes).
. Pop internal keys (drawings, events, position meta).
. Return similar envelope with mode: "compile", plus objectmode / generatedcode when present.
PineSeries (backend/series.py)
History deque (default maxlen ), series[] current / series[n] lookback, arithmetic on current values, None propagates as na-like absence. Negative indices raise — Pine does not allow them.
Context namespaces
Defined in runtime.py: Syminfo, Chartinfo, Timeframe, Barstate, Chart — attribute names aim at Pine v/v surface (timeframe.isdaily, syminfo.isin, …).
Internals
| Path | Role |
| --- | --- |
| backend/runtime.py | Runtime |
| backend/evaluator.py | CustomEvaluator specialization |
| backend/series.py | PineSeries |
| src/pynescript/ast/helper.py | parse |
| src/pynescript/ast/evaluator/ | Builtin + strategy semantics |
| src/pynescript/compiler/engine.py | Compile mode |
Flask wiring: backend/app.py constructs a fresh Runtime per request (and per batch job) so runid and drawing registries do not leak across users.
Invariants and edge cases
. Parse once, visit many — AST is not re-parsed per bar.
. Defs locked after bar — performance invariant for large function tables.
. Drawing registry reset at run start — isolation between requests.
. Primary plots list is plot index for backward compatibility; AXIS should prefer series + plotmeta.
. Errors are dicts, not exceptions, for control-flow at the HTTP boundary ("error" in result).
. Compile mode supports a subset (documented in engine — ta/math/plots family); unsupported scripts should use interpret.
Worked example
`python
from backend.runtime import Runtime
rt = Runtime(symbol="NASDAQ:AAPL")
out = rt.run(
'//@version=\nindicator("x")\nplot(close, "C")',
[{"time": i, "open": , "high": , "low": ., "close": . + i ., "volume": } for i in range()],
)
assert "error" not in out
assert out["count"] ==
assert "C" in out["series"] or "plot" in out["series"]
`
Failure modes
| Message prefix | Cause |
| --- | --- |
| Parse Error: | Grammar / syntax |
| Runtime Error at bar | Evaluator exception |
| Order fill error at bar | Broker sim pending orders |
| Compile mode requires numba | Missing dependency |
| Compile Error: / Compiled Runtime Error:` | Engine path |
See also
Contract
Run endpoint
Series and history
Strategy events
---
FILE: docs/pyne/api/services/chart-renderer.mdx
Chart Renderer
Abstract
backend/services/chartrenderer.py is a headless plotting façade. It forces matplotlib into Agg, draws dark-theme charts for desk thumbnails, and returns base-encoded PNG strings suitable for JSON transport. Callers are the preview and backtest routes; the module has no Flask dependency.
Conceptual model
Interface surface
renderlinechart
| Param | Default | Role |
| --- | --- | --- |
| values | required | Y series; None → NaN gaps |
| dates | None | Optional x tick labels (≤ ~ ticks) |
| title | "Chart" | Title string |
| color | F | Line + fill |
| height / width | / | Pixel intent → figsize /, dpi |
| showvolume | false | Twin axis bars from ohlcv["volume"] |
| ohlcv | None | Volume source when enabled |
Empty / all-null values → renderemptychart placeholder.
renderohlcvchart
Candlestick + volume subplot pair from dict keys open, high, low, close, volume. Green/red body colors (CAF / F). Shared x-axis.
renderequitycurve
Single series with profit/loss fill relative to zero baseline and a dashed reference at the first equity point. Title fixed "Equity Curve".
Return type
All public renderers → str (base PNG without data-URI prefix). Clients typically use:
``text
data:image/png;base,{chart}
`
Internals
| Detail | Implementation |
| --- | --- |
| Backend | matplotlib.use("Agg") before pyplot import |
| Theme | fig EEE, axes , muted grids |
| Cleanup | plt.close(fig) after savefig |
| Empty state | Centered “No data available” text |
| Path | Role |
| --- | --- |
| backend/services/chartrenderer.py | Renderers |
| backend/api/preview.py | Consumers |
| backend/services/backtest.py | Equity chart hook |
Invariants and edge cases
. Not interactive — no GUI backend; safe for servers.
. NaNs break lines at gaps (numpy nan).
. Candle body height uses + e- to keep zero-range bodies visible.
. Thread safety: matplotlib global state is historically process-local; multi-threaded gunicorn workers should avoid concurrent pyplot use on one worker or serialize renders.
. Exceptions in callers often become HTTP RENDERERROR or empty equity chart strings.
Worked example
`python
from backend.services.chartrenderer import renderlinechart
b = renderlinechart([., ., ., .], title="demo", color="AE")
assert isinstance(b, str) and len(b) >
``
Failure modes
| Symptom | Cause |
| --- | --- |
| ImportError Agg | Incomplete matplotlib install |
| Huge base | High width/height — routes clamp preview sizes |
| Blank image | Empty input path succeeded with placeholder |
See also
Preview endpoints
Backtest
---
FILE: docs/pyne/contributing.mdx
Contributing
Abstract
Contributions to PYNE are welcome when they respect the repo’s mechanical invariants: generated code boundaries, src-layout packaging, dual console scripts, test corpus costs, and encryption secrets. This page distills CONTRIBUTING.md and AGENTS.md into an operational contract for humans and agents.
Conceptual model
Interface surface
Baseline workflow
. Fork and branch from main
. Install: make install or hatch envs
. Run tests: make test / hatch run test:test
. Implement with tests (prefer TDD for non-trivial behavior)
. Lint/format: make lint / make fmt or hatch run lint:style + lint:typing
. Open a PR with rationale + test evidence
Official short form also lives in root CONTRIBUTING.md (points at Sphinx docs historically).
Everyday commands
| Goal | Command |
| --- | --- |
| Editable install + LSP | make install |
| Full tests | make test |
| LSP only | make test-lsp |
| Backend only | make test-backend |
| Lint / format | make lint / make fmt |
| Fast build sanity | make build-check |
| API | make run |
| LSP process | make run-lsp |
Hard constraints
Grammar & generated code
Edit only src/pynescript/ast/grammar/antlr/resource/.g for grammar work
Never hand-edit …/antlr/generated/ or …/asdl/generated/
Regenerate via hatch lint:gen-parser / project scripts; see grammar guides under .opencode/context/
No stale backups
Do not recreate removed backups such as builder.py.bak or evaluator/builtins/technicalrefactored.py. Live builder.py and technical.py are sole sources of truth.
Future annotations
Every new Python file must start with:
``python
from future import annotations
`
Enforced by ruff isort required-imports.
Console scripts
| Script | Role |
| --- | --- |
| pynescript | Click CLI |
| pynescript-lsp | Language Server (pygls) |
Do not conflate entrypoints in docs, packaging, or process managers.
Test corpus fixture
tests/conftest.py parametrizes pinescriptfilepath over every tests/data/builtinscripts/.pine. A new test using that fixture may run hundreds of cases. Narrow with --example-scripts-dir=... when iterating.
tests/data/library/ is reference material, not auto-parametrized.
Builtin metadata
builtinmetadata.json is generated from code. After adding builtins:
`bash
python scripts/generatebuiltinmetadata.py
`
Do not hand-maintain the catalog long-term.
Secrets & crypto
scripts/build/.metadata.key is gitignored
CI must supply stable CRYPTOKEY from METADATAKEY / METADATAKEY for reproducible .enc blobs
Never commit Fernet keys or API admin tokens
Internals (where to change what)
| Want to… | Look at |
| --- | --- |
| Parse / unparse | src/pynescript/ast/helper.py |
| Add builtin | src/pynescript/ast/evaluator/builtins/.py + metadata generator |
| LSP feature | src/pynescript/langserver/features/ + server.py |
| CLI subcommand | src/pynescript/main.py |
| Grammar | resource/.g then regenerate |
| Pro API route | backend/api/preview.py, backend/app.py |
| TS port | pine-worker/ |
Style
Ruff line length ; broad rule set in pyproject.toml
Black target py (via toolchain config)
Mypy strict-ish; exemptions for generated grammar, evaluator.builtins., tests
Voice for docs: academic nerdy-cool per docs/WRITING.md — no empty marketing adjectives
pine-worker contributions
Bun for install/test/typecheck
Python remains oracle; update parity fixtures via generator
Converter stubs are starting points only
Failure modes for contributors
| Mistake | Fallout |
| --- | --- |
| Editing generated ANTLR/ASDL | Overwritten / review hell |
| Skipping corpus cost awareness | -minute “unit” tests |
| Hand-editing metadata JSON | Drift from dispatch |
| Committing .metadata.key | Secret leak; rotate immediately |
| PR without lint | CI red on ruff/mypy |
See also
Local development
CI
pine-worker
Roadmap
Root AGENTS.md, CONTRIBUTING.md, CODEOFCONDUCT.md`
---
FILE: docs/pyne/core/asdl-schema.mdx
ASDL Schema
The abstract syntax tree is not free-form dicts. It is an algebraic data type declared in ASDL and materialized as Python dataclasses.
Abstract
src/pynescript/ast/grammar/asdl/resource/Pinescript.asdl defines the Pinescript module: roots (Script, Expression), statements, expressions (including control structures that are also expressions), operators, params, args, cases, and comments. Codegen writes grammar/asdl/generated/PinescriptASTNode.py. src/pynescript/ast/node.py re-exports that module so the rest of the codebase imports from pynescript.ast import node as ast (or from pynescript.ast.node import Script).
ASDL is the contract between builder (produces nodes), evaluator/LSP (consume nodes), and unparser (serializes nodes). Changing a field without regenerating and updating all three is a schema break.
Conceptual model
Each product sum type becomes a Python class hierarchy under AST with:
fields — logical children (walked by iterfields / visitors)
attributes — location metadata (lineno, coloffset, endlineno, endcoloffset) where declared
Interface surface
Module roots (mod)
| Constructor | Fields | Use |
| --- | --- | --- |
| Script | stmt body, string annotations | Full file (mode="exec") |
| Expression | expr body | Single expression (mode="eval") |
Statements (stmt)
| Constructor | Notable fields |
| --- | --- |
| FunctionDef | name, param args, body, method?, export?, annotations |
| TypeDef | UDT type Name body (fields + methods as stmts) |
| EnumDef | enum Name members |
| Assign | target, optional value/type/mode (Var/VarIp), export?, annotations |
| ReAssign | := reassignment |
| AugAssign | += … ops |
| Import | namespace/name/version, optional alias |
| Expr | Expression statement |
| Break / Continue | Loop control |
All stmt and most other attributed products carry location attributes.
Expressions (expr)
Operators and primaries:
BoolOp, BinOp, UnaryOp, Conditional (? :), Compare
Call, Constant, Attribute, Subscript, Name, Tuple
Structures as expressions: ForTo, ForIn, While, If, Switch — same shapes can appear as values (Pine’s expression-oriented control flow)
Qualify — const/input/simple/series applied to a type/expression
Specialize — generic specialization form (value + args)
Auxiliaries
| Product | Role |
| --- | --- |
| declmode | Var \| VarIp |
| typequal | Const \| Input \| Simple \| Series |
| exprcontext | Load \| Store |
| param | Function/method parameter |
| arg | Call argument (optional keyword name) |
| case | Switch arm (pattern?, body) |
| cmnt / Comment | Comment with value and kind (annotation classification) |
Operators are empty product constructors (Add, And, Eq, …) used as tags — the unparser maps class names back to tokens.
Importing nodes
``python
from pynescript.ast import node as ast
script = ast.Script(body=[
ast.Expr(value=ast.Call(
func=ast.Name(id="plot", ctx=ast.Load()),
args=[ast.Arg(value=ast.Name(id="close", ctx=ast.Load()))],
))
])
`
Prefer constructing via parse() unless synthesizing trees for tests.
Internals
Paths
| Path | Edit? |
| --- | --- |
| src/pynescript/ast/grammar/asdl/resource/Pinescript.asdl | Yes — schema source of truth |
| src/pynescript/ast/grammar/asdl/generated/PinescriptASTNode.py | No — regenerated |
| src/pynescript/ast/grammar/asdl/tool/generate.py | Generator entry |
| src/pynescript/ast/node.py | Thin re-export (from .grammar.asdl.generated import ) |
Generated class shape
Excerpt of generated form (do not edit this file; shown for orientation):
`python
@dataclasses.dataclass
class Script(mod):
body: list[stmt] = field(defaultfactory=list)
annotations: list[string] = field(defaultfactory=list)
fields: ClassVar[list[str]] = ["body", "annotations"]
`
stmt base injects location attributes. Hashability is forced to object.hash so nodes can live in sets/maps by identity when needed.
Regeneration
When the schema changes:
`bash
Project convention (pyasdl / asdl tool — see asdl/tool/generate.py)
pyasdl src/pynescript/ast/grammar/asdl/resource/Pinescript.asdl \
-o src/pynescript/ast/grammar/asdl/generated
`
Then update, in the same change set:
. builder.py construction sites
. unparser.py visit methods
. Evaluator expression/statement handlers
. Any isinstance checks in LSP features
Dual nature of structures
ASDL places If, ForTo, ForIn, While, Switch under expr, while the builder often wraps structure-as-statement under Expr(value=…). Annotation collection (StatementCollector) treats assign/reassign/augassign/expr that hold structure values specially so nested statements remain visible for //@… pairing.
Invariants
. Schema first. New language constructs need an ASDL constructor (or a clear encoding into an existing one) before evaluator semantics.
. Fields vs attributes. Visitors walk fields only. Location is metadata; missing locations are fillable via fixmissinglocations.
. Optional flags as int?. method and export are integer flags (truthy when set), not booleans — match builder assignments (export = ).
. Generated file is not a style playground. Formatting/lint of PinescriptASTNode.py is overwritten on regen.
Worked examples
Map ASDL to a short script
`pine
//@version=
f(x) => x +
a = f(close)
`
Approximate tree:
`text
Script(
annotations=["//@version=", ...],
body=[
FunctionDef(name="f", args=[Param(name="x")], body=[Expr(BinOp(...))]),
Assign(target=Name("a", Store), value=Call(...)),
],
)
`
Dump fields programmatically
`python
from pynescript.ast.helper import parse, iterfields
tree = parse("x = ")
for name, value in iterfields(tree):
print(name, type(value).name)
`
Failure modes
| Failure | Cause |
| --- | --- |
| AttributeError on new field | Schema regenerated but consumer not updated |
| Builder constructs wrong product | ASDL and grammar out of sync |
| Unparser omits new node | Missing visitNewNode → generic_visit may emit nothing useful |
| Hand-edit of generated nodes lost | Edited generated/ instead of .asdl` |
See also
Builder
Visitor / transformer
Unparser
Grammar (ANTLR)
---
FILE: docs/pyne/core/builder.mdx
AST Builder
The builder is the semantic boundary between ANTLR’s concrete parse tree and PYNE’s ASDL abstract tree.
Abstract
src/pynescript/ast/builder.py defines three cooperating mixins:
. PinescriptASTLocator — maps ParserRuleContext.start/stop tokens to lineno / coloffset / end.
. PinescriptCommentParser — classifies //… text into annotation kinds (@=S, @F, , plain //, …).
. PinescriptASTBuilder — subclasses generated PinescriptParserVisitor, implements visit for every meaningful rule, returns ASDL nodes.
helper.parse constructs a builder, runs builder.visit(rulecontext), then (in exec mode) uses StatementCollector + comment tokens to attach annotations. The builder itself does not walk the default token channel for comments; that is a post-pass.
Conceptual model
Interface surface
You rarely instantiate the builder directly; parse() does. For tools that already hold a parse tree:
``python
from pynescript.ast.builder import PinescriptASTBuilder
from pynescript.ast.grammar.antlr.parser import PinescriptParser
... construct parser, parse to ctx ...
node = PinescriptASTBuilder().visit(ctx)
`
Locator API
| Method | Role |
| --- | --- |
| getLocations(ctx) | Dict of four location keys |
| setLocations(node, ctx) | Mutates node attributes from ctx.start / ctx.stop |
End column calculation accounts for embedded newlines in the stop token text.
Comment kinds
parseComment returns (kind, parts):
| Pattern | Kind prefix | Suffix | Example |
| --- | --- | --- | --- |
| //@version = | @= | S (script) | version assignment |
| //@description … | @ | S | script description |
| //@function / @returns | @ | F | function docs |
| //@type | @ | T | type docs |
| //@variable | @ | V | variable docs |
| //@param name … | @ | F | param docs |
| //@field name … | @ | T | field docs |
| // region / endregion | | — | editor regions |
| other // | // | — | ordinary comment |
helper.addannotations filters by suffix when attaching to nodes.
Store context
setstorectx walks assignment targets (Name, nested Tuple) and sets ctx = Store() so load/store distinction is available to later passes (Python-ast style).
Representative visit methods
| Rule visitor | Emits |
| --- | --- |
| visitStartscript | Script(body) |
| visitStartexpression | Expression(body) |
| visitFunctiondeclaration / visitMethoddeclaration | FunctionDef (method= for methods) |
| visitTypedeclaration | TypeDef |
| visitEnumdeclaration | EnumDef |
| visitCompound / visitSimple assignment family | Assign / ReAssign / AugAssign |
| visitIfstructure / visitFor / visitWhile / visitSwitch | Structure expressions |
| visitexpression (disjunction → … → primary) | Operator trees with correct associativity |
| visitLiteral | Constant (numbers via ast.literaleval, strings, bools, colors) |
| visitPrimaryexpressioncall | Call + Arg list (positional/keyword) |
Expression climbing follows the grammar’s layered rules (conditional → disjunction → conjunction → equality → inequality → additive → multiplicative → unary → primary).
Internals
Path
Implementation: src/pynescript/ast/builder.py
Visitor base: src/pynescript/ast/grammar/antlr/visitor.py → generated PinescriptParserVisitor
Node types: src/pynescript/ast/node.py
Annotation pairing: src/pynescript/ast/collector.py + helper.addannotations
Statement list flattening
visitStatements flattens each statement visitor’s result: some rules return a list (simple multi-statement lines joined by commas), so the body is always a flat list[stmt].
Export flag
When EXPORT is present on library declarations or compound name initialization, the builder sets export = on the corresponding node (integer flag per ASDL int?).
Field / enum members
Field definitions become Assign-like statements inside TypeDef.body (with optional VarIp mode). Enum members become assignment-shaped stmts under EnumDef.body. The unparser special-cases method members of types when pretty-printing.
Invariants
. Every constructed stmt/expr that has a source span should receive setLocations so LSP diagnostics and getsourcesegment work.
. Visitor method names must track generated rule names. Renaming a parser rule without updating the builder is a silent genericvisit fallback (wrong or empty trees).
. Do not edit builder.py.bak. Live builder is builder.py only (stale backups are forbidden under project policy).
. Grammar and builder co-evolve. A resource-only grammar change that adds rules is incomplete until visit exists.
Worked examples
What x = close + becomes
Rough structure after build:
`text
Assign(
target=Name(id="x", ctx=Store()),
value=BinOp(
left=Name(id="close", ctx=Load()),
op=Add(),
right=Constant(value=),
),
)
`
Export const initialization
Grammar: EXPORT? variabledeclaration EQUAL structureexpression
Builder sets assign.export = when ctx.EXPORT() is present — required for library export of typed constants.
Debugging a missing visitor
`python
from pynescript.ast.helper import parse, dump
print(dump(parse(yoursnippet), indent=))
`
If a subtree is None or a raw unexpected type, find the rule in PinescriptParser.g and the matching visitRulename in builder.py.
Failure modes
| Symptom | Cause |
| --- | --- |
| AttributeError: '…Context' object has no attribute '…' | Full parser regen changed accessors; builder outdated |
| Locations all zero / missing | Forgot _setLocations on new visit method |
| Annotations not attached | Comment kind suffix wrong, or statement not yielded by StatementCollector |
| Tuples not assignable | Store context not propagated through Tuple.elts` |
| Structure expression lost | Wrong rule branch between statement vs expression path |
See also
Grammar (ANTLR)
ASDL schema
Helper API
Error model
---
FILE: docs/pyne/core/error-model.mdx
Error Model
Parse failures in PYNE are not bare ANTLR console messages. They are structured exceptions with file, line, column, source excerpt, and a caret — suitable for CLI, tests, and LSP adapters.
Abstract
Two layers cooperate:
. src/pynescript/ast/error.py — SyntaxErrorDetails, SyntaxError, IndentationError.
. src/pynescript/ast/grammar/antlr/errorlistener.py — PinescriptErrorListener implements ANTLR’s ErrorListener.syntaxError, builds details from the recognizer + offending token, and raises the project SyntaxError.
helper.parse installs this singleton listener on both lexer and parser after removing default listeners. Indentation problems raised from PinescriptLexerBase use the same exception hierarchy (IndentationError subclass).
Note: these names intentionally shadow Python builtins (SyntaxError, IndentationError) inside the package — import carefully:
``python
from pynescript.ast.error import SyntaxError as PineSyntaxError
`
Conceptual model
Interface surface
SyntaxErrorDetails (NamedTuple)
| Field | Type | Meaning |
| --- | --- | --- |
| filename | str | Path or |
| lineno | int | -based line |
| offset | int | -based column |
| text | str | Source line (or excerpt) |
| endlineno | int \| None | Multi-line span end |
| endoffset | int \| None | End column |
SyntaxError(Exception)
`python
class SyntaxError(Exception):
def init(self, message: str, details): ...
`
details may be:
a single SyntaxErrorDetails instance, or
positional components matching the NamedTuple fields.
Attributes: message, details.
str renders:
`text
{message}
File "{filename}", line {lineno}
{strippedline}
{spaces}^
`
Offset is adjusted when the displayed line is lstrip()’d so the caret still points at the logical column.
IndentationError(SyntaxError)
Empty subclass for indent-stack failures from the lexer base (wrong nest, inconsistent levels). Catch either specifically or via the base SyntaxError.
PinescriptErrorListener
`python
from pynescript.ast.grammar.antlr.errorlistener import PinescriptErrorListener
listener = PinescriptErrorListener.INSTANCE singleton
lexer.removeErrorListeners()
lexer.addErrorListener(listener)
parser.removeErrorListeners()
parser.addErrorListener(listener)
`
| Method | Role |
| --- | --- |
| syntaxError(...) | Build details; raise SyntaxError |
| getFilenameFrom(recognizer) | Walk Parser → TokenStream → Lexer → InputStream/FileStream |
| getInputTextFrom(recognizer, lineno) | Full buffer or single line |
| splitLines | Portable newline split (\r\n/\n/\r) |
If the ANTLR exception e is already a project SyntaxError, its details are updated; otherwise the new error’s cause is set to e.
Internals
Paths
| Path | Role |
| --- | --- |
| src/pynescript/ast/error.py | Exception types |
| src/pynescript/ast/grammar/antlr/errorlistener.py | ANTLR bridge |
| src/pynescript/ast/grammar/antlr/resource/PinescriptLexerBase.py | May raise IndentationError / SyntaxError during tokenization |
| src/pynescript/ast/helper.py | Wires listener; maps filename onto streams |
Filename resolution order (InputStream)
. getSourceName() if present
. sourceName attribute
. inputstream.name (set by helper to the user filename)
. ""
FileStream uses fileName.
End span from offending token
`text
symbollen = stop - start +
endlineno = token.line + newlinesintext
endoffset = adjusted if multiline else column + symbollen
`
Mirrors the builder’s location math so diagnostics and AST spans speak the same coordinate system.
Relationship to linter E
PineLinter.checksyntax catches any Exception from parse and wraps it as LintWarning(code="E", severity="error"), losing structured details. For rich carets, catch pynescript.ast.error.SyntaxError at the call site instead of only reading linter output.
LSP / tooling
Adapters typically map:
| Exception field | LSP Diagnostic |
| --- | --- |
| lineno / offset | Range.start |
| endlineno / end_offset | Range.end |
| message | message |
| IndentationError | same, possibly different code |
Invariants
. Default ANTLR listeners must stay removed on the parse path — otherwise errors print twice and may not raise.
. Singleton listener is stateless enough to share (INSTANCE); thread-local needs would require new instances.
. Raising aborts parse immediately — no error recovery producing a partial tree through the public helper.
. Package SyntaxError ≠ builtin — except SyntaxError without import may catch the wrong class.
Worked examples
Catching a parse error
`python
from pynescript.ast.helper import parse
from pynescript.ast.error import SyntaxError as PineSyntaxError
try:
parse("f( =>\n", filename="bad.pine")
except PineSyntaxError as e:
print(e) caret form
print(e.details.lineno, e.details.offset)
print(e.details.filename)
`
Distinguishing indentation
`python
from pynescript.ast.error import IndentationError as PineIndentError
from pynescript.ast.error import SyntaxError as PineSyntaxError
try:
parse("if true\nplot()\n") missing indent under if — may indent-error depending on tokens
except PineIndentError as e:
print("indent", e.message)
except PineSyntaxError as e:
print("syntax", e.message)
`
Manual construction (tests)
`python
from pynescript.ast.error import SyntaxError, SyntaxErrorDetails
details = SyntaxErrorDetails("t.pine", , , " plot(\n", , )
err = SyntaxError("missing RPAR", details)
assert "t.pine" in str(err)
assert "^" in str(err)
`
Failure modes
| Failure | Cause |
| --- | --- |
| TypeError: unexpected type of input | Listener received a recognizer/stream combo it does not support |
| Caret misaligned | Tabs vs spaces; display strips leading WS while offset counts raw |
| Filename always | Stream name not set (bypassed helper) |
| Exception swallowed as generic E | Only used linter path |
| except SyntaxError misses Pine errors | Caught builtin only |
| Partial trees | Not available via parse()` after error — redesign would need recovery listeners |
See also
Helper API
Grammar (ANTLR)
Linter
LSP diagnostics
Language core hub
---
FILE: docs/pyne/core/grammar-antlr4.mdx
Grammar (ANTLR)
Concrete syntax for Pine Script in PYNE is defined by a split lexer/parser ANTLR grammar plus hand-written base classes that inject Python-like indentation tokens.
Abstract
Two grammars live under src/pynescript/ast/grammar/antlr/resource/:
PinescriptLexer.g — keywords, operators, literals (including v triple-quoted strings), comments, hidden whitespace.
PinescriptParser.g — statements, expressions, type/enum/function/method declarations, control structures.
Python runtime classes under generated/ are outputs of antlr -Dlanguage=Python. They are committed so end users never need a JDK. Never edit files under generated/ — changes evaporate on the next regeneration and fight the build pipeline.
Hand-written bases (copied into generated/ by the generate tool):
resource/PinescriptLexerBase.py — INDENT/DEDENT, newline elision, multiline string handling.
resource/PinescriptParserBase.py — parser superClass hooks.
Conceptual model
Interface surface
Application code should not import generated symbols directly when a wrapper exists:
``python
from pynescript.ast.grammar.antlr.lexer import PinescriptLexer
from pynescript.ast.grammar.antlr.parser import PinescriptParser
from pynescript.ast.grammar.antlr.visitor import PinescriptParserVisitor
from pynescript.ast.grammar.antlr.errorlistener import PinescriptErrorListener
`
The public parse path is higher level — Helper API — which constructs lexer, token stream, parser, and strips default console error listeners in favor of PinescriptErrorListener.
Entry rules (PinescriptParser.g)
| Rule | Purpose |
| --- | --- |
| start → startscript | Full script (statements? EOF) |
| startexpression | Single expression + optional NEWLINE + EOF |
| startcomments | Comment stream only |
Lexer structural facts
| Token / channel | Behavior |
| --- | --- |
| INDENT / DEDENT | Imaginary tokens emitted by PinescriptLexerBase (declared in tokens { }) |
| COMMENT | // line comments → COMMENTCHANNEL (not the default channel) |
| WS | Spaces/tabs/form-feed → HIDDEN |
| ERRORTOKEN | Catch-all . for recovery/reporting |
| TRIPLEDQSTRING / TRIPLESQSTRING | v multiline strings, typed as STRING |
Keywords include var, varip, method, export, enum, type, qualifiers const/input/simple/series, and control forms if/for/while/switch.
Parser structural facts
Compound vs simple statements. Functions, methods, types, enums, and structure-valued assignments are compound; imports, break/continue, expression statements, and simple assignments terminate with NEWLINE.
Structures are dual-use. if/for/while/switch appear both as statements and as expressions (structureexpression) — the classic Pine “if returns a value” form.
Local blocks. Either indented (NEWLINE INDENT statements DEDENT) or inline single statement.
Library export. EXPORT? prefixes function/method/type/enum and June- export const T name = … compound initializers.
Internals
Paths
| Path | Edit? | Role |
| --- | --- | --- |
| src/pynescript/ast/grammar/antlr/resource/PinescriptLexer.g | Yes | Lexer grammar |
| src/pynescript/ast/grammar/antlr/resource/PinescriptParser.g | Yes | Parser grammar |
| src/pynescript/ast/grammar/antlr/resource/PinescriptLexerBase.py | Yes | Indentation machine |
| src/pynescript/ast/grammar/antlr/resource/PinescriptParserBase.py | Yes | Parser base |
| src/pynescript/ast/grammar/antlr/generated/ | No | ANTLR output + copied bases |
| src/pynescript/ast/grammar/antlr/tool/generate.py | Tool | Runs antlr, copies .py bases |
| src/pynescript/ast/grammar/antlr/errorlistener.py | Hand | Maps ANTLR errors → ast.error.SyntaxError |
LexerBase responsibilities
PinescriptLexerBase is not optional sugar; it is part of the language definition:
Ignore leading / collapse consecutive newlines; ensure trailing newline.
Suppress newlines inside open () / [], after operators, and for soft wrap lines whose indent width is not a multiple of four.
Push INDENT/DEDENT with tab length and indent length = .
Special-case multiline string tokens so wrap indentation inside strings is not mis-tokenized as structure.
Indent mistakes surface as pynescript.ast.error.IndentationError (subclass of the project SyntaxError).
Regeneration
`bash
Preferred hatch env (pulls antlr-cli)
hatch run lint:gen-parser
Or: python -m pynescript.ast.grammar.antlr.tool.generate
which invokes: $(python)/antlr -o generated -lib resource -listener -visitor -Dlanguage=Python resource/.g
then copies resource/.py into generated/
`
Requires a working antlr CLI (often antlr-cli on PATH, sometimes under ~/.local/bin or next to sys.executable). Generated modules are excluded from mypy/ruff strictness in pyproject.toml — do not “lint-fix” them by hand.
Downstream after grammar change
. Regenerate (or selectively refresh lexer — see case study).
. Add/adjust visit methods in src/pynescript/ast/builder.py for new or renamed rules.
. If the shape of the language changed (new statement/expression kind), update Pinescript.asdl and regenerate ASDL nodes — ASDL schema.
. Smoke-test with parse / unparse on a minimal snippet before the full tests/data/builtinscripts/ corpus.
Invariants
. Never hand-edit generated/. Patches belong in resource/.g or resource/Base.py.
. Visitor contract. Generated PinescriptParserVisitor method names follow ANTLR rule names; the builder subclass must track renames.
. Comments are off the default channel. Annotation logic in helper.collectcommentnodes iterates tokenstream.tokens for COMMENT type after fill().
. Indent unit is four spaces. Soft-wrap lines that are not multiples of four are newline-elided, not nested blocks.
Worked examples
Minimal grammar-level mental model
`text
//@version=
indicator("x")
if close > open
plot()
else
plot()
`
Lexer emits NEWLINE, INDENT, DEDENT around the if bodies. Parser builds ifstructure → builder yields ast.If with body / orelse statement lists.
v triple-quoted strings
Resource lexer rules (illustrative — see live PinescriptLexer.g):
`g
TRIPLEDQSTRING : '"""' ( ~'"' | '"' ~'"' | '""' ~'"' ) '"""' -> type(STRING) ;
TRIPLESQSTRING : '\'\'\'' ( ~'\'' | '\'' ~'\'' | '\'\'' ~'\'' ) '\'\'\'' -> type(STRING) ;
`
ANTLR quoting pitfall: putting "'''" or '"""' as fragment literals can produce tool errors (quote came as a complete surprise). Factor starters when needed:
`g
fragment TRIPLESQSTART: '\'' '\'' '\'';
`
Content and source indentation inside the triple quotes are preserved literally (no automatic dedent).
Case study: targeted lexer refresh (-)
Full regeneration once produced a PinescriptParser.py whose context accessors diverged from what the hand-written builder expected (e.g. missing templatespecsuffix()), breaking even trivial parses. The practical recovery:
. Edit only resource/ grammar.
. Generate the lexer in a clean temp directory (avoids nested src/ mirror paths).
. Copy only PinescriptLexer.py (+ refreshed LexerBase.py) into generated/.
. Leave committed PinescriptParser.py and visitors untouched unless prepared to patch the builder in the same change.
. Verify immediately:
`python
from pynescript.ast.helper import parse, unparse
ast = parse('indicator("x")\ns = """\nfoo\n bar\n"""\n')
assert "foo" in unparse(ast)
`
Failure modes
| Failure | Cause | Mitigation |
| --- | --- | --- |
| mismatched input after """ | Old lexer ATN without triple rules | Refresh generated lexer from resource |
| Full regen breaks builder | Parser context API drift | Selective copy; or update all visit together |
| antlr not found | CLI not on PATH | Use hatch env or full path to antlr-cli |
| Nested generated/src/pynescript/... junk | Running antlr with wrong -o cwd | Generate from clean temp + explicit -o |
| IndentationError mid-file | Mixed tabs/spaces or wrong nest | Align to -space blocks; check soft-wrap rules |
| Silent loss of hand patches | Edited generated/ | Revert; re-apply in resource/` |
See also
ASDL schema
Builder
Error model
Helper API
Missing features
---
FILE: docs/pyne/core/helper-api.mdx
Helper API
src/pynescript/ast/helper.py is the stable library face of the language core. It orchestrates ANTLR, the builder, annotations, and the unparser behind functions deliberately similar to CPython’s ast module.
Abstract
Call parse to obtain an ASDL tree; dump / walk / iter to inspect it; unparse to regenerate source; literaleval for constant-folding-safe evaluation of literal expressions. Location helpers (copylocation, fixmissinglocations, getsourcesegment, incrementlineno) support tooling that rewrites or reports on trees.
Everything higher in the stack (CLI, LSP, Pro API, workers) ultimately routes through this module for front-end work.
Conceptual model
Interface surface
parse(source, filename="", mode="exec") -> AST
| Parameter | Meaning |
| --- | --- |
| source | Full script or expression text |
| filename | Error reporting name; resolved to absolute path if the file exists |
| mode | "exec" → startscript → Script; "eval" → startexpression → Expression |
Raises:
ValueError — invalid mode
pynescript.ast.error.SyntaxError — lexer/parser failure (via PinescriptErrorListener)
Side effects during exec mode:
. Temporarily raises sys.getrecursionlimit to at least .
. Collects COMMENT tokens and attaches annotations to eligible statements.
unparse(node) -> str
Delegates to NodeUnparser().visit(node). See Unparser.
dump(node, , annotatefields=True, includeattributes=False, indent=None) -> str
Recursive pretty-print of node trees. With indent (int spaces or string), multi-line layout is used for non-trivial nodes.
literaleval(nodeorstring, context=None, datafeed=None, dataprovider=None) -> Any
Strings are parse(..., mode="eval") first.
Unwraps Expression.body.
Uses NodeLiteralEvaluator — not full script evaluation. Non-literal graphs raise.
Tree navigation
| Function | Behavior |
| --- | --- |
| iterfields(node) | Yields (name, value) for set fields |
| iterchildnodes(node) | Direct AST children (incl. list items) |
| walk(node) | BFS over the full subtree |
Location utilities
| Function | Behavior |
| --- | --- |
| copylocation(new, old) | Copy lineno/col/end when both define the attribute |
| fixmissinglocations(node) | Fill gaps from parent defaults starting at (, ) |
| incrementlineno(node, n=) | Shift all line numbers |
| getsourcesegment(source, node, , padded=False) | Slice original source by node span; None if incomplete ends |
all export list
copylocation, dump, fixmissinglocations, getsourcesegment, incrementlineno, iterchildnodes, iterfields, literaleval, parse, unparse, walk.
Internals
Path
src/pynescript/ast/helper.py — all public helpers above.
Related:
src/pynescript/ast/collector.py — StatementCollector for annotation pairing
src/pynescript/ast/unparser.py — unparse backend
src/pynescript/ast/evaluator — NodeLiteralEvaluator (literaleval only)
Annotation algorithm (addannotations)
. Merge comments and statements; sort by (lineno, coloffset).
. Group consecutive comments vs statements.
. Keep only kinds starting with @.
. Script-level: first group members with kind ending in S → script.annotations.
. Pair remaining comment groups with following statements; attach F/T/V filters to FunctionDef / TypeDef / Assign.
Comment nodes are not left in Script.body; only string annotation lists are stored on targets.
Stream helpers
| Internal | Role |
| --- | --- |
| parseinputstream | InputStream(source) + stream.name = filename |
| parsefilestream | FileStream for on-disk scripts |
| getabsolutepath | Resolve existing paths; leave ` alone |
Invariants
. mode is a closed set. Only exec and eval.
. Default listeners are removed. Console ANTLR spam is replaced by raising project SyntaxError.
. dump requires an AST instance. Non-nodes raise TypeError.
. Round-trip tests should use parse + unparse, not raw builder access, so annotation behavior matches production.
Worked examples
Inspect a tree
`python
from pynescript.ast.helper import parse, dump, walk
tree = parse("""
//@version=
indicator("demo")
plot(close)
""")
print(dump(tree, indent=))
print(sum( for in walk(tree)), "nodes")
`
Source segment
`python
from pynescript.ast.helper import parse, getsourcesegment
src = "a = \nb = a + \n"
tree = parse(src)
assignb = tree.body[]
print(getsourcesegment(src, assignb)) "b = a + \n" or similar span
`
Literal evaluation
`python
from pynescript.ast.helper import literaleval
assert literaleval(" + ") ==
assert literaleval("'hi'") == "hi"
`
Failure modes
| Failure | Notes |
| --- | --- |
| Deeply nested ternaries hit recursion | Mitigated by temporary limit ≥ ; pathological depth can still fail |
| getsourcesegment returns None | Missing endlineno / endcoloffset |
| literal_eval raises | Non-literal AST (names, calls beyond allowed set) |
| Filename in errors | Expected when parsing pure strings without a path |
| Annotations missing | Comment not @…` form, or not immediately preceding eligible stmt |
See also
Builder
Unparser
Visitor / transformer
Error model
---
FILE: docs/pyne/core/index.mdx
Language Core
The language core is the front half of the evaluate contract: source text becomes a typed tree; the tree can be walked, rewritten, dumped, and unparsed without ever entering the bar loop.
Abstract
PYNE treats Pine Script as a formal pipeline, not a host plugin. Lexing and parsing are ANTLR-driven. The concrete syntax tree is lowered by a hand-written visitor (PinescriptASTBuilder) into ASDL-generated dataclasses. Public helpers in helper.py mirror Python’s ast module: parse, unparse, dump, walk, literaleval. Static tools (linter, type registry, LSP) consume the same tree. Evaluation is out of scope here — see Runtime.
Hard rule: edit grammar only under resource/. Paths under generated/ are regenerated artifacts and must not be hand-edited.
Conceptual model
| Stage | Role | Primary path |
| --- | --- | --- |
| Grammar | Concrete syntax | src/pynescript/ast/grammar/antlr/resource/.g |
| ASDL | Algebraic node shape | src/pynescript/ast/grammar/asdl/resource/Pinescript.asdl |
| Builder | CST → AST | src/pynescript/ast/builder.py |
| Helper | Public parse/unparse API | src/pynescript/ast/helper.py |
| Nodes | Re-export generated types | src/pynescript/ast/node.py |
Interface surface
Consumers almost always enter through pynescript.ast.helper:
``python
from pynescript.ast.helper import parse, unparse, dump, walk, literaleval
tree = parse('//@version=\nindicator("x")\nplot(close)')
print(dump(tree, indent=))
print(unparse(tree))
`
Modes:
| mode | Entry rule | Root node |
| --- | --- | --- |
| "exec" (default) | startscript | Script |
| "eval" | startexpression | Expression |
Full page inventory:
| Page | Topic |
| --- | --- |
| Grammar (ANTLR) | Resource vs generated; regen; v triple strings |
| ASDL schema | Algebraic AST; codegen |
| Builder | CST visitor → ASDL nodes |
| Helper API | parse, dump, walk, locations |
| Unparser | Precedence-aware source regen |
| Visitor / transformer | Read vs rewrite |
| Type system | Qualifiers, UDT registry |
| Linter | Static rules, codes |
| Error model | SyntaxError, indent, listener |
Internals
`
src/pynescript/ast/
helper.py public pipeline orchestration
builder.py PinescriptASTBuilder
unparser.py NodeUnparser + Precedence
visitor.py NodeVisitor
transformer.py NodeTransformer
collector.py StatementCollector (annotation pairing)
linter.py PineLinter
typesystem.py Type, TypeRegistry, MethodResolver
error.py SyntaxError / IndentationError
node.py from grammar.asdl.generated import
grammar/
antlr/
resource/ EDIT HERE: .g + Base.py
generated/ NEVER HAND-EDIT
errorlistener.py
tool/generate.py
asdl/
resource/ EDIT HERE: Pinescript.asdl
generated/ NEVER HAND-EDIT
tool/generate.py
`
Thin wrappers grammar/antlr/lexer.py, parser.py, visitor.py re-export generated classes so application code imports stable paths.
Invariants
. Resource is source of truth. Lexer/parser .g and Pinescript.asdl define syntax and node shape; generated Python is a build product (committed so install does not require Java).
. Round-trip is a test oracle. For the supported surface, unparse(parse(src)) should re-parse and preserve semantics (not byte-identical formatting).
. Locations are first-class. Statements and expressions carry lineno, coloffset, optional end for diagnostics and LSP.
. Annotations are not free-floating. Comment tokens on COMMENTCHANNEL are reified as Comment nodes and attached to Script / FunctionDef / TypeDef / Assign via kind suffixes (S, F, T, V).
. Recursion budget. Deep ternary chains raise the Python recursion limit temporarily during parse (cap ≥ ).
Worked examples
Minimal script:
`python
from pynescript.ast.helper import parse, dump
src = """
//@version=
indicator("demo")
x = ta.sma(close, )
plot(x)
"""
tree = parse(src)
assert tree.class.name == "Script"
assert any("version" in a for a in (tree.annotations or []))
print(dump(tree, includeattributes=True, indent=))
`
Expression-only eval mode:
`python
from pynescript.ast.helper import parse, literaleval
expr = parse(" + ", mode="eval")
assert literaleval(expr) ==
`
Failure modes
| Symptom | Likely cause | Where to look |
| --- | --- | --- |
| SyntaxError with caret | Lexer/parser reject | Error model |
| IndentationError | Bad INDENT/DEDENT from LexerBase | PinescriptLexerBase.py |
| Missing visit_ after grammar change | Builder not updated for new rule | Builder |
| Hand-edit of generated/ lost on regen | Violated resource-only rule | Grammar |
| Full regen breaks builder.py` | Parser context accessors changed | Prefer targeted lexer refresh (v case study) |
See also
Grammar (ANTLR)
Helper API
Runtime
LSP
Compatibility
---
FILE: docs/pyne/core/linter.mdx
Linter
The core linter is a lightweight static checker that sits on the parse pipeline plus regex rules. It is not a full dataflow analyzer; it is the first automated gate for “does this even look like modern Pine?”
Abstract
src/pynescript/ast/linter.py exports:
LintWarning — dataclass (code, message, line, column, severity)
PineLinter — stateful runner accumulating warnings
lintscript(source, filename) / lintfile(filepath) — convenience entry points
Rules run in fixed order: syntax → version → deprecated patterns → naming → style. Syntax failures become severity "error" with code E; other rules are mostly "warning".
Conceptual model
Interface surface
``python
from pynescript.ast.linter import lintscript, PineLinter, LintWarning
warnings = lintscript("""
indicator("x")
plot(close)
""")
for w in warnings:
print(w) warning: [W] Missing @version ... at line
`
LintWarning
| Field | Type | Meaning |
| --- | --- | --- |
| code | str | Stable id (E, W, C, …) |
| message | str | Human-readable explanation |
| line | int \| None | -based when known |
| column | int \| None | Reserved / optional |
| severity | str | "warning" (default) or "error" |
str format: {severity}: [{code}] {message} at {location}.
PineLinter.lint(source, filename="") -> list[LintWarning]
Resets self.warnings, runs all checks, returns the list (also stored on the instance).
File helper
`python
from pynescript.ast.linter import lintfile
issues = lintfile("strategies/meanreversion.pine")
`
Reads UTF- text then delegates to lintscript.
Rule catalog
Syntax
| Code | Severity | Condition |
| --- | --- | --- |
| E | error | parse(source, filename) raises any exception; message includes exception text |
Does not attempt recovery; one syntax error check per run.
Version
| Code | Condition |
| --- | --- |
| W | No //@version = N (flexible whitespace) match |
| W | Version integer |
| C | Line matches ^\s+if\s+ — flagged as “single-line if without braces” heuristic |
| C | File does not end with newline |
Internals
Path
src/pynescript/ast/linter.py
Dependencies:
pynescript.ast.parse (re-exported path via from pynescript.ast import parse)
Standard library re, dataclasses
Design stance
The linter intentionally uses regex over AST for several rules so it still partially works when parse fails (version/deprecations/style still run after a failed syntax check — note: checksyntax records E but does not abort the pipeline). That means:
False positives/negatives on fancy formatting are possible.
Deep semantic issues (wrong series type, undefined names) belong to evaluator/LSP diagnostics, not these codes.
Integration points
CLI, editor save hooks, and CI can call lintscript without spinning up the full evaluator. LSP diagnostics may layer additional semantic checks beyond this module.
Invariants
. lint() always clears prior warnings on that instance before running.
. Codes are stable public strings — treat renames as breaking for tooling.
. Syntax errors are not fatal to the function — you get E plus any later regex hits.
. No mutation of source — pure analysis.
Worked examples
Clean modern script
``python
from pynescript.ast.linter import lintscript
src = """//@version=
indicator("ok")
length =
basis = ta.sma(close, length)
plot(basis)
"""
assert lintscript(src) == [] may still flag C if naming heuristic fires
`
Note: basis = ta.sma(...) triggers C under the current rule (lowercase LHS). Prefer documenting that heuristic when teaching style.
Missing version
`python
ws = lintscript('indicator("x")\nplot(close)\n')
assert any(w.code == "W" for w in ws)
`
Programmatic filter
`python
errors = [w for w in lintscript(src) if w.severity == "error"]
if errors:
raise SystemExit()
`
Failure modes
| Issue | Explanation |
| --- | --- |
| C on legitimate indented if blocks | Regex is coarse; multi-line if with body still matches ^\s+if\s+ |
| C noise | Many valid snakecase or lower identifiers |
| E opaque message | Exception string from parse; see Error model for caret formatting when catching SyntaxError directly |
| Encoding errors in lint_file | Non-UTF- files raise at read time |
| False security deprecation | Pattern looks for quoted EXCHANGE:SYM form only |
See also
Helper API — underlying parse`
Error model
LSP diagnostics
End-user troubleshooting
---
FILE: docs/pyne/core/type-system.mdx
Type System
PYNE’s type layer models Pine’s qualified types and user-defined types (UDTs) as Python objects usable by analysis and evaluation — not as a full independent typechecker IR (yet), but as the shared vocabulary for “what kind of value is this?”
Abstract
src/pynescript/ast/typesystem.py defines:
Qualifiers — const, simple, series, input (TypeQualifier)
Builtin kinds — int, float, bool, string, color, na, enum
Composite types — array, matrix, map
UDTs — UserDefinedType with Fields and MethodSignatures
Runtime instances — ObjectInstance for UDT values
Registry / resolution — TypeRegistry, MethodResolver (.new, .copy, user methods)
ASDL already has Qualify / Specialize expression forms and typequal constructors; the type system module is the semantic counterpart used when evaluating UDT construction and when tools reason about types.
Conceptual model
Interface surface
Qualifiers
``python
from pynescript.ast.typesystem import TypeQualifier
TypeQualifier.CONST compile-time constant
TypeQualifier.SIMPLE bar-invariant
TypeQualifier.SERIES per-bar series
TypeQualifier.INPUT input. style parameter
`
Type.str renders "series float" when a qualifier is set.
Builtins and factories
`python
from pynescript.ast.typesystem import (
BuiltinTypeKind,
inttype,
floattype,
booltype,
stringtype,
colortype,
TypeQualifier,
)
t = floattype(TypeQualifier.SERIES) "series float"
`
| Factory | Kind |
| --- | --- |
| inttype | INT |
| floattype | FLOAT |
| booltype | BOOL |
| stringtype | STRING |
| colortype | COLOR |
Collections
`python
from pynescript.ast.typesystem import ArrayType, MatrixType, MapType, inttype, stringtype
ArrayType(inttype()) array
MatrixType(floattype()) matrix
MapType(stringtype(), inttype()) map
`
UDTs
`python
from pynescript.ast.typesystem import UserDefinedType, Field, MethodSignature, floattype, inttype
point = UserDefinedType("point")
point.addfield(Field("x", floattype(), defaultvalue=.))
point.addfield(Field("y", floattype(), defaultvalue=., varip=False))
point.addmethod(MethodSignature("length", [], returntype=floattype()))
`
Field supports optional defaultvalue and varip (mirrors varip fields in type declarations).
TypeRegistry
`python
from pynescript.ast.typesystem import TypeRegistry
reg = TypeRegistry()
reg.registertype(point)
assert reg.isbuiltintype("float")
assert reg.isuserdefinedtype("point")
assert reg.gettype("float") is not None
`
Builtins are seeded in initbuiltintypes (int, float, bool, string, color, na, enum).
ObjectInstance + MethodResolver
`python
from pynescript.ast.typesystem import ObjectInstance, MethodResolver
inst = ObjectInstance(point)
inst.setfield("x", .)
resolver = MethodResolver(reg)
.new / .copy handled specially
copy = resolver.resolvemethod(inst, "copy", [])
`
| Method name | Behavior |
| --- | --- |
| new | Construct instance; positional args fill fields in declaration order |
| copy | Shallow copy of field dict |
| other | Lookup UserDefinedType.methods; missing → AttributeError |
Unknown field get/set on ObjectInstance raises AttributeError with type name context.
Relation to ASDL typequal
| ASDL | TypeQualifier |
| --- | --- |
| Const | CONST |
| Input | INPUT |
| Simple | SIMPLE |
| Series | SERIES |
Builder produces Qualify(qualifier=…, value=…) for annotated types in the AST; the type system module is used when those need runtime/type-registry meaning (especially UDTs in builtins/evaluator).
Internals
Path
src/pynescript/ast/typesystem.py — sole definition site for the classes above.
Consumers (outside this doc’s page set, for orientation):
Evaluator UDT / collection code (evaluator/builtins/, matrixevaluator, etc.)
Future/static analysis hooks (LSP hover may grow toward this model)
Compatibility
Type.iscompatiblewith is currently shallow: BuiltinType compares by equality; the base method returns False for other pairs. Treat richer subtyping (series/simple lattice, numeric promotion) as not fully encoded here — bar-loop semantics still own much of the coercion behavior.
na and enum
Registered as builtin kinds for name lookup. Enum definitions in scripts appear as AST EnumDef; linking enum members into BuiltinTypeKind.ENUM instances is evaluator-side work.
Invariants
. Registry lookup prefers builtins over UDTs of the same name.
. Field presence is schema-true. Setting an undeclared field is an error (no open bags).
. .new does not type-check arguments against field types in the resolver — it assigns positionally.
. varip is metadata on Field, not a separate type qualifier enum member.
Worked examples
Model a simple UDT from Pine
`pine
type candle
float o
float h
float l
float c
`
`python
from pynescript.ast.typesystem import UserDefinedType, Field, floattype, TypeRegistry, MethodResolver, ObjectInstance
candle = UserDefinedType("candle")
for name in "o h l c".split():
candle.addfield(Field(name, floattype()))
reg = TypeRegistry()
reg.registertype(candle)
c = MethodResolver(reg).resolvemethod(ObjectInstance(candle), "new", [, , ., .])
assert c.getfield("h") ==
`
Qualified series float
`python
from pynescript.ast.typesystem import floattype, TypeQualifier
assert str(floattype(TypeQualifier.SERIES)) == "series float"
`
Failure modes
| Failure | Cause |
| --- | --- |
| AttributeError: Field '…' not found | Typo or outdated UDT schema |
| AttributeError: Method '…' not found | Method never addmethod’d; not new/copy |
| Silent type confusion | iscompatiblewith too weak — do not rely on it alone for safety |
| Name shadowing | UDT registered with a builtin name is unreachable via gettype |
See also
ASDL schema — Qualify, TypeDef, EnumDef`
Runtime — bar-loop use of values
Builder — parsing type declarations
Linter — style-level checks (not full typing)
---
FILE: docs/pyne/core/unparser.mdx
Unparser
The unparser is the inverse of the builder: ASDL nodes become source text with correct operator parenthesization and Pine idioms (=>, :=, var/varip, export, method vs function).
Abstract
src/pynescript/ast/unparser.py implements NodeUnparser(NodeVisitor) and a Precedence enum. helper.unparse(node) constructs a fresh unparser and returns "".join of the internal buffer. Output is semantically faithful, not a pretty-printer of original trivia: whitespace is normalized; comments that never became annotations are not reconstructed; annotation strings that were attached are re-emitted.
Round-trip tests (parse → unparse → parse) are the primary oracle for grammar/builder/unparser coherence.
Conceptual model
Parenthesization rule of thumb: if a child expression’s stored precedence is weaker (higher binding needed) than the parent operator, wrap the child in ().
Interface surface
``python
from pynescript.ast.helper import parse, unparse
src = """
//@version=
f(x) => x
plot(f(close))
"""
print(unparse(parse(src)))
`
Precedence (low → high binding)
| Level | Operators / forms |
| --- | --- |
| TEST | ternary ? : (weakest) |
| OR | or |
| AND | and |
| EQ | == != |
| INEQ / CMP | = |
| ARITH | + - |
| TERM | / % |
| FACTOR / NOT | unary + - not |
| ATOM | names, literals, calls, subscripts |
Precedence.next() steps one level tighter for right-hand associativity control.
Buffer helpers (internal but useful when subclassing)
| Method | Role |
| --- | --- |
| write(text) | Append fragments |
| fill(text="") | Newline + indent + text |
| block() | Indent context manager |
| delimit(start, end) | Wrap sub-output |
| requireparens(prec, node) | Conditional parentheses |
| itemsview / interleave | Comma-separated lists |
| buffered() | Capture substring generation |
Node coverage (high level)
| Node | Emission sketch |
| --- | --- |
| Script | annotations then body |
| FunctionDef | optional export/method, name(args) => body (inline single Expr or indented block) |
| TypeDef | type Name with fields then methods |
| EnumDef | enum Name body |
| Assign | annotations, export?, var/varip?, type?, target = value |
| ReAssign | target := value |
| AugAssign | target op= value |
| ForTo / ForIn / While / If / Switch | Pine control syntax; else if chain collapse |
| Import | import ns/name/version as alias? |
| BinOp / BoolOp / UnaryOp / Compare / Conditional | precedence-aware |
| Call / Attribute / Subscript / Name / Constant / Tuple | primaries |
Internals
Path
src/pynescript/ast/unparser.py — Precedence, NodeUnparser
Entry: helper.unparse → NodeUnparser().visit(node)
Visit vs traverse
visit resets source and calls traverse. traverse accepts lists (statement bodies) or single nodes, then dispatches via the visitor cache. This split lets statement lists and expression trees share code without double-buffering.
If / else if chains
visitIf detects orelse == [Expr(If(...))] and emits else if rather than nested else + if blocks — matching idiomatic Pine.
Type body ordering
visitTypeDef partitions body into field statements vs FunctionDef with method set, emitting fields first then methods.
Constants
String constants use JSON-style quoting helpers where needed; colors and numbers re-serialize from their Python values. Exact original quote style (single vs double) is not guaranteed.
Invariants
. Precedence tables must stay aligned with the parser’s expression layers. A mismatch causes either redundant parens (benign) or wrong binding after re-parse (severe).
. Annotations are part of the round-trip surface when attached to Script/FunctionDef/TypeDef/Assign.
. Unparser does not type-check. Ill-typed but well-shaped trees still print.
. Indent unit is four spaces (" " self.indent).
Worked examples
Precedence
`python
from pynescript.ast.helper import parse, unparse
Builder produces BinOp(Add, left=, right=BinOp(Mult, , )) or inverse depending on parse
print(unparse(parse(" + ", mode="eval")))
Expect something that re-parses to the same value: e.g. " + "
`
Function forms
`python
Single-expression body → f(x) => expr
Multi-statement body → f(x) =>\n stmt\n stmt
`
Reassignment vs declaration
`pine
// Assign with mode → var float x = .
// ReAssign → x := .
// AugAssign → x +=
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Missing visit output | Falls through genericvisit which only walks children — may emit empty string for leaf-like unhandled nodes |
| Extra parentheses everywhere | Over-conservative precedence |
| Re-parse differs in binding | Under-parenthesized output |
| Lost comments | Only annotation strings are stored; plain // comments are dropped after parse |
| Method printed as function | FunctionDef.method` flag not set by builder |
See also
Helper API
Builder
ASDL schema
Visitor / transformer
---
FILE: docs/pyne/core/visitor-transformer.mdx
Visitor & Transformer
Tree walks in PYNE follow the same dual pattern as CPython’s ast module: a visitor for analysis, a transformer for structural rewrite.
Abstract
NodeVisitor (src/pynescript/ast/visitor.py) — dispatch visit, default genericvisit walks children via iterfields. Method lookups are cached per visitor instance.
NodeTransformer (src/pynescript/ast/transformer.py) — same dispatch, but genericvisit replaces list items and fields according to return values (None removes, list splices, AST replaces).
Specialized visitors elsewhere:
| Class | Path | Role |
| --- | --- | --- |
| PinescriptASTBuilder | builder.py | CST visitor (ANTLR), not NodeVisitor |
| NodeUnparser | unparser.py | Source emission |
| StatementCollector | collector.py | Yield statements for annotations |
| Evaluator classes | evaluator/.py | Runtime interpretation |
Conceptual model
Interface surface
NodeVisitor
``python
from pynescript.ast.visitor import NodeVisitor
from pynescript.ast.helper import parse
class CallCounter(NodeVisitor):
def init(self):
super().init()
self.count =
def visitCall(self, node):
self.count +=
self.genericvisit(node)
tree = parse("plot(ta.sma(close, ))")
c = CallCounter()
c.visit(tree)
assert c.count >=
`
| Method | Contract |
| --- | --- |
| visit(node) | Resolve visit with cache; fallback genericvisit |
| genericvisit(node) | Recurse into AST fields and lists of AST |
Returns are whatever the override returns (analysis often returns None).
NodeTransformer
`python
from pynescript.ast import node as ast
from pynescript.ast.transformer import NodeTransformer
from pynescript.ast.helper import parse, unparse
class RenameClose(NodeTransformer):
def visitName(self, node: ast.Name):
if node.id == "close":
return ast.Name(id="open", ctx=node.ctx)
return node
tree = parse("plot(close)")
tree = RenameClose().visit(tree)
assert "open" in unparse(tree)
`
Return value semantics when transforming a list parent (e.g. body):
| Return | Effect |
| --- | --- |
| None | Remove this element |
| AST | Replace element |
| non-AST iterable | Splice multiple nodes in place of one |
| same node | Keep |
For a single AST field, None deletes the attribute (delattr); otherwise setattr replaces.
StatementCollector
Generator-style visitor used only for annotation attachment:
Yields FunctionDef, TypeDef, EnumDef, Assign, ReAssign, AugAssign, Import, Expr, Break, Continue.
Descends into nested structures and into structure-valued assignments.
Does not yield the structure nodes themselves as statements when they appear only as values — it yields their inner bodies.
`python
from pynescript.ast.collector import StatementCollector
from pynescript.ast.helper import parse
stmts = list(StatementCollector().visit(parse("f() => \nx = ")))
`
Internals
Paths
| File | Symbols |
| --- | --- |
| src/pynescript/ast/visitor.py | NodeVisitor |
| src/pynescript/ast/transformer.py | NodeTransformer |
| src/pynescript/ast/collector.py | StatementCollector, Structure tuple |
| src/pynescript/ast/helper.py | iterfields, iterchildnodes, walk |
Dispatch cache
`text
nodeclass = type(node).name
visitor = cache.get(nodeclass) or getattr(self, "visit" + nodeclass, genericvisit)
`
Subclass instances that dynamically add methods after first visit of a type will not see them unless the cache is cleared — prefer defining methods on the class body.
In-place lists
Transformers mutate list fields with oldvalue[:] = newvalues rather than rebinding, so aliased references to the same list see updates.
Invariants
. Always call super().init() on subclasses so visitorcache exists.
. Transformers should return the node from overrides that use genericvisit (the default returns node after rewriting children).
. Do not assume identity stability after a transformer pass if you replaced roots.
. walk is not a visitor — it is a pure BFS helper without dispatch.
Worked examples
Strip all Expr statements that call plot
`python
from pynescript.ast import node as ast
from pynescript.ast.transformer import NodeTransformer
from pynescript.ast.helper import parse, unparse
class DropPlots(NodeTransformer):
def visitExpr(self, node):
if isinstance(node.value, ast.Call):
func = node.value.func
if isinstance(func, ast.Name) and func.id == "plot":
return None
return self.genericvisit(node)
tree = DropPlots().visit(parse("x = \nplot(x)\n"))
print(unparse(tree))
`
Collect all string constants
`python
from pynescript.ast.visitor import NodeVisitor
from pynescript.ast.helper import parse
class Strings(NodeVisitor):
def init(self):
super().init()
self.found = []
def visitConstant(self, node):
if isinstance(node.value, str):
self.found.append(node.value)
s = Strings()
s.visit(parse('indicator("hello")'))
`
Failure modes
| Failure | Cause |
| --- | --- |
| AttributeError: visitorcache | Forgot super().init() |
| Transformer silently no-ops | Override returns nothing (None) unintentionally → node deleted |
| Infinite recursion | visitX calls self.visit(node) on same node without change |
| Partial rewrite | Overrode visit without generic_visit — children untouched |
| Collector misses nested assign | Structure not in Structure` tuple and not assigned as value |
See also
Helper API
Unparser
Builder
ASDL schema
---
FILE: docs/pyne/devops/ci.mdx
Continuous integration
Abstract
CI is the public invariant check for the monorepo-like layout. A single PR can touch the Python evaluator, the Flask API, the VS Code extension, the AXIS PWA, and a Cloudflare Worker typecheck — the workflow graph is deliberately fan-out with concurrency cancellation on newer pushes to the same ref.
Primary definition: .github/workflows/ci.yml. Nightly AXIS expansion lives in axis-nightly.yml. Release builds are separate (release.yml).
Conceptual model
Interface surface
Triggers
| Workflow | On | Purpose |
| --- | --- | --- |
| ci.yml | push/PR to main | Required-path confidence |
| axis-nightly.yml | cron + workflowdispatch | Full Playwright + security + coverage gate |
| release.yml | tags v + dispatch | LSP binaries + VSIX + GitHub Release |
ci.yml jobs
| Job | Runtime | What it asserts |
| --- | --- | --- |
| lint | Python . | ruff check src/ tests/; mypy src/ --ignore-missing-imports |
| test | Matrix .–. | Editable .[lsp]; LSP unit/ee tests; backend tests; Codecov upload on . only |
| backend-test | . | Explicit Flask/numpy/matplotlib install + testbackend.py |
| vscode-ext-test | Node | npm ci → compile → vsce package → upload VSIX artifact (-day) |
| pwa | Bun . | Root + worker install, dual tsc, frontend coverage gate + security tests; upload lcov |
| axis-ee | Bun . | Playwright Chromium smoke (test:ee:smoke) |
| cf-worker | Bun . | frontend/worker typecheck only |
Concurrency:
``yaml
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
`
Nightly (axis-nightly.yml)
Full ee suite (bun run test:ee) with report artifact on failure
Coverage gate + security tests (stricter than PR smoke)
Internals
| Path | Role |
| --- | --- |
| .github/workflows/ci.yml | PR/main fan-out |
| .github/workflows/axis-nightly.yml | Scheduled AXIS depth |
| .github/workflows/release.yml | Tag release (see Release) |
| frontend/scripts/check-coverage.mjs | Coverage gate helper for AXIS |
| codecov action | Optional; failciiferror: false |
Secrets referenced in CI surface
| Secret | Used by | Notes |
| --- | --- | --- |
| VSCEPAT | VSIX package / publish | Extension packaging |
| METADATAKEY | Release / Cloud Build encrypt | Stable Fernet key for metadata |
| GITHUBTOKEN | Release upload | Default Actions token |
— CI test job currently does not run the full tests/ corpus on every matrix cell; it prioritizes LSP + backend. Local make test is broader.
Invariants & edge cases
. Fail-fast is off on the Python matrix — one version failure does not hide others.
. Codecov only on . to avoid noisy multi-upload; coverage XML path depends on pytest-cov configuration in the job.
. PWA install is resilient: bun install --frozen-lockfile || bun install — lockfile drift should still be fixed, not papered forever.
. VSIX artifacts are not the Marketplace. Packaging on every PR proves buildability; publish is release-gated.
. Worker typecheck ≠ deploy. cf-worker job does not call wrangler deploy.
Worked examples
Reproduce lint locally as CI does
`bash
pip install ruff flake mypy types-python-dateutil antlr-python-runtime
ruff check src/ tests/ --output-format=github
mypy src/ --ignore-missing-imports
`
Reproduce Python job slice
`bash
pip install -e ".[lsp]"
pip install pytest pytest-cov pytest-xdist pytest-asyncio
python -m pytest tests/testlangserver.py tests/testlspfeatures.py -v --tb=short
python -m pytest tests/testbackend.py -v --tb=short
`
Reproduce PWA job
`bash
bun install --frozen-lockfile
cd frontend/worker && bun install --frozen-lockfile && bunx tsc --noEmit
cd ../.. && bunx tsc -p tsconfig.json
cd frontend && bun install && bun run test:coverage:gate && bun run test:security
`
Failure modes
| Symptom | Likely cause | Fix |
| --- | --- | --- |
| Matrix red only on . | Version-specific typing / API | Guard with sys.version_info or drop unsupported API |
| Playwright smoke flake | Timing / missing deps | Install chromium with deps; re-run; check nightly for full suite |
| VSIX job fails on vsce` | Node/engine mismatch | Pin Node as workflow does |
| Coverage gate red | New untested AXIS paths | Add unit tests or adjust gate intentionally |
| Cancelled mid-run | Newer push to same branch | Expected concurrency behavior |
See also
Release
Local development
Observability
Security
---
FILE: docs/pyne/devops/docker.mdx
Docker
Abstract
Containers package the Pro API (Flask + evaluator) for reproducible local runs and Cloud Run deploys. Two Dockerfiles express different trade-offs: a simple single-stage image for development, and a multi-stage production image with non-root user, gunicorn, and healthchecks.
Compose wires volume mounts and optional Redis/LSP profiles for desk development.
Conceptual model
Interface surface
Dockerfiles
| File | Audience | Process model |
| --- | --- | --- |
| Dockerfile | Local / compose | CMD ["python", "-m", "backend.app"], Python .-slim, exposes |
| Dockerfile.api | Production / make docker-build | Multi-stage; USER appuser; gunicorn workers × threads; healthcheck curl |
Make / compose
``bash
make docker-build docker build -f Dockerfile.api -t pynescript-api .
make docker-run docker compose up api --build
`
docker-compose.yml services:
| Service | Profile | Notes |
| --- | --- | --- |
| api | default | Port :, FLASKENV=development, ro mounts of src/ + backend/ |
| redis | with-redis | redis:-alpine on |
| lsp | with-lsp | Same image, python -m pynescript.langserver |
Network name: pynescript-dev.
Cloud Build image tags
From cloudbuild.yaml:
`
gcr.io/$PROJECTID/pynescript/pynescript-pro-api:$COMMITSHA
gcr.io/$PROJECTID/pynescript/pynescript-pro-api:latest
`
Internals
Production stages (Dockerfile.api)
. Builder: install build-essential, pip install --prefix=/install -r backend/requirements.txt
. Runtime: copy install prefix, copy backend/ + src/, pip install ., drop privileges to appuser
. Env: PORT=, FLASKENV=production
. Healthcheck: HTTP GET / every s
. CMD:
`text
gunicorn --bind : --workers --threads --timeout backend.app:app
`
Dev image (Dockerfile)
Copies backend/requirements.txt, pyproject.toml, README/LICENSE, installs requirements, copies src + backend, pip install ., exposes .
Compose healthchecks
API: curl -f http://localhost:/ (inside container). Redis: redis-cli ping.
Invariants & edge cases
. Host port vs container . AXIS local docs assume API on when using compose; Cloud Run uses .
. Read-only mounts in compose mean image reinstall is required for dependency changes; source edits hot-reload only if the process reloads Python modules.
. Production image installs the package non-editable — good for immutability, bad for live code mount (use compose + Dockerfile for that).
. Non-root appuser cannot write arbitrary paths; configure APIKEYSTORE to a writable volume if using file-backed keys.
. Cloud Build currently builds context . with default Dockerfile in cloudbuild.yaml — confirm which Dockerfile is intended for production deploys when changing images. if Cloud Build should pin -f Dockerfile.api.
Worked examples
Production-like local API
`bash
docker build -f Dockerfile.api -t pynescript-api .
docker run --rm -p : \
-e FLASKENV=production \
-e ALLOWEDORIGINS=http://localhost: \
pynescript-api
curl -s http://...:/ | jq .
`
Compose with Redis
`bash
docker compose --profile with-redis up --build
`
Optional LSP container
`bash
docker compose --profile with-lsp up lsp
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Healthcheck fails | curl missing in image | Production Dockerfile installs curl; slim custom images may not |
| Permission denied writing keys | Running as appuser | Mount writable volume; set APIKEYSTORE |
| Import errors after mount | PYTHONPATH / non-editable install | Prefer compose dev Dockerfile + pip install -e patterns |
| OOM under load | Single small instance | Raise Cloud Run memory; reduce concurrency |
| CORS failures from PWA | Origin not allowed | Set ALLOWEDORIGINS` |
See also
GCP
Security
Pro API lifecycle
Local development
---
FILE: docs/pyne/devops/gcp.mdx
GCP & Cloud Run
Abstract
Production-shaped hosting for the Pro API targets Google Cloud Run, fed by Cloud Build (cloudbuild.yaml). The pipeline builds a container, pushes multi-tags to Container Registry, deploys a managed service, and can optionally compile the LSP binary in the same build graph.
This page codifies the as-checked-in config and the cost envelope from docs/gcpcostestimate.md — treat dollar figures as planning estimates, not invoices.
Conceptual model
Interface surface
Substitutions (cloudbuild.yaml)
| Substitution | Default | Meaning |
| --- | --- | --- |
| SERVICENAME | pynescript-pro-api | Cloud Run service |
| REGION | us-central | Deploy region |
| REPOSITORY | pynescript | Image path segment |
| METADATAKEY | (secret/subst) | Fernet key → CRYPTOKEY for LSP stage |
Build steps
. build — docker build with BuildKit; tags $COMMITSHA + latest; cache-from latest
. push — docker push --all-tags
. deploy — gcloud run deploy with managed platform flags
. build-lsp — python:.-slim image runs scripts/build/cibuild.py --jobs= with CRYPTOKEY=${METADATAKEY}
Cloud Run flags (as configured)
| Flag | Value | Rationale |
| --- | --- | --- |
| --allow-unauthenticated | on | Public HTTP API (auth via API keys at app layer) |
| --min-instances | | Scale to zero |
| --max-instances | | Cap spend / blast radius |
| --memory | Mi | Evaluator headroom |
| --cpu | | Single vCPU per instance |
| --concurrency | | Parallel requests per instance |
| --timeout | s | Aligns with gunicorn timeout |
| --set-env-vars | FLASKENV=production | App mode |
Machine / budget knobs
``yaml
options:
machineType: EHIGHCPU
logging: CLOUDLOGGINGONLY
timeout: s
`
Internals
| Path | Role |
| --- | --- |
| cloudbuild.yaml | Pipeline definition |
| Dockerfile / Dockerfile.api | Image recipes (confirm which Cloud Build uses) |
| backend/app.py | Health /, CORS, body size limit |
| docs/gcpcostestimate.md | Scenario costs (April draft) |
Cost envelope (planning)
| Scale | Rough monthly | Notes |
| --- | --- | --- |
| Free / hobby | ~$ | Within Cloud Run free tier if tiny |
| MVP (~ pro users) | ~$ | Cloud Run + micro SQL |
| Growth (~) | ~$ | Add Redis caching |
| Scale (~) | ~$ | Dominated by evaluator CPU-sec |
Dominant cost driver: script execution CPU time (~– CPU-sec per typical backtest over k bars). Memory ~–MB per concurrent Python worker.
Free-tier leverage (GCP)
Cloud Run: M requests, K CPU-sec, K GB-sec / month
Cloud Build: build-minutes / day
Artifact Registry / GCS small free slices
Invariants & edge cases
. App-layer auth still required when Cloud Run allows unauthenticated invoke — see Security.
. Scale-to-zero cold starts add latency; min-instances= trades money for snappiness.
. LSP build step does not deploy the binary to Cloud Run; it is a release-adjacent compile in the same YAML.
. Mi may be tight for large OHLCV payloads — watch OOM kills; raise memory before raising concurrency.
. Cost estimate doc also lists Railway / Render / Fly / self-host alternatives if GCP is overkill.
Worked examples
Manual deploy sketch (operator)
`bash
gcloud builds submit --config cloudbuild.yaml \
--substitutions=METADATAKEY="$METADATAKEY"
`
Smoke health after deploy
`bash
curl -s "https://SERVICE-URL/"
expect JSON status healthy, service pynescript-pro-api
``
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Deploy timeout | Nuitka step + docker on min budget | Split LSP compile to release pipeline |
| / queue | max-instances or concurrency | Raise caps carefully; add queue |
| High bill | No cache; chatty clients | Redis TTL; Cloud Tasks for backtests |
| Auth bypass illusion | Unauthenticated Cloud Run | Enforce API keys in Flask |
| Image pull errors | Wrong project / registry perms | IAM on GCR + Cloud Run service agent |
See also
Docker
Observability
Security
Pro API
---
FILE: docs/pyne/devops/index.mdx
DevOps
Abstract
PYNE is not a single binary. It is a multi-surface product: a pure-Python library, a Click CLI, a pygls Language Server, a Flask Pro API, a VS Code extension, an AXIS PWA AXIS, and colocated edge workers. DevOps here means keeping those surfaces buildable, testable, and deployable without drifting from the same evaluate contract.
This tab documents the operational graph: local loops, GitHub Actions, release tags, Docker images, Nuitka compilation, Fernet-encrypted LSP metadata, GCP Cloud Build/Run, observability hooks, and security controls.
Conceptual model
Invariant: CI green on main is necessary but not sufficient for a release. Tag-triggered release workflows build platform LSP binaries and VSIX; Cloud Build deploys the API image on its own pipeline.
Interface surface
| Concern | Entry | Doc |
| --- | --- | --- |
| Local install & loops | Makefile, hatch envs | Local development |
| PR / push CI | .github/workflows/ci.yml | CI |
| Versioned LSP + VSIX | .github/workflows/release.yml | Release |
| Pro API containers | Dockerfile, Dockerfile.api, docker-compose.yml | Docker |
| Compiled LSP | scripts/build/compile.py, cibuild.py | Nuitka build |
| Builtin metadata crypto | Fernet key + .enc | Metadata crypto |
| Cloud Run | cloudbuild.yaml | GCP |
| Logs / health | Flask /, gunicorn, Cloud Logging | Observability |
| Auth, CORS, secrets | backend/middleware/auth.py | Security |
Internals (repo map)
| Path | Role |
| --- | --- |
| Makefile | Human-facing orthography: install, test, lint, build, docker, worker |
| pyproject.toml | Hatch envs (test, lint, docs), optional extras (lsp, data) |
| .github/workflows/ | ci.yml, release.yml, axis-nightly.yml |
| scripts/build/ | Nuitka compile + CI build + Fernet metadata stage |
| scripts/generatebuiltinmetadata.py | Regenerates LSP builtinmetadata.json from live builtins |
| Dockerfile / Dockerfile.api | Dev vs multi-stage production API images |
| docker-compose.yml | Local API (+ optional Redis / LSP profiles) |
| cloudbuild.yaml | Build → GCR push → Cloud Run deploy; optional LSP compile |
| vscode-extension/ | Node extension package / vsce |
Invariants & edge cases
. Two console scripts, two entrypoints. pynescript (Click) ≠ pynescript-lsp (pygls). Ops scripts must not conflate them.
. Generated artifacts are not hand-edited. ANTLR/ASDL under generated/ and builtinmetadata.json are code-derived.
. Fernet key is gitignored. Without a stable CRYPTOKEY / METADATAKEY secret, every CI encrypt produces a different .enc blob (harmless functionally, bad for reproducibility).
. Python matrix vs local default. CI tests .–.; Nuitka release currently pins ..
. AXIS nightly is not PR-blocking. Full Playwright + security gates run on schedule (axis-nightly.yml).
Worked examples
``bash
Fast local confidence loop
make install
make lint
make test-lsp
make build-check import check only, ~s, no Nuitka compile
API in Docker (production Dockerfile)
make docker-build
make docker-run
`
Failure modes
| Symptom | Likely cause | Fix |
| --- | --- | --- |
| CI lint red, local green | Different ruff/mypy versions | Match CI install pins in workflow |
| Metadata decrypt fails in binary | Missing key at runtime | Set PYNESCRIPTMETADATA_KEY or embed key at build |
| Cloud Run after deploy | Image missing deps / gunicorn bind | Confirm Dockerfile.api CMD and PORT= |
| VSIX empty / missing | npm ci / vsce not run | Use make build-vscode or release job artifacts |
| Nuitka Anaconda link error | Static libpython missing | conda install libpython-static or keep --static-libpython=no` |
See also
Contributing
LSP architecture
Pro API contract
pine-worker
---
FILE: docs/pyne/devops/local-dev.mdx
Local development
Abstract
A PYNE checkout is a polyglot workspace: Python (src-layout package + Flask backend), TypeScript/Bun (AXIS frontend, Cloudflare worker, pine-worker), and Node (VS Code extension). Local DevOps is about choosing the right orthography for the surface you are changing without spinning up the entire mesh.
Conceptual model
Interface surface
Make targets (primary)
| Target | Effect |
| --- | --- |
| make install | pip install -e ".[lsp]" |
| make install-bun | Root + frontend/worker Bun deps |
| make test | Frontend Bun tests + full pytest tests/ |
| make test-lsp | testlangserver.py + testlspfeatures.py |
| make test-backend | tests/testbackend.py |
| make lint / make fmt | Ruff check / format on src/, tests/, backend/ |
| make typecheck | Root TS + frontend worker tsc |
| make build | Nuitka LSP via scripts/build/compile.py --jobs= |
| make build-check | Fast import resolution only (~s) |
| make build-vscode | Compile + package VSIX |
| make run | python -m backend.app |
| make run-lsp | python -m pynescript.langserver |
| make run-frontend | AXIS PWA on : (expects API on :) |
| make docker-build / docker-run | Production API image / compose |
Hatch equivalents
``bash
hatch run test:test
hatch run test:test-cov
hatch run lint:style
hatch run lint:typing
hatch run lint:gen-parser ANTLR regeneration — see grammar guide
`
See also CONTRIBUTING.md and AGENTS.md for hatch-first workflows.
Internals
| Path | Why it matters locally |
| --- | --- |
| src/pynescript/ | Editable package root (src-layout) |
| tests/conftest.py | Parametrizes pinescriptfilepath over every tests/data/builtinscripts/.pine (~hundreds of cases) |
| tests/data/library/ | Reference corpus only — not auto-parametrized |
| backend/requirements.txt | Extra deps for Pro API (Flask, numpy, matplotlib, …) |
| pine-worker/ | Colocated TS port; Bun tests independent of Python wheel |
Python requirements
Requires-Python: >=. (pyproject.toml)
CI matrix: .–.
Recommended local: . or . with a virtualenv / hatch env
Optional extras
`bash
pip install -e ".[lsp]" pygls + lsprotocol
pip install -e ".[dev-lsp]" + pytest-lsp for ee LSP
pip install -e ".[data]" ccxt for data providers
pip install -r backend/requirements.txt
`
Invariants & edge cases
. from future import annotations is mandatory on every new Python file (ruff isort required-imports).
. Do not edit generated grammar. Change only src/pynescript/ast/grammar/antlr/resource/.g, then regenerate.
. No stale backups in src/. Live builder.py and technical.py are sole sources of truth.
. Corpus fixture cost. Any test using pinescriptfilepath multiplies across the builtin script corpus. Narrow with --example-scripts-dir=... when iterating.
. Console scripts are separate. Installing .[lsp] gives both pynescript and pynescript-lsp; invoking the wrong one is a common first-day footgun.
Worked examples
Library + parse round-trip
`bash
make install
python -c "from pynescript.ast.helper import parse, unparse; print(unparse(parse('//@version=\nindicator(\"x\")\nplot(close)')))"
`
Pro API + AXIS
`bash
terminal
make run
terminal
make run-frontend
open http://...: — /run hits backend on : per compose/docs convention
`
LSP without Nuitka
`bash
make install
make run-lsp
point editor to stdio server: python -m pynescript.langserver
`
pine-worker only
`bash
cd pine-worker
bun install
bun test
bun run typecheck
`
Failure modes
| Symptom | Cause | Mitigation |
| --- | --- | --- |
| ModuleNotFoundError: pynescript | Not editable-installed | make install |
| Hundreds of unexpected test failures | Editing shared conftest fixture | Isolate unit tests; avoid corpus fixture |
| Frontend fails calling API | Backend not on expected port | Start make run; check CORS / ALLOWEDORIGINS |
| antlr not found | CLI not on PATH | Use hatch lint:gen-parser or full path to antlr |
| Ruff I import errors | Missing future annotations | Add from future import annotations` |
See also
CI
Docker
Nuitka build
Contributing
Installation (end user)
---
FILE: docs/pyne/devops/metadata-crypto.mdx
Metadata encryption
Abstract
LSP completion, hover, and signature help depend on a large builtin metadata document derived from the live evaluator surface. In open development the JSON is plaintext and git-tracked. For compiled Nuitka binaries, the build pipeline can ship an encrypted blob so the binary does not leave a trivially scrapable metadata file on disk.
Mechanism: Fernet (symmetric, cryptography package) + optional SHA integrity sidecar.
Conceptual model
Interface surface
Files
| Path | Tracked? | Role |
| --- | --- | --- |
| src/pynescript/langserver/providers/builtinmetadata.json | yes | Plaintext for dev / hatch |
| …/builtinmetadata.json.enc | often yes | Encrypted payload for binaries |
| …/builtinmetadata.json.sha | yes | Truncated SHA of plaintext |
| scripts/build/.metadata.key | no (gitignored) | Local Fernet key material |
| scripts/generatebuiltinmetadata.py | yes | Regenerates JSON from code |
Environment variables
| Variable | Context | Meaning |
| --- | --- | --- |
| CRYPTOKEY | Build scripts / CI / Cloud Build | Key bytes for encrypt stage |
| PYNESCRIPTMETADATAKEY | Runtime decrypt fallback | Env-supplied key if file missing |
| GitHub secrets.METADATAKEY | Actions | Stable secret → CRYPTOKEY |
| Cloud Build METADATAKEY | cloudbuild.yaml substitution | CRYPTOKEY=${METADATAKEY} |
Code entrypoints
Encrypt stage: scripts/build/compile.py (encryptmetadata), scripts/build/cibuild.py (stagemetadata)
Decrypt: src/pynescript/langserver/providers/metadatadecrypt.py
Loader preference: plaintext if present, else encrypted (getmetadatacached)
Internals
Generation (always code-derived)
``bash
python scripts/generatebuiltinmetadata.py
`
Do not hand-edit the JSON for permanent feature work — re-run the generator after adding builtins.
Encrypt (local sketch)
`python
from cryptography.fernet import Fernet
key = Fernet.generatekey() or load stable secret
fernet = Fernet(key)
encrypted = fernet.encrypt(plaintextbytes)
`
Build scripts chmod the key file to o when writing .metadata.key.
Decrypt integrity check
After Fernet decrypt, if a .sha sidecar exists, the loader compares sha(plaintext)[:] and raises on mismatch.
Dev vs binary
| Mode | Behavior |
| --- | --- |
| Editable install / repo checkout | Prefer plaintext JSON |
| Nuitka onefile | Encrypted data dir + key resolution via MEIPASS / env |
| Missing both | FileNotFoundError directing to compile script |
Invariants & edge cases
. Stable key ⇒ reproducible .enc. Random key every CI run yields a different ciphertext even if plaintext is identical — not a functional bug, but pollutes diffs and caches.
. Plaintext remains in git for open-source transparency of the language surface. Encryption protects the bundled binary layout, not the secret of the API catalog.
. Truncated hash is integrity, not secrecy. It detects bitrot / wrong plaintext, not attackers with the key.
. Never commit .metadata.key. Rotate if leaked; regenerate .enc with the new key for binary builds.
. After adding builtins: regenerate metadata, re-encrypt with the same CI key, commit updated JSON (and .enc if tracked).
Worked examples
Regenerate + local encrypt
`bash
python scripts/generatebuiltinmetadata.py
pip install cryptography
python scripts/build/compile.py --check stages metadata paths as configured
`
CI snippet
`yaml
name: Build LSP binary
run: python scripts/build/cibuild.py --jobs
env:
CRYPTOKEY: ${{ secrets.METADATAKEY }}
`
Runtime with env key
`bash
export PYNESCRIPTMETADATAKEY="$(cat scripts/build/.metadata.key)"
pynescript-lsp
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| No metadata decryption key found | Missing file + env | Supply PYNESCRIPTMETADATAKEY or rebuild with key |
| Metadata integrity check failed | Stale hash or wrong ciphertext | Re-encrypt from current plaintext; commit both |
| Completion empty in binary only | Data dir not included in Nuitka | Fix --include-data-dir |
| Divergent .enc every CI | Unset CRYPTOKEY | Configure METADATAKEY` secret |
See also
Nuitka build
LSP builtin metadata
Release
Security
---
FILE: docs/pyne/devops/nuitka-build.mdx
Nuitka LSP build
Abstract
The Language Server can run as pure Python (python -m pynescript.langserver) or as a Nuitka-compiled binary for editor distribution. Compilation freezes the pygls server, providers, and (when configured) encrypted builtin metadata into a deployable artifact under dist/.
Primary tools: scripts/build/compile.py (local) and scripts/build/cibuild.py (CI/Cloud Build).
Conceptual model
Interface surface
Commands
``bash
Prerequisites
pip install nuitka cryptography
Full onefile build (default path via Make)
make build
≡ python scripts/build/compile.py --jobs=
Fast import check only (~s)
make build-check
≡ python scripts/build/compile.py --check
Explicit modes
python scripts/build/compile.py --standalone
python scripts/build/compile.py --onefile --jobs
python scripts/build/cibuild.py --jobs=
`
Build modes (approx. wall time)
| Mode | Time | Notes |
| --- | --- | --- |
| --check | ~s | Import resolution / no full compile |
| --standalone | – min | Directory distribution, faster iteration |
| --onefile | – min | Self-extracting single binary |
| CI cores | – min | High parallelism |
Expected layout
`text
dist/
├── lsp/
│ └── pynescript-lsp onefile (path may vary by script)
├── vsix/
│ └── pynescript-lsp.vsix optional bundle
└── pynescript-lsp cibuild final move target (script-dependent)
`
Also see scripts/build/README.md.
Internals
Key Nuitka flags (compile.py)
--static-libpython=no — default; critical for many Linux/Anaconda envs
--python-flag=nosite,nodocstrings
--include-data-dir=…/providers=pynescript/langserver/providers
--lto=auto, --jobs=N
--onefile or --standalone
--follow-imports on local compile path
Entry module
Compilation targets src/pynescript/langserver (package entry), product name pynescript-lsp.
Metadata stage
Before compile, scripts may:
. Run scripts/generatebuiltinmetadata.py if plaintext metadata missing
. Fernet-encrypt to builtinmetadata.json.enc
. Write truncated SHA sidecar
. Write scripts/build/.metadata.key (gitignored) or consume CRYPTOKEY env in CI
Anaconda
If static libpython is unavailable:
`bash
conda install libpython-static
or rely on --static-libpython=no (already the default in compile.py)
`
Invariants & edge cases
. C compiler required (gcc/clang/MSVC). Containers need build-essential or equivalent if compiling inside Docker.
. Python .+; release workflow pins ..
. Providers data dir must ship — completion/hover collapse without metadata.
. Onefile startup cost includes extract-to-temp; standalone is snappier for local soak tests.
. Do not commit scripts/build/.metadata.key.
Worked examples
Local onefile with explicit jobs
`bash
python scripts/build/compile.py --jobs "$(nproc)"
find dist -name 'pynescript-lsp' -type f
`
CI-shaped build
`bash
export CRYPTOKEY="$METADATAKEYSTABLE"
python scripts/build/ci_build.py --jobs=
`
Verify without compile
`bash
python scripts/build/compile.py --check
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Link error on libpython | Anaconda static lib missing | Install static lib or keep --static-libpython=no |
| Binary missing builtins | Data dir not included | Confirm --include-data-dir` providers path |
| Huge binary | Full stdlib pull-in | Review Nuitka plugins / unused deps |
| CI timeout | Low cores / onefile | Raise jobs; use larger runner; prefer standalone intermediate |
| Decrypt fail at runtime | Key not embedded / env unset | See Metadata crypto |
See also
Metadata crypto
Release
LSP builtin metadata
---
FILE: docs/pyne/devops/observability.mdx
Observability
Abstract
PYNE’s production surface today optimizes for simple, hostable signals rather than a full OpenTelemetry mesh: HTTP health checks, process managers (gunicorn), cloud platform logs, CI artifacts (coverage, Playwright reports), and application-level API key usage counters. Treat this page as the map of what exists in-repo, not a claim of enterprise APM parity.
Conceptual model
Interface surface
Health
backend/app.py exposes:
``http
GET /
→ { "status": "healthy", "service": "pynescript-pro-api", … }
`
Used by:
Dockerfile.api HEALTHCHECK (curl -f http://localhost:/)
docker-compose.yml api healthcheck
Load balancers / Cloud Run startup-ish probes (platform-dependent)
Process model (production)
`text
gunicorn --bind : --workers --threads --timeout backend.app:app
`
| Knob | Effect on signals |
| --- | --- |
| workers | Process isolation; multiplies memory |
| threads | Concurrent requests within worker |
| timeout s | Aligns with Cloud Run --timeout=s — long evals die predictably |
Logs
| Environment | Where logs go |
| --- | --- |
| Local make run | Process stdout/stderr |
| Docker | Container logs (docker logs, compose) |
| Cloud Build | logging: CLOUDLOGGINGONLY |
| Cloud Run | Cloud Logging (request + container) |
Application modules use standard library / Flask logging patterns; there is no mandatory structured-log schema enforced repo-wide. if a future branch introduces OpenTelemetry exporters.
CI / quality signals
| Signal | Source |
| --- | --- |
| Ruff / mypy | ci.yml lint job |
| Pytest results | LSP + backend jobs |
| Codecov (optional) | Python . matrix cell |
| AXIS coverage lcov | axis-coverage-lcov artifact |
| Playwright report | Uploaded on nightly/ee failure |
| Security tests | bun run test:security (PWA job + nightly) |
Usage metering (app-layer)
backend/middleware/auth.py tracks per-key:
callsused / callslimit / tier
lastused timestamps
These are business metrics for rate limits, not Prometheus time series — export them if you need dashboards.
Internals
| Path | Role |
| --- | --- |
| backend/app.py | Health route, CORS, MAXCONTENT_LENGTH |
| backend/middleware/auth.py | Key store, rate limit counters |
| backend/services/backtest.py | Backtest metrics objects for API responses |
| Dockerfile.api | HEALTHCHECK + gunicorn CMD |
| cloudbuild.yaml | Cloud Logging-only build logs |
| logs/ (repo) | Local combined/error log files if generated by tools — not the production contract |
Invariants & edge cases
. Health ≠ readiness of evaluator warm caches. A on / does not mean the first /run will be fast (cold import / JIT / numba).
. Timeouts double-bind. Gunicorn s and Cloud Run s — the tighter effective limit is still s.
. Scale-to-zero hides idle metrics. No traffic ⇒ no samples; use synthetic uptime checks if SLOs matter.
. Coverage artifacts expire (– days retention on Actions uploads).
. Do not scrape secrets from logs. API keys and admin tokens must never be logged at info level.
Worked examples
Local health loop
`bash
make run
curl -s localhost:/ | python -m json.tool
`
Docker health
`bash
docker inspect --format='{{json .State.Health}}' CONTAINER
`
Tail Cloud Run (gcloud)
`bash
gcloud run services logs read pynescript-pro-api --region us-central --limit
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Health green, /run | Exception in evaluator path | Inspect request logs; reproduce with payload |
| Worker kills | Soft timeout | Optimize script / raise timeout consistently |
| Missing CI artifact | Path wrong / tests passed | if-no-files-found: ignore` hides absence |
| Rate limit false positives | Shared key / limit too low | Adjust tier limits; multi-key |
| Silent deploy failure | Build logging-only + unread | Check Cloud Build history UI |
See also
GCP
Docker
Security
API lifecycle
---
FILE: docs/pyne/devops/release.mdx
Release
Abstract
A PYNE release is a multi-artifact event, not a single wheel upload. The primary automation (.github/workflows/release.yml) builds platform-native Language Server binaries with Nuitka, packages the VS Code extension, attaches them to a GitHub Release, and optionally publishes to the VS Code Marketplace.
Library wheels may still be built separately via hatch / PyPI workflows; the documented release pipeline optimizes for editor distribution (LSP + VSIX).
Conceptual model
Interface surface
Triggers
| Event | Behavior |
| --- | --- |
| Push tag matching v | Full build + create release + marketplace publish |
| workflowdispatch with version input | Manual build path (release job still gated on tag push condition) |
Jobs
| Job | Matrix / OS | Output |
| --- | --- | --- |
| build-lsp | ubuntu-latest, windows-latest, macos-latest | Named artifacts: pynescript-lsp-linux-x, …-windows-x.exe, …-macos-arm |
| build-vscode | ubuntu-latest | pynescript-vscode-extension VSIX |
| release | needs both; only if tag push | GitHub Release body + asset globs |
| publish-vscode | needs VSIX; only if tag push | vsce publish --pat |
Environment
``yaml
env:
PYTHONVERSION: "."
NUITKAJOBS:
`
Build steps rely on:
`bash
pip install nuitka==.. cryptography
python scripts/build/compile.py --no-encrypt --check metadata stage
python scripts/build/cibuild.py --jobs
CRYPTOKEY from secrets.METADATAKEY
`
Local make-side packaging
`bash
make build scripts/build/compile.py --jobs=
make build-vscode npm install && compile && vsce package
`
Artifacts land under dist/lsp/, dist/vsix/, and vscode-extension/.vsix depending on script path.
Internals
| Path | Role |
| --- | --- |
| .github/workflows/release.yml | Orchestration |
| scripts/build/cibuild.py | CI-oriented Nuitka + metadata encrypt |
| scripts/build/compile.py | Local/full compile options (--onefile, --standalone, --check) |
| vscode-extension/package.json | Extension version / engines |
| src/pynescript/about.py | Hatch dynamic version for Python package |
Release asset contract (from release body)
| Artifact | Install sketch |
| --- | --- |
| pynescript-lsp-linux-x | chmod +x → /usr/local/bin/pynescript-lsp |
| pynescript-lsp-windows-x.exe | Place on PATH |
| pynescript-lsp-macos-arm | Place on PATH (Apple Silicon runners) |
| pynescript-.vsix | code --install-extension pynescript-.vsix |
Invariants & edge cases
. CRYPTOKEY must be stable across builds if you care about byte-identical builtinmetadata.json.enc. Supply secrets.METADATAKEY.
. Tag name is the version source for the GitHub Release title (v prefix stripped).
. Marketplace publish needs VSCEPAT. Without it, packaging may still succeed while publish fails.
. Artifact retention is short ( days on build jobs) — releases must attach files immediately.
. macOS artifact is ARM from macos-latest — Intel Mac users may need separate builds if required.
. Draft vs prerelease: workflow currently sets draft: false, prerelease: false for tag releases.
Worked examples
Cut a release
`bash
ensure main is green
git checkout main && git pull
bump versions in extension / about as needed
git tag -a v.. -m "v.."
git push origin v..
watch Actions → Build & Release LSP
`
Smoke-test a downloaded binary
`bash
chmod +x pynescript-lsp-linux-x
./pynescript-lsp-linux-x --help
or wire stdio into an editor client
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Binary not found after Nuitka | Path layout drift in cibuild.py | Inspect dist/; fix finder step |
| Decrypt errors in field | Key mismatch between encrypt and runtime | Align METADATAKEY with embedded key strategy |
| Empty GitHub Release assets | Artifact download path mismatch | Match files: globs to download-artifact layout |
| Marketplace | Bad/expired VSCEPAT | Rotate PAT with Marketplace scopes |
| Windows .exe not marked executable | Finder uses -type f -executable` | Fallback non-executable find branch in workflow |
See also
Nuitka build
Metadata crypto
CI
VS Code extension
---
FILE: docs/pyne/devops/security.mdx
Security
Abstract
PYNE security is layered: transport/hosting (Cloud Run, containers), application (API keys, admin tokens, CORS, body size caps), build secrets (Fernet metadata keys, VSCE PAT), and supply chain (CI dependency installs, extension packaging). This page documents controls that exist in code and workflows — not a full threat model certification.
Conceptual model
Interface surface
HTTP application controls (backend/app.py)
| Control | Default / mechanism |
| --- | --- |
| Body size | MAXCONTENTLENGTH = ( MiB) — DoS/OOM mitigation |
| CORS | ALLOWEDORIGINS env (comma-separated); default includes product hosts + localhost regex |
| Methods | GET, POST only via flask-cors config |
| Headers | Content-Type, Authorization, X-Admin-Token |
| Credentials | supportscredentials=False |
API keys (backend/middleware/auth.py)
Keys stored as hashes, not raw secrets (store path via APIKEYSTORE)
Tiers: free, hobby (k), pro (k), team (k), enterprise (∞)
Rate limit by monthly call counters (callsused / callslimit)
Decorators: requireapikey, requireadmintoken
Build / release secrets
| Secret | Purpose |
| --- | --- |
| METADATAKEY / METADATAKEY | Stable Fernet key for LSP metadata encrypt |
| VSCEPAT | VS Code Marketplace / vsce package |
| GITHUBTOKEN | Release asset upload |
| PYNESCRIPTMETADATAKEY | Runtime decrypt env fallback |
Container hardening (Dockerfile.api)
Multi-stage build (no compiler toolchain in final image)
Non-root appuser
Production env flags
Healthcheck without privileged ops
Internals
| Path | Role |
| --- | --- |
| backend/app.py | CORS, body limit, blueprint registration |
| backend/middleware/auth.py | Key store, tiers, decorators |
| backend/middleware/keystoreredis.py / keystoresqlite.py | Alternative stores |
| backend/middleware/schemas.py | Request validation schemas |
| src/pynescript/langserver/providers/metadatadecrypt.py | Encrypted metadata load |
| .github/workflows/.yml | Secret consumption |
| frontend security tests | bun run test:security in CI |
Audit notes encoded in source
Comments in app.py reference a -- audit (findings S body size, S/S CORS). Prefer reading the code over assuming defaults never change.
Invariants & edge cases
. Cloud Run --allow-unauthenticated does not mean open data. It means IAM does not gate invoke; the Flask app must.
. Free tier with callslimit= is special-cased as unlimited remaining in some helpers — verify tier semantics before production billing.
. Localhost CORS regex permits any port on localhost/... — fine for desk, wrong for prod ALLOWEDORIGINS.
. Fernet protects binary metadata packaging, not user script confidentiality.
. Admin token routes are high privilege — rotate and store separately from user API keys.
. Never log raw API keys. Hash lookups only.
Worked examples
Harden CORS for a single origin
``bash
export ALLOWEDORIGINS=https://app.example.com
python -m backend.app
`
Point key store at a volume
`bash
export APIKEYSTORE=/data/apikeys.json
ensure process user can write /data
`
Rotate metadata key (ops procedure sketch)
. Generate new Fernet key; store as secret METADATAKEY
. Rebuild LSP with CRYPTOKEY set
. Ship new binary; retire old key after client upgrade window
. Do not leave dual-key decrypt unless you implement it
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Browser CORS error | Origin not listed | Set ALLOWED_ORIGINS |
| Payload Too Large | Body > MB | Reduce OHLCV batch; raise limit knowingly |
| / on /run | Missing/invalid key | Check Authorization header scheme |
| Rate limited | Tier exhausted | Upgrade tier / wait reset |
| Secret in fork PR logs | Misconfigured workflow echo | Never echo secrets; use masked env |
| Root-owned files in volume | Old image ran as root | Align UID with appuser` |
See also
Auth and keys (API)
Metadata crypto
Docker
GCP
---
FILE: docs/pyne/enduser/getting-started/configuration.mdx
Configuration
Abstract
PYNE is mostly convention-over-config at the library layer: parse and evaluate take explicit arguments rather than a global config file. Configuration appears at three boundaries: install extras (what code is available), process environment (Pro API host, CORS, API keys store), and editor / LSP client settings (diagnostics, formatting, binary path). This page inventories those knobs for end users.
Conceptual model
There is no required pynescript.toml for core use.
Interface surface
Package extras (pyproject.toml)
| Extra | Dependencies | Enables |
| --- | --- | --- |
| (none) | antlr-runtime, click, requests, tqdm | CLI + AST + linter + literaleval |
| lsp | pygls, lsprotocol | pynescript-lsp |
| dev-lsp | pygls, lsprotocol, pytest-lsp | LSP + protocol tests |
| data | ccxt | Exchange historical data |
| datafeed | ccxt | Same as data (alias for realtime feed paths) |
Console scripts
| Name | Entry point | Notes |
| --- | --- | --- |
| pynescript | pynescript.main:cli | Always with core install |
| pynescript-lsp | pynescript.langserver.main:main | Needs importable pygls |
Invoke without scripts on PATH:
``bash
python -m pynescript --help
python -m pynescript.langserver
`
CLI flags that act as “config”
Per-invocation only (no persistence):
| Command | Notable options |
| --- | --- |
| parse-and-dump | --encoding, --indent, --output-file |
| parse-and-unparse | --encoding, --output-file |
| lint | --encoding, --fix, --fail-on {errors,warnings,all} |
| data | --provider, --period, --interval, --api-key, --secret, --exchange |
| download-builtin-scripts | --script-dir (required) |
--fix on lint is accepted by Click; auto-fix behavior may be incomplete — .
Pro API environment variables
Consumed by backend/app.py and middleware:
| Variable | Default | Purpose |
| --- | --- | --- |
| HOST | ... | Bind address when running main |
| PORT | | Bind port |
| ALLOWEDORIGINS | https://pynescript.ai, https://app.pynescript.ai, localhost regex | CORS allowlist (comma-separated; may include regex strings) |
| ADMINTOKEN | unset | Required for POST /auth/createkey; if unset, create returns |
| APIKEYSTORE | /root/pynescript/data/apikeys.json | Path for key store (override for local) |
Also relevant:
| Variable | Context |
| --- | --- |
| MAXCONTENTLENGTH | Hardcoded MiB on Flask app config (not env) — large OHLCV POSTs fail above this |
Local run:
`bash
export HOST=...
export PORT=
export ALLOWEDORIGINS="http://localhost:,http://...:"
export ADMINTOKEN="dev-only-token"
export APIKEYSTORE="$PWD/.data/apikeys.json"
python -m backend.app
or: make run
`
Data providers
CLI pynescript data / library providers:
| Provider | Auth | Notes |
| --- | --- | --- |
| mock | none | Deterministic offline bars (default) |
| yahoo | none | Yahoo Finance path |
| alphavantage | --api-key (falls back to demo) | Limited with demo key |
| ccxt | optional key/secret; --exchange | Requires [data] extra |
Pro API /run optional fields:
| Field | Values | Role |
| --- | --- | --- |
| datasource | "", mock, ccxt, ccxtpro, yahoo, alphavantage | Wires request. resolution |
| dataoptions | object | exchange, apikey, seed, … |
| mode | interpret (default), compile | AST walker vs compiled subset |
Editor / LSP client settings
VS Code extension keys (from clients/README.md):
| Setting | Default | Meaning |
| --- | --- | --- |
| pynescript.lsp.enabled | true | Toggle server |
| pynescript.lsp.command | pynescript-lsp | Binary path |
| pynescript.formatting.enabled | true | Format document |
| pynescript.diagnostics.enabled | true | Lint squiggles |
| pynescript.completion.snippets | true | Snippet completions |
Neovim (clients/neovim.lua):
`lua
settings = {
pinescript = {
formatting = { enabled = true },
diagnostics = { enabled = true },
completion = { snippets = true },
},
}
`
Zed: languageservers.pynescript.command / arguments: ["--stdio"] — see Editors.
AXIS / frontend coupling (optional)
When running SuperChart Lite PWA against local Flask:
| Make target | Port (typical) |
| --- | --- |
| make run | API : |
| make run-frontend | PWA : |
| make worker-dev | CF Worker : |
CORS must allow the PWA origin via ALLOWEDORIGINS.
Internals (repo paths)
| Path | Config surface |
| --- | --- |
| pyproject.toml | extras, scripts, Python requires |
| src/pynescript/main.py | CLI options |
| backend/app.py | Flask, CORS, HOST/PORT, body size |
| backend/middleware/auth.py | APIKEYSTORE, key tiers |
| backend/middleware/schemas.py | Request field defaults (mode, symbol, …) |
| clients/.json|lua|el | Editor LSP wiring |
| vscode-extension/ | Extension contribution points |
Invariants & edge cases
Unknown JSON fields on Pro API are rejected (UNKNOWNFIELDS) — schemas are strict.
CORS defaults exclude arbitrary production domains — set ALLOWEDORIGINS deliberately.
Key store default path is production-oriented (/root/...) — always override locally.
Lint --fail-on only affects process exit code; it does not change which rules run.
Recursion limit during parse is raised to at least temporarily inside helper.parse for deep nests; not user-configurable.
Worked examples
Desk-only machine
`bash
python -m venv .venv && source .venv/bin/activate
pip install pynescript
no env vars required
pynescript lint strategy.pine --fail-on errors
`
Editor workstation
`bash
pip install "pynescript[lsp]"
ensure which pynescript-lsp
paste clients/neovim.lua or clients/zed.json as documented
`
Local API + AXIS
`bash
pip install -e ".[lsp]"
pip install -r backend/requirements.txt
export APIKEYSTORE="$PWD/.data/apikeys.json"
export ALLOWEDORIGINS="http://localhost:"
make run
other terminal:
make run-frontend
`
Scripted Alpha Vantage fetch
`bash
export AVKEY=yourkey
pynescript data EUR/USD --provider alphavantage --api-key "$AVKEY" --period mo
`
Failure modes
| Symptom | Config fix |
| --- | --- |
| Browser CORS error on /run | Add origin to ALLOWEDORIGINS |
| createkey always | Set ADMINTOKEN and send X-Admin-Token |
| API keys vanish / permission error | Point APIKEYSTORE to writable path |
| Payload too large | Split bars or stay under MiB body limit |
| Editor “server failed to start” | Fix pynescript.lsp.command PATH; install [lsp] |
| ccxt import errors | Install [data]` extra |
See also
Installation
Pro API usage
Editors
CLI commands
API auth (systems)
AXIS docs
---
FILE: docs/pyne/enduser/getting-started/installation.mdx
Installation
Abstract
PYNE ships as the Python package pynescript (src-layout under src/pynescript/). The default install gives you the Click CLI (pynescript) and the AST library (parse, unparse, dump, walk, literaleval, linter). Optional extras add the Language Server (pynescript-lsp) and CCXT-backed market data. A Hatch-managed checkout is the supported path for contributors who need tests, grammar tooling, or the Flask Pro API in-tree.
Conceptual model
| Install flavor | What you get | When |
| --- | --- | --- |
| pip install pynescript | CLI + library | Desk use, scripts, CI lint |
| pip install "pynescript[lsp]" | + pygls LSP server | Editors |
| pip install "pynescript[data]" | + ccxt | Live/historical exchange data |
| pip install -e ".[dev-lsp]" | Editable + LSP test harness | Contributing to LSP |
| Hatch env | Matrix tests, lint, docs | Full monorepo workflow |
Interface surface
Prerequisites
Python ≥ . (classifiers declare .–.; . may work in CI — pin to .+ for production)
pip or an equivalent installer; virtualenv strongly recommended
For Pro API locally: flask, flask-cors, numpy, matplotlib (see backend/requirements.txt)
For VS Code extension build: Node + under vscode-extension/ (optional)
Core install (PyPI / local package)
From a release or built wheel:
``bash
pip install pynescript
`
From a cloned monorepo root (non-editable):
`bash
pip install .
`
Editable (preferred for development):
`bash
pip install -e .
or with Make:
make install
`
Extras
Defined in pyproject.toml under [project.optional-dependencies]:
`bash
Language Server (pygls + lsprotocol)
pip install "pynescript[lsp]"
CCXT for exchange data / datafeed
pip install "pynescript[data]"
alias:
pip install "pynescript[datafeed]"
LSP + pytest-lsp for protocol tests
pip install -e ".[dev-lsp]"
`
Combine extras:
`bash
pip install -e ".[lsp,data]"
`
Hatch (contributor)
`bash
Install Hatch if needed, then:
hatch shell
hatch run test:test
hatch run lint:style
`
Make targets used in-repo (see AGENTS.md):
`bash
make install pip install -e ".[lsp]"
make test pytest tests/ -v --tb=short
make lint ruff check src/ tests/ backend/
make run python -m backend.app
make run-lsp python -m pynescript.langserver
make build-check fast import check without Nuitka
`
Console scripts
After install, two entry points should resolve:
| Command | Purpose |
| --- | --- |
| pynescript | Click CLI group |
| pynescript-lsp | Language server (requires [lsp]) |
`bash
pynescript --version
pynescript --help
which pynescript-lsp only after [lsp]
`
Internals (repo paths)
| Path | Role |
| --- | --- |
| pyproject.toml | Package metadata, deps, optional extras, scripts |
| src/pynescript/ | Library + CLI + LSP package tree |
| src/pynescript/about.py | version (e.g. ..) |
| src/pynescript/main.py | CLI implementation |
| src/pynescript/langserver/main.py | pynescript-lsp entry |
| backend/requirements.txt | Pro API server deps (not in core extras) |
| Makefile | Convenience install / test / run targets |
| CONTRIBUTING.md | Hatch-based contributor workflow |
Base runtime dependencies (core):
antlr-python-runtime>=..
click>=..
requests
tqdm
Invariants & edge cases
License. Package license is AGPL-3.0-or-later (pyproject.toml). Link dynamically against the library; network use of modified versions requires offering corresponding source (AGPL §13).
Src layout. Imports are pynescript., not a flat package. Editable install requires hatchling to map src/.
LSP is not a CLI subcommand of pynescript in the current Click group. README sometimes says pynescript lsp; the registered script is pynescript-lsp.
Pro API is not a pip extra. Run from monorepo with backend deps, or call a hosted endpoint. Local: make run binds HOST/PORT (default ...:).
Anaconda / Nuitka. Building the onefile LSP binary may need libpython-static or --static-libpython=no (see build docs). End users rarely need this.
Generated artifacts. Do not hand-edit src/pynescript/ast/grammar/antlr/generated/ or ASDL generated modules.
Worked examples
Smoke-test CLI and library
`bash
python - /dev/null || true server may only speak stdio
`
Monorepo desk + Pro API
`bash
git clone https://github.com/jango-blockchained/pynescript.git
cd pynescript
pip install -e ".[lsp]"
pip install -r backend/requirements.txt
make run Flask on :
`
CI-style install
`bash
pip install .
or matrix:
pip install -e ".[dev-lsp]"
pytest tests/testparseandunparse.py -q
`
Failure modes
| Failure | Diagnosis | Fix |
| --- | --- | --- |
| ModuleNotFoundError: pynescript | Wrong env / not installed | Activate venv; reinstall |
| No module named 'pygls' | LSP import without extra | pip install "pynescript[lsp]" |
| No module named 'ccxt' | data CLI/provider path | pip install "pynescript[data]" |
| pynescript not found | Scripts not on PATH | Use python -m pynescript or fix venv bin/ |
| Import error from editable | Incomplete install / broken path | Re-run pip install -e . from repo root |
| Backend ImportError: flask | Pro API without backend deps | pip install -r backend/requirements.txt` |
| ANTLR / grammar mismatch on odd syntax | Older package vs newer scripts | Upgrade package; see Troubleshooting |
See also
Quick start
Configuration
CLI guide
Editors / LSP
Pro API usage
End User hub
---
FILE: docs/pyne/enduser/getting-started/quick-start.mdx
Quick Start
Abstract
This page is a linear path from a working install to the five desk operations most users need: parse, dump, unparse, lint, and evaluate. Full option trees live in CLI commands; deeper evaluation semantics live in Evaluate scripts.
Conceptual model
Interface surface
Assumes:
``bash
pip install pynescript
optional for data:
pip install "pynescript[data]"
`
Sample script in-tree: examples/rsistrategy.pine.
Internals (repo paths)
| Step | Implementation |
| --- | --- |
| CLI parse/dump | src/pynescript/main.py → parseanddump |
| CLI unparse | parseandunparse |
| CLI lint | lint → pynescript.ast.linter.lintscript |
| Library helpers | src/pynescript/ast/helper.py |
| Example scripts | examples/parsedumpunparse.py, examples/evaluateexpressions.py |
Invariants & edge cases
Always declare //@version= or //@version= at the top; the linter warns (W) if missing.
parse-and-unparse is a formatter via AST, not a semantic optimizer.
literaleval is for expressions / built-in calls with optional series context — not a full multi-bar strategy backtester. For bar loops, use backend.runtime.Runtime or the Pro API /run.
Filename arguments to the CLI must exist and be readable; lint may also read stdin (- or no file).
Worked examples
. Parse and dump the AST
`bash
pynescript parse-and-dump examples/rsistrategy.pine
`
Pretty-print with indent and optional file output:
`bash
pynescript parse-and-dump examples/rsistrategy.pine --indent --output-file /tmp/rsi.ast.txt
`
Library equivalent:
`python
from pynescript.ast import parse, dump
with open("examples/rsistrategy.pine", encoding="utf-") as f:
tree = parse(f.read(), "examples/rsistrategy.pine")
print(dump(tree, indent=))
`
. Round-trip unparse (normalize)
`bash
pynescript parse-and-unparse examples/rsistrategy.pine
pynescript parse-and-unparse messy.pine --output-file clean.pine
`
`python
from pynescript.ast import parse, unparse
source = """
//@version=
indicator("My RSI")
rsi(close, )
"""
print(unparse(parse(source)))
`
Canonical demo: examples/parsedumpunparse.py.
. Lint before upload
`bash
pynescript lint examples/rsistrategy.pine
pynescript lint --fail-on warnings examples/rsistrategy.pine
echo '//@version=\nindicator("x")\nplot(close)' | pynescript lint -
`
Library:
`python
from pynescript.ast.linter import lintscript
issues = lintscript(open("examples/rsistrategy.pine").read(), "rsistrategy.pine")
for w in issues:
print(w) severity: [CODE] message at line N
`
. Evaluate expressions
`python
from pynescript.ast.helper import literaleval
print(literaleval(" + "))
print(literaleval("math.sqrt()"))
prices = [, , , , , , , , , ]
print(literaleval(f"ta.rsi({prices}, )"))
Series history with context
ctx = {
"close": [, , , , ],
"open": [, , , , ],
}
print(literaleval("close[] - close[]", ctx))
`
Broader demo: examples/evaluateexpressions.py.
. Fetch sample market data (CLI)
`bash
mock provider (no network)
pynescript data AAPL --provider mock
Yahoo (network)
pynescript data AAPL --provider yahoo --period mo --interval d
CCXT (needs pynescript[data])
pynescript data BTC/USDT --provider ccxt --exchange binance
`
. Optional: local Pro API /run
`bash
from monorepo with backend deps
make run
`
`bash
curl -s -X POST http://...:/run \
-H 'Content-Type: application/json' \
-d '{
"script": "//@version=\nindicator(\"t\")\nplot(close)",
"data": [
{"time": , "open": , "high": , "low": ., "close": ., "volume": },
{"time": , "open": ., "high": ., "low": ., "close": ., "volume": }
]
}' | python -m json.tool
`
See Pro API usage for schemas and batch mode.
. Optional: start LSP for editors
`bash
pip install "pynescript[lsp]"
pynescript-lsp stdio; normally launched by the editor
`
Wire Neovim / Zed / Emacs using Editors and clients/.
Failure modes
| Step fails | Check |
| --- | --- |
| Dump raises SyntaxError | Script version / unsupported construct — try a minimal indicator + plot(close) |
| Unparse output differs cosmetically | Expected; compare semantic structure via dump |
| Lint exits non-zero | --fail-on threshold; inspect codes E, W, … |
| literaleval NotImplementedError | Expression not in literal/builtin subset; use full Runtime |
| data provider error | Network, API key, missing ccxt` extra |
See also
Installation
Configuration
CLI guide
Library API
Evaluate scripts
CLI command reference
---
FILE: docs/pyne/enduser/guides/cli.mdx
CLI Guide
Abstract
The pynescript console script is a thin Click group over the same helpers used by the library API. It is optimized for shell pipelines, CI gates, and one-shot inspection — not for multi-bar evaluation (use the library Runtime or Pro API for that). This guide covers workflows; exhaustive flags are in CLI commands.
Conceptual model
Commands mirror pynescript.ast.helper and friends:
| CLI | Library |
| --- | --- |
| parse-and-dump | parse + dump |
| parse-and-unparse | parse + unparse |
| lint | lintscript |
| data | pynescript.util.data.getprovider |
| download-builtin-scripts | pynescript.util.pinefacade.downloadbuiltinscripts |
Interface surface
``bash
pynescript --help
pynescript --version
pynescript --help
`
| Command | Input | Output |
| --- | --- | --- |
| parse-and-dump PATH | file | AST dump (stdout or --output-file) |
| parse-and-unparse PATH | file | Pine source |
| lint [PATH\|-] | file or stdin | human diagnostics |
| data SYMBOL | symbol + provider opts | summary stats |
| download-builtin-scripts | --script-dir | downloads reference scripts |
Not part of this Click group: starting the language server — use pynescript-lsp.
Internals (repo paths)
| Path | Role |
| --- | --- |
| src/pynescript/main.py | All command implementations |
| src/pynescript/ast/helper.py | parse, dump, unparse |
| src/pynescript/ast/linter.py | lintscript |
| src/pynescript/util/data.py | providers + DataProviderError |
| src/pynescript/util/pinefacade.py | builtin script download |
| examples/ | sample .pine inputs |
Invariants & edge cases
Encoding default is UTF- for file reads/writes.
--output-file - means stdout (Click allowdash=True).
Lint without a path reads stdin — useful in git hooks and cat script.pine | pynescript lint.
--fail-on defaults to errors (syntax/E); warnings fails on any warning severity too; all fails if any issue exists.
data default provider is mock — offline-safe.
Alpha Vantage without key prints a warning and uses demo.
Download command requires an explicit directory; create parents as needed (implementation writes into the given path).
Worked examples
Inspect structure of a strategy
`bash
pynescript parse-and-dump examples/rsistrategy.pine --indent | less
`
Compare two scripts’ shapes in CI:
`bash
pynescript parse-and-dump a.pine --output-file /tmp/a.ast
pynescript parse-and-dump b.pine --output-file /tmp/b.ast
diff -u /tmp/a.ast /tmp/b.ast
`
Format / normalize
`bash
pynescript parse-and-unparse messy.pine --output-file clean.pine
in-place-ish via shell:
pynescript parse-and-unparse strategy.pine --output-file strategy.pine.tmp \
&& mv strategy.pine.tmp strategy.pine
`
Lint gate in CI
`bash
set -e
pynescript lint strategies/ --help note: single file PATH, not directory recursion
for f in strategies/.pine; do
pynescript lint --fail-on warnings "$f"
done
`
Stdin pipeline:
`bash
git show HEAD:strategies/rsi.pine | pynescript lint --fail-on errors -
`
Market data smoke checks
`bash
pynescript data AAPL
pynescript data AAPL --provider yahoo --period y --interval d
pynescript data BTC/USDT --provider ccxt --exchange binance
pynescript data EUR/USD --provider alphavantage --api-key "$AVKEY"
`
Typical stdout shape (fields from main.py):
`text
Symbol: AAPL
Bars: N
Date range: N bars
First close: ...
Last close: ...
Volume (avg): ...
`
Refresh builtin corpus (contributors / offline tests)
`bash
pynescript download-builtin-scripts --script-dir tests/data/builtinscripts
`
Use sparingly: network + rate limits; CI may cache the corpus.
Compose with library for evaluation
CLI stops at parse/lint/data. Chain:
`bash
normalize then evaluate in Python
pynescript parse-and-unparse strat.pine --output-file /tmp/s.pine
python - <<'PY'
from pathlib import Path
from pynescript.ast.helper import parse
hand off to Runtime or your own visitor
print(parse(Path("/tmp/s.pine").readtext()).class.name)
PY
`
Failure modes
| Exit / message | Cause | Mitigation |
| --- | --- | --- |
| Click path errors | missing file | check PATH exists and is a file |
| Parse exception during dump | invalid syntax | fix source; use smaller repro |
| Lint failed with errors. | --fail-on threshold | read Ex / Wx lines |
| DataProviderError → ClickException | network / auth / missing dep | install [data], fix keys |
| Empty dump / unexpected tree | mode always exec for CLI | expression-only needs library mode="eval"` |
See also
CLI commands reference
Library API
Quick start
Troubleshooting
---
FILE: docs/pyne/enduser/guides/editors.mdx
Editors & LSP
Abstract
pynescript-lsp is a stdio Language Server (pygls) over the same parse/lint/metadata stack as the desk tools. Editors do not reimplement Pine intelligence; they speak LSP. This guide covers install, client configs in clients/, and feature expectations. Server architecture is documented under LSP.
Conceptual model
| Feature | Role |
| --- | --- |
| Diagnostics | Lint rules + parse errors as squiggles |
| Completion | Builtins (ta., strategy., …) + snippets |
| Hover | Signature / docs from metadata |
| Definition / references | Symbol index in document |
| Document symbols | Outline / breadcrumb |
| Formatting | Full document + range via unparser path |
Builtin metadata is generated (scripts/generatebuiltinmetadata.py); encrypted blobs may appear in release builds.
Interface surface
Install server
``bash
pip install "pynescript[lsp]"
monorepo:
pip install -e ".[lsp]"
or: make install
command -v pynescript-lsp
python -m pynescript.langserver equivalent entry
`
The process speaks LSP over stdio by default (PynescriptLanguageServer.startio()). Do not expect a useful interactive TTY UI.
VS Code / Cursor / Antigravity
. Build or install the extension from vscode-extension/:
`bash
cd vscode-extension
npm ci
npm run compile
package: npx vsce package (or make build-vscode)
`
. Open a .pine / .pinev / .pinev file — LSP activates when enabled.
Settings:
| Key | Default | Notes |
| --- | --- | --- |
| pynescript.lsp.enabled | true | Master switch |
| pynescript.lsp.command | pynescript-lsp | Absolute path if not on PATH |
| pynescript.formatting.enabled | true | |
| pynescript.diagnostics.enabled | true | |
| pynescript.completion.snippets | true | |
Marketplace availability may lag git — .
Neovim (nvim-lspconfig)
`lua
require("lspconfig").pynescript.setup({})
`
Manual config from clients/neovim.lua:
`lua
return {
cmd = { "pynescript-lsp" },
filetypes = { "pinescript" },
rootdir = function(fname)
return vim.fs.root(fname, { ".git", ".pine", ".pinev", ".pinev", "pyproject.toml" })
or vim.fn.getcwd()
end,
settings = {
pinescript = {
formatting = { enabled = true },
diagnostics = { enabled = true },
completion = { snippets = true },
},
},
}
`
Suggested maps: gd definition, gr references, K hover, format via vim.lsp.buf.format.
Ensure filetype detection maps .pine → pinescript if your distro does not ship it.
Zed
Merge clients/zed.json or:
`json
{
"languages": {
"Pine Script": {
"languageservers": ["pynescript"]
}
},
"languageservers": {
"pynescript": {
"command": "pynescript-lsp",
"arguments": ["--stdio"]
}
}
}
`
Emacs (lsp-mode)
`elisp
(use-package lsp-mode
:hook ((pinescript-mode . lsp))
:config
(lsp-register-client
(make-lsp-client
:new-connection (lsp-stdio-connection '("pynescript-lsp" "--stdio"))
:major-modes '(pinescript-mode)
:server-id 'pynescript)))
`
Or (load-file "clients/emacs.el") from a checkout.
Helix
~/.config/helix/languages.toml:
`toml
[[language]]
name = "pinescript"
scope = "source.pinescript"
file-types = ["pine", "pinev", "pinev"]
roots = ["pyproject.toml"]
command = "pynescript-lsp"
args = ["--stdio"]
`
Sublime Text (LSP package)
`json
{
"clients": {
"pynescript": {
"command": ["pynescript-lsp", "--stdio"],
"selector": "source.pinescript",
"initializationOptions": {}
}
}
}
`
Generic
Any LSP client:
`text
Command: pynescript-lsp
Args: (none or --stdio depending on client)
Transport: stdio
`
Internals (repo paths)
| Path | Role |
| --- | --- |
| src/pynescript/langserver/main.py | CLI entry main() |
| src/pynescript/langserver/server.py | PynescriptLanguageServer |
| src/pynescript/langserver/features/ | diagnostics, completion, hover, … |
| clients/README.md | Canonical client matrix |
| clients/neovim.lua, zed.json, emacs.el | Drop-in snippets |
| vscode-extension/ | VS Code extension |
| scripts/generatebuiltinmetadata.py | Completion/hover corpus |
Invariants & edge cases
Python .+ required on the machine running the server.
Server must be on PATH or configured with absolute command.
One server per workspace is typical; large monorepos with many .pine files still parse on demand.
Formatting uses the unparser pipeline — expect normalization, not clang-format style knobs.
Diagnostics reuse linter codes (E, W, …); severity maps to LSP DiagnosticSeverity.
Nuitka binary (dist/lsp/pynescript-lsp) may replace the Python entry in packaged distributions — point the editor at that binary if you ship it.
Worked examples
Verify server starts (stdio smoke)
`bash
Editors start this; manual smoke is limited.
python -c "from pynescript.langserver.server import PynescriptLanguageServer; print(PynescriptLanguageServer)"
`
Neovim minimal init.lua fragment
`lua
vim.api.nvimcreateautocmd("FileType", {
pattern = "pinescript",
callback = function()
vim.lsp.start({
name = "pynescript",
cmd = { "pynescript-lsp" },
root_dir = vim.fn.getcwd(),
})
end,
})
`
Force a specific venv binary in VS Code
`json
{
"pynescript.lsp.command": "/home/you/project/.venv/bin/pynescript-lsp"
}
`
Failure modes
| Symptom | Fix |
| --- | --- |
| Client: executable not found | Install [lsp]; fix PATH / command setting |
| No module named pygls` | Reinstall extra in the same env as the command |
| No diagnostics | Enable diagnostics; confirm filetype; check server logs |
| Stale completions | Regenerate metadata in dev builds; restart server |
| Format does nothing | Enable formatting; ensure document is valid enough to parse |
| High CPU on huge files | Split libraries; wait for parse; check for pathological nesting |
See also
Installation
LSP architecture
VS Code extension
LSP clients (systems)
Troubleshooting
---
FILE: docs/pyne/enduser/guides/evaluate-scripts.mdx
Evaluate Scripts
Abstract
Evaluation is the second half of the PYNE pipeline: given an AST (or source) and market context, produce plots, series, strategy events, and drawings. End users typically choose among three graded tools:
. literaleval — pure/builtin expressions, optional series context
. AST evaluator (NodeLiteralEvaluator / full evaluator mixins) — script-shaped evaluation in-process
. backend.runtime.Runtime or Pro API POST /run — bar-loop over OHLCV with shared evaluate contract
This guide stays at the consumer level; series semantics and builtin inventories are under Runtime.
Conceptual model
| Mode | Bar loop | Strategy | Typical use |
| --- | --- | --- | --- |
| literaleval | no (single shot) | limited | TA on arrays, math, strings |
| Evaluator in tests / tools | optional | yes via state | Unit tests, notebooks |
| Runtime.run / /run | yes | yes | Charts, AXIS, backtests |
| mode="compile" | yes (subset) | limited | Faster SMA/EMA/RSI-style paths |
Interface surface
Expression evaluation
``python
from pynescript.ast.helper import literaleval
literaleval(" + ")
literaleval("math.max(, , )")
literaleval('str.upper("hi")')
literaleval("array.size([, , ])")
prices = [, , , , , , , , , ]
literaleval(f"ta.sma({prices}, )")
literaleval(f"ta.rsi({prices}, )")
bb = literaleval(f"ta.bb({prices}, , )") middle, upper, lower
`
Series history context (Pine-style [] current / [] previous as implemented):
`python
context = {
"close": [, , , , ],
"open": [, , , , ],
"high": [, , , , ],
"low": [, , , , ],
}
literaleval("close[]", context)
literaleval("close[] - close[]", context)
`
Optional live/historical wiring:
`python
literaleval(expr, context, datafeed=feed, dataprovider=provider)
`
Script evaluation helper
`python
from pynescript.ast.evaluator import NodeLiteralEvaluator
ev = NodeLiteralEvaluator()
result = ev.evaluatescript(
"""
//@version=
indicator("demo")
// body depends on what the evaluator implements for statements
"""
)
`
Libraries:
`python
ev.registerlibrarysource(namespace="MyNs", name="Lib", version=, source=libsource)
mod = ev.lookuplibrary(namespace="MyNs", name="Lib", version=)
`
Bar-loop Runtime (HTTP contract shape)
`python
from backend.runtime import Runtime monorepo; not always installed as package path
runtime = Runtime(symbol="AAPL")
ohlcv = [
{"time": , "open": , "high": , "low": , "close": ., "volume": },
{"time": , "open": ., "high": , "low": , "close": ., "volume": },
]
result = runtime.run(
sourcecode='//@version=\nindicator("t")\nplot(close)\n',
ohlcvdata=ohlcv,
mode="interpret", or "compile" for supported subset
)
result: plots, series, plotmeta, events, drawings, scriptid, runid, count, ...
or {"error": "..."}
`
Pro API wraps the same runtime — see Pro API usage.
Data providers and feeds
Historical CLI/library:
`python
from pynescript.util.data import getprovider
prov = getprovider("mock")
or yahoo / alphavantage / ccxt with kwargs
bars = prov.fetch("AAPL", "mo", "d")
bars: dict with close/open/... lists
`
Realtime (requires ccxt / pro):
`python
examples/realtimedatafeed.py
from pynescript.util.datafeed import getdatafeed
feed = getdatafeed("ccxtpro", exchange="binance")
async with feed: async for candle in feed.watchohlcv("BTC/USDT", "m"): ...
`
Educational bar executor
examples/executescript.py implements a teaching RSI strategy executor with a custom visitor and pandas history (examples/historicaldata.py / yfinance). It is not the production Runtime, but demonstrates bar iteration patterns.
Internals (repo paths)
| Path | Role |
| --- | --- |
| src/pynescript/ast/helper.py | literaleval |
| src/pynescript/ast/evaluator/base.py | Context, series, visitor base |
| src/pynescript/ast/evaluator/init.py | NodeLiteralEvaluator composition |
| src/pynescript/ast/evaluator/builtins/ | ta., strategy., arrays, … |
| src/pynescript/ast/evaluator/events.py | Event emission hooks |
| backend/runtime.py | Multi-bar Runtime.run |
| backend/series.py | PineSeries history model |
| src/pynescript/util/data.py | Providers + resolverequestsources |
| src/pynescript/util/datafeed.py | Realtime feeds |
| examples/evaluateexpressions.py | Expression gallery |
| examples/executescript.py | Didactic strategy loop |
| examples/rsistrategy.pine | Sample strategy source |
Invariants & edge cases
Determinism. Same source + same OHLCV + same mode should yield reproducible series (mock providers seedable via options where supported).
na / warmup. TA functions need enough bars; early bars may be na/None-like — guard like Pine (not na(x)).
mode="compile" only supports a subset (docs in runtime: sma/ema/rsi, plots, math). Unsupported constructs fall back or error — .
History indexing. Confirm []/[] semantics against your evaluator version before porting TV scripts that assume closed-bar rules.
Strategy events include run/script ids when stamped by Runtime — useful for HOOX downstream.
Body size / bar count. HTTP path caps payload size ( MiB); very long histories should be downsampled or chunked.
Worked examples
RSI on a synthetic series
`python
from pynescript.ast.helper import literaleval
closes = [float(x) for x in range(, )]
print(literaleval(f"ta.rsi({closes}, )"))
`
Multi-indicator expressions
`python
highs = [c + for c in closes]
lows = [c - for c in closes]
macd, signal, hist = literaleval(f"ta.macd({closes}, , , )")
atr = literaleval(f"ta.atr({highs}, {lows}, {closes}, )")
`
Full script via Runtime (monorepo)
`python
from pathlib import Path
from backend.runtime import Runtime
script = Path("examples/rsistrategy.pine").readtext(encoding="utf-")
Build ohlcv list from your provider...
rt = Runtime(symbol="EXAMPLE")
out = rt.run(script, ohlcvdata=ohlcv, mode="interpret")
if "error" in out:
raise RuntimeError(out["error"])
print(out.get("count"), len(out.get("events", [])), out.get("series", {}).keys())
`
Strategy event inspection
`python
for ev in out.get("events", []):
shape depends on StrategyState / event schema
print(ev)
`
request. with resolved sources
When calling /run or Runtime, pass datasource / providers so request.security and friends resolve; without configuration they may use chart bars or mocks.
Failure modes
| Symptom | Interpretation |
| --- | --- |
| NotImplementedError / incomplete builtin | Expression outside supported surface — check missing features |
| {"error": "Parse Error: ..."} | Runtime caught parse failure |
| EXECUTIONERROR via HTTP | Exception during bar loop |
| Empty plots | No plot/plotshape executed; wrong declaration type |
| Numerical mismatch vs TV | Warmup, float path, or builtin parity gap — numerical validation |
| DataProviderError` | Provider misconfig |
| Async feed errors | Missing ccxt.pro / network |
See also
Pro API usage
Library API
Runtime series model
Strategy builtins
Evaluate contract
AXIS — visualizing series
HOOX — consuming strategy events on the edge
---
FILE: docs/pyne/enduser/guides/library-api.mdx
Library API
Abstract
The public Python surface for language work lives primarily under pynescript.ast: parse source into an ASDL-backed tree, serialize it (dump), regenerate source (unparse), traverse (walk, visitors), rewrite (NodeTransformer), and lint (lintscript). Expression evaluation (literaleval) and full script execution (evaluator / Runtime) sit adjacent; this page focuses on structural APIs. Evaluation workflows are covered in Evaluate scripts.
Conceptual model
Design intent mirrors Python’s ast module: stable tree operations, location attributes, and visitor protocols — specialized for Pine’s script model (annotations, series-aware semantics live at evaluation time).
Interface surface
Imports
``python
Re-exported convenience (node classes, helpers, visitors, transformers)
from pynescript.ast import parse, unparse, dump, walk
from pynescript.ast import NodeVisitor, NodeTransformer
from pynescript.ast.helper import literaleval, iterchildnodes, getsourcesegment
from pynescript.ast.linter import lintscript, lintfile, PineLinter, LintWarning
`
pynescript.ast.init star-imports helper, node, visitor, transformer, and error.
parse(source, filename="", mode="exec") -> AST
| Arg | Meaning |
| --- | --- |
| source | Full script or expression text |
| filename | Error reporting; absolutized if file exists |
| mode | "exec" → script statements; "eval" → single expression |
Raises ValueError on bad mode; SyntaxError (via error listener) on grammar failure.
`python
tree = parse('//@version=\nindicator("x")\nplot(close)\n')
expr = parse("close > open", mode="eval")
`
unparse(node) -> str
Round-trips an AST through NodeUnparser.
`python
normalized = unparse(parse(messysource))
`
dump(node, , annotatefields=True, includeattributes=False, indent=None) -> str
Human-readable tree serialization (debug, tests, CLI).
`python
print(dump(tree, indent=, includeattributes=True))
`
walk(node) -> Iterator[AST]
Breadth-first (deque) traversal yielding every node.
`python
from pynescript.ast import walk
names = [n for n in walk(tree) if n.class.name == "Name"]
`
Location helpers
| Function | Role |
| --- | --- |
| copylocation(new, old) | Copy lineno/col offsets |
| fixmissinglocations(node) | Fill missing positions |
| incrementlineno(node, n=) | Shift lines |
| getsourcesegment(source, node, , padded=False) | Slice original text |
| iterfields / iterchildnodes | Schema-aware iteration |
Visitors and transformers
`python
from pynescript.ast import NodeVisitor, NodeTransformer, parse
class CallCounter(NodeVisitor):
def init(self):
self.calls =
def visitCall(self, node):
self.calls +=
self.genericvisit(node)
class RenameClose(NodeTransformer):
def visitName(self, node):
if getattr(node, "id", None) == "close":
node.id = "price"
return node
tree = parse(source)
CallCounter().visit(tree)
tree = RenameClose().visit(tree)
`
Linter
`python
from pynescript.ast.linter import lintscript, LintWarning
warnings: list[LintWarning] = lintscript(source, "strat.pine")
for w in warnings:
print(w.code, w.severity, w.line, w.message)
`
| Code family | Examples |
| --- | --- |
| E | Syntax error (error severity) |
| W–W | Missing / old @version |
| W–W | Deprecated patterns |
| C–C | Naming / style (line length , trailing newline, …) |
PineLinter.lint runs: syntax → version → deprecated → naming → style.
literaleval (bridge to evaluation)
`python
from pynescript.ast.helper import literaleval
literaleval("ta.sma([,,,,], )")
literaleval("close[]", {"close": [., ., .]})
`
Optional datafeed / dataprovider for request. in literal contexts.
Internals (repo paths)
| Path | Role |
| --- | --- |
| src/pynescript/ast/helper.py | Public parse/dump/walk/unparse/literaleval |
| src/pynescript/ast/builder.py | Parse tree → ASDL nodes |
| src/pynescript/ast/unparser.py | NodeUnparser |
| src/pynescript/ast/visitor.py | NodeVisitor |
| src/pynescript/ast/transformer.py | NodeTransformer |
| src/pynescript/ast/node.py | Node re-exports / helpers |
| src/pynescript/ast/grammar/asdl/resource/Pinescript.asdl | Algebraic schema |
| src/pynescript/ast/linter.py | Static rules |
| src/pynescript/ast/error.py | Error types |
| examples/parsedumpunparse.py | Round-trip demo |
Pipeline inside parse:
. InputStream / FileStream
. PinescriptLexer + PinescriptParser (error listener)
. PinescriptASTBuilder.visit
. In exec mode: StatementCollector, comment collection, @ annotation attach
Invariants & edge cases
Annotation comments (//@version, etc.) attach to script / function / type / assign nodes by kind suffix — see addannotations in helper.py.
Deep nesting: temporary sys.setrecursionlimit(max(old, )) during parse.
dump requires a true AST node — TypeError otherwise.
Transformers mutate or replace nodes depending on your visit methods; prefer returning new nodes if you need purity.
Round-trip fidelity is tested against a large corpus but is not a proof of bit-identical source; whitespace and some sugar may normalize.
Do not edit generated grammar modules; extend via resource grammars if contributing.
Worked examples
End-to-end inspect
`python
from pynescript.ast import parse, dump, unparse, walk
source = open("examples/rsistrategy.pine", encoding="utf-").read()
tree = parse(source, "examples/rsistrategy.pine")
print(dump(tree, indent=)[:], "...")
print("---")
print(unparse(tree))
print("node count", sum( for in walk(tree)))
`
Collect all call names
`python
from pynescript.ast import parse, NodeVisitor
class Calls(NodeVisitor):
def init(self):
self.names = []
def visitCall(self, node):
func = node.func
Attribute vs Name depending on ta.rsi vs f()
self.names.append(func)
self.genericvisit(node)
c = Calls()
c.visit(parse(source))
`
Lint programmatically with fail semantics
`python
from pynescript.ast.linter import lintscript
def assertclean(path: str) -> None:
text = open(path, encoding="utf-").read()
issues = lintscript(text, path)
errors = [i for i in issues if i.severity == "error"]
if errors:
raise SystemExit("\n".join(map(str, errors)))
`
Expression mode for tooling
`python
from pynescript.ast import parse, dump
print(dump(parse("ta.rsi(close, )", mode="eval"), indent=))
`
Failure modes
| Exception / result | Meaning |
| --- | --- |
| SyntaxError | Lexer/parser rejection |
| ValueError: invalid argument mode | mode not in {exec,eval} |
| Empty / odd tree | Empty body scripts; annotations-only edge cases |
| Unparse mismatch | Node kinds without unparser support — file issue against unparser |
| Linter false positives | Style rules are heuristic (regex naming) — treat Cx` as advisory |
See also
Evaluate scripts
CLI guide
Helper API (core)
Visitor / transformer (core)
Linter (core)
---
FILE: docs/pyne/enduser/guides/pro-api-usage.mdx
Pro API Usage
Abstract
The Pro API is the HTTP evaluate contract for PYNE: POST Pine source plus OHLCV, receive plots, series, events, and drawings. Free-tier evaluate endpoints (/run, /run/batch) are usable without a key; Pro preview/backtest routes track usage via API keys. This page is for callers (AXIS, scripts, workers). Server lifecycle and auth design are under API.
Conceptual model
| Endpoint | Auth | Purpose |
| --- | --- | --- |
| GET / | none | Health + endpoint map |
| POST /run | none (free) | Single script evaluate |
| POST /run/batch | none (free) | ≤ scripts, shared OHLCV |
| POST /preview/chart | Pro usage | Chart thumbnail |
| POST /preview/indicator | Pro usage | Indicator-style preview |
| POST /backtest/quick | Pro usage | Equity / summary |
| POST /auth/createkey | admin token | Mint key |
| GET /auth/usage | Pro | Usage stats |
| POST /auth/validate | body key | Validate key |
Default local bind: ...:.
Interface surface
Run the server (local)
``bash
pip install -r backend/requirements.txt
export APIKEYSTORE="$PWD/.data/apikeys.json"
python -m backend.app
or: make run
`
Health
`bash
curl -s http://...:/ | python -m json.tool
`
POST /run
Schema (RUNSCHEMA):
| Field | Type | Required | Default |
| --- | --- | --- | --- |
| script | string | yes | |
| data | list (OHLCV bars) | yes | |
| symbol | string | no | "CHART" |
| datasource | string | no | "" |
| dataoptions | object | no | {} |
| mode | string | no | "interpret" |
Bar objects are expected to carry at least open/high/low/close/time (volume optional depending on script).
`bash
curl -s -X POST http://...:/run \
-H 'Content-Type: application/json' \
-d '{
"script": "//@version=\nindicator(\"Demo\")\nplot(ta.sma(close, ))",
"symbol": "AAPL",
"mode": "interpret",
"data": [
{"time": , "open": , "high": , "low": , "close": ., "volume": },
{"time": , "open": ., "high": , "low": , "close": ., "volume": },
{"time": , "open": ., "high": ., "low": , "close": , "volume": }
]
}'
`
Success shape (conceptual):
`json
{
"status": "success",
"plots": [],
"series": {},
"plotmeta": {},
"events": [],
"drawings": [],
"scriptid": "...",
"runid": "...",
"count": ,
"mode": "interpret",
"datasource": "chart"
}
`
Error codes include NOSCRIPT, NODATA, DATASOURCEERROR, EXECUTIONERROR, plus schema codes INVALIDBODY, MISSINGFIELD, INVALIDFIELD, UNKNOWNFIELDS.
POST /run/batch
Shared OHLCV, multiple scripts (max ). Envelope:
`json
{
"scripts": [
{"id": "sma", "script": "//@version=\nindicator(\"s\")\nplot(ta.sma(close, ))"},
{"id": "rsi", "script": "//@version=\nindicator(\"r\")\nplot(ta.rsi(close, ))"}
],
"data": [/ bars /],
"symbol": "AAPL",
"mode": "interpret"
}
`
String entries in scripts are accepted and auto-id’d as script, …. Per-script errors do not necessarily fail the whole HTTP status (route returns with per-item status — verify against deployment).
POST /preview/chart (Pro)
`json
{
"script": "",
"data": {"close": [, , ], "volume": [, , ]},
"options": {
"type": "line",
"color": "F",
"width": ,
"height": ,
"showvolume": false
}
}
`
Response includes base PNG chart + meta. Width/height clamped (e.g. max ×).
POST /preview/indicator (Pro)
`json
{
"expression": "ta.sma(close, )",
"data": {"close": [/ ... /]},
"options": {}
}
`
POST /backtest/quick (Pro)
`json
{
"script": "//@version=\nstrategy(\"x\")\n...",
"data": {},
"initialcapital": .,
"mockdata": true,
"mockbars":
}
`
Auth helpers
`bash
Mint key (admin)
curl -s -X POST http://...:/auth/createkey \
-H "Content-Type: application/json" \
-H "X-Admin-Token: $ADMINTOKEN" \
-d '{"tier":"hobby"}'
Validate
curl -s -X POST http://...:/auth/validate \
-H "Content-Type: application/json" \
-d '{"apikey":"pyn..."}'
`
Send Pro requests with the key as required by middleware (typically Authorization bearer or documented header — ).
Hosted SDK (if published)
README sketches:
`python
from pynescript.api import PynescriptAPI
api = PynescriptAPI(apikey="pyn...")
chart = api.preview.chart(data={"close": [...]}, options={"type": "line"})
result = api.backtest.quick(script="...", mockdata=True, mockbars=)
`
If the client module is not present in your install, use raw HTTP as above.
Internals (repo paths)
| Path | Role |
| --- | --- |
| backend/app.py | Routes /, /run, /run/batch, auth |
| backend/api/preview.py | /preview/, /backtest/quick |
| backend/middleware/schemas.py | Strict request schemas |
| backend/middleware/auth.py | Keys, usage, admin token |
| backend/runtime.py | Evaluate bridge |
| backend/services/chartrenderer.py | PNG rendering |
| backend/services/backtest.py | Quick backtest + mock OHLCV |
Invariants & edge cases
Max body MiB — oversized JSON → Flask .
Strict schemas — unknown fields rejected.
CORS — browser clients need listed origins; server-to-server without Origin is fine.
mode=compile — subset only; prefer interpret unless you know the script is in the compiled surface.
Batch cap — TOOMANYSCRIPTS if exceeded.
Free /run is still CPU-expensive — rate-limit at your reverse proxy in production.
AXIS multi-indicator UIs prefer /run/batch to share bar payloads.
Worked examples
Python requests client for /run
`python
import requests
SCRIPT = """
//@version=
indicator("SMA", overlay=true)
plot(ta.sma(close, ))
"""
bars = [
{"time": i, "open": + i, "high": + i, "low": + i, "close": . + i, "volume": }
for i in range()
]
r = requests.post(
"http://...:/run",
json={"script": SCRIPT, "data": bars, "symbol": "DEMO", "mode": "interpret"},
timeout=,
)
r.raiseforstatus()
payload = r.json()
assert payload["status"] == "success", payload
print(payload["count"], list(payload.get("series", {})))
`
Batch two indicators
`python
r = requests.post(
"http://...:/run/batch",
json={
"scripts": [
{"id": "sma", "script": "//@version=\nindicator(\"s\")\nplot(ta.sma(close, ))"},
{"id": "ema", "script": "//@version=\nindicator(\"e\")\nplot(ta.ema(close, ))"},
],
"data": bars,
},
timeout=,
)
print(r.json())
`
Wire optional CCXT data source
`json
{
"script": "...",
"data": [/ chart bars /],
"symbol": "BTC/USDT",
"datasource": "ccxt",
"dataoptions": {"exchange": "binance"}
}
`
Requires server environment with ccxt installed.
Failure modes
| Code / HTTP | Meaning | Action |
| --- | --- | --- |
| MISSINGFIELD | No script/data | Fix body |
| UNKNOWNFIELDS | Extra keys | Strip undocumented fields |
| DATASOURCEERROR | Provider config | Fix dataoptions / deps |
| EXECUTIONERROR | Runtime exception | Simplify script; check message |
| on createkey | ADMINTOKEN unset/wrong | Export token |
| | Body too large | Fewer bars / compress strategy |
| CORS browser error | Origin not allowed | Update ALLOWED_ORIGINS |
| Empty series | Script never plotted | Add plot / check declaration |
See also
Evaluate scripts
Configuration
API contract
Auth and keys
AXIS — AXIS over /run` results
HOOX — edge mesh after strategy events
---
FILE: docs/pyne/enduser/guides/troubleshooting.mdx
Troubleshooting
Abstract
Failures cluster by layer: environment, parse, lint, evaluation, data I/O, LSP, HTTP. Work top-down: confirm the binary/module, then a minimal script, then full strategy complexity. This page is a runbook; deep protocol and runtime docs are linked per section.
Conceptual model
Interface surface (diagnostic commands)
``bash
python -c "import pynescript; from pynescript import about; print(about.version)"
python -c "from pynescript.ast import parse, unparse; print(unparse(parse('//@version=\nindicator(\"t\")\nplot(close)\n')))"
pynescript --version
pynescript lint - /tmp/a.ast
pynescript parse-and-unparse a.pine | pynescript parse-and-dump /dev/stdin
stdin may not work for parse-and-dump (file path required) — use a temp file:
pynescript parse-and-unparse a.pine --output-file /tmp/a.pine
pynescript parse-and-dump /tmp/a.pine > /tmp/a.ast
diff -u /tmp/a.ast /tmp/a.ast
`
Structural diff empty ⇒ cosmetic only.
. Lint noise / CI red
| Code | Action |
| --- | --- |
| E | Fix syntax first |
| W | Add //@version= |
| W | Upgrade version pragma |
| C–C | Style; use --fail-on errors if style should not gate CI |
`bash
pynescript lint script.pine --fail-on errors
`
. literaleval fails
Use mode="eval"-compatible expressions, not full strategy() scripts
Pass series as Python lists inside the expression or via context
Missing builtin → incomplete implementation
Escalate to Runtime.run for multi-bar scripts.
. Runtime / /run execution error
`bash
curl -s -X POST http://...:/run -H 'Content-Type: application/json' -d '{"script":"//@version=\nindicator(\"t\")\nplot(close)","data":[{"time":,"open":,"high":,"low":,"close":,"volume":}]}'
`
If minimal works but strategy fails: remove request., drawings, then strategy entries until isolated. Check mode; try "interpret".
. Schema validation errors
UNKNOWNFIELDS means you sent a key not in the schema — remove it. Do not rely on servers ignoring extras.
. CORS from AXIS / browser
`bash
export ALLOWEDORIGINS="http://localhost:,http://...:"
`
Restart Flask after changing env.
. LSP will not start
`bash
pip install "pynescript[lsp]"
python -c "import pygls, lsprotocol"
command -v pynescript-lsp
Run with editor logging enabled; check stderr for import traces
`
Ensure the editor’s command matches the venv. For Neovim, confirm filetypes include your buffer’s filetype.
. Data provider failures
| Provider | Checklist |
| --- | --- |
| mock | Should always work offline |
| yahoo | Network; symbol format |
| alphavantage | API key; demo limits |
| ccxt | pip install "pynescript[data]"; exchange id; symbol BTC/USDT |
`bash
pynescript data AAPL --provider mock
`
. Version / parity confusion
PYNE is independent of TradingView. Behavioral gaps are expected for edge builtins, broker emulators, and UI-only calls. Use compatibility and implementation status.
Failure modes (quick matrix)
| Symptom | Layer | First check |
| --- | --- | --- |
| Command not found | env | python -m pynescript |
| Import error pygls | extra | [lsp] |
| Import error ccxt | extra | [data] |
| SyntaxError | parse | minimal script + version |
| Lint CI fail | lint | --fail-on level |
| Empty plots | eval | plot() present; enough bars |
| UNKNOWNFIELDS | HTTP | schema |
| createkey | HTTP | ADMINTOKEN` |
| | HTTP | body size |
| Squiggles missing | LSP | diagnostics enabled + filetype |
| Numerical drift | eval | warmup / parity docs |
See also
Installation
Configuration
CLI guide
Evaluate scripts
Pro API usage
Editors
FAQ
---
FILE: docs/pyne/enduser/index.mdx
End User Hub
This track is for people who consume PYNE: parse and reformat Pine Script, lint before upload, evaluate expressions or full scripts against OHLCV, wire an editor via LSP, or call the HTTP evaluate contract. Contributor internals (grammar regeneration, AST builder, bar-loop semantics) live under Core, Runtime, and DevOps.
Abstract
PYNE exposes one language pipeline through several thin surfaces:
``
.pine source
→ parse (ANTLR + ASDL AST)
→ optional: dump | unparse | lint | walk/transform
→ optional: evaluate (literaleval | bar-loop Runtime)
→ plots / events / drawings / metrics
`
The desk path is the pynescript Click CLI and pynescript.ast library API. The editor path is pynescript-lsp (stdio). The HTTP path is the Flask Pro API (POST /run, preview, backtest). The optional chart is AXIS; the optional trade mesh is HOOX. Evaluation does not require either.
Conceptual model
| Surface | Entry | Typical job |
| --- | --- | --- |
| CLI | pynescript … | One-shot parse, format, lint, market data |
| Library | from pynescript.ast import parse, unparse | Embed parse/transform/eval in tools |
| LSP | pynescript-lsp | Diagnostics, completion, hover in editors |
| Pro API | POST /run | Shared evaluate contract for AXIS / workers |
| AXIS | separate product | Chart PWA AXIS over evaluate results |
| HOOX | separate product | Edge execution mesh after strategy events |
Choose your path
. Getting started
New install, first parse, and environment knobs:
Installation — PyPI, extras (lsp, data), Hatch checkout, smoke checks.
Quick start — Parse → dump → unparse → lint → literaleval in under ten minutes.
Configuration — Extras, console scripts, Pro API env vars, editor settings.
. Operational guides
Daily workflows against the same pipeline:
CLI — parse-and-dump, parse-and-unparse, lint, data, download-builtin-scripts.
Library API — parse, unparse, dump, walk, visitors, transformers, linter.
Evaluate scripts — Expression eval, bar-loop Runtime, strategy events, mock vs live data.
Editors — VS Code, Neovim, Zed, Emacs, Helix; clients/ snippets.
Pro API usage — /run, /run/batch, preview, backtest as a consumer.
Troubleshooting — Parse failures, recursion limits, missing extras, CORS, auth.
. Reference
CLI commands — Full option trees for every Click command.
Glossary — AST, series, bar-loop, Runtime, ASDL, and related terms.
FAQ — Version support, TradingView parity, licensing, AXIS/HOOX boundaries.
Interface surface (consumer map)
| Want | Command / import | Docs |
| --- | --- | --- |
| Install core | pip install pynescript | Installation |
| Install LSP | pip install "pynescript[lsp]" | Editors |
| Install market data deps | pip install "pynescript[data]" | CLI data |
| Parse file | pynescript parse-and-dump path.pine | CLI |
| Normalize source | pynescript parse-and-unparse path.pine | CLI |
| Lint | pynescript lint path.pine | CLI |
| Library parse | from pynescript.ast import parse, unparse | Library API |
| Expression eval | from pynescript.ast.helper import literaleval | Evaluate |
| Start LSP | pynescript-lsp (stdio) | Editors |
| Local Pro API | make run / python -m backend.app | Pro API usage |
Two console scripts are registered in pyproject.toml:
| Script | Module | Role |
| --- | --- | --- |
| pynescript | pynescript.main:cli | Desk CLI (Click group) |
| pynescript-lsp | pynescript.langserver.main:main | Language server (pygls) |
Do not conflate them: lint and parse live on pynescript; editor features live on pynescript-lsp.
Internals (repo paths)
Consumer-relevant code only — deeper design is linked from each guide:
| Path | Role |
| --- | --- |
| src/pynescript/main.py | Click CLI: parse-and-dump, parse-and-unparse, lint, data, download-builtin-scripts |
| src/pynescript/ast/helper.py | Public parse, unparse, dump, walk, literaleval |
| src/pynescript/ast/linter.py | lintscript / PineLinter |
| src/pynescript/ast/evaluator/ | Bar-aware and literal evaluators |
| src/pynescript/langserver/ | LSP features and server |
| src/pynescript/util/data.py | Historical data providers (mock, yahoo, alphavantage, ccxt) |
| backend/app.py | Flask Pro API entry (/run, auth, blueprints) |
| backend/runtime.py | HTTP-facing Runtime.run bar loop |
| examples/ | Worked scripts (parse, RSI strategy, datafeed) |
| clients/ | Editor client configs (Neovim, Zed, Emacs) |
| vscode-extension/ | VS Code extension package |
Invariants & edge cases
Round-trip is first-class. parse → unparse should preserve semantics; use it to normalize formatting, not as a semantic rewrite.
Mode split. parse(..., mode="exec") yields a script tree; mode="eval" parses a single expression (used by literaleval).
No proprietary host required. Evaluation works offline with mock or supplied OHLCV; AXIS/HOOX are optional.
Extras are opt-in. Core parse/lint needs only base deps (antlr-python-runtime, click, …). LSP needs [lsp]; CCXT paths need [data] / [datafeed].
Parametrized corpus. Tests under tests/data/builtinscripts/ are large; a fixture using pinescriptfilepath can explode CI runtime if misused.
Worked examples
Minimal library round-trip:
`python
from pynescript.ast import parse, unparse
source = """
//@version=
indicator("My RSI")
plot(ta.rsi(close, ))
"""
tree = parse(source)
print(unparse(tree))
`
Minimal CLI:
`bash
pip install pynescript
pynescript parse-and-dump examples/rsistrategy.pine
pynescript lint examples/rsistrategy.pine
`
Minimal HTTP evaluate (local server running):
`bash
curl -s http://...:/ \
| python -m json.tool
`
Failure modes
| Symptom | Likely cause | Where to go |
| --- | --- | --- |
| pynescript: command not found | Package not on PATH / venv inactive | Installation |
| pynescript-lsp missing | Installed without [lsp] extra | Editors |
| SyntaxError on parse | Grammar mismatch or truncated source | Troubleshooting |
| /run NO_DATA` | Missing OHLCV list | Pro API usage |
| Numerical drift vs TradingView | Builtin / series edge case | Compatibility |
See also
PYNE product map — full stack overview
Installation
Quick start
Evaluate contract (API)
AXIS docs — optional chart
HOOX docs — optional edge trade mesh
---
FILE: docs/pyne/enduser/reference/cli-commands.mdx
CLI Commands Reference
Abstract
Machine-oriented reference for every command registered on the pynescript Click group in src/pynescript/main.py. For workflows and pipelines, see the CLI guide. The language server is a separate console script (pynescript-lsp) and is documented under Editors.
Conceptual model
``text
pynescript [--version] COMMAND [ARGS]...
`
| Command | Short help (Click) |
| --- | --- |
| parse-and-dump | Parse pinescript file to AST tree. |
| parse-and-unparse | Parse pinescript file and unparse back to pinescript. |
| download-builtin-scripts | Download builtin scripts. |
| lint | Lint Pine Script file for issues. |
| data | Fetch market data from providers. |
Global:
`bash
pynescript --help
pynescript --version
`
Version source: package version option on the Click group (@click.versionoption()), aligned with pynescript.about.version.
Interface surface
pynescript parse-and-dump
Parse a file and print an AST dump.
| Kind | Name | Type / default | Description |
| --- | --- | --- | --- |
| argument | PATH | existing readable file | Input .pine path |
| option | --encoding | str, default utf- | File text encoding |
| option | --indent | int, default | Indent width for dump |
| option | --output-file | path or -, default - | Output path; - = stdout |
Semantics: parse(text, filename) then dump(node, indent=indent); write with same encoding.
Examples:
`bash
pynescript parse-and-dump strategy.pine
pynescript parse-and-dump strategy.pine --indent --output-file tree.txt
pynescript parse-and-dump strategy.pine --encoding utf- --output-file -
`
Exit codes: Click/path errors non-zero; uncaught parse exceptions propagate (non-zero).
---
pynescript parse-and-unparse
Parse a file and emit regenerated source.
| Kind | Name | Type / default | Description |
| --- | --- | --- | --- |
| argument | PATH | existing readable file | Input path |
| option | --encoding | str, default utf- | Text encoding |
| option | --output-file | path or -, default - | Output path; - = stdout |
Semantics: parse → unparse → write.
Examples:
`bash
pynescript parse-and-unparse messy.pine
pynescript parse-and-unparse messy.pine --output-file clean.pine
`
---
pynescript lint
Lint a file or stdin.
| Kind | Name | Type / default | Description |
| --- | --- | --- | --- |
| argument | PATH | optional string | File path; omit or - for stdin |
| option | --encoding | str, default utf- | File encoding (ignored for stdin) |
| option | --fix | flag | Attempt to fix issues where possible |
| option | --fail-on | choice errors\|warnings\|all, default errors | Non-zero exit threshold |
Semantics:
. Read source (stdin if PATH is missing or -).
. warnings = lintscript(source, filename).
. If empty → print No issues found. and return.
. Print each issue with emoji prefix by severity; print summary count.
. Raise click.ClickException when threshold met:
| --fail-on | Fails when |
| --- | --- |
| errors | any severity == "error" |
| warnings | any error or warning |
| all | any issue |
Examples:
`bash
pynescript lint strategy.pine
pynescript lint --fail-on warnings strategy.pine
pynescript lint --fail-on all strategy.pine
pynescript lint -
cat strategy.pine | pynescript lint
pynescript lint --encoding utf- strategy.pine
`
Issue line format (via LintWarning.str):
`text
{severity}: [{code}] {message} at line {n}|unknown location
`
CLI prefixes (error) or ️ (other).
---
pynescript data
Fetch market data for a symbol and print a short summary.
| Kind | Name | Type / default | Description |
| --- | --- | --- | --- |
| argument | SYMBOL | string | Ticker / market symbol |
| option | --provider | mock\|yahoo\|alphavantage\|ccxt, default mock | Backend |
| option | --period | str, default y | e.g. d, w, mo, mo, mo, y, y, y |
| option | --interval | str, default d | e.g. m, m, m, m, m, d, w |
| option | --api-key | str, default "" | Alpha Vantage or CCXT |
| option | --secret | str, default "" | CCXT secret |
| option | --exchange | str, default binance | CCXT exchange id |
Semantics:
. Build provider kwargs (AV demo key warning if missing; CCXT exchange/key/secret).
. getprovider(provider, kwargs).fetch(symbol, period, interval).
. Print symbol, bar count, first/last close, average volume.
Examples:
`bash
pynescript data AAPL
pynescript data AAPL --provider=yahoo --period=mo
pynescript data EUR/USD --provider=alphavantage --api-key=YOURKEY
pynescript data BTC/USDT --provider=ccxt --exchange=binance
`
Errors: DataProviderError → ClickException with message.
Deps: ccxt providers require pip install "pynescript[data]".
---
pynescript download-builtin-scripts
Download TradingView reference scripts used for tests/corpus.
| Kind | Name | Type / default | Description |
| --- | --- | --- | --- |
| option | --script-dir | writable directory path | Required. Destination (e.g. tests/data/builtinscripts) |
Semantics: pynescript.util.pinefacade.downloadbuiltinscripts(scriptdir).
Examples:
`bash
pynescript download-builtin-scripts --script-dir tests/data/builtinscripts
pynescript download-builtin-scripts --script-dir /tmp/pinecorpus
`
Notes: Network-dependent; not required for ordinary desk use.
---
Related: pynescript-lsp (not a subcommand)
`bash
pynescript-lsp
python -m pynescript.langserver
`
Entry: pynescript.langserver.main:main → PynescriptLanguageServer().startio().
Internals (repo paths)
| Item | Path |
| --- | --- |
| CLI group | src/pynescript/main.py |
| Script entry | pyproject.toml → [project.scripts] pynescript = ... |
| parse/dump/unparse | src/pynescript/ast/helper.py |
| lint | src/pynescript/ast/linter.py |
| data | src/pynescript/util/data.py |
| download | src/pynescript/util/pinefacade.py |
Invariants & edge cases
File arguments for parse commands use click.Path(exists=True, fileokay=True, dirokay=False) — no directory recursion.
lint deliberately accepts a free string path so stdin modes work; invalid paths surface as open errors.
Dump indent is passed as integer spaces to dump(..., indent=indent).
Output files are opened with Click openfile (creates/truncates as applicable).
There is currently no pynescript evaluate / pynescript run CLI subcommand — evaluation is library or HTTP.
Worked examples
CI lint gate
`bash
!/usr/bin/env bash
set -euo pipefail
for f in "$@"; do
pynescript lint --fail-on errors "$f"
done
`
Normalize and re-dump for structural snapshot tests
`bash
pynescript parse-and-unparse in.pine --output-file /tmp/norm.pine
pynescript parse-and-dump /tmp/norm.pine --indent --output-file /tmp/norm.ast
`
Offline data smoke
`bash
pynescript data TEST --provider mock --period mo --interval d
`
Failure modes
| Situation | Result |
| --- | --- |
| Missing input file | Click path validation error |
| Parse error in dump/unparse | Python traceback / non-zero |
| Lint threshold exceeded | Error: Lint failed with ... |
| Provider failure | Error: |
| Missing --script-dir` | Click missing option error |
| Unknown provider | Click choice validation error |
See also
CLI guide
Library API
Configuration
FAQ
---
FILE: docs/pyne/enduser/reference/faq.mdx
FAQ
Abstract
Short answers to questions that recur when adopting PYNE as a consumer. Where behavior is evolving, answers point at status docs rather than overclaiming parity.
Conceptual model
Questions group into: install & package shape, language coverage, evaluation fidelity, tooling surfaces, and product boundaries (AXIS / HOOX / workers).
Interface surface
No dedicated FAQ API — pointers only.
Internals (repo paths)
Answers cite:
pyproject.toml — version, extras, license
src/pynescript/main.py — CLI reality
docs/missingfeatures.md / reference status pages — coverage
backend/app.py — HTTP contract
Invariants & edge cases
Prefer reading code + status docs over README marketing tables when they disagree; mark drift with issues.
Trademark disclaimer lives on the product index once — not repeated here in full.
Questions
What is PYNE vs pynescript?
pynescript is the Python package / repo name. PYNE is the product documentation brand for the open evaluate stack (grammar → AST → runtime → LSP → Pro API → workers). You still pip install pynescript.
What Python versions are supported?
requires-python = ">=.". Classifiers list .–.. Newer CPython may work; CI matrices evolve — .
How do I install just the parser?
``bash
pip install pynescript
`
No extra required for parse / unparse / lint / basic literaleval.
How do I install the LSP?
`bash
pip install "pynescript[lsp]"
`
Run pynescript-lsp (not a Click subcommand of pynescript in current code).
Why is pynescript lsp missing?
The registered entry point is the separate script pynescript-lsp. Some docs historically mentioned a lsp subcommand; trust pyproject.toml [project.scripts] and main.py for ground truth.
Which Pine versions are supported?
v is the primary surface; v features land incrementally (e.g. multiline strings). See implementation status, missing features, and pine v surface.
Is this affiliated with TradingView?
No. Independent open-source toolchain. Pine Script and TradingView are trademarks of TradingView, Inc.
Will my TradingView strategy produce identical PnL?
Not guaranteed. Broker emulator rules, fill models, request. data, and some builtins differ. Use PYNE for offline analysis, CI, and self-hosted evaluate; validate critical strategies numerically (numerical validation).
Can I format Pine like Black?
parse-and-unparse / LSP format normalize via the AST unparser. There is no rich style config comparable to Black’s profile system.
Does the CLI evaluate strategies?
No evaluate subcommand today. Use:
literaleval for expressions
backend.runtime.Runtime or POST /run for bar-loops
examples/executescript.py as a didactic sample
How do I lint in CI?
`bash
pynescript lint --fail-on errors path/to/script.pine
`
Loop files yourself; the CLI is single-file / stdin.
What lint rules exist?
Syntax (E), version (W–W), deprecated patterns (W–), style/naming (C–C). See src/pynescript/ast/linter.py and the library API guide.
How do I get market data?
`bash
pynescript data AAPL --provider yahoo
pip install "pynescript[data]" for ccxt
`
Or providers in Python via pynescript.util.data.getprovider.
What is the Pro API free tier?
POST /run and POST /run/batch are exposed as free evaluate endpoints (still your CPU if self-hosted). Preview/backtest routes are Pro-tier with API keys. Hosted pricing in README is product-side — .
How do I self-host the API?
Install backend requirements, set APIKEYSTORE / ALLOWEDORIGINS / optional ADMINTOKEN, run python -m backend.app or make run. See configuration and Pro API usage.
What is AXIS?
Optional charting PWA that can call the evaluate contract and render series. Docs: AXIS. PYNE evaluates without AXIS.
What is HOOX?
Optional edge trading mesh (webhooks, trade-worker, etc.). Docs: HOOX. Strategy events from PYNE can feed HOOX; not required for parsing/eval.
What is pine-worker vs pyne-worker?
pine-worker — TypeScript port / tool in this monorepo (pine-worker/)
pyne-worker — Python Cloudflare Worker sister repo depending on pynescript
Both aim at edge evaluate; different runtimes.
What license is pynescript?
AGPL-3.0-or-later per pyproject.toml. Proprietary embedding usually needs a commercial license; consult your counsel.
Can I use this commercially?
AGPL permits commercial use under its terms. Hosted Pro API terms (if any) are separate from the library license.
Why is parse slow on huge files?
ANTLR parse + AST build scale with file size and nesting. Deep expression nests also stress recursion (limit temporarily raised to ≥). Split libraries when possible.
Round-trip changed my spacing — is that a bug?
Usually no. Unparser emits a normalized style. File a bug if semantics change (identifiers, call structure, annotations lost).
Where is the builtin list for autocomplete?
Generated metadata used by the LSP (scripts/generatebuiltinmetadata.py). Do not hand-edit encrypted release blobs.
How do I report a grammar bug?
Minimal .pine repro + expected TV behavior + PYNE version. Prefer failing pytest if contributing.
Is Jupyter supported?
README mentions magic commands — . You can always parse / literaleval in notebooks with the library.
Can I transform scripts programmatically?
Yes — NodeVisitor / NodeTransformer on the AST, then unparse. See library API.
Worked examples
“Is my install healthy?”
`bash
python -c "from pynescript.ast import parse, unparse; print(unparse(parse('//@version=\nindicator(\"t\")\nplot(close)\n')))"
pynescript lint - <<< $'//@version=\nindicator("t")\nplot(close)\n'
`
“Does HTTP evaluate work?”
`bash
curl -s http://...:/ | python -m json.tool
`
“Is LSP importable?”
`bash
python -c "import pygls; from pynescript.langserver.server import PynescriptLanguageServer"
`
Failure modes
If an FAQ answer appears wrong against your checkout:
. Check package version (pynescript --version).
. Read the cited source file.
. Prefer main.py / pyproject.toml` over secondary docs.
. Open an issue or PR against the MDX page.
See also
End User hub
Installation
Troubleshooting
Glossary
Compatibility
CLI commands
---
FILE: docs/pyne/enduser/reference/glossary.mdx
Glossary
Abstract
Shared vocabulary for the End User track. Terms are defined as used in this repo, not as MarketingView marketing copy. Cross-links point at deeper systems manuals where useful.
Conceptual model
Terms fall into four buckets:
. Language representation — grammar, AST, unparser
. Execution — series, bar-loop, builtins, strategy events
. Surfaces — CLI, library, LSP, Pro API
. Ecosystem — AXIS, HOOX, pine-worker, pyne-worker
Interface surface
This page is pure reference — no commands required.
Internals (repo paths)
| Term cluster | Primary paths |
| --- | --- |
| Parse / AST | src/pynescript/ast/helper.py, builder.py, grammar/ |
| Evaluate | src/pynescript/ast/evaluator/, backend/runtime.py |
| Lint | src/pynescript/ast/linter.py |
| LSP | src/pynescript/langserver/ |
| HTTP | backend/app.py |
Invariants & edge cases
Pine Script / TradingView are trademarks of TradingView, Inc. PYNE is independent (see product index).
“Parity” is empirical (tests/corpus), not a legal claim of equivalence.
Terms
ASDL
Abstract Syntax Description Language. Schema language used to generate Python AST node classes (Pinescript.asdl → generated node module). Gives a single algebraic description of script structure.
Annotation
Special comments such as //@version= attached to script or declaration nodes during parse (addannotations in helper.py).
AST
Abstract Syntax Tree. In-memory tree of ASDL-generated nodes produced by parse. Root is typically a Script (mode="exec") or Expression (mode="eval").
AXIS
Optional charting PWA “AXIS” product (frontend/, docs at /axis/docs). Consumes evaluate results (series/plots); not required for evaluation.
Bar / bar-loop
A discrete OHLCV time step. A bar-loop evaluator re-executes script logic once per bar with updated series history (Runtime.run).
Builtin
Host-provided namespace function or value (ta.rsi, strategy.entry, math.sqrt, …). Implemented under ast/evaluator/builtins/ and catalogued for LSP metadata.
CLI
The pynescript Click command group (parse-and-dump, parse-and-unparse, lint, data, download-builtin-scripts).
Compile mode
mode="compile" on Runtime / /run: attempts a Numba (or similar) compiled subset of bar logic for speed. Not full language coverage.
Data provider
Historical OHLCV source (mock, yahoo, alphavantage, ccxt) via pynescript.util.data.
Data feed
Realtime stream abstraction (datafeed, e.g. CCXT Pro) for live candles/trades.
Dump
dump(node) — pretty textual representation of an AST for debugging and tests (not executable Pine).
Evaluate / evaluator
Execution of AST nodes to values. Ranges from literaleval (expression-safe) to full statement/builtin evaluators and HTTP Runtime.
Extra (package extra)
Optional dependency set in pyproject.toml: lsp, dev-lsp, data, datafeed.
HOOX
Edge trade execution mesh (separate monorepo / docs under /docs). Can consume strategy events from evaluate pipelines; optional.
Interpret mode
Default Runtime mode: AST walking per bar without the compiled subset path.
literaleval
Helper that parses an expression (or accepts an AST) and evaluates literals plus supported builtins/series context — analogous in spirit to Python’s ast.literaleval but Pine-aware and broader.
Lint / LintWarning
Static diagnostics with code, message, line, severity (error|warning). Entry: lintscript.
LSP
Language Server Protocol. pynescript-lsp process implementing diagnostics, completion, hover, navigation, formatting.
NodeVisitor / NodeTransformer
Patterns for traversing or rewriting ASTs (visitor.py, transformer.py).
OHLCV
Open, high, low, close, volume — bar fields passed to Runtime / /run (plus time).
pine-worker
TypeScript port / worker tool colocated under pine-worker/ for parity experiments and edge TS evaluate paths.
pyne-worker
Python Cloudflare Worker (sister repo) embedding pynescript for edge /run.
Plot / series / plotmeta
Evaluate outputs: plotted values, named multi-series maps, and metadata for AXIS UIs (AXIS).
Pro API
Flask application in backend/ exposing /run, preview, backtest, and auth endpoints.
PYNE
Product name for this evaluation stack (docs under /pyne/docs); package name on PyPI remains pynescript.
Round-trip
parse → unparse (optionally → parse again). Used to normalize formatting and test fidelity.
Runtime
backend.runtime.Runtime — multi-bar execution engine used by Pro API.
Series
Time-indexed value stream with history access (close[]). Implementation includes PineSeries and evaluator series objects.
Strategy event
Structured record of entries/exits/orders emitted during strategy evaluation for downstream systems (HOOX, analytics).
Unparse
unparse(node) — regenerate Pine source text from an AST.
//@version
Version pragma annotation; linter expects v+ for modern scripts.
Warmup
Bars required before a TA function has enough history; early outputs may be na.
Worked examples
Using terms together:
``text
Source --parse--> AST --unparse--> normalized source
AST + OHLCV --Runtime bar-loop--> series + strategy events
AST --lintscript--> LintWarnings
AST --LSP--> editor diagnostics/completion
series --AXIS--> chart
events --HOOX--> exchange orders
``
Failure modes
Misusing terms often causes support confusion:
| Confused pair | Distinction |
| --- | --- |
| dump vs unparse | dump is debug AST text; unparse is Pine source |
| literal_eval vs Runtime | single-shot expressions vs multi-bar scripts |
| pynescript vs pynescript-lsp | desk CLI vs language server |
| AXIS vs PYNE | AXIS vs evaluate engine |
| data provider vs data feed | historical fetch vs realtime stream |
See also
End User hub
FAQ
PYNE product map
AXIS docs
HOOX docs
---
FILE: docs/pyne/index.mdx
PYNE Documentation
PYNE is the open evaluation stack for TradingView Pine Script: a formal ANTLR grammar, an ASDL algebraic AST, a deterministic bar-loop evaluator, a Language Server Protocol surface, a Flask Pro API, and edge workers that share one evaluate contract.
Pine Script and TradingView are trademarks of TradingView, Inc. This project is independent and not affiliated with or endorsed by TradingView, Inc.
Abstract
Where closed hosts couple the language to a proprietary charting host, PYNE treats the language as an inspectable pipeline:
``
Source (.pine)
→ ANTLR lexer/parser (resource grammar)
→ ASDL AST (builder)
→ bar-loop evaluator (builtins, strategy, drawings)
→ plots / events / metrics
`
The same pipeline powers the desk CLI, the LSP binary, the Pro API, browser Pyodide (via AXIS), and Cloudflare workers. Round-trip fidelity (parse → unparse) and a large builtin corpus are first-class invariants, not marketing claims — see compatibility and implementation status.
Product map
| Surface | Role | Start here |
| --- | --- | --- |
| Library + CLI | Parse, lint, unparse, evaluate on your machine | End User hub |
| Language core | Grammar, AST, visitors, type system | Core |
| Runtime | Series model, builtins, strategy, compiler | Runtime |
| LSP | Diagnostics, completion, hover, format | LSP |
| Pro API | HTTP /run, preview, backtest | API |
| Ops | CI, Docker, Nuitka, secrets, GCP | DevOps |
| AXIS (separate product) | Charting PWA AXIS | AXIS docs |
| HOOX | Edge trade mesh | HOOX docs |
Conceptual model
Invariant: evaluation never requires a proprietary chart host. AXIS is an optional AXIS; HOOX is an optional execution mesh.
Tracks
End User
Install the package, run the CLI, embed the library API, wire an editor, or call the Pro API as a consumer.
→ End User hub
Core / Runtime / LSP / API
Contributor and systems manuals: how the front-end, semantics, editor protocol, and HTTP bridge are built.
DevOps
CI matrices, Fernet metadata encryption, Nuitka LSP binaries, containers, Cloud Build.
→ DevOps hub
Reference
Compatibility guarantees, missing surface, numerical validation, TypeScript pine-worker parity tool.
Sister surfaces
pyne-worker (separate repo) — Python Cloudflare Worker for production /run.
pine-worker (this monorepo, pine-worker/) — TypeScript port + converter; see pine-worker.
AXIS (frontend/) — installable PWA; documented at hoox.sh/axis/docs.
Offline & agent exports
Auto-generated like HOOX manuals (bun run docs:exports):
| Kind | Path |
| --- | --- |
| Full-corpus LLM pack | llm.txt |
| LLM site map (llmstxt.org) | llms.txt |
| Track PDFs (A) | /exports/pyne--manual.pdf` |
See also
Installation
Quick start
Grammar pipeline
Evaluate contract
---
FILE: docs/pyne/lsp/architecture.mdx
LSP Architecture
Abstract
The language server is a single-process pygls application. It owns a Workspace of open TextDocumentState records, re-parses on every mutation, and dispatches LSP methods to pure-ish feature handlers that take (params, source) rather than holding server globals. Capabilities are declared once at initialize; transport defaults to STDIO for editor integration.
Conceptual model
Interface surface
Process entry
``bash
Editable install
pip install -e ".[lsp]"
python -m pynescript.langserver same as pynescript-lsp
make run-lsp
`
Entry: src/pynescript/langserver/main.py constructs PynescriptLanguageServer() and calls server.startio() (STDIO JSON-RPC).
pyproject.toml registers:
`text
pynescript-lsp = "pynescript.langserver.main:main"
`
Server class
PynescriptLanguageServer (server.py):
Subclasses pygls.lsp.server.LanguageServer with name="Pynescript", version="..".
Instantiates self.pineworkspace = Workspace().
Registers handlers in setupmethodhandlers() via @self.feature(...).
Document sync
| Event | Behavior |
| --- | --- |
| didOpen | putdocument → parse/lint → push diagnostics |
| didChange | Incremental or full-text apply → re-parse/lint → push diagnostics |
| didClose | Remove document; publish empty diagnostic list |
| didSave | Re-publish diagnostics for current buffer |
Sync options (config.getservercapabilities):
TextDocumentSyncKind.Incremental
openclose=True
save with includetext=True
willsave / willsavewaituntil off
Capability set (declared)
From config.py:
Diagnostic provider (identifier="pynescript-diagnostics", workspacediagnostics=False in options; workspace pull handler still exists)
Completion: trigger ".", resolveprovider=True
Hover, definition, references
Document + workspace symbols
Full + range formatting
Inlay hints (resolveprovider=False)
Semantic tokens full (legend of standard token types/modifiers; range=false)
Signature help and code action options are advertised; dedicated handlers are not fully wired as first-class feature modules yet — treat as capability stubs for clients that probe the table
File filters (getfilteroptions): .pine, .pinev, .pinev under language id pinescript.
Internals
Workspace
Workspace (workspace.py) maps uri → TextDocumentState:
`text
TextDocumentState
uri, source, version
ast | None
diagnostics: list[LintWarning]
parseerror, parseerrorline
`
parseandlint:
. parse(source, filename=uri) — on success, lintscript(source, filename=uri).
. On exception: clear AST, store parseerror string, extract line via regex line[:\s]+(\d+).
Incremental edits: applytextedit splits on \n, replaces the LSP range, rejoins. Whole-document change events replace source wholesale.
Diagnostics conversion (lintwarningstodiagnostics):
Map severity strings → DiagnosticSeverity.
Source tag "PineScript", code from lint rule.
Append synthetic E error for parse failures at the extracted line.
Feature dispatch pattern
Handlers are functions, not server methods, e.g.:
`python
@self.feature(lsp.TEXTDOCUMENTCOMPLETION)
def textcompletion(params):
source = self.pineworkspace.getsource(params.textdocument.uri)
return completionfeature.handlecompletion(params, source)
`
Definition / references / symbols also pass uri for Location construction. Inlay hints and semantic tokens re-parse from source inside the feature module.
Pull diagnostics
textDocument/diagnostic → RelatedFullDocumentDiagnosticReport with resultid=f"{uri}-{version}".
workspace/diagnostic → one full report per open document from getalldiagnostics().
Workspace symbols
workspace/symbol walks each open document’s AST via collectworkspacesymbols: FunctionDef → Function, TypeDef → Class, Assign to Name → Variable. Filter is case-insensitive substring on params.query.
Invariants and edge cases
. Parse failure is non-fatal for the process. AST-dependent features return None or []; diagnostics still surface E.
. Source of truth is the open buffer, not disk. Save only re-publishes; it does not re-read files.
. Incremental sync assumes the client sends coherent ranges. Out-of-bounds ranges leave source unchanged (applytextedit early-return).
. Version is stored and used in pull-diagnostic resultid; the server does not reject out-of-order versions itself.
. Nuitka onefile embeds providers data and may rely on encrypted metadata — see builtin metadata. Architecture of handlers is identical to the pure-Python path.
Worked example — minimal initialize
Client → server (conceptual):
`json
{
"method": "initialize",
"params": {
"capabilities": {},
"clientInfo": { "name": "example", "version": "" }
}
}
`
Server returns InitializeResult with serverInfo.name = "Pynescript Language Server" and the full ServerCapabilities table from getserver_capabilities().
Failure modes
| Symptom | Likely cause |
| --- | --- |
| No diagnostics on open | Client did not send didOpen or language id is not pinescript |
| Completions empty | Metadata file missing/undecryptable; see builtin metadata |
| Format no-ops | Parse error (handler returns []) or unparsed text equals source |
| Extension can't spawn server | pynescript-lsp not on PATH`; check VS Code extension |
See also
Diagnostics
Builtin metadata
VS Code extension
Nuitka build
---
FILE: docs/pyne/lsp/builtin-metadata.mdx
Builtin Metadata
Abstract
Builtin metadata is the LSP documentation corpus: a map from fully qualified names (ta.sma, strategy.entry, …) to labels, signatures, briefs, snippets, and categories. It is generated from code (not hand-edited as source of truth), loaded by the language server for completion/hover/inlay, and optionally Fernet-encrypted for Nuitka onefile distribution so casual string extraction does not dump the full catalog.
This is obscurity for distribution hygiene, not a cryptographic access-control boundary.
Conceptual model
Interface surface (metadata record)
Typical entry in builtinmetadata.json:
``json
{
"ta.sma": {
"label": "ta.sma",
"kind": "function",
"detail": "ta.sma(series, int) → series float",
"brief": "…",
"documentation": "…",
"snippet": "ta.sma(${:param}, ${:param})",
"category": "ta.technicalanalysis"
}
}
`
Categories drive completion headers (ta.technicalanalysis → “Technical Analysis (ta.)”, builtin → “Built-in Variables”, etc.).
Loader API
providers/builtinmetadata.py:
| Function | Behavior |
| --- | --- |
| getmetadata() | Singleton cache; prefer .enc decrypt, else plaintext JSON, else {} |
| getbuiltin(name) | Single entry or None |
| getbuiltinsbycategory / getallcategories | Category queries |
| fuzzyfilter | Scored substring match for completion |
Consumers
Completion list / module / resolve
Hover cards
Inlay return-type extraction from detail after →
Internals
| Path | Role |
| --- | --- |
| scripts/generatebuiltinmetadata.py | Introspect builtins → JSON |
| src/pynescript/langserver/providers/builtinmetadata.json | Dev plaintext |
| …/builtinmetadata.json.enc | Encrypted blob for release |
| …/builtinmetadata.json.sha | First hex chars of SHA- of plaintext |
| …/metadatadecrypt.py | Fernet load + integrity check |
| scripts/build/compile.py | encryptmetadata(), Nuitka include of providers dir |
| scripts/build/cibuild.py | stagemetadata() for CI |
| scripts/build/.metadata.key | Gitignored Fernet key material |
Generation
After adding evaluator builtins:
`bash
python scripts/generatebuiltinmetadata.py
`
Do not hand-edit the JSON as the long-term source of truth; regenerate from code.
Encryption (build)
scripts/build/compile.py → encryptmetadata():
. Require cryptography and plaintext JSON.
. Generate Fernet key → write scripts/build/.metadata.key (mode o).
. Encrypt → builtinmetadata.json.enc.
. Write truncated SHA- of plaintext to .sha.
Local encryptmetadata currently always generates a new key when invoked. For byte-stable encrypted artifacts across CI runs, supply a stable secret:
| Context | Mechanism |
| --- | --- |
| GitHub Actions | secrets.METADATAKEY → env (documented as CRYPTOKEY in workflows / AGENTS) |
| Cloud Build | substitution ${METADATAKEY} → CRYPTOKEY |
| Runtime decrypt | File .metadata.key next to providers or env PYNESCRIPTMETADATAKEY |
Without a stable key, each CI encrypt produces a different .enc blob — not a functional bug, but noisy diffs and cache misses.
Decrypt path
metadatadecrypt.py:
. Resolve key: PyInstaller/Nuitka data dir .metadata.key, else providers dir, else PYNESCRIPTMETADATAKEY.
. Fernet-decrypt .enc.
. If .sha present, compare sha(plaintext).hexdigest()[:].
. json.loads → dict.
getmetadatacached() prefers plaintext when both exist (developer-friendly).
getmetadata() in builtinmetadata.py: if .enc exists, try decrypt cache; on failure empty dict; elif plaintext, load it.
Invariants and edge cases
. Regenerate after builtin changes or completion/hover/inlay drift.
. Empty dict on failure fails soft (no completions) rather than crashing the server.
. Integrity mismatch raises in decrypt — compiled binary should not silently serve corrupt docs.
. Key not in git — never commit .metadata.key.
. Fernet is symmetric — anyone with the binary + key material can recover plaintext; design goal is casual reverse-engineering friction and future paid-metadata swap hooks.
Worked example — local dev
`bash
python scripts/generatebuiltinmetadata.py
edit-free check
python -c "from pynescript.langserver.providers.builtinmetadata import getbuiltin; print(getbuiltin('ta.sma'))"
`
Worked example — encrypted binary path
`bash
build (encrypts + Nuitka) — see DevOps
python scripts/build/compile.py
export PYNESCRIPTMETADATAKEY="$(cat scripts/build/.metadata.key)"
or embed .metadata.key in the onefile data dir during compile
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Completions empty in release binary | Missing key / wrong PYNESCRIPTMETADATAKEY |
| Metadata integrity check failed | .enc and .sha out of sync |
| CI churn on .enc | New random key each build without CRYPTOKEY / METADATA_KEY |
| Stale docs for new ta.` | Forgot to re-run generator |
See also
Metadata crypto (DevOps)
Nuitka build
Completion
Hover
---
FILE: docs/pyne/lsp/clients.mdx
Editor Clients
Abstract
Any editor that speaks LSP over STDIO can host PYNE intelligence. First-class snippets live in clients/ (Neovim Lua, Zed JSON, Emacs Lisp) plus documented Helix/Sublime fragments. The VS Code path is a dedicated extension (vscode-extension); this page covers the rest.
Prerequisite: pynescript-lsp on PATH (Python .+):
``bash
pip install "pynescript[lsp]"
development
pip install -e ".[lsp]"
`
Conceptual model
Interface surface — repo files
| File | Editor |
| --- | --- |
| clients/neovim.lua | Neovim (nvim-lspconfig or manual) |
| clients/zed.json | Zed settings.json fragment |
| clients/emacs.el | Emacs lsp-mode + pinescript-mode |
| clients/README.md | Human-oriented install notes |
| vscode-extension/ | VS Code / Cursor-compatible |
Neovim
Return table from clients/neovim.lua:
`lua
return {
cmd = { 'pynescript-lsp' },
filetypes = { 'pinescript' },
rootdir = function(fname)
return vim.fs.root(fname, { '.git', '.pine', '.pinev', '.pinev', 'pyproject.toml' })
or vim.fn.getcwd()
end,
settings = {
pinescript = {
formatting = { enabled = true },
diagnostics = { enabled = true },
completion = { snippets = true },
},
},
}
`
Suggested maps when attaching: gd definition, gr references, K hover, format via vim.lsp.buf.format.
Register filetype if needed:
`vim
au BufRead,BufNewFile .pine,.pinev,.pinev setfiletype pinescript
`
Zed
Merge clients/zed.json into ~/.config/zed/settings.json:
`json
{
"languages": {
"Pine Script": {
"languageservers": ["pynescript"]
}
},
"languageservers": {
"pynescript": {
"command": "pynescript-lsp",
"arguments": ["--stdio"],
"languages": ["Pine Script"]
}
}
}
`
Emacs
clients/emacs.el registers an lsp-mode client:
`elisp
(lsp-register-client
(make-lsp-client
:new-connection (lsp-stdio-connection '("pynescript-lsp" "--stdio"))
:major-modes '(pinescript-mode)
:server-id 'pynescript))
`
Defines a minimal pinescript-mode with font-lock keywords and auto-mode-alist for \\.pine\\' / \\.pinev[-]+\\'. Optional keys: C-c C-c format, M-. definition, M-? references.
Helix
~/.config/helix/languages.toml:
`toml
[[language]]
name = "pinescript"
scope = "source.pinescript"
file-types = ["pine", "pinev", "pinev"]
roots = ["pyproject.toml"]
command = "pynescript-lsp"
args = ["--stdio"]
`
Sublime Text
With Package Control LSP:
`json
{
"clients": {
"pynescript": {
"command": ["pynescript-lsp", "--stdio"],
"selector": "source.pinescript",
"initializationOptions": {}
}
}
}
`
Generic / Cursor
`bash
pynescript-lsp
or explicitly:
pynescript-lsp --stdio
`
Point the host’s “external language server” command at that binary. Cursor users can also load the VS Code extension in compatible mode when packaged as VSIX.
Internals
Server entry always uses STDIO (startio in main.py). Capability negotiation is identical across hosts — differences are client UI only (how inlay hints render, whether pull diagnostics are used, etc.).
Tests: tests/testlspfeatures.py (handlers with fake params), tests/test_langserver.py (ee with pytest-asyncio) — not editor-specific.
Invariants and edge cases
. Language id should be pinescript for VS Code parity; some editors invent their own names — map filetypes carefully.
. Root directory heuristics vary; wrong root rarely breaks single-file Pine editing (workspace is URI-keyed open buffers).
. Settings keys under pinescript in Neovim are conventional; the Python server does not currently enforce a rich workspace/configuration schema for all of them.
. Nuitka onefile can replace the module entry for air-gapped machines — same command name if installed on PATH.
Failure modes
| Symptom | Cause |
| --- | --- |
| Server exits immediately | Missing pygls / incomplete install .[lsp] |
| Filetype never attaches | Extension not mapped to pinescript` |
| Features missing in Helix | Older Helix without inlay / semantic support — grammar still useful |
See also
VS Code extension
Architecture
End-user editors guide
---
FILE: docs/pyne/lsp/features/completion.mdx
Completion
Abstract
Completion is metadata-driven, not evaluator-driven. The server loads a catalog of builtins (plaintext JSON in development, Fernet-encrypted in compiled binaries), filters by prefix or module, and returns CompletionItem rows with optional snippet insertText. Resolve re-hydrates documentation for a single item.
Conceptual model
Interface surface
| Method | Capability |
| --- | --- |
| textDocument/completion | triggercharacters=["."], returns CompletionList |
| completionItem/resolve | resolveprovider=True |
Handler: features/completion.py → handlecompletion / handlecompletionresolve.
Trigger logic
. Compute textbeforecursor on the current line.
. gettriggerchar — if the previous character is . (or (, ,, space), treat as trigger.
. If the last token contains . (e.g. ta. or ta.sm), call buildmodulecompletion(module).
. Else buildcompletionlist(prefix=prefix) over the full catalog.
Word boundaries for Pine identifiers: [a-zA-Z][a-zA-Z-.] (protocol/utils.py).
Item fields
Built by providers/completionitems.py:
| Field | Source |
| --- | --- |
| label | metadata label (e.g. ta.sma) |
| kind | CompletionItemKind.Function |
| detail | signature / detail string |
| documentation | Markdown from metadata |
| inserttext | Snippet if ${...} present, else plain label |
| inserttextformat | Snippet or PlainText |
| filtertext | dotted parts + brief |
| sorttext | modules first (\x…), root second (\x…) |
Category headers may appear as Folder-kind items with empty insert text and labels like --- Technical Analysis (ta.) (N) ---.
Internals
| Path | Role |
| --- | --- |
| features/completion.py | Request context + dispatch |
| providers/completionitems.py | List / item / module builders |
| providers/builtinmetadata.py | Load cache, fuzzyfilter, getbuiltin |
| protocol/utils.py | Word + trigger helpers |
Fuzzy filter scores
fuzzyfilter(query, items, limit=):
| Match | Score |
| --- | --- |
| Exact label | |
| Prefix | |
| Substring in label | |
| Category | |
| Brief | |
Results sorted by score descending, capped at limit.
Resolve
handlecompletionresolve looks up params.label in metadata; if found, rebuilds a full CompletionItem. Unknown labels return unchanged.
Invariants and edge cases
. No user-symbol completion yet — only catalog builtins (not local myFunc unless it appears in metadata).
. Empty metadata dict yields an empty list (failed decrypt / missing files).
. Module completion uses startswith(module + ".") — nested namespaces beyond one dot still work if labels are fully qualified.
. Snippets are generated at metadata build time (scripts/generatebuiltinmetadata.py); incompleteness of placeholders is a generator concern, not the LSP handler.
Worked example
User types ta. → module completion for all ta. labels.
User types sma without a module → fuzzy filter may return ta.sma among others (substring score).
Resolve on ta.sma reloads full documentation markup for the detail pane.
Failure modes
| Symptom | Cause |
| --- | --- |
| Zero items always | Metadata not loaded — check builtin metadata |
| Dot trigger ignored | Client did not set completion trigger characters from server capabilities |
| Snippets inserted raw | Client disabled snippet support; extension setting pynescript.completion.snippets |
See also
Hover — shares metadata
Builtin metadata
Inlay hints — uses return types from detail
---
FILE: docs/pyne/lsp/features/diagnostics.mdx
Diagnostics
Abstract
Diagnostics are the primary static feedback channel. The workspace re-parses and re-lints on every open/change/save, converts LintWarning objects (and parse failures) into lsp.Diagnostic, then pushes them with textDocument/publishDiagnostics. Clients that support LSP .+ pull diagnostics can also request textDocument/diagnostic or workspace/diagnostic.
Conceptual model
Interface surface
Push (always on)
Triggered from server.py on didOpen, didChange, didSave. Close clears diagnostics (items=[]).
Pull
| Method | Response |
| --- | --- |
| textDocument/diagnostic | RelatedFullDocumentDiagnosticReport with kind=full, resultid="{uri}-{version}" |
| workspace/diagnostic | One WorkspaceFullDocumentDiagnosticReport per open URI |
Capability: diagnosticprovider with identifier="pynescript-diagnostics", interfiledependencies=False.
Diagnostic shape
| Field | Value |
| --- | --- |
| range | -based line; column from warning or ; end spans remainder of line (capped) |
| severity | error / warning / information / hint |
| message | Human-readable lint or parse message |
| source | "PineScript" |
| code | Lint code (e.g. W) or E for parse errors |
The richer conversion in features/diagnostics.py also supports:
codedescription hrefs for codes starting with E / W (docs URLs)
tags: W → Unnecessary, W → Deprecated
Workspace conversion in workspace.py is the path used on push; the feature module API is available for shared logic and quick-fix helpers.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/langserver/workspace.py | parseandlint, lintwarningstodiagnostics |
| src/pynescript/langserver/features/diagnostics.py | Conversion helpers, tags, createquickfix |
| src/pynescript/ast/linter.py | Rule engine producing LintWarning |
Severity map
``text
error → DiagnosticSeverity.Error
warning → Warning
info|information → Information
hint → Hint
(default) → Warning
`
Parse errors
On exception from parse:
ast = None, diagnostics = [] (lint skipped)
Message stored as parseerror
Synthetic diagnostic code E, severity Error, line from regex on the exception text
Quick fixes (feature module)
createquickfix (not yet fully wired through workspace/executeCommand / codeAction handlers in server.py):
W → insert //@version=\n at document start
C (long line) → suggest format document command
Capability advertises QuickFix / Refactor / SourceOrganizeImports kinds for future wiring.
Invariants and edge cases
. Lint does not run when parse fails — only E is shown.
. Warnings without a line number are dropped (None conversion).
. End column heuristic uses full line length (or +) and clamps to characters — not a precise token span.
. interfile_dependencies=False: no cross-file analysis; each URI is independent.
. Push and pull should agree for a given buffer version; both read the same TextDocumentState.
Worked example
Source:
`pinescript
indicator("x")
plot(close)
`
If the linter emits a missing-version warning (code W), the client receives a Warning diagnostic on the relevant line with source: "PineScript" and may offer the version-header quick fix when code actions are connected.
On a hard parse failure at line , expect:
`json
{
"severity": ,
"code": "E",
"source": "PineScript",
"message": ""
}
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Stale squiggles after edit | Client using full-document sync incorrectly, or version mismatch |
| Empty diagnostics on broken file | Unexpected: parse errors should still emit E — check client filter on source` |
| No workspace pull results | Only open documents are tracked; closed files are absent |
See also
Linter (core)
Error model
Architecture
---
FILE: docs/pyne/lsp/features/formatting.mdx
Formatting
Abstract
Formatting is canonical unparse, not a style linter. The handler parses the buffer, runs NodeUnparser, and if the result differs from the source, replaces the document (or the requested line range) with a single TextEdit. There is no prettier-like option matrix; FormattingOptions from the client are currently unused.
Conceptual model
Interface surface
| Method | Handler |
| --- | --- |
| textDocument/formatting | handleformatting |
| textDocument/rangeFormatting | handlerangeformatting |
Capabilities: both providers True in config.py.
Full document
. Parse with filename "".
. formatted = NodeUnparser().visit(tree).
. If identical to source → [].
. Else one TextEdit spanning (,) → last line/col with newtext=formatted.
Range
. Unparse the entire document.
. Slice formatted lines and source lines by the client range’s start/end line indices.
. If the in-range slices match → [].
. Else replace params.range with the formatted line slice.
This is line-aligned, not AST-node-aligned: partial mid-line ranges may produce surprising splices when unparse changes line structure.
Internals
| Path | Role |
| --- | --- |
| features/formatting.py | LSP handlers |
| src/pynescript/ast/helper.py | parse |
| src/pynescript/ast/unparser.py | NodeUnparser |
VS Code command pynescript.formatDocument delegates to editor.action.formatDocument, which hits this provider when the language client is active.
Invariants and edge cases
. Parse failure → [] (no partial format; silent no-op from the client’s perspective).
. Idempotence goal: format twice should stabilize if unparse is canonical; any non-idempotent unparse is a core bug, not an LSP bug.
. Comments / trivia: fidelity is whatever the unparser preserves — not a full CST pretty-printer.
. Range formatting still requires a full successful parse of the whole file.
. Tab size / insert spaces from FormattingOptions are ignored today.
Worked example
Before:
``pinescript
//@version=
indicator("x")
plot( close )
`
After full format (illustrative — exact whitespace follows NodeUnparser):
`pinescript
//@version=
indicator("x")
plot(close)
`
If source already matches unparse output, the server returns an empty edit list and the editor leaves the buffer alone.
Failure modes
| Symptom | Cause |
| --- | --- |
| Format does nothing on broken syntax | Expected: exception → []` |
| Range format rewrites wrong lines | Unparse shifted line count; prefer full-document format |
| Client “format on save” loops | Unparse not idempotent — fix unparser, not client |
See also
Unparser
Architecture
VS Code extension
---
FILE: docs/pyne/lsp/features/hover.mdx
Hover
Abstract
Hover answers “what is this identifier?” for builtins. The handler extracts the word under the cursor (including dotted forms), looks up metadata, and returns a Markdown card: signature fence, brief, optional long documentation, curated examples, and related symbols. User-defined symbols without metadata return null.
Conceptual model
Interface surface
Method: textDocument/hover
Capability: hoverprovider=True
Handler: features/hover.py → handlehover
Lookup order
. Exact word at cursor via getwordatposition (identifiers may include .).
. If no metadata hit, inspect text before the word: if it ends with module., try module.word.
. Build hover with range covering start, end) of the word on the line.
Card structure
``markdown
`pinescript
{detail}
`
{brief}
---
{first documentation paragraph, ≤ chars}
Example: …
See also: ta.ema, …
`
Examples and related maps are hardcoded for a small high-value set (ta.sma, ta.ema, ta.rsi, ta.macd, strategy entry/exit, etc.). Others show signature + brief only.
External TradingView doc links are intentionally omitted (builddocslink returns empty) to keep hover self-contained.
Internals
| Path | Role |
| --- | --- |
| features/hover.py | Word resolution + Markdown assembly |
| providers/builtinmetadata.py | getbuiltin |
| protocol/utils.py | Word extraction |
Invariants and edge cases
. No source → null. Missing buffer aborts early.
. Line OOB → null. Cursor past last line is ignored.
. Empty word → null. Whitespace / punctuation yields no hover.
. User symbols (local variables, UDTs) are not inferred for hover; use [definition / outline instead.
. Documentation truncation is first-paragraph / -char only — full docs live in metadata and completion resolve.
Worked example
Cursor on sma in plot(ta.sma(close, )):
Word may be ta.sma if the dotted identifier is one match, or sma with module recovery from ta..
Response contents.kind = markdown with fence of the metadata detail line.
Failure modes
| Symptom | Cause |
| --- | --- |
| Hover never appears | Metadata empty or word not in catalog |
| Wrong symbol | Identifier regex swallowed neighboring text — rare with _.` pattern |
| Stale docs after adding builtin | Regenerate metadata — builtin metadata |
See also
Completion
Builtin metadata
Navigation
---
FILE: docs/pyne/lsp/features/inlay-hints.mdx
Inlay Hints
Abstract
Inlay hints surface inferred types next to simple name = expr assignments when no explicit type annotation is present. Inference is deliberately shallow: constants, bare series builtins (close, …), input. constructors, and ta/math/str/color calls whose metadata detail contains a → return type. Ambiguous operators are skipped rather than guessed wrong.
Conceptual model
Example mental model (from module docstring):
``text
length = → length: const int
rsi = ta.rsi(close, ) → rsi: series float (from metadata detail)
n = input.int(, "n") → n: input int
`
Interface surface
Method: textDocument/inlayHint
Capability: InlayHintOptions(resolveprovider=False)
Handler: features/inlayhints.py → handleinlayhints
Each hint:
| Field | Value |
| --- | --- |
| position | End of target identifier (lineno-, coloffset + len(id)) |
| label | ": {typelabel}" |
| kind | InlayHintKind.Type |
| tooltip | Inferred type: {typelabel} |
Returns [] for empty source, None if parse fails (client typically treats as no hints).
Internals
Constant mapping
| Python value | Label |
| --- | --- |
| bool | const bool |
| int | const int |
| float | const float |
| str | const string |
Builtin bare names
open, high, low, close, volume, hl, hlc, ohlc → series float; time / bar index vars → series int; na → na.
Calls
input.{int,float,bool,string,color,symbol,session,source,time,timeframe,price} → input {attr}
ta|math|str|color.{attr} → parse detail after →, else fallback "series"
Walk
Recursive over fields (same pattern as workspace symbol collection) — not limited to script top-level; nested function assigns can receive hints.
| Path | Role |
| --- | --- |
| features/inlayhints.py | Collection + inference |
| providers/builtinmetadata.py | Return-type detail for modules |
Invariants and edge cases
. Explicit name: type = … suppresses hints (node.type is not None).
. Tuple / attribute targets are ignored.
. BinOp / Compare / Conditional return None — no speculative arithmetic types.
. Metadata quality bounds accuracy — missing → in detail yields coarse series.
. No resolve step — tooltips are final.
Worked example
`pinescript
//@version=
indicator("hints")
length =
src = close
avg = ta.sma(src, length)
period = input.int(, "Period")
`
Expected hints (when parse + metadata succeed):
length → : const int
src → : series float
avg → type from ta.sma detail (typically series float) or series
period → : input int
avg will not get a hint derived from length’s type through the call graph — only the call’s known return shape.
Failure modes
| Symptom | Cause |
| --- | --- |
| No hints on file | Parse error → None |
| ta. always : series | Metadata detail lacks → segment |
| Hints mid-identifier | Incorrect col_offset` on AST node — builder issue |
See also
Completion
Builtin metadata
Type system
---
FILE: docs/pyne/lsp/features/navigation.mdx
Navigation
Abstract
Navigation features operate on user AST symbols, not the builtin catalog. Definition and references walk a freshly parsed tree with NodeVisitor subclasses; document symbols build a hierarchical outline; workspace symbols scan all open documents. Locations currently anchor to line starts (coarse ranges) — enough for jump lists, not character-precise rename.
Conceptual model
Interface surface
| Method | Handler | Returns |
| --- | --- | --- |
| textDocument/definition | definitions.handledefinition | list[Location] \| None |
| textDocument/references | references.handlereferences | list[Location] (possibly empty) |
| textDocument/documentSymbol | symbols.handledocumentsymbols | list[DocumentSymbol] |
| workspace/symbol | server.collectworkspacesymbols | list[SymbolInformation] |
Capabilities: definitionprovider, referencesprovider, document + workspace symbol options with work-done progress flags.
Definition (DefinitionFinder)
Matches target word against:
FunctionDef.name
TypeDef.name
Assign targets (Name or nested names)
Ignores mere call sites (visitCall returns early when the callee name matches).
References (ReferencesFinder)
Name with Load / Store contexts
Function and type definitions when includedeclaration is true
Call sites of bare Name callees
params.context.includedeclaration controls whether declarations appear.
Document symbols
Hierarchical:
| AST | SymbolKind | Children |
| --- | --- | --- |
| FunctionDef | Function | Variables assigned inside body |
| TypeDef | Class | Fields (Assign targets) |
| Top-level Assign | Variable | — |
Function detail is a coarse function name() string; assignments may show callee module attribute as detail when RHS is a call.
Workspace symbols
Flat SymbolInformation for functions, types, and assigned names across open buffers; filtered by lowercase query substring.
Internals
| Path | Role |
| --- | --- |
| features/definitions.py | Go-to-definition visitor |
| features/references.py | Find-all-references visitor |
| features/symbols.py | Document outline |
| server.py | Workspace symbol aggregation |
Parse is per request inside definition/references/symbols (not the workspace cached AST). That trades a little CPU for isolation if cache and buffer ever diverge.
Invariants and edge cases
. Parse failure → empty / null (definition None, others []).
. Builtin names have no definition in-file; clients should not expect ta.sma to resolve to a library file URI.
. Ranges are line-coarse for many locations (character=); selection ranges for document symbols use coloffset when present.
. Scope is naive — same identifier in nested functions may collect multiple definitions; no shadowing analysis.
. Closed documents disappear from workspace symbol search.
Worked example
``pinescript
//@version=
indicator("nav")
length =
mySma(src, len) =>
ta.sma(src, len)
v = mySma(close, length)
`
Definition on mySma at the call site → function line.
References on length → assign + call argument (and declaration if included).
Outline → length variable, mySma function (with locals), v variable.
Failure modes
| Symptom | Cause |
| --- | --- |
| Jump lands column | Expected with current Location encoding |
| Missing references inside function when targeting another function | visitFunctionDef` only walks body when the def name differs from target — intentional for declaration handling; calls inside same-named functions are a known edge |
| Empty outline | Parse error or empty source |
See also
Architecture
Hover — builtins vs user symbols
Diagnostics
---
FILE: docs/pyne/lsp/features/semantic-tokens.mdx
Semantic Tokens
Abstract
Semantic tokens let the server paint identifiers with roles richer than TextMate scopes (e.g. distinguish library functions from user variables). PYNE advertises a full semantic-tokens provider with a standard legend, and implements textDocument/semanticTokens/full as a safe stub: parse succeeds, data is an empty integer array. Editors that require tokens degrade to grammar-only highlighting without protocol errors.
Conceptual model
Interface surface
Capability (config.py → semantictokensprovider):
``text
legend.tokentypes:
namespace, type, class, enum, interface, struct, typeParameter,
parameter, variable, property, enumMember, event, function,
method, macro, keyword, modifier, comment, string, number,
regexp, operator
legend.tokenmodifiers:
declaration, definition, readonly, static, deprecated,
abstract, async, modification, documentation, defaultLibrary
range: false
full: true
`
Handler: features/semantictokens.py → handlesemantictokens.
Response type: lsp.SemanticTokens(data=[...]) — LSP-encoded five-tuples as a flat int array.
Internals
| Path | Role |
| --- | --- |
| features/semantictokens.py | Stub handler |
| config.py | Legend + capability |
| server.py | TEXTDOCUMENTSEMANTICTOKENSFULL registration |
The module docstring states intent: a visitor would emit (line, col, len, tokentypeindex, modifiers_bitfield) with relative line/character deltas per the LSP spec.
Until that lands, TextMate highlighting from vscode-extension/syntaxes/pinescript.tmLanguage.json remains the primary colorization path for VS Code.
Invariants and edge cases
. Never throws — exceptions from parse yield empty data.
. No range provider — clients must not request semanticTokens/range.
. Legend is fixed at initialize; changing indices later would break cached clients.
. Empty data is valid protocol, not an error.
Worked example
Request textDocument/semanticTokens/full for any open buffer:
`json
{ "data": [] }
`
Client falls back to TextMate / tree-sitter grammar scopes.
Failure modes
| Symptom | Cause |
| --- | --- |
| “Semantic highlighting” setting appears on but changes nothing | Stub returns empty data — expected |
| Client errors on range request | range=False` in capabilities; client should not call it |
See also
VS Code extension — TextMate grammar
Architecture
Inlay hints — complementary visual annotations
---
FILE: docs/pyne/lsp/index.mdx
Language Server
The PYNE Language Server is the editor-facing half of the evaluation stack: a pygls process that speaks LSP over STDIO (or TCP in tooling), parses Pine with the same ANTLR → ASDL pipeline as the runtime, and never requires a proprietary chart host.
Abstract
Editors want static intelligence at typing cadence. Evaluation over OHLCV is dynamic and bar-loop expensive. PYNE therefore splits the stack:
| Process | Cadence | Shared substrate |
| --- | --- | --- |
| LSP (pynescript-lsp) | keystroke / open / save | parse, lint, unparse, builtin metadata |
| Pro API (backend.app) | request / batch | parse, evaluate, plots / events |
| CLI / library | batch / embed | same AST + evaluator |
The LSP publishes diagnostics from the linter and parse errors, completes and hovers from Fernet-ready builtin metadata, navigates user symbols via AST visitors, and formats by round-tripping through the unparser. Feature modules live under src/pynescript/langserver/features/; the VS Code extension and clients/ configs are thin protocol hosts.
Conceptual model
Invariant: open/change/close always re-parse and re-lint the in-memory buffer. Features that need an AST re-parse on demand if the workspace cache is missing source.
Interface surface
| Capability | LSP method(s) | Module |
| --- | --- | --- |
| Sync | textDocument/didOpen|didChange|didClose|didSave | server.py + workspace.py |
| Diagnostics (push + pull) | publishDiagnostics, textDocument/diagnostic | workspace.py, features/diagnostics.py |
| Completion (+ resolve) | textDocument/completion, completionItem/resolve | features/completion.py |
| Hover | textDocument/hover | features/hover.py |
| Definition / references | textDocument/definition, textDocument/references | definitions.py, references.py |
| Symbols | textDocument/documentSymbol, workspace/symbol | symbols.py, server.py |
| Formatting | textDocument/formatting, rangeFormatting | features/formatting.py |
| Inlay hints | textDocument/inlayHint | features/inlayhints.py |
| Semantic tokens | textDocument/semanticTokens/full | features/semantictokens.py (stub data) |
Declared capabilities: src/pynescript/langserver/config.py (getservercapabilities).
Internals
| Path | Role |
| --- | --- |
| src/pynescript/langserver/server.py | PynescriptLanguageServer, method registration |
| src/pynescript/langserver/workspace.py | Document map, incremental edits, parse+lint |
| src/pynescript/langserver/config.py | ServerCapabilities |
| src/pynescript/langserver/features/ | Per-method handlers |
| src/pynescript/langserver/providers/ | Metadata + completion builders |
| src/pynescript/langserver/main.py | STDIO entry (pynescript-lsp) |
| vscode-extension/ | Language client + TextMate grammar |
| clients/ | Neovim, Zed, Emacs, Helix snippets |
Console scripts are separate: pynescript (Click CLI) vs pynescript-lsp (pygls). Do not conflate them — see architecture.
Tracks in this tab
. Architecture — server lifecycle, workspace, capabilities
. Diagnostics through Inlay hints — feature manuals
. Builtin metadata — generation + Fernet .enc / CRYPTO_KEY
. VS Code extension — client packaging
. Clients — Neovim, Zed, Emacs, Helix, Sublime
See also
Pro API — HTTP evaluate path sharing the same AST
Evaluate contract
Linter
Unparser
Editors guide (end user)
---
FILE: docs/pyne/lsp/vscode-extension.mdx
VS Code Extension
Abstract
The VS Code extension is a LanguageClient host plus a TextMate grammar. It does not reimplement diagnostics or completion in TypeScript; it spawns pynescript-lsp (or a configured command) over STDIO via vscode-languageclient, maps .pine / .pinev / .pinev to language id pinescript, and exposes a small settings surface for enablement and feature toggles.
Conceptual model
Interface surface
Package identity
From vscode-extension/package.json:
| Field | Value |
| --- | --- |
| name | pynescript |
| displayName | Pine Script |
| engines.vscode | ^.. |
| main | ./out/extension.js |
| dependency | vscode-languageclient ^ |
Language contribution
id: pinescript
extensions: .pine, .pinev, .pinev
configuration: language-configuration.json
grammar: syntaxes/pinescript.tmLanguage.json (scopeName: source.pinescript)
Activation
``text
onLanguage:pinescript
workspaceContains:/.pine
workspaceContains:/.pinev
workspaceContains:/.pinev
`
Settings (pynescript.)
| Key | Default | Meaning |
| --- | --- | --- |
| lsp.enabled | true | Skip activation when false |
| lsp.command | pynescript-lsp | Server executable |
| formatting.enabled | true | Passed as init option |
| diagnostics.enabled | true | Passed as init option |
| completion.snippets | true | Passed as init option |
Initialization options object:
`json
{
"formattingEnabled": true,
"snippetsEnabled": true,
"diagnosticsEnabled": true
}
`
Commands
| Command | Action |
| --- | --- |
| pynescript.restartServer | client.stop() then client.start() |
| pynescript.formatDocument | editor.action.formatDocument |
Client options
Document selector: pinescript for file and untitled schemes
File watcher: /.pine (not automatically the v/v suffixes)
Diagnostic collection name: pynescript
Internals
| Path | Role |
| --- | --- |
| vscode-extension/src/extension.ts | Activate / deactivate / client |
| vscode-extension/package.json | Contributes + scripts |
| vscode-extension/syntaxes/pinescript.tmLanguage.json | Grammar |
| vscode-extension/language-configuration.json | Brackets / comments |
| out/extension.js | Compiled JS |
Server launch:
`typescript
const serverOptions: ServerOptions = {
command: lspCommand, // default pynescript-lsp
args: ['--parent-dir', scriptPath],
transport: TransportKind.stdio,
options: { env: { ...process.env, PYTHONPATH: scriptPath } },
};
`
scriptPath is resolved to ../src/pynescript/langserver relative to the extension for monorepo development. Release packaging typically expects pynescript-lsp on PATH (pip or Nuitka binary).
Build
`bash
cd vscode-extension
npm ci
npm run compile tsc → out/
npx vsce package VSIX
or monorepo:
make build-vscode
`
Requires Node tooling as noted in project agents docs for packaging.
Invariants and edge cases
. lsp.enabled === false short-circuits activate — no client, no restart command registration beyond early return (commands not registered).
. Binary vs module: production users should install pynescript[lsp] or the Nuitka onefile and set pynescript.lsp.command if not on PATH.
. Grammar works without LSP — open a .pine file offline and still get TextMate highlighting.
. File watcher only lists /.pine; v/v still activate via language id on open.
Worked example — extension development
`bash
pip install -e ".[lsp]"
cd vscode-extension && npm ci && npm run compile
Launch Extension Development Host pointing at vscode-extension
`
Open a fixture .pine file; confirm Problems panel shows parse/lint diagnostics from the Python server.
Failure modes
| Symptom | Cause |
| --- | --- |
| “Language client failed to start” | pynescript-lsp missing / wrong command |
| No squiggles, highlighting OK | LSP disabled or server crash; check Output → Pine Script Language Server |
| Restart does nothing | Client never started (lsp.enabled` false) |
See also
Clients — non-VS Code hosts
Architecture
Nuitka build
Editors guide
---
FILE: docs/pyne/pine-worker/converter.mdx
Python → TypeScript converter
Abstract
pine-worker/scripts/convert-python-to-ts.py is a porting accelerator, not a verified transpiler. It reads Python builtin modules (AST of the Python source), emits TypeScript scaffolding with imports, function signatures, best-effort JSDoc from docstrings, and TODO bodies.
The human (or follow-on agent) still implements semantics to match the Python oracle and locks them with parity tests.
Conceptual model
Interface surface
CLI
``bash
python pine-worker/scripts/convert-python-to-ts.py \
src/pynescript/ast/evaluator/builtins/numeric.py
redirect
python pine-worker/scripts/convert-python-to-ts.py \
src/pynescript/ast/evaluator/builtins/numeric.py \
pine-worker/src/evaluator/builtins/numeric.ts
namespace hint
python pine-worker/scripts/convert-python-to-ts.py \
--namespace technical \
--module src/pynescript/ast/evaluator/builtins/technical.py
`
Emitted properties
| Feature | Behavior |
| --- | --- |
| Function signatures | Args + crude TS types from annotations |
| Docstrings | JSDoc blocks when present |
| Bodies | TODO / stubs — not executable ports |
| Types mapping | int/float→number, str→string, containers→broad any[] / Record |
Explicit non-goals
Full semantic translation of Python control flow
NumPy / Pine series awareness
Automatic parity proof
Keeping output green without edits
Internals
| Path | Role |
| --- | --- |
| pine-worker/scripts/convert-python-to-ts.py | Converter implementation |
| src/pynescript/ast/evaluator/builtins/.py | Preferred inputs |
| pine-worker/src/evaluator/ | Destination tree for ports |
Type mapping (simplified)
The script applies regex-ish substitutions for common typing names (Optional, List, Dict, …) and strips generics crudely. Expect to hand-fix types toward Zod-validated AST nodes and PineSeries.
Invariants & edge cases
. Re-running clobbers manual bodies if you redirect over a finished file — convert to a new path or diff carefully.
. Mixin methods / dynamic dispatch in Python may not map : to TS functions.
. Default args and args need manual TS redesign.
. Prefer converting one namespace at a time to keep reviewable diffs.
Worked examples
Port math helpers workflow
`bash
mkdir -p pine-worker/src/evaluator/builtins
python pine-worker/scripts/convert-python-to-ts.py \
src/pynescript/ast/evaluator/builtins/numeric.py \
pine-worker/src/evaluator/builtins/numeric.ts
implement TODOs
cd pine-worker && bun test
`
Compare against oracle
`bash
python tests/fixtures/parity/generate_fixtures.py
add TS assertions consuming tests/fixtures/parity/json/.json
`
Failure modes
| Symptom | Fix |
| --- | --- |
| Empty output | Path wrong / no functions found |
| any` everywhere | Annotations missing in Python source |
| Runtime divergence | Implement NA rules; don’t trust stubs |
| Import cycles in TS | Align module graph with evaluator design |
See also
pine-worker overview
Testing & parity
Runtime builtins
---
FILE: docs/pyne/pine-worker/index.mdx
pine-worker
Abstract
pine-worker is a TypeScript implementation track for the Pine Script evaluator (AST types, visitor dispatch, series history, and eventually builtins + strategy events). It lives at pine-worker/ inside the pynescript repository as an extra tool, not a replacement for the Python package.
Colocation goals:
Share parity fixtures under tests/fixtures/parity/
Keep the ANTLR resource grammar as single syntactic truth (TS parser target can be generated later)
Allow cross-language porting in one checkout
Python (pynescript + backend/) remains the semantic oracle.
Conceptual model
Interface surface
Quick start (Bun)
``bash
cd pine-worker
bun install
bun test
bun run typecheck
`
package.json scripts:
| Script | Action |
| --- | --- |
| test | bun test |
| test:watch | watch mode |
| typecheck | tsc --noEmit |
| parity | reserved — harness not fully wired |
Dependencies: Zod (AST schemas), TypeScript; runtime assumes Bun for tests.
Current skeleton
| Area | State |
| --- | --- |
| AST Zod schemas + types | Partial (mirrors Python ASDL intent) |
| Base evaluator + visitor dispatch | Partial |
| PineSeries historical access | Partial |
| Core NA / loop signal semantics | Partial |
| Full builtins | In progress |
| ANTLR TS parser | Not complete |
| Strategy event emission | Plan-driven (see opencode plans) |
| Parity harness test/parity/ | Planned |
Relationship to HOOX
When mature, pine-worker may be vendored/submoduled into hoox-setup as a Cloudflare Worker bound to trade-worker. Until then it develops here. Sister: pyne-worker (Python edge) is a different repo.
Internals
| Path | Role |
| --- | --- |
| pine-worker/src/ast/types.ts | AST typing / Zod |
| pine-worker/src/evaluator/evaluator.ts | Visitor evaluator |
| pine-worker/src/evaluator/series.ts | Series model |
| pine-worker/src/evaluator/types.ts | NA, signals, registries |
| pine-worker/test/.test.ts | Unit tests |
| pine-worker/scripts/convert-python-to-ts.py | Porting aid |
| tests/fixtures/parity/ | Shared oracle fixtures |
| .opencode/plans/pine-worker | Strategy-events plans |
Invariants & edge cases
. Python wins ties. If TS and Python disagree, fix TS (unless Python is proven wrong — then fix both + fixtures).
. NA is a sentinel, not JS null alone — truthiness differs from ordinary JS symbols.
. Do not fork grammar ad hoc in TS; prefer shared resource .g.
. Converter emits stubs, not proofs — manual port + parity required.
. License follows parent pynescript project.
Worked examples
Minimal evaluator unit (conceptual)
TS tests construct literal AST nodes (Literal, Assign, Script) and assert visitor results — see pine-worker/test/evaluator.test.ts.
Regenerate Python fixtures
`bash
python tests/fixtures/parity/generate_fixtures.py
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| bun test fails on NA | JS truthiness | Use Pine truth helpers |
| Parity script no-ops | Harness not wired | Implement test/parity` per plan |
| Drift from Python builtins | Manual port error | Re-run converter diff + fixtures |
See also
Converter
Testing
Runtime events
Roadmap
---
FILE: docs/pyne/pine-worker/testing.mdx
pine-worker testing
Abstract
Testing for pine-worker has two layers:
. Local Bun unit tests — AST/evaluator/series behavior in pure TS
. Cross-language parity — Python oracle generates JSON fixtures; TS must match
Until the parity harness is fully wired (bun run parity is a placeholder), treat Python fixtures as the contract and unit tests as constructive proofs of the port’s foundations.
Conceptual model
Interface surface
Commands
``bash
cd pine-worker
bun install
bun test
bun test --watch
bun run typecheck
bun run parity prints note until harness lands
`
Unit test files
| File | Focus |
| --- | --- |
| test/ast.test.ts | AST schema / types |
| test/evaluator.test.ts | Visitor eval, assign, literals |
| test/series.test.ts | Historical series indexing |
Tests use bun:test (describe / test / expect).
Parity fixtures (shared with Python)
| Path | Content |
| --- | --- |
| tests/fixtures/parity/pine/.pine | Minimal strategy scripts |
| tests/fixtures/parity/json/.json | Expected events / series snapshots |
| tests/fixtures/parity/generatefixtures.py | Oracle generator |
| tests/fixtures/parity/ohlcv.py | Shared bars |
Example scenarios: entry long/short, close, closeall, exit stop/limit, cancel, no-action, var counters.
Python-side consumers
Parity is also exercised from the main suite (e.g. strategy event tests). Regenerating fixtures:
`bash
python tests/fixtures/parity/generatefixtures.py
`
Internals
Evaluator test style
Tests hand-build AST node objects (plain dictionaries with type discriminators) rather than parsing Pine text — because the TS parser target is incomplete. That decouples evaluator progress from lexer work.
NA and control-flow signals
evaluator/types.ts defines NA, BreakSignal, ContinueSignal, ReturnSignal. Unit tests should cover:
arithmetic with NA → NA
comparisons with NA → false (Pine rule)
truthiness where NA is falsy (unlike some JS values)
Future parity harness
Intended location: pine-worker/test/parity/ with bun test test/parity --timeout . Design goal: load each JSON fixture, evaluate corresponding AST/script path, assert event lists match oracle.
Invariants & edge cases
. Never edit JSON fixtures by hand to make TS pass — regenerate from Python or fix Python if oracle is wrong.
. OHLCV must match between generators and consumers (ohlcv.py).
. Timeouts: strategy loops can be heavy; parity script reserves s.
. Typecheck is mandatory before claiming port milestones (tsc --noEmit).
Worked examples
Run a single unit file
`bash
cd pine-worker
bun test test/series.test.ts
`
Add a unit case for assignment
Follow patterns in evaluator.test.ts: build Script → Assign → Identifier load → expect.
Oracle refresh after Python strategy change
`bash
python tests/fixtures/parity/generatefixtures.py
git diff tests/fixtures/parity/json/
pytest tests/teststrategyevents.py -q
`
Failure modes
| Symptom | Cause | Fix |
| --- | --- | --- |
| Fixture mismatch after Python fix | Stale JSON | Regenerate |
| TS passes units, fails parity | Missing strategy builtins | Port + converter |
| Flaky floats | Compare with tolerances | Mirror Python assert style |
| parity script always “not wired” | Expected until harness lands | Implement under test/parity/` |
See also
pine-worker overview
Converter
Runtime events
Compatibility
---
FILE: docs/pyne/reference/compatibility.mdx
Compatibility guarantee
Abstract
PYNE aims for high syntactic fidelity and strong semantic compatibility with TradingView Pine Script v/v on the language core: parser, AST round-trip, type system, collections, technical builtins, and strategy primitives. Compatibility is not a claim of full platform parity (hosted chart, proprietary market data, editor-only features).
Source distillations: docs/compatibilityguarantee.md (v., --) and live test inventory. Metrics move as the suite grows — prefer CI and the implementation status matrix for current checkmarks.
Conceptual model
Interface surface
Compatibility metrics (summary)
| Category | Claim | Status posture |
| --- | --- | --- |
| Parser & syntax | Full grammar on real scripts (+ builtin corpus) | Solid |
| AST round-trip | Structural match on corpus | Verified |
| Builtins | Broad coverage (+ / dispatch hundreds) | Good → excellent by namespace |
| Technical indicators | Core + advanced; numerical cross-checks | Validated in tests / reports |
| Strategy | entry/exit/close, events, risk, OCA, commission | Functional / improved - |
| Type system | int/float/bool/string/color/series/UDT/collections | Strong |
| Real-world scripts | High parse success on corpus | Tested |
Historical guarantee doc cited automated tests at last update — re-run make test for live counts.
Language features (core)
Documented at % support posture for:
var / varip, type annotations, functions/methods
Control flow, UDTs, operators, string interpolation
Imports/libraries, v enum, switch expressions
Series history, array/matrix/map, tuple unpacking, method chaining
Builtins (guarantee lens)
| Family | Guarantee style |
| --- | --- |
| math. | IEEE-oriented precision; exact for discrete ops |
| ta. | Cross-validated vs reference implementations within tight relative error |
| str. / array. | Behavioral parity on exercised surface |
| Plotting | Signature + registry effects; rendering is external (AXIS / clients) |
| strategy. | Execution model in-process; not a licensed broker |
Internals
| Artifact | Role |
| --- | --- |
| docs/compatibilityguarantee.md | Long-form guarantee text |
| tests/data/builtinscripts/.pine | Parametrized corpus |
| tests/testparseandunparse.py | Round-trip |
| tests/testrealworldcompatibility.py | Real-script posture |
| tests/fixtures/parity/ | Cross-impl strategy events |
| reports/compatibilityreport.json | Machine-readable reports if generated |
Invariants & edge cases
. Parse success ≠ evaluate success. A script may round-trip yet depend on mock request. data.
. Plot functions are not TradingView’s renderer. They validate and record; pixels are optional.
. Undocumented / beta TV syntax may fail (guarantee doc notes a v-beta edge).
. Floating-point parity is statistical, not bit-identical for all recursive smoothers — see Numerical validation.
. Library export / import require in-process registration paths when not on TV’s CDN.
Worked examples
Round-trip check
``python
from pynescript.ast.helper import parse, unparse
src = open("strategy.pine").read()
assert unparse(parse(src)) structural fidelity; exact whitespace may differ
`
Corpus pytest (narrow)
`bash
pytest tests/testparseandunparse.py -q --example-scripts-dir=tests/data/builtinscripts
`
Failure modes
| Observation | Interpretation |
| --- | --- |
| Parse error on new TV release notes feature | Surface gap — see Missing features |
| Numerical drift on custom indicator | Compare bar-mode vs list-mode; check na` propagation |
| Strategy equity mismatch vs TV | Commission/slippage/fill model differences; seed data |
| Import resolution fails | Library not registered in runtime |
See also
Implementation status
Missing features
Pine v surface inventory
Numerical validation
---
FILE: docs/pyne/reference/implementation-status.mdx
Implementation status
Abstract
This annex summarizes the feature matrix maintained in docs/pinescriptimplementationstatus.md: a large // inventory of series variables, strategy fields, and related surfaces. It is the human-readable twin of the full surface inventory (dispatch-oriented, auto-countable).
Status here is implementation judgment, not marketing completion percentage.
Conceptual model
Interface surface
Legend
| Mark | Meaning |
| --- | --- |
| | Implemented and usable |
| | Partial / stub / mock |
| | Missing |
| ️ | By design out of scope |
| | Editor/platform N/A |
Component rollup
| Component | Posture (- consolidation) |
| --- | --- |
| Parser (ANTLR) | Complete for v/v core + recent multiline / export work |
| Evaluator | Full bar-loop with var/varip, reassignment |
| Builtins | + historical count; callables on live dispatch inventory (--) |
| TA | + ta. keys in inventory |
| Collections | array / matrix / map complete on exercised surface |
| Strategy events | StrategyEvent, parity corpus, risk enforcement |
| Drawing / plot | Registry effects; AXIS is external |
| Linter | Present (pynescript lint) |
| Data providers | Mock, Yahoo, AlphaVantage, CCXT + datafeed wiring |
| LSP | Diagnostics, completion, hover, format, symbols, defs/refs, … |
| pine-worker | Colocated TS port (skeleton → growing) |
| Numba compile path | MVP + object-mode fallback |
Series & context (illustrative families)
Fully listed in the source doc; representative groups:
Price: open/high/low/close/volume/hl/ohlc/…
Time: year…second, timeclose, timenow, …
Barstate / chart: barindex, barstate., lastbar
syminfo. ticker, mintick, session, fundamentals fields tracked as
session. / dividends. / earnings. context fields tracked as
strategy. position, trades, equity, risk, OCA, commission constants
July enhancements called out in source
Full strategy event emission with bar context
strategy.long / strategy.short constants
var / varip + := reassignment
Parity fixtures for cross-implementation validation
Risk helpers and performance series expansion
Internals
| Path | Role |
| --- | --- |
| docs/pinescriptimplementationstatus.md | Exhaustive checklist (do not paste wholesale here) |
| docs/pinevfullsurfaceinventory.md | Dispatch-centric inventory |
| src/pynescript/ast/evaluator/builtins/ | Per-namespace implementations |
| scripts/regenerateinventorysummary.py | Inventory regeneration aid |
Invariants & edge cases
. Checklist lag: a may trail a commit by a day; prefer tests when shipping.
. Context fields may be defaults rather than live exchange feeds.
. Partial stubs still appear callable — inventory marks when docstrings/heuristics say mock.
. Namespace counts ≠ quality. Many drawing setters are thin mutators; TA is heavier.
Worked examples
Smoke a strategy series field
``python
from pynescript.ast.helper import parse
evaluate via Runtime / library API with OHLCV — see runtime docs
`
Regenerate inventory summary
`bash
python scripts/regenerateinventorysummary.py
``
Failure modes
| Symptom | Action |
| --- | --- |
| Doc says , runtime AttributeError | File bug; fix dispatcher or demote status |
| Partial mock accepted silently | Assert data_feed wiring in tests |
| Status doc conflicts inventory | Prefer live dispatch + tests |
See also
Compatibility
Missing features
Pine v surface
Roadmap
---
FILE: docs/pyne/reference/missing-features.mdx
Missing features
Abstract
“Missing” in PYNE is a moving frontier. As of --, core v support remains very high (~%+ for core language), with post-launch TV items closed (multiline strings, library export const, footprint mocks, risk enforcement, matrix LA, compile-mode strategy, …) plus open-source corpus parse/Runtime hardening and incremental bar-mode TA for hot indicators. Remaining gaps cluster into by-design platform differences, editor-only features, fidelity edges, and a long-tail Runtime/corpus bucket (truncated scrapes, timeouts, exotic arity).
Authoritative narrative: docs/missingfeatures.md (last updated --).
Conceptual model
Interface surface
Recently completed (do not re-open as “missing”)
Summarized — full prose in source doc:
| Area | Notes |
| --- | --- |
| Multiline """ / ''' | Lexer + LexerBase + unparser |
| Library export const + types/enums | Parse/AST/runtime registry |
| UDT sortfield on array/matrix sort | Implemented |
| input..active | Metadata-driven |
| Dynamic request. | Shared resolvers; mock/feed scaling |
| Enums runtime + input.enum | Type kind + LSP partial |
| Strategy open/closed trades, risk, OCA, commission | Golden tests |
| Drawing .all, plot registry | Effects recorded |
| Numba compile + object mode | See compiler docs |
| Official matrix LA & predicates | det/inv/eigen/… |
| missing vs public TV v function reference list ( symbols, --) |
| Corpus sanitize + soft keywords / bitwise / typed UDF returns | set– parse ~.% |
| Runtime pinedefslocked + append-only currentseries | Host hygiene (backend + pyne-worker) |
| Incremental ta.sma/ema/rma/rsi/macd/atr | Bar mode; PYNETAINCREMENTAL= to disable |
Still incomplete / by design
| Item | Status posture |
| --- | --- |
| Real (non-mock) request. market data | ️ By design for library core; feeds optional |
| Pixel chart host | External (AXIS / clients) |
| Some unlimited-history / trim edge cases | High-level support; exotic TV behaviors may differ |
| Editor word-wrap / UI-only release notes | N/A |
| pine-worker full builtin port | In progress (extra tool) |
| Bit-identical every recursive smoother vs TV | Numerical bounds, not bits |
| Corpus Runtime tail (TIMEOUT / RUNFAIL / stubs) | ️ Execution + scrape quality, not missing grammar |
| Incremental for remaining heavy kernels | Partial — hot path done; BB/nested internal full paths remain |
| Dual Runtime host copies | Unify backend vs pyne-worker still open |
| TV Wilder RMA-ATR (if re-baselined) | ️ Current oracle uses EMA-of-TR; change is correctness, not silent perf |
Post-v launch themes (historical gap list)
v launch items (dynamic requests, strict bool, enums, polylines, log., negative indices, truediv, …) are largely supported or intentionally stubbed where platform-bound.
– monthly TV updates introduced footprint types, active inputs, multiline strings, export const, UDT sort fields — tracked and largely closed in this repo’s July work.
Corpus + performance snapshot (--)
| Metric | Value |
| --- | --- |
| set– parse | ~.% OK |
| set– Runtime (projected after fail re-runs) | ~.% OK (≈/) |
| PARSEFAIL residual | ~ files — almost all truncated/non-Pine |
| tasma bar loop (≈.k bars) | ~.× vs full recompute |
| taatr bar loop | ~× |
| macd+atr+rsi+sma combo | ~.× |
Plans: .opencode/plans/---runtime-performance.md. Skill: .grok/skills/pynescript-perf/. Golden: tests/testtaincremental.py.
Internals
| Path | Role |
| --- | --- |
| docs/missingfeatures.md | Living gap analysis |
| tests/testvfeatures.py | v feature coverage |
| tests/testpinesurfacegaps.py | Surface gap probes |
| tests/testtaincremental.py | Incremental TA ≡ full recompute |
| docs/pinevfullsurfaceinventory.md | Full name-level inventory |
| resource/.g + builder | Grammar-level feature work |
| .../technicalsubmodules/core.py | smaincupdate, macdincupdate, atrincupdate, … |
Invariants & edge cases
. Closing a syntax gap requires grammar + builder + unparser + tests, not only an evaluator stub.
. Mock data can green tests while production feeds differ — declare data source explicitly.
. “ missing vs reference list” ≠ identical semantics for every edge case.
. Perf changes must keep bar-by-bar semantics — no whole-script vectorization; prefer golden tests vs current oracle.
. Update this annex when missingfeatures.md changes major status lines.
Worked examples
Detect multiline string support
``python
from pynescript.ast.helper import parse, unparse
src = '''//@version=
indicator("m")
s = """a
b"""
plot()
'''
tree = parse(src)
assert '"""' in unparse(tree) or "'''" in unparse(tree)
`
Footprint mock call
Prefer tests in tests/testvfeatures.py as executable specification for request.footprint methods.
Incremental TA parity smoke
`bash
.venv/bin/python -m pytest tests/testtaincremental.py -q
Disable hot path if needed:
PYNETAINCREMENTAL= python -c "from backend.runtime import Runtime; ..."
`
Failure modes
| Symptom | Likely gap class |
| --- | --- |
| Lexer error on triple quotes | Stale generated lexer (should be fixed on main) |
| Import member missing | Library export registration |
| Empty footprint rows | Mock seed / datafeed not configured |
| Risk not blocking entry | Old runtime path; ensure risk kwargs applied |
| Runtime TIMEOUT on method-heavy scripts | Missing pinedefslocked on host (fixed --) |
| Corpus PARSEFAIL mid-block | Truncated scrape — not a grammar hole |
See also
Implementation status
Pine v surface
Roadmap
Compatibility
Technical analysis (ta.`)
---
FILE: docs/pyne/reference/numerical-validation.mdx
Numerical validation
Abstract
PYNE’s technical and mathematical builtins are validated for numerical agreement with reference Pine Script implementations under IEEE realities. The long-form report (docs/numericalvalidationreport.md, v., Nov ) documents methodology, per-family error tables, and edge cases.
Headline (from that report): max observed relative error .% (ADX, extreme volatility); average error &lt; .%; deterministic ops (SMA, OBV, discrete math) often exact.
Treat percentages as report-era measurements, not eternal SLAs — re-validate after changing smoothers or bar-mode semantics.
Conceptual model
Interface surface
Acceptable thresholds (report)
| Band | Relative error | Verdict |
| --- | --- | --- |
| Excellent | &lt; .% | Pass |
| Good | &lt; .% | Pass |
| Acceptable | &lt; .% | Review |
| Unacceptable | ≥ .% | Fail |
Metric definitions
Absolute error: |ref − pyne|
Relative error: |ref − pyne| / |ref| × % (guard zero refs)
Max / mean / std over bars and scenarios
Category highlights
| Family | Notes |
| --- | --- |
| Moving averages | SMA exact; EMA/WMA/… tiny FP error |
| Oscillators | RSI/stoch slightly higher smoothing error still ≪ .% |
| Trend | ADX highest in report (.% max) |
| Volume | OBV exact cumulative; MFI inherits typical-price FP |
| Stats | percentrank exact; variance/stdev excellent |
| Math | Discrete exact; transcendentals ~e- class |
Error distribution (report aggregate)
| Error range | Share (approx.) |
| --- | --- |
| Exact | ~% |
| &lt; .% | ~% |
| .–.% | ~% |
| .–.% | ~% |
| ≥ .% | % |
Internals
| Path | Role |
| --- | --- |
| docs/numericalvalidationreport.md | Full tables + bias analysis |
| tests/testtaindicators.py | Executable TA tests |
| tests/testindicators.py / builtins tests | Additional numeric guards |
| Bar-mode vs list-mode (pinebarmode) | Scalar-per-bar vs full series — compare like with like |
Methodology (report)
. Generate synthetic k-bar OHLCV across regimes (trend, range, vol, gaps)
. Validate with real histories (equities, crypto, FX samples)
. Export reference outputs; run PYNE; element-wise compare
. Search systematic bias; inspect tails
Invariants & edge cases
. na propagation must match Pine truthiness — numerical compare should mask na pairs.
. Bar-mode scalars vs full-series list mode can look like “bugs” if misaligned.
. Seeded mocks (request.seed) stabilize stochastic feeds for regression.
. Recursive smoothers accumulate FP differently across languages; bounds &gt; bits.
. Extreme prices (near-zero, huge) tested; still within excellent band in report.
Worked examples
Relative error helper
``python
def relerr(a: float, b: float) -> float:
if a == and b == :
return .
denom = abs(a) if a != else abs(b)
return abs(a - b) / denom .
`
Run TA tests
`bash
pytest tests/testtaindicators.py tests/testtaindicators.py -q
`
Failure modes
| Symptom | Investigation |
| --- | --- |
| Sudden ADX drift | Check Wilder smoothing / seed bars |
| SMA not exact | Off-by-one window or float input casts |
| Good unit tests, bad TV export compare | Timezone/session alignment of bars |
| Only bar differs | Warm-up / na` initialization |
See also
Compatibility
Runtime technical builtins
Implementation status
---
FILE: docs/pyne/reference/pine-v6-surface.mdx
Pine v surface inventory
Abstract
The full surface inventory enumerates every registered evaluator builtin, major series/variable, language construct, and known gap against Pine Script v. It is deliberately large (tens of thousands of tokens when fully expanded). This page is the navigable summary; the exhaustive tables live in-repo at:
docs/pinevfullsurfaceinventory.md
Generated snapshot referenced here: --, from live NodeLiteralEvaluator dispatch, builtinmetadata.json, and design docs.
Do not paste the entire inventory into product docs — regenerate and link.
Conceptual model
Interface surface
Status schema
| Status | Definition |
| --- | --- |
| implemented | Handler/parser path exists and is usable |
| partial | Mock/stub, incomplete semantics, or metadata-only |
| missing | Expected on TV v; not resolving |
| ️ by design | Intentionally not full TV platform |
| N/A | Editor/UI-only |
Each inventory row also carries: Name, Namespace, Kind, Metadata (LSP JSON present?), Source (e.g. dispatch), Notes.
Summary counts (--)
| Metric | Count |
| ---: | ---: |
| Dispatch builtins (callable) | |
| Dispatch partial-heuristic (stub/mock docstring) | |
| Top-level namespaces | |
| Official TV v function reference list | symbols — missing in dispatch |
Top namespaces by dispatch keys
| Namespace | Count |
| --- | ---: |
| ta | |
| matrix | |
| strategy | |
| array | |
| box | |
| label | |
| table | |
| math | |
| line | |
| str | |
| input | |
| map | |
| request | |
| footprint | |
| ticker | |
| (remaining namespaces) | smaller |
Kinds covered
function · series/var · constant · declaration · control · operator · type system · literal · semantics · export · import
Internals
| Path | Role |
| --- | --- |
| docs/pinevfullsurfaceinventory.md | Full tables + architecture diagrams |
| scripts/regenerateinventorysummary.py | Regeneration helper |
| scripts/generatebuiltinmetadata.py | LSP metadata from live surface |
| Evaluator buildbuiltinmap() | Source of dispatch truth |
How to use the full file
. Open docs/pinevfullsurfaceinventory.md in the repo or Sphinx/site mirror.
. Search by namespace ( ta, strategy, …).
. Trust dispatch source rows for “is there a callable?”; read Notes for mock/feed caveats.
. After large builtin PRs, regenerate summary counts before quoting them publicly.
Invariants & edge cases
. Inventory size is a feature — summarization must not drop rows when claiming coverage.
. Metadata presence ≠ semantic completeness.
. Partial heuristic count () is docstring-based — real partials may be higher.
. Language constructs (control/export) may be parser-complete without every runtime edge.
Worked examples
Quote coverage honestly
“Against the public Pine v function reference ( symbols), pynescript registered missing dispatch entries as of --; total callables across namespaces. See docs/pinevfullsurfaceinventory.md.”
Programmatic check (sketch)
``bash
regenerate after evaluator changes
python scripts/regenerateinventorysummary.py
``
Failure modes
| Mistake | Consequence |
| --- | --- |
| Pasting KB inventory into Mintlify page | Unreadable docs; broken UX |
| Citing counts without date | Stale marketing |
| Equating dispatch hit with TV gold | False precision claims |
See also
Implementation status
Missing features
Compatibility
Runtime builtins
---
FILE: docs/pyne/reference/roadmap.mdx
Roadmap
Abstract
The roadmap is a living prioritization document, not a contract. Source: docs/ROADMAP.md (updated -- consolidation) plus subsequent surface work. Many historically “remaining” items — Jupyter, linter, data providers, strategy events, pine-worker skeleton, deep strategy runtime — are done. What remains clusters into performance, packaging polish, and ecosystem integration.
Conceptual model
Interface surface
Status snapshot
| Component | Status |
| --- | --- |
| Parser ANTLR | |
| Evaluator | incl. var/varip / reassign |
| Builtins / TA / collections | broad |
| Strategy events + parity | |
| pine-worker (TS + converter) | colocated tool (port ongoing) |
| Drawing/input/request | (plots registry; data mock/feed) |
| Linter / Jupyter / data providers | |
| LSP advanced features | core set |
| Numba compile path | MVP |
| Full TV platform identity | ️ out of scope |
Completed themes (do not re-plan as greenfield)
Error message / typing hygiene
%%pinescript Jupyter magic + sample data helpers
pynescript lint
Yahoo / AlphaVantage / CCXT + realtime datafeed wiring
StrategyEvent emission + parity fixtures
pine-worker extra tool + Python→TS converter
Remaining high-value work
Phase A — Performance
JIT / deeper numba coverage beyond MVP paths
Vectorized array hot loops
Caching for repeated calculations
Phase B — Developer experience
Deeper debugging / profiling tools
Docs: tutorials, interactive examples (this Mintlify tree is part of that)
Editor client polish beyond VS Code
Phase C — Integration
Hardened REST API productization (auth, SLOs)
Webhook alerts beyond stubs
Cloud deploy recipes (see DevOps)
Phase D — Longer horizon
Pine v v converter maturity (scripts/convertpineversion.py started)
Automatic refactor / optimization suggestions
Parallel / distributed evaluation experiments
Priority recommendation (from source)
| Horizon | Focus |
| --- | --- |
| Short | API server productization, LSP/editor depth |
| Medium | Performance (JIT, cache, vectorize), advanced stats/ML wrappers |
| Long | Converters, parallel execution |
Internals
| Path | Role |
| --- | --- |
| docs/ROADMAP.md | Canonical roadmap prose |
| docs/PROGRESSREPORT.md | Historical completion narrative |
| docs/COMPILERPLAN.md | Compile/numba plan |
| .opencode/plans/ | Engineering plans (strategy events, consolidation) |
| pine-worker/ | TS port track |
Invariants & edge cases
. Roadmap dates lag code — always diff against missingfeatures.md and tests.
. Sister repos (pyne-worker, hoox-setup, pine-worker packaging) may absorb “integration” items.
. AXIS is a separate product tree (docs/axis) — chart UX roadmap is not PYNE-core.
. Avoid reintroducing deleted backups (builder.py.bak, etc.) as “plan items.”
Worked examples
Track a roadmap item with a test
``bash
pick a gap, write failing test first, implement, update missingfeatures.md
pytest tests/testvfeatures.py -k footprint -v
`
pine-worker maturity gate
`bash
cd pine-worker && bun test && bun run typecheck
future: bun run parity once harness lands
``
Failure modes
| Anti-pattern | Why it hurts |
| --- | --- |
| Planning features already | Duplicated work |
| Roadmap without tests | Unverifiable “done” |
| Ignoring by-design limits | Infinite platform chase |
See also
Missing features
Implementation status
pine-worker
Contributing
---
FILE: docs/pyne/runtime/builtins/collections.mdx
Collections
Abstract
Pine collections—arrays, matrices, and maps—are mutable values that live in the evaluator context (often under var so identity persists across bars). PYNE maps them to Python list, a dedicated Matrix type, and a Map / dict-like type respectively, with builtin functions and instance-method sugar (a.push(x) → array.push(a, x)).
Conceptual model
Interface surface
Arrays (array.)
Represented as Python lists. Highlights from the dispatch map:
| Category | Functions |
| --- | --- |
| Construct | array.new, array.new, array.from |
| Mutate | push, pop, shift, unshift (if present), insert, remove, set, fill, clear, reverse, sort |
| Access | get, first, last, size, includes, indexof, lastindexof, slice |
| Aggregate | sum, avg, min, max, median, mode, range, stdev, variance, covariance, percentiles, percentrank |
| Search | binarysearch, binarysearchleftmost, binarysearchrightmost |
| Higher-order | every, some, copy, concat, join, abs |
Indexing: Pine array indices are -based from the start of the list (unlike series history). Negative indices are rejected for series; array paths use list semantics consistent with the builtin handlers.
Series-like values passed into array ops unwrap via .history (reversed to chronological) when duck-typed.
Matrices (matrix.)
Matrix supports:
Construction: matrix.new
Element access: matrix.get / set, and m[row, col] via subscript
Shape: rows, columns, elementscount
Row/column ops: add/remove/copy, sum/avg/min/max/mode, fill
Aggregates: sum/avg/min/max/mode all
Transforms: transpose, fill diagonal, and further linear helpers in the evaluator map
Method sugar: m.rows() → matrix.rows(m).
Maps (map.)
| Function | Role |
| --- | --- |
| map.new | Empty map |
| put / putall / get / remove / clear | Mutation |
| contains / keys / values / size / copy | Query |
Keys follow Pine map rules as implemented (hashable Python keys). Compile object-mode lowers maps to ordinary Python dicts.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/builtins/arrays.py | Array builtins |
| src/pynescript/ast/evaluator/builtins/matrix.py | Matrix type |
| src/pynescript/ast/evaluator/builtins/matrixevaluator.py | matrix. handlers |
| src/pynescript/ast/evaluator/builtins/map.py | Map type |
| src/pynescript/ast/evaluator/builtins/mapevaluator.py | map. handlers |
| src/pynescript/ast/evaluator/names.py | Method markers for list/Matrix |
| tests/testcollections.py, testmapcollections.py, testmatrix.py | Coverage |
Multi-dispatch type tags (array.float, matrix.string, …) interact with method overloading and na exclusion lists so na-typed optionals do not bind to collection tostring overloads.
Invariants & edge cases
. Identity under var. var a = array.newfloat() keeps one list across bars; reassigning a := array.newfloat() replaces it.
. Drawing-typed arrays. array.newlabel etc. store drawing object references; delete semantics remain drawing-registry concerns.
. Empty array overload match. Empty arrays match any array. element tag in dispatch (no sample element).
. Matrix unpack hazard. StatementEvaluator refuses to treat matrices as general iterables for tuple unpack—prevents corrupting multi-assign from row iteration.
. Compile mode. Maps/UDTs force object mode; pure array numeric work may still numeric-compile depending on visitor detection—verify generated code when performance-critical.
Worked examples
Rolling buffer
``pine
//@version=
indicator("buf")
var floats = array.new_float()
array.push(floats, close)
if array.size(floats) >
array.shift(floats)
plot(array.avg(floats))
`
Matrix element
`pine
var m = matrix.new(, , .)
matrix.set(m, , , close)
plot(matrix.get(m, , ))
`
Map counter
`pine
var counts = map.new()
map.put(counts, "n", nz(map.get(counts, "n")) + )
`
Failure modes
| Symptom | Cause |
| --- | --- |
| map.get requires map and key | Wrong arity or non-Map receiver |
| Index errors | OOB get/set without Pine na` guard in that handler |
| Method not found | Typo or non-list receiver after series unwrap failed |
| Shared mutation across runs | Reused evaluator context without reset |
See also
Expressions & statements
Drawing & plotting — arrays of labels/lines
Compiler object mode
---
FILE: docs/pyne/runtime/builtins/drawing-plotting.mdx
Drawing and plotting
Abstract
Visual effects in Pine are side channels: they do not change pure arithmetic results, but they are first-class runtime outputs. PYNE records plots in PlotRegistry and drawing primitives (lines, boxes, labels, tables, polylines, linefills) in DrawingRegistry, then serializes them for the Pro API and optional AXIS. Evaluation never requires a UI—tests assert on registries alone.
Conceptual model
Interface surface
Plotting (PlottingFunctionsMixin)
| Function | Kind | Notes |
| --- | --- | --- |
| plot | plot | Returns Plot id for fill(plot, plot) |
| plotshape / plotchar / plotarrow | markers | location, char, text metadata |
| plotbar / plotcandle | OHLC visuals | open/high/low/close fields |
| hline | horizontal | price level |
| bgcolor / barcolor | coloring | series-driven colors |
| fill | fill | references two plot ids |
Style constants: plot.linestylesolid / dashed / dotted.
Each call constructs a Plot dataclass (kind, series, title, color, linewidth, text formatting, forceoverlay, …) and appends it to PlotRegistry.plots.
Hosts often also capture per-bar plot values on the evaluator (plotoutputs) for time-aligned series maps—see backend/runtime.py.
Drawing objects (DrawingBuiltinsMixin)
Object types: Line, Box, Label, Table, Polyline, LineFill, ChartPoint.
Typical factories: line.new, box.new, label.new, table.new, polyline.new, linefill.new, plus getters/setters/deletes and .all / last-bar helpers where implemented.
Instance methods resolve through namespace markers:
``text
la.gettext() → label.gettext(la)
`
DrawingRegistry.exportforapi(bartimes) maps xloc=barindex coordinates to wall times for chart clients, normalizes colors, and skips deleted objects.
Registry lifecycle
`python
DrawingRegistry.reset() clears drawings + PlotRegistry
`
Hosts must reset at run start so labels from a previous evaluation do not leak.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/builtins/plotting.py | Plot, PlotRegistry, plot handlers |
| src/pynescript/ast/evaluator/builtins/drawing.py | Drawing types, registry, builtins, export |
| src/pynescript/ast/evaluator/names.py | DRAWINGMETHODNS |
| tests/testplottingeffects.py, testdrawingallandlastbar.py | Behavior |
| Compiler object mode | Accumulates drawings event list instead of full registry parity |
Invariants & edge cases
. plot returns an id. Required for fill; treating it as “void” breaks band fills.
. Deleted flag. Soft-delete keeps history for debugging; active() filters deleted plots.
. Series in drawings. Prices may be series wrappers—export coerces .current and maps NaN → omit.
. Compile path. Numeric mode supports plot arrays; drawings force object mode with a simplified event list—not full delete/style parity with the interpreter.
. No GPU/canvas in-process. Registries are data; AXIS/HOOX render elsewhere.
Worked examples
Dual plot + fill
`pine
//@version=
indicator("BB mid")
basis = ta.sma(close, )
p = plot(basis)
p = plot(basis .)
fill(p, p, color=color.new(color.blue, ))
`
Label on last bar
`pine
//@version=
indicator("lbl", overlay=true)
if barstate.islast
label.new(barindex, high, str.tostring(close))
`
Hosts read DrawingRegistry.labels or API export after the run.
Failure modes
| Symptom | Cause |
| --- | --- |
| Empty drawings in API | Forgot DrawingRegistry.reset timing / export; or script never called .new |
| fill no-ops | plot ids not retained from plot() returns |
| Stale labels across HTTP runs | Missing registry reset between Runtime.run` calls (Runtime resets—custom hosts must) |
| Object mode missing delete | Known compile limitation—use interpret mode |
See also
Builtins hub
Pro API preview / chart renderer
AXIS docs
Compiler overview
---
FILE: docs/pyne/runtime/builtins/index.mdx
Builtins
Abstract
Pine’s standard library is not a single module: it is a flat qualified-name dispatch table ("ta.sma", "strategy.entry", "array.push", …) assembled from category mixins. Each entry is a handler (args, kwargs?) → value with side effects when the language requires them (orders, plots, drawings, inputs). This hub orients the surface; child pages go deep on technical, strategy, collections, drawing/plotting, and request/input.
Conceptual model
BuiltinEvaluator merges mixin maps in a fixed order and then registers ticker, logging, color, timeframe, and script-declaration functions. The strategy() declaration handler is wrapped so broker settings apply to StrategyState.
Interface surface
| Namespace / area | Mixin / module | Docs |
| --- | --- | --- |
| ta. | TechnicalAnalysisMixin + technicalsubmodules/ | Technical |
| strategy. | StrategyBuiltinsMixin, StrategyConstantsMixin | Strategy |
| array. / matrix. / map. | ArrayBuiltinsMixin, matrix/map evaluators | Collections |
| plot / hline / line / label / … | PlottingFunctionsMixin, DrawingBuiltinsMixin | Drawing & plotting |
| request. / input. | RequestBuiltinsMixin, InputBuiltinsMixin | Request & input |
| math., min/max, nz, … | NumericBuiltinsMixin | (this page) |
| str. | StringBuiltinsMixin | (this page) |
| color. | registercolorfunctions | (this page) |
| syminfo / ticker. | registertickerfunctions | (this page) |
| timeframe. | registertimeframefunctions | (this page) |
| alert / logging | AlertsMixin, registerloggingfunctions | (this page) |
| indicator / strategy / library | registerscriptdeclarationfunctions | Libraries |
| Footprint / volumerow | FootprintBuiltinsMixin | Request & input |
Numeric and string essentials
Numeric: math. constants from BaseEvaluator (math.pi, math.e, golden-ratio helpers); functions cover abs, min/max, rounding, logs, trig, and NA helpers (nz, na predicates via utility).
String: str. formatting, conversion (str.tostring respects format. constants), and manipulation aligned with common Pine scripts in the builtin corpus.
Color, ticker, timeframe
Colors: hex constants (color.red → F, etc.) plus composition helpers.
syminfo / ticker: host-provided Syminfo object or flat context keys; ticker construction for request.security symbols.
timeframe: period flags (isintraday, isdaily, …) default daily in base constants; hosts override from bar spacing.
Alerts
alert() / related helpers record structured messages with optional barindex / time from context—side channel for hosts, not strategy events.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/builtins/init.py | BuiltinEvaluator composition |
| src/pynescript/ast/evaluator/builtins/base.py | Dispatch mixin, callbuiltin, registration |
| src/pynescript/ast/evaluator/builtins/.py | Category implementations |
| scripts/generatebuiltinmetadata.py | LSP/metadata surface (not hand-edited JSON) |
Handlers are looked up by the fully qualified string built during name/attribute evaluation. Zero-arg series (e.g. strategy.equity) are registered as builtins that ignore empty args.
Invariants & edge cases
. One map, many mixins. New builtins require a map entry and (usually) metadata regeneration for LSP.
. Kwargs and positionals. Handlers accept both; Pine named arguments arrive as kwargs when the call AST carries them.
. Bar mode. Stateful ta.crossover / similar use per-bar call indices reset by the host (crosscalli).
. Mock fallbacks. request. without a feed still returns synthetic data so offline evaluation does not hard-fail—hosts must wire real providers for production accuracy.
. Compile path subset. Numba builtins reimplement only a fraction of this table; object mode covers more via Python helpers—see Compiler.
Worked example — resolution path
``pine
plot(ta.sma(close, ))
`
. Attribute/call chain builds qualified name ta.sma.
. callbuiltin("ta.sma", [closeseries, ]).
. Handler coerces series → list, computes SMA, finalizes scalar or list.
. plot registers a Plot with that series value and returns the plot id.
Failure modes
| Symptom | Fix |
| --- | --- |
| Unknown built-in function | Typo, version surface gap, or failed attribute chain |
| Always mock prices | Wire datafeed / dataprovider` into evaluator context |
| LSP knows a function runtime lacks | Metadata ahead of implementation—check missing features |
| Handler arity errors | Positional vs keyword mismatch in script |
See also
Technical
Strategy
Collections
Drawing & plotting
Request & input
Builtin metadata (LSP)
---
FILE: docs/pyne/runtime/builtins/request-input.mdx
request. and input.
Abstract
Two namespaces couple scripts to the host environment: input. declares parameters (with defaults and UI metadata), and request. pulls data from other symbols, timeframes, or fundamental/economic sources. PYNE implements both as builtins with explicit extension points—datafeed, dataprovider, and inputoverrides—so the same script can run offline with mocks or online with real market data.
Conceptual model
Interface surface
input. (InputBuiltinsMixin)
| Builtin | Purpose |
| --- | --- |
| input | Generic defval + title/tooltip/inline/group/confirm/active |
| input.bool / int / float / string / color | Typed scalars |
| input.price / source / time / timeframe / session / symbol | Domain inputs |
| input.enum / input.textarea | Enumerations and multiline text |
Runtime value: each call returns the resolved value (override if title matches, else default). Metadata is appended to inputdeclarations for settings panels and LSP-adjacent hosts.
Overrides live on the evaluator as inputoverrides: dict[title, value].
request. (RequestBuiltinsMixin)
| Builtin | Role |
| --- | --- |
| request.security | Other symbol / timeframe expression |
| request.securitylowertf | Lower-TF array expansion |
| request.dividends / earnings / splits | Corporate actions |
| request.financial / economic / quandl | Fundamentals / external series |
| request.currencyrate | FX conversion helper |
| request.seed | Deterministic pseudo-series |
| request.footprint | Volume footprint object (v surface) |
request.security(symbol, timeframe, expression, …)
Resolution order (simplified):
. Normalize symbol (series → last element) and timeframe.
. Try datafeed.fetchlatestohlcv / ticker.
. Try dataprovider.fetch.
. Fall back to mock synthetic prices so evaluation continues offline.
Expression may be a simple series name (close) or more complex forms depending on handler paths; production hosts should supply real data for multi-asset accuracy.
Footprint (FootprintBuiltinsMixin)
Types Footprint and VolumeRow expose buy/sell volume, delta, VAH/VAL/POC rows, and per-row imbalance helpers—aligned with the v footprint surface inventory.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/builtins/input.py | Input handlers + declarations |
| src/pynescript/ast/evaluator/builtins/request.py | request. + footprint types |
| src/pynescript/ast/evaluator/base.py | Injects datafeed / dataprovider into context |
| backend/runtime.py | resolverequestsources wiring from chart bars |
| tests/testrequestdatafeed.py, testdatafeed.py | Feed contracts |
Invariants & edge cases
. Inputs are pure values at runtime. Titles matter only for override keys and UI metadata—not for Pine type identity.
. Mocks are intentional. Silent mock fallback is not a hard error; production must validate feed wiring.
. Dynamic symbols. List/series symbols resolve to the latest element—supports loops constructing ticker ids.
. Lower TF. request.securitylowertf returns array-like structures; length scales with simulated lower-TF density when mocking.
. Compile mode. input. defaults lower into numeric compile; live request.security is interpret-centric—do not assume multi-asset compile coverage.
Worked examples
Parameterized length
``pine
//@version=
indicator("len")
len = input.int(, "Length", minval=)
plot(ta.sma(close, len))
`
Host:
`python
ev.inputoverrides = {"Length": }
`
Multi-timeframe close
`pine
htf = request.security(syminfo.tickerid, "D", close)
plot(htf)
`
With Runtime.run(..., data_provider=...), daily closes come from the provider; without it, mocks apply.
Failure modes
| Symptom | Cause |
| --- | --- |
| Always ~ mock prices | No feed/provider; or fetch exceptions swallowed |
| Input override ignored | Title string mismatch (including empty title) |
| Footprint fields zero | request.footprint` without configured footprint data |
| Lookahead surprises | Host data alignment / gaps—not automatic TV replay guarantees |
See also
Builtins hub
Pro API run endpoint
Runtime hub
---
FILE: docs/pyne/runtime/builtins/strategy.mdx
Strategy builtins
Abstract
PYNE’s strategy layer is a per-run broker simulation driven by strategy. calls during the bar loop. It is not a live exchange adapter: orders become fills against bar OHLC (market immediately; limit/stop via pending book), commissions and slippage adjust PnL, and risk helpers can block entries. Structured StrategyEvent records form the parity contract with pine-worker and the HOOX trade mesh.
Conceptual model
Fill-before-script matches the interpreter Runtime and the compile-path strategy broker.
Interface surface
Order placement
| Builtin | Role |
| --- | --- |
| strategy.entry | Open/add (or reverse) by direction; market or pending limit/stop |
| strategy.exit | Bracket-style exit (limit/stop) against position |
| strategy.close / strategy.closeall | Flatten by id or all |
| strategy.order | Lower-level order with OCA name/type, partial fill cap |
| strategy.cancel / strategy.cancelall | Remove pending orders |
Directions accept Pine constants (strategy.long / strategy.short) and common string aliases.
Position and performance series
Zero-arg builtins include strategy.positionsize (signed: +long / −short), positionavgprice, positionentryname, opentrades / closedtrades counts, netprofit, openprofit, equity, cash, gross win/loss stats, averages, max drawdown/runup, and max contracts held.
Trade queries
Indexed accessors:
strategy.closedtrades.entry / exit / profit / size / commission
strategy.opentrades.entry / size / profit / commission
Risk
| Builtin | Effect |
| --- | --- |
| strategy.risk.maxpositionsize | Cap size as % of equity |
| strategy.risk.maxintradayloss | Intraday loss gate |
| strategy.risk.maxintradayfilledorders | Order count cap |
| strategy.risk.maxdrawdown | Absolute / percent drawdown halt |
| strategy.risk.maxconslossdays | Consecutive losing calendar days → entriesblocked |
| strategy.risk.allowentryin | "all" \| "long" \| "short" |
Blocked entries still emit a diagnostic-style event with comment riskblocked where implemented.
Declaration
strategy(title, …) applies broker settings onto StrategyState: initialcapital, commission type/value, slippage ticks, pyramiding, etc.
Internals
StrategyState (strategy.py)
Per-evaluator instance fields include:
Position: direction (flat/long/short), size (non-negative internal), entry metadata
Books: pendingorders, opentrades, closedtrades
Economics: capital, commission model, slippage, mintick
Risk: max position %, drawdown, consecutive loss days, entriesblocked
Equity curve peak/trough for max DD / runup
events: list[StrategyEvent] drained each bar
signedpositionsize() implements Pine’s signed strategy.positionsize.
Order and fills
``text
Order: market | limit | stop | stop-limit
ocaname / ocatype ∈ {none, cancel, reduce}
maxfillperbar ( = fill remaining)
`
processpendingorders(open, high, low, close) walks the book, computes trigger prices from OHLC (gap-aware open logic for limits/stops), applies commission/slippage, updates trades, runs OCA side effects, and emits events.
OCA
| ocatype | On fill of a group member |
| --- | --- |
| cancel | Cancel other pending in group |
| reduce | Reduce sibling quantities by fill size |
| none | Independent |
Covered by tests/testocacommission.py and order-fill suites.
Invariants & edge cases
. Isolation. Never use class-level strategy state; concurrent runs must not share books.
. Pyramiding. Additional entries respect pyramiding from the declaration.
. Partial fills. maxfillperbar / defaults allow multi-bar completion of large orders.
. Calendar buckets for cons-loss. Exit timestamps in ms vs seconds vs bar index are normalized in noteclosedtradeday.
. Compile path. Object-mode uses CompileStrategyBroker—aligned fill rules, smaller surface than full interpreter metrics. Prefer interpret mode for full strategy.closedtrades. analytics unless compile coverage is verified.
Worked examples
Market long / close
`pine
//@version=
strategy("Demo", overlay=true, initialcapital=)
if barindex ==
strategy.entry("L", strategy.long, qty=)
if barindex ==
strategy.close("L")
`
Parity fixtures under tests/fixtures/parity/pine/ encode expected event sequences for such scripts.
Pending stop entry
`pine
strategy.entry("Break", strategy.long, stop=high[])
`
Creates a pending stop; subsequent bars’ processpendingorders fill when high trades through the stop, using open-gap-aware fill prices.
OCA bracket sketch
`pine
strategy.order("TP", strategy.short, qty=, limit=tp, ocaname="br", ocatype=strategy.oca.cancel)
strategy.order("SL", strategy.short, qty=, stop=sl, ocaname="br", ocatype=strategy.oca.cancel)
`
First fill cancels the sibling.
Failure modes
| Symptom | Cause |
| --- | --- |
| Entry never fills | Pending stop/limit; OHLC never trades through; or risk block |
| Double entries | Pyramiding > or missing close |
| Events missing scriptid | Host forgot to stamp after drainevents (Runtime does this) |
| PnL off vs TV | Commission type, slippage ticks, mintick, or fill price model |
| Interpret vs compile event drift | Broker subset / timing—diff with tests/testcompiler_strategy.py` |
See also
Events
Strategy broker (compile)
Parity / pine-worker
Pro API backtest
---
FILE: docs/pyne/runtime/builtins/technical.mdx
Technical analysis (ta.)
Abstract
The ta. namespace is the largest pure-compute surface in PYNE: moving averages, oscillators, volatility bands, volume studies, pattern helpers, and cross/rise/fall predicates. Implementations live under technicalsubmodules/ and are composed into TechnicalAnalysisMixin. Numerical behavior is validated against TradingView reference series (see numerical validation); the design goal is IEEE-limit parity, not “approximate TA.”
Conceptual model
Hosts in production set bar mode so indicator calls return scalars for the active bar, matching how Pine expressions compose (ta.ema(a) - ta.ema(b)).
Incremental hot path (--)
When pinebarmode and pinetaincremental are enabled (Runtime default; disable with PYNETAINCREMENTAL=), these builtins update call-site state once per bar instead of recomputing full history:
| Builtin | State model |
| --- | --- |
| ta.sma | Rolling window (deque) |
| ta.ema / ta.rma | Recursive seed + step |
| ta.rsi | Dual RMA of gains/losses |
| ta.macd | Fast/slow/signal EMAs in one slot |
| ta.atr | TR stream; mean warm-up then EMA-of-TR (matches current full oracle) |
Call sites are indexed like crossovers (tacalli reset each bar). Nested forms such as ta.ema(ta.sma(close, ), ) stay correct because each call keeps its own slot. Golden suite: tests/testtaincremental.py (incremental last values ≡ full recompute).
Interface surface
Dispatch keys (non-exhaustive; map is authoritative in technical.py):
| Family | Examples |
| --- | --- |
| Moving averages | ta.sma, ta.ema, ta.wma, ta.rma, ta.hma, ta.vwma, ta.swma, ta.alma |
| Oscillators | ta.rsi, ta.stoch, ta.cci, ta.cmo, ta.mfi, ta.roc, ta.wpr, ta.tsi |
| Trend / channels | ta.macd, ta.adx, ta.dmi, ta.supertrend, ta.sar, ta.linreg |
| Volatility | ta.atr, ta.tr, ta.bb, ta.bbw, ta.kc, ta.kcw, ta.stdev, ta.variance |
| Volume | ta.obv, ta.vwap, ta.mfi (shared), volume submodule helpers |
| Structure | ta.highest / lowest / highestbars / lowestbars, ta.pivothigh / pivotlow |
| Predicates | ta.crossover, ta.crossunder, ta.cross, ta.rising, ta.falling |
| Stats / other | ta.change, ta.mom, ta.cum, ta.median, ta.mode, percentiles, ta.barssince, ta.valuewhen, ta.correlation |
Argument conventions
Typical form: ta.fn(series, length).
Some functions allow period-only calls (e.g. ta.highest()) defaulting source to high / context series via expectseries(..., allowperiodonly=True).
Multi-value returns (e.g. MACD, BB, Supertrend) unpack as tuples/lists; assignment uses StatementEvaluator unpack rules.
Fractional lengths floor to int (TradingView-compatible).
Stateful crosses
ta.crossover / ta.crossunder need previous-bar pairs. In bar-mode runs the host resets a call-site index (crosscalli) each bar so multiple cross calls in one script keep independent state.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/builtins/technical.py | Dispatch map aggregation |
| .../technicalsubmodules/core.py | Series coerce, expect helpers, bar finalize, incremental kernels |
| .../movingaverages.py, oscillators.py, volatility.py, volume.py, … | Kernels + inc wiring |
| .../common.py, basic.py, advanced.py, patterns.py, strategies.py, synthesizer.py, economics.py | Additional families |
| tests/testtaindicators.py, tests/testindicators.py | Regression |
| tests/testtaincremental.py | Inc ≡ full recompute golden |
| docs/numericalvalidationreport.md | Published precision summary |
History for wrapper series is reversed to chronological order and may be truncated (SERIESMAX) before full kernels run. Incremental path only needs series[-] per call, so truncation does not freeze state.
Invariants & edge cases
. Warm-up → na. Insufficient bars (e.g. SMA length \(N\) needs \(N\) samples) yield None / leading na in full-series mode.
. NA in window. Many kernels propagate na if any window element is missing (mirrors Pine strictness for SMA-like sums).
. Default sources. ta.atr pulls high/low/close from currentseries / context when not passed explicitly.
. Compile path kernels differ. Numba numbasma / numbaema / numbarsi / highest/lowest are separate implementations—parity tests should cover both modes for scripts that compile.
. No chart look-ahead. Kernels only see history available at the current bar index; request.security lookahead is a separate concern.
. Incremental ≡ full recompute (oracle). Changing seed rules (e.g. ATR Wilder vs EMA-of-TR) is a correctness project, not a silent perf flag.
Worked examples
Classic overlay
``pine
//@version=
indicator("SMA ")
v = ta.sma(close, )
plot(v)
`
In bar mode each visit returns one float (or na); the host stacks them into a series for the response envelope.
Cross entry signal
`pine
//@version=
strategy("X")
fast = ta.ema(close, )
slow = ta.ema(close, )
if ta.crossover(fast, slow)
strategy.entry("L", strategy.long)
`
Cross state is bar-local; ensure the runtime resets cross call indices when reusing an evaluator.
Multi-value unpack
`pine
[macdLine, signal, hist] = ta.macd(close, , , )
plot(hist)
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Always na | Length longer than available history; or series not updated |
| Wrong vs TV by large margin | Wrong source series; mock OHLCV; or non-bar-mode list composition bug |
| Cross never fires | Call-index state not reset / shared incorrectly across bars |
| Compile diverges | Numba RSI/EMA seed conventions—check numbabuiltins.py` |
See also
Series & history
Strategy builtins
Numba builtins
Numerical validation
---
FILE: docs/pyne/runtime/compiler/numba.mdx
Numba path
Abstract
Numeric compile mode emits a single @numba.njit function whose body is the script’s bar loop. Pine builtins that cannot cross the JIT boundary are replaced by hand-written kernels in numbabuiltins.py that take full arrays plus the current index. The result is large speedups on multi-thousand-bar series after a one-time JIT warm-up—at the cost of a restricted language subset and np.nan as the missing-value sentinel.
Conceptual model
Interface surface
Kernels (all @numba.njit(cache=True) at definition site; generated entry uses cache=False):
| Kernel | Semantics sketch |
| --- | --- |
| numbasma(arr, period, i) | Mean of arr[i-period+ : i]; nan if warm-up or any window nan |
| numbaema(arr, period, i) | SMA seed over first period bars, then EMA to i |
| numbarsi(arr, period, i) | Average gain/loss form; if avg loss |
| numbahighest / numbalowest | Window extrema; window clamps at |
| numbanz(val, replacement) | Replace nan |
| numbaabs / numbamin / numbamax | Scalar helpers |
Generated code imports from pynescript.compiler.numbabuiltins import so call sites are bare names.
History access
close[n] lowers to array indexing at baridx - n with bounds → nan (parallel to interpreter None, different sentinel).
Control flow
if / for / while emit Python control flow inside the njit function—supported Numba subset only (no heap objects, no Python lists of mixed types).
Internals
| Path | Role |
| --- | --- |
| src/pynescript/compiler/numbabuiltins.py | Kernels |
| src/pynescript/compiler/compiler.py | emitnumericmode, call lowering |
| src/pynescript/compiler/engine.py | Requires HASNUMBA when not objectmode |
| tests/testcompilernumba.py | Correctness |
Expanding coverage is primarily new kernels + visitor call mapping, not changes to the bar-loop skeleton.
Invariants & edge cases
. No Python objects in numeric mode. Strings, UDTs, maps, drawings force object mode before njit is applied.
. Warm-up cost. First compilescript invokes a dummy run; production servers should cache CompiledScript.
. EMA/RSI definitions are explicit in source—document any intentional deviation when comparing to interpret ta. for edge seeds.
. cache=False on entry. Do not expect Numba’s on-disk cache to key generated functions across processes without further work.
. Anaconda static libpython. Packaging notes in AGENTS/build docs apply when shipping JIT-heavy binaries.
Worked examples
Benchmark mental model
``text
interpret: O(bars × astnodes × pythondispatch)
numeric: O(bars × loweredops) in machine code after JIT
`
For simple SMA scripts, internal notes cite order-of-magnitude speedups versus list-based interpretation on long series (see docs/COMPILERPLAN.md qualitative claims).
Using from Pro API
`python
Runtime(...).run(source, ohlcv, mode="compile")
`
Converts bar dicts → arrays, compiles, reshapes plots into the standard envelope, flags objectmode when applicable.
Failure modes
| Symptom | Cause |
| --- | --- |
| TypingError from Numba | Unsupported construct leaked into numeric emit |
| All-nan plots | Period longer than series; or history index bugs |
| ImportError numba | Environment missing dependency |
| Divergent RSI vs interpret | Seed / average method—file fixture and compare both paths |
| Object mode unexpectedly | Visitor detected drawing/UDT/map—inspect visitor.objectmode` |
Performance notes
Prefer one compile, many runs with different OHLCV.
Keep scripts in the numeric subset for realtime ticks.
Object mode still wins over AST walking for UDT-heavy scripts but will not match peak njit throughput.
See also
Compiler overview
Technical builtins (interpret)
Series & history
---
FILE: docs/pyne/runtime/compiler/overview.mdx
Compiler overview
Abstract
The compile path lowers Pine’s ASDL AST to executable Python that walks bars with an explicit index (_baridx) over contiguous numpy arrays—avoiding per-node visitor overhead. Pure numeric scripts wrap the loop in @numba.njit; scripts that touch UDTs, maps, or drawings select object mode (still a tight Python/numpy loop, no AST walk). The public façade is pynescript.compiler.engine (transpile, compilescript, runscript), also reachable as Runtime.run(..., mode="compile").
This is an MVP with growing surface—not a full substitute for the interpreter on every script.
Conceptual model
Interface surface
``python
from pynescript.compiler.engine import transpile, compilescript, runscript, hasnumba
src = transpile(pinetext) generated Python string
cs = compilescript(pinetext) CompiledScript
result = cs.run(open, high, low, close, volume)
result: dict of plot title → float array, optional drawings, events, …
`
| API | Role |
| --- | --- |
| transpile | Parse + visit → source string (inspect/debug) |
| compilescript | exec generated code, warm-up call, return CompiledScript |
| runscript | One-shot compile+run (prefer cache CompiledScript for batches) |
| hasnumba | Numeric mode prerequisite |
CompiledScript fields: source, generatedcode, execute, plottitles, objectmode.
Mode selection
CompilerVisitor sets objectmode = True when it sees:
User-defined types / enums
Map operations
Drawing APIs / certain plot variants
Strategy usage (object mode + CompileStrategyBroker)
Otherwise numeric mode requires Numba installed.
What numeric mode lowers today
Series assigns, arithmetic/logic, history (close[n]), if/for/while, selected ta. (sma, ema, rsi, highest, lowest), scalar math helpers, plot, input. defaults, var/varip carry—executed under @numba.njit(cache=False) (cache=False because code is exec’d from a string without a stable disk locator).
Object mode extras
UDT instances as field dicts
Maps as Python dicts
drawings list of structured events
Optional strategy broker: pending fills, position, events, equity snapshots
Internals
| Path | Role |
| --- | --- |
| src/pynescript/compiler/compiler.py | CompilerVisitor, emit numeric/object |
| src/pynescript/compiler/numbabuiltins.py | JIT kernels |
| src/pynescript/compiler/strategybroker.py | Compile broker |
| src/pynescript/compiler/engine.py | Façade + result normalize |
| backend/runtime.py | runcompiled envelope → plots/series |
| tests/testcompilernumba.py, testcompilerobjects.py, testcompilerstrategy.py | Coverage |
| docs/COMPILERPLAN.md | Design history and remaining work |
Data layout
Unlike interpreter PineSeries deques:
`text
openarr, higharr, lowarr, closearr, volarr inputs
userarr = np.full(nbars, np.nan) or dtype=object for UDT
ploti = np.full(nbars, np.nan)
for baridx in range(nbars):
userarr[_baridx] = …
ploti[baridx] = …
`
var lowers to “init only when still na / first write” patterns so carry matches Pine without declaration sets.
Invariants & edge cases
. Same AST front-end. No second parser—compile bugs are lowering bugs.
. OHLCV length equality enforced in CompiledScript.run.
. Warm-up ignores exceptions on a dummy -bar series (JIT or first-run).
. Result normalization converts Numba typed maps to plain dicts; _drawings stays a Python list.
. Interpret remains default. Full strategy analytics, request., and exotic builtins stay interpret-first until lowered.
Worked example
Pine:
`pine
//@version=
indicator("c")
s = ta.sma(close, )
plot(s, "sma")
`
Generated shape (illustrative):
`python
@numba.njit(cache=False)
def executescriptcompiled(openarr, higharr, lowarr, closearr, volarr):
nbars = len(closearr)
sarr = np.full(nbars, np.nan)
plot = np.full(nbars, np.nan)
for _baridx in range(nbars):
sarr[_baridx] = numbasma(closearr, , _baridx)
plot[baridx] = sarr[baridx]
return {'sma': plot}
`
Failure modes
| Symptom | Cause |
| --- | --- |
| numba is required for numeric compile mode | Install numba or simplify script into object mode triggers |
| Empty generated code | Visitor returned blank—unsupported top-level shape |
| Missing executescriptcompiled | Emit bug or failed exec |
| Silent semantic drift | Kernel seed differences—diff against interpret on fixtures |
| Strategy metrics missing | Use interpret or check events / broker fields only |
Remaining work (summary)
Expand Numba ta. surface, richer drawings, nested UDT methods, disk cache of generated modules, and systematic interpretcompile parity tests per lowered construct. See docs/COMPILERPLAN.md` for the living checklist.
See also
Numba path
Strategy broker
Runtime hub
Numerical validation
---
FILE: docs/pyne/runtime/compiler/strategy-broker.mdx
Compile strategy broker
Abstract
When CompilerVisitor detects strategy usage, object-mode emission instantiates CompileStrategyBroker—a lightweight broker aligned with the interpreter’s fill-before-script cadence, but designed for generated loops (no AST, no full StrategyState metric surface). It supports market entry/close and pending limit/stop/stop-limit orders with per-bar OHLC triggers, OCA cancel/reduce, commission, and slippage.
Conceptual model
Interface surface
Construction (from strategy() kwargs when lowered)
``python
CompileStrategyBroker(
initialcapital=.,
commissionvalue=.,
commissiontype="percent", percent | cashperorder | cashpercontract
slippageticks=,
mintick=.,
)
`
Bar context
`python
broker.setbar(barindex, bartime, mark, open=…, high=…, low=…, close=…)
broker.processpendingorders(open=…, high=…, low=…, close=…)
`
Orders
| Method | Role |
| --- | --- |
| entry(...) | Market or pending entry; reverse flattens opposite |
| close / closeall | Reduce/flat with optional price |
| Pending book | PendingOrder with type, direction, qty, limit/stop, OCA, maxfillperbar, isentry |
Fill logic (triggerprice)
| Type | Long trigger | Short trigger | Fill price idea |
| --- | --- | --- | --- |
| market | — | — | close (immediate path) |
| limit | low = limit | gap-aware vs open |
| stop | high >= stop | low <= stop | gap-aware vs open |
| stop-limit | stop traded and limit available | symmetric | limit price |
Partial fills: maxfillperbar caps quantity per bar; residual stays pending.
OCA
After a fill, siblings in the same ocaname:
cancel — remove sibling, emit cancel
reduce — decrease sibling qty by fill size
none — no interaction
Economics
Slippage: ± slippageticks mintick by direction on fills.
Commission: percent of notional, cash per order, or cash per contract.
Position: signed positionsize, positionavgprice, netprofit, equity helpers for return payload.
Events
emit appends dicts with keys compatible with strategy event serialization (kind, id, direction, qty, ordertype, limit, stop, ocaname, comment, barindex, bartime, ohlc). Object-mode return includes 'events': strategy.toevents() plus position/netprofit/equity snapshots.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/compiler/strategybroker.py | Broker implementation |
| src/pynescript/compiler/compiler.py | Emits _strategy wiring in object mode |
| src/pynescript/ast/evaluator/builtins/strategy.py | Interpreter counterpart (richer) |
| tests/testcompilerstrategy.py | Compile strategy coverage |
NA helpers treat None, blank/na strings, and float NaN as missing prices when classifying order types.
Invariants & edge cases
. Same order as interpreter: setbar → processpendingorders → script body.
. Not a full metric engine. Prefer interpret mode for strategy.closedtrades.profit(i) style analytics until parity is complete.
. Direction normalization accepts strategy.long, long, , buy, etc.
. Reverse on opposite entry via closeall(comment="reverse") then open.
. Event dicts are mutable lists—fine for run output; not frozen dataclasses.
Worked examples
Generated prologue (illustrative)
`python
strategy = CompileStrategyBroker(initialcapital=.)
for _baridx in range(nbars):
strategy.setbar(_baridx, , float(closearr[baridx]),
open=float(openarr[_baridx]), high=float(higharr[baridx]),
low=float(lowarr[baridx]), close=float(closearr[baridx]))
_strategy.processpendingorders(...)
lowered strategy.entry / close calls
return {..., 'events': strategy.toevents(),
'_positionsize': _strategy.positionsize,
'netprofit': strategy.netprofit,
'equity': strategy.equity}
`
Limit buy pending
`pine
strategy.entry("L", strategy.long, limit=low - syminfo.mintick)
`
Object-mode lowers to a pending limit; subsequent bars fill when low trades through.
Failure modes
| Symptom | Cause |
| --- | --- |
| No fills | Triggers never met; or market path not invoked |
| OCA sibling remains | ocatype none / name mismatch |
| PnL vs interpret drift | Commission/slippage/mintick defaults differ |
| Missing events in API envelope | Host only maps plots—not _events` |
See also
Strategy builtins (interpret)
Events
Compiler overview
Numba path
---
FILE: docs/pyne/runtime/events.mdx
Strategy events
Abstract
A strategy event is the immutable, structured record of one strategy. action (or fill side-effect) at a specific bar. Events are the wire format between PYNE’s Python evaluator, the TypeScript pine-worker port, and downstream HOOX trade workers. Changing the dataclass without updating the TS mirror and parity fixtures is a contract break.
Conceptual model
Interface surface
StrategyEvent fields
Defined in src/pynescript/ast/evaluator/events.py:
| Field | Type (logical) | Notes |
| --- | --- | --- |
| kind | entry \| exit \| close \| closeall \| cancel \| cancelall \| order | Discriminator |
| id | str \| None | Order / entry id |
| direction | long \| short \| None | |
| qty | float \| None | |
| ordertype | market \| limit \| stop \| None | Stop-limit may appear as composite handling elsewhere |
| limit / stop | float \| None | Prices when relevant |
| ocaname | str \| None | OCA group |
| comment | str \| None | Includes system comments (riskblocked, fill:…) |
| barindex | int | From context at emit |
| bartime | int | From context |
| ohlc | -tuple → list in JSON | Bar snapshot |
| scriptid | str | Host-stamped (source hash) |
| runid | str | Host-stamped run uuid |
Frozen dataclass → todict() always includes keys (JSON nulls for unspecified fields) so parity diffs are stable.
Emission sources
. Explicit builtins — strategy.entry, exit, close, closeall, cancel, cancelall, order push events as they execute.
. Broker fills — processpendingorders emits fill-related order / entry / cancel (OCA) events.
. Risk gates — blocked entries may emit with comment="riskblocked".
Host collection pattern
``python
evaluator.resetevents()
... processpendingorders + visit(tree)
for ev in evaluator.strategystate.drainevents():
d = ev.todict()
d["scriptid"] = scriptid
d["runid"] = runid
allevents.append(d)
`
Runtime.run implements this and returns events in the response envelope.
Compile path
CompileStrategyBroker accumulates plain dict events (events in the return mapping) with the same field names for kinds it supports—use for throughput; full kind coverage trails the interpreter.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/events.py | StrategyEvent |
| src/pynescript/ast/evaluator/builtins/strategy.py | Emit sites + drainevents |
| src/pynescript/compiler/strategybroker.py | Compile-mode emit |
| tests/teststrategyevents.py | Unit behavior |
| tests/testparity.py + tests/fixtures/parity/ | Cross-port oracle |
| pine-worker/src/evaluator/events.ts | TS twin |
Parity corpus scripts (strategyentrylong.pine, …) regenerate expected JSON via:
`bash
python tests/fixtures/parity/generatefixtures.py
`
Tests strip scriptid / runid before compare.
Invariants & edge cases
. Immutability. Do not mutate events after construction; drain copies and clears the buffer.
. Per-bar reset. resetevents / drain prevents cross-bar leakage in tests that share evaluators.
. OHLC list vs tuple. todict converts to list for JSON round-trips.
. Kind vocabulary is closed. New kinds require TS + fixtures + this doc.
. Not alerts. alert() uses a separate alerts channel—not StrategyEvent.
Worked examples
Expected single entry
Script enters long on a fixed bar; fixture JSON lists one event:
`json
{
"kind": "entry",
"id": "L",
"direction": "long",
"qty": .,
"ordertype": "market",
"barindex":
}
`
(plus nullables and ohlc—see real fixtures for full keys).
Cancel after OCA
Fill of one OCA sibling yields cancel events for others with comment reflecting ocacancel / ocareduce.
Failure modes
| Symptom | Cause |
| --- | --- |
| Empty events with live strategy | Forgot drain; or strategy never called |
| Parity flake on ids | Comparing scriptid/runid |
| TS/Python mismatch | Field rename without dual update |
| Duplicate events | Not clearing buffer; double processpending_orders` |
See also
Strategy builtins
Strategy broker
pine-worker
HOOX docs
---
FILE: docs/pyne/runtime/expressions-statements.mdx
Expressions and statements
Abstract
Evaluation is pure visitor dispatch: each ASDL node type maps to a visit method on a mixin. Expressions produce values (with Pine na and series rules); statements mutate context, register types/functions, or emit side effects via builtins. This page is the operational semantics of the interpreter path—what runs inside evaluator.visit(tree) each bar.
Conceptual model
Interface surface
Expressions (ExpressionEvaluator)
| Node | Behavior |
| --- | --- |
| BoolOp (and / or) | Short-circuit via Python all / any on visited values |
| BinOp | NA-safe + - / %; division by zero → na |
| UnaryOp | not, unary +/- with NA and list map |
| Compare | Chained comparisons with NA-safe operators |
| IfExp | Ternary: condition, then, else |
| Call | Builtin dispatch, user functions, methods, multi-dispatch overloads |
| Tuple / lists | Literal sequences for unpacking and multi-arg returns |
Binary ops coerce PineSeries-like objects to .current before operating. List operands broadcast or zip with trailing alignment (see Series & history).
Names (NameEvaluator)
| Node | Behavior |
| --- | --- |
| Name | Context lookup; bare series builtins (na, year, …); else string id (lazy) |
| Attribute | Qualified context keys, builtins (strategy.positionsize), library modules, UDT fields/methods, array/drawing/matrix method markers, Python getattr fallback |
| Subscript | Series history / array / matrix indexing |
Critical: qualified names like strategy.opentrades.entryprice are resolved via AST path construction (astqualifiedname) without evaluating intermediate zero-arg series that would collapse the path to an int.
Statements (StatementEvaluator)
| Construct | Behavior |
| --- | --- |
| Script | Walk body; finalize library exports if library(...) active |
| Assign | var/varip once-per-declaration; const; ordinary assign; tuple unpack; export registration |
| ReAssign (:=) | Update existing binding (and UDT fields) |
| If / For / While | Standard control flow; loop variables in context |
| FunctionDef | Store callable in context; multi-dispatch overload lists; export; methods tagged pinemethod_ |
| TypeDef / EnumDef | Type registry / enum member dicts |
| Import | Resolve LibraryRegistry → bind alias to LibraryModule |
| Declarations | indicator / strategy / library via declaration builtins |
Function invocation binds parameters into context (with restore of prior bindings on exit) so nested calls and bar-local state interact cleanly. After bar , hosts set pinedefslocked so re-visiting the script does not re-register overload tables.
Calls and multi-dispatch
Method markers returned from attribute evaluation:
| Marker | Meaning |
| --- | --- |
| ("methodcall", instance, name) | UDT method |
| ("arraymethod", list, name) | array.(self, …) |
| ("nsmethod", obj, "label.gettext") | Drawing/matrix namespace method |
| ("extmethod", receiver, name) | Free method with receiver as first arg |
Overloads with typed first parameters (e.g. matrix.float vs array.label) select via coarse type tags; na receivers deliberately exclude matrix/array/drawing tags so optional UDT fields do not mis-dispatch to tostring on collections.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/expressions.py | Ops, calls, conditionals |
| src/pynescript/ast/evaluator/names.py | Names, attributes, subscripts |
| src/pynescript/ast/evaluator/statements.py | Script structure, assign, defs, import |
| src/pynescript/ast/evaluator/literals.py | Constants, colors, strings |
| src/pynescript/ast/evaluator/base.py | Context, math constants, library registry hook |
| src/pynescript/ast/evaluator/types.py | EvaluatorProtocol typing for mixins |
Invariants & edge cases
. Context is the single store. Builtins read OHLCV and strategy series from self.context; hosts must keep it coherent each bar.
. String fallback names. An unbound Name returns its id string so later attribute/call resolution can still hit the builtin map ("ta.sma" patterns via attribute chains).
. Parameter unbind. Missing-before-bind params use a sentinel so unbind pops rather than leaving None ghosts.
. na normalization. String "na" / "nan" / "none" may coerce to None at dispatch boundaries for UDT optional args.
. User functions re-enter the visitor. Recursive and higher-order patterns are limited by Python stack, not a Pine-specific trampoline.
Worked examples
Ternary and comparison chain
``pine
//@version=
indicator("cmp")
v = close > open ? close - open : open - close
plot(v)
`
Each operator path is NA-safe: if close or open were missing, comparisons and arithmetic yield na.
User function with series
`pine
//@version=
indicator("fn")
f(x, n) =>
ta.sma(x, n)
plot(f(close, ))
`
f is stored as a Python callable that rebinds x/n and visits the function body AST each call.
Method multi-dispatch sketch
`pine
method log(this, string s) => ...
method log(this, float x) => ...
`
Both register under the same name with pineoverloads_; call sites pick by receiver/arg type tags.
Failure modes
| Error | Meaning |
| --- | --- |
| unexpected type of node | Missing visit for an AST construct |
| Unsupported binary operator | Op class not in the dispatch table |
| Negative indices not supported | Pine forbids series[-] Python style |
| Wrong overload / destroyed receiver | Historical na` string path; now normalized—report residual mismatches as bugs |
| Stack overflow | Deep recursion in user functions |
See also
Series & history
Builtins hub
Libraries
Type system (core)
---
FILE: docs/pyne/runtime/index.mdx
Runtime
Abstract
PYNE’s runtime is the deterministic bar-loop that turns a parsed ASDL AST into series values, drawing side-effects, and strategy events. It is deliberately host-agnostic: the same visitor mixins power the library API, the Flask Pro API (Runtime.run), browser Pyodide, and edge workers. A second path—source-to-source compilation into a Numba or object-mode bar loop—trades interpretive flexibility for throughput on long histories.
This section documents semantics, not the chart. AXIS consumes plots; HOOX consumes strategy events; neither is required to evaluate a script.
Conceptual model
Two execution modes share the same parse tree:
| Mode | Entry | Bar loop | Typical use |
| --- | --- | --- | --- |
| Interpret | NodeLiteralEvaluator.visit(tree) per bar | Host updates PineSeries / context, re-visits AST | Full language surface, strategy, drawings |
| Compile | pynescript.compiler.engine.compilescript | Generated for baridx in range(n) | Numeric indicators (Numba) or object-mode subset |
Interface surface
| Concern | Start here |
| --- | --- |
| Series history, [], var/varip, na | Series & history |
| Expressions, statements, control flow | Expressions & statements |
| Builtin namespaces (ta., strategy., …) | Builtins hub |
| Library export / import | Libraries |
| StrategyEvent parity contract | Events |
| Numba / object-mode compiler | Compiler overview |
Library callers typically use NodeLiteralEvaluator (or the Pro API Runtime) rather than wiring the bar loop by hand:
``python
from pynescript.ast.evaluator import NodeLiteralEvaluator
ev = NodeLiteralEvaluator(context={"close": [., ., .], "barindex": })
result = ev.evaluatescript('//@version=\nindicator("x")\nplot(close)')
`
Hosts that own OHLCV (Pro API, workers) implement the loop: push bar → fill pending orders → visit(tree) → drain events → collect plots. See backend/runtime.py for the reference implementation.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/ | Mixin evaluators (base, names, expressions, statements, libraries, events) |
| src/pynescript/ast/evaluator/builtins/ | Builtin dispatch tables |
| src/pynescript/compiler/ | CompilerVisitor, Numba builtins, strategy broker, engine façade |
| backend/runtime.py | Production bar loop + mode="compile" bridge |
| tests/testevaluator.py, tests/testparity.py, tests/teststrategy.py | Semantic oracles |
Composition of the full interpreter:
`
BaseEvaluator
LiteralEvaluator
NameEvaluator
ExpressionEvaluator
StatementEvaluator
BuiltinEvaluator
→ NodeLiteralEvaluator
`
BuiltinEvaluator itself aggregates mixins (TechnicalAnalysisMixin, StrategyBuiltinsMixin, PlottingFunctionsMixin, …) into one dispatch map built at construction time.
Invariants & edge cases
. Bar re-entrancy. The AST is visited once per bar. Function/type definitions lock after bar (pinedefslocked) so multi-dispatch tables do not grow \(O(\text{bars}^)\).
. Order of broker steps. Pending limit/stop fills run before script body evaluation on each bar (interpreter and compile object-mode agree).
. na is None. Unresolved missing data, OOB history, and arithmetic with missing operands produce Python None, not IEEE NaN—except on the Numba path, which uses np.nan as the array sentinel.
. Strategy state is per-evaluator. StrategyState lives on the instance (evaluator.strategystate), never as a process-global singleton—required for concurrent runs and parity tests.
. Drawing/plot registries are side channels. Visual objects do not participate in pure expression values except where plot() returns a plot id for fill().
Worked example — minimal bar loop mental model
`text
for barindex, bar in ohlcv:
open/high/low/close.update(bar)
context[barindex, time, barstate, …] = …
processpendingorders(OHLC) broker sim
evaluator.visit(ast) script body
events += drainevents()
series.append(plot values)
`
Compile mode collapses this into one generated function over full arrays, with _baridx standing in for barindex.
Failure modes
| Symptom | Likely cause |
| --- | --- |
| unexpected type of node: … | AST node not implemented in any mixin’s visit |
| Unknown built-in function: … | Missing dispatch key or wrong qualified-name resolution |
| Divergent strategy events vs TV | Partial-fill / OCA / risk settings, or fill timing vs bar |
| Numba path raises / falls back | Script uses UDT/map/drawing → object mode; or Numba not installed |
| Plots empty but no error | Host forgot to collect plot_outputs / PlotRegistry` after visit |
See also
Language core — grammar, AST, types before evaluation
Pro API runtime bridge
Numerical validation
pine-worker parity
AXIS docs — optional AXIS for series/drawings
---
FILE: docs/pyne/runtime/libraries.mdx
Libraries
Abstract
TradingView libraries publish as namespace/Name/version with exported members. PYNE implements an in-process analogue: evaluate a library("Title") script (or register source by path), collect exports (export const, exported functions, types), then import binds an alias to a LibraryModule for alias.member access. There is no network publish step—the registry is the host’s responsibility.
Conceptual model
Interface surface
Library script
``pine
//@version=
library("MyMath")
export add(float a, float b) =>
a + b
export const float PHI = .
`
On script completion, StatementEvaluator finalizes pendinglibraryexports onto the active LibraryModule and calls LibraryRegistry.register.
Consumer import
`pine
//@version=
indicator("use")
import user/MyMath/ as M
plot(M.add(close, M.PHI))
`
Resolution (LibraryRegistry.lookup):
. Exact (namespace, name, version) if provided and registered.
. Else title-only match (MyMath) for local evaluation workflows without publisher path.
API on the evaluator
`python
ev.registerlibrarysource(namespace, name, version, sourcestr)
mod = ev.lookuplibrary(namespace=..., name=..., version=...)
`
registerlibrarysource stores Pine text for lazy load on import; evaluating a library script eagerly populates exports.
LibraryModule
Dataclass with title, optional namespace/version, and exports: dict[str, Any]. Attribute access reads exports; missing members raise AttributeError with a clear message. NameEvaluator routes alias.member when the base value is a LibraryModule.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/libraries.py | LibraryModule, LibraryRegistry |
| src/pynescript/ast/evaluator/statements.py | export registration, import visit, library finalize |
| src/pynescript/ast/evaluator/base.py | registry fields on BaseEvaluator |
| src/pynescript/ast/evaluator/init.py | registerlibrarysource, lookuplibrary |
| tests/testlibraryexportimport.py | Round-trip coverage |
Exportable values include callables (user functions), constants, and type-related bindings depending on declaration paths. Exported methods participate in the same call machinery as local functions once retrieved from the module.
Invariants & edge cases
. Evaluate library before consumer (or register source for lazy import). Empty registry → import failure.
. Same evaluator instance (or shared registry) must see both scripts—registries are not process-global unless the host shares them.
. Version is part of the path key. Mismatched version does not fall back to another version automatically when namespace is specified.
. Title registration always updates bytitle so simple local titles work without publisher metadata.
. No TV account auth. Publishing/versioning outside the process is out of scope.
Worked examples
Explicit registration
`python
from pynescript.ast.evaluator import NodeLiteralEvaluator
ev = NodeLiteralEvaluator()
ev.evaluatescript(librarysource) registers by title
ev.evaluatescript(consumersource) import by title or path
`
Path registration without pre-eval
`python
ev.registerlibrarysource("user", "MyMath", , librarysource)
import user/MyMath/ loads and evaluates source on demand
`
Failure modes
| Symptom | Cause |
| --- | --- |
| Import resolves to nothing | Library never registered / wrong title |
| Library 'X' has no exported member 'y' | Missing export or typo |
| Stale exports | Re-evaluated library without re-import; host cache |
| Version conflict | Multiple libs same title—last register` wins for title key |
See also
Expressions & statements
Builtins declarations
End-user library API guide
---
FILE: docs/pyne/runtime/series-and-history.mdx
Series and history
Abstract
Pine’s core data structure is the series: a value that evolves bar by bar, with random access to past values via the history operator []. PYNE implements that model with host-managed series wrappers (PineSeries in the Pro API runtime), plain Python lists in unit tests, and flat numpy arrays on the compile path. Understanding history indexing and na is prerequisite to every builtin and strategy claim.
Conceptual model
Pine indices are most-recent-first:
| Pine | Meaning | List representation (chronological) |
| --- | --- | --- |
| close[] | Current bar | list[-] |
| close[] | Previous bar | list[-] |
| close[n] with \(n \ge \text{len}\) | Out of range | na (None) |
Negative offsets are not supported; they raise rather than wrap.
Interface surface
History operator (visitSubscript)
Implemented in NameEvaluator.visitSubscript:
Float indices coerce to int (e.g. depth / ); NaN index → na.
Lists map Pine index \(i\) to Python index -(i + ).
Scalars: x[] is x; x[i] for \(i > \) is na (no history buffer).
Matrices use [row, col] (tuple/list of length ), not series semantics.
Series wrappers
Hosts expose OHLCV as objects with:
.current — scalar for the active bar
.history — most-recent-first buffer (deque or list)
Arithmetic in ExpressionEvaluator coerces such wrappers via asscalaroperand so close + does not attempt object addition.
var / varip / const
| Qualifier | Semantics in PYNE |
| --- | --- |
| var / varip | Initializer runs on first execution of that declaration site (tracked in vardeclarations), not only barindex == . Later bars skip re-init so the value carries. |
| const (v) | Always initializes when the statement runs; not a cross-bar carry lock like var. |
| bare assign | Re-evaluates every bar; series “history” is the host series or prior list values. |
Nested var inside if barstate.islast or a function therefore initializes on first path-taken bar—matching Pine’s execution-based persistence.
na propagation
| Operation | Rule |
| --- | --- |
| Binary arithmetic / comparison | Any None operand → None (or element-wise for lists) |
| Division by zero | → None (not exception) |
| Soft type errors ("a" + ) | → None |
| Unary | None stays None |
| History OOB | → None |
| Bare name na | Builtin / sentinel resolving to None |
| nz(x, r) (Numba path) | nan → replacement |
List-valued series apply operators element-wise, aligning on the trailing edge when lengths differ (pad leading None).
Bar mode vs full-series mode
Technical helpers (technicalsubmodules/core.py) distinguish:
Full-series mode (unit tests with explicit lists): ta.sma may return a full list of values.
Bar mode (pinebarmode): returns the current scalar so expressions like ta.ema(close,) - ta.ema(close,) stay numeric per bar.
History buffers for indicators are truncated to a rolling window (SERIESMAX = for wrapper histories) to avoid \(O(n^)\) full-history recomputation every bar.
Internals
| Path | Role |
| --- | --- |
| src/pynescript/ast/evaluator/names.py | visitName, visitAttribute, visitSubscript |
| src/pynescript/ast/evaluator/expressions.py | NA-safe binary/unary ops, series coerce |
| src/pynescript/ast/evaluator/statements.py | var/varip/const assign |
| src/pynescript/ast/evaluator/builtins/technicalsubmodules/core.py | asseries, bar mode finalize |
| src/pynescript/ast/evaluator/builtins/utility.py | maxbarsback, lastbarindex, na helpers |
| backend/runtime.py | PineSeries.update per bar |
Compile path: series are np.full(nbars, np.nan) written at baridx; history is arr[_baridx - n] (see Compiler overview).
Invariants & edge cases
. Chronology of .history. Most-recent-first; reverse when converting to chronological lists for ta. / array..
. No negative Pine indices. Explicit error—different from Python lists.
. None vs float('nan'). Interpreter prefers None; Numba kernels use np.nan. Cross-mode comparisons must normalize.
. maxbarsback. Declares intended depth; PYNE does not hard-cap OHLCV history the way a hosted chart might, but indicator helpers still bound rolling windows for cost.
. Tuple returns from multi-value ta.. Unpacking [a, b, c] = ta.macd(...) uses current when it is a sequence of matching arity; otherwise history heuristics apply.
Worked examples
History lag
``pine
//@version=
indicator("lag")
delta = close - close[]
plot(delta)
`
On bar , close[] is na → delta is na. On bar +, arithmetic is ordinary floats (or series lists in batch tests).
var counter
`pine
//@version=
indicator("count")
var int n =
n := n +
plot(n)
`
n initializes once; subsequent bars reassignment (:=) increments. Declaration sites are recorded in vardeclarations.
Element-wise NA
Given two list series of different lengths, a + b aligns tails and pads the longer prefix with None—preserving “most recent bars line up” intuition.
Failure modes
| Symptom | Cause |
| --- | --- |
| All na after first bar | Host not updating series / barindex between visits |
| var re-inits every bar | Fresh evaluator or resetvar_declarations each bar incorrectly |
| TypeError on close + | Series wrapper missing .current / not coerced |
| Compile vs interpret plot mismatch | nan vs None`, or seed differences in EMA/RSI kernels |
See also
Expressions & statements
Technical builtins
Numba path
Runtime hub
---