Numerical validation

Methodology and error bounds for PYNE technical and math builtins versus reference Pine Script™ implementations.

This page

Numerical validation

Abstract

PYNE’s technical and mathematical builtins are validated for numerical agreement with reference Pine Script™ implementations under IEEE 754 realities. The long-form report (docs/numerical_validation_report.md, v1.0, Nov 2025) documents methodology, per-family error tables, and edge cases.

Headline (from that report): max observed relative error 0.0022% (ADX, extreme volatility); average error < 0.0005%; deterministic ops (SMA, OBV, discrete math) often exact.

Treat percentages as report-era measurements, not eternal SLAs — re-validate after changing smoothers or bar-mode semantics.

Conceptual model

Diagram

Rendering…

Interface surface

Acceptable thresholds (report)

BandRelative errorVerdict
Excellent< 0.001%Pass
Good< 0.01%Pass
Acceptable< 0.1%Review
Unacceptable≥ 0.1%Fail

Metric definitions

  • Absolute error: |ref − pyne|
  • Relative error: |ref − pyne| / |ref| × 100% (guard zero refs)
  • Max / mean / std over bars and scenarios

Category highlights

FamilyNotes
Moving averagesSMA exact; EMA/WMA/… tiny FP error
OscillatorsRSI/stoch slightly higher smoothing error still ≪ 0.01%
TrendADX highest in report (0.0022% max)
VolumeOBV exact cumulative; MFI inherits typical-price FP
Statspercentrank exact; variance/stdev excellent
MathDiscrete exact; transcendentals ~1e-15 class

Dual-host formula alignment (interpret ↔ compile)

Where fixed, these kernels share one formula contract across interpret (evaluator / incremental) and compile (numba_builtins) so plot series match bit-identical or maxdiff ≈ 0 on dual-run goldens:

BuiltinShared contract (summary)
ta.rsiWilder: SMA seed of first period deltas, then RMA gain/loss; first valid at bar period
ta.rocStandard lookback % change; na until lookback (no early 0.0)
ta.wmaFull non-na window required; no partial-window reweight (nested ROC/WMA safe)
ta.cumRunning sum; NaNs as 0; user series (ad = ta.cum(...)) must not alias builtin A/D
ta.highestbars / lowestbarsNegative bars-back offset; short/all-NaN → -1; oldest extreme on ties

Goldens: tests/test_compiler_numba.py (TestInterpCompilePlotParityFixes) and scripts/compare_interp_compile.py. Residual dual-host drift (if any) tends to cluster in nested EMA seed families (DEMA/TEMA SMA-seed vs first-value) rather than these aligned kernels — see Compatibility.

Error distribution (report aggregate)

Error rangeShare (approx.)
Exact 0~38%
< 0.0001%~46%
0.0001–0.001%~14%
0.001–0.01%~2%
≥ 0.1%0%

Internals

PathRole
docs/numerical_validation_report.mdFull tables + bias analysis
tests/test_ta_indicators_*.pyExecutable TA tests
tests/test_indicators.py / builtins testsAdditional numeric guards
tests/test_compiler_numba.pyInterpret ↔ compile formula goldens (RSI/ROC/WMA/…)
scripts/compare_interp_compile.pyCorpus plot-series parity harness
Bar-mode vs list-mode (_pine_bar_mode)Scalar-per-bar vs full series — compare like with like

Methodology (report)

  1. Generate synthetic 1k-bar OHLCV across regimes (trend, range, vol, gaps)
  2. Validate with real histories (equities, crypto, FX samples)
  3. Export reference outputs; run PYNE; element-wise compare
  4. Search systematic bias; inspect tails

Invariants & edge cases

  1. na propagation must match Pine truthiness — numerical compare should mask na pairs.
  2. Bar-mode scalars vs full-series list mode can look like “bugs” if misaligned.
  3. Seeded mocks (request.seed) stabilize stochastic feeds for regression.
  4. Recursive smoothers accumulate FP differently across languages; bounds > bits.
  5. Extreme prices (near-zero, huge) tested; still within excellent band in report.

Worked examples

Relative error helper

def rel_err(a: float, b: float) -> float:
    if a == 0 and b == 0:
        return 0.0
    denom = abs(a) if a != 0 else abs(b)
    return abs(a - b) / denom * 100.0

Run TA tests

pytest tests/test_ta_indicators_1.py tests/test_ta_indicators_2.py -q

Failure modes

SymptomInvestigation
Sudden ADX driftCheck Wilder smoothing / seed bars
SMA not exactOff-by-one window or float input casts
Good unit tests, bad TV export compareTimezone/session alignment of bars
Only bar 0 differsWarm-up / na initialization

See also