Language Core

Language Core

The language core is the front half of the evaluate contract: source text becomes a typed tree; the tree can be walked, rewritten, dumped, and unparsed without ever entering the bar loop.

Abstract

PYNE treats Pine Script as a formal pipeline, not a host plugin. Lexing and parsing are ANTLR4-driven. The concrete syntax tree is lowered by a hand-written visitor (PinescriptASTBuilder) into ASDL-generated dataclasses. Public helpers in helper.py mirror Python’s ast module: parse, unparse, dump, walk, literal_eval, plus a process-local sha256 LRU parse cache (clear_parse_cache, parse_cache_info). Static tools (linter, type registry, LSP) consume the same tree. Evaluation is out of scope here — see Runtime.

Hard rule: edit grammar only under resource/. Paths under generated/ are regenerated artifacts and must not be hand-edited.

Conceptual model

Rendering…

Interface surface

Consumers almost always enter through pynescript.ast.helper:

from pynescript.ast.helper import parse, unparse, dump, walk, literal_eval

tree = parse('//@version=6\nindicator("x")\nplot(close)')
print(dump(tree, indent=2))
print(unparse(tree))

Successful parse results are cached by sha256(source) + mode (default ON; PYNE_PARSE_CACHE=0 disables). Treat returned trees as read-only.

Modes:

Full page inventory:

Internals

src/pynescript/ast/
  helper.py              # public pipeline + sha256 LRU parse cache
  builder.py             # PinescriptASTBuilder (_setLocations = direct attrs)
  unparser.py            # NodeUnparser + Precedence
  visitor.py             # NodeVisitor
  transformer.py         # NodeTransformer
  collector.py           # StatementCollector (annotation pairing)
  linter.py              # PineLinter
  type_system.py         # Type*, TypeRegistry, MethodResolver
  error.py               # SyntaxError / IndentationError
  node.py                # from grammar.asdl.generated import *
  grammar/
    antlr4/
      resource/          # EDIT HERE: .g4 + *Base.py
      generated/         # NEVER HAND-EDIT
      error_listener.py
      tool/generate.py
    asdl/
      resource/          # EDIT HERE: Pinescript.asdl
      generated/         # NEVER HAND-EDIT
      tool/generate.py

Thin wrappers grammar/antlr4/lexer.py, parser.py, visitor.py re-export generated classes so application code imports stable paths.

parse uses a thread-local ANTLR engine and a two-stage SLL then LL strategy; successful trees go into a sha256 LRU (PYNE_PARSE_CACHE, default 128).

Invariants

1. Resource is source of truth. Lexer/parser .g4 and Pinescript.asdl define syntax and node shape; generated Python is a build product (committed so install does not require Java).
2. Round-trip is a test oracle. For the supported surface, unparse(parse(src)) should re-parse and preserve semantics (not byte-identical formatting).
3. Locations are first-class. Statements and expressions carry lineno, col_offset, optional end_* for diagnostics and LSP.
4. Annotations are not free-floating. Comment tokens on COMMENT_CHANNEL are reified as Comment nodes and attached to Script / FunctionDef / TypeDef / Assign via kind suffixes (S, F, T, V).
5. Recursion budget. Deep ternary chains raise the Python recursion limit temporarily during parse (cap ≥ 5000).
6. Cached trees are shared by identity. Mutating a parse() result (transformer, increment_lineno) mutates the LRU entry. Call clear_parse_cache() after intentional mutation, or disable the cache.
7. FunctionDef.returns is a type spec, not an executable expression. Optional leading type_specification on UDFs/methods is stored there (0.3.9+).

Worked examples

Minimal script:

from pynescript.ast.helper import parse, dump

src = """
//@version=6
indicator("demo")
x = ta.sma(close, 14)
plot(x)
"""
tree = parse(src)
assert tree.__class__.__name__ == "Script"
assert any("version" in a for a in (tree.annotations or []))
print(dump(tree, include_attributes=True, indent=2))

Expression-only eval mode:

from pynescript.ast.helper import parse, literal_eval

expr = parse("1 + 2 * 3", mode="eval")
assert literal_eval(expr) == 7

Failure modes

See also

• Grammar (ANTLR4)
• Helper API
• Runtime
• LSP
• Compatibility