[Language Core]

Language Core

The language core is the front half of the evaluate contract: source text becomes a typed tree; the tree can be walked, rewritten, dumped, and unparsed without ever entering the bar loop.

Abstract

PYNE treats Pine Script as a formal pipeline, not a host plugin. Lexing and parsing are ANTLR4-driven. The concrete syntax tree is lowered by a hand-written visitor (PinescriptASTBuilder) into ASDL-generated dataclasses. Public helpers in helper.py mirror Python’s ast module: parse, unparse, dump, walk, literal_eval. Static tools (linter, type registry, LSP) consume the same tree. Evaluation is out of scope here — see Runtime.

Hard rule: edit grammar only under resource/. Paths under generated/ are regenerated artifacts and must not be hand-edited.

Conceptual model

Interface surface

Consumers almost always enter through pynescript.ast.helper:

from pynescript.ast.helper import parse, unparse, dump, walk, literal_eval

tree = parse('//@version=5\nindicator("x")\nplot(close)')
print(dump(tree, indent=2))
print(unparse(tree))

Modes:

Full page inventory:

Internals

src/pynescript/ast/
  helper.py              # public pipeline orchestration
  builder.py             # PinescriptASTBuilder
  unparser.py            # NodeUnparser + Precedence
  visitor.py             # NodeVisitor
  transformer.py         # NodeTransformer
  collector.py           # StatementCollector (annotation pairing)
  linter.py              # PineLinter
  type_system.py         # Type*, TypeRegistry, MethodResolver
  error.py               # SyntaxError / IndentationError
  node.py                # from grammar.asdl.generated import *
  grammar/
    antlr4/
      resource/          # EDIT HERE: .g4 + *Base.py
      generated/         # NEVER HAND-EDIT
      error_listener.py
      tool/generate.py
    asdl/
      resource/          # EDIT HERE: Pinescript.asdl
      generated/         # NEVER HAND-EDIT
      tool/generate.py

Thin wrappers grammar/antlr4/lexer.py, parser.py, visitor.py re-export generated classes so application code imports stable paths.

Invariants

1. Resource is source of truth. Lexer/parser .g4 and Pinescript.asdl define syntax and node shape; generated Python is a build product (committed so install does not require Java).
2. Round-trip is a test oracle. For the supported surface, unparse(parse(src)) should re-parse and preserve semantics (not byte-identical formatting).
3. Locations are first-class. Statements and expressions carry lineno, col_offset, optional end_* for diagnostics and LSP.
4. Annotations are not free-floating. Comment tokens on COMMENT_CHANNEL are reified as Comment nodes and attached to Script / FunctionDef / TypeDef / Assign via kind suffixes (S, F, T, V).
5. Recursion budget. Deep ternary chains raise the Python recursion limit temporarily during parse (cap ≥ 5000).

Worked examples

Minimal script:

from pynescript.ast.helper import parse, dump

src = """
//@version=5
indicator("demo")
x = ta.sma(close, 14)
plot(x)
"""
tree = parse(src)
assert tree.__class__.__name__ == "Script"
assert any("version" in a for a in (tree.annotations or []))
print(dump(tree, include_attributes=True, indent=2))

Expression-only eval mode:

from pynescript.ast.helper import parse, literal_eval

expr = parse("1 + 2 * 3", mode="eval")
assert literal_eval(expr) == 7

Failure modes

See also

• Grammar (ANTLR4)
• Helper API
• Runtime
• LSP
• Compatibility