Helper API

Helper API

src/pynescript/ast/helper.py is the stable library face of the language core. It orchestrates ANTLR, the builder, annotations, and the unparser behind functions deliberately similar to CPython’s ast module.

Abstract

Call parse to obtain an ASDL tree; dump / walk / iter_* to inspect it; unparse to regenerate source; literal_eval for constant-folding-safe evaluation of literal expressions. Location helpers (copy_location, fix_missing_locations, get_source_segment, increment_lineno) support tooling that rewrites or reports on trees. clear_parse_cache / parse_cache_info control the process-local sha256 LRU.

Everything higher in the stack (CLI, LSP, Pro API, workers) ultimately routes through this module for front-end work.

Conceptual model

Rendering…

Interface surface

parse(source, filename="<unknown>", mode="exec") -> AST

Raises:

• ValueError — invalid mode
• pynescript.ast.error.SyntaxError — lexer/parser failure (via PinescriptErrorListener)

Side effects during exec mode:

1. Temporarily raises sys.getrecursionlimit to at least 5000.
2. Collects COMMENT tokens containing @ and attaches annotations to eligible statements. If the source has no @, the annotation pass is skipped.

filename is not part of the cache key (only diagnostics).

Parse cache

Successful trees are stored in a process-local, thread-safe OrderedDict LRU keyed by (sha256(source.encode("utf-8")), mode).

On a cache hit, the same AST object is returned (after scrubbing stale _pine_call_site attrs). Treat it as read-only: a NodeTransformer or increment_lineno mutates the cached entry for every later caller. After intentional mutation, call clear_parse_cache() or parse with the cache disabled.

unparse(node) -> str

Delegates to unparse_node — a per-thread reused NodeUnparser (warm visitor cache). Public return value is still a new str. See Unparser.

dump(node, *, annotate_fields=True, include_attributes=False, indent=None) -> str

Recursive pretty-print of node trees. With indent (int spaces or string), multi-line layout is used for non-trivial nodes.

literal_eval(node_or_string, context=None, data_feed=None, data_provider=None) -> Any

• Strings are parse(..., mode="eval") first.
• Unwraps Expression.body.
• Uses NodeLiteralEvaluator — not full script evaluation. Non-literal graphs raise.

Tree navigation

Location utilities

__all__ export list

clear_parse_cache, copy_location, dump, fix_missing_locations, get_source_segment, increment_lineno, iter_child_nodes, iter_fields, literal_eval, parse, parse_cache_info, unparse, walk.

Internals

Path

src/pynescript/ast/helper.py — all public helpers above.

Related:

• src/pynescript/ast/collector.py — StatementCollector for annotation pairing
• src/pynescript/ast/unparser.py — unparse_node (thread-local NodeUnparser)
• src/pynescript/ast/evaluator — NodeLiteralEvaluator (literal_eval only)

Parse pipeline (_parse)

1. Validate mode ∈ {exec, eval}.
2. Raise recursion limit to ≥ 5000 if needed.
3. Bind a thread-local lexer + token stream + parser (_ThreadParseEngine). Process-wide reuse is unsafe (indent / token-stream bleed).
4. SLL first (BailErrorStrategy). On ParseCancellationException, reset and re-parse LL (DefaultErrorStrategy). Trees match pure-LL on success.
5. PinescriptASTBuilder.visit via the shared stateless builder.
6. Exec mode: if source contains @, collect statements + @ comments and _add_annotations.

Default ANTLR console listeners are removed; only PinescriptErrorListener.INSTANCE is installed.

Annotation algorithm (_add_annotations)

1. Merge comments and statements; sort by (lineno, col_offset).
2. Group consecutive comments vs statements.
3. Keep only kinds starting with @.
4. Script-level: first group members with kind ending in S → script.annotations.
5. Pair remaining comment groups with following statements; attach F/T/V filters to FunctionDef / TypeDef / Assign.

Comment nodes are not left in Script.body; only string annotation lists are stored on targets. Collection itself skips tokens whose text has no @ (plain // and //# region never become Comment nodes).

Stream helpers

Invariants

1. mode is a closed set. Only exec and eval.
2. Default listeners are removed. Console ANTLR spam is replaced by raising project SyntaxError.
3. dump requires an AST instance. Non-nodes raise TypeError.
4. Round-trip tests should use parse + unparse, not raw builder access, so annotation behavior matches production.
5. Cache identity is shared. Do not mutate a cached tree unless you also clear_parse_cache().
6. filename is diagnostic-only and is not part of the sha256 key.

Worked examples

Inspect a tree

from pynescript.ast.helper import parse, dump, walk

tree = parse("""
//@version=6
indicator("demo")
plot(close)
""")
print(dump(tree, indent=2))
print(sum(1 for _ in walk(tree)), "nodes")

Source segment

from pynescript.ast.helper import parse, get_source_segment

src = "a = 1\nb = a + 2\n"
tree = parse(src)
assign_b = tree.body[1]
print(get_source_segment(src, assign_b))  # "b = a + 2\n" or similar span

Literal evaluation

from pynescript.ast.helper import literal_eval

assert literal_eval("2 * 3 + 4") == 10
assert literal_eval("'hi'") == "hi"

Parse cache

from pynescript.ast.helper import parse, parse_cache_info, clear_parse_cache

src = 'indicator("x")\nplot(close)\n'
a = parse(src)
b = parse(src)
assert a is b  # same object on hit
print(parse_cache_info())  # enabled, size, maxsize, hits, misses
clear_parse_cache()

Failure modes

See also

• Builder
• Unparser
• Visitor / transformer
• Error model