AST Analysis Phase

Before Pyvorin can optimise anything, it has to decide what it is allowed to touch. That decision happens in the AST analysis phase: a set of walks over the parsed syntax tree that classify every function in a module as a native compilation candidate, a partial candidate, or a function that must run under CPython compatibility semantics. The walk is conservative by design. When it is unsure, it says so, and the function keeps its ordinary Python behaviour.

This page describes the analysis as shipped in Pyvorin Native 1.0.9: what the walker extracts, the constructs that disqualify a function, how imports are classified into four support tiers, and how the results surface in the support and inspect commands you can run against your own code.

What the walk extracts

Compilation begins with the standard library parser: ast.parse(source). The compiler then deep-copies the target function's subtree before any rewriting happens, so that later analysis sees the code exactly as written rather than as reshaped by optimisation. Three broad kinds of information are gathered from the tree.

First, structure: which functions are defined, which calls reference which definitions, which imports introduce which names, and which functions are decorated or defined inside classes. This builds the call graph used to decide whether an import used only by an unrelated helper should affect the function you actually want to compile — it should not, and the analysis takes care to separate the two.

Second, constructs: the walker enumerates every node type present in each function — loops, exception handling, context managers, generators, async constructs, lambdas, f-strings, subscripts, set operations. Each node type maps to a verdict: natively lowerable, lowerable only in a restricted shape, or incompatible with native code.

Third, dependencies: every import and every attribute access on an imported module is classified against a central capability registry. The result of this classification, per module and even per attribute, is what the four support tiers express.

The extraction is cheap. The walk runs once per compilation, and its cost is proportional to the size of the AST, not to the runtime of your program.

What disqualifies a function

The compatibility gate, implemented in frontend/compat_analyzer.py, returns a single verdict per function: natively compilable, or requiring compatibility mode. The disqualifying constructs, verified against the source, are as follows.

Async and generators. async def and await are not natively lowerable in the general case, and yield/yieldFrom force compatibility mode unless the function is a recognised simple generator or a @contextmanager. A generator expression that is not a direct argument of a call — sum(x*x for x in xs) is fine as a call argument; a bare genexp statement is not — also disqualifies.

Module-state mutation. A global statement disqualifies in the default full-compilation mode, because native code cannot honour arbitrary rebinding of module-level names and keep Python semantics.

Context managers. A with statement is natively compilable only when each context expression is a call to open() or to a known class constructor, and the body contains no return, break or continue and no nested with. Multiple context managers in one statement are outside the supported shape.

Exception handling. Simple patterns are supported: raise ExceptionType("constant message"), try/except ExceptionType, and try/finally. Disqualifying shapes include raise ... from ..., try-else, a nested try inside a try body, and return/break/continue inside a try that has a finally.

F-strings with format specs. An f-string containing a formatted value with a format specification (such as f"{x:.2f}") forces compatibility mode; plain interpolation is natively lowerable.

Miscellaneous shapes. enumerate or zip used anywhere other than directly as the iterable of a for loop; a lambda that captures enclosing-function locals and escapes (as a return value or call argument); and set union/intersection/difference through the |, & and - operators, which native code would otherwise misread as bitwise operations on handles.

Two engineering decisions deserve explicit mention. The gate is syntactic: it reasons about node shapes, not runtime values, so it is fast and deterministic — but it will occasionally be stricter than the program actually requires, declining a construct that a smarter analysis could prove safe. That over-strictness is intentional. A false negative (wrong native code) is a correctness bug; a false positive (unnecessary compatibility mode) is a performance note. The gate is built to make only the second kind of mistake.

The four import support tiers

Every import is classified by frontend/module_capabilities.py into one of four tiers. The tier decides whether the import blocks native compilation and, if not, how the compiler handles it.

Classification is per attribute, not just per module, and the granularity matters. math.sqrt, math.sin and the rest of the math surface have native LLVM lowering, so a numeric loop over math calls stays fully native. re splits: literal search, match, fullmatch, sub, findall, split and escape lower natively through the regex runtime, while compile and finditer stay runtime-callable. Runtime-value attributes such as os.environ, sys.platform and os.name have no native representation and force compatibility mode when used on a runtime-callable module alias. An unknown module defaults to unsupported — the registry is an allowlist, not a blocklist.

The enforcement is a safety property, applied regardless of licence state: unsupported and compatibility-only imports generate explicit fallback reasons, and a strict import check raises ImportPolicyError naming the offending module. You never get silently half-compiled code around an import the compiler does not understand.

From analysis to candidate selection

The verdicts roll up per function. A function with no disqualifying constructs and only compile-safe or runtime-callable dependencies is a full native candidate. A function that trips the gate is not thrown away: it is marked for compatibility execution, and in partial-compilation mode (compile_partial, exposed as --partial on the CLI) the supported islands of the function are still compiled natively while the unsupported sections route through the compatibility path. In the default auto backend, the router tries the native compiler first and falls back to honest CPython execution on failure — the fallback is recorded, never hidden.

The analysis is also deliberately selective about blame. An unsupported import used only by an unrelated helper function does not disqualify the function you asked to compile; the walk computes the set of names the target function and its callees actually use before flagging imports. This keeps one exotic import at the bottom of a large module from poisoning everything above it.

Where the tiers surface

You do not have to take the compiler's word for any of this. The support command prints the per-function tier report for any file — the same classification the pipeline itself uses:

python -m pyvorin support your_script.py
python -m pyvorin support your_script.py --json   # machine-readable reasons

Run against a small module mixing a numeric function, a generator and a global-mutating counter, the report looks like this (captured from a live run):

Support report for mixed.py
Function                       Status               Unsupported
------------------------------------------------------------
checksum                       COMPATIBILITY_EXECUTED 0
stream_rows                    COMPILED_PARTIAL     1
tally                          COMPILED_PARTIAL     1

The statuses map onto the compiler's internal ExecutionTier enumeration: COMPILED_FULL for complete native compilation, COMPILED_PARTIAL when some islands fell back, COMPATIBILITY_EXECUTED when the function ran under CPython semantics, plus DEOPTED and UNSUPPORTED for the runtime outcomes covered in Unsupported code and the fallback path. python -m pyvorin inspect wraps the same information with a summary count and an overall risk level.

One practical recommendation falls out of all this. Run support before you benchmark anything. It takes seconds, it is derived from the exact gate the compiler will apply, and it converts "will Pyvorin help my code" from a marketing question into a table you can read line by line. The functions that come back COMPILED_FULL are where the measured speedups in the benchmarks come from; the ones that do not are where your effort, if you want it, should go.

Where to go next

• Compiler pipeline overview — where the AST analysis phase sits in the full journey to native code.
• Type inference system — how the analysed AST acquires the types native code needs.
• Unsupported code and the fallback path — what compatibility execution means at runtime, in detail.
• Quick start — install Pyvorin and run your first support report.

Last reviewed 19 January 2026 against Pyvorin Native 1.0.9 (installed package and source tree). Disqualifying constructs, tier names and CLI output were confirmed against the installed package by running the documented commands locally.