Type Inference System

Native code needs types. A CPU instruction that adds two 64-bit integers is not the same instruction as one that concatenates two strings, and a compiled function must commit to machine-level representations long before your data arrives. Pyvorin's type inference system is the machinery that closes that gap: it traces types through dynamic Python well enough to generate native code, records every assumption it makes, and installs guards so that a wrong assumption degrades gracefully instead of corrupting your program. This page describes how that works in Pyvorin Native 1.0.9.

The problem, stated precisely

Python gives a compiler almost nothing statically. A parameter has no declared type; the same variable may hold an int in one call and a float in the next; a function's return type can depend on values only known at runtime. Yet the compiled artefact must pick concrete representations — an unboxed i64, a double, a pointer to a list runtime structure, an opaque object handle — at compile time.

Pyvorin's answer is evidence-based inference. Rather than demanding annotations, it derives logical types from the code itself and from observed behaviour, then treats each derived type as a hypothesis guarded at the boundary. Where evidence runs out, the compiler keeps the object representation. That is the whole philosophy: specialise where provable, box where not.

Local inference: tracing types through assignments

The core pass, _local_var_types in frontend/return_type_analysis.py, walks a function's AST and infers a logical type for every local variable from the expressions assigned to it. A variable initialised from a float literal is a float. A list built by appending ints is an int list. A variable assembled from string pieces is a string. Types propagate through the function in a fixpoint loop — up to ten iterations — so that line → parts → val chains settle even when the assignments appear out of order.

Beneath the textual inference sits a formal type lattice in frontend/type_system.py: a ScalarType with kinds for bool, int, float, string, list, tuple, dict and array, optional flags, element types and shape information, plus a verifier, constant propagation and value-range tracking. The range tracking is what downstream passes build on — bounds-check elimination reasons about index ranges, and the vectoriser's cost model reasons about trip counts.

Two details are worth noticing. Dict literals get their value type tracked only when homogeneous: a dict where all values share one inferable type is treated as typed; a mixed dict falls back to a generic representation. And function-local imports are folded into the alias maps, so from struct import unpack inside a function body types just as well as a module-level import. Small conveniences, but they decide whether real-world code qualifies.

From logical types to native signatures

Inference is only useful if it changes the generated code, and here it does, concretely. The compiler infers ctypes argument types from the untransformed AST, then the LLVM IR generator promotes parameters to unboxed machine types: ints become i64, floats become double, and objects cross as handles that are materialised only where semantics require. Return-type analysis runs the same logic in reverse — what a function returns determines how the wrapper unboxes the result on the way back to Python.

Type-derived information also seeds deeper specialisation. With warm-up arguments available, the compiler inspects actual parameter values — read-only instance lists receive the native instance-field path, and field kinds (bool, int, float, string) are seeded from the warm-up objects so struct fields load as typed values rather than raw pointers. A separate cloning pass produces type-specialised copies of hot helpers, so a function called with ints in one site and floats in another can have both shapes compiled rather than one generic compromise.

The trade-off should be named. Every unboxed parameter is a promise that future calls will honour the same representation, and every specialised copy is code spent on a hypothesis. The compiler's defence is twofold: inference stays conservative on anything it cannot trace, and everything it does commit to is checked at the boundary — which brings us to guards.

Where the guards come from

A guard is a runtime check that a compile-time assumption still holds. Pyvorin installs them at the point where native code meets the interpreter: the call wrapper.

The most common guard is the argument-shape check. Before dispatching to native code, the wrapper verifies that each argument fits the native signature — an integer parameter is checked to fit in a signed 64-bit range, a float parameter is checked to be an actual float. These checks are cheap, and they exist because Python ints are unbounded: a value of 10**30 is a perfectly legal Python int that simply has no i64 representation. Without the check, the compiled function would truncate silently.

A second, opt-in layer adds inline guards inside the generated code itself, controlled by PYVORIN_INLINE_GUARDS=1. In this mode, the LLVM IR calls a deopt stub when an internal assumption fails; the stub records the failure in thread-local storage, the native function returns a sentinel, and the wrapper raises GuardRuntimeError after the call. The current implementation is explicitly a safe V0 — a ctypes callback rather than a longjmp-based frame reconstruction, chosen because it needs no C extension build and preserves the exception chain. Full frame reconstruction is documented in the source as future work. That is an honest limitation, and worth knowing before you enable the flag.

Specialisation is a third source. A specialised copy compiled for int arguments carries the assumption that its arguments are ints; the wrapper dispatches to it only after the corresponding check passes.

When a runtime type breaks the assumption

Here is the part that matters most for correctness. When a guard fails, the result is never a wrong answer — it is a diversion.

The wrapper catches the guard failure, the out-of-range value, or the unrepresentable result, and reruns the call through a lazily compiled copy of the original Python source. On first use, that fallback is compiled from the untouched source and cached; from then on, a guard failure costs one exception and one interpreted call, and execution continues with exactly the semantics CPython would have given you. Overflow on checked arithmetic is the instructive case: an int addition whose result exceeds the native range raises a marked exception that propagates when it must, and everything else — a negative exponent producing a float, an out-of-range power — diverts to CPython so Python semantics are preserved.

These events are recorded, not swallowed. The execution report tracks deopt counts, fallback counts and guard failures per function, and a function that repeatedly fails its assumptions is classified into the DEOPTED tier — the compiler's way of saying the hypothesis did not survive contact with the data. Profile-guided optimisation feeds the same machinery in the other direction: warm-up runs collect loop trip counts, branch frequencies, and argument and return type histograms, and the PGO engine uses those measurements to switch vectorisation, unrolling and parallelism on or off. Observed types inform; observed behaviour decides.

For the full mechanics of the guard-and-deopt path — including the sentinel protocol and the thread-local failure slot — see Guard and deoptimisation. The one-sentence summary is the design contract: an assumption is a hypothesis with an escape hatch, and the escape hatch leads back to correct Python.

What inference cannot see

Honesty requires the boundaries. Type inference in Pyvorin is intra-procedural and evidence-based: it reasons about assignments, calls and literals it can trace, and it does not attempt whole-program type reconstruction. Dynamic attribute access, values produced by runtime-only code paths, and functions whose behaviour depends on data the compiler never observes will keep the generic object representation rather than a wrong unboxed one. In practice this means the system is strongest exactly where its users need it most — numeric and data-shuffling loops with stable types — and weakest where Python is most dynamic, which is precisely where native compilation has little to offer anyway. The conservative fallback is not a gap in the system; it is the system working as designed.

Seeing it in practice

# Which functions compiled fully, and which fell back
python -m pyvorin support your_script.py --json

# Per-function optimisation detail, including specialisation
python -m pyvorin explain your_script.py

# Isolate type-dependent stages when benchmarking
python -m pyvorin bench your_script.py --function your_entrypoint --no-pgo --runs 5

The --json support report exposes deopt_count, fallback_count and fallback_reason per function, so after a run you can check whether your data honoured the inferred types or kept escaping to the fallback path. If the escape rate is high, the right response is not to force native compilation — it is to look at the types your function actually receives, because the compiler's assumptions are only as good as the evidence it was given.

Where to go next

• Guard and deoptimisation — the runtime contract in detail: sentinels, the deopt stub, and frame reconstruction.
• Compiler pipeline overview — where type inference sits among the full set of stages.
• AST analysis phase — the gate that decides which functions reach type inference at all.
• Benchmarks — what the typed, unboxed path delivers on measured workloads, including where it does not.

Last reviewed 21 January 2026 against Pyvorin Native 1.0.9 (installed package and source tree). Inference passes, guard checks, fallback semantics and CLI flags were confirmed against the installed package; no mechanism on this page is described that could not be verified there.