Guards and Deoptimisation in Pyvorin

Compiled Pyvorin code runs on assumptions. The compiler assumed your list held floats when it emitted a vectorised loop; the ABI assumed an argument's dtype and layout when it marshalled a pointer. Guards are the runtime machinery that checks those assumptions, and deoptimisation is what happens when a check fails. This page states the contract precisely, shows the mechanism, and is equally precise about the honest cost — guard-heavy workloads are among the slower patterns in the published benchmark suite.

Everything described here was read from the pyvorin-native 1.0.9 source and installed binaries. Where the old version of this page implied behaviour we could not find in the code, we say so.

The contract

The rule, stated in the guard subsystem's own documentation, is short: a guard failure must never silently alter semantics. It must transfer execution to the Python reference implementation of the same function at the same program point, with the same arguments, producing the same result the uncompiled program would have produced. Everything in the guard machinery — the checks, the error types, the fallback path, the logging — exists to honour that single rule.

Two corollaries follow. First, guards are a correctness device that happens to cost performance, not a performance device that might compromise correctness. Second, a guard is allowed to be conservative: checking more than strictly necessary is safe, checking less is a bug. The implementation leans conservative throughout.

What the guards check

The guard layer validates arguments against the plan the compiler committed to. For array-style kernel arguments the checks are: the value is not null and exposes a usable data pointer; the dtype matches exactly — float64 promised, float64 delivered; the layout is C-contiguous, because a vectorised kernel reads memory linearly; pairs of arrays that the kernel assumes independent do not share memory, checked via np.shares_memory with a raw-address-range fallback; and platform length limits are respected. Scalars are range-checked where the native arithmetic requires it, since native integer paths cannot always represent Python's arbitrary-precision results.

The checks live at two levels. A small runtime module provides the primitive predicates — contiguous, dtype, no-alias — plus an Assumption cache, an LRU of up to 256 entries keyed by object identity that memoises guard results across repeated calls with the same arrays. Above it, a plan-aware layer ties the predicates to a specific kernel's argument descriptors and raises a single structured failure type, GuardRuntimeError carrying a GuardFailure record with the guard's name, a human-readable reason and the argument involved.

The identity-keyed cache is a deliberate engineering trade-off worth naming. Keying by id() is the cheapest possible lookup, and it is only sound while the caller keeps the guarded objects alive — a freed object could in principle leave a stale entry that a new object at the same address would match. In practice guard inputs are live call arguments held for the duration of the call, and the LRU bound keeps memory flat. The alternative — hashing array contents — would be correct under all lifetimes and catastrophically slow. Pyvorin chose the cache that is fast and documented its assumption, which is the honest version of the choice.

Inline guards: the V0 deopt mechanism

Paying the Python/ctypes boundary on every call is wasteful for kernels that run long enough to ignore it, and expensive for short ones. An inline-guard subsystem addresses this by emitting checks directly into the LLVM IR, before optimised regions. Each guard lowers to a comparison and a conditional branch: on success, execution continues; on failure, control branches to a shared deopt stub. The documented cost table is one LLVM instruction for a type-tag or null-pointer check, three to four for a contiguity check.

The deopt stub itself is V0, and the source is candid about it. When a guard fires inside native code, the stub — a ctypes callback whose address is mapped into the module with llvm.add_global_mapping — records the guard ID and actual value in thread-local storage and returns. The compiled function then returns a sentinel, and the Python wrapper inspects the thread-local slot after the call and raises GuardRuntimeError if a failure was recorded. True frame reconstruction, the longjmp-style deopt that would resume execution mid-function with rebuilt CPython state, is explicitly marked future work: both reconstruct_frame and the trampoline allocation raise NotImplementedError if called.

Inline guards are also off by default in 1.0.9. The feature activates only when PYVORIN_INLINE_GUARDS=1 is set; the wrapper-level guard path described next is the production default. Treat inline guards as an optimisation preview with a safety net, not as a dependency.

What actually happens on failure

Here is the behaviour we verified, stated without ornamentation. A guard failure raises GuardRuntimeError at the boundary. The compiled wrapper catches it and calls the fallback — the original Python source, compiled lazily by the ordinary CPython compiler and cached — with the same arguments. The result of that fallback call is returned to the caller. Semantics are preserved exactly, because the fallback is the original program.

Two details of that sentence carry the weight. The diversion is per call: a function whose guard fails does not get recompiled, re-specialised or deoptimised-and-resumed in 1.0.9. We searched for a recompilation path keyed on guard failures and did not find one; the lazily-built fallback serves every subsequent call that fails its checks. And failure does not poison the function — the native code and its cache entry remain in place, so a transient mismatch and a later conforming call both behave correctly.

One narrow exception to silent fallback exists and is itself principled: when native code has already mutated parameter state that a CPython rerun would double-apply, or when the exception was raised by the native side with mutation hints attached, the wrapper re-raises rather than reruns. Partially-applied mutation cannot be replayed honestly, so the system stops loudly instead of corrupting state. A surprise exception is recoverable; a silently wrong result is not.

Where failures are recorded

Guard failures and deoptimisation events are not invisible. A dedicated diagnostics module defines a structured event — function name, a canonical reason (of which type_guard_failure is one), optional line and detail, timestamp, and a flag distinguishing compile-time fallbacks from runtime events. A thread-safe logger batches these events in memory (up to 10,000), appends them to a JSON file when configured, and forwards them to the telemetry pipeline when telemetry is enabled. The routing layer likewise marks functions that ran on the fallback path, and every result exposes which backend executed it. If you are auditing why a workload ran slower than expected, the record of which calls diverted to CPython is the first place to look.

The honest cost

Guards are not free, and Pyvorin's own measurements say so plainly. The benchmark suite's sixteen slower-than-CPython workloads cluster in shapes where per-call overheads show through: web request handling at 0.42x, database operations at 0.33x, parsing at 0.31x by geometric mean, with a worst case of 0.01x on a workload of millions of tiny object interactions. Code like that does short bursts of work per call; the guard suite, the marshalling layer and the boundary crossing are a visible fraction of each call's total time. The same guard machinery is negligible on the suite's winners — the 272x trapezoidal-integration loop amortises its checks over millions of vectorised iterations.

The design response to this cost is not to weaken the guards. It is to decline compilation when there is nothing worth guarding: the router sends functions it cannot take natively to the fallback path upfront, and the pyvorin support command shows the per-function tier breakdown before you benchmark anything.

# See which functions are native candidates and which will guard-divert
python -m pyvorin support your_script.py

# Measure with the contribution of each stage isolated
python -m pyvorin bench your_script.py --function your_entrypoint --runs 5

If your profile shows a hot function spending its time crossing the boundary rather than inside the loop, the remedy is the usual one: batch work into fewer, larger calls so guard cost amortises, and keep argument types stable so the checks stay cheap. That advice is not Pyvorin-specific folklore; it is the arithmetic of per-call overhead stated in the open.

Where to go next

Unsupported code and the fallback path — the compile-time sibling of runtime guard failure, in detail.
Runtime libraries — the ABI and argument marshalling the guards validate.
Native code generation — what the guards protect, and the vectorisation they unlock.
Benchmarks — the measured distribution, including the guard-heavy slower cases.

Last reviewed 2 February 2026 against pyvorin-native 1.0.9: guard checks, the V0 deopt stub, the per-call fallback behaviour and the diagnostics event model were read from the installed package and source tree. The earlier claim that failed guards trigger recompilation with updated type information was not found in the 1.0.9 code and has been corrected on this page.