Anomaly Detection with Pyvorin

Anomaly detection over event streams is a statistics problem wearing a latency costume. Strip the costume and the expensive part is almost always the same: for every window of readings, compute aggregates — mean, variance, correlation, a rolling threshold — and flag the windows that deviate. That computation is pure-Python-shaped work, and it is where Pyvorin Native 1.0.9 has measured, reproducible wins. This page shows the architecture that delivers them: windowed aggregates compiled ahead of time and called from behind your stream processor, with the measured anchors from the canonical benchmark run and an honest account of where the pattern stops.

What "real time" means here

Start with the boundary, because marketing pages usually blur it. Pyvorin compiles Python functions to native machine code; it does not turn a Python process into an in-line event handler, and this page makes no such claim. The transport — reading from the broker, deserialising messages, acknowledging offsets, fanning out to consumers — stays in your stream processor's own stack, where it belongs. Pyvorin sits one step behind the transport, on the computation step: the function that takes a window of values and returns a decision. That boundary is where the measured gains live, and drawing it correctly is the difference between a system that is fast and a slide deck that says it is.

The pattern: a compiled detector behind the processor

The detector is an ordinary Python function over a window. Mark it, compile it ahead of time, and call it per window from the processor's callback:

# detector.py — plain Python, no framework imports in the hot path
def window_score(readings, mean_hist, std_hist):
    n = len(readings)
    total = 0.0
    for r in readings:
        total += r
    mean = total / n
    var = 0.0
    for r in readings:
        diff = r - mean
        var += diff * diff
    std = (var / n) ** 0.5
    if std == 0.0:
        return 0.0
    return abs(mean - mean_hist) / (std_hist + 1e-9)

# Compile once at build time; the cache ships with the service
python -m pyvorin compile detector.py --fail-on-fallback

# Verify the speedup and correctness on your data shape
python -m pyvorin run detector.py --function window_score --compare --runs 5

Per window, the processor hands window_score a list of floats and history values; the function returns a z-score-style deviation that the surrounding code thresholds and routes. Compilation happens once, at build time, and the compiled artefact travels in the service image — nothing compiles on the event path, and no source leaves the build pipeline. The surrounding orchestration — windowing by time or count, handling late data, emitting alerts — remains the stream framework's job. Pyvorin's role is confined to making the arithmetic native.

Keep the compiled function pure and self-contained. Pass the window contents and the history values in as arguments, and take the decision logic — threshold, hysteresis, minimum-suspicion interval — as parameters or return a score and let the processor apply the policy. This is not stylistic advice: a function that reaches for module-level state, clocks or network clients gives the compiler less to work with and gives you a harder correctness story when the fallback path runs the original instead. Pure-in, score-out keeps the two paths equivalent by construction, which is what makes the honest fallback a safety net rather than a subtle fork in behaviour.

Measured anchors

The canonical 2026-09-13 benchmark run, with memory caps fixed and a quiet host, contains the workloads closest to this pattern:

Read all five rows before quoting any of them. The wins — anomaly detection at 45.85x, correlation at 52.53x, the windowed aggregate at 134.62x — come from tight loops over numeric data with no library calls in the middle. The flat and negative rows are the caution: mean-and-variance at 1.04x and histogram at 0.85x spend their time in operations the compiler cannot improve, and they show what happens when the loop is not the bottleneck. The statistics category as a whole averages 3.59x across its three workloads precisely because the category mixes both shapes. The suite-wide picture is the same lesson at larger scale: median 1.35x, best 202.83x, worst 0.05x, seventeen of 71 workloads slower than CPython. Your detector's position in that spread depends on its loop, and only measuring it will tell you where.

Where the pattern stops helping

Three limits deserve explicit statements. First, per-event handlers: if the "detector" is a function called once per message on a tight per-event budget, the compiled path's fixed overheads can dominate, and the suite's worst rows — 0.05x on micro-workloads — are the honest forecast. Batch the work into windows. Second, string-heavy detection: if the signal lives in log lines, headers or payloads that must be parsed before statistics can run, expect the parsing categories, not the statistics ones — string manipulation averaged 0.85x and parsing 0.86x in the suite. Do the parsing in the processor or upstream, and hand the compiled function numbers. Third, model inference beyond simple statistics: distance-based and linear scoring fit the compiler; anything that bottoms out in a native library — a tensor runtime, a tree ensemble from a C extension — was never interpreted to begin with, and compilation has nothing to remove. The general test is unglamorous: profile the detector, and if the flame shows an interpreted Python loop, it is a candidate; if it shows library calls or I/O, it is not.

Building and gating the detector

The build pipeline for a detector is the same speed-proof workflow used everywhere else:

# Which functions compile fully, and which fall back
python -m pyvorin support detector.py

# Compile as a release gate
python -m pyvorin compile detector.py --fail-on-fallback

# Measured comparison on representative windows
python -m pyvorin run detector.py --function window_score --compare --runs 5 --json

Feed the --compare step windows that match production shape and size — a benchmark on 10,000-element windows says little about 200-element ones. Keep the --fail-on-fallback gate in CI so a dependency change that silently diverts the detector to the interpreter fails the build rather than the service; if you prefer resilience over strictness at runtime, the fallback path keeps the detector correct at CPython speed, and it is recorded. The fallback mechanics are on the unsupported code and fallback page, and the measurement discipline — warm-up, same host, correctness checked per run — is on the benchmark methodology page.

The judgement call: window size

Every deployment of this pattern eventually faces the same trade-off: smaller windows detect sooner but shrink the compiled function's advantage, because the fixed cost of the call is amortised over fewer values. The suite supplies the two ends of the ruler — 45.85x on a substantive anomaly loop, 0.05x when the workload is so small the overhead owns the runtime. Our engineering rule: size the window to the physics of what you monitor — a bearing vibration signature needs milliseconds; a warehouse temperature excursion does not — and then verify the chosen size with the speed proof, rather than choosing the window to flatter the benchmark. A detector that is fast on the wrong window is just wrong, sooner.

Where to go next

• How to run a speed proof — the measurement workflow, end to end.
• Benchmarks — every measured result, including the losing rows, from the canonical run.
• Unsupported code and the fallback path — what happens when the compiler declines your detector.
• Implementing Pyvorin in microservices — deploying the compiled detector service itself.

Last reviewed 18 June 2026 against pyvorin-native 1.0.9 installed at /root/pvfinal (verified build of 12 September 2026). All workload figures are from the canonical 2026-09-13 benchmark artefact with memory caps active. No wire-path or latency-boundary claims are made beyond the measured workload timings.