Pyvorin for Regulatory Reporting

Regulatory reporting pipelines share one anatomy: ingest transaction records, validate them against rules, enrich them against reference data, aggregate them into reportable totals, format and submit. The deadlines are fixed by regulators, the volumes grow every year, and the compute is pure Python because the rules change too often to freeze into another language. That anatomy maps almost one-to-one onto the benchmark table — with the usual split. Some stages have strong measured anchors; others regress under compilation. This page walks the pipeline stage by stage with the named evidence.

No institution appears here; the deployment is an illustrative example. Every speedup below names a workload from the benchmark artefact dated 13 September 2026, extracted from the published table, and combined arithmetic is labelled an illustrative model.

Stage by stage, with the anchors

Ingestion and parsing. Trade reports, order records and reference files arrive as CSV, fixed-width or message payloads. This is string work, and the suite is consistent: data_processing.csv_parse measured 0.8x, parsing.log_line_parse measured 0.9x, and the parsing category as a whole sits at a 0.86x geomean. The string category is similar at 0.85x. Verdict: do not compile the ingest stage. Materialise parsed records once and move on.

Validation. Field-range checks, checksum verification, referential-integrity tests — per record, in pure Python loops. Small per-record work lives at the modest end of the measured range, but accumulation loops have anchors: crypto.simple_checksum measured 6.8x, and the rolling-window shape of threshold checks appears in iot.sensor_anomaly at 45.9x. Validation batches large enough to amortise compile cost are legitimate targets; per-record micro-checks invoked once each are the suite's cautionary micro-workload shape, where compilation overhead can dominate.

Enrichment and reference data. Attaching ISINs, classifications and counterparty attributes per record is dictionary-lookup work backed by CPython's C-implemented dict code — never slow. The suite's own near-flat rows make the point: stdlib.dict_lookup measured 1.0x, database.group_by_sum measured 1.1x, and database.word_count_index measured 1.3x. Hash-heavy work resists. The compiler's contribution here is nil; the engineer's contribution is algorithmic — pre-hash reference tables once rather than looking up per record in a loop-shaped way.

Aggregation. This is where the pipeline's hours live, and the anchors are the strongest on this page: etl.windowed_aggregate, per-category windowed totals, measured 134.6x; etl.top_n_selection, largest-exposure lists, measured 7.8x; etl.distinct_count, distinct counterparty or instrument counts, measured 2.9x; statistics.correlation, exposure-correlation matrices, measured 52.5x. The ETL category's geomean is 4.6x, but the honest way to quote it is per workload, exactly as this page does — the same category contains etl.merge_join at 0.6x.

Formatting and submission. String formatting, serialisation, file writing. serialization.json_serialize measured 3.6x where serialisation is pure Python, but the string and parsing categories tell the dominant story: near or below 1x. Treat formatting as fixed cost.

An illustrative deployment

Consider a transaction-reporting batch with a regulatory deadline in the morning. Assume profiling attributes 10% of runtime to parsing, 10% to validation, 15% to enrichment and lookups, 50% to aggregation, 15% to formatting and I/O. Assume — labelled, not measured — 25x on the aggregation share, inside the measured 2.9x to 134.6x range for the matching workloads. The model: 0.10 + 0.10 + 0.15 + 0.50/25 + 0.15 = 0.52 of the old runtime, roughly 1.9x end to end. On those assumptions, a five-hour batch finishes in two and a half hours — which is often the difference between a pipeline that fits comfortably inside its reporting window and one that has never once had headroom for a bad-data day.

The model's margins deserve the usual scrutiny. Pipelines whose ingest stage re-parses payloads multiple times across stages will find the parsing share larger and the model optimistic; parsing does not accelerate. Pipelines whose aggregation is already delegated to a database or a vectorised library have no interpreted loop to remove and belong outside this model entirely. And the enrichment share rewards pre-hashing: at 15% of runtime, restructuring it to a single hash build is worth more than compiling anything around it.

What a pilot looks like

A contained pilot precedes any change to a filing path. Take the heaviest aggregation function — the windowed totals are the usual candidate — extract it into a standalone module with a zero-argument entrypoint over a representative reporting period, and run the proof sequence on it. The pilot answers the only material question: whether your loop, on your record volumes, measures anywhere near the published anchors. The honest outcomes are the familiar three. Near the anchors, and the pilot proceeds with a correctness record attached. Near 1x, and the aggregation is hash- or library-bound, needing a data-structure fix rather than a compiler. Declined, and the fallback record explains why. In a regulated pipeline, all three outcomes belong in the change log beside the code they describe.

One pilot detail matters here: use a reporting period with its bad data included. Every production period contains late records, malformed fields and restatements, and an aggregation function measured only over clean input is measured over a workload that does not exist. The proof subject should include the awkward rows, labelled, because the compiled path must be correct precisely when the data is not.

Where the engineering judgement sits

The judgement specific to reporting is audit design. A report that is faster but differs from the validated output is a filing defect with a regulator's name on it, which is as expensive as software defects get. The compiled surface must therefore be bounded by the correctness gate as a matter of governance, not preference: both backends returning the identical result, quoted in the run record, with fallback_count and deopt_count at zero so no record passed through uncompiled code unobserved. Regulated teams should store the proof records alongside the filings they support — same discipline as model validation, same reason.

The second judgement is revalidation cadence. Reporting rules change on regulatory schedules; the Python that encodes them changes with them. Each rule change re-opens the measured claim, because the compiled function is new code. The speed-proof method is cheap enough to run per release, and in a regulated pipeline it should be a release gate: no changed aggregation ships to production without a fresh Correct: YES beside it. How to run a speed proof is written to be repeatable for exactly this rhythm.

Proving it on your own reporting batch

python -m pyvorin support your_reporting.py
python -m pyvorin run your_reporting.py --function entry --compare --runs 5 --warmup 2
python -m pyvorin bench your_reporting.py --function entry --runs 7 --warmup 2 --json

Quote only runs showing Correct: YES, COMPILED_FULL, and matched warmup and run counts; report the median of the raw samples with the spread kept in the record; state the one-off compile cost next to the steady-state figure. The harness mechanics are in how to benchmark a function, and the full context for every anchor is on the benchmarks page. Both are cheap enough to run per release, which is exactly the cadence a pipeline feeding regulated filings should keep — a standing proof, re-run on every rule change, costs less than one incident of unexplained drift in a submitted number.

Limits, stated plainly

The anchors come from one host, one date, one suite; the shape of the distribution transfers better than any absolute figure. Parsing, enrichment and formatting will not accelerate and can slow slightly if compiled — the fallback path in unsupported code and the fallback path keeps any declined function correct in the meantime. Workloads whose aggregation runs in a database or vectorised library are outside scope. Deadlines fixed by regulators are hard deadlines: the value of the modelled speedup is headroom against failure, not a licence to cut the reporting window.

Where to go next

• How to run a speed proof — the repeatable, auditable measurement method reporting pipelines need.
• Benchmarks — the full 71-workload table with all seventeen slower rows intact.
• Example workload: ETL pipeline — the same stage-by-stage logic on a generic pipeline.
• Unsupported code and the fallback path — the semantics that keep declined functions correct.

Last reviewed 1 June 2026 against the benchmark artefact dated 13 September 2026 (71 workloads, 54 faster / 17 slower than CPython, ETL category range 0.6x–134.6x). Anchor figures are extracted from that artefact; end-to-end arithmetic is a labelled illustrative model. This page describes an illustrative deployment; it contains no customer claims.