What Is Pyvorin Edge?

Introduction: The Edge Computing Problem

Every day, billions of IoT sensors, industrial gateways, and smart devices generate enormous volumes of data. Traditionally, that data is sent straight to the cloud for processing, storage, and analysis. This approach works, but it creates three expensive problems:

• Cloud egress costs. Moving data out of devices and into cloud data centres costs money. On AWS, outbound data transfer can cost £0.07 per GB or more. A factory with 500 sensors sampling every second can easily generate terabytes per month. At scale, these bills become unsustainable.
• Latency. If your leak detector sends a reading to a cloud server 200 milliseconds away, then waits for a cloud function to process it and send an alert back, you have already lost precious time. For safety-critical systems, every millisecond counts.
• Privacy and compliance. Sending raw sensor data to the cloud may violate GDPR, industry regulations, or internal data policies. Personal identifiable information, precise location data, or sensitive industrial telemetry should not leave the premises unfiltered.

Pyvorin Edge solves all three problems by moving computation as close to the source as possible: directly onto the gateway device sitting next to your sensors.

What Is Pyvorin Edge?

Pyvorin Edge is a lightweight Python runtime and software development kit (SDK) designed for ARM64 edge gateways, particularly the Raspberry Pi 4 and 5. It runs on-device, ingests data from local sensors, processes that data through privacy-aware pipelines, compiles hot paths to native ARM64/NEON code for speed, and only synchronises the minimum necessary data to the cloud.

Think of it as a miniature data centre that lives in your cupboard, plant room, or equipment rack. It is small enough to run on a £50 computer, yet powerful enough to handle hundreds of sensors, enforce complex privacy rules, and compile Python logic into machine code that rivals C performance.

The Three Pillars

Pyvorin Edge is built on three pillars that directly address the problems above:

1. Local Processing and Compilation

Instead of sending raw readings to the cloud, Pyvorin Edge processes them on the gateway itself. It can calculate rolling averages, detect anomalies, evaluate event rules, and aggregate time windows, all without a network round-trip.

For performance-critical paths, Pyvorin Edge uses a compiler bridge that translates Python functions into native shared libraries (.so files) optimised for ARM64 with NEON vector instructions. In plain English: your Python code gets turned into fast machine code that the Pi's CPU can execute directly, bypassing the normal Python interpreter overhead.

# A simple rule condition written in Python
def check_temperature(reading):
    return reading.value > 30.0

# Pyvorin Edge can compile this to a native .so library
# via the CompilerBridge, making it run at near-C speed.
from pyvorin_edge.compiler_bridge import CompilerBridge

bridge = CompilerBridge()
bridge.compile_hotpath(
    source_code="def check_temperature(v): return v > 30.0",
    function_name="check_temperature",
    output_path="/home/pi/pyvorin-edge/kernels/temp_check.so"
)
  

2. Privacy-First Design

Pyvorin Edge includes a privacy firewall that evaluates every sensor reading against configurable rules before it is stored or forwarded. Rules can drop sensitive fields, mask values, or hash identifiers.

# From pyv_edge_agent.privacy — how rules work under the hood
from pyv_edge_agent.privacy import PrivacyPolicy, PrivacyRule

policy = PrivacyPolicy(
    enabled=True,
    rules=[
        PrivacyRule(sensor_pattern="camera_*", action="drop"),
        PrivacyRule(sensor_pattern="room_*", action="mask"),
        PrivacyRule(sensor_pattern="badge_id", action="hash"),
    ],
    default_action="allow"
)

# A reading from a camera is dropped entirely
reading = SensorReading(sensor_name="camera_01", timestamp=1717000000.0, value=1.0)
result = policy.evaluate(reading)  # Returns None — reading is discarded
  

3. Store-and-Forward Cloud Sync

When the cloud does need data, Pyvorin Edge does not send it immediately. Instead, it places readings into a persistent SQLite-backed queue with priority levels. The queue batches items, retries on failure, and respects time-to-live (TTL) limits so that stale data does not clog the pipe.

# From pyv_edge_agent.cloud_sync.queue
from pyv_edge_agent.cloud_sync import CloudSyncQueue, Priority

queue = CloudSyncQueue(db_path="/home/pi/pyvorin-edge/sync_queue.db")
item_id = queue.enqueue(
    payload={"sensor": "temp_01", "value": 22.5},
    priority=Priority.TELEMETRY,
    ttl_seconds=86400
)
  

Core Concepts

To use Pyvorin Edge effectively, you need to understand five core concepts. These map directly to modules in the source code.

Concept 1: The Gateway

The gateway is the physical device. Usually this is a Raspberry Pi 4 or 5 running Raspberry Pi OS or Ubuntu 24.04. The gateway sits on your local network, reads from sensors via GPIO, USB, MQTT, HTTP webhooks, or file replay, and runs the Pyvorin Edge Agent.

Concept 2: The Pipeline

A pipeline is a declarative description of how data flows from sensors through windows, rules, and policies to storage and cloud sync. In code, a pipeline is an instance of pyvorin_edge.pipeline.Pipeline.

from pyvorin_edge.pipeline import Pipeline, WindowConfig, RuleConfig

pipeline = Pipeline(name="factory_floor")
pipeline.add_source({"type": "mqtt", "broker": "192.168.1.50"})
pipeline.add_window(WindowConfig(duration_seconds=60.0, window_type="rolling"))
pipeline.add_rule(RuleConfig(name="overheat", condition_expr="ctx.value > 35.0"))
  

Concept 3: The Compiler

The compiler bridge (pyvorin_edge.compiler_bridge.CompilerBridge) is the component that transforms Python logic into native ARM64 machine code. It analyses your Python source using the abstract syntax tree (AST), decides whether a workload is "scalar" (single values) or "kernel" (array reductions), and routes it to the appropriate backend.

For scalar workloads, it uses PyvorinCompiler. For kernel workloads involving loops and array operations, it uses CKernelBackend, which generates C code with NEON intrinsics. The result is a .so shared library that Python loads via ctypes.

Concept 4: The Privacy Filter

The privacy filter (pyv_edge_agent.privacy.PrivacyPolicy) acts like a firewall for data. Every reading passes through it. Rules use Unix glob patterns (sensor_*, camera_*) to match sensor names, and actions include:

• allow — pass the reading through unchanged.
• drop — discard the reading entirely.
• mask — replace the value with 0.0 and mark the unit as "masked".
• hash — replace the sensor name with a SHA-256 hash so the source is anonymised but trends remain traceable.

Concept 5: Cloud Sync

Cloud sync is the process of uploading aggregated, privacy-filtered data to a remote endpoint. The CloudSyncQueue persists items to SQLite so that reboots do not cause data loss. The HTTPCloudUploader posts batches as JSON over HTTPS. If the upload fails, items are nack'd with exponential backoff.

How ARM64/NEON Compilation Works in Plain English

Most Python code runs in an interpreter. The interpreter reads your code line by line, translates it into bytecode, and the Python virtual machine executes that bytecode. This is flexible but slow compared to languages like C or Rust.

Pyvorin Edge bypasses the interpreter for hot paths. Here is what happens step by step:

1. Parse. The compiler reads your Python function and builds an abstract syntax tree (AST), a structured representation of the code.
2. Analyse. It looks for loops, list comprehensions, and calls to functions like sum, min, max. If it finds array-style operations, it classifies the workload as a "kernel".
3. Plan. For kernels, it builds a TypedKernelPlan that describes input arrays, scalar parameters, the operations to perform, and the output shape.
4. Generate. The C kernel backend turns the plan into C source code that uses NEON intrinsics. NEON is ARM's single-instruction-multiple-data (SIMD) extension, which means one CPU instruction can process multiple numbers at once.
5. Compile. A C compiler (usually GCC or Clang) compiles the C source into a .so shared library for ARM64.
6. Load. Python loads the .so via ctypes and calls it like a normal function.

The result is that a Python function that might take 50 milliseconds to sum a million numbers in pure Python can take less than a millisecond when compiled to NEON-optimised machine code. On a Raspberry Pi, where CPU cycles are precious, this matters enormously.

Architecture Diagram (Text)

  +------------------+     +------------------+     +------------------+
  |   Sensors        |     |   Edge Agent     |     |   Cloud          |
  |   (MQTT/HTTP/    | --> |   (Pyvorin Edge) | --> |   Endpoint       |
  |   Simulator)     |     |                  |     |   (HTTPS)        |
  +------------------+     +------------------+     +------------------+
            |                       |
            v                       v
    +---------------+      +------------------+
    | Ingest        |      | Compiler Bridge  |
    | Adapters      |      | (.so generation) |
    +---------------+      +------------------+
            |                       |
            v                       v
    +---------------+      +------------------+
    | Ring Buffers  |      | Privacy Firewall |
    | (per device)  |      | (drop/mask/hash) |
    +---------------+      +------------------+
            |                       |
            +-----------+-----------+
                        v
              +------------------+
              | SQLite Store     |
              | (local evidence) |
              +------------------+
                        |
                        v
              +------------------+
              | Cloud Sync Queue |
              | (batch & retry)  |
              +------------------+
  

When Should You Use Pyvorin Edge?

Pyvorin Edge is ideal for the following scenarios:

• Smart buildings and facilities. Temperature, humidity, occupancy, and energy monitoring where data must stay on-site for GDPR or tenant privacy.
• Industrial IoT. Vibration, current, and pressure sensors on manufacturing lines where sub-second anomaly detection prevents costly downtime.
• Agriculture and remote sites. Locations with intermittent connectivity where store-and-forward queuing is essential.
• Cost-sensitive deployments. Startups and SMEs that cannot justify £500+ per month in cloud ingress and compute fees for basic telemetry.

Summary

Pyvorin Edge is a gateway-native Python runtime that processes sensor data locally, compiles performance-critical logic to ARM64/NEON machine code, enforces privacy rules before any data leaves the device, and synchronises only what is necessary to the cloud. By doing so, it slashes egress costs, reduces latency to near zero, and keeps sensitive data under your physical control.

In the next articles, we will cover system requirements, installation, and your first working pipeline.