Edge Project Structure

Introduction

After installing Pyvorin Edge, your gateway's disk contains more than a single Python script. There are configuration files, databases, log directories, compiled kernel caches, and the virtual environment itself. This article maps every directory and file you will encounter, explains its purpose, and shows you which files are safe to edit, back up, or delete.

Top-Level Directory Layout

Assuming you installed into /home/pi/pyvorin-edge, the directory tree looks like this after a typical installation and a few hours of runtime:

/home/pi/pyvorin-edge/
├── .venv/                          # Python virtual environment
│   ├── bin/
│   │   ├── python3 -> ...
│   │   ├── pip
│   │   └── pyv-edge               # CLI entry point
│   ├── lib/python3.12/site-packages/
│   │   ├── pyv_edge_agent/        # Edge runtime source
│   │   ├── pyvorin_edge/          # Edge SDK source
│   │   └── ...
│   └── pyvenv.cfg
├── config/
│   └── (optional extra configs)
├── config.toml                     # Main agent configuration
├── data/
│   ├── edge_store.db               # Local SQLite evidence store
│   ├── edge_store.db-shm           # SQLite WAL shared-memory file
│   ├── edge_store.db-wal           # SQLite WAL journal
│   └── sync_queue.db               # Cloud sync persistent queue
├── logs/
│   └── (optional file logs)
├── cache/
│   └── __pycache__/                # Compiled Python bytecode
├── kernels/
│   └── temp_check.so               # Example compiled native library
└── demo_pipeline.py                # Your custom scripts (user-created)
  

Directory by Directory

.venv/ — The Virtual Environment

This directory is a self-contained Python installation. It isolates Pyvorin Edge and its dependencies from the system Python packages. This prevents version conflicts and makes uninstallation as simple as deleting the directory.

Key Subdirectories

What Lives in site-packages/pyv_edge_agent/

pyv_edge_agent/
├── __init__.py
├── main.py              # EdgeAgent daemon and CLI
├── config.py            # Config loader with env substitution
├── privacy.py           # PrivacyPolicy and PrivacyRule
├── types.py             # SensorReading dataclass
├── health_monitor/
│   ├── __init__.py
│   ├── metrics.py       # SystemMetrics collector
│   └── watchdog.py      # Software watchdog
├── buffers/
│   ├── __init__.py
│   ├── ring_buffer.py   # Thread-safe RingBuffer
│   └── typed_arrays.py  # Typed array helpers
├── cloud_sync/
│   ├── __init__.py
│   ├── queue.py         # CloudSyncQueue (SQLite-backed)
│   ├── uploader.py      # HTTPCloudUploader
│   ├── retry.py         # Retry policies
│   └── http_client.py   # Low-level HTTP utilities
├── ingest/
│   ├── __init__.py
│   ├── simulator_adapter.py  # SimulatorAdapter, SensorConfig
│   ├── mqtt_adapter.py       # MQTTAdapter
│   ├── http_adapter.py       # HTTPAdapter
│   └── file_replay_adapter.py # FileReplayAdapter
├── local_store/
│   ├── __init__.py
│   ├── sqlite_store.py  # SQLiteStore with pooling
│   └── schema.sql       # Database schema
├── module_host/
│   ├── __init__.py
│   ├── abi.py           # ABIContract for .so modules
│   ├── loader.py        # ModuleLoader (ctypes)
│   └── fallback.py      # Fallback Python implementations
├── privacy_firewall/
│   ├── __init__.py
│   ├── policy.py
│   ├── minimiser.py
│   └── audit.py
└── reports/
    ├── __init__.py
    └── reduction_report.py
  

What Lives in site-packages/pyvorin_edge/

pyvorin_edge/
├── __init__.py
├── pipeline.py          # Pipeline, WindowConfig, RuleConfig
├── sensors.py           # Sensor, SensorRegistry
├── windows.py           # RollingWindow, TumblingWindow
├── policies.py          # CloudPolicy, PrivacyPolicy, EventRule
├── reports.py           # ReportBuilder
├── compiler_bridge.py   # CompilerBridge, KernelOp
├── cost_model.py        # Egress cost estimation
├── attestation.py       # Binary attestation
├── packaging/
│   ├── __init__.py
│   ├── signer.py
│   ├── verifier.py
│   └── legacy.py
└── licensing/
    ├── __init__.py
    └── stub.py
  

config/ — Extra Configurations

The config/ directory is optional. Some users prefer to split their configuration into multiple TOML files and reference them from the main config.toml using environment variable substitution:

# config.toml
[cloud]
endpoint = "${CLOUD_ENDPOINT}"
api_key = "${CLOUD_API_KEY}"
  

# Load secrets from a separate file
export $(cat /home/pi/pyvorin-edge/config/secrets.env | xargs)
  

config.toml — The Main Configuration File

This is the single most important file in your deployment. It is read at startup and optionally reloaded at runtime via agent.reload_config(). The Config class merges your settings over built-in defaults, so you only need to specify values that differ from the defaults.

Valid formats: TOML (preferred), JSON, or plain text that parses as either.

Safe to edit? Yes, but you must restart the agent (or call reload) for changes to take effect.

Safe to back up? Absolutely. Version-control it with Git if possible.

data/ — SQLite Databases

This directory contains the persistent state of your edge gateway. Treat it like production data.

edge_store.db

The evidence store. Schema includes:

• readings — every sensor reading ever collected (unless purged).
• events — rule-triggered events with severity and message.
• summaries — time-window aggregations.
• audit_log — administrative actions.
• schema_version — migration tracking.

WAL mode is enabled, so you will also see edge_store.db-shm and edge_store.db-wal. These are normal and required for WAL operation. Do not delete them while the agent is running.

sync_queue.db

The cloud sync queue. Each item is a JSON payload with a priority level, timestamp, TTL, and retry count. Because it is SQLite-backed, queue items survive reboots. If the cloud endpoint is unreachable, the queue grows until storage limits or TTL expiry.

logs/ — Log Files

By default, the agent logs to stdout in JSON format. If you configure file logging (by wrapping the agent with a process manager like systemd or by adding a FileHandler in a custom wrapper), this is where logs should go. The directory is user-created; the agent itself does not write here unless configured to do so.

cache/ — Compiled Bytecode and Manifests

Python writes __pycache__ directories here after importing modules. You can safely delete cache/__pycache__ to force recompilation, but there is usually no reason to do so.

In future versions, this directory may also hold pipeline manifest JSON files and compiled deployment bundles.

kernels/ — Native Compiled Libraries

When you use the compiler bridge to turn Python functions into .so shared libraries, the output files are written here by convention. Each .so is a standalone ARM64 binary that Python loads via ctypes.CDLL.

from pyvorin_edge.compiler_bridge import CompilerBridge

bridge = CompilerBridge()
bridge.compile_hotpath(
    source_code="def f(x): return x * 2",
    function_name="f",
    output_path="/home/pi/pyvorin-edge/kernels/double.so"
)
  

Safe to delete? Yes, but you will need to recompile. These are build artifacts, not data.

Runtime File Paths

When the agent starts, it resolves paths relative to the current working directory unless you specify absolute paths in config.toml. This is important if you run the agent from different directories.

# Relative path — resolves to CWD/store/edge_store.db
[store]
path = "store/edge_store.db"

# Absolute path — always the same regardless of CWD
[store]
path = "/home/pi/pyvorin-edge/data/edge_store.db"
  

Best practice: always use absolute paths in production configurations to eliminate ambiguity.

What to Back Up

What to Exclude from Git

If you version-control your project, add these to .gitignore:

# .gitignore for a Pyvorin Edge project
.venv/
data/*.db
data/*.db-shm
data/*.db-wal
logs/*.log
logs/*.jsonl
cache/__pycache__/
kernels/*.so
*.pyc
  

Summary

The Pyvorin Edge project structure separates code (.venv/), configuration (config.toml), data (data/*.db), logs (logs/), caches (cache/), and compiled artifacts (kernels/). Understanding this layout lets you back up effectively, troubleshoot file-related issues, and keep your deployment organised.