Campus Sensor Monitoring Backend

A high-throughput, asynchronous backend service built with FastAPI and SQLModel to ingest campus sensor readings and evaluate real-time and time-window alert rules.

Instructions for Installation

1. Create and activate a virtual environment:

   python -m venv .venv
   
   # Windows:
   .\.venv\Scripts\activate
   
   # macOS/Linux:
   source .venv/bin/activate

2. Install the required dependencies:

   pip install fastapi uvicorn sqlmodel aiosqlite pydantic greenlet apscheduler httpx

Running the Service

Start the FastAPI server using Uvicorn:

uvicorn main:app --reload

Once running, the interactive API documentation and testing UI is accessible at: http://127.0.0.1:8000/docs

Load-Test Report

To verify the system can handle bursts of 10,000 readings in one second, a custom asynchronous simulator (simulator.py) was built using httpx to simulate multiple sensor gateways firing concurrently.

To run the load test: Ensure the API is running, open a separate terminal, activate the virtual environment, and run:

python simulator.py

Performance Results:

• Total Readings Sent: 10,000
• Time Taken: 0.845 seconds
• Throughput: ~11,834 readings/second
• Successful Gateway Requests: 20 / 20 (100% accepted with HTTP 202 Accepted status)

Architecture & Reliability Trade-offs

1. The Ingestion Queue (Throughput over Absolute Persistence)

Architecture: If the API attempted to write 10,000 readings directly to the database during a single second, the database would lock up and requests would drop. To solve this, the /readings endpoint acts purely as a Producer. It accepts the JSON payload, dumps it into an in-memory asyncio.Queue, and immediately returns a 202 Accepted to the client. A background worker (Consumer) pulls from this queue and performs bulk database inserts. Trade-off: We are trading absolute persistence for massive throughput. Because the queue lives in memory, if the server experiences a hard crash or power loss before the background worker commits a batch to the database, the readings currently in the queue are lost. For high-volume campus sensor data, dropping a fraction of a second of data during a catastrophic crash is preferable to crippling the API under normal burst loads.

2. SQLite WAL Mode

Architecture: Standard SQLite locks the entire database file during a write transaction. To support background bulk-inserts without blocking read queries from the organizer dashboard, Write-Ahead Logging (WAL) mode is explicitly enabled (PRAGMA journal_mode=WAL;). Trade-off: WAL mode requires extra -shm and -wal temporary files on the disk, but it dramatically increases write concurrency and overall read performance.

3. Asynchronous Rule Evaluation

Architecture:

• Single-reading rules are evaluated in memory by the background worker immediately prior to bulk-inserting the batch. This allows the generated alerts and the raw readings to be committed simultaneously in a single transaction.
• Time-window rules (e.g., "10-minute rolling average") are handled by an independent APScheduler background job. Instead of loading thousands of readings into Python memory to do the math, the heavy aggregation is offloaded to the database engine via a highly optimized GROUP BY SQL query, minimizing the Python event loop's payload.