Real-time object detection from your webcam — FastAPI + Ultralytics YOLO on the back, React + Vite + Tailwind on the front, SQLite underneath.
Detects objects live, tracks them across frames, and counts people, cars and motorcycles crossing a line you draw on the video.
Built as a first practical computer-vision project, so every non-obvious decision is explained in the code and in docs/YOLO_EXPLAINED.md — a ground-up walkthrough of every important YOLO component, written for someone who has not done CV before.
- Live detection — webcam frames stream to the backend over WebSocket and come back annotated, at whatever rate your hardware sustains (see Performance).
- Bounding boxes — drawn in the browser over the live video (smooth, low bandwidth) or server-rendered into the returned JPEG. Toggle between them.
- Object statistics — per-class counts for the current frame, confidence levels, all-time totals.
- FPS + timing breakdown — where every millisecond goes: pre-process, inference, NMS, annotation.
- Object tracking — each object gets a stable ID across frames.
- Virtual counting line — drag a line across the scene; it counts people, cars and motorcycles that cross it, with direction, each counted exactly once.
- Persistent history — every detection is written to SQLite with timestamp, object name and confidence.
Requirements: Python 3.10+, Node 18+, a webcam.
Windows PowerShell users: run each command on its own line. Windows PowerShell 5.1 (the default in VS Code's terminal) does not support
&&for chaining — it fails with «&&» n'est pas un séparateur d'instruction valide. Use;instead, or just run the lines separately as shown below.
cd backend
python -m venv .venvActivate it:
# Windows (PowerShell)
.venv\Scripts\Activate.ps1
# macOS / Linux
source .venv/bin/activateInstall and run:
pip install -r requirements.txtuvicorn app.main:app --reload --port 8000If activation gives you trouble on Windows (execution policy, wrong shell), skip it entirely and call the venv's Python directly — this always works:
.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload --port 8000The first start downloads yolov8n.pt (~6 MB) into backend/models/. Wait for:
Model loaded in 2.77s (80 classes, task=detect)
Ready on http://0.0.0.0:8000 (device=cpu)
In a second terminal:
cd frontend
npm installnpm run devhttp://localhost:5173 → Start camera → Start detection.
The browser will ask for camera permission.
getUserMediaonly works in a secure context — HTTPS orlocalhost. Video is sent only to your own backend; nothing leaves your machine.
docker compose up --buildThen open http://localhost:8080.
The first build downloads PyTorch and takes several minutes. The compose file pulls the CPU-only PyTorch wheel (~200 MB instead of ~2.5 GB with CUDA), and pre-downloads the model weights at build time so the container starts fast and works offline.
docker compose logs -f backend # follow logs
docker compose down # stop
docker compose down -v # stop and delete the databaseNamed volumes persist the SQLite database and the model cache across rebuilds.
yolo-app/
├── backend/
│ ├── app/
│ │ ├── main.py app factory, startup/shutdown ordering
│ │ ├── core/ config (pydantic-settings), logging
│ │ ├── api/routes/ health, detect (REST), stream (WS), stats
│ │ ├── services/ detector, tracker, line counter, annotator,
│ │ │ fps, session, writer, pipeline
│ │ ├── db/ engine, ORM models, queries
│ │ └── schemas/ the wire contract
│ ├── tests/ 116 tests
│ ├── requirements.txt
│ └── Dockerfile
├── frontend/
│ ├── src/
│ │ ├── hooks/ useWebcam, useDetectionStream, useStats
│ │ ├── components/ VideoStage, ControlPanel, stats panels
│ │ ├── lib/ API client, config, formatting
│ │ └── types/api.ts mirrors the backend schemas
│ ├── nginx.conf production proxy for /api and /ws
│ └── Dockerfile
├── docs/
│ ├── YOLO_EXPLAINED.md ← start here if YOLO is new to you
│ └── ARCHITECTURE.md how and why the pieces fit together
├── data/ SQLite database (created on first run)
└── docker-compose.yml
Copy backend/.env.example to backend/.env and edit. Everything has a
sensible default — the file documents each option.
The settings you are most likely to change:
| Variable | Default | Effect |
|---|---|---|
MODEL_NAME |
yolov8n.pt |
yolov8s.pt = more accurate, ~2× slower |
IMAGE_SIZE |
640 |
320 ≈ 2.4× faster, misses small objects |
DEVICE |
auto |
cpu, cuda, mps |
CONFIDENCE_THRESHOLD |
0.35 |
higher = fewer false positives |
IOU_THRESHOLD |
0.45 |
NMS duplicate removal aggressiveness |
COUNTING_CLASSES |
person,car,motorcycle |
what the line counts |
Confidence, IoU, frame rate and class filter are also adjustable live in the dashboard sidebar — changing them while watching the video is the fastest way to build intuition for what they do.
The default install is CPU-only. For an NVIDIA GPU:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121then DEVICE=cuda and HALF_PRECISION=true. Expect an order-of-magnitude
speedup — inference stops being the bottleneck entirely.
Measured on the development machine (8-core CPU, no GPU, yolov8n.pt). Two
columns, because the difference matters: inference on an idle machine is
roughly twice as fast as inference while you are actually using the dashboard,
with a browser encoding frames and a dev server running alongside it.
IMAGE_SIZE |
Inference (idle) | Inference (dashboard running) |
|---|---|---|
| 320 | ~31 ms | ~53 ms |
| 416 | ~46 ms | ~109 ms |
| 640 | ~74 ms | ~145 ms |
End-to-end through the browser at 640px — capture, JPEG-encode, upload, detect, respond — lands around 4–7 FPS on this machine. Benchmark numbers you see quoted for YOLO are almost always the idle-machine kind; treat them as a ceiling, not a forecast.
Expect wide variance on a loaded machine: inference at 640px ranged from 138 ms to 982 ms in the same run depending on what else was competing for CPU.
Inference dominates; everything else (decode, NMS, drawing) is under 10 ms
combined. If it feels slow, lower IMAGE_SIZE first — compute scales with
area, so halving it is roughly a 4× speedup.
The dashboard shows the full breakdown per frame, so you can verify this rather than guess.
cd backend
pytest116 tests, ~18 seconds, no model download needed (the API tests use a stub detector). They cover the parts most likely to be silently wrong:
- Tracker — identity stability, class-consistent matching, occlusion survival, fast-object association.
- Counting line — no double counting, correct rejection of objects passing beyond the line's endpoints, direction, per-class filtering.
- FPS meter — sliding-window averaging, staleness.
- API — every endpoint, plus malformed input and WebSocket error recovery.
- Persistence — detections actually reach SQLite.
- End-to-end — a car driven across the counting line through the real pipeline is counted exactly once, keeps one track ID, and lands in the database.
Integration tests against the real weights:
pytest -m slowwebcam ──▶ canvas ──▶ base64 JPEG ──WebSocket──▶ FastAPI
│
decode ▶ DETECT ▶ TRACK ▶ COUNT ▶ ANNOTATE
│ │ │
└── queued batch writes ──▶ SQLite
│
boxes + stats + FPS ◀────────────────────────────┘
Three decisions worth knowing about:
Backpressure. A webcam produces 30 fps; a CPU handles ~13. The client sends one frame and waits for its result before capturing the next. Sending on a fixed timer would build a queue and the video would drift further behind reality every second. This way latency stays at exactly one frame.
Tracking is separate from detection. YOLO is stateless — it does not know the car in frame 51 is the car from frame 50. Counting needs identity, so a per-session tracker assigns stable IDs. Without it, one car crossing the line would be counted once per frame.
Writes are batched. 25 FPS × 8 objects = 200 INSERTs/second. Committing each inside the request would dominate the frame budget, so rows go onto a bounded queue and a background task flushes them in batches.
The full reasoning is in docs/ARCHITECTURE.md.
Interactive docs at http://localhost:8000/docs.
curl -X POST http://localhost:8000/api/detect \
-F "file=@photo.jpg" \
-F "confidence=0.4"Get an annotated image straight back:
curl -X POST http://localhost:8000/api/detect/image -F "file=@photo.jpg" -o out.jpgKey endpoints: POST /api/detect, WS /ws/detect, GET /api/stats/summary,
GET /api/stats/detections, PUT /api/detect/sessions/{id}/line. Full table in
docs/ARCHITECTURE.md.
"Cannot reach the backend" — the model takes a few seconds to load on first
start. Check http://localhost:8000/api/health shows "model_loaded": true,
then reload the page.
Camera permission denied — allow it in the browser's site settings. The page
must be on localhost or HTTPS; plain HTTP to another machine will not work.
Camera in use by another application — close Zoom/Teams/OBS and retry.
Low FPS — set IMAGE_SIZE=320, lower the max frame rate slider, or restrict
the class filter. See Performance.
Boxes flicker — the confidence threshold is close to the model's score for
those objects. Lower it, or raise TRACK_MAX_AGE so tracks coast through gaps.
One object counted several times — track identity is breaking. Raise
TRACK_MAX_AGE or TRACK_MAX_CENTER_DISTANCE, or send frames faster so objects
move less between them.
Nothing appears in the history table — only detections at or above
DB_MIN_CONFIDENCE (default 0.50) are persisted. Lower it to store more.
This is a learning project, deliberately scoped. Before deploying anywhere real
you would need: authentication (every endpoint is open, including
DELETE /api/stats/detections), rate limiting, HTTPS, and — if scaling past one
worker — shared session state, since it is currently in-memory per process.
- docs/YOLO_EXPLAINED.md — every important YOLO component from first principles: backbone/neck/head, confidence, IoU and NMS, letterboxing, tracking, and the mistakes that are easy to make.
- docs/ARCHITECTURE.md — system design and rationale.
- Ultralytics documentation
The code in this repository is MIT licensed.
One dependency is not: Ultralytics YOLO is AGPL-3.0, a copyleft licence that extends to network use — deploying a derivative of this project as a public service means making the source available to its users. Ultralytics sells a commercial licence for cases where that does not work.
Full breakdown in NOTICE.md. Worth two minutes before you build anything commercial on top of this.