A localized Retrieval-Augmented Generation (RAG) system designed to analyze technical documentation on-premise, ensuring data privacy and zero cloud dependency.
In corporate environments, technical documents (SOPs, financial reports, and project charters) often contain proprietary information that cannot be processed via public LLM APIs due to compliance (SOC2/GDPR) and security concerns.
I built this system to demonstrate a Privacy-First alternative. By utilizing local hardware to process data, organizations can gain the benefits of AI-driven insights without exposing intellectual property to third-party providers.
- Localized Processing: Powered by Llama 3.1 via Ollama; data never leaves the local machine.
- On-Demand Ingestion: Extracts and analyzes unstructured data from PDF documentation.
- Hybrid-Ready Architecture: Includes a secure configuration for OpenAI integration, allowing for a "Provider-Agnostic" deployment based on data sensitivity.
- Language: Python 3.12
- Model: Llama 3.1 (8B)
- Library Support:
pypdffor extraction,ollamafor local inference. - Environment: Virtualized macOS environment optimized for Apple Silicon.
To demonstrate that the agent is successfully retrieving information from the local knowledge base rather than general training data:
- Ingestion: Place a document in the
/docsfolder (e.g., an internal project update). - Execution: Run
python app.py. - Test Query: Ask a question specific to the document, such as: "What was the stakeholder satisfaction rating mentioned in the 2025 change initiative?"
- Result: The system will return the exact value (e.g., 98%) extracted directly from the PDF text.
- Environment Stability: Successfully mitigated Python 3.14 (experimental) build errors by pivoting to a stable Python 3.12 production environment.
- Security Best Practices: Implemented
.envcredential management and.gitignoreprotocols to prevent accidental exposure of API keys.