Skip to content

About

Awesome AI Engineer - Curated roadmap and resources from AI Engineer to AI Systems Architect, with 13 production-ready portfolio projects

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

5 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Awesome AI Engineer Awesome

A curated roadmap and resource collection for leveling up from AI Engineer to AI Systems Architect β€” with 18 production-ready portfolio projects, enterprise architecture patterns, and modern 2025–2026 LLM & agent engineering paradigms.

Awesome AI Engineer

Roadmap β€’ Projects β€’ Learning β€’ Agent Frameworks β€’ Structured Outputs β€’ Reasoning & DSPy β€’ Memory & State β€’ AI Gateways β€’ MLOps & Serving β€’ System Design β€’ Guardrails β€’ Realtime & Voice β€’ Edge AI β€’ Books β€’ Contributing


Why This List?

The AI engineering landscape is evolving fast. Beyond basic prompting and wrapper scripts, modern AI systems require rigorous architecture, stateful memory, constrained decoding, test-time compute, resilient gateways, and automated evaluation. This repo cuts through the noise with:

  • A clear career progression from junior AI engineer to enterprise systems architect
  • 18 hands-on portfolio projects covering LLMs, multi-agent systems, RAG, gateways, and protocols
  • Curated 2025–2026 production stack β€” structured outputs, prompt compilers, reasoning models, and realtime agents
  • Enterprise architecture focus β€” production patterns, latency/cost routing, and safety guardrails

πŸ—ΊοΈ Career Roadmap

AI Engineer to Systems Architect Career Progression Roadmap

πŸ“– Click to view Text & Level-by-Level Breakdown
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ AI ENGINEER β†’ SYSTEMS ARCHITECT β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”‚ LEVEL 1: AI Engineer β”‚ β”‚ β”œβ”€β”€ LLM fundamentals (tokenization, generation, fine-tuning) β”‚ β”‚ β”œβ”€β”€ Prompt engineering, structured outputs & JSON schemas β”‚ β”‚ β”œβ”€β”€ RAG pipelines, dense retrieval & vector databases β”‚ β”‚ β”œβ”€β”€ API design for AI services & streaming (SSE/WebSockets) β”‚ β”‚ └── Projects: #1 #2 #5 #7 β”‚ β”‚ β”‚ β”‚ LEVEL 2: Senior AI Engineer β”‚ β”‚ β”œβ”€β”€ Agent architectures (ReAct, Plan-and-Execute, Code Agents) β”‚ β”‚ β”œβ”€β”€ Multi-agent orchestration patterns (Supervisor, Hierarchical) β”‚ β”‚ β”œβ”€β”€ Agentic frameworks (LangGraph, CrewAI, PydanticAI, ADK) β”‚ β”‚ β”œβ”€β”€ Prompt optimization & compilers (DSPy, TextGrad) β”‚ β”‚ β”œβ”€β”€ Streaming voice & real-time AI systems β”‚ β”‚ └── Projects: #3 #4 #6 #8 β”‚ β”‚ β”‚ β”‚ LEVEL 3: AI Platform Engineer β”‚ β”‚ β”œβ”€β”€ MLOps & high-throughput serving (vLLM, SGLang, TensorRT-LLM) β”‚ β”‚ β”œβ”€β”€ AI Gateways & smart routing (LiteLLM, RouteLLM, Portkey) β”‚ β”‚ β”œβ”€β”€ Agent memory systems & persistent state (Letta, Mem0, Zep) β”‚ β”‚ β”œβ”€β”€ Kubernetes for AI workloads & GPU orchestration β”‚ β”‚ β”œβ”€β”€ Observability, tracing & evaluation (LangSmith, Phoenix) β”‚ β”‚ β”œβ”€β”€ Protocol design & tool standards (MCP, A2A, UCP) β”‚ β”‚ └── Projects: #9 #10 #11 #12 #13 β”‚ β”‚ β”‚ β”‚ LEVEL 4: AI Systems Architect β”‚ β”‚ β”œβ”€β”€ Enterprise multi-agent system design at scale β”‚ β”‚ β”œβ”€β”€ Reasoning models, test-time compute & inference scaling β”‚ β”‚ β”œβ”€β”€ Context caching economics & KV-cache optimization β”‚ β”‚ β”œβ”€β”€ Runtime guardrails, security & defense against injection β”‚ β”‚ β”œβ”€β”€ Cross-framework interoperability & model cascading β”‚ β”‚ └── Projects: #14 #15 #16 #17 #18 β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ```

πŸš€ Portfolio Projects

18 production-ready projects covering foundational LLMs to enterprise-scale AI architecture, each equipped with Docker setup, automated tests, CI/CD, and Kubernetes manifests.

Foundation & Agent Workflows (Level 1-2)

# Project What You'll Build Key Skills Repo
1 LLM Playground Interactive tokenization & generation explorer BPE tokenization, sampling strategies, temperature GitHub
2 Customer Support Chatbot Production chatbot with fine-tuning & evaluation LoRA/PEFT, prompt engineering, evaluation metrics GitHub
3 Ask-the-Web Agent Perplexity-like search & synthesis agent LangGraph, tool use, citation generation GitHub
4 Deep Research Multi-strategy reasoning engine Chain-of-Thought, Tree-of-Thought, inference scaling GitHub
5 Image Generation Multi-provider image service Diffusion models, DALL-E, FLUX, Stable Diffusion GitHub
6 Capstone Multi-Agent Supervisor-pattern agent platform Multi-agent orchestration, task decomposition GitHub
7 Agent RAG Hierarchical RAG system Single/multi/hierarchical retrieval, query decomposition GitHub
8 MCP & A2A Protocol implementation MCP SDK, A2A protocol, AgentCards GitHub

Platform, Gateways & Realtime Streaming (Level 3)

# Project What You'll Build Key Skills Repo
9 UCP Merchant Server Commerce protocol platform Checkout state machine, AP2 verification, MCP bindings GitHub
10 UCP Shopping Agent AI shopping assistant Multi-merchant discovery, comparison, LangGraph GitHub
11 Enterprise AI Gateway Resilient multi-provider routing gateway LiteLLM, RouteLLM, semantic caching, rate limiting GitHub
12 Realtime Voice Agent Sub-second bidirectional voice & multimodal agent WebRTC / WebSockets, streaming STT/TTS, tool interrupts GitHub
13 Stateful Memory Agent Tiered memory agent with entity knowledge graph Letta/Mem0 paradigms, episodic recall, semantic store GitHub

Enterprise Systems, Compilers & Governance (Level 4)

# Project What You'll Build Key Skills Repo
14 DSPy Prompt Optimizer Self-improving declarative prompt compiler DSPy MIPROv2, teleprompters, metric optimization GitHub
15 AI Guardrails Firewall Zero-trust GenAI security & injection defense NeMo Guardrails, Llama Guard 3, PII redaction GitHub
16 Compliance Audit Agents Regulatory compliance audit system Microsoft Agent Framework, graph workflows, middleware GitHub
17 Incident Response ADK IT incident orchestrator Google ADK / Genkit, Sequential/Parallel/Loop agents GitHub
18 Contract Lifecycle Crew Contract management platform CrewAI, role-based agents, Flow orchestration GitHub

Quick Start (Any Project)

# Clone any project repository
git clone https://github.com/Skopaq-AI/<project-name>.git
cd <project-name>

# Option 1: Docker (recommended)
docker compose up --build

# Option 2: Local development
pip install -e ".[dev]"
uvicorn src.<package>.main:app --reload --port 8000

# Run test suite
pytest tests/ -v

πŸ“š Foundational Learning

Courses & Tutorials

Resource Description Level
Neural Networks: Zero to Hero Andrej Karpathy's legendary series β€” build GPT from scratch Beginner β†’ Intermediate
fast.ai Practical Deep Learning Top-down practical approach to deep learning Beginner
Stanford CS224N: NLP with Deep Learning Comprehensive NLP & transformer theory Intermediate
Stanford CS229: Machine Learning Andrew Ng's foundational ML course Beginner β†’ Intermediate
DeepLearning.AI Short Courses Bite-sized courses on LangChain, RAG, fine-tuning, agents All levels
Hugging Face NLP Course Hands-on transformers & NLP with the HF ecosystem Beginner β†’ Intermediate
Full Stack LLM Bootcamp Production LLM application development Intermediate
LLM University by Cohere Comprehensive LLM concepts and applications Beginner β†’ Intermediate

Prompt Engineering & Optimization

Resource Description
Anthropic Prompt Engineering Guide Official Claude prompt engineering best practices & context caching
OpenAI Prompt Engineering Guide Official GPT prompt engineering techniques & structured outputs
Prompt Engineering Guide Community-driven comprehensive guide to prompting techniques
DAIR.AI Prompt Engineering Research-backed prompting patterns, few-shot, and reasoning traces

πŸ€– Agent Frameworks & Tools

Core Frameworks

Framework Best For Maintainer Link
LangGraph Complex stateful agent workflows with cycles and persistence LangChain Docs
PydanticAI Type-safe, production-grade agent framework built on Pydantic Pydantic Docs
CrewAI Role-based multi-agent collaboration and hierarchical flows CrewAI Docs
Smolagents Lightweight code-agent framework running secure sandboxed Python Hugging Face GitHub
Google Genkit / ADK Agent Development Kit with enterprise workflows and Firebase integration Google Docs
Microsoft Agent Framework Enterprise agents with middleware pipelines and Azure integration Microsoft GitHub
OpenAI Agents SDK Lightweight agent loops with multi-agent handoffs OpenAI Docs
LlamaIndex Data-centric RAG and document-native agent workflows LlamaIndex Docs
AutoGen Event-driven multi-agent conversation patterns Microsoft Docs
Semantic Kernel Enterprise AI orchestration for .NET, Python, and Java Microsoft Docs

Agent Protocols & Standards

Protocol Purpose Link
MCP (Model Context Protocol) Standardized protocol for connecting AI models to tools and resources Spec
A2A (Agent-to-Agent) Standardized inter-agent communication protocol Spec
UCP (Universal Commerce Protocol) Secure agentic commerce and checkout transaction standard Spec
OpenAI Function Calling Structured tool use standard adopted across major providers Docs

Observability & Tracing

Tool Purpose Link
LangSmith LLM observability, step-level tracing, and dataset evaluations Docs
Arize Phoenix Open-source AI observability, evals, and OpenTelemetry tracing GitHub
Weights & Biases Weave Lightweight AI application tracing, scoring, and evaluation Docs
Braintrust Enterprise evaluation platform, logging proxy, and prompt playground Docs
OpenLLMetry OpenTelemetry-based instrumentation for LLM applications GitHub

πŸ“ Structured Outputs & Constrained Decoding

Deterministic schema enforcement is essential for production agents, APIs, and data extraction pipelines.

Tool Purpose Link
Instructor Structured outputs for LLMs via Pydantic β€” validation, retries, and streaming Docs
Outlines Fast, guided text generation enforcing regex and JSON schemas via CFGs GitHub
Guidance Constrained decoding language controlling generation structure and interleaving GitHub
TypeChat Schema-guided structured output generation using TypeScript definitions GitHub
Jsonformer Bulletproof JSON generation generating only schema-fill tokens GitHub

🧠 Reasoning, Prompt Compilers & Optimization

Modern AI engineering compiles prompts programmatically and leverages test-time search.

Tool / Concept Purpose Link
DSPy Stanford framework for compiling declarative pipelines into optimized prompts & weights Docs
TextGrad Automatic "differentiation" via text feedback for prompt and system optimization GitHub
DeepSeek-R1 Open reasoning models demonstrating reinforcement learning test-time scaling GitHub
Process Reward Models (PRMs) Step-by-step verification and tree-search scoring for complex problem solving Paper
Tree-of-Thought (ToT) Deliberate exploration and heuristic evaluation of reasoning paths GitHub

πŸ’Ύ Agent Memory & State Management

Stateful agents require distinct memory layers: short-term context, working memory, episodic memory, and long-term semantic memory.

Tool Purpose Link
Letta (formerly MemGPT) Stateful agent platform managing working memory and archival storage GitHub
Mem0 Personalized memory layer for AI agents, extracting entity and user preferences GitHub
Zep Long-term memory server providing auto-summarization and knowledge graphs Docs
LangGraph Store Built-in key-value and semantic cross-thread persistence for stateful agents Docs

🌐 AI Gateways, Routing & Cost Optimization

Production AI architectures implement unified gateways for rate-limiting, failover, prompt caching, and cost-aware routing.

Tool Purpose Link
LiteLLM Unified proxy for 100+ LLMs with load balancing, retries, and spend tracking GitHub
RouteLLM Framework for serving and evaluating cost-effective LLM routing GitHub
Portkey Gateway Blazing fast AI gateway with 50+ integrations, semantic cache, and fallbacks GitHub
OpenRouter Multi-provider unified routing and competitive pricing API Site
Helicone Developer observability proxy for caching, rate limiting, and cost analytics Docs

βš™οΈ MLOps, Serving & Infrastructure

The Modern Production AI Systems Architecture Stack

High-Throughput Model Serving

Tool Purpose Link
vLLM High-throughput LLM serving with PagedAttention and prefix caching GitHub
SGLang High-performance serving engine for LLMs/VLMs with RadixAttention GitHub
TensorRT-LLM NVIDIA-optimized LLM inference execution engine GitHub
Triton Inference Server Multi-framework production model serving at scale GitHub
Ollama Lightweight CLI to run open-source models locally with one command GitHub
llama.cpp Ultra-efficient CPU/GPU LLM inference in C/C++ GitHub

Vector Databases & Retrieval Engines

Database Strengths Link
Qdrant Rust-based vector search engine with payload filtering and quantization Docs
Pinecone Serverless managed vector database with hybrid search Docs
Weaviate Open-source vector database with multi-modal capabilities and hybrid search Docs
pgvector PostgreSQL extension enabling vector embeddings in relational DBs GitHub
Milvus Distributed, highly scalable vector database for billion-scale search Docs
ChromaDB Lightweight, open-source embedding database for rapid prototyping Docs

Fine-Tuning & Training

Tool Purpose Link
Unsloth 2-5x faster LLM fine-tuning with 80% less VRAM GitHub
Hugging Face TRL Transformer Reinforcement Learning (DPO, ORPO, GRPO, SFT) Docs
Axolotl Production-ready LLM fine-tuning configured through YAML GitHub
PEFT Parameter-efficient fine-tuning (LoRA, QLoRA, Prompt Tuning) Docs
LitGPT Hackable 20+ LLM pretraining and fine-tuning engine by Lightning AI GitHub

πŸ—οΈ System Design for AI

Architecture Patterns

AI Agent Architecture Patterns in Production

Pattern When to Use Reference / Project Example
Agentic RAG Dynamic multi-step retrieval, query routing & self-correction Agent RAG (#7)
GraphRAG Complex entity-relationship reasoning across document corpora Microsoft GraphRAG
ReAct (Reason + Act) Interleaving thought steps with tool invocations Ask-the-Web (#3)
Supervisor Pattern Central orchestrator routing sub-tasks to specialized domain agents Capstone (#6)
Prompt Caching Long-context apps to reduce latency and API costs by up to 90% Anthropic Prompt Caching
Sequential Pipeline Deterministic ordered multi-step processing Incident Response (#12)
Parallel Fan-Out Concurrent independent agent executions Incident Response (#12)
Human-in-the-Loop Approval gates, validation interrupts & escalation Compliance Audit (#11)
Model Cascading Routing simple queries to SLMs and complex reasoning to frontier models RouteLLM
State Machine Transactional, verifiable multi-step commerce workflows UCP Merchant (#9)

Evaluation & Testing

Tool Purpose Link
RAGAS Automated evaluation framework for RAG pipelines (faithfulness, answer relevancy) Docs
DeepEval Unit testing framework for LLMs with 14+ production metrics GitHub
Promptfoo Automated testing, red-teaming, and regression testing for prompts Docs
Inspect AI Framework for large language model evaluations and safety benchmarks by AISI GitHub

πŸ›‘οΈ Runtime Guardrails & Security

Tool / Framework Purpose Link
NeMo Guardrails Programmable rails for dialogue safety, topical control, and hallucination reduction GitHub
Guardrails AI Real-time output validation, structured assertions, and schema correction GitHub
Llama Guard Fine-tuned safeguard model for input/output safety classification by Meta Docs
Lakera AI Real-time defense against prompt injection, jailbreaking, and data leakage Site
Garak LLM vulnerability scanner identifying hallucination, injection, and security flaws GitHub
OWASP Top 10 for LLMs Standardized threat model for generative AI and LLM application security Docs

πŸŽ™οΈ Realtime Voice & Multimodal Systems

Tool / Standard Purpose Link
LiveKit Agents Realtime voice, video, and multimodal agent framework with WebRTC Docs
OpenAI Realtime API Low-latency, speech-to-speech bidirectional streaming over WebSockets Docs
Faster-Whisper Reimplementation of OpenAI's Whisper model using CTranslate2 for ultra-fast transcription GitHub
ColPali Efficient vision-language retrieval model indexing documents as images GitHub

πŸ“± Edge AI & On-Device Deployment

Tool / Framework Target Link
Apple MLX Optimized machine learning framework designed for Apple Silicon GPUs/Unified Memory GitHub
ExecuTorch PyTorch end-to-end solution for on-device inference on mobile and edge devices GitHub
ONNX Runtime Cross-platform accelerator for high-performance ML inference on edge and cloud Site
Transformers.js Run state-of-the-art transformer models directly in browsers and Node.js via WebGPU GitHub

☁️ Cloud & Kubernetes for AI

Cloud AI Platforms

Provider Key Services Link
Google Cloud Vertex AI, Gemini APIs, Cloud TPUs, GKE Enterprise for AI Docs
AWS Amazon Bedrock, SageMaker HyperPod, Trainium/Inferentia accelerators Docs
Azure Azure OpenAI Service, Azure AI Foundry, AKS with GPU operators Docs

Kubernetes for AI Workloads

Resource Description Link
KServe Serverless model serving standard on Kubernetes Docs
KubeFlow Cloud-native end-to-end ML platform on Kubernetes Docs
Ray on Kubernetes (KubeRay) Distributed ML compute and batch training on K8s Docs
NVIDIA GPU Operator Automated GPU driver, container runtime, and telemetry management in K8s Docs

πŸ“– Books

Essential Reading

Book Author Why It Matters
AI Engineering Chip Huyen Definitive guide to building reliable LLM applications in production
Designing Machine Learning Systems Chip Huyen System design fundamentals from data pipelines to model serving
Build a Large Language Model (From Scratch) Sebastian Raschka Deep understanding of transformer internals and tokenizers
Hands-On Large Language Models Jay Alammar, Maarten Grootendorst Visual, practical patterns for modern LLM applications
Designing Data-Intensive Applications Martin Kleppmann Foundational data architecture principles essential for AI systems architects

πŸ“° Blogs, Podcasts & Newsletters

Resource Type Focus Link
Lil'Log Blog Deep technical surveys on LLMs and generative models by Lilian Weng Blog
Eugene Yan Blog Applied ML systems design, search, and production patterns Blog
Latent Space Podcast / Newsletter The leading technical podcast and community for AI engineers Listen
The Batch Newsletter Weekly AI news and insights curated by Andrew Ng Subscribe
Interconnects Newsletter Nathan Lambert on RLHF, open models, and reasoning architectures Subscribe

🀝 Contributing

Contributions are welcome! Please check out CONTRIBUTING.md for contribution guidelines, table schema formats, and PR rules.


πŸ“Š Star History

Star History Chart


License

CC0

This work is licensed under CC0 1.0 Universal.

About

Awesome AI Engineer - Curated roadmap and resources from AI Engineer to AI Systems Architect, with 13 production-ready portfolio projects

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages