Modern Backend Structure (AI Era)

Python-based AI backend — reference architecture for projects using LLMs, RAG, and agents.

Directory Layout

src/
├── ai/                       # AI Core
│   ├── agents/               # LangChain/CrewAI agents
│   ├── llm/                  # OpenAI/Anthropic/Gemini clients
│   ├── rag/                  # RAG pipeline (retrieval + generation)
│   └── embeddings/           # Vector embeddings
├── api/
│   ├── routes/
│   │   ├── chat.py           # Streaming chat endpoints
│   │   ├── rag.py            # Document search endpoints
│   │   └── agents.py         # Agent management
│   └── websocket/            # Real-time AI streaming
├── core/
│   ├── config.py             # AI provider configs
│   ├── database.py           # PostgreSQL + Vector DB
│   └── cache.py              # Redis for LLM caching
├── domain/                   # Business logic
│   ├── models/               # Pydantic models
│   ├── services/             # AI services
│   └── repositories/         # Data access
├── infrastructure/
│   ├── vector_dbs/           # Pinecone / Weaviate / Qdrant
│   └── monitoring/           # LangSmith, OpenTelemetry
└── workflows/                # AI pipelines
    ├── document_processing.py
    └── chat_orchestration.py

data/                         # Knowledge base for RAG
├── documents/                # Source files
├── chunks/                   # Processed chunks
└── embeddings/               # Vector embeddings

prompts/                      # Prompt management
├── system/
├── user/
└── templates/

tests/
├── ai/                       # LLM & RAG tests
├── integration/
└── e2e/

docker-compose.yml            # Vector DBs, Redis, monitoring
requirements-ai.txt           # AI-specific packages
.env                          # API keys & configs

Key Features This Structure Supports

Notes

• domain/ follows DDD: models → services → repositories
• core/database.py manages both relational (PostgreSQL) and vector store connections
• cache.py (Redis) avoids redundant LLM calls for repeated queries
• requirements-ai.txt kept separate from general requirements.txt to isolate heavy AI dependencies