AI system foundations

6 topics.

• 01FoundationReadChoosing the right approachPlain code, a prompt, RAG, fine-tuning or an agent: pick the cheapest one that does the job.
• 02Mid-levelReadHow LLM inference worksPrefill, decode and the KV cache explain almost every latency and cost number you will see.
• 03Mid-levelReadEmbeddings and vector searchTurn meaning into vectors, then find the nearest ones in milliseconds without checking them all.
• 04Mid-levelReadRetrieval-augmented generationGive the model the right pages at the right moment, and know the ways it still gets it wrong.
• 05Mid-levelReadEvaluation and guardrailsYou cannot improve what you cannot score: eval sets, judges, and filters that hold in production.
• 06SeniorReadRecommendation and rankingShrink a billion items to the ten a user sees, in two stages and under 200 ms.

More in AI systems

• Agents
• Build it yourself
• Problems
• AI systems overview