LLMs in practice

Prompt engineering, chain-of-thought, external memory (RAG)

In this chapter, you will learn to

• Get reliable, well-formatted answers from an LLM
• Make it reason step by step
• Give it access to documents (RAG)

Tools: Hugging Face Transformers

5 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   Plugging an LLM into real software means getting an answer a program can use, and that is true. Without touching its weights: everything goes through what you send it. - Reading documents instead of a human.…

2. Talking to a giant

   You have built the generation machinery (decoding, temperature). Time for a real LLM: without retraining it, we make it perform NLP tasks just by talking to it: prompting. Open the Colab notebook. You will…

3. Taming the Format

   You are plugging an LLM (the K2 model) into your point-of-sale software. A cashier types in free language what they are selling, and your code must recover a clean product, price list to record it. The catch…

4. The external memory

   Your LLM answers questions with confidence, but it sometimes makes up facts. Worse, it knows nothing about anything that happened after its training. A user asks: "What was the score of yesterday's game?": the model…

5. How many passages?

   Your RAG pipeline works. But a crucial question arises: how many documents should you give the LLM? - Too few: you risk missing the crucial piece of information - Too many: you drown the LLM in irrelevant text and…