Large language model
Why it matters
An LLM on its own only produces text. That is already useful for a small team: first drafts of emails and proposals, summaries of call notes, sorting inbound enquiries into categories, and extracting fields from messy documents. The time saved goes to the checking and the decisions.
An Autonomous agent is an LLM given a goal and tools (Tool use), so that it can search, read files, send messages and change records. Knowing the engine explains why agents are useful and why they need checks.
It also helps you judge claims. A tool described as "AI-powered" is usually a model plus instructions plus your data. The quality of the last two decides the result more than the brand of the first.
How to apply it
- Give clear instructions and the relevant material. See Prompt engineering and System prompt.
- Supply the facts, for example through Retrieval-Augmented Generation (RAG), instead of relying on its memory for prices, policies or numbers.
- Check the output wherever a mistake is costly, and keep a Human-in-the-loop approval step for actions that cannot be undone.
- Do not paste secrets or customer personal data into a tool unless its data terms allow it.
What it is
An LLM is a program that learned the patterns of language by processing huge amounts of text, such as books, websites and code. Given some text, it produces what is likely to come next, one small piece at a time. These pieces are called tokens and are roughly word fragments. At scale this lets it answer questions, summarise, translate, write code and follow instructions. Claude, GPT and Gemini are examples of model families. A chat assistant is a product built around a model.
Common mistakes
- Treating fluent writing as correct writing.
- Assuming the biggest model is always the best choice. Smaller models are often cheaper and faster for simple tasks.
Strengths and limits
- Strong at drafting, rewriting, summarising, sorting text into categories, pulling facts out of messy text and writing code.
- It does not look things up by default. What it knows was fixed at training time, so it can be out of date.
- It can state wrong things with confidence. This is called Hallucination.
- It can only consider a limited amount of text at once, its Context window.
- It does not remember earlier conversations unless the product stores them and sends them again.
