LLMs 101: A Comprehensive Introduction to Large Language Models

LLMs are the most talked-about technology in software right now, and most people only use them through chat interfaces like ChatGPT. Earlier this month I gave a talk at the NYC AI Users meetup as a comprehensive introduction to large language models and their ecosystem, and this article is the written version of it.

The Evolution of Natural Language Processing

Before LLMs, computers trying to understand language went through a few eras.

1. Rule-Based

   1950s-1980s

   Handwritten Rules

2. Statistical

   1990s-2012

   n-grams, SVMs

3. Deep Learning

   2013-2018

   Embeddings, RNNs, Attention

4. LLMs

   2019-Present

   BERT, GPT, LLaMA

Rule-based (1950s-1980s). Computers relied on complex, hand-crafted rules to try and understand language, a rule for plurals, another for verb tense, another for every edge case, and they were used for tasks like text classification and information extraction. Natural language has more edge cases than anyone can write down, so the limitations were brittleness and high maintenance.

Statistical (1990s-early 2010s). Text vectorization: sentences are represented as vectors (list of numbers), each number representing a token, and computers began learning patterns directly from massive amounts of text. Limitations: limited understanding of word meaning and sentence order.

Deep learning (2013-2018). Word embeddings: words represented as vectors captured meaning and relationships, allowing computers to understand analogies (“King” - “Man” + “Woman” ≈ “Queen”).

Transformers

Attention mechanism: models ‘attend’ to different parts of a sentence simultaneously, regardless of distance or position. Introduced by Bahdanau et al.[1] in 2014 for machine translation, it significantly improved the performance of Seq2Seq models by improving input to output alignment.

Then came Attention Is All You Need[2] in 2017 and the Transformer revolution: models use attention to improve text understanding and positional encoding to understand sentence order, and they can be trained much faster on specialized hardware, enabling massive growth in model size, and ChatGPT, Claude, and LLaMA are all Transformers. If you want to go deeper on self-attention, I recommend Jay Alammar's The Illustrated Transformer.

How LLMs Work

There are three main stages to training an LLM, and each one costs a fraction of the one before it.

Alignment

Pre-Training

Training on massive text datasets

SFT

Fine-tuning on specific tasks

RLHF

Human feedback as a reward signal

ICL or Fine-Tuning

Prompts, RAG, or more training

Most expensive step (by far)

* Pre-Training: Most expensive step (by far)

Pretraining: the model goes through massive text datasets guessing hidden words in sentences, trillions of times, until it understands language patterns (Common Crawl alone, the open web archive GPT-3 pulled from, is petabytes of text).

Supervised Fine-Tuning: fine-tuning on specific tasks (like question-answering) using labeled data, for LLMs usually instruction tuning so they behave like an assistant.

Reinforcement Learning with Human Feedback (RLHF): refines models by using human feedback as a “reward” signal, which gets them closer to Anthropic’s “helpful, harmless, and honest”.

The ML community has a meme for this, the “Shoggoth with a smiley mask”, with the pre-trained model drawn as a Lovecraftian monster, SFT as a small pink face on one of its tentacles, and RLHF as the smiley mask in front. The polished assistant you talk to is a thin layer on top of a much bigger and weirder pre-trained model.

Next-Word Prediction

Under the hood, LLMs calculate probabilities, trying to guess the next word that makes sense based on the prompt and their “memory” (what they learned in training, the preferences from SFT and RLHF, and the tokens in their context window). For “The sky is”, the model might give “blue” 47%, “clear” 24%, and smaller weights to “usually”, “the”, and “falling”, then it picks one, adds it to the text, and repeats. LLMs build text step by step, one token at a time, continually updating their guesses as they create sentences, and every response you’ve gotten from one was generated this way.

The sky is ▊

blue

47%

clear

24%

usually

12%

the

9%

falling

5%

Tokens vs Words

Tokens are the basic building blocks of LLMs. They’re similar to words, but can also be parts of words, punctuation, and special characters, and they’re selected using a process called tokenization, usually Byte-Pair Encoding (BPE), which iteratively merges the most frequent pairs of characters or character sequences in a large dataset. Common words like “the” are a single token, while “Tokenization” might become “Token” + “ization”, which is why LLMs struggle with counting letters in a word, they never see individual characters. The Tiktokenizer playground shows you how GPT’s tokenizer breaks text apart.

Limitations

Hallucinations. LLMs aim to produce natural language but can generate plausible yet untrue information, as they are not designed to prioritize factual accuracy. Ask one how many ‘m’s are in “Weather” and it might confidently tell you there’s one.

Limited Context Windows. While LLMs work well with conversations or contained summaries, they aren’t able to process very long inputs (in early 2024 GPT-4 Turbo takes 128K tokens and Claude 2.1 takes 200K).

Outdated Knowledge. LLMs may not keep up with the latest events, language usage, slang, and cultural references, affecting their relevance and accuracy, a model trained on data up to April 2023 doesn’t know anything that happened after.

Limited to Language, Not Action. An LLM might write a perfect email, but it can’t send it. It can describe actions to take, but can’t execute them directly.

Working Around the Limitations

Prompt Engineering

Prompts are key, they give LLMs directions, specifying the kind of text and style of output we want. Ask for “Suggest some books.” and you’ll get a generic list of classics, ask for contemporary novels about AI ethics and you’ll get something you’d actually want to read.

Few-Shot Prompting

You can also show the LLM what you want instead of describing it, one example is one-shot prompting, and a handful (usually 2-5) is few-shot prompting, which helps it pick up the task and the pattern of outputs you’re after.

Chain of Thought

Complex tasks can overwhelm LLMs, so chain of thought breaks them down into smaller steps they can follow by asking the model to show its reasoning first. It leads to better accuracy, helps with longer problems, and the answers come out more logical.

Prompt Chaining

Think of prompt chaining as giving LLMs a step-by-step recipe, where the output of one call becomes the input of the next. This unlocks more complex task workflows, longer content, and analysis within text, and humans can be added in the loop to curate intermediary outputs. It’s how I built the talk's demo.

Retrieval Augmented Generation (RAG)

With RAG, LLMs can rely on search-like abilities to find information, provide accurate answers backed by sources, and stay updated beyond their initial knowledge. It uses vector databases (like Chroma or Pinecone) to store sentences and their embeddings, and the relevant ones get added to the prompt as context.

Without RAG

Question

For which club does Lionel Messi play?

Prompt

Answer the following question to the best of your abilities: For which club does Lionel Messi play?

LLM Response

As of my latest update in 2021, Messi plays for Paris Saint-Germain FC.

With RAG

Question

For which club does Lionel Messi play?

Retrieved Context

Messi has signed with Inter Miami CF, joining David Beckham's club until 2025.

source: news article

Prompt

Based on the following context: Messi has signed with Inter Miami CF, joining David Beckham's club until 2025.

Answer the following question: For which club does Lionel Messi play?

LLM Response

Lionel Messi plays for Inter Miami CF.

I used Messi for the example (of course 😄), a model with a 2021 cutoff will tell you he plays for Paris Saint-Germain, but put a news snippet about him signing with Inter Miami in 2023 in the prompt and it gets it right.

Function Calling

Instead of just describing an action, an LLM’s output can become code that acts on other applications or triggers processes. Think summarizing your data directly and then generating charts from it.

Models, Benchmarks, and Tooling

Choosing your path: open source models (LLaMA, Mistral) can be fine-tuned for specific use cases and run yourself, while proprietary models (GPT-4, Gemini, Claude) are easy to access via APIs without major setup efforts.

How to decide which LLM is better? Performance on benchmark datasets. MMLU (Massive Multitask Language Understanding): the benchmark covers 57 subjects across STEM, the humanities, the social sciences, and more. HumanEval: measures functional correctness for synthesizing programs from docstrings. It consists of 164 original programming problems. HellaSwag: a challenge dataset for evaluating commonsense NLI that is especially hard for state-of-the-art models, though its questions are trivial for humans (>95% accuracy). Head-to-head comparison. Chatbot Arena: uses ELO ratings based on head to head crowd-sourced blind testing. As of February 2024, GPT-4 variants hold the top spots, with Gemini Pro, Mistral Medium, and Claude in the top 10.

LLM Frameworks (LangChain, LlamaIndex): provide structure for building LLM-powered applications, streamlining interaction, and chaining different LLM capabilities. Vector Databases (Chroma, Pinecone): specialized databases for storing and searching LLM-generated embeddings. Fine Tuning & Quantization (Axolotl, DeepSpeed): customizing LLMs with new knowledge and behaviors without large scale retraining. Local Runners (Ollama, LLaMa.cpp): tools for running and experimenting with LLMs locally.

Putting It All Together: Super Bowl Ad Brief Generator

For the demo I built something fun, the Super Bowl Ad Brief Generator! You give it a one-sentence product description, like “dog food that makes dogs able to talk”, and it gives you back an ad storyboard. Behind the scenes there are five GPT-4 Turbo calls chained with LangChain (idea, creative brief, creative concepts, script, then storyboard), and DALL-E 3 draws the storyboard panels at the end.

Everything’s in the repo (slides, notebook, and code), and you can run it on Google Colab, all you need is an OpenAI API key.

Use Responsibly!

LLMs inherit biases from their training data, can convincingly generate false information, and might process sensitive user data depending on your application, so be careful how you use them, and don’t take eloquent-sounding text as fact-checked truth.

References

1. Neural Machine Translation by Jointly Learning to Align and Translate — Bahdanau et al., 2014.
2. Attention Is All You Need — Vaswani et al., 2017.