1. How AI is Trained
Training large language models is a pipeline with three distinct phases. Understanding these phases changes how you evaluate models, interpret their failures (hallucinating, refusing, inconsistency), and make smarter deployment decisions.
Pre-training: This creates a base model built on a transformer architecture. The input is a massive corpus of web crawls, books, and code. To put this in perspective, the entire English Wikipedia is roughly 3 billion tokens—a rounding error at this scale. Because it solves for next-token prediction across human writing, it internalizes logic. This requires tens to hundreds of millions of dollars and months of GPU time.
Fine-tuning: Makes the base model useful. Through Instruction Tuning and Reinforcement Learning from Human Feedback (RLHF), human raters compare outputs so a reward model learns human preference. This optimization target differs from raw prediction. Note: Fine-tuning shapes behavior but does not update knowledge (which requires Retraining or RAG).
Alignment: Makes the model safe and honest. Anthropic's Constitutional AI trains the model to critique and revise its outputs against written principles. A major side-effect is over-refusal. Furthermore, models have a static knowledge cutoff; they determine confidence based on training data volume, not ground truth, which often leads to them being confidently wrong.
Architect Perspective
Design systems assuming the model has a static knowledge cutoff. You must architecture RAG (Retrieval-Augmented Generation) pipelines to inject live retrieval data at inference time to supplement frozen knowledge.
2. Neural Network & Backpropagation
Tokens map to integer IDs, which project into dense numerical vectors called embeddings. These form the input to a neural network—layers of interconnected neurons acting as a function approximator.
The Learning Loop
- Forward Pass: The neuron multiplies inputs by its weight, adds a bias, and passes it through an activation function to introduce nonlinearity. Without nonlinearity, stacking layers collapses into a single linear transformation.
- Loss Function: Measures the error between the prediction and the correct answer.
- Backpropagation: Applies calculus chain rules backward to compute a gradient for each weight.
- Optimizer: Applies gradients via gradient descent. A learning rate that is too high causes instability; too low stalls convergence.
Gradient Stability Interventions
Backpropagation requires differentiable operations. When networks scale deeply, two failure modes occur:
- Vanishing Gradients: Gradients shrink to zero, stalling early layers. Fixed by replacing hard step functions with smooth ReLU activations.
- Exploding Gradients: Handled structurally via Residual Connections and Batch Normalization to stabilize gradient flow.
3. The Big Picture: How AI Learns (Backpropagation)
Imagine you are trying to teach a student (the AI model) a new skill by showing them examples. To learn, the student has to look at an answer, see how wrong they were, and then adjust their understanding for the next time. This process of figuring out how to adjust is called Backpropagation.
Visualizing Gradient Failure Modes
1. The Problem: Everything Must Be Smooth
For the AI to learn, the math inside it needs to flow smoothly, like water flowing gently, not jumping abruptly.
- Differentiable Operation: Think of this as needing a smooth ramp or slope on a hill. If you try to use a "hard step" (like an on/off switch—if the input is 0, output is 0; if input is 1, output is 1), the math breaks down when trying to learn.
- The Fix: We use smooth functions (like ReLU) that create a gentle slope. This allows the error signal to flow back and forth smoothly through all layers.
2. The Failure Modes: Gradient Problems
When the learning process runs through many layers, the "error signal" or "gradient" can get messed up in two ways:
A. Vanishing Gradients
Imagine passing a message down a very long chain of people. If each person reduces the message slightly every time they pass it on, by the time it reaches the start of the line, it is almost zero. Result: The initial layers never get strong enough signals to learn.
B. Exploding Gradients
Imagine every person in the chain decides to double the message. The signal doesn't fade; it gets ridiculously huge, exploding into infinity! Result: This massive error signal completely overwhelms the learning process and breaks the math.
3. The Solutions: Keeping Things Stable
We need special tools built into the AI structure to act like traffic controllers to keep the learning signal steady:
- Residual Connections & Batch Normalization: Checkpoints that constantly check the flow of information and adjust it so the gradient stays in a healthy range.
- Careful Weight Initialization: Setting up the initial starting values sensibly so the learning process has a better chance of staying stable.
Summary in One Sentence:
The goal of these techniques is to ensure that the learning signal (the gradient) flows smoothly and stays at a useful strength, no matter how deep the AI network gets.
4. Prompt Engineering
Prompt engineering is a deliberate design skill, not a shortcut. It structures inputs to guide the model toward reliable, verifiable outputs instead of generic baselines.
Zero Shot
Asking directly. Yields a generic, technically correct baseline.
Role Prompting
Assigns a persona (e.g., Senior Engineer) anchoring perspective and vocabulary depth.
Few-Shot
Provides Q&A examples. Enforces strict format, tone, and output length.
Chain of Thought
Adds "think step by step". Surfaces intermediate logic, making answers verifiable.
Self-Consistency
Runs prompt multiple times and uses majority voting to improve reliability.
Tree of Thoughts
Explores reasoning paths simultaneously, pruning weak approaches to synthesize the best.
ReAct
Interleaves reasoning with tool use (thought, action, observation loops) for autonomous agents.
Prompt Chaining
Decomposes tasks; one output feeds into the next prompt iteratively.
5. Tokenization
Computers process numbers, not text. Text is converted to unique IDs via Byte Pair Encoding (BPE), which iteratively merges frequent adjacent characters into single tokens to build the vocabulary (typically 30k-100k tokens).
- Roughly 75 words equal 100 tokens for English prose.
- Multilingual prose (Thai, Arabic) and code cost more tokens because they are underrepresented in training.
- Tokens are the unit of attention. The model cannot spell words character-by-character natively because tokens do not preserve character-level logic.(Ask a model to count the letters in a word, or to reverse a string character by character, and it will often fail.)
Every model has a maximum token limit. If exceeded via system prompts, history, and docs, the model drops earlier content silently. Track it!
6. Transformer Architecture
Introduced in "Attention Is All You Need", transformers replaced sequential bottleneck processing with Self-Attention, allowing every token to simultaneously reference every other token, making massive parallel training tractable.
Layer Mechanics
Built by stacking identical layers (dozens to >100). Each layer repeats:
- Self-Attention: Tokens exchange context across the sequence.
- Feed-Forward Network: Two linear transformations applied independently to each token.
Stabilized by residual connections and layer normalization.
Architectural Variants
- Encoder-Only (e.g., BERT): Bidirectional attention for comprehension tasks (search, classification). Not for generation.
- Decoder-Only (e.g., GPT, Claude): Autoregressive generation. Each token only sees what came before. Predicts next token, appends it, and loops.
7. Frontier Models
The frontier is a moving position, not a fixed product. Leadership changes frequently across benchmarks assessing fluid intelligence, scientific reasoning, coding, and long context recall.
Closed Labs
Access via APIs. Typically lead on raw capability, reasoning, and multimodal. Vendor controls pricing and updates.
Open-Source
Publicly released weights. Allows self-hosting and specialized fine-tuning, rapidly closing the gap with closed labs.
Reasoning Models: These allocate more compute at inference time to generate internal chain-of-thought steps before answering. They trade latency and cost for significantly higher accuracy on hard math or logic problems.
Choosing the Right Model
| Use Case | Approach | Why |
|---|---|---|
| Complex reasoning and scientific problems | Prioritize models that lead on hard reasoning benchmarks. | Reasoning models allocate more compute at inference time to generate internal chain-of-thought steps, trading speed for significantly higher accuracy on multi-step problems. |
| Coding | Evaluate on real-world software engineering tasks, not synthetic tests. | Benchmarks age quickly and once saturated, the field moves on; real-world evaluations cut through vendor positioning better than static tests. |
| Long documents or codebases | Context length and recall quality matter more than raw benchmark scoree. | Models must be able to reason over extended texts and documents, making long context capability a standard requirement rather than just a differentiator. |
| Writing and instruction following | Human preference evaluations are more reliable than automated metrics . | Models are fine-tuned via RLHF (Reinforcement Learning from Human Feedback) to maximize a reward model based on human preference, optimizing for what humans actually prefer rather than raw prediction . |
| Cost-sensitive production workloads | The efficient tier handles most tasks at a fraction of frontier pricing . | Not every task requires a frontier model; smaller, cheaper models inherit previous capability levels as the frontier advances, making them perfect for routine tasks . |
| Self-hosted or fine-tuned deployments | Open-weight models offer flexibility closed APIs cannot . | Publicly released weights allow for self-hosted deployment and domain-specific fine-tuning with cost structures that closed APIs cannot match . |
8. Multimodal AI
Multimodal systems integrate text, images, audio, and video to achieve unified reasoning and shared semantic understanding. It moves computing toward general machine perception.
Image & Video (VQA)
Interprets diagrams, screenshots, spatial relationships, and temporal events across frames.
Audio & Speech
Extends beyond transcription to detect tone, identify speakers, and recognize environmental sounds.
Training requires aligned cross-modal data (e.g., images paired with captions). The goal isn't just recognition—a screenshot becomes a debugging task; a video becomes a searchable summary.