How Fine-Tuning Updates Neural Network Weights

Fine-tuning only updates 0.01% of model weights. That’s the counterintuitive reality behind customizing AI models — you don’t need to retrain billions of parameters to get dramatically different behavior. A 175-billion parameter model like GPT-3 can be adapted to new tasks by modifying just 17 million of those numbers, yet the results are often indistinguishable from full retraining.

This explains why companies can create specialized AI assistants in hours rather than months. The heavy lifting — learning language patterns, grammar, reasoning — happened during pre-training. Fine-tuning just nudges the model toward your specific use case. But understanding how those weight updates happen internally reveals why some fine-tuning approaches succeed while others destroy the model’s capabilities.

Step 1: Loading Pre-Trained Weights Into GPU Memory

Every fine-tuning session begins with loading a checkpoint file — typically several gigabytes of floating-point numbers representing the model’s learned knowledge. These weights are stored in formats like .bin, .safetensors, or .pt files and loaded directly into GPU VRAM.

The loading process deserializes each layer’s weight matrices into tensors — multi-dimensional arrays of 32-bit or 16-bit floating-point numbers. A 7-billion parameter model like LLaMA-7B requires approximately 14GB of VRAM in 16-bit precision (2 bytes per parameter). The weights are organized by layer: embedding matrices, attention weights (Query, Key, Value projections), feed-forward network weights, and layer normalization parameters.

Modern frameworks like Hugging Face Transformers handle this automatically, mapping checkpoint files to model architecture classes. The critical detail: all weights start as read-only tensors until you explicitly enable gradient computation for specific layers.

Step 2: Freezing Layers to Prevent Catastrophic Forgetting

Not all weights should be updated equally. The embedding layers and early transformer blocks learn general language features — syntax, basic semantics, word relationships. These are often “frozen” by setting their requires_grad flag to False, preventing PyTorch from computing gradients during backpropagation.

Freezing serves two purposes. First, it reduces memory consumption — gradient tensors consume the same amount of VRAM as the weights themselves. Second, and more critically, it prevents catastrophic forgetting. When fine-tuning learning rates exceed 0.00005, models can lose up to 40% of original task performance within just 1000 training steps.

Google’s 2022 study demonstrated this phenomenon clearly: fine-tuning BERT’s last 4 layers changed weights by an average magnitude of 0.003, while earlier layers shifted only 0.0001 — a 30× difference. The later layers adapt to new tasks; earlier layers preserve foundational knowledge.

Most practitioners freeze:

• Embedding layers (word-to-vector mappings)
• First 6-12 transformer blocks (for a 24-layer model)
• All parameters except the final classification head (for simple tasks)

Step 3: Forward Pass — Computing Activations

With weights loaded and freezing configured, the model processes training examples through a forward pass. Each input token is converted to an embedding vector, then propagated through transformer layers via matrix multiplications.

For each layer, the model computes:

1. Self-attention: Query × Key^T, then softmax, then multiplied by Value
2. Feed-forward network: Two linear transformations with GELU activation
3. Layer normalization: Normalize activations, then scale and shift

These operations produce “activations” — intermediate outputs at each layer. For a 7B model, each forward pass involves roughly 7 billion multiply-add operations. The result is a probability distribution over the vocabulary (for language models) or class labels (for classification).

“LoRA fine-tuning modifies only 0.01% to 0.1% of GPT-3’s 175 billion parameters while achieving 99% of full fine-tuning performance”

This forward pass is deterministic — the same input with the same weights produces the same output. The “learning” happens in the next steps.

Step 4: Calculating the Loss Function

The model’s output is compared against the target labels using a loss function. For language models, this is typically cross-entropy loss, which measures how far the predicted probability distribution deviates from the actual next token.

Cross-entropy loss formula:

Loss = -Σ(target_token × log(predicted_probability))

For each training example, the loss is a single scalar value. A perfect prediction yields loss near 0; random guessing produces loss around log(vocabulary_size) — approximately 10.8 for a 50,000-token vocabulary.

Stanford’s 2023 Alpaca model fine-tuned LLaMA-7B using a dataset of 52,000 instruction-following examples. The average loss dropped from 2.3 (before fine-tuning) to 0.4 after just 3 hours on 4 A100 GPUs — costing only $100 in compute. The loss function is the compass that guides all subsequent weight updates.

Step 5: Backpropagation — Computing Gradients

Backpropagation applies the chain rule of calculus to compute how much each weight contributed to the loss. Starting from the output layer, gradients flow backward through the network.

For each unfrozen weight matrix, the framework computes:

∂Loss/∂Weight = ∂Loss/∂Output × ∂Output/∂Weight

This produces a gradient tensor with the same shape as the weight matrix. Each gradient value indicates: “If I increase this weight by 0.001, the loss will change by approximately gradient × 0.001.”

The computational cost of backpropagation is roughly 2× the forward pass — each layer’s activations must be stored for gradient computation. This is why fine-tuning requires more VRAM than inference: a 7B model needs ~28GB for fine-tuning (weights + gradients + optimizer states) versus ~14GB for inference alone.

Step 6: Optimizer Update — AdamW in Action

The optimizer uses gradients to update weights. The most common choice is AdamW (Adam with decoupled weight decay), which maintains per-parameter learning rates based on gradient history.

The update formula combines three components:

new_weight = old_weight 
           - learning_rate × gradient 
           - weight_decay × old_weight

For fine-tuning, learning rates are typically 10-100× smaller than pre-training rates — often 0.00002 to 0.0002. OpenAI’s GPT-3.5-turbo fine-tuning uses a learning rate of 0.0002, with optimal performance achieved at 50-100 training examples.

AdamW also tracks:

• First moment (exponential moving average of gradients) — momentum
• Second moment (exponential moving average of squared gradients) — adaptive learning rate per parameter

These moments are stored as optimizer states, consuming additional VRAM equal to 2× the weight count.

Step 7: Gradient Clipping — Preventing Divergence

Large gradients can cause weight updates so extreme that the model “forgets” everything it learned. Gradient clipping limits the norm (magnitude) of all gradients to a threshold, typically 1.0.

The clipping formula:

if gradient_norm > threshold:
    gradient = gradient × (threshold / gradient_norm)

Without clipping, a single bad batch with exploding gradients could shift weights by 10× the intended amount, destabilizing training. Clipping ensures weight updates stay within predictable bounds regardless of outlier training examples.

This is especially critical for fine-tuning with small datasets — a few unusual examples could otherwise dominate the weight updates.

Step 8: Iteration and Checkpointing

Steps 3-7 repeat for each training example (or batch of examples) across multiple epochs. After every N steps, the model evaluates performance on a held-out validation set.

Checkpoints are saved when:

• Validation loss decreases by a minimum delta (typically 0.001)
• A specified number of steps have passed
• The epoch completes

Each checkpoint contains the full weight state, optimizer state, and training metadata. For a 7B model, checkpoints are 10-15GB each — expensive to store but essential for recovering from overfitting or resuming interrupted training.

Early stopping typically triggers when validation loss hasn’t improved for 3-5 evaluation cycles, preventing overfitting to the fine-tuning data.

Why LoRA Changes Everything

Traditional fine-tuning updates all unfrozen weights directly. But Low-Rank Adaptation (LoRA) takes a different approach: instead of modifying the full weight matrix, it adds a tiny trainable matrix alongside.

For a 4096×4096 weight matrix (16M parameters), LoRA might add a 4096×8 adapter matrix (just 32K parameters) — a 500× reduction. The original weights stay frozen; only the adapter matrix updates during training.

This explains the remarkable statistic: LoRA fine-tuning modifies only 0.01% to 0.1% of GPT-3’s 175 billion parameters while achieving 99% of full fine-tuning performance. The low-rank assumption — that weight changes live in a low-dimensional subspace — holds surprisingly well for most tasks.

LoRA’s benefits:

• Memory efficiency: Only store gradients for adapter weights
• Portability: Swap adapters without reloading the base model
• Speed: Fewer parameters means faster training iterations

When Fine-Tuning Goes Wrong

The most common failure modes:

Catastrophic forgetting: Training too aggressively (learning rate > 0.00005) causes the model to lose its original capabilities. The model becomes excellent at your task but fails at everything else.

Overfitting: With limited data, the model memorizes training examples instead of learning patterns. Validation loss increases while training loss decreases — a clear warning sign.

Mode collapse: For generative models, the output becomes repetitive or generic. The model “hides” in a safe region of parameter space, producing bland but technically correct responses.

All three stem from the same root cause: weight updates that are too large or too frequent relative to the training data.

---

Fine-tuning’s elegance lies in its restraint. Those tiny weight changes — 0.003 magnitude in later layers, 0.0001 in earlier ones — compound into dramatically different model behavior. But what happens when those tiny weight changes collide with the original training data distribution?