Skip to content
FDE 101

LoRA

LoRA, or Low-Rank Adaptation, freezes the pretrained weights and learns smaller low-rank matrices that modify selected layers.1

Tiny example

Section titled “Tiny example”

Instead of updating every weight in a large model for a classifier, train a compact adapter and keep the base checkpoint unchanged.

FDE note

Section titled “FDE note”

LoRA changes how you fine-tune, not whether fine-tuning is the correct lever. Begin with a stable task, good examples, and a versioned eval; use retrieval for facts that change.

Footnotes

Section titled “Footnotes”

  1. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models”, 2021. ↩