LLM Vocabulary for Engineers: Embeddings, Evals, Quantization, and LoRA

Part 2 of 3 · ← Previous: practical LLM map · Next: LLM tools & ERPNext

In the first post I shared a practical map of LLMs for full-stack engineers: tokens and context, chat “memory,” and the four levers — prompting, RAG, agents/tools, and fine-tuning.

The next wall is vocabulary. APIs, JSON schemas, and UIs felt familiar. Embeddings, training loops, evaluation metrics, quantization, and LoRA / QLoRA did not. The hardest stretch for me was neural network concepts.

This post is for busy engineers who need enough of that vocabulary to follow what they’re reading and building.

Familiar vs foreign

Felt familiar

Calling models like any other service
JSON schemas for structured input/output
Chat and admin UIs around those APIs

Felt foreign

Embeddings — text as vectors
Training — updating weights with data and a loss
Evals — measuring retrieval and answers beyond “looks good”
Quantization — lower-precision weights for memory and cost
LoRA / QLoRA — efficient fine-tuning with adapters (and a quantized base)

As a backend/frontend engineer, it helps to recognize this shift. You are not starting from scratch — you are adding a new layer of ML concepts and terminology to skills you already have.

Learn enough to follow, then go deep

My working rule: first learn enough to follow the pipeline, then go into details.

You do not need to master neural network math before you can understand RAG or ship a tool-calling demo. You do need short, accurate definitions so terms stop being fog.

Embeddings

Dense vectors that represent information such as text in a way that captures useful semantic relationships. Similar meanings often land closer in vector space, depending on the embedding model and similarity measure. That is why embedding-based retrieval can “search by meaning,” not only by keywords.

As an engineer: think index + similarity search, with vectors instead of (or alongside) classic search.

Evals

How you know whether retrieval or answers are any good. Without evals, you are tuning by vibes. More on mindset below. As a software engineer, I already know the importance of tests.

Quantization

Store and run model weights in lower precision (for example 8-bit or 4-bit). You can often trade a small amount of quality for fitting larger models on limited GPUs and reducing memory and inference cost. The actual quality impact depends on the quantization method and model.

As an engineer: compression with a quality budget.

LoRA and QLoRA

LoRA (Low-Rank Adaptation) trains small adapter matrices instead of updating every weight — much cheaper than full fine-tuning. QLoRA keeps the base model quantized and trains LoRA adapters on top, allowing large models to be fine-tuned with less memory and less compute.

Treat these as “efficient transfer learning for LLMs,” not as something you must implement on day one.

Neural nets — enough to follow

When people say “forward pass, loss, backward pass, optimizer,” they mean:

predict → measure how wrong → compute gradients → update the weights

Language models typically predict a distribution over next tokens; training pushes probability toward the right tokens.

That is enough to follow fine-tuning discussions. Go deeper on architectures, gradients, and hyperparameters when you feel confident about the basics and ready to explore more.

Eval mindset (short, but important)

Do not trust vibes alone.

Especially with RAG, separate two questions:

Retrieval quality — did you fetch the right chunks?
Final answer quality — did the model use them well and answer correctly?

You will see retrieval measures such as Precision@K, Recall@K, MRR, and nDCG, and approaches such as LLM-as-judge for evaluating answers. These are useful signals, not perfect ground truth. You do not need to memorize formulas on day one. You need to know they exist so you can build a small test set instead of demo-driven confidence.

One discipline line that transfers from any serious engineering: don’t tune on the test set. Keep evaluation data separate from the data you use to develop and tune the system.

Where this fits the four levers

Vocabulary	Mostly serves
Embeddings + retrieval evals	RAG
Quantization	Running models yourself / fitting hardware
LoRA / QLoRA	Fine-tuning
Answer evals / LLM-as-judge	All levers — knowing whether you improved anything

The map from post 1 still holds: prompting, RAG, and tools are often simpler places to start than fine-tuning. The vocabulary in this post is what makes those levers readable in docs, notebooks, and design reviews.

What’s next

In post 3, I share what we learned from an ERPNext chatbot experiment: how giving an LLM access to ERP data through tools changed it from a general chatbot into something that could answer questions using real purchase and sales order data.

These notes draw on common LLM engineering curricula (for reference: LLM Engineering on Udemy).

Series

A Practical LLM Map for Full-Stack Engineers
LLM Vocabulary for Engineers: Embeddings, Evals, Quantization, and LoRA (this post)
Tools Make the LLM Useful: Lessons from an ERPNext Chatbot Experiment