How to Hire an LLM / RAG Engineer

• aAlways pick the largest model, since accuracy is all that matters
• bPick the cheapest model to protect margin
• cBuild a labeled eval set from real traffic, benchmark several sizes on accuracy, latency, and cost, and pick the smallest that clears your quality barCorrect
• dUse whatever model the vendor blog recommends this month

Explanation: Model selection should be evidence-based on your distribution: a labeled eval set from real traffic with explicit accuracy, latency, and cost targets. Largest-always wastes money, cheapest-always risks quality, and vendor marketing is not measurement.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the mid-level experience tier.

• aSelf-attention compares every token with every other token, so attention compute grows roughly quadratically with sequence lengthCorrect
• bThe tokenizer slows down on long inputs
• cGPUs throttle on long sequences
• dThe model loads its weights twice

Explanation: Standard self-attention forms an n-by-n interaction matrix, so both compute and attention memory scale quadratically in sequence length; this is why long-context inference is expensive and why techniques like sliding-window or sparse attention exist. Tokenization and weight loading are linear or constant.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the mid-level experience tier.

• aNever; prompting always beats fine-tuning for format
• bWhen the residual error rate times volume has real cost, and you have enough examples of ideal input-output pairs to teach the format durably while shrinking per-call prompt tokensCorrect
• cOnly if the vendor discontinues your model
• dFine-tuning cannot affect output format, only tone

Explanation: Fine-tuning earns its overhead when residual prompt-level errors carry real cost at volume and you have quality training pairs; a side benefit is dropping long formatting instructions from every call. It is not a last resort of discontinued models, and format is precisely what fine-tuning teaches well.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the mid-level experience tier.

• aPretraining teaches chat formatting; SFT teaches facts; preferences teach tokenization
• bAll three stages do the same thing with different data sizes
• cPretraining aligns the model to human values; SFT adds world knowledge; preferences add reasoning
• dPretraining builds broad language and knowledge capability, SFT teaches instruction-following and task format, preference optimization shapes helpfulness, tone, and safetyCorrect

Explanation: The stages are functionally distinct: capability and knowledge come mostly from pretraining, instruction-following behavior from SFT, and preference alignment (helpfulness, style, refusals) from RLHF or DPO-style optimization. The distractors scramble these roles.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the mid-level experience tier.

• aIt trades model accuracy for speed
• bIt compresses responses to save bandwidth
• cIt keeps the GPU busy by admitting new requests as others finish, trading a little per-request latency for much higher aggregate throughputCorrect
• dIt caches answers so identical questions skip the model

Explanation: Continuous (in-flight) batching fills GPU capacity by admitting requests as slots free up instead of waiting for whole batches, raising throughput at slight per-request latency cost. It does not change accuracy, compress payloads, or cache answers.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the mid-level experience tier.

• aThe GPU is too old for long context
• bThe served configuration kept the model's native positional-encoding regime (e.g., RoPE trained at 8k) without the required scaling (NTK/YaRN-style interpolation), so positions beyond training length degradeCorrect
• cThe tokenizer rejects long documents
• dThe KV cache is disabled by default

Explanation: Claimed long contexts often rely on positional-encoding scaling configured at serve time; running RoPE beyond its trained length without interpolation makes attention patterns degrade into gibberish. GPUs affect speed, not positional generalization, and tokenizer or KV-cache toggles would error, not degrade.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the senior experience tier.

• aThis is normal; keep training through it
• bThe model achieved convergence; stop immediately
• cThe eval set leaked into training; celebrate the low loss
• dLikely a bad data batch, too-aggressive learning rate, or hardware fault; roll back to the last healthy checkpoint, clip gradients, and audit or skip the offending dataCorrect

Explanation: Loss spikes usually trace to corrupted batches, LR/optimizer instability, or flaky hardware; the playbook is checkpoint rollback, gradient clipping, LR reduction, and data audit. Training through spikes bakes in damage, and a spike is the opposite of convergence.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the senior experience tier.

• aAn auxiliary load-balancing loss that penalizes uneven expert usage, plus routing tricks like capacity factors or noisy gatingCorrect
• bDeleting the starving experts
• cIncreasing the learning rate until collapse reverses
• dConverting the model back to dense mid-training

Explanation: Routing collapse is a known MoE failure: a rich-get-richer dynamic starves most experts. Load-balancing auxiliary losses and capacity/noise mechanisms spread tokens across experts. Deleting experts or dense conversion abandons the architecture's benefit, and higher LR worsens instability.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the senior experience tier.

• aParameters should be maximized even if data runs out
• bData volume barely matters next to architecture choice
• cFor a fixed compute budget, model size and training tokens should scale together; many early large models were undertrained on data, and smaller well-fed models can beat them cheaperCorrect
• dScaling laws make loss curves unpredictable

Explanation: Compute-optimal training balances parameters and tokens (roughly 20 tokens per parameter as a rule of thumb), implying several famous large models were oversized for their data. The actionable result: favor smaller models trained on more tokens for better quality per compute dollar.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the senior experience tier.

• aInt4 weights silently reload in fp16
• bAt high concurrency and long contexts, the KV cache dominates memory, not the weights; quantizing the KV cache and capping context length target the real bottleneckCorrect
• cMemory pressure is caused by tokenization buffers
• dWeights quantize but activations triple in size

Explanation: Decode-phase memory at scale is dominated by per-sequence KV state, which grows with concurrency times context length; weight-only quantization leaves it untouched. KV-cache quantization, context caps, and paged KV management address the actual constraint.

Why it matters: Evaluates practical understanding of Large Language Models (LLMs) concepts and problem solving for LLM / RAG Engineer at the senior experience tier.