LLM Engineering11 min read• Published July 16, 2026

The End-to-End LLM Pipeline: Pretraining, SFT, RLHF, DPO & LoRA/QLoRA Fine-Tuning

Parameter scaling laws (Chinchilla 20 tokens/param), dataset deduplication, instruction tuning, direct preference alignment, and PEFT adapters.

YP
Yashika P
Founder @ AscendiaEdu & Lead DevOps Engineer

⚡ Executive Summary

From pretraining FLOP scaling laws ($6ND$) to Supervised Fine-Tuning (SFT), DPO alignment, and parameter-efficient fine-tuning with LoRA & 4-bit QLoRA.

Key Takeaways

  • ✓Pretraining & Scaling Laws — covered in depth with practical examples, formulas, and code.
  • ✓Supervised Fine-Tuning (SFT) & Alignment (RLHF / DPO) — covered in depth with practical examples, formulas, and code.
  • ✓Parameter-Efficient Fine-Tuning (LoRA & QLoRA) — covered in depth with practical examples, formulas, and code.

The Full LLM Pipeline & Alignment Guide

#1. Pretraining & Scaling Laws

  • Compute Budget Math: FLOPs ≈ 6 N D (where N = parameters, D = tokens).
  • Chinchilla Scaling: Optimal training ratio is ~20 tokens per parameter.

    • #2. Supervised Fine-Tuning (SFT) & Alignment (RLHF / DPO)

  • SFT: Fine-tunes pretrained base models on instruction-response pairs.
  • RLHF vs DPO: Direct Preference Optimization (DPO) optimizes preference policy directly without PPO reward models:

    • MATHEMATICAL FORMULA mathcalL_DPO = - E_(x, y_w, y_l) ≤ft[ log σ ≤ft( β log (π_θ(y_w|x)) / (π_ref)(y_w|x) - β log (π_θ(y_l|x)) / (π_ref)(y_l|x) right) right]

    #3. Parameter-Efficient Fine-Tuning (LoRA & QLoRA)

    LoRA freezes base weights W₀ and injects low-rank trainable decomposition matrices A and B:

    MATHEMATICAL FORMULA W = W₀ + (α) / (r) (B × A)

    QLoRA quantizes base model weights to 4-bit NormalFloat (NF4) while maintaining 16-bit LoRA gradients.

    python
    from peft import LoraConfig, get_peft_model
peft_config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
lora_model = get_peft_model(model, peft_config)
    Tags:#Pretraining#SFT#RLHF#DPO#LoRA#QLoRA#Fine-Tuning
    ABOUT THE AUTHOR
    YP

    Yashika P

    Founder @ AscendiaEdu & Lead DevOps Engineer

    Yashika P is part of the Junglans Solutions engineering team, specializing in llm engineering. Junglans builds a 20-product ecosystem of local-first enterprise software — AI developer tools, encrypted communication, and data infrastructure with zero cloud telemetry.

    Meet the full Junglans engineering team ↗
    RELATED RESOURCES & REFERENCES

    This article is part of the Junglans Research knowledge base, produced alongside the engineering teams that build our production AI tools. Explore related product documentation and research:

    Related Research & Articles

    LLM Evaluation & Benchmarking

    RAG Evaluation Frameworks & LLM Serving Metrics: TTFT, Throughput, Latency & VRAM Sizing

    A practical operational guide covering RAG metrics (Context Precision, Recall, Groundedness) and physical serving metrics (TTFT, VRAM sizing, Throughput).

    Read Article ↗
    LLM Engineering

    Building Large Language Models: Tokenization (BPE/WordPiece), RoPE Embeddings, Self-Attention & Transformer Blocks

    Uncover how LLMs are engineered: Byte-Pair Encoding, Rotary Position Embeddings (RoPE), Self-Attention QKV matrices, RMSNorm, and SwiGLU activations.

    Read Article ↗
    LLM Engineering

    LLM Inference Hyperparameters: Temperature, Top-P Nucleus, Top-K, Repetition Penalty & Context Windows

    Master decoding parameters: Temperature logits scaling, Top-P nucleus sampling, Top-K truncation, Repetition Penalties, and system prompt engineering.

    Read Article ↗