SmallLM-Forge: A Small-Scale Language Model Training and Alignment Stack

Wed, 09 Apr 2025 00:00:00 +0000

What Is SmallLM-Forge?

SmallLM-Forge is a small-scale language-model stack for studying how the main components of modern language models are built and connected. It covers tokenization, decoder-only Transformer pretraining, autoregressive generation, parameter-efficient adaptation, supervised fine-tuning, and preference alignment in implementations that remain small enough to inspect and modify directly.

The project focuses on the ideas behind these components and how they become working code: how byte-level BPE constructs a vocabulary, how RoPE and grouped-query attention fit into a decoder-only Transformer, how low-rank adapters modify selected layers while the base model stays frozen, how supervised fine-tuning restricts the loss to response tokens, and how Direct Preference Optimization turns pairwise preferences into a training objective.

Small experiments are used throughout the repository to exercise these ideas in practice. They serve as examples of how the components can be assembled, while the main focus remains on the underlying mechanisms and their implementation.

From Tokens to Alignment

SmallLM-Forge follows three stages of language-model development:

Build — start from raw text, train a tokenizer, construct a decoder-only Transformer, and pretrain it with next-token prediction.

Adapt — keep an existing model as the starting point and modify its behavior efficiently with LoRA or DoRA and supervised fine-tuning.

Align — move from supervised targets to pairwise preferences and optimize the model with Direct Preference Optimization.

The three parts below follow the same progression: build, adapt, and align.

Explore the Stack

Part 1 — Build a Small Language Model

From byte-level BPE to a decoder-only Transformer, causal language modeling, and autoregressive generation.

Part 2 — Adapt a Language Model with LoRA and SFT

Implement LoRA and DoRA, inject lightweight adapters, freeze the base model, and train on supervised responses.

Part 3 — Align a Language Model with DPO

Build the DPO objective from policy and reference log-probabilities and train from chosen and rejected responses.

Why Build It at Small Scale?

Modern language-model libraries provide powerful high-level interfaces, but those abstractions can make the mechanics underneath a training run difficult to follow. SmallLM-Forge keeps the models, objectives, and training loops compact enough that the path from an idea or equation to the corresponding tensor operations can be inspected directly.

This makes the stack useful for experimenting with individual design choices. Tokenization, attention, feed-forward layers, generation, low-rank adaptation, supervised fine-tuning, and preference optimization remain separate components with clear responsibilities. Each one can be studied, modified, and tested without requiring a production-scale training system.

The small scale also keeps complete experiments practical. The goal is to make language-model architecture and training behavior concrete through executable implementations, while preserving enough structure to connect those implementations to the techniques used in larger systems.

Models, Artifacts, and Resources

The repository includes example workflows that exercise the components described in the three parts. The pretraining workflow has also produced two public Hugging Face artifacts:

Byte-level BPE tokenizer: liuhailin0123/wiki103-bpe-tokenizer
Pretrained mini model: liuhailin0123/wiki103-llama-mini

The source code, playground notebooks, tests, and implementation details are available in the SmallLM-Forge repository.