Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
2026-09-04 · Hugging Face
Background
Structured output is one of the most practical demands placed on language models in production. Whether a model reliably returns valid, parseable data in the required format — what the authors call schema compliance — often determines whether it can be wired into a downstream system at all. Yet most benchmarks fold structured-output performance into broader reasoning or extraction scores rather than measuring it on its own.
This guide presents a fully public, low-cost recipe: fine-tune LFM2.5-350M using Group Relative Policy Optimization (GRPO) with the TRL library, run for 500 samples and 100 training steps on a free-tier Colab or Kaggle GPU, and evaluate on the IFStruct benchmark. The result is a jump from 22.6% to 29.7%. The full code is available on GitHub.
One important note: the training pipeline here is not the one used to produce the RL model described in the IFStruct blog post. The goal is not to reproduce that official score, but to show how task-specific fine-tuning of small models can close the gap with much larger ones.
Setup
The workflow runs across two environments:
- Fine-tuning requires a GPU. The accompanying notebook is sized for free-tier Colab or Kaggle.
- Evaluation runs locally via llama.cpp on a MacBook (the article uses a MacBook Pro with Apple M5 Max and 36 GB of unified memory). llama.cpp exposes an OpenAI-compatible server endpoint that the IFStruct evaluator connects to directly.
Required tooling: `uv` for Python dependency management and `llama.cpp` installed via Homebrew.
Baseline: LFM2.5-350M Without Fine-tuning
Before training, the base model is evaluated on IFStruct to establish a local baseline.
IFStruct is an open benchmark designed specifically to test LLM output validity and schema adherence. The dataset is available on Hugging Face at `LiquidAI/ifstruct-v1.0` and the evaluation code is open-sourced at `Liquid4All/ifstruct`.
The model is served in BF16 GGUF format with a 32,768-token context window and four parallel slots. Running the full evaluation on 2,000 samples yields:
- Overall pass rate: 22.6% (452/2000)
- JSON: 18.0% | YAML: 27.2%
- Wrapper-key top-level: 28.5% | Bare list: 16.6%
The IFStruct release blog reports 21.1% for this model; the local result of 22.6% is close, with the difference attributable to serving-stack variation. This local figure serves as the comparison baseline.
Top error categories: required field missing (7,228 occurrences), wrong item count (738), type mismatch (540), unclosed code blocks (317), and several extraneous fields such as `notes`, `path`, and `constraints`.
GRPO Fine-tuning
Training Data
Training uses the `nvidia/Nemotron-RL-instruction_following-structured_outputs` dataset, which pairs each prompt with a target JSON Schema and an expected field count. Approximately 500 samples are selected for training.
Because the Nemotron data distribution differs from the IFStruct evaluation, prompts are augmented to close the gap between the two distributions (the article references a 40% augmentation rate, though full details are not included in the available excerpt).
Training Configuration
- Algorithm: GRPO (Group Relative Policy Optimization)
- Library: TRL
- Training steps: 100
- Sample count: ~500
- Hardware: Free-tier Colab or Kaggle GPU
The complete runnable pipeline is provided in the accompanying notebook on GitHub.
Results
After fine-tuning, IFStruct performance improves from 22.6% to 29.7% — roughly a 7-point gain. This shows that even a very lightweight fine-tuning procedure produces a meaningful improvement in structured-output compliance.
The broader takeaway: task-specific fine-tuning of sub-400M parameter models is a practical, compute-efficient path toward matching the structured-output capabilities of far larger models.