# Deepspec

URL: https://docs.hanzo.ai/docs/projects/hanzoai/deepspec

> DeepSpec (DSpark/DFlash/Eagle3 draft training) + Hanzo V4 pipeline — fork of deepseek-ai/DeepSpec

DeepSpec is a full-stack codebase for training and evaluating draft models for speculative decoding. It contains data preparation utilities, draft model implementations, training code, and evaluation scripts.

## Environment

Install the Python dependencies:

```bash
python -m pip install -r requirements.txt
```

Data preparation additionally requires an inference engine to serve the target model when regenerating answers; see [scripts/data/README.md](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec) for details.

## Workflow

Run the stages in order — each stage's output feeds the next:

1. **Data Preparation** — download prompts, regenerate target answers, and build the target cache.
2. **Training** — train a draft model against the cached target outputs.
3. **Evaluation** — measure speculative-decoding acceptance on benchmark tasks.

## Data Preparation

See [scripts/data/README.md](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec) for the step-by-step data pipeline:

1. download and split training data,
2. regenerate answers,
3. prepare the target cache (storage warning: this can be very large — roughly 38 TB for the default `Qwen/Qwen3-4B` setting).

## Training

```bash
bash scripts/train/train.sh
```

`train.sh` launches `train.py`, which spawns one worker per visible GPU. Select the algorithm and target model by pointing `config_path` at one of the configs under [config/](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec) (e.g. `config/dspark/dspark_qwen3_4b.py`); see the script header for the full list of configs, how to override `config_path` / `target_cache_dir`, and how to use `--opts` to override individual config fields. Checkpoints are written to `~/checkpoints/<project_name>/<exp_name>/step_*`.

Hardware: the default configs and scripts assume a single node with 8 GPUs. For fewer GPUs, reduce `CUDA_VISIBLE_DEVICES`.

## Evaluation

```bash
bash scripts/eval/eval.sh
```

`eval.sh` runs `eval.py` against a trained draft checkpoint over the speculative-decoding benchmarks in [eval\_datasets/](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec) (gsm8k, math500, aime25, humaneval, mbpp, livecodebench, mt-bench, alpaca, arena-hard-v2). Set:

- `target_name_or_path` — the target model the draft was trained against (e.g. `Qwen/Qwen3-4B`),
- `draft_name_or_path` — the draft checkpoint, e.g. `~/checkpoints/deepspec/dspark_block7_qwen3_4b/step_latest`, or one of the Hugging Face repo IDs listed in [Released Checkpoints](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec#released-checkpoints).

### Released Checkpoints

The checkpoints below are the ones used for Table 1 in the [paper](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec). Each checkpoint was trained on [open-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend) data generated by its corresponding target model in non-thinking mode, and is the direct output of the corresponding training configuration under [config/](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec).

| Algorithm | `Qwen/Qwen3-4B`                                                                                    | `Qwen/Qwen3-8B`                                                                                    | `Qwen/Qwen3-14B`                                                                                     | `google/gemma-4-12B-it`                                                                                |
| --------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| Eagle3    | [deepseek-ai/eagle3\_qwen3\_4b\_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_4b_ttt7)     | [deepseek-ai/eagle3\_qwen3\_8b\_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_8b_ttt7)     | [deepseek-ai/eagle3\_qwen3\_14b\_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_14b_ttt7)     | [deepseek-ai/eagle3\_gemma4\_12b\_ttt7](https://huggingface.co/deepseek-ai/eagle3_gemma4_12b_ttt7)     |
| DFlash    | [deepseek-ai/dflash\_qwen3\_4b\_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_4b_block7) | [deepseek-ai/dflash\_qwen3\_8b\_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_8b_block7) | [deepseek-ai/dflash\_qwen3\_14b\_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_14b_block7) | [deepseek-ai/dflash\_gemma4\_12b\_block7](https://huggingface.co/deepseek-ai/dflash_gemma4_12b_block7) |
| DSpark    | [deepseek-ai/dspark\_qwen3\_4b\_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_4b_block7) | [deepseek-ai/dspark\_qwen3\_8b\_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_8b_block7) | [deepseek-ai/dspark\_qwen3\_14b\_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_14b_block7) | [deepseek-ai/dspark\_gemma4\_12b\_block7](https://huggingface.co/deepseek-ai/dspark_gemma4_12b_block7) |

> \[!IMPORTANT]
> If you cite these results in a new paper, align your setup with the training settings in this repository; otherwise, the comparison is not meaningful. For domain-specific use, fine-tune the draft model again for better results, especially if the target model is expected to run in thinking mode.

## Supported Algorithms

Currently, DeepSpec includes three draft models: [DSpark](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec), [DFlash](https://arxiv.org/abs/2602.06036) and [Eagle3](https://arxiv.org/abs/2503.01840).

## License

DeepSpec is released under the [MIT License](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec). It includes code adapted
from third-party projects under their own licenses; see [NOTICE](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec) for the
full attribution.

## Acknowledgements

DeepSpec builds on the ideas and code of several excellent open-source projects:

- [SpecForge](https://github.com/sgl-project/SpecForge) (Apache-2.0) — the overall training framework and Eagle3 implementation; portions of the Eagle3 modeling, loss, optimizer, attention, and evaluation code are adapted from it. Adapted files carry an in-file attribution comment, and the full notice is recorded in [NOTICE](https://docs.hanzo.ai/docs/projects/hanzoai/deepspec).
- [DFlash](https://github.com/z-lab/dflash) (MIT) — the DFlash draft-model design and training recipe.
- [Qwen3](https://github.com/QwenLM/Qwen3) and [Gemma](https://github.com/google-deepmind/gemma) — the target model families supported in this repo.

We thank the authors and maintainers of these projects. Contributions of new algorithms are welcome.

## Links

- [Source](https://github.com/hanzoai/deepspec)
- Licence: MIT