tau-ds

This repository contains the HuggingFace safetensors export of a DeepSeek-V4-Flash SFT checkpoint.

• Base model: deepseek-ai/DeepSeek-V4-Flash
• Training framework: Megatron-SWIFT / MCore
• Exported checkpoint: checkpoint-5934
• HF export format: sharded safetensors with model.safetensors.index.json

Convert Megatron-SWIFT / MCore Checkpoint to HuggingFace Safetensors

The uploaded weights were converted from the Megatron-SWIFT MCore checkpoint with megatron export --to_hf true.

1. Paths

Example paths used for this export:

MODEL=/mnt/models/basemodel/DeepSeek-V4-Flash
CKPT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934
OUT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934-hf-safetensors

2. Runtime Patch Used During Export

For this DeepSeek-V4-Flash FP8 checkpoint, the export container used a small runtime patch via PYTHONPATH:

export PYTHONPATH=/mnt/data/patches/deepseek_hf_export:${PYTHONPATH:-}
export DEEPSEEK_EXPORT_PATCH_REQUIRED=1

The patch handled two compatibility details:

1. mcore_bridge.GPTBridge.save_weights compatibility with the current swift export call signature.
2. Pipeline-parallel-safe export of DeepSeek-V4 grouped wo_a weights when PP > 1.

3. Export Command

The core command was:

megatron export \
  --to_hf true \
  --model "$MODEL" \
  --mcore_model "$CKPT" \
  --output_dir "$OUT" \
  --exist_ok true \
  --model_type deepseek_v4 \
  --template deepseek_v4 \
  --agent_template deepseek_v4 \
  --tensor_model_parallel_size 1 \
  --pipeline_model_parallel_size 8 \
  --pipeline_model_parallel_layout 'Et*6|t*6|t*6|t*6|t*6|t*5|t*5|t*3,m,L' \
  --context_parallel_size 1 \
  --expert_model_parallel_size 8 \
  --expert_tensor_parallel_size 1 \
  --sequence_parallel true \
  --max_length 16384 \
  --bf16 true \
  --fp8_format e4m3 \
  --fp8_recipe blockwise \
  --fp8_param_gather true \
  --mtp_num_layers 1 \
  --moe_grouped_gemm true \
  --moe_permute_fusion true \
  --attention_backend flash \
  --apply_dsa_kernel_fusion true \
  --test_convert_precision false

4. Multi-node Launch

The export was launched on 8 nodes, one container per node, using the same image and shared filesystem mounts:

IMG=modelscope-qwen36:deepseek-v4-flashmla-nvdev
WORK=/mnt/cephfs/ubuntu/qwen36sft
MODELS=/mnt/cephfs/ubuntu/models
SCRIPT=/mnt/data/scripts/export_deepseek_v4_flash_5934_hf.sh

# inside each node container
bash "$SCRIPT" <node_rank_0_to_7>

The container was started with host networking, host IPC, all GPUs, and the shared data/model mounts:

docker run -d \
  --gpus all \
  --network host \
  --ipc=host \
  --privileged \
  --shm-size=64g \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  -v ${WORK}:/mnt/data \
  -v ${MODELS}:/mnt/models \
  ${IMG} \
  bash ${SCRIPT} <node_rank>

5. Expected Output

A successful export writes a HuggingFace model folder like:

config.json
generation_config.json
tokenizer.json / tokenizer_config.json / special_tokens_map.json
model.safetensors.index.json
model-00001-of-00060.safetensors
...
model-00060-of-00060.safetensors

You can sanity-check the result with:

python - <<'PY'
from transformers import AutoConfig, AutoTokenizer
path = "/path/to/checkpoint-5934-hf-safetensors"
print(AutoConfig.from_pretrained(path, trust_remote_code=True).model_type)
print(type(AutoTokenizer.from_pretrained(path, trust_remote_code=True)).__name__)
PY

SGLang Serving Example

The exported safetensors checkpoint was tested with SGLang using DeepEP:

SGLANG_DSV4_FP4_EXPERTS=0 \
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=256 \
sglang serve \
  --trust-remote-code \
  --model-path /path/to/checkpoint-5934-hf-safetensors \
  --tp 4 \
  --dp 4 \
  --enable-dp-attention \
  --moe-a2a-backend deepep \
  --cuda-graph-max-bs-decode 128 \
  --max-running-requests 256 \
  --deepep-config '{"normal_dispatch":{"num_sms":96},"normal_combine":{"num_sms":96}}' \
  --host 0.0.0.0 \
  --port 30000