Text Generation
Transformers
PyTorch
Safetensors
English
Chinese
llama
text-generation-inference
How to use from
vLLM
Install from pip and serve model
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GeneZC/MiniLoong-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GeneZC/MiniLoong-3B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Use Docker
docker model run hf.co/GeneZC/MiniLoong-3B
Quick Links

MiniLoong-3B

πŸ“‘ arXiv | πŸ‘» GitHub | πŸ€— HuggingFace-MiniMA-3B | πŸ€— HuggingFace-MiniChat-3B | πŸ€– ModelScope-MiniMA-3B | πŸ€– ModelScope-MiniChat-3B | πŸ€— HuggingFace-MiniChat-1.5-3B | πŸ€— HuggingFace-MiniMA-2-3B | πŸ€— HuggingFace-MiniChat-2-3B | πŸ€— HuggingFace-MiniMA-2-1B | πŸ€— HuggingFace-MiniLoong-3B | πŸ€— HuggingFace-MiniMix-2/4x3B

❗ Must comply with LICENSE of LLaMA-2 since it is derived from LLaMA-2.

teaser_d

Bibtex

@article{zhang2023law,
    title={Towards the Law of Capacity Gap in Distilling Language Models},
    author={Zhang, Chen and Song, Dawei and Ye, Zheyu and Gao, Yan},
    year={2023},
    url={https://arxiv.org/abs/2311.07052}
}
Downloads last month
12
Safetensors
Model size
3B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for GeneZC/MiniLoong-3B

Quantizations
1 model

Datasets used to train GeneZC/MiniLoong-3B

Space using GeneZC/MiniLoong-3B 1

Collection including GeneZC/MiniLoong-3B

Paper for GeneZC/MiniLoong-3B