Instructions to use Qwen/CodeQwen1.5-7B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/CodeQwen1.5-7B-Chat with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Qwen/CodeQwen1.5-7B-Chat") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Qwen/CodeQwen1.5-7B-Chat") model = AutoModelForCausalLM.from_pretrained("Qwen/CodeQwen1.5-7B-Chat") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use Qwen/CodeQwen1.5-7B-Chat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Qwen/CodeQwen1.5-7B-Chat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/CodeQwen1.5-7B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Qwen/CodeQwen1.5-7B-Chat
- SGLang
How to use Qwen/CodeQwen1.5-7B-Chat with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Qwen/CodeQwen1.5-7B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/CodeQwen1.5-7B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Qwen/CodeQwen1.5-7B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/CodeQwen1.5-7B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Qwen/CodeQwen1.5-7B-Chat with Docker Model Runner:
docker model run hf.co/Qwen/CodeQwen1.5-7B-Chat
Maybe a silly question...
Sry, but why can't a coding model provide a working "snake game" code for pygame, while llama-3-8b-instruct can? (not always on the first try, but it can).
Promt: Provide complete working code for a snake game in pygame please.
Another promt: Write a Python function that prints the next 20 leap years. Reply with only the function.
Also fails, while other coding models succeed. Update: It actually worked, but after like 5 attempts.
Running the model in LM Studio.
You can use the prompt "Write a snake game with pygame".
The generated code is listed below:
import pygame
import random
# Initialize pygame
pygame.init()
# Set up the display
width, height = 800, 600
win = pygame.display.set_mode((width, height))
pygame.display.set_caption("Snake Game")
# Set up colors
black = (0, 0, 0)
white = (255, 255, 255)
red = (255, 0, 0)
green = (0, 255, 0)
# Set up the snake
snake_speed = 15
snake_block = 10
snake_list = []
snake_length = 1
x1 = width // 2
y1 = height // 2
direction = "RIGHT"
# Set up the food
foodx = round(random.randrange(0, width-snake_block) / 10.0) * 10.0
foody = round(random.randrange(0, height-snake_block) / 10.0) * 10.0
clock = pygame.time.Clock()
game_over = False
while not game_over:
for event in pygame.event.get():
if event.type == pygame.QUIT:
game_over = True
elif event.type == pygame.KEYDOWN:
if event.key == pygame.K_LEFT and direction != "RIGHT":
direction = "LEFT"
elif event.key == pygame.K_RIGHT and direction != "LEFT":
direction = "RIGHT"
elif event.key == pygame.K_UP and direction != "DOWN":
direction = "UP"
elif event.key == pygame.K_DOWN and direction != "UP":
direction = "DOWN"
# Update the snake's position
if direction == "RIGHT":
x1 += snake_block
elif direction == "LEFT":
x1 -= snake_block
elif direction == "UP":
y1 -= snake_block
elif direction == "DOWN":
y1 += snake_block
# Check for collisions with the edges of the screen
if x1 < 0 or x1 >= width or y1 < 0 or y1 >= height:
game_over = True
# Check for collisions with the snake's body
for x in snake_list[:-1]:
if x == (x1, y1):
game_over = True
# Add the new head of the snake to the list
snake_list.insert(0, (x1, y1))
# Check if the snake has eaten the food
if x1 == foodx and y1 == foody:
foodx = round(random.randrange(0, width-snake_block) / 10.0) * 10.0
foody = round(random.randrange(0, height-snake_block) / 10.0) * 10.0
snake_length += 1
# Remove the tail of the snake if it's too long
if len(snake_list) > snake_length:
snake_list.pop()
# Draw everything
win.fill(black)
for pos in snake_list:
pygame.draw.rect(win, green, pygame.Rect(pos[0], pos[1], snake_block, snake_block))
pygame.draw.rect(win, red, pygame.Rect(foodx, foody, snake_block, snake_block))
pygame.display.update()
# Cap the frame rate
clock.tick(snake_speed)
# Game over screen
pygame.quit()
Hi, I try a simple prompt "snake game with pygame" in the codeqwen-chat huggingface space.
codeqwen is able to generate functional code with a few trials, due to the sampling strategy in the generation config. Maybe you should double check with generation config.
