swnometal666/finnish-healthcare-terminology-v3

Finnish Healthcare Terminology Model v3

Model Description

This is a fine-tuned Finnish language model specialized for healthcare terminology and medical education. Built on TurkuNLP/gpt3-finnish-small using LoRA (Low-Rank Adaptation).

Training Details

• Base Model: TurkuNLP/gpt3-finnish-small
• Training Strategy: Ultra-conservative with early stopping
• Dataset: 20 high-quality Finnish medical articles
• Training Loss: 2.43 (optimal checkpoint)
• Training Steps: 10 (early stopped at 16)
• Learning Rate: 5e-6
• LoRA Configuration: r=6, alpha=12

Intended Use

• Educational healthcare content in Finnish
• Medical terminology explanations
• Health information chatbots
• NOT for medical diagnosis or treatment advice

Model Performance

• Superior medical terminology usage
• Clean, coherent Finnish responses
• No formatting artifacts
• Outperforms base model in medical contexts

Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("TurkuNLP/gpt3-finnish-small")
tokenizer = AutoTokenizer.from_pretrained("TurkuNLP/gpt3-finnish-small")

# Load fine-tuned model
model = PeftModel.from_pretrained(base_model, "swnometal666/finnish-healthcare-terminology-v3")

# Generate response
prompt = "Kysymys: Mitä tarkoittaa diabetes? Vastaus:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Limitations

• Designed for educational purposes only
• Should not replace professional medical advice
• May contain inaccuracies - always verify medical information
• Optimized for Finnish language only

Training Data

• Curated Finnish healthcare articles
• Cleaned format without artifacts
• Quality-focused selection (20 articles)

License

Apache 2.0