RichardErkhov/Ericu950_-_Papy_1_Llama-3.1-8B-Instruct_date-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Papy_1_Llama-3.1-8B-Instruct_date - GGUF

• Model creator: https://huggingface.co/Ericu950/
• Original model: https://huggingface.co/Ericu950/Papy_1_Llama-3.1-8B-Instruct_date/

Name	Quant method	Size
Papy_1_Llama-3.1-8B-Instruct_date.Q2_K.gguf	Q2_K	2.96GB
Papy_1_Llama-3.1-8B-Instruct_date.IQ3_XS.gguf	IQ3_XS	3.28GB
Papy_1_Llama-3.1-8B-Instruct_date.IQ3_S.gguf	IQ3_S	3.43GB
Papy_1_Llama-3.1-8B-Instruct_date.Q3_K_S.gguf	Q3_K_S	3.41GB
Papy_1_Llama-3.1-8B-Instruct_date.IQ3_M.gguf	IQ3_M	3.52GB
Papy_1_Llama-3.1-8B-Instruct_date.Q3_K.gguf	Q3_K	3.74GB
Papy_1_Llama-3.1-8B-Instruct_date.Q3_K_M.gguf	Q3_K_M	3.74GB
Papy_1_Llama-3.1-8B-Instruct_date.Q3_K_L.gguf	Q3_K_L	4.03GB
Papy_1_Llama-3.1-8B-Instruct_date.IQ4_XS.gguf	IQ4_XS	4.18GB
Papy_1_Llama-3.1-8B-Instruct_date.Q4_0.gguf	Q4_0	4.34GB
Papy_1_Llama-3.1-8B-Instruct_date.IQ4_NL.gguf	IQ4_NL	4.38GB
Papy_1_Llama-3.1-8B-Instruct_date.Q4_K_S.gguf	Q4_K_S	4.37GB
Papy_1_Llama-3.1-8B-Instruct_date.Q4_K.gguf	Q4_K	4.58GB
Papy_1_Llama-3.1-8B-Instruct_date.Q4_K_M.gguf	Q4_K_M	4.58GB
Papy_1_Llama-3.1-8B-Instruct_date.Q4_1.gguf	Q4_1	4.78GB
Papy_1_Llama-3.1-8B-Instruct_date.Q5_0.gguf	Q5_0	5.21GB
Papy_1_Llama-3.1-8B-Instruct_date.Q5_K_S.gguf	Q5_K_S	5.21GB
Papy_1_Llama-3.1-8B-Instruct_date.Q5_K.gguf	Q5_K	5.34GB
Papy_1_Llama-3.1-8B-Instruct_date.Q5_K_M.gguf	Q5_K_M	5.34GB
Papy_1_Llama-3.1-8B-Instruct_date.Q5_1.gguf	Q5_1	5.65GB
Papy_1_Llama-3.1-8B-Instruct_date.Q6_K.gguf	Q6_K	6.14GB
Papy_1_Llama-3.1-8B-Instruct_date.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: apache-2.0 datasets: - Ericu950/Papyri_1 base_model: - meta-llama/Meta-Llama-3.1-8B-Instruct library_name: transformers tags: - papyrology - epigraphy - philology

Papy_1_Llama-3.1-8B-Instruct_date

This is a fine-tuned version of the Llama-3.1-8B-Instruct model, specialized in assigning a date to Greek documentary papyri. On a test set of 2,295 unseen papyri its predictions were, on average, 21.7 years away from the actual date spans. See https://arxiv.org/abs/2409.13870.

Dataset

This model was finetuned on the Ericu950/Papyri_1 dataset, which consists of Greek documentary papyri editions and their corresponding dates and geographical attributions sourced from the amazing Papyri.info.

Usage

To run the model on a GPU with large memory capacity, follow these steps:

1. Download and load the model

import json
from transformers import pipeline, AutoTokenizer, LlamaForCausalLM
import torch
model_id = "Ericu950/Papy_1_Llama-3.1-8B-Instruct_date"
model = LlamaForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
generation_pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto",
)

2. Run inference on a papyrus fragment of your choice

# This is a rough transcription of Pap.Ups. 106
papyrus_edition = """
ετουσ τεταρτου αυτοκρατοροσ καισαροσ ουεσπασιανου σεβαστου ------------------ 
ομολογει παυσιριων απολλωνιου του παuσιριωνοσ μητροσ ---------------τωι γεγονοτι αυτωι 
εκ τησ γενομενησ και μετηλλαχυιασ αυτου γυναικοσ ------------------------- 
απο τησ αυτησ πολεωσ εν αγυιαι συγχωρειν ειναι ---------------------------------- 
--------------------σ αυτωι εξ ησ συνεστιν ------------------------------------ 
----τησ αυτησ γενεασ την υπαρχουσαν αυτωι οικιαν ------------ 
------------------ ---------καὶ αιθριον και αυλη απερ ο υιοσ διοκοροσ -------------------------- 
--------εγραψεν του δ αυτου διοσκορου ειναι ------------------------------------ 
---------- και προ κατενγεγυηται τα δικαια -------------------------------------- 
νησ κατα τουσ τησ χωρασ νομουσ· εαν δε μη --------------------------------------- 
υπ αυτου τηι του διοσκορου σημαινομενηι -----------------------------------ενοικισμωι του 
ημισουσ μερουσ τησ προκειμενησ οικιασ --------------------------------- διοσκοροσ την τουτων αποχην 
---------------------------------------------μηδ υπεναντιον τουτοισ επιτελειν μηδε 
------------------------------------------------ ανασκευηι κατ αυτησ τιθεσθαι ομολογιαν μηδε 
----------------------------------- επιτελεσαι η χωρισ του κυρια ειναι τα διομολογημενα 
παραβαινειν, εκτεινειν δε τον παραβησομενον τωι υιωι διοσκορωι η τοισ παρ αυτου καθ εκαστην 
εφοδον το τε βλαβοσ και επιτιμον αργυριου δραχμασ 0 και εισ το δημοσιον τασ ισασ και μηθεν 
ησσον· δ -----ιων ομολογιαν συνεχωρησεν·
"""
system_prompt = "Date this papyrus fragment to an exact year!"
input_messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": papyrus_edition},
]
terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = generation_pipeline(
    input_messages,
    max_new_tokens=4,
    num_beams=45, # Set this as high as your memory will allow!
    num_return_sequences=1,
    early_stopping=True,
)
beam_contents = []
for output in outputs:
    generated_text = output.get('generated_text', [])
    for item in generated_text:
        if item.get('role') == 'assistant':
            beam_contents.append(item.get('content'))
real_response = "71 or 72 AD"
print(f"Year: {real_response}")
for i, content in enumerate(beam_contents, start=1):
    print(f"Suggestion {i}: {content}")

Expected Output:

Year: 71 or 72 AD
Suggestion 1: 71

Usage on free tier in Google Colab

If you don’t have access to a larger GPU but want to try the model out, you can run it in a quantized format in Google Colab. The quality of the responses might deteriorate significantly. Follow these steps:

Step 1: Connect to free GPU

1. Click Connect arrow_drop_down near the top right of the notebook.
2. Select Change runtime type.
3. In the modal window, select T4 GPU as your hardware accelerator.
4. Click Save.
5. Click the Connect button to connect to your runtime. After some time, the button will present a green checkmark, along with RAM and disk usage graphs. This indicates that a server has successfully been created with your required hardware.

Step 2: Install Dependencies

!pip install -U bitsandbytes
import os
os._exit(00)

Step 3: Download and quantize the model

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
import torch
quant_config = BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_use_double_quant=True,
   bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained("Ericu950/Papy_1_Llama-3.1-8B-Instruct_date",
device_map = "auto", quantization_config = quant_config)
tokenizer = AutoTokenizer.from_pretrained("Ericu950/Papy_1_Llama-3.1-8B-Instruct_date")
generation_pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto",
)

Step 4: Run inference on a papyrus fragment of your choice

# This is a rough transcription of Pap.Ups. 106
papyrus_edition = """
ετουσ τεταρτου αυτοκρατοροσ καισαροσ ουεσπασιανου σεβαστου ------------------ 
ομολογει παυσιριων απολλωνιου του παuσιριωνοσ μητροσ ---------------τωι γεγονοτι αυτωι 
εκ τησ γενομενησ και μετηλλαχυιασ αυτου γυναικοσ ------------------------- 
απο τησ αυτησ πολεωσ εν αγυιαι συγχωρειν ειναι ---------------------------------- 
--------------------σ αυτωι εξ ησ συνεστιν ------------------------------------ 
----τησ αυτησ γενεασ την υπαρχουσαν αυτωι οικιαν ------------ 
------------------ ---------καὶ αιθριον και αυλη απερ ο υιοσ διοκοροσ -------------------------- 
--------εγραψεν του δ αυτου διοσκορου ειναι ------------------------------------ 
---------- και προ κατενγεγυηται τα δικαια -------------------------------------- 
νησ κατα τουσ τησ χωρασ νομουσ· εαν δε μη --------------------------------------- 
υπ αυτου τηι του διοσκορου σημαινομενηι -----------------------------------ενοικισμωι του 
ημισουσ μερουσ τησ προκειμενησ οικιασ --------------------------------- διοσκοροσ την τουτων αποχην 
---------------------------------------------μηδ υπεναντιον τουτοισ επιτελειν μηδε 
------------------------------------------------ ανασκευηι κατ αυτησ τιθεσθαι ομολογιαν μηδε 
----------------------------------- επιτελεσαι η χωρισ του κυρια ειναι τα διομολογημενα 
παραβαινειν, εκτεινειν δε τον παραβησομενον τωι υιωι διοσκορωι η τοισ παρ αυτου καθ εκαστην 
εφοδον το τε βλαβοσ και επιτιμον αργυριου δραχμασ 0 και εισ το δημοσιον τασ ισασ και μηθεν 
ησσον· δ -----ιων ομολογιαν συνεχωρησεν·"""
system_prompt = "Date this papyrus fragment to an exact year!"
input_messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": papyrus_edition},
]
outputs = generation_pipeline(
    input_messages,
    max_new_tokens=4,
    num_beams=10,
    num_return_sequences=1,
    early_stopping=True,
)
beam_contents = []
for output in outputs:
    generated_text = output.get('generated_text', [])
    for item in generated_text:
        if item.get('role') == 'assistant':
            beam_contents.append(item.get('content'))
real_response = "71 or 72 AD"
print(f"Year: {real_response}")
for i, content in enumerate(beam_contents, start=1):
    print(f"Suggestion {i}: {content}")

Expected Output:

Year: 71 or 72 AD
Suggestion 1: 71