atisharma/narratrix-exl2-4.0bpw-h6

Exllama v2 Quantization of Narratrix

Used ExLlamaV2 v0.2.8 for quantization at 4.0bpw, otherwise with default settings.

Narratrix

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Linear merge method.

Models Merged

The following models were included in the merge:

• gghfez/Writer-Large-2411-v2.1
• NeverSleep/Lumimaid-v0.2-123B
• mistralai/Mistral-Large-Instruct-2411
• nbeerbower/Gigaberg-Mistral-Large-123B
• TheDrummer/Behemoth-123B-v2.2
• migtissera/Tess-3-Mistral-Large-2-123B
• anthracite-org/magnum-v4-123b

Configuration

The following YAML configuration was used to produce this model:

merge_method: linear
models:
    - model: mistralai/Mistral-Large-Instruct-2411
      parameters:
        weight: 0.13
    - model: nbeerbower/Gigaberg-Mistral-Large-123B
      parameters:
        weight: 0.145
    - model: migtissera/Tess-3-Mistral-Large-2-123B
      parameters:
        weight: 0.145
    - model: anthracite-org/magnum-v4-123b
      parameters:
        weight: 0.145
    - model: NeverSleep/Lumimaid-v0.2-123B
      parameters:
        weight: 0.145
    - model: TheDrummer/Behemoth-123B-v2.2
      parameters:
        weight: 0.145
    - model: gghfez/Writer-Large-2411-v2.1
      parameters:
        weight: 0.145
dtype: bfloat16