Ling-3.0-tiny-Uncensored-Abliterated

An uncensored, abliterated derivative of inclusionAI/Ling-3.0-tiny (BailingMoeV3) — the refusal direction removed at the weights level for direct, complete answers on cybersecurity, red-teaming, and penetration-testing topics where aligned models refuse.

What makes this different

It runs on Apple Silicon (MPS) — the original can't. Ling-3.0-tiny's KDA linear-attention requires fla / Triton kernels, which have no Apple-Silicon backend. This repo ships a triton-free pure-torch port of the BailingMoeV3 modeling code (KDA recurrence, gated RMSNorm, short causal convolution) so the model loads and generates on a Mac's GPU with plain transformers — no CUDA, no Triton, no fla. The abliteration itself was performed on an M4 Max using that port.

• Weights-level uncensored — refusal direction ablated (Heretic / Optuna TPE) across both attention paths (MLA o_proj + KDA dense) and all 128 experts + shared expert per layer. Refusals dropped 35/100 → 8/100 at KL 0.046 (minimal capability change).
• Apple-Silicon runnable — triton-free modeling code included; loads on MPS out of the box.
• MoE — 7.9B total / 1.3B active (128 routed + 1 shared expert), 24 layers, hybrid MLA + KDA linear attention.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "Securelayer7/Ling-3.0-tiny-Uncensored-Abliterated"
tok = AutoTokenizer.from_pretrained(m, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    m, torch_dtype=torch.bfloat16, trust_remote_code=True).to("mps").eval()

msgs = [{"role": "user", "content": "Explain how a SQL injection works and how to prevent it."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
ids = {k: v.to("mps") for k, v in ids.items()}
out = model.generate(**ids, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

The bundled modeling_bailing_moe_v3.py uses a pure-torch fallback for the KDA linear-attention (no Triton), so it also runs on CPU. On CUDA with fla installed you may prefer the original upstream modeling code for speed.

How it was made

1. Triton-free port of BailingMoeV3 so it runs without fla/Triton (math from fla's own MIT naive references; identical weights).
2. Abliteration (Heretic, Optuna TPE multi-objective: minimize refusals + KL) targeting the residual-writing projections of both attention types and every expert down-projection.

Known behavior

Ling is a bilingual (English/Chinese) model; after answering it may occasionally drift into Chinese. Recommended sampling: do_sample=True, temperature=0.7, top_p=0.95. Greedy decoding can degrade. A short SFT pass cleans up drift.

Responsible use

Uncensored ≠ lawless — for legitimate research and authorized security work. Illegal content (incl. CSAM) must be blocked at the serving layer; the weights carry no such guard, and the operator is responsible for a lawful, policy-gated deployment.

License & attribution

MIT — see LICENSE. Derivative of inclusionAI/Ling-3.0-tiny (BailingMoeV3, © Antgroup, MIT). Modifications (triton-free port + abliteration) disclosed in NOTICE.