Anurag33Gaikwad/legal-led-judgment-summarizer-16384

📘 Legal LED – Court Judgment Summarizer (Long-Document)

Fine-tuned version of NSI’s Legal LED for abstractive summarization of long judicial decisions, case law, and court judgments.

This model fine-tunes nsi319/legal-led-base-16384, a legally adapted LED with a 16k token context window, using a mix of Indian court judgment data and custom curated summaries.
It is optimized to handle factual background, issues, arguments, legal reasoning, and final orders — producing structured and accurate case summaries.

---

🧠 Base Model

This model extends:

👉 nsi319/legal-led-base-16384

The original base model is pretrained on large-scale legal corpora including:

• Supreme Court & High Court judgments
• Legislation
• Legal treatises
• Court orders
• Regulatory material

---

📚 Fine-Tuning Datasets

1️⃣ ai4bharat/IndicCourtSummaries (Recommended dataset)

Contains Indian Supreme Court & High Court judgments with human-written headnotes.

Includes sections like:

• Fact summary
• Legal issues
• Arguments
• Court reasoning
• Final conclusion

2️⃣ Custom Legal Judgment Summaries

Additional curated summaries cleaned manually.

Document Length

Judgments ranged from 8,000 to 40,000 tokens, making LED’s long-context attention crucial.

---

⚙️ Training Configuration

Setting	Value
Base model	nsi319/legal-led-base-16384
Epochs	4
Batch size	1
Gradient accumulation	8
Learning rate	1e-5
Optimizer	AdamW
Weight decay	0.01
FP16	Yes
Warmup steps	500
Max input length	14,000 tokens
Max output length	512 tokens
Attention	Global attention on first token
Scheduler	Linear

Training was performed on NVIDIA TESLA P100 GPU (16GB).

---

🧪 Evaluation Metrics

Training Progress

Epoch	Training Loss	Validation Loss
1	2.25	2.32
2	2.03	2.11
3	1.85	2.03
4	1.79	2.01

ROUGE (Judgment Test Set)

Metric	F1
ROUGE-1	0.4610
ROUGE-2	0.2353
ROUGE-L	0.2584

BERTScore

Metric	Score
Precision	~0.86
Recall	~0.85
F1	~0.86

BERTScore shows strong semantic preservation — crucial for legal tasks.

---

🏗️ Long-Document Summarization Pipeline

Since judgments exceed 20k–30k tokens, this model uses:

1️⃣ Paragraph-aware adaptive chunking

2️⃣ Sliding window segmentation

3️⃣ Chunk-wise LED summarization

4️⃣ Top-K reranking using BERTScore

5️⃣ Final LED rewrite pass

This produces coherent, concise, and semantically accurate summaries.

---

📌 Intended Use

Designed for:

• Legal judgment summarization
• Case-law research tools
• Document automation
• Student case summaries
• Legal AI assistance
• Preprocessing for RAG/legal LLM tasks

---

⚠️ Limitations

• English only
• Requires chunking for >16k tokens
• May condense minority arguments
• Not suitable for citation extraction or legal reasoning verification
• Not a substitute for legal advice

---

🔧 Usage Example

from transformers import AutoTokenizer, LEDForConditionalGeneration
import torch

model_name = "Anurag33Gaikwad/legal-led-judgment-summarizer-16384"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = LEDForConditionalGeneration.from_pretrained(model_name)

text = """Your long court judgment here..."""

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=4096
)

global_attention_mask = torch.zeros_like(inputs["input_ids"])
global_attention_mask[:, 0] = 1

summary_ids = model.generate(
    inputs["input_ids"],
    global_attention_mask=global_attention_mask,
    num_beams=5,
    max_length=512,
    early_stopping=True
)

print(tokenizer.decode(summary_ids[0], skip_special_tokens=True))