---
library_name: transformers
tags:
- stance-detection
- political-science
- multilingual
- nli
- deberta
- group-appeals
language:
- en
- de
base_model: MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7
---

# Model Card for mDeBERTa Stance Detection

A multilingual stance detection model fine-tuned for detecting political stance towards specific groups in text.

## Model Details

### Model Description

This model is a fine-tuned mDeBERTa-v3-base that performs stance classification using Natural Language Inference (NLI) to determine whether political text expresses positive, negative, or neutral stance towards specific target groups.

- **Developed by:** Will Horne, Alona O. Dolinsky and Lena Maria Huber
- **Model type:** Sequence Classification (NLI-based stance detection)
- **Language(s) (NLP):** English, German (multilingual)
- **Finetuned from model:** MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7

### Model Sources

- **Repository:** rwillh11/mdeberta_NLI_stance_NoContext
- **Base Model:** [MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7](https://huggingface.co/MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7)

## Uses

### Direct Use

The model is designed for researchers analyzing political discourse and stance towards specific groups in political text, trained and validated using party manifestos. It takes a natural sentence and a target group as input and classifies the stance as positive, negative, or neutral.

### Downstream Use

This model can be integrated into larger political text analysis pipelines for:
- Political manifestos analysis
- Group appeals detection in political communication
- Comparative political research across countries and languages

### Out-of-Scope Use

This model should not be used for:
- General sentiment analysis (not group-specific)
- Real-time social media monitoring without human oversight
- Making decisions about individuals or groups
- Content moderation without additional validation

## Bias, Risks, and Limitations

### Technical Limitations
- Trained specifically on political manifesto text; performance may vary on other text types
- Limited to three stance categories (positive, negative, neutral)

### Bias Considerations
- Training data consists of political manifestos from specific countries and time periods
- May reflect biases present in political discourse of training data
- Stance classification may vary across different political contexts

### Recommendations

Users should be aware that this model:
- Is designed for research purposes in political science
- Should be validated on specific domains before deployment
- May require human oversight for sensitive applications
- Performance may vary across different types of groups and political contexts

## How to Get Started with the Model

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "rwillh11/mdeberta_NLI_stance_NoContext"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example usage
text = "We will increase funding for schools to better support students."
target_group = "students"

# Create hypotheses for each stance
hypotheses = {
    "positive": f"The text is positive towards {target_group}.",
    "negative": f"The text is negative towards {target_group}.",
    "neutral": f"The text is neutral, or contains no stance, towards {target_group}."
}

# Get predictions for each hypothesis
results = {}
for stance, hypothesis in hypotheses.items():
    inputs = tokenizer(text, hypothesis, return_tensors="pt", truncation=True)
    with torch.no_grad():
        outputs = model(**inputs)
        probs = torch.softmax(outputs.logits, dim=-1)
        entailment_prob = probs[0][0].item()  # Probability of entailment
        results[stance] = entailment_prob

# Select stance with highest entailment probability
predicted_stance = max(results, key=results.get)
print(f"Predicted stance towards '{target_group}': {predicted_stance}")
```

## Training Details

### Training Data

The model was trained on political manifesto data containing:
- **Languages:** English and German
- **Text Type:** Political manifesto sentences at natural sentence level
- **Labels:** Three-class stance classification (positive, negative, neutral)
- **Groups:** Various political target groups (citizens, specific demographics, etc.)
- **Original dataset:** 7,567 text-group pairs
  - **English:** 4,110 examples (54.3%)
  - **German:** 3,457 examples (45.7%)
- **Training Size:** 12,104 expanded training examples (~6,054 original texts × 2 hypotheses each)
- **Test Size:** 4,539 expanded test examples (~1,513 original texts × 3 hypotheses each)

### Training Procedure

#### Preprocessing
- Texts tokenized using mDeBERTa tokenizer with max length 512
- NLI format: premise (political text) + hypothesis (stance towards group)
- Each text paired with both true and false hypotheses for binary classification

#### Training Hyperparameters
- **Training regime:** Mixed precision training
- **Optimizer:** AdamW with weight decay
- **Learning rate:** Optimized via Optuna (range: 1e-5 to 4e-5)
- **Weight decay:** Optimized via Optuna (range: 0.01 to 0.3)
- **Warmup ratio:** Optimized via Optuna (range: 0.0 to 0.1)
- **Epochs:** 10 per trial
- **Batch size:** 16 (train and eval)
- **Trials:** 20 total (10 + 10 batches)
- **Metric for selection:** F1 Macro
- **Seed:** 42 (deterministic training)

#### Training Infrastructure
- **Hardware:** CUDA-enabled GPU
- **Framework:** Transformers, PyTorch
- **Hyperparameter optimization:** Optuna
- **Deterministic training:** All random seeds fixed

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data
- 20% holdout from original dataset
- Multilingual political manifesto sentences

#### Factors
The model was evaluated across:
- **Languages:** English and German text
- **Additional validation on held out sets:** English, German, Dutch, Danish, Spanish, French, Italian, Swedish
- **Stance classes:** Positive, negative, neutral
- **Group types:** Various socia-demographic groups

#### Metrics
Primary metrics used for evaluation:
- **F1 Macro:** Primary optimization metric (treats all classes equally)
- **Balanced Accuracy:** Accounts for class imbalance
- **Precision/Recall (Macro):** Detailed performance measures

### Results

**Best Model Performance (Trial 19, Epoch 9):**
- **Accuracy:** 0.86
- **Balanced Accuracy:** 0.81
- **Precision:** 0.83
- **Recall:** 0.81
- **F1 Macro:** 0.81

Additional validation on held-out sets return the following metrics:

**English**
- **Accuracy:** 0.86
- **Precision:** 0.84
- **Recall:** 0.81
- **F1 Macro:** 0.82

**German (using texts translated from English)**
- **Accuracy:** 0.86
- **Precision:** 0.85
- **Recall:** 0.79
- **F1 Macro:** 0.82

**Dutch (using texts translated from English)**
- **Accuracy:** 0.82
- **Precision:** 0.80
- **Recall:** 0.74
- **F1 Macro:** 0.76

**Danish (using texts translated from English)**
- **Accuracy:** 0.83
- **Precision:** 0.80
- **Recall:** 0.75
- **F1 Macro:** 0.77

**Spanish (using texts translated from English)**
- **Accuracy:** 0.85
- **Precision:** 0.84
- **Recall:** 0.77
- **F1 Macro:** 0.80

**French (using texts translated from English)**
- **Accuracy:** 0.84
- **Precision:** 0.84
- **Recall:** 0.76
- **F1 Macro:** 0.79

**Italian (using texts translated from English)**
- **Accuracy:** 0.86
- **Precision:** 0.86
- **Recall:** 0.77
- **F1 Macro:** 0.81

**Swedish (using texts translated from English)**
- **Accuracy:** 0.82
- **Precision:** 0.79
- **Recall:** 0.77
- **F1 Macro:** 0.78

The model demonstrates strong performance across stance categories with deterministic results confirmed through multiple prediction runs.

## Model Examination

The model uses Natural Language Inference to transform stance detection into a binary entailment task:
- For each text-group pair, generates three hypotheses (positive/negative/neutral stance)
- Selects the hypothesis with highest entailment probability
- This approach leverages pre-trained NLI capabilities for stance classification

## Environmental Impact

Training involved hyperparameter optimization with 20 trials, each training for 10 epochs.

- **Hardware Type:** CUDA-enabled GPU
- **Hours used:** Estimated 10-15 hours (including hyperparameter search)
- **Cloud Provider:** Google Colab
- **Compute Region:** Variable
- **Carbon Emitted:** Not precisely measured

## Technical Specifications

### Model Architecture and Objective
- **Base Architecture:** mDeBERTa-v3-base (278M parameters)
- **Task:** Natural Language Inference for stance detection
- **Input:** Text pair (political sentence + stance hypothesis)
- **Output:** Binary classification (entailment/non-entailment)
- **Objective:** Cross-entropy loss with F1 Macro optimization

### Compute Infrastructure

#### Hardware
- GPU-accelerated training (CUDA)
- Mixed precision training support

#### Software
- Transformers library
- PyTorch framework
- Optuna for hyperparameter optimization
- scikit-learn for metrics

## Citation

If you use this model in your research, please cite:

**BibTeX:**
```bibtex
@misc{mdeberta_stance_nocontext,
  title={mDeBERTa Stance Detection Model for Political Group Appeals},
  author={Research Team},
  year={2024},
  url={https://huggingface.co/rwillh11/mdeberta_NLI_stance_NoContext}
}
```

## Model Card Authors

Research team studying group appeals in political discourse.

## Model Card Contact

For questions about this model, please open an issue in the repository or contact the research team through appropriate academic channels.