README.md

metadata

title: Attention Atlas
emoji: 🌍
colorFrom: pink
colorTo: blue
sdk: docker
pinned: false
license: mit
short_description: Tool for exploring attention patterns, assessing bias, etc.
app_port: 8000

Attention Atlas 🌍

An interactive application for visualizing and exploring Transformer architectures (BERT, GPT-2) in detail, with special focus on multi-head attention patterns, head specializations, bias detection, and inter-sentence attention analysis.

Overview

Attention Atlas is an educational and analytical tool that allows you to visually explore every component of BERT and GPT-2 architectures:

• Token Embeddings & Positional Encodings
• Q/K/V Projections & Scaled Dot-Product Attention
• Multi-Head Attention (Interactive Maps & Flow)
• Head Specialization Radar (Syntax, Semantics, etc.)
• Bias Detection (Token-level & Attention interaction)
• Token Influence Tree (Hierarchical dependencies)
• Inter-Sentence Attention (ISA)

Features

• Interactive Visualizations: Powered by Plotly and D3.js.
• Real-Time Inference: Uses PyTorch backend to run BERT/GPT-2 models on the fly.
• Bias Analysis: Detects generalizations, stereotypes, and unfair language, analyzing how attention mechanisms process them.
• Full Architecture Explorer: Inspect every layer, head, and residual connection.

Technologies

• Shiny for Python
• Transformers (Hugging Face)
• PyTorch
• Plotly

Usage

Simply enter a sentence in the input box, select a model (BERT or GPT-2), and click Generate / Analyze Bias.

---

Part of a Master's thesis on Interpretable Large Language Models.