ShivankXD/stasis-1

Stasis 1 (base, step 36,420)

Stasis 1 is a 109.5M-parameter decoder-only language model trained from scratch on FineWeb-Edu as a personal research project, entirely on free compute (2× NVIDIA T4). The goal is to study how far a small model can be pushed with careful, verifiable training on a tight compute budget. Weights, modeling code, training logs, frozen evaluation gates and per-item benchmark rows are all released.

Base model, research use only. This is a raw pretrained checkpoint: not instruction-tuned, chat-tuned or safety-tuned. It does not match GPT-2 small (see results) and is not suited to chat, coding or any production use.

Summary

Checkpoint	optimizer step 36,420 (end of the tenth training run)
Parameters	109,529,856
Training tokens	~2.947B cumulative (839.4M in the latest run)
Hardware	free Kaggle 2× T4 (DDP, fp16 AMP), ~10.0 h of training for the latest run (10.36 h session)
Held-out result (run 10)	all 8 pre-registered gates passed (−1.39% CE on the fresh primary held-out panel TENTH_FINAL256, −1.47% on the fresh TENTH_DEV64 diagnostic panel, vs step 23,612)
vs GPT-2 small	matched comparison at step 36,420: still behind on WikiText-2, LAMBADA and HellaSwag; PIQA and ARC-Easy statistically inconclusive; no parity

Architecture

Type	Decoder-only Transformer (causal LM)
Parameters	109.5M (109,529,856)
Layers / hidden / heads	12 / 768 / 12 (head dim 64)
MLP	SwiGLU, intermediate size 2048
Normalization	Pre-RMSNorm (eps 1e-5)
Positions	RoPE, theta 1e4
Context length	2048 tokens
Vocabulary	32,000, byte-level BPE (tokenizer.json)
Output head	Tied to the input embedding

Training details

Data. English web text from FineWeb-Edu (HuggingFaceFW/fineweb-edu, sample-100BT, pinned revision), packed into 2048-token windows. Held-out panels are drawn so they are domain-disjoint from training documents and are never trained on.

Lineage. Training proceeds in sequential tranches, each resuming the full optimizer state of the previous endpoint.

Stage	Optimizer steps	Tokens in stage	Cumulative tokens
Runs 1–5	0 → 4,126	~830.5M	~830.5M
Run 6	4,126 → 12,269 (8,143 updates)	533.7M	~1.364B
Run 7	12,269 → 15,106 (2,837 updates)	185.9M	~1.550B
Run 8	15,106 → 16,112 (1,006 updates)	65.9M	~1.616B
Run 9	16,112 → 23,612 (7,500 updates)	491.5M	~2.108B
Run 10 (this checkpoint)	23,612 → 36,420 (12,808 updates)	839.4M	~2.947B

Optimizer and schedule (run 6). AdamW (β1 0.9, β2 0.95, weight decay 0.1, grad clip 1.0); 65,536 target tokens per update; 200-update re-warmup to 1.5e-4, cosine decay to 1.5e-5; fp16 AMP (2 skipped updates out of 8,143).

Optimizer and schedule (run 7). Same optimizer, resumed with its full state from step 12,269; fresh FineWeb-Edu shard not used in earlier runs. 100-update re-warmup to 1e-4, then cosine decay toward 1e-5 over a schedule frozen at 3,150 updates after 30 calibration updates. Throughput was lower than the calibration estimate (median 3.12 s/update vs 2.73 s measured during calibration), so the run hit its wall-clock stop at update 2,837 with the learning rate at 1.23e-5. The decay was about 90% complete, not the full planned anneal. 1 skipped AMP update out of 2,837.

Optimizer and schedule (run 8). Same optimizer, resumed with its full state from step 15,106; another fresh FineWeb-Edu shard not used in earlier runs. A short tranche: 50-update re-warmup to 5e-5 (half the run-7 peak), then cosine decay to 1e-5 over a schedule frozen at 1,006 updates after 30 calibration updates. To avoid the run-7 mis-calibration, the frozen length used a conservative 3.18 s/update floor (measured 2.66 s during calibration). The schedule completed in full (1,006 of 1,006 updates, final LR 1.0e-5). 0 skipped AMP updates.

Optimizer and schedule (run 9). Same optimizer, resumed with its full state from step 16,112; another fresh FineWeb-Edu shard not used in earlier runs (240,000 windows, 491.5M target tokens). 100-update re-warmup to 6e-5, then cosine decay to 1e-5 over a schedule frozen at 7,500 updates after 30 calibration updates. The schedule completed in full (7,500 of 7,500 updates, final LR 1.0e-5). 4 skipped AMP updates.

Optimizer and schedule (run 10). Same optimizer, resumed with its full state from step 23,612; another fresh FineWeb-Edu shard pair not used in earlier runs (409,856 of 416,000 prepared windows used, 839.4M target tokens). 100-update re-warmup to 6e-5, then cosine decay to 1e-5 over a schedule frozen at 12,808 updates after 30 calibration updates. The schedule completed in full (12,808 of 12,808 updates, final LR 1.0e-5). 5 skipped AMP updates.

Hardware. Free Kaggle 2× NVIDIA T4, PyTorch DDP (NCCL). Run 6: ~2.76 s/update, 7.13 h of training. Run 7: ~3.12 s/update, 2.48 h of training. Run 8: ~2.82 s/update, 0.79 h of training. Run 9: ~2.82 s/update (median), ~5.9 h of training (6.11 h session). Run 10: ~2.82 s/update (median), 10.36 h session.

Training loss of runs 6, 7, 8, 9 and 10 (per-update raw values and 100-update moving average). Per-update logs for runs 1–5 are not published, so they are not plotted. The jumps at steps 12,269, 15,106, 16,112 and 23,612 are re-warmups on new data shards (training loss is measured on different data in each run, so levels are not directly comparable across runs). Raw logs: metrics/run6_step12269/train_log.jsonl, metrics/run7_step15106/train_log.jsonl, metrics/run8_step16112/train_log.jsonl, logs/run9_step23612/train_log.jsonl, logs/run10_step36420/train_log.jsonl. Run 7 training loss went from 3.358 to 3.313 (mean of first vs last 100 updates); last update 3.292. Run 8: 3.341 to 3.328 (mean of first vs last 100 updates); first/last update 3.412 / 3.327. Run 9: 3.355 to 3.302 (mean of first vs last 100 updates); first/last update 3.370 / 3.270. Run 10: 3.260 to 3.199 (mean of first vs last 100 updates); first/last update 3.350 / 3.191.

Training data

Stasis 1 was trained on FineWeb-Edu (HuggingFaceFW/fineweb-edu) by Hugging Face, licensed under the Open Data Commons Attribution License (ODC-By) v1.0. Use of the dataset is also subject to CommonCrawl's Terms of Use.

A small English reference set (the English93 and English599 panels) was used only for retention evaluation (the retention gates in the tables below); it was not used for training.

Citation for FineWeb-Edu (from the dataset page):

@misc{lozhkov2024fineweb-edu,
    author       = { Lozhkov, Anton and Ben Allal, Loubna and von Werra, Leandro and Wolf, Thomas },
    title        = { FineWeb-Edu: the Finest Collection of Educational Content },
    year         = 2024,
    url          = { https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu },
    doi          = { 10.57967/hf/2497 },
    publisher    = { Hugging Face }
}

Related datasets

Released alongside Stasis (CC-BY-4.0, generated by the author's own Python template scripts, no third-party AI model). They were not used in base pretraining, which used FineWeb-Edu only (see Training data). They are intended for later fine-tuning.

• ShivankXD/stasis-synthetic-data: 7,070 chat-format coding examples in 12 configs (debugging, execution-verified generations, UI-to-code, state management, full-stack wiring, architecture reasoning, frontend subsets)
• ShivankXD/stasis-thinking-data: 141 long-form reasoning examples in 5 configs (algorithmic, architecture trade-offs, code review, deep debugging, edge cases)

Results

Frozen acceptance gates (run 10, current checkpoint)

All gates were written to configs/run10_step36420/FROZEN_GATES.json (sha256 af59ace0…a673a, byte-identical to the frozen original) before run 10 started and were not changed afterwards. Same criteria as run 9 (not loosened), with the run-9 primary panel added as a no-regression gate. Both the step-23,612 parent and the step-36,420 candidate were scored live with the same code. 95% upper bounds come from a paired bootstrap (1,000 draws). The primary panel (TENTH_FINAL256) is fresh: 256 whole web domains from a shard never used for training, none seen in any training run or earlier panel.

Gate	Step 23,612 → step 36,420	Upper 95% bound	Result
Primary TENTH_FINAL256 CE (gain ≥ 0, UB95 delta < 0)	3.3426 → 3.2963 (−1.39%)	delta -0.0448	PASS
NINTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3529 → 3.3106 (−1.26%)	delta -0.0401	PASS
EIGHTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3416 → 3.2951 (−1.39%)	delta -0.0449	PASS
SEVENTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.2703 → 3.2215 (−1.49%)	delta -0.0473	PASS
SIXTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.2789 → 3.2304 (−1.48%)	delta -0.0472	PASS
FINAL126 no-regression (gain ≥ 0, UB95 < 0)	3.3319 → 3.2825 (−1.48%)	delta -0.0473	PASS
English93 retention ratio vs early reference (UB ≤ 1.01)	ratio 0.9104	0.9171	PASS
English599 retention ratio vs step 3,403 (UB ≤ 1.01)	ratio 0.9269	0.9294	PASS
TENTH_DEV64 (diagnostic only, not a gate)	3.2880 → 3.2397 (−1.47%)	delta -0.0451	n/a

Decision: all eight gates passed, so this is recorded as a research gain (RESEARCH_GAIN_ESTABLISHED_NO_AUTOMATIC_PROMOTION). The gain is modest (about 1.3–1.5% CE on every held-out panel) and is measured against Stasis's own earlier checkpoint. Results: eval/run10_step36420/EVAL_RESULT.json (also metrics/eval/), per-unit rows in EVAL_ROWS.json.

Frozen acceptance gates (run 9, history)

All gates were written to configs/run9_step23612/FROZEN_GATES.json (sha256 35f0de23…6759, byte-identical to the frozen original) before run 9 started and were not changed afterwards. Same strictness as run 8: the primary gate requires a statistically resolved improvement (CE gain ≥ 0 and paired UB95 of the CE delta < 0), and every earlier held-out panel must not regress. Both the step-16,112 parent and the step-23,612 candidate were scored live with the same code. 95% upper bounds come from a paired bootstrap (1,000 draws). The primary panel (NINTH_FINAL256) is fresh: 256 whole web domains from a shard never used for training, none seen in any training run or earlier panel.

Gate	Step 16,112 → step 23,612	Upper 95% bound	Result
Primary NINTH_FINAL256 CE (gain ≥ 0, UB95 delta < 0)	3.4000 → 3.3529 (−1.38%)	delta −0.0446	PASS
EIGHTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3866 → 3.3416 (−1.33%)	delta −0.0430	PASS
SEVENTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3125 → 3.2703 (−1.28%)	delta −0.0406	PASS
SIXTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3221 → 3.2789 (−1.30%)	delta −0.0420	PASS
FINAL126 no-regression (gain ≥ 0, UB95 < 0)	3.3715 → 3.3319 (−1.17%)	delta −0.0378	PASS
English93 retention ratio vs early reference (UB ≤ 1.01)	ratio 0.9247	0.9308	PASS
English599 retention ratio vs step 3,403 (UB ≤ 1.01)	ratio 0.9405	0.9425	PASS
NINTH_DEV64 (diagnostic only, not a gate)	3.4220 → 3.3781 (−1.28%)	delta −0.0415	n/a

Decision: all seven gates passed, so this is recorded as a research gain (RESEARCH_GAIN_ESTABLISHED_NO_AUTOMATIC_PROMOTION). The gain is modest (about 1.2–1.4% CE on every held-out panel). It is measured against Stasis's own earlier checkpoint, not against GPT-2. Results: eval/run9_step23612/EVAL_RESULT.json (also metrics/eval/), per-unit rows in EVAL_ROWS.json.

Frozen acceptance gates (run 8, history)

All gates were written to metrics/run8_step16112/FROZEN_GATES.json before run 8 started and were not changed afterwards. Because run 8 was a short ~1,000-update tranche at a low learning rate, the primary gate was pre-registered as "any statistically resolved improvement" (CE gain ≥ 0 and paired UB95 of the CE delta < 0), with no fixed minimum, instead of run 7's 0.5% floor. Both the step-15,106 parent and the step-16,112 candidate were scored live with the same code; on the three panels shared with run 7, the parent scores reproduce the recorded run-7 rows exactly. 95% upper bounds come from a paired bootstrap (1,000 draws). The primary panel (EIGHTH_FINAL256) is fresh: 256 whole web domains from a shard never used for training, none seen in any training run or earlier panel.

Gate	Step 15,106 → step 16,112	Upper 95% bound	Result
Primary EIGHTH_FINAL256 CE (gain ≥ 0, UB95 delta < 0)	3.3939 → 3.3866 (−0.21%)	delta −0.0066	PASS
SEVENTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3201 → 3.3125 (−0.23%)	delta −0.0069	PASS
SIXTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3301 → 3.3221 (−0.24%)	delta −0.0073	PASS
FINAL126 no-regression (gain ≥ 0, UB95 < 0)	3.3792 → 3.3715 (−0.23%)	delta −0.0068	PASS
English93 retention ratio vs early reference (UB ≤ 1.01)	ratio 0.9392	0.9450	PASS
English599 retention ratio vs step 3,403 (UB ≤ 1.01)	ratio 0.9528	0.9547	PASS
EIGHTH_DEV64 (diagnostic only, not a gate)	3.4783 → 3.4688 (−0.27%)	delta −0.0082	n/a

Decision: all six gates passed, so this is recorded as a research gain (RESEARCH_GAIN_ESTABLISHED_NO_AUTOMATIC_PROMOTION). The gain is small (about 0.2% CE on every held-out panel), consistent with a short, low-learning-rate tranche. Results: metrics/run8_step16112/eval/EVAL_RESULT.json, per-unit rows in metrics/run8_step16112/eval/EVAL_ROWS.json.

Frozen acceptance gates (run 7, history)

All gates were written to metrics/run7_step15106/FROZEN_GATES.json before run 7 started and were not changed afterwards. Both the step-12,269 parent and the step-15,106 candidate were scored live with the same code. 95% upper bounds come from a paired bootstrap (1,000 draws). The primary panel (SEVENTH_FINAL256) is fresh: 256 whole web domains from a shard never used for training, none seen in any training run.

Gate	Step 12,269 → step 15,106	Upper 95% bound	Result
Primary SEVENTH_FINAL256 CE (≥0.5% gain, UB95 delta < 0)	3.3391 → 3.3201 (−0.57%)	delta −0.0177	PASS
SIXTH_FINAL256 no-regression (gain ≥ 0, UB95 < 0)	3.3491 → 3.3301 (−0.57%)	delta −0.0180	PASS
FINAL126 no-regression (gain ≥ 0, UB95 < 0)	3.3977 → 3.3792 (−0.54%)	delta −0.0170	PASS
English93 retention ratio vs early reference (UB ≤ 1.01)	ratio 0.9416	0.9474	PASS
English599 retention ratio vs step 3,403 (UB ≤ 1.01)	ratio 0.9550	0.9568	PASS
SEVENTH_DEV128 (diagnostic only, not a gate)	3.3005 → 3.2815 (−0.58%)	delta −0.0172	n/a

Decision: all five gates passed, so this is recorded as a research gain (RESEARCH_GAIN_ESTABLISHED_NO_AUTOMATIC_PROMOTION). The gain is small: the primary panel cleared the 0.5% threshold by only about 0.07 percentage points. Results: metrics/run7_step15106/eval/EVAL_RESULT.json.

Frozen acceptance gates (run 6, history)

These gates were frozen before run 6 (metrics/run6_step12269/FROZEN_GATES.json). Comparisons are against the step-4,126 parent.

Gate	Parent → step 12,269	Result
Primary held-out panel CE (≥0.5% gain, UB95 < 0)	3.503 → 3.349 (−4.40%)	PASS
FINAL126 held-out CE	3.523 → 3.398 (−3.57%)	PASS
English93 retention ratio (UB ≤ 1.01)	0.947	PASS
English599 retention ratio (UB ≤ 1.01)	0.960	PASS

Matched comparison with GPT-2 small (step 36,420, current checkpoint)

Stasis 1 step 36,420 vs GPT-2 small (124M, openai-community/gpt2), using exactly the same frozen package as the step-12,269 comparison: same items, tokenization, scoring script, metric definitions (sha256 579ab8c1…afa24137) and paired bootstrap (10,000 draws). Both models were re-scored on CPU in fp32; the GPT-2 CPU scores reproduce the earlier T4 GPT-2 scores (identical accuracies on every item, max relative NLL difference 9e-5). Delta = Stasis − GPT-2. The step-12,269 column is from the earlier comparison, for reference.

Benchmark (unit)	Stasis 1 step 12,269	Stasis 1 step 36,420	GPT-2 small	Delta 36,420 − GPT-2 [95% CI]	Verdict
WikiText-2 test, full, BPB ↓ (62 articles)	1.2746	1.2256	1.0565	+0.169 [+0.141, +0.202] (16.0% worse)	GPT-2 better
WikiText-2 256-row panel, BPB ↓	1.5037	1.4612	1.3179	+0.143 [+0.121, +0.168] (10.9% worse)	GPT-2 better
LAMBADA (OpenAI), accuracy ↑ (5,153)	19.1%	20.0%	33.7%	-13.7 pts [-14.9, -12.4]	GPT-2 better
LAMBADA target BPB ↓	1.2692	1.1871	0.7685	+0.419 [+0.403, +0.435] (54.5% worse)	GPT-2 better
HellaSwag val, acc_norm ↑ (10,042)	28.0%	29.4%	31.3%	-1.9 pts [-2.6, -1.2]	GPT-2 better
HellaSwag val, acc ↑	27.2%	28.0%	29.0%	-1.1 pts [-1.6, -0.6]	GPT-2 better
PIQA val, acc_norm ↑ (1,838)	59.4%	60.7%	62.5%	-1.8 pts [-3.6, +0.1]	Inconclusive
PIQA val, acc ↑	60.3%	61.3%	63.1%	-1.7 pts [-3.5, +0.0]	Inconclusive
ARC-Easy test, acc_norm ↑ (2,376)	39.6%	40.7%	40.5%	+0.2 pts [-1.4, +1.8]	Inconclusive
ARC-Easy test, acc ↑	43.7%	45.6%	43.9%	+1.7 pts [-0.1, +3.6]	Inconclusive

Bottom line (step 36,420): still no GPT-2-small parity. Stasis improved on every benchmark since step 12,269 and the WikiText-2 gap narrowed (full test BPB 20.6% → 16.0% worse), but GPT-2 small is still clearly better on WikiText-2, LAMBADA and HellaSwag. PIQA and ARC-Easy are statistically inconclusive (95% CIs include zero); this is not evidence of parity. Files: benchmarks/gpt2_matched_step36420/ (RESULTS.json, COMPARISON.md/json, per-item rows for both models in rows/, run log; metric definitions are the frozen file in benchmarks/gpt2_matched_step12269/). Note: the public METRIC_DEFINITIONS.json had one private output path (outputs field) replaced with <redacted>/RESULTS.json on 2026-10-27, so it no longer hashes to the frozen original 579ab8c1…; no metric definition was changed and the original is in this repo's history; scripts code/gpt2eval/run_step36420.py, summarize_step36420.py.

Step 36,420. Error bars on the Stasis bars show the 95% CI of the paired delta, placed relative to GPT-2's score.

Matched comparison with GPT-2 small (measured at step 12,269, history)

Stasis 1 step 12,269 (109.5M) vs GPT-2 small (124M, openai-community/gpt2), scored by the same script on one T4 in fp32 with no truncation. Metric definitions were frozen before any scoring (benchmarks/gpt2_matched_step12269/METRIC_DEFINITIONS.json, sha256 579ab8c1…afa24137). 95% CIs from a paired bootstrap (10,000 draws). Delta = Stasis − GPT-2.

Benchmark (unit)	Stasis 1	GPT-2 small	Delta [95% CI]	Verdict
WikiText-2 test, full, BPB ↓ (62 articles)	1.2746	1.0565	+0.218 [+0.190, +0.252] (20.6% worse)	GPT-2 better
WikiText-2 256-row panel, BPB ↓	1.5037	1.3179	+0.186 [+0.163, +0.210] (14.1% worse)	GPT-2 better
LAMBADA (OpenAI), accuracy ↑ (5,153)	19.1%	33.7%	−14.6 pts [−15.8, −13.3]	GPT-2 better
LAMBADA target BPB ↓	1.2692	0.7685	+0.501 [+0.484, +0.518] (65.2% worse)	GPT-2 better
HellaSwag val, acc_norm ↑ (10,042)	28.0%	31.3%	−3.3 pts [−4.1, −2.6]	GPT-2 better
PIQA val, acc_norm ↑ (1,838)	59.4%	62.5%	−3.1 pts [−5.0, −1.3]	GPT-2 better
ARC-Easy test, acc_norm ↑ (2,376)	39.6%	40.5%	−1.0 pts [−2.7, +0.7]	Inconclusive
ARC-Easy test, acc ↑	43.7%	43.9%	−0.2 pts [−2.0, +1.7]	Inconclusive

Bottom line (step 12,269): Stasis 1 is not at GPT-2-small parity. It is clearly behind on language modeling (WikiText, LAMBADA), slightly behind on HellaSwag and PIQA, and statistically indistinguishable on ARC-Easy.

Trend: step 3,403 → step 36,420

The 256-row WikiText-2 panel uses the same method at every checkpoint (GPT-2 re-scored at 1.3179 BPB, reproducing its earlier value).

Step	Stasis BPB	GPT-2 small BPB	Gap
3,403	1.6061	1.3179	21.9% worse
12,269	1.5037	1.3179	14.1% worse
36,420	1.4612	1.3179	10.9% worse

Contamination notes

A 13-gram overlap check (Stasis token space) of every benchmark against the run-6 training tokens (533,965,302 tokens) found shared 13-grams in 12/62 WikiText articles (0.15% of 13-grams), 2/114 panel rows, 2/5,153 LAMBADA items, 23/10,042 HellaSwag items, 4/1,756 PIQA items and 5/2,366 ARC-Easy items. This is descriptive only: data from other runs (including runs 7 and 8) and GPT-2's WebText were not checked. Details: benchmarks/gpt2_matched_step12269/CONTAMINATION_13GRAM.json.

How to load

# pip install torch safetensors tokenizers huggingface_hub
# see load_example.py for a complete script
import sys, torch
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
from tokenizers import Tokenizer

path = snapshot_download("ShivankXD/stasis-1", allow_patterns=["config.json", "tokenizer.json", "model.safetensors", "frozen_stasis/*"])  # skips the large checkpoints/ folder
sys.path.insert(0, path)
from frozen_stasis.config import ModelConfig
from frozen_stasis.model import StasisDecoder

model = StasisDecoder(ModelConfig())   # defaults match config.json
model.load_state_dict(load_file(f"{path}/model.safetensors"), strict=True)
model.eval()
tok = Tokenizer.from_file(f"{path}/tokenizer.json")
ids = torch.tensor([tok.encode("The water cycle").ids])
print(tok.decode(model.generate(ids, max_new_tokens=60, top_k=50)[0].tolist()))

This is a custom architecture, so it does not load through transformers.AutoModel.

Roadmap / planned future work

• Continued pretraining on free compute: further resumable tranches on free 2× T4 GPUs, each judged by evaluation gates frozen before the run.
• Matched GPT-2 re-evaluations after each run: the same frozen-metric comparison against GPT-2 small, with confidence intervals, published whether the gap closes or not.
• Later: instruction/chat fine-tuning, then a public chat demo Space. This is planned and not built yet. The current model is a base checkpoint with no chat or instruction-following ability.

Limitations and intended use

• Intended use: research on small-model pretraining, training dynamics, evaluation methodology and reproducibility.
• Out of scope: chat, instruction following, coding, factual question answering, or any high-stakes or production use.
• Small base model trained on ~2.95B tokens; outputs are often repetitive, incoherent or factually wrong.
• Not instruction-tuned or aligned; may reproduce biases or problematic content from web data.
• English only; evaluated only on the panels and benchmarks listed here.

Repository layout

Path	Contents
model.safetensors, config.json, tokenizer.json	Current weights (step 36,420), config and tokenizer
frozen_stasis/, load_example.py	Exact modeling code and a minimal loader
checkpoints/step-<N>/endpoint.model_only.pt	Raw PyTorch model-only endpoint of each run (steps 12,269, 15,106, 16,112, 23,612, 36,420). Load with torch.load(path, weights_only=True)["model"]; ~438 MB each. Optimizer/recovery checkpoints are not published
code/	Training/eval/checkpoint scripts per run (run6_sixth … run10_tenth_kaggle), frozen_stasis/, and the GPT-2 matched-eval scripts (gpt2eval/)
notebooks/	The Kaggle notebooks used for runs 6–10 and the GPT-2 matched eval
configs/run<k>_step<N>/	RUN_CONFIG, FROZEN_GATES, SCHEDULE_FROZEN, TRAIN_STATUS (+ runs 9–10 RUN_PLAN, SESSION_SUMMARY)
logs/run<k>_step<N>/	train_log.jsonl (per-update loss/LR/grad-norm) and torchrun output
eval/run<k>_step<N>/	Frozen-gate EVAL_RESULT.json and per-unit EVAL_ROWS.json
benchmarks/gpt2_matched_step12269/, benchmarks/gpt2_matched_step36420/	GPT-2 matched comparisons: metric definitions, results, rows, contamination report
metrics/	Latest run's files (run 10) plus archived per-run copies (older layout, kept for history)
assets/	Charts and make_charts.py

Training data (train.bin, document lists) and held-out panel contents are not published, so the held-out evaluations stay uncontaminated. Private storage paths in configs, logs, code and notebooks are replaced with <redacted>/<basename>; no values were changed.

Files

• model.safetensors: step-36,420 weights, fp32 (converted from training endpoint sha256 5b2dc755…bcf85; reload verified bitwise identical, strict load OK). Optimizer/recovery state not included. Earlier versions stay available at tags step-12269, step-15106, step-16112 and step-23612.
• config.json, tokenizer.json, frozen_stasis/ (exact training modeling code), load_example.py
• benchmarks/gpt2_matched_step12269/: frozen metric definitions, results, run log, contamination report, per-item rows for both models
• metrics/: latest run (run 10) status, config, schedule, frozen gates, eval results/rows, train_log.jsonl; archived per-run copies in metrics/run6_step12269/, metrics/run7_step15106/, metrics/run8_step16112/ and metrics/run9_step23612/ and metrics/run10_step36420/. Private storage paths are replaced with <redacted>, so the public FROZEN_GATES.json files no longer hash to the originals (run 6 94b62c38…, run 7 c9e73d49…, run 8 cd830ceb…; runs 9 and 10 are unchanged at 35f0de23… and af59ace0…); no values were changed.
• assets/: charts and make_charts.py to regenerate them

License

Code and weights: Apache-2.0 (see LICENSE). Training data: FineWeb-Edu, ODC-By v1.0, also subject to CommonCrawl's Terms of Use.

Citation

@misc{prabhudessai2026stasis1,
  author       = {Prabhudessai, Shivank Samir},
  title        = {Stasis 1: a 110M-parameter language model trained from scratch on free compute},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/ShivankXD/stasis-1}},
  note         = {Personal research project; checkpoint step 36,420}
}

FineWeb-Edu:

@misc{lozhkov2024fineweb-edu,
  author    = {Lozhkov, Anton and Ben Allal, Loubna and von Werra, Leandro and Wolf, Thomas},
  title     = {FineWeb-Edu: the Finest Collection of Educational Content},
  year      = 2024,
  url       = {https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu},
  doi       = {10.57967/hf/2497},
  publisher = {Hugging Face}
}