yannelli commited on
Commit
be53bd0
·
verified ·
1 Parent(s): 60bd5d7

Fine-tuned checkpoint and model card

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: unknown
4
+ license_link: https://huggingface.co/C-Tianyu/NanoJev
5
+ base_model: C-Tianyu/NanoJev
6
+ library_name: pytorch
7
+ pipeline_tag: text-classification
8
+ tags:
9
+ - decision-model
10
+ - typed-decisions
11
+ - system-one
12
+ - code-review
13
+ - javascript
14
+ - typescript
15
+ ---
16
+
17
+ # NanoJev Code Review Decisions
18
+
19
+ [C-Tianyu/NanoJev](https://huggingface.co/C-Tianyu/NanoJev) fine-tuned to review JavaScript and TypeScript snippets. It answers
20
+ 13 typed questions about a snippet: 10 flaw checks (yes/no), `has_flaw`, `primary_flaw` (a choice over 11 options)
21
+ and `quality` (a 5-level score).
22
+
23
+ ## Results
24
+
25
+ Test split of JS/TS Code Review Decisions (`decisions` config): 300 tasks, 1,200 snippets, 15,600 decisions,
26
+ scored with the dataset's `score_decisions.py`. Latency is the median time for one snippet (13 questions) alone on
27
+ one H100 SXM.
28
+
29
+ | Model | Macro F1 @ 0.5 | Accuracy | Log loss | Brier | ECE | p50 ms |
30
+ |---|---:|---:|---:|---:|---:|---:|
31
+ | This model | 0.452 | 0.831 | 0.459 | 0.173 | 0.038 | 115.4 |
32
+ | This model + validation temperatures | 0.452 | 0.831 | 0.456 | 0.172 | 0.031 | 115.4 |
33
+ | Base model, zero-shot | 0.237 | 0.192 | 1.413 | 0.559 | 0.510 | 116.0 |
34
+ | TF-IDF + logistic regression | 0.502 | 0.830 | 0.449 | 0.172 | 0.030 | - |
35
+
36
+ Per-flag F1 at 0.5: unclear naming 0.93, noise comments 0.99, dead code 0.41, inconsistent style 0.43, over abstracted 0.00, too clever 0.18, duplicated logic 0.00, error handling problem 0.79, type safety hole 0.53, unsafe input 0.25.
37
+
38
+ ## Use
39
+
40
+ This folder is a NanoJev bundle (`best.safetensors`, `config.json`, `backbone_config/`, `tokenizer/`).
41
+ Load it with NanoJev's `DecisionPredictor` from the
42
+ [NanoJev release](https://huggingface.co/C-Tianyu/NanoJev) (revision `unified-games-v1`), or with
43
+ `training/nanojev.py predict --ckpt <folder>`, which also builds the 13 questions.
44
+
45
+ Known weakness: after half an epoch `has_flaw` P(true) never drops below 0.545, so every clean snippet is called
46
+ flawed, and `over_abstracted` and `duplicated_logic` were not learned.
47
+
48
+ ## Training
49
+
50
+ - Data: train split of JS/TS Code Review Decisions (2,400 tasks, 9,600 snippets); state = task, language, code.
51
+ - Full fine-tune of the Qwen3-0.6B backbone and choice head for half an epoch, 1,200 steps of 4 snippets, backbone at 2e-5 and head at 2e-4, bf16 autocast, one H100 (40 min). Every option is encoded with the state and question (27 paths per snippet).
52
+ - Best validation loss: 0.453. `training/` holds the script that trained and scored this model.
53
+
54
+ ## Limits
55
+
56
+ The labels are synthetic and model-reviewed, so scores measure agreement with the dataset's reviewers, not with
57
+ human reviewers. Snippets are 5 to 80 lines of JS/TS; other languages and whole files are untested.
58
+
59
+ ## License
60
+
61
+ Not stated. The NanoJev source is MIT and the Qwen3-0.6B base is Apache 2.0, but the NanoJev release does not state a license for its weights. Settle this with the NanoJev author before making this repo public.
backbone_config/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "dtype": "float32",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 1024,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 3072,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": null,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.17.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
best.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:25d6e2db15fec76421f6c5f5dabf79903d13bb960f43ad7417399d08e57d663b
3
+ size 2385039280
config.json ADDED
@@ -0,0 +1,197 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "input": "/data/rwang/nanojev_sonic_sft_20260920/unified/hard",
3
+ "init_checkpoint": "/tmp/nanojev_sonic_sft_20260920/initial_checkpoint",
4
+ "output_dir": "/data/rwang/nanojev_sonic_sft_20260920/experiment/hard_lr1e5",
5
+ "stage": "sft",
6
+ "loss": "ce",
7
+ "balance": "task",
8
+ "policy_pool_weights": "/data/rwang/nanojev_sonic_sft_20260920/experiment/population_weights.json",
9
+ "retention_fraction": 0.25,
10
+ "reward_samples": 32,
11
+ "steps": 600,
12
+ "head_steps": 0,
13
+ "batch_questions": 24,
14
+ "microbatch_questions": 8,
15
+ "max_microbatch_tokens": 32768,
16
+ "max_length": 8192,
17
+ "eval_every": 100,
18
+ "seed": 17,
19
+ "backbone_lr": 1e-05,
20
+ "head_lr": 0.0001,
21
+ "head_warmup_lr": 0.001,
22
+ "weight_decay": 0.01,
23
+ "precision": "bf16",
24
+ "gradient_checkpointing": true,
25
+ "disable_native_triton": true,
26
+ "td_episodes": null,
27
+ "td_n_step": 3,
28
+ "td_weight": 0.0,
29
+ "target_update_every": 25,
30
+ "validate_only": false,
31
+ "resolved_policy_pool_weights": {
32
+ "maze/policy": 0.3333333333333333,
33
+ "snake/policy": 0.3333333333333333,
34
+ "shooting/policy/basic": 0.16666666666666666,
35
+ "shooting/policy/predict_position": 0.16666666666666666
36
+ },
37
+ "schema_version": "nanojev-unified-games-v1",
38
+ "model": "Qwen/Qwen3-0.6B",
39
+ "set_head": "attention",
40
+ "resolved_model_revision": "c1899de289a04d12100db370d81485cdf75e47ca",
41
+ "initialization": "local DecisionModel warm start; fresh optimizer",
42
+ "init_weights_sha256": "38116340795de1c82369b7fe15819d92d79600a7b4dc7a3cd0d4390cb6782639",
43
+ "data_sha256": {
44
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/manifest.json": "e79f0c3a8559e480db3161781963b1ac7c69be83772eb889208532e4cc5ab80a",
45
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/train.jsonl": "bda8712f89d5db94d80f840f9620f18312c92baca2d8a6df3456218fc807acd0",
46
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/dev.jsonl": "605b40f56e4834b0f234f3328b1e6a8ed41174641986b229e94927a3f05f70bd",
47
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/calibration.jsonl": "6408aa09b9238c724c557c86b68b23f2359140911d84daac7693f9e347fa091b",
48
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/test.jsonl": "2f96cc4e45c67fa375980ff4f355b47716985cf5fb4a1ef034c5e65481b072c9",
49
+ "/data/rwang/nanojev_sonic_sft_20260920/unified/hard/ood.jsonl": "7a778752e0f7699bd709aa6c83bb1ffaaa205568c304909000039b40fb7ff636"
50
+ },
51
+ "implementation_sha256": "a4aedc3b7ad25d875af67943bef6dbc017799f46c4ae0a845fc200e1866fa6cd",
52
+ "continuation_policy_id": null,
53
+ "population_weights": {
54
+ "maze/policy": 0.3333333333333333,
55
+ "snake/policy": 0.3333333333333333,
56
+ "shooting/policy/basic": 0.16666666666666666,
57
+ "shooting/policy/predict_position": 0.16666666666666666
58
+ },
59
+ "policy_pool_weights_sha256": "66afd9cdb9f0479a77896671c201834db1e5889ea081f81aa068dc73359f0449",
60
+ "eligible_by_split_sampling_pool": {
61
+ "calibration/maze/policy": 188,
62
+ "calibration/shooting/policy/basic": 377,
63
+ "calibration/shooting/policy/predict_position": 1006,
64
+ "calibration/snake/policy": 137,
65
+ "dev/maze/policy": 213,
66
+ "dev/shooting/policy/basic": 384,
67
+ "dev/shooting/policy/predict_position": 958,
68
+ "dev/snake/policy": 157,
69
+ "ood/maze/policy": 202,
70
+ "ood/shooting/policy/basic": 589,
71
+ "ood/shooting/policy/predict_position": 1008,
72
+ "ood/snake/policy": 141,
73
+ "test/maze/policy": 211,
74
+ "test/shooting/policy/basic": 756,
75
+ "test/shooting/policy/predict_position": 1413,
76
+ "test/snake/policy": 116,
77
+ "train/maze/policy": 651,
78
+ "train/shooting/policy/basic": 3054,
79
+ "train/shooting/policy/predict_position": 6788,
80
+ "train/snake/policy": 400
81
+ },
82
+ "sampling": "stratified with replacement; per-cell exact population weights; train rows only",
83
+ "parameter_storage": "float32",
84
+ "forward_autocast": "bf16",
85
+ "objective": "Choice CE with explicit API or expert targets plus observed Boolean event loss",
86
+ "selection": "minimum fixed population-weighted dev CE including initial checkpoint; test only after selection",
87
+ "temperature": 1.0,
88
+ "temperature_fitted": false,
89
+ "deps": {
90
+ "torch": "2.14.0",
91
+ "transformers": "5.17.0",
92
+ "safetensors": "0.8.0"
93
+ },
94
+ "gpu": "NVIDIA A100-SXM4-80GB",
95
+ "schema_counts": {
96
+ "records": 18760,
97
+ "outcome_questions": 0,
98
+ "continuation_policy_id": null,
99
+ "questions_by_split_task_role": {
100
+ "calibration/maze/policy": 188,
101
+ "calibration/shooting/policy": 1383,
102
+ "calibration/snake/policy": 138,
103
+ "dev/maze/policy": 215,
104
+ "dev/shooting/policy": 1342,
105
+ "dev/snake/policy": 158,
106
+ "ood/maze/policy": 202,
107
+ "ood/shooting/policy": 1597,
108
+ "ood/snake/policy": 143,
109
+ "test/maze/policy": 211,
110
+ "test/shooting/policy": 2169,
111
+ "test/snake/policy": 116,
112
+ "train/maze/policy": 653,
113
+ "train/shooting/policy": 9842,
114
+ "train/snake/policy": 403
115
+ },
116
+ "eligible_by_split_task_role": {
117
+ "calibration/maze/policy": 188,
118
+ "calibration/shooting/policy": 1383,
119
+ "calibration/snake/policy": 137,
120
+ "dev/maze/policy": 213,
121
+ "dev/shooting/policy": 1342,
122
+ "dev/snake/policy": 157,
123
+ "ood/maze/policy": 202,
124
+ "ood/shooting/policy": 1597,
125
+ "ood/snake/policy": 141,
126
+ "test/maze/policy": 211,
127
+ "test/shooting/policy": 2169,
128
+ "test/snake/policy": 116,
129
+ "train/maze/policy": 651,
130
+ "train/shooting/policy": 9842,
131
+ "train/snake/policy": 400
132
+ },
133
+ "quarantined_policy_questions": 11,
134
+ "policy_questions_by_split_task_target_kind": {
135
+ "calibration/maze/api_policy_distribution": 188,
136
+ "calibration/shooting/expert_action": 1383,
137
+ "calibration/snake/api_policy_distribution": 138,
138
+ "dev/maze/api_policy_distribution": 215,
139
+ "dev/shooting/expert_action": 1342,
140
+ "dev/snake/api_policy_distribution": 158,
141
+ "ood/maze/api_policy_distribution": 202,
142
+ "ood/shooting/expert_action": 1597,
143
+ "ood/snake/api_policy_distribution": 143,
144
+ "test/maze/api_policy_distribution": 211,
145
+ "test/shooting/expert_action": 2169,
146
+ "test/snake/api_policy_distribution": 116,
147
+ "train/maze/api_policy_distribution": 653,
148
+ "train/shooting/expert_action": 9842,
149
+ "train/snake/api_policy_distribution": 403
150
+ },
151
+ "unique_episodes_by_split_task_role": {
152
+ "calibration/maze/policy": 10,
153
+ "calibration/shooting/policy": 128,
154
+ "calibration/snake/policy": 8,
155
+ "dev/maze/policy": 10,
156
+ "dev/shooting/policy": 128,
157
+ "dev/snake/policy": 8,
158
+ "ood/maze/policy": 10,
159
+ "ood/shooting/policy": 256,
160
+ "ood/snake/policy": 8,
161
+ "test/maze/policy": 10,
162
+ "test/shooting/policy": 256,
163
+ "test/snake/policy": 8,
164
+ "train/maze/policy": 32,
165
+ "train/shooting/policy": 1024,
166
+ "train/snake/policy": 28
167
+ },
168
+ "rows_missing_episode_id_by_split_task_role": {},
169
+ "outcome_episode_counts_by_split_task": {},
170
+ "outcome_state_label_counts_by_split_task": {},
171
+ "outcome_question_label_counts_by_split_task": {},
172
+ "episode_audit_note": "Repeated states/questions are not independent episodes. Single-outcome cells are retained without forced balancing. Missing episode IDs are counted explicitly and excluded from episode totals."
173
+ },
174
+ "predictive_samples_are_physical_actions": false,
175
+ "frozen_continuation_policy_is_updated_by_this_script": false,
176
+ "td_enabled": false,
177
+ "td_audit": null,
178
+ "td_implementation_sha256": null,
179
+ "td_contract": {
180
+ "active": false,
181
+ "gamma": 1,
182
+ "n_step_unit": "recorded decision transitions",
183
+ "continuation_policy_id": null,
184
+ "bootstrap_policy": "recorded frozen behavior_probs; never online greedy",
185
+ "target_microbatch_questions": 4,
186
+ "max_microbatch_tokens": 32768,
187
+ "heldout_target": "observed terminal outcome only",
188
+ "outcome_loss": "(1-td_weight)*MC_Brier + td_weight*detached_soft_TD_Brier",
189
+ "initial_target": "deepcopy of initial online model; eval mode; no gradients",
190
+ "prediction_cache": "none; only bootstrap input tokens and indices are cached",
191
+ "bootstrap_cache": null
192
+ },
193
+ "fine_tuned": {
194
+ "from": "C-Tianyu/NanoJev@unified-games-v1",
195
+ "data": "js-ts-code-review-decisions-3000/decisions"
196
+ }
197
+ }
state.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0,
3
+ "step_in_epoch": 1200,
4
+ "step": 1200,
5
+ "val_loss": 0.45309440126499306,
6
+ "best_val_loss": 0.45309440126499306,
7
+ "history": [
8
+ {
9
+ "epoch": 0,
10
+ "step": 1200,
11
+ "val_loss": 0.45309440126499306
12
+ }
13
+ ],
14
+ "train_layers": -1
15
+ }
tokenizer/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
tokenizer/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": true,
24
+ "local_files_only": true,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
training/calibrate.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Temperature scaling for any model's answers: one temperature per question, fitted on validation answers.
2
+
3
+ python calibrate.py --val runs/m/finetuned.validation.jsonl --test runs/m/finetuned.test.jsonl \
4
+ --out runs/m/finetuned-calibrated.test.jsonl
5
+
6
+ Each answer's distribution p becomes softmax(log p / T). T is chosen by validation log loss, so the argmax and the
7
+ noul ranking never change; only confidence moves. Latencies are copied unchanged.
8
+ """
9
+ from __future__ import annotations
10
+
11
+ import argparse
12
+ import json
13
+ import math
14
+ from pathlib import Path
15
+
16
+ import numpy as np
17
+
18
+ from common import gold_index, load_split, options_of, to_answer
19
+
20
+ FLOOR = 1e-12
21
+
22
+
23
+ def read(path: Path) -> dict[str, dict]:
24
+ return {r["id"]: r for r in map(json.loads, path.read_text().splitlines())}
25
+
26
+
27
+ def distribution(answer: dict, question: dict) -> np.ndarray:
28
+ if question["type"] == "noul":
29
+ return np.array([answer["noul"], 1 - answer["noul"]])
30
+ return np.array([answer["probabilities"][o] for o in options_of(question)])
31
+
32
+
33
+ def scaled(p: np.ndarray, t: float) -> np.ndarray:
34
+ z = np.log(np.clip(p, FLOOR, 1.0)) / t
35
+ z = np.exp(z - z.max(axis=-1, keepdims=True))
36
+ return z / z.sum(axis=-1, keepdims=True)
37
+
38
+
39
+ def fit(p: np.ndarray, gold: np.ndarray) -> float:
40
+ """The temperature with the lowest mean log loss, by golden-section search over log T in [-3, 3]."""
41
+ def loss(log_t: float) -> float:
42
+ q = scaled(p, math.exp(log_t))
43
+ return float(-np.log(np.clip(q[np.arange(len(gold)), gold], FLOOR, 1.0)).mean())
44
+
45
+ lo, hi, ratio = -3.0, 3.0, (math.sqrt(5) - 1) / 2
46
+ a, b = hi - ratio * (hi - lo), lo + ratio * (hi - lo)
47
+ for _ in range(60):
48
+ if loss(a) < loss(b):
49
+ hi, b = b, a
50
+ a = hi - ratio * (hi - lo)
51
+ else:
52
+ lo, a = a, b
53
+ b = lo + ratio * (hi - lo)
54
+ return math.exp((lo + hi) / 2)
55
+
56
+
57
+ def main() -> None:
58
+ parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
59
+ parser.add_argument("--val", type=Path, required=True)
60
+ parser.add_argument("--test", type=Path, required=True)
61
+ parser.add_argument("--out", type=Path, required=True)
62
+ args = parser.parse_args()
63
+ val, test = read(args.val), read(args.test)
64
+ cases = [c for c in load_split("validation") if c["id"] in val]
65
+ questions = cases[0]["questions"]
66
+ temperatures = {}
67
+ for name, q in questions.items():
68
+ p = np.stack([distribution(val[c["id"]]["answers"][name], q) for c in cases])
69
+ gold = np.array([gold_index(q, c["gold"][name]) for c in cases])
70
+ temperatures[name] = fit(p, gold)
71
+ with args.out.open("w") as handle:
72
+ for case_id, record in test.items():
73
+ answers = {name: to_answer(q, scaled(distribution(record["answers"][name], q), temperatures[name]).tolist())
74
+ for name, q in questions.items()}
75
+ handle.write(json.dumps({**record, "answers": answers}) + "\n")
76
+ print(json.dumps({"validation_cases": len(cases), "test_cases": len(test),
77
+ "temperatures": {k: round(v, 3) for k, v in temperatures.items()}}, indent=1))
78
+
79
+
80
+ if __name__ == "__main__":
81
+ main()
training/common.py ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Shared pieces for fitting and answering the `decisions` config of JS/TS Code Review Decisions.
2
+
3
+ Every model script reads cases with `load_split` and writes answers with `PredictionWriter`, in the JSONL shape that
4
+ `score_decisions.py` scores: {"id", "answers": {name: System One answer}, "latency_ms"}.
5
+ """
6
+ from __future__ import annotations
7
+
8
+ import json
9
+ import os
10
+ import random
11
+ import time
12
+ from pathlib import Path
13
+
14
+ import pyarrow.parquet as pq
15
+
16
+ HERE = Path(__file__).resolve().parent
17
+ DATA = Path(os.environ.get("DECISIONS_DATA", HERE.parent / "js-ts-code-review-decisions-3000"))
18
+
19
+
20
+ def load_split(split: str, limit: int | None = None, seed: int = 0) -> list[dict]:
21
+ """Cases of one split: id, task_slug, state (task, language, code), questions, and gold label per question.
22
+ `limit` keeps whole tasks (all 4 variants) chosen at random, so a small run still has clean and flawed code."""
23
+ table = pq.read_table(DATA / "decisions" / f"{split}.parquet", columns=["id", "task_slug", "state", "questions", "gold"])
24
+ rows = table.to_pylist()
25
+ if limit is not None and limit < len(rows):
26
+ tasks = sorted({r["task_slug"] for r in rows})
27
+ random.Random(seed).shuffle(tasks)
28
+ keep = set(tasks[: max(1, limit // 4)])
29
+ rows = [r for r in rows if r["task_slug"] in keep]
30
+ return [{"id": r["id"], "task_slug": r["task_slug"], "state": json.loads(r["state"]),
31
+ "questions": json.loads(r["questions"]),
32
+ "gold": {name: g["label"] for name, g in json.loads(r["gold"]).items()}} for r in rows]
33
+
34
+
35
+ def options_of(question: dict) -> list[str]:
36
+ """The option labels `score_decisions.py` scores, in order: true/false, choice keys, or level indexes."""
37
+ if question["type"] == "noul":
38
+ return ["true", "false"]
39
+ if question["type"] == "choice":
40
+ return list(question["criteria"])
41
+ return [str(i) for i in range(len(question["criteria"]))]
42
+
43
+
44
+ def gold_index(question: dict, label: str) -> int:
45
+ return options_of(question).index(label)
46
+
47
+
48
+ def to_answer(question: dict, probs: list[float]) -> dict:
49
+ """A System One answer from probabilities in `options_of` order (a noul's are [true, false])."""
50
+ probs = [float(p) for p in probs]
51
+ if question["type"] == "noul":
52
+ return {"type": "noul", "noul": probs[0]}
53
+ return {"type": question["type"], "probabilities": dict(zip(options_of(question), probs))}
54
+
55
+
56
+ def state_text(state: dict) -> str:
57
+ """One string for models that take plain text: the task, the language, then the code."""
58
+ return f"Task: {state['task']}\nLanguage: {state['language']}\n\n{state['code']}"
59
+
60
+
61
+ class PredictionWriter:
62
+ """Appends one answered case per line and flushes it, so a stopped run resumes where it left off."""
63
+
64
+ def __init__(self, path: str | Path):
65
+ self.path = Path(path)
66
+ self.path.parent.mkdir(parents=True, exist_ok=True)
67
+ self.done: set[str] = set()
68
+ if self.path.exists():
69
+ for line in self.path.read_text().splitlines():
70
+ try:
71
+ self.done.add(json.loads(line)["id"])
72
+ except (json.JSONDecodeError, KeyError):
73
+ break # a line cut short by a stop; everything before it is kept
74
+ self._handle = self.path.open("a")
75
+
76
+ def pending(self, cases: list[dict]) -> list[dict]:
77
+ return [c for c in cases if c["id"] not in self.done]
78
+
79
+ def write(self, case_id: str, answers: dict, latency_ms: float | None = None) -> None:
80
+ record = {"id": case_id, "answers": answers}
81
+ if latency_ms is not None:
82
+ record["latency_ms"] = round(latency_ms, 3)
83
+ self._handle.write(json.dumps(record) + "\n")
84
+ self._handle.flush()
85
+ self.done.add(case_id)
86
+
87
+ def close(self) -> None:
88
+ self._handle.close()
89
+
90
+
91
+ def log(path: str | Path | None, **fields) -> None:
92
+ """Prints a JSON line and appends it to `path` when given (training curves, timings)."""
93
+ line = json.dumps({"time": round(time.time(), 1), **fields})
94
+ print(line, flush=True)
95
+ if path:
96
+ Path(path).parent.mkdir(parents=True, exist_ok=True)
97
+ with open(path, "a") as handle:
98
+ handle.write(line + "\n")
training/nanojev.py ADDED
@@ -0,0 +1,383 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """NanoJev (C-Tianyu/NanoJev @ unified-games-v1) on the `decisions` config: answer zero-shot, fine-tune, answer again.
3
+
4
+ NanoJev encodes every candidate path as state + question + candidate in a Qwen3-0.6B backbone and scores the offered
5
+ set with its own head, with no decoding. A noul is posed as a NanoJev boolean (one path, logits [0, z]), `primary_flaw`
6
+ as a choice (11 paths and the attention set head) and `quality` as an ordered score (5 paths). Request format,
7
+ tokenization, forward and probabilities all come from NanoJev's own source, which is hash-checked before it is loaded.
8
+
9
+ python nanojev.py zeroshot --split test --out PRED.jsonl [--limit N] [--device D]
10
+ python nanojev.py train --ckpt DIR [--epochs 2] [--limit N] [--max-steps S] [--device D] [--log PATH]
11
+ python nanojev.py predict --ckpt DIR --split test --out PRED.jsonl [--limit N] [--device D]
12
+
13
+ Set NANOJEV_REPO to use another download directory; a missing release is fetched there (inference bundle only).
14
+ """
15
+ from __future__ import annotations
16
+
17
+ import argparse
18
+ import contextlib
19
+ import functools
20
+ import hashlib
21
+ import importlib.util
22
+ import json
23
+ import math
24
+ import os
25
+ import random
26
+ import shutil
27
+ import time
28
+ from pathlib import Path
29
+
30
+ import torch
31
+ import torch.nn.functional as F
32
+
33
+ from common import PredictionWriter, load_split, log, options_of, state_text, to_answer
34
+
35
+ REPO_ID, REVISION = "C-Tianyu/NanoJev", "unified-games-v1"
36
+ REPO = Path(os.environ.get("NANOJEV_REPO", Path.home() / ".cache" / "decision-models-scratch" / "nanojev-repo"))
37
+ BUNDLE = ["best.safetensors", "config.json", "backbone_config/*", "tokenizer/*", "source/*", "*.md"]
38
+ SOURCE_SHA256 = {
39
+ "predict_toy_decisions.py": "6eedc49aaf9a81763677e3cff4e87fc4e499bfc5decf0bb4678f452f578dd92b",
40
+ "train_toy_decisions.py": "4f39babd5575e43d7acf3eca357329a93c60041f334bc2bd7424929bf9c0da86",
41
+ }
42
+ STATE_CAP = 1024 # state tokens; the longest train state is 992 tokens, so the cap truncates no train case
43
+ TYPES = {"noul": "boolean", "choice": "choice", "score": "score"}
44
+
45
+
46
+ @functools.cache
47
+ def nanojev_source():
48
+ """NanoJev's `predict_toy_decisions` module, loaded by path after its files match the reviewed hashes."""
49
+ if not (REPO / "best.safetensors").exists():
50
+ from huggingface_hub import snapshot_download
51
+ snapshot_download(REPO_ID, revision=REVISION, local_dir=REPO, allow_patterns=BUNDLE, token=False)
52
+ scripts = REPO / "source" / "scripts"
53
+ for name, digest in SOURCE_SHA256.items():
54
+ if hashlib.sha256((scripts / name).read_bytes()).hexdigest() != digest:
55
+ raise RuntimeError(f"{scripts / name} is not the reviewed NanoJev source; refusing to run it")
56
+ spec = importlib.util.spec_from_file_location("nanojev_predict", scripts / "predict_toy_decisions.py")
57
+ module = importlib.util.module_from_spec(spec)
58
+ spec.loader.exec_module(module)
59
+ return module
60
+
61
+
62
+ def load_model(root: Path, device: torch.device) -> tuple[torch.nn.Module, object, int]:
63
+ """Model, tokenizer and path-length limit of one NanoJev bundle (the release or a fine-tuned checkpoint)."""
64
+ nj = nanojev_source()
65
+ if device.type == "cuda":
66
+ predictor = nj.DecisionPredictor(str(root), device_name=str(device), precision="bf16",
67
+ disable_native_triton=True)
68
+ return predictor.model, predictor.tokenizer, predictor.limit
69
+ # DecisionPredictor refuses non-CUDA devices, so on CPU these are its own loading steps without that check.
70
+ from safetensors.torch import load_file
71
+ from torch._native import triton_utils
72
+ from transformers import AutoConfig, AutoModel, AutoTokenizer
73
+ triton_utils.deregister_op_overrides()
74
+ _, paths = nj.local_checkpoint_files(root)
75
+ run = nj.read_json(paths["run_config"])
76
+ tokenizer = AutoTokenizer.from_pretrained(str(paths["tokenizer"]), local_files_only=True)
77
+ if tokenizer.pad_token_id is None:
78
+ tokenizer.pad_token = tokenizer.eos_token
79
+ config = AutoConfig.from_pretrained(str(paths["body_config"]), local_files_only=True)
80
+ config.use_cache = False
81
+ body = AutoModel.from_config(config, attn_implementation="sdpa").float()
82
+ model = nj.load_decision_model_class()(body, run["set_head"])
83
+ model.load_state_dict(load_file(str(paths["weights"])), strict=True)
84
+ return model.to(device).eval(), tokenizer, run.get("max_length", 512)
85
+
86
+
87
+ def autocast(device: torch.device):
88
+ """BF16 autocast over FP32 weights on CUDA, as NanoJev was trained; plain FP32 on CPU."""
89
+ return torch.autocast("cuda", dtype=torch.bfloat16) if device.type == "cuda" else contextlib.nullcontext()
90
+
91
+
92
+ def capped_state(case: dict, tokenizer, cap: int) -> tuple[str, bool]:
93
+ """`common.state_text`, cut to `cap` tokens with a visible marker when longer."""
94
+ text = state_text(case["state"])
95
+ ids = tokenizer.encode(text, add_special_tokens=False)
96
+ if len(ids) <= cap:
97
+ return text, False
98
+ return tokenizer.decode(ids[:cap]) + "\n[code truncated]", True
99
+
100
+
101
+ def request(cases: list[dict], states: dict[str, str]) -> dict:
102
+ """NanoJev's request: one state per case with its 13 questions in NanoJev's boolean, choice and score types."""
103
+ return {"states": [{"id": case["id"], "state": states[case["id"]],
104
+ "questions": {name: {"type": TYPES[q["type"]], "instructions": q["instructions"],
105
+ "criteria": q["criteria"]} for name, q in case["questions"].items()}}
106
+ for case in cases]}
107
+
108
+
109
+ def examples_for(cases: list[dict], states: dict[str, str], tokenizer, limit: int, gold: bool = True) -> list[dict]:
110
+ """NanoJev's tokenized questions (all candidate paths each), with the gold candidate index for training."""
111
+ examples = nanojev_source().prepare_examples(request(cases, states), tokenizer, limit)
112
+ golds = {case["id"]: case["gold"] for case in cases} if gold else {}
113
+ for ex in examples if gold else []:
114
+ ex["gold_index"] = ex["candidate_ids"].index(golds[ex["state_id"]][ex["qid"]])
115
+ return examples
116
+
117
+
118
+ def answers_of(case: dict, examples: list[dict], logits: torch.Tensor) -> dict:
119
+ """System One answers in `options_of` order from NanoJev's per-question softmax over its candidates."""
120
+ answers = {}
121
+ for ex, values in zip(examples, logits):
122
+ probs = values[: len(ex["candidate_ids"])].float().softmax(-1).cpu().tolist()
123
+ result = nanojev_source().answer_from_probabilities(ex, probs)
124
+ question = case["questions"][ex["qid"]]
125
+ if question["type"] == "noul":
126
+ vector = [result["p_true"], 1 - result["p_true"]]
127
+ else:
128
+ vector = [result["probabilities"][o] for o in options_of(question)]
129
+ answers[ex["qid"]] = to_answer(question, vector)
130
+ return answers
131
+
132
+
133
+ def microbatches(examples: list[dict], budget: int) -> list[list[dict]]:
134
+ """Whole questions sorted by length and grouped so paths x longest path stays within `budget` tokens."""
135
+ groups, group, paths, width = [], [], 0, 0
136
+ for ex in sorted(examples, key=lambda e: max(map(len, e["leaf_tokens"]))):
137
+ n, size = len(ex["leaf_tokens"]), max(map(len, ex["leaf_tokens"]))
138
+ if group and (paths + n) * max(width, size) > budget:
139
+ groups.append(group)
140
+ group, paths, width = [], 0, 0
141
+ group.append(ex)
142
+ paths, width = paths + n, max(width, size)
143
+ return groups + [group] if group else groups
144
+
145
+
146
+ def padded_tokens(group: list[dict]) -> int:
147
+ return sum(len(ex["leaf_tokens"]) for ex in group) * max(len(p) for ex in group for p in ex["leaf_tokens"])
148
+
149
+
150
+ def group_loss(model: torch.nn.Module, group: list[dict], pad: int, device: torch.device) -> torch.Tensor:
151
+ """Summed cross entropy of each question's gold candidate; padded candidates sit at -1e9 and get no mass."""
152
+ with autocast(device):
153
+ logits, _ = model(group, pad)
154
+ gold = torch.tensor([ex["gold_index"] for ex in group], device=logits.device)
155
+ return F.cross_entropy(logits.float(), gold, reduction="sum")
156
+
157
+
158
+ def bundle_of(ckpt: Path) -> Path:
159
+ """The bundle `predict` loads from a checkpoint directory: final, else the last saved step, else DIR itself."""
160
+ for path in (ckpt / "final", ckpt / "last", ckpt / "last.old"):
161
+ if (path / "best.safetensors").exists():
162
+ return path
163
+ return ckpt
164
+
165
+
166
+ def answer_split(root: Path, args: argparse.Namespace) -> None:
167
+ """Answers each pending case alone (all 13 questions in one forward) and records its wall-clock latency."""
168
+ device = torch.device(args.device)
169
+ model, tokenizer, limit = load_model(root, device)
170
+ writer = PredictionWriter(args.out)
171
+ cases = writer.pending(load_split(args.split, args.limit))
172
+ log(None, event="answer", split=args.split, bundle=str(root), device=str(device), pending=len(cases),
173
+ done=len(writer.done), state_cap=args.state_cap)
174
+ truncated, latencies = 0, []
175
+ with torch.inference_mode():
176
+ for i, case in enumerate(cases, 1):
177
+ start = time.perf_counter()
178
+ text, cut = capped_state(case, tokenizer, args.state_cap)
179
+ examples = examples_for([case], {case["id"]: text}, tokenizer, limit, gold=False)
180
+ with autocast(device):
181
+ logits, _ = model(examples, tokenizer.pad_token_id)
182
+ answers = answers_of(case, examples, logits)
183
+ if device.type == "cuda":
184
+ torch.cuda.synchronize()
185
+ elif device.type == "mps":
186
+ torch.mps.synchronize()
187
+ latencies.append((time.perf_counter() - start) * 1000)
188
+ writer.write(case["id"], answers, latencies[-1])
189
+ truncated += cut
190
+ if i % 50 == 0 or i == len(cases):
191
+ log(None, event="progress", answered=i, of=len(cases), last_ms=round(latencies[-1], 1))
192
+ writer.close()
193
+ if latencies:
194
+ ordered = sorted(latencies)
195
+ log(None, event="answered", cases=len(latencies), truncated=truncated, out=str(args.out),
196
+ median_ms=round(ordered[len(ordered) // 2], 1), mean_ms=round(sum(ordered) / len(ordered), 1))
197
+
198
+
199
+ def save_bundle(model: torch.nn.Module, dest: Path, state: dict, optimizer=None) -> None:
200
+ """A NanoJev bundle that DecisionPredictor loads, plus state.json: written to DIR.tmp, then swapped in by rename."""
201
+ from safetensors.torch import save_file
202
+ tmp, old = dest.with_name(dest.name + ".tmp"), dest.with_name(dest.name + ".old")
203
+ shutil.rmtree(tmp, ignore_errors=True)
204
+ tmp.mkdir(parents=True)
205
+ for name in ("backbone_config", "tokenizer"):
206
+ shutil.copytree(REPO / name, tmp / name)
207
+ run = json.loads((REPO / "config.json").read_text())
208
+ run["fine_tuned"] = {"from": f"{REPO_ID}@{REVISION}", "data": "js-ts-code-review-decisions-3000/decisions"}
209
+ (tmp / "config.json").write_text(json.dumps(run, indent=1))
210
+ save_file({k: v.detach().cpu().contiguous() for k, v in model.state_dict().items()}, str(tmp / "best.safetensors"))
211
+ if optimizer is not None:
212
+ torch.save(optimizer.state_dict(), tmp / "optimizer.pt")
213
+ (tmp / "state.json").write_text(json.dumps(state, indent=1))
214
+ shutil.rmtree(old, ignore_errors=True)
215
+ if dest.exists():
216
+ dest.rename(old)
217
+ tmp.rename(dest)
218
+ shutil.rmtree(old, ignore_errors=True)
219
+
220
+
221
+ def prepare_states(cases: list[dict], tokenizer, cap: int, limit: int) -> tuple[dict[str, str], int, int]:
222
+ """Capped state text per case, how many were truncated, and the unpadded candidate-path tokens of one pass."""
223
+ states, truncated, tokens = {}, 0, 0
224
+ for case in cases:
225
+ states[case["id"]], cut = capped_state(case, tokenizer, cap)
226
+ truncated += cut
227
+ examples = examples_for([case], states, tokenizer, limit)
228
+ tokens += sum(len(path) for ex in examples for path in ex["leaf_tokens"])
229
+ return states, truncated, tokens
230
+
231
+
232
+ def validation_loss(model: torch.nn.Module, cases: list[dict], states: dict[str, str], tokenizer, limit: int,
233
+ device: torch.device, budget: int) -> float:
234
+ """Mean per-question cross entropy over the validation cases."""
235
+ model.eval()
236
+ total, count = 0.0, 0
237
+ with torch.no_grad():
238
+ for i in range(0, len(cases), 4):
239
+ for group in microbatches(examples_for(cases[i:i + 4], states, tokenizer, limit), budget):
240
+ total += float(group_loss(model, group, tokenizer.pad_token_id, device))
241
+ count += len(group)
242
+ model.train()
243
+ return total / count
244
+
245
+
246
+ def set_trainable(model: torch.nn.Module, layers: int) -> list[torch.nn.Parameter]:
247
+ """Trains the head plus the last `layers` decoder layers (and final norm); all layers means a full fine-tune."""
248
+ blocks = model.backbone.layers
249
+ full = layers >= len(blocks)
250
+ for param in model.backbone.parameters():
251
+ param.requires_grad_(full)
252
+ if not full:
253
+ for module in [*blocks[len(blocks) - layers:], model.backbone.norm] if layers else []:
254
+ module.requires_grad_(True)
255
+ return [p for p in model.parameters() if p.requires_grad]
256
+
257
+
258
+ def lr_scale(step: int, total: int, warmup: int) -> float:
259
+ """Linear warmup over `warmup` steps, then linear decay to zero at `total`."""
260
+ return min(1.0, (step + 1) / warmup) * max(0.0, min(1.0, (total - step) / max(1, total - warmup)))
261
+
262
+
263
+ def train(args: argparse.Namespace) -> None:
264
+ """Supervised cross entropy over every question's options; resumes from DIR/last and keeps the best as DIR/final."""
265
+ device, ckpt = torch.device(args.device), Path(args.ckpt)
266
+ last = next((p for p in (ckpt / "last", ckpt / "last.old") if (p / "state.json").exists()), None)
267
+ state = json.loads((last / "state.json").read_text()) if last else {
268
+ "epoch": 0, "step_in_epoch": 0, "step": 0, "val_loss": None, "best_val_loss": None, "history": [],
269
+ "train_layers": args.train_layers if args.train_layers is not None else (-1 if device.type == "cuda" else 2)}
270
+ train_cases = load_split("train", args.limit)
271
+ per_epoch = math.ceil(len(train_cases) / args.batch_cases)
272
+ total = args.epochs * per_epoch
273
+ stop = min(total, args.max_steps or total)
274
+ if state["step"] >= stop:
275
+ log(args.log, event="done", step=state["step"], stop=stop, best_val_loss=state["best_val_loss"],
276
+ final=str(ckpt / "final"))
277
+ return
278
+ model, tokenizer, limit = load_model(last or REPO, device)
279
+ val_cases = load_split("validation", min(args.val_cases, args.limit or args.val_cases))
280
+ started = time.perf_counter()
281
+ states, train_cut, epoch_tokens = prepare_states(train_cases, tokenizer, args.state_cap, limit)
282
+ val_states, val_cut, _ = prepare_states(val_cases, tokenizer, args.state_cap, limit)
283
+ states.update(val_states)
284
+ layers = len(model.backbone.layers) if state["train_layers"] < 0 else state["train_layers"]
285
+ params = set_trainable(model, layers)
286
+ head = {id(p) for n, p in model.named_parameters() if not n.startswith("backbone.")}
287
+ optimizer = torch.optim.AdamW([{"params": [p for p in params if id(p) not in head], "lr": args.lr},
288
+ {"params": [p for p in params if id(p) in head], "lr": args.head_lr}],
289
+ weight_decay=0.01)
290
+ if last:
291
+ optimizer.load_state_dict(torch.load(last / "optimizer.pt", map_location=device))
292
+ model.backbone.gradient_checkpointing_enable(gradient_checkpointing_kwargs={"use_reentrant": False})
293
+ model.train()
294
+ warmup = max(1, min(args.warmup, total // 10))
295
+ log(args.log, event="start", resume_from=str(last) if last else None, step=state["step"], stop=stop,
296
+ steps_per_epoch=per_epoch, train_cases=len(train_cases), val_cases=len(val_cases), state_cap=args.state_cap,
297
+ train_truncated=train_cut, val_truncated=val_cut, unpadded_tokens_per_epoch=epoch_tokens,
298
+ train_layers=layers, trainable_params=sum(p.numel() for p in params), warmup=warmup,
299
+ prepare_s=round(time.perf_counter() - started, 1), device=str(device))
300
+ for epoch in range(state["epoch"], args.epochs):
301
+ order = list(range(len(train_cases)))
302
+ random.Random(args.seed + epoch).shuffle(order)
303
+ for b in range(state["step_in_epoch"], per_epoch):
304
+ tick = time.perf_counter()
305
+ batch = [train_cases[i] for i in order[b * args.batch_cases:(b + 1) * args.batch_cases]]
306
+ examples = examples_for(batch, states, tokenizer, limit)
307
+ scale = lr_scale(state["step"], stop, warmup)
308
+ for group, base in zip(optimizer.param_groups, (args.lr, args.head_lr)):
309
+ group["lr"] = base * scale
310
+ optimizer.zero_grad(set_to_none=True)
311
+ loss_sum, tokens = 0.0, 0
312
+ for group in microbatches(examples, args.micro_tokens):
313
+ loss = group_loss(model, group, tokenizer.pad_token_id, device) / len(examples)
314
+ loss.backward()
315
+ loss_sum, tokens = loss_sum + float(loss.detach()), tokens + padded_tokens(group)
316
+ grad_norm = torch.nn.utils.clip_grad_norm_(params, 1.0)
317
+ optimizer.step()
318
+ state.update(step=state["step"] + 1, step_in_epoch=b + 1)
319
+ seconds = time.perf_counter() - tick
320
+ if state["step"] % args.log_every == 0 or state["step"] in (1, stop):
321
+ log(args.log, event="step", epoch=epoch, step=state["step"], loss=round(loss_sum, 5),
322
+ lr=args.lr * scale, grad_norm=round(float(grad_norm), 4), padded_tokens=tokens,
323
+ tokens_per_s=round(tokens / seconds), step_s=round(seconds, 2))
324
+ if state["step"] >= stop:
325
+ break
326
+ if state["step"] % args.save_every == 0 and b + 1 < per_epoch:
327
+ save_bundle(model, ckpt / "last", state, optimizer)
328
+ log(args.log, event="saved", step=state["step"], checkpoint=str(ckpt / "last"))
329
+ if state["step_in_epoch"] >= per_epoch:
330
+ state.update(epoch=epoch + 1, step_in_epoch=0)
331
+ state["val_loss"] = validation_loss(model, val_cases, states, tokenizer, limit, device, args.micro_tokens)
332
+ state["history"].append({"epoch": state["epoch"], "step": state["step"], "val_loss": state["val_loss"]})
333
+ best = state["best_val_loss"] is None or state["val_loss"] < state["best_val_loss"]
334
+ if best:
335
+ state["best_val_loss"] = state["val_loss"]
336
+ save_bundle(model, ckpt / "final", state)
337
+ save_bundle(model, ckpt / "last", state, optimizer)
338
+ log(args.log, event="validated", epoch=state["epoch"], step=state["step"], val_loss=round(state["val_loss"], 5),
339
+ best=best, elapsed_s=round(time.perf_counter() - started, 1))
340
+ if state["step"] >= stop:
341
+ break
342
+ log(args.log, event="finished", step=state["step"], best_val_loss=state["best_val_loss"], final=str(ckpt / "final"))
343
+
344
+
345
+ def main() -> None:
346
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347
+ commands = parser.add_subparsers(dest="command", required=True)
348
+ for name in ("zeroshot", "train", "predict"):
349
+ command = commands.add_parser(name)
350
+ command.add_argument("--limit", type=int, help="whole tasks covering about N cases")
351
+ command.add_argument("--device", default="cuda:0" if torch.cuda.is_available() else "cpu")
352
+ command.add_argument("--state-cap", type=int, default=STATE_CAP, help="state tokens kept per case")
353
+ if name != "train":
354
+ command.add_argument("--split", choices=["validation", "test"], required=True)
355
+ command.add_argument("--out", required=True)
356
+ if name != "zeroshot":
357
+ command.add_argument("--ckpt", required=True)
358
+ fit = commands.choices["train"]
359
+ fit.add_argument("--epochs", type=int, default=2)
360
+ fit.add_argument("--max-steps", type=int, help="stop after this many optimizer steps in total")
361
+ fit.add_argument("--log", help="also append the JSON progress lines here")
362
+ fit.add_argument("--batch-cases", type=int, default=4, help="cases (13 questions each) per optimizer step")
363
+ fit.add_argument("--micro-tokens", type=int, help="padded tokens per forward (default 32768 CUDA, 8192 CPU)")
364
+ fit.add_argument("--lr", type=float, default=2e-5, help="backbone learning rate")
365
+ fit.add_argument("--head-lr", type=float, default=2e-4, help="head learning rate (NanoJev used 10x backbone)")
366
+ fit.add_argument("--warmup", type=int, default=100)
367
+ fit.add_argument("--train-layers", type=int, help="last decoder layers to train (default all on CUDA, 2 on CPU)")
368
+ fit.add_argument("--val-cases", type=int, default=300)
369
+ fit.add_argument("--save-every", type=int, default=500)
370
+ fit.add_argument("--log-every", type=int, default=20)
371
+ fit.add_argument("--seed", type=int, default=0)
372
+ args = parser.parse_args()
373
+ if args.command == "zeroshot":
374
+ answer_split(REPO, args)
375
+ elif args.command == "predict":
376
+ answer_split(bundle_of(Path(args.ckpt)), args)
377
+ else:
378
+ args.micro_tokens = args.micro_tokens or (32768 if args.device.startswith("cuda") else 8192)
379
+ train(args)
380
+
381
+
382
+ if __name__ == "__main__":
383
+ main()