mlboydaisuke commited on
Commit
ba61d25
·
verified ·
1 Parent(s): 886422f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +32 -13
README.md CHANGED
@@ -24,7 +24,6 @@ All variants take and return fp32 tensors — swap the `.pte` file, keep your ap
24
  |-----------|------|-----------|------------------------------------|------------------|
25
  | fp32 | `clip_vit_b32_image_xnnpack_fp32.pte` | 351.6 | 1.000000 | 19.0 |
26
  | fp16 | `clip_vit_b32_image_xnnpack_fp16.pte` | 180.7 | 0.999996 | 26.1 |
27
- | int8 (dynamic) | `clip_vit_b32_image_xnnpack_int8.pte` | 95.9 | 0.995739 | 18.4 |
28
  | Core ML (fp16, iOS) | `clip_vit_b32_image_coreml_all.pte` | 176.2 | 0.999998 | 3.5 |
29
 
30
 
@@ -40,9 +39,39 @@ only, not a device number (torch eager fp32 on the same machine: 18.5 ms).
40
 
41
  ### Checked in the task's own units
42
 
43
- Correlation is a first filter. These are the numbers that decide:
 
 
44
 
45
- - **int8 (dynamic)** measured in the units that matter for this model cosine similarity of the image embeddings: median 0.9988 over 10 real images, worst 0.9957.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
  ## Verification (executorch 1.4.0, torch 2.13.0)
48
 
@@ -63,13 +92,3 @@ torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
63
  This repo holds **both towers**: `clip_vit_b32_image_xnnpack_fp32.pte` (image) and
64
  `clip_vit_b32_text_xnnpack_fp32.pte` (text, fixed len 77 + attention mask).
65
  L2-normalize both embeddings, then cosine-match.
66
-
67
- <!-- funnel:v1 -->
68
-
69
- ---
70
-
71
- **More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.
72
-
73
- **Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.
74
-
75
- <!-- /funnel:v1 -->
 
24
  |-----------|------|-----------|------------------------------------|------------------|
25
  | fp32 | `clip_vit_b32_image_xnnpack_fp32.pte` | 351.6 | 1.000000 | 19.0 |
26
  | fp16 | `clip_vit_b32_image_xnnpack_fp16.pte` | 180.7 | 0.999996 | 26.1 |
 
27
  | Core ML (fp16, iOS) | `clip_vit_b32_image_coreml_all.pte` | 176.2 | 0.999998 | 3.5 |
28
 
29
 
 
39
 
40
  ### Checked in the task's own units
41
 
42
+ Correlation is a first filter, and on an embedding it is a weak one: a build can read
43
+ 0.999 against eager and still move an answer, because what decides is whether the error is
44
+ smaller than the gap between the answer a query gets and the runner-up.
45
 
46
+ The test is **near-duplicate retrieval** over 52 photographs, each also present centre-cropped to 90% and re-encoded as JPEG. Every item has one obviously
47
+ correct nearest neighbour, and each build has to find it.
48
+
49
+ | build | correct neighbour kept | worst score shift | budget spent |
50
+ |---|---|---|---|
51
+ | fp32 | 104 of 104 | 0.0000 | 0% |
52
+ | fp16 | 104 of 104 | 0.0023 | 32% |
53
+ | Core ML (fp16, iOS) | 104 of 104 | 0.0030 | 43% |
54
+
55
+ The closest decision eager makes on this set is **0.0071**, and the bar is half of
56
+ it. An earlier version of this card cleared int8 on "cosine similarity of the embeddings",
57
+ which is correlation wearing a task metric's clothes — it never asked whether a retrieval
58
+ answer moved.
59
+
60
+ ## Withdrawn: int8 (2026-08-27)
61
+
62
+ `clip_vit_b32_image_xnnpack_int8.pte` was published and has been **withdrawn**. It was
63
+ cleared on "cosine similarity of the image embeddings, median 0.9988 over 10 real images",
64
+ which never asked whether a retrieval answer moved.
65
+
66
+ Re-measured on the test above it still matches all 104 copies to their originals, but it
67
+ moves a pair score by **0.0404** against a closest decision of **0.0071** — **567% of the
68
+ room available**, against a bar of 50%. Its zero-shot label also moved on one image of the
69
+ 104, where no other build moved any. And the number is generous: this shelf calibrated the
70
+ int8 build on these same photographs.
71
+
72
+ At 95.9 MB it was the smallest build and size was the reason to reach for it. fp16 is
73
+ 180.7 MB and spends 32% of the same budget; the Core ML build is 176.2 MB, spends 43%, and
74
+ runs 5x faster.
75
 
76
  ## Verification (executorch 1.4.0, torch 2.13.0)
77
 
 
92
  This repo holds **both towers**: `clip_vit_b32_image_xnnpack_fp32.pte` (image) and
93
  `clip_vit_b32_text_xnnpack_fp32.pte` (text, fixed len 77 + attention mask).
94
  L2-normalize both embeddings, then cosine-match.