Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -24,7 +24,6 @@ All variants take and return fp32 tensors — swap the `.pte` file, keep your ap
|
|
| 24 |
|-----------|------|-----------|------------------------------------|------------------|
|
| 25 |
| fp32 | `clip_vit_b32_image_xnnpack_fp32.pte` | 351.6 | 1.000000 | 19.0 |
|
| 26 |
| fp16 | `clip_vit_b32_image_xnnpack_fp16.pte` | 180.7 | 0.999996 | 26.1 |
|
| 27 |
-
| int8 (dynamic) | `clip_vit_b32_image_xnnpack_int8.pte` | 95.9 | 0.995739 | 18.4 |
|
| 28 |
| Core ML (fp16, iOS) | `clip_vit_b32_image_coreml_all.pte` | 176.2 | 0.999998 | 3.5 |
|
| 29 |
|
| 30 |
|
|
@@ -40,9 +39,39 @@ only, not a device number (torch eager fp32 on the same machine: 18.5 ms).
|
|
| 40 |
|
| 41 |
### Checked in the task's own units
|
| 42 |
|
| 43 |
-
Correlation is a first filter
|
|
|
|
|
|
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
## Verification (executorch 1.4.0, torch 2.13.0)
|
| 48 |
|
|
@@ -63,13 +92,3 @@ torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
|
|
| 63 |
This repo holds **both towers**: `clip_vit_b32_image_xnnpack_fp32.pte` (image) and
|
| 64 |
`clip_vit_b32_text_xnnpack_fp32.pte` (text, fixed len 77 + attention mask).
|
| 65 |
L2-normalize both embeddings, then cosine-match.
|
| 66 |
-
|
| 67 |
-
<!-- funnel:v1 -->
|
| 68 |
-
|
| 69 |
-
---
|
| 70 |
-
|
| 71 |
-
**More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.
|
| 72 |
-
|
| 73 |
-
**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.
|
| 74 |
-
|
| 75 |
-
<!-- /funnel:v1 -->
|
|
|
|
| 24 |
|-----------|------|-----------|------------------------------------|------------------|
|
| 25 |
| fp32 | `clip_vit_b32_image_xnnpack_fp32.pte` | 351.6 | 1.000000 | 19.0 |
|
| 26 |
| fp16 | `clip_vit_b32_image_xnnpack_fp16.pte` | 180.7 | 0.999996 | 26.1 |
|
|
|
|
| 27 |
| Core ML (fp16, iOS) | `clip_vit_b32_image_coreml_all.pte` | 176.2 | 0.999998 | 3.5 |
|
| 28 |
|
| 29 |
|
|
|
|
| 39 |
|
| 40 |
### Checked in the task's own units
|
| 41 |
|
| 42 |
+
Correlation is a first filter, and on an embedding it is a weak one: a build can read
|
| 43 |
+
0.999 against eager and still move an answer, because what decides is whether the error is
|
| 44 |
+
smaller than the gap between the answer a query gets and the runner-up.
|
| 45 |
|
| 46 |
+
The test is **near-duplicate retrieval** over 52 photographs, each also present centre-cropped to 90% and re-encoded as JPEG. Every item has one obviously
|
| 47 |
+
correct nearest neighbour, and each build has to find it.
|
| 48 |
+
|
| 49 |
+
| build | correct neighbour kept | worst score shift | budget spent |
|
| 50 |
+
|---|---|---|---|
|
| 51 |
+
| fp32 | 104 of 104 | 0.0000 | 0% |
|
| 52 |
+
| fp16 | 104 of 104 | 0.0023 | 32% |
|
| 53 |
+
| Core ML (fp16, iOS) | 104 of 104 | 0.0030 | 43% |
|
| 54 |
+
|
| 55 |
+
The closest decision eager makes on this set is **0.0071**, and the bar is half of
|
| 56 |
+
it. An earlier version of this card cleared int8 on "cosine similarity of the embeddings",
|
| 57 |
+
which is correlation wearing a task metric's clothes — it never asked whether a retrieval
|
| 58 |
+
answer moved.
|
| 59 |
+
|
| 60 |
+
## Withdrawn: int8 (2026-08-27)
|
| 61 |
+
|
| 62 |
+
`clip_vit_b32_image_xnnpack_int8.pte` was published and has been **withdrawn**. It was
|
| 63 |
+
cleared on "cosine similarity of the image embeddings, median 0.9988 over 10 real images",
|
| 64 |
+
which never asked whether a retrieval answer moved.
|
| 65 |
+
|
| 66 |
+
Re-measured on the test above it still matches all 104 copies to their originals, but it
|
| 67 |
+
moves a pair score by **0.0404** against a closest decision of **0.0071** — **567% of the
|
| 68 |
+
room available**, against a bar of 50%. Its zero-shot label also moved on one image of the
|
| 69 |
+
104, where no other build moved any. And the number is generous: this shelf calibrated the
|
| 70 |
+
int8 build on these same photographs.
|
| 71 |
+
|
| 72 |
+
At 95.9 MB it was the smallest build and size was the reason to reach for it. fp16 is
|
| 73 |
+
180.7 MB and spends 32% of the same budget; the Core ML build is 176.2 MB, spends 43%, and
|
| 74 |
+
runs 5x faster.
|
| 75 |
|
| 76 |
## Verification (executorch 1.4.0, torch 2.13.0)
|
| 77 |
|
|
|
|
| 92 |
This repo holds **both towers**: `clip_vit_b32_image_xnnpack_fp32.pte` (image) and
|
| 93 |
`clip_vit_b32_text_xnnpack_fp32.pte` (text, fixed len 77 + attention mask).
|
| 94 |
L2-normalize both embeddings, then cosine-match.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|