neonforestmist commited on
Commit
00a79fc
·
1 Parent(s): 243ef26

Refine model card positioning and benchmark context

Browse files
Files changed (1) hide show
  1. README.md +103 -70
README.md CHANGED
@@ -11,7 +11,13 @@ tags:
11
  - stable-diffusion
12
  - knowledge-distillation
13
  - compact
 
14
  - local-inference
 
 
 
 
 
15
  widget:
16
  - text: "a glass of red wine"
17
  example_title: "Red wine"
@@ -51,7 +57,7 @@ widget:
51
  url: examples/prompt-gallery/original/image_81.png
52
  ---
53
 
54
- # Clover Image Tiny 🍀
55
 
56
  ![Clover Image Tiny mosaic banner](assets/clover-image-tiny-banner.png)
57
 
@@ -61,41 +67,82 @@ Windows, or Linux.
61
  **323,384,964 denoiser parameters · about 1.67 GB · 4–100 inference steps ·
62
  PyTorch/Diffusers**
63
 
64
- Clover Image Tiny is intentionally a compact, local-first 512×512 model rather
65
- than a frontier-scale checkpoint. Its denoiser has 323,384,964 parameters; the
66
- small-model comparison below gives that scale practical runtime context.
 
67
 
68
- Clover Image Tiny is the public PyTorch/Diffusers checkpoint release behind
69
- these examples. Its output has a recognizable, playful
70
- **DALL·E mini-ish** character. That is a visual description, not a claim of
71
- equivalent architecture, training scale, or benchmark performance.
 
72
 
73
  [**Try Clover Image Tiny in the live ZeroGPU demo →**](https://huggingface.co/spaces/neonforestmist/Clover-Image-Tiny-Demo)
74
 
75
- [**Download the native iPhone project →**](Clover-iOS/)
76
 
77
  The demo exposes prompt, negative prompt, seed, guidance, dimensions,
78
  scheduler, and 4–100 conventional Diffusers inference steps. It creates one
79
- image per request and keeps the packaged safety checker enabled.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
 
81
  ## Contents
82
 
83
- 1. [Examples](#1-examples)
84
- 2. [Small-model benchmark](#2-small-model-benchmark)
85
- 3. [iPhone and Core ML](#3-iphone-and-core-ml)
86
- 4. [Run locally](#4-run-locally)
87
- 5. [Generation controls](#5-generation-controls)
88
- 6. [Hardware and operating systems](#6-hardware-and-operating-systems)
89
- 7. [Python API](#7-python-api)
90
- 8. [About this release](#8-about-this-release)
91
- 9. [Quality and known behavior](#9-quality-and-known-behavior)
92
- 10. [Safety](#10-safety)
93
- 11. [Training lineage and data](#11-training-lineage-and-data)
94
- 12. [Citation](#12-citation)
95
- 13. [Licenses](#13-licenses)
96
- 14. [Reproducibility and artifact identity](#14-reproducibility-and-artifact-identity)
97
-
98
- ## 1. Examples
 
99
 
100
  ### Prompt gallery
101
 
@@ -114,38 +161,24 @@ style adapters. These are visual references for the corresponding LoRA cards.
114
  |---|---|---|---|
115
  | [![Base model greenhouse](examples/normal-to-lora/normal.png)](https://huggingface.co/neonforestmist/Clover-Image-Tiny) | [![Monet greenhouse](examples/normal-to-lora/monet.png)](https://huggingface.co/neonforestmist/clover-image-tiny-monet-lora) | [![Pointillism greenhouse](examples/normal-to-lora/pointillism.png)](https://huggingface.co/neonforestmist/clover-image-tiny-pointillism-lora) | [![Watercolor Anime greenhouse](examples/normal-to-lora/watercolor_anime.png)](https://huggingface.co/neonforestmist/clover-image-tiny-watercolor-anime-lora) |
116
 
117
- ### Reproducible baseline
118
-
119
- ![Eight paired baseline and Clover Image Tiny examples](assets/clover-image-tiny-paired-contact-sheet.png)
120
-
121
- Each row uses the same prompt and seed. The left column is the pinned
122
- BK-SDM-Tiny-2M starting model; the right column is Clover Image Tiny. The
123
- gallery used an NVIDIA L4 in bfloat16, 50 PNDM steps, guidance 7.5, an empty
124
- negative prompt, and 512×512 output. With Diffusers 0.39.0, 50 requested PNDM
125
- steps use 51 U-Net calls because PLMS repeats its first retained timestep.
126
-
127
- All eight Clover images were finite, nonblank, nonblack, and cleared by the
128
- packaged upstream safety checker in this run. The set covers objects, a person,
129
- an animal, a landscape, an interior, food, a product, and a night scene.
130
-
131
- The local MPS reference used “a compact modern library with arched windows,”
132
- seed 1469, and the same 50-step configuration:
133
-
134
- ![Clover Image Tiny local MPS library example](assets/clover-image-tiny-local-mps-library-seed-1469.png)
135
-
136
- ## 2. Small-model benchmark
137
 
138
  Clover is compared with its pinned BK-SDM-Tiny-2M base and two public
139
  same-family references using 16 prompts, identical seeds, 512×512 output, 30
140
  DDIM steps, guidance 7.5, and a shared NVIDIA A10G runtime. The measurement is
141
  an engineering comparison, not a human-preference leaderboard.
142
 
143
- | Model | U-Net parameters | Mean latency | Peak CUDA | Mean CLIP cosine |
144
- |---|---:|---:|---:|---:|
145
- | [Clover Image Tiny](https://huggingface.co/neonforestmist/Clover-Image-Tiny) | 323.4M | 1.024 s | 2,233 MB | 0.3195 |
146
- | [BK-SDM-Tiny-2M](https://huggingface.co/nota-ai/bk-sdm-tiny-2m) | 323.4M | 1.027 s | 2,230 MB | 0.3246 |
147
- | [Segmind Tiny-SD](https://huggingface.co/segmind/tiny-sd) | 323.4M | 1.028 s | 1,649 MB | 0.3345 |
148
- | [BK-SDM-v2-Tiny](https://huggingface.co/nota-ai/bk-sdm-v2-tiny) | 326.8M | 0.957 s | 2,067 MB | 0.3303 |
 
 
 
 
 
149
 
150
  CLIP cosine is only a prompt-adherence proxy. It is not a human-quality score,
151
  FID, safety evaluation, or evidence that these models are interchangeable.
@@ -155,7 +188,7 @@ The complete protocol, machine-readable results, and generated examples are in
155
 
156
  ![Four-prompt small-model comparison](benchmarks/text-to-image/results/clover-small-model-comparison-20260825/contact-sheet.png)
157
 
158
- ## 3. iPhone and Core ML
159
 
160
  The SwiftUI app in [`Clover-iOS`](Clover-iOS/) follows Apple platform
161
  conventions and exposes prompt, negative prompt, steps, guidance, seed, image
@@ -180,7 +213,7 @@ own Core ML picker download:
180
  See [`COREML.md`](COREML.md) for conversion details and
181
  [`training/README.md`](training/README.md) for the pinned LoRA jobs.
182
 
183
- ### 3.1 Inpainting track
184
 
185
  The 9-channel SD 1.4-class inpainting adaptation is trained and packaged separately:
186
  [`neonforestmist/Clover-Image-Tiny-Inpaint`](https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint).
@@ -196,12 +229,12 @@ diverse free-form and object-like masks. It improved held-out masked MAE by
196
  96-pixel mask-context crop; the runtime composites through the exact mask so
197
  unmasked pixels remain unchanged.
198
 
199
- ## 4. Run locally
200
 
201
  Download once, then generate offline with the bundled runner. Python 3.11 and
202
  3.12 are supported.
203
 
204
- ### 4.1 macOS — Apple silicon
205
 
206
  ~~~bash
207
  mkdir clover-image-tiny-local
@@ -232,7 +265,7 @@ open clover-image-tiny.png
232
 
233
  Use `python3.11` instead if that is the installed supported Python.
234
 
235
- ### 4.2 Windows — PowerShell
236
 
237
  ~~~powershell
238
  mkdir clover-image-tiny-local
@@ -264,7 +297,7 @@ Invoke-Item .\clover-image-tiny.png
264
  Use `py -3.11` if needed. With `--device auto`, the runner selects an
265
  available NVIDIA CUDA GPU and otherwise uses CPU.
266
 
267
- ### 4.3 Linux
268
 
269
  ~~~bash
270
  mkdir clover-image-tiny-local
@@ -293,7 +326,7 @@ python model/examples/generate.py \
293
  uses CPU. After the first download, `--local-files-only` prevents network
294
  access during generation.
295
 
296
- ## 5. Generation controls
297
 
298
  The command above is ready to copy. Change these flags to explore the model:
299
 
@@ -324,7 +357,7 @@ outputs are never overwritten.
324
 
325
  Run `python model/examples/generate.py --help` for the complete CLI reference.
326
 
327
- ## 6. Hardware and operating systems
328
 
329
  | System | Automatic backend | Precision | Current evidence |
330
  |---|---|---|---|
@@ -344,7 +377,7 @@ image in 18.21 seconds with fp16 MPS. Its process-lifetime maximum RSS was
344
  631,341,056 bytes. This is a measured point, not a minimum-RAM claim. No Core
345
  ML package is required for the Python path.
346
 
347
- ## 7. Python API
348
 
349
  ~~~python
350
  import torch
@@ -380,7 +413,7 @@ image.save("clover-image-tiny.png")
380
  Seeded generation is repeatable within the selected runtime. Different
381
  devices, dtypes, kernels, and dependency builds can produce different pixels.
382
 
383
- ## 8. About this release
384
 
385
  Clover Image Tiny is a conventional knowledge-distillation checkpoint trained
386
  for 500 optimizer steps on an exact licensed 1,000-pair calibration set. The
@@ -399,7 +432,7 @@ reproducible Core ML conversion and iPhone app source. The downloadable Core ML
399
  artifacts and style adapters are versioned in the separate repositories linked
400
  above.
401
 
402
- ## 9. Quality and known behavior
403
 
404
  - The included gallery demonstrates recognizable subjects across colorful
405
  scenes, products, food, an animal, a landscape, and an interior.
@@ -407,11 +440,11 @@ above.
407
  steps increase runtime but do not guarantee a better result.
408
  - Hands, anatomy, exact counts and relationships, and readable text can be
409
  difficult.
410
- - The paired eight-prompt gallery is a reproducible engineering sample, not a
411
- controlled benchmark or broad human-preference study.
412
  - Resolution and batch size multiply memory use.
413
 
414
- ## 10. Safety
415
 
416
  The upstream safety checker is packaged and enabled in both the supported
417
  runner and hosted demo. A flagged output may be returned as a black placeholder;
@@ -424,7 +457,7 @@ outputs before sharing them. Do not use the model for consequential decisions,
424
  identity claims, medical or legal conclusions, harassment, exploitation,
425
  illegal activity, or uses prohibited by CreativeML OpenRAIL-M.
426
 
427
- ## 11. Training lineage and data
428
 
429
  - Clover fine-tuning data: exactly 1,000 accepted image-caption pairs from
430
  `Spawning/PD3M@2a5eb24a8dccf245acd8e56341761aee06da0bdf`
@@ -444,7 +477,7 @@ for all foundational pretraining is not available to this project.
444
  See `DATA_PROVENANCE.md` for the portable manifest identity and
445
  `MODEL_DATA_LICENSES.md` for the complete component ledger.
446
 
447
- ## 12. Citation
448
 
449
  If Clover Image Tiny is useful in your work, please cite the model release:
450
 
@@ -457,7 +490,7 @@ If Clover Image Tiny is useful in your work, please cite the model release:
457
  }
458
  ```
459
 
460
- ## 13. Licenses
461
 
462
  The model weights are a derivative under **CreativeML OpenRAIL-M**. The example
463
  runner and packaging code are under **Apache-2.0**. Dataset and item-level terms
@@ -469,7 +502,7 @@ request for display in this public model repository. It is not benchmark
469
  evidence, its panel-generation provenance is not claimed, and this package
470
  does not grant a downstream reuse license for it.
471
 
472
- ## 14. Reproducibility and artifact identity
473
 
474
  | Field | Value |
475
  |---|---|
 
11
  - stable-diffusion
12
  - knowledge-distillation
13
  - compact
14
+ - small-model
15
  - local-inference
16
+ - edge-inference
17
+ - mobile-inference
18
+ - core-ml
19
+ - iphone
20
+ - sd-1.4-class
21
  widget:
22
  - text: "a glass of red wine"
23
  example_title: "Red wine"
 
57
  url: examples/prompt-gallery/original/image_81.png
58
  ---
59
 
60
+ # Clover Image Tiny 🍀🤗
61
 
62
  ![Clover Image Tiny mosaic banner](assets/clover-image-tiny-banner.png)
63
 
 
67
  **323,384,964 denoiser parameters · about 1.67 GB · 4–100 inference steps ·
68
  PyTorch/Diffusers**
69
 
70
+ Clover Image Tiny 🤗 is an SD-1.4-class 512×512 diffusion checkpoint distilled
71
+ for compact local inference. It follows the conventional Stable Diffusion 1.x
72
+ text-to-image pipeline contract rather than a modern large-model architecture;
73
+ results vary with the prompt, seed, scheduler, and number of denoising steps.
74
 
75
+ The small-model comparison below is intentionally candid: Clover is not
76
+ presented as a quality winner. Its distinction is the complete, practical
77
+ package around a compact denoiser—local Python inference, Core ML/iPhone
78
+ integration, linked style adapters, prompt-linked examples, and a model card
79
+ with enough provenance to reproduce the documented paths.
80
 
81
  [**Try Clover Image Tiny in the live ZeroGPU demo →**](https://huggingface.co/spaces/neonforestmist/Clover-Image-Tiny-Demo)
82
 
83
+ [**Explore the native iPhone/Core ML implementation →**](Clover-iOS/)
84
 
85
  The demo exposes prompt, negative prompt, seed, guidance, dimensions,
86
  scheduler, and 4–100 conventional Diffusers inference steps. It creates one
87
+ image per request and keeps the packaged safety checker enabled. The Hugging
88
+ Face model pages carry the model cards, tags, prompt galleries, and links to
89
+ the corresponding LoRA and Core ML releases.
90
+
91
+ ## 1. Overview
92
+
93
+ ### Model profile
94
+
95
+ | Profile | Value |
96
+ |---|---:|
97
+ | Published PyTorch/Diffusers package | **~1.67 GB** |
98
+ | Denoiser / U-Net | **323,384,964 parameters** · ~646.8 MB |
99
+ | Text encoder | ~246.1 MB |
100
+ | VAE | ~167.3 MB |
101
+ | Safety checker | ~608.0 MB |
102
+ | Native output | 512×512 |
103
+ | Inference range | 4–100 steps |
104
+ | Published formats | PyTorch/Diffusers + Core ML adapters |
105
+
106
+ The package figure includes the text encoder, VAE, tokenizer/configuration, and
107
+ the enabled upstream safety checker; it is therefore larger than the denoiser
108
+ alone. The component sizes are a practical way to understand the footprint,
109
+ not a claim that every byte is active in every runtime.
110
+
111
+ ### Why it stands out
112
+
113
+ - **A small-model target with a complete workflow.** The 323.4M-parameter
114
+ denoiser is paired with a conventional Diffusers API, deterministic seeds,
115
+ scheduler controls, a safety path, and offline generation after the initial
116
+ download.
117
+ - **Designed for local surfaces.** The same release is documented for Apple
118
+ silicon, CUDA, CPU, and an iPhone/Core ML path, with the heavier resources
119
+ split into versioned Hugging Face repositories.
120
+ - **An extensible style family.** Monet, Pointillism, and Watercolor Anime
121
+ adapters are linked directly from the examples and published as both
122
+ Diffusers LoRAs and Core ML variants.
123
+ - **Clear trade-offs.** The benchmark makes the quality/runtime limits visible
124
+ instead of hiding them behind a frontier-model comparison. The value here is
125
+ compactness, portability, and an unusually integrated 🤗 release surface.
126
 
127
  ## Contents
128
 
129
+ 1. [Overview](#1-overview)
130
+ 2. [Examples](#2-examples)
131
+ 3. [Small-model benchmark](#3-small-model-benchmark)
132
+ 4. [iPhone and Core ML](#4-iphone-and-core-ml)
133
+ 5. [Run locally](#5-run-locally)
134
+ 6. [Generation controls](#6-generation-controls)
135
+ 7. [Hardware and operating systems](#7-hardware-and-operating-systems)
136
+ 8. [Python API](#8-python-api)
137
+ 9. [About this release](#9-about-this-release)
138
+ 10. [Quality and known behavior](#10-quality-and-known-behavior)
139
+ 11. [Safety](#11-safety)
140
+ 12. [Training lineage and data](#12-training-lineage-and-data)
141
+ 13. [Citation](#13-citation)
142
+ 14. [Licenses](#14-licenses)
143
+ 15. [Reproducibility and artifact identity](#15-reproducibility-and-artifact-identity)
144
+
145
+ ## 2. Examples
146
 
147
  ### Prompt gallery
148
 
 
161
  |---|---|---|---|
162
  | [![Base model greenhouse](examples/normal-to-lora/normal.png)](https://huggingface.co/neonforestmist/Clover-Image-Tiny) | [![Monet greenhouse](examples/normal-to-lora/monet.png)](https://huggingface.co/neonforestmist/clover-image-tiny-monet-lora) | [![Pointillism greenhouse](examples/normal-to-lora/pointillism.png)](https://huggingface.co/neonforestmist/clover-image-tiny-pointillism-lora) | [![Watercolor Anime greenhouse](examples/normal-to-lora/watercolor_anime.png)](https://huggingface.co/neonforestmist/clover-image-tiny-watercolor-anime-lora) |
163
 
164
+ ## 3. Small-model benchmark
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
165
 
166
  Clover is compared with its pinned BK-SDM-Tiny-2M base and two public
167
  same-family references using 16 prompts, identical seeds, 512×512 output, 30
168
  DDIM steps, guidance 7.5, and a shared NVIDIA A10G runtime. The measurement is
169
  an engineering comparison, not a human-preference leaderboard.
170
 
171
+ | Model | U-Net parameters | Loaded pipeline parameters | Mean latency | Peak CUDA | Mean CLIP cosine |
172
+ |---|---:|---:|---:|---:|---:|
173
+ | [Clover Image Tiny](https://huggingface.co/neonforestmist/Clover-Image-Tiny) | 323.4M | 834.1M | 1.024 s | 2,233 MB | 0.3195 |
174
+ | [BK-SDM-Tiny-2M](https://huggingface.co/nota-ai/bk-sdm-tiny-2m) | 323.4M | 834.1M | 1.027 s | 2,230 MB | 0.3246 |
175
+ | [Segmind Tiny-SD](https://huggingface.co/segmind/tiny-sd) | 323.4M | 530.1M | 1.028 s | 1,649 MB | 0.3345 |
176
+ | [BK-SDM-v2-Tiny](https://huggingface.co/nota-ai/bk-sdm-v2-tiny) | 326.8M | 750.9M | 0.957 s | 2,067 MB | 0.3303 |
177
+
178
+ The table keeps denoiser size and loaded pipeline size separate. The former is
179
+ the most useful apples-to-apples model comparison; the latter includes the
180
+ text encoder, VAE, and other loaded components and is runtime context rather
181
+ than a download-size metric.
182
 
183
  CLIP cosine is only a prompt-adherence proxy. It is not a human-quality score,
184
  FID, safety evaluation, or evidence that these models are interchangeable.
 
188
 
189
  ![Four-prompt small-model comparison](benchmarks/text-to-image/results/clover-small-model-comparison-20260825/contact-sheet.png)
190
 
191
+ ## 4. iPhone and Core ML
192
 
193
  The SwiftUI app in [`Clover-iOS`](Clover-iOS/) follows Apple platform
194
  conventions and exposes prompt, negative prompt, steps, guidance, seed, image
 
213
  See [`COREML.md`](COREML.md) for conversion details and
214
  [`training/README.md`](training/README.md) for the pinned LoRA jobs.
215
 
216
+ ### 4.1 Inpainting track
217
 
218
  The 9-channel SD 1.4-class inpainting adaptation is trained and packaged separately:
219
  [`neonforestmist/Clover-Image-Tiny-Inpaint`](https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint).
 
229
  96-pixel mask-context crop; the runtime composites through the exact mask so
230
  unmasked pixels remain unchanged.
231
 
232
+ ## 5. Run locally
233
 
234
  Download once, then generate offline with the bundled runner. Python 3.11 and
235
  3.12 are supported.
236
 
237
+ ### 5.1 macOS — Apple silicon
238
 
239
  ~~~bash
240
  mkdir clover-image-tiny-local
 
265
 
266
  Use `python3.11` instead if that is the installed supported Python.
267
 
268
+ ### 5.2 Windows — PowerShell
269
 
270
  ~~~powershell
271
  mkdir clover-image-tiny-local
 
297
  Use `py -3.11` if needed. With `--device auto`, the runner selects an
298
  available NVIDIA CUDA GPU and otherwise uses CPU.
299
 
300
+ ### 5.3 Linux
301
 
302
  ~~~bash
303
  mkdir clover-image-tiny-local
 
326
  uses CPU. After the first download, `--local-files-only` prevents network
327
  access during generation.
328
 
329
+ ## 6. Generation controls
330
 
331
  The command above is ready to copy. Change these flags to explore the model:
332
 
 
357
 
358
  Run `python model/examples/generate.py --help` for the complete CLI reference.
359
 
360
+ ## 7. Hardware and operating systems
361
 
362
  | System | Automatic backend | Precision | Current evidence |
363
  |---|---|---|---|
 
377
  631,341,056 bytes. This is a measured point, not a minimum-RAM claim. No Core
378
  ML package is required for the Python path.
379
 
380
+ ## 8. Python API
381
 
382
  ~~~python
383
  import torch
 
413
  Seeded generation is repeatable within the selected runtime. Different
414
  devices, dtypes, kernels, and dependency builds can produce different pixels.
415
 
416
+ ## 9. About this release
417
 
418
  Clover Image Tiny is a conventional knowledge-distillation checkpoint trained
419
  for 500 optimizer steps on an exact licensed 1,000-pair calibration set. The
 
432
  artifacts and style adapters are versioned in the separate repositories linked
433
  above.
434
 
435
+ ## 10. Quality and known behavior
436
 
437
  - The included gallery demonstrates recognizable subjects across colorful
438
  scenes, products, food, an animal, a landscape, and an interior.
 
440
  steps increase runtime but do not guarantee a better result.
441
  - Hands, anatomy, exact counts and relationships, and readable text can be
442
  difficult.
443
+ - The small-model comparison is an engineering benchmark with a CLIP
444
+ prompt-adherence proxy, not a controlled human-preference study.
445
  - Resolution and batch size multiply memory use.
446
 
447
+ ## 11. Safety
448
 
449
  The upstream safety checker is packaged and enabled in both the supported
450
  runner and hosted demo. A flagged output may be returned as a black placeholder;
 
457
  identity claims, medical or legal conclusions, harassment, exploitation,
458
  illegal activity, or uses prohibited by CreativeML OpenRAIL-M.
459
 
460
+ ## 12. Training lineage and data
461
 
462
  - Clover fine-tuning data: exactly 1,000 accepted image-caption pairs from
463
  `Spawning/PD3M@2a5eb24a8dccf245acd8e56341761aee06da0bdf`
 
477
  See `DATA_PROVENANCE.md` for the portable manifest identity and
478
  `MODEL_DATA_LICENSES.md` for the complete component ledger.
479
 
480
+ ## 13. Citation
481
 
482
  If Clover Image Tiny is useful in your work, please cite the model release:
483
 
 
490
  }
491
  ```
492
 
493
+ ## 14. Licenses
494
 
495
  The model weights are a derivative under **CreativeML OpenRAIL-M**. The example
496
  runner and packaging code are under **Apache-2.0**. Dataset and item-level terms
 
502
  evidence, its panel-generation provenance is not claimed, and this package
503
  does not grant a downstream reuse license for it.
504
 
505
+ ## 15. Reproducibility and artifact identity
506
 
507
  | Field | Value |
508
  |---|---|