florianvoss commited on
Commit
7200ce4
·
verified ·
1 Parent(s): 6c7f3d0

Align Whisper model card and compilation scripts with SiMa.ai packages

Browse files
Files changed (4) hide show
  1. README.md +198 -56
  2. SHA256SUMS +3 -2
  3. compile.sh +23 -10
  4. gen_models--openai--whisper.py +99 -0
README.md CHANGED
@@ -1,8 +1,9 @@
1
  ---
2
  library_name: llima
3
  license: apache-2.0
4
- base_model: openai/whisper-medium
5
  pipeline_tag: automatic-speech-recognition
 
 
6
  language:
7
  - en
8
  - zh
@@ -103,97 +104,238 @@ language:
103
  - ba
104
  - jw
105
  - su
106
- inference: false
107
  tags:
108
- - llima
109
  - whisper
110
  - audio
111
  - automatic-speech-recognition
 
 
112
  - sima
113
  - modalix
114
- - compiled
115
  - a16w8
 
116
  - gelu-tanh
 
117
  ---
118
- # whisper-medium-a16w8 for SiMa.ai Modalix
119
 
120
- Compiled [OpenAI Whisper medium](https://huggingface.co/openai/whisper-medium)
121
- for the LLiMa speech-to-text runtime. The package contains unpacked `devkit/`
122
- assets and `elf_files/`, ready to copy to a Modalix device.
123
 
124
- | Setting | Value |
125
- | --- | --- |
126
- | Source checkpoint | `openai/whisper-medium` |
127
- | Source revision | `abdf7c39ab9d0397620ccaea8974cc764cd0953e` |
128
- | Weights / activations | INT8 / BF16 (A16W8) |
129
- | Token and position embeddings / KV caches | BF16 |
130
- | Activation | GELU-tanh for encoder convolutions and encoder/decoder MLPs |
131
- | Audio input | 16 kHz; 30-second encoder input, with shorter audio padded |
132
- | Mel features | 80 |
133
- | Encoder / decoder layers | 24 / 24 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
134
  | Maximum decoder positions | 448 |
135
- | Decoder future-token mask | Enabled |
136
- | Log-probability outputs | Enabled (`--enable_log_probe`) |
137
- | Compiler | Model SDK 3.0.0; LLiMa 0.6.0 at `0747a38b99e88ab45786c3141a0f96afb980b3c7` |
 
138
  | Compiled units | 98 ELF files |
139
 
140
- ## Download and use
 
 
 
 
141
 
142
- On Modalix, install a compatible LLiMa runtime and download the unpacked model:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
143
 
144
  ```bash
145
  hf download florianvoss/whisper-medium-a16w8 \
146
- --local-dir /media/nvme/llima/models/whisper-medium-a16w8
147
  ```
148
 
149
- The existing LLiMa CLI loads Whisper as the speech-to-text component alongside
150
- an LLM or VLM:
151
 
152
  ```bash
153
- llima run /path/to/compiled-language-model \
154
- --stt_model_path /media/nvme/llima/models/whisper-medium-a16w8
 
 
 
155
  ```
156
 
157
- In the interactive CLI:
158
 
159
- ```text
160
- set language en
161
- set audio /path/to/audio.wav
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
162
  ```
163
 
164
- This transcribes the audio and submits the transcription as the language-model
165
- query. Applications can use LLiMa's `WhisperModel` interface directly for ASR.
166
- See the [LLiMa runtime guide](https://github.com/sima-neat/llima/blob/0747a38b99e88ab45786c3141a0f96afb980b3c7/docs/runtime.md).
167
 
168
- ## Reproduce
169
 
170
- Activate a matching SiMa Model Compiler environment and use the LLiMa checkout
171
- at [commit `0747a38`](https://github.com/sima-neat/llima/commit/0747a38b99e88ab45786c3141a0f96afb980b3c7).
172
- The original source weights are quantized by AFE during compilation:
 
173
 
174
  ```bash
175
  hf download openai/whisper-medium \
176
- --revision abdf7c39ab9d0397620ccaea8974cc764cd0953e \
177
- --local-dir models--openai--whisper-medium
178
- LLIMA_REPO=/path/to/llima bash compile.sh \
179
- models--openai--whisper-medium compiled-whisper-medium-a16w8
180
- llima-deploy compiled-whisper-medium-a16w8 whisper-medium-a16w8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
181
  ```
182
 
183
- `compile.sh` compiles every component with 16 workers by default; set `JOBS`
184
- to change the worker count. See `compilation.json` for the recorded settings.
185
- The source revision comes from the local Hugging Face download metadata.
 
 
 
 
 
 
186
 
187
- ## Validation
 
 
 
 
188
 
189
- All 98 components compiled successfully and were unpacked with
190
- `llima-deploy`. The ELF inventory, file headers and runtime configuration were
191
- checked. This new compiled build has not yet been tested on Modalix.
192
 
193
- Verify the downloaded files with `sha256sum -c SHA256SUMS`.
 
 
 
 
 
 
 
 
 
 
194
 
195
- ## License
196
 
197
- The upstream checkpoint declares `apache-2.0`. See the
198
- [source model card](https://huggingface.co/openai/whisper-medium/blob/abdf7c39ab9d0397620ccaea8974cc764cd0953e/README.md).
199
- A compatible SiMa.ai runtime is required to execute these compiled artifacts.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: llima
3
  license: apache-2.0
 
4
  pipeline_tag: automatic-speech-recognition
5
+ base_model: openai/whisper-medium
6
+ base_model_relation: quantized
7
  language:
8
  - en
9
  - zh
 
104
  - ba
105
  - jw
106
  - su
 
107
  tags:
 
108
  - whisper
109
  - audio
110
  - automatic-speech-recognition
111
+ - speech-translation
112
+ - embedded
113
  - sima
114
  - modalix
 
115
  - a16w8
116
+ - compiled
117
  - gelu-tanh
118
+ inference: false
119
  ---
 
120
 
121
+ # Whisper Medium A16W8: Optimized for SiMa.ai Modalix
 
 
122
 
123
+ **Build: Model SDK 3.0.0; LLiMa 0.6.0; GELU-tanh.**
124
+
125
+ Use a compatible Neat/LLiMa runtime supporting layered Whisper encoders and
126
+ log-probability decoder outputs.
127
+
128
+
129
+ ## Overview
130
+
131
+ This repository contains a precompiled and quantized version of
132
+ [OpenAI Whisper Medium](https://huggingface.co/openai/whisper-medium), optimized
133
+ for execution on the **SiMa.ai Modalix** platform.
134
+
135
+ | Property | Value |
136
+ |---|---|
137
+ | Model architecture | Whisper encoder-decoder Transformer |
138
+ | Parameters | 769 million |
139
+ | Quantization | A16W8: BF16 activations, INT8 weights |
140
+ | Activation | GELU-tanh in encoder convolutions and encoder/decoder MLPs |
141
+ | Embeddings / KV cache | BF16 |
142
+ | Decoder future-token mask / log-probability outputs | Enabled / enabled |
143
+ | Target | SiMa.ai Modalix |
144
+ | Tasks | Multilingual transcription and speech translation to English |
145
+ | Languages | 99 |
146
+ | Maximum audio window | 30 seconds |
147
  | Maximum decoder positions | 448 |
148
+ | Build LLiMa version | 0.6.0 |
149
+ | Source model | [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) |
150
+ | Source revision | `abdf7c39ab9d0397620ccaea8974cc764cd0953e` |
151
+ | LLiMa revision | `0747a38b99e88ab45786c3141a0f96afb980b3c7` |
152
  | Compiled units | 98 ELF files |
153
 
154
+ > This repository is not a standard Transformers checkpoint. The `elf_files/`
155
+ > and `devkit/` directories contain compiled artifacts consumed by the SiMa.ai
156
+ > Neat runtime on Modalix.
157
+
158
+ ## Prerequisites
159
 
160
+ To use this model, you need:
161
+
162
+ 1. A **SiMa.ai Modalix device**.
163
+ 2. The **SiMa.ai Neat Runtime** installed or updated on Modalix.
164
+ 3. The **Hugging Face CLI**, optionally, for downloading the model on a host
165
+ before copying it to Modalix.
166
+
167
+ Follow the [SiMa.ai Neat getting started guide](https://developer.sima.ai/software/getting-started/)
168
+ to install or update the Neat Runtime.
169
+
170
+ ## Installation and Deployment
171
+
172
+ ### Download the Model
173
+
174
+ On Modalix, download this GELU-tanh build into the model directory:
175
 
176
  ```bash
177
  hf download florianvoss/whisper-medium-a16w8 \
178
+ --local-dir /media/nvme/llima/models/whisper-medium-a16w8
179
  ```
180
 
181
+ Alternatively, download it on a host and copy it to Modalix:
 
182
 
183
  ```bash
184
+ hf download florianvoss/whisper-medium-a16w8 \
185
+ --local-dir whisper-medium-a16w8
186
+
187
+ scp -r whisper-medium-a16w8 \
188
+ sima@<modalix-ip>:/media/nvme/llima/models/
189
  ```
190
 
191
+ Replace `<modalix-ip>` with the IP address or hostname of the Modalix device.
192
 
193
+ ## Usage
194
+
195
+ This model is intended for applications built with the SiMa.ai Neat GenAI
196
+ APIs. Use `ASRModel` for direct, in-process transcription or `GenAIServer` when
197
+ clients access the model over HTTP.
198
+
199
+ ### Direct Application Integration
200
+
201
+ The Neat `ASRModel` API provides the lowest-overhead path for speech
202
+ transcription inside an application:
203
+
204
+ ```cpp
205
+ #include "neat/genai.h"
206
+
207
+ #include <iostream>
208
+
209
+ int main() {
210
+ simaai::neat::genai::ASRModel model(
211
+ "/media/nvme/llima/models/whisper-medium-a16w8");
212
+
213
+ simaai::neat::genai::GenerationRequest request;
214
+ request.audio_file = "speech.wav";
215
+ request.language = "en";
216
+
217
+ auto result = model.run(request);
218
+ std::cout << result.text << "\n";
219
+ }
220
+ ```
221
+
222
+ For the complete C++ and Python workflows, including audio tensors and
223
+ streaming, see
224
+ [GenAI Model](https://developer.sima.ai/software/develop-apps/development-workflow/genai-model).
225
+
226
+ ### Serve with the Neat GenAI Server
227
+
228
+ Use `GenAIServer` when a browser, service, or remote client needs to access the
229
+ model through an HTTP API. The packaged Neat tutorial can serve this ASR model
230
+ by itself:
231
+
232
+ ```bash
233
+ ./lib/sima-neat/tutorials/tutorial_021_serve_genai_models \
234
+ --asr /media/nvme/llima/models/whisper-medium-a16w8
235
+ ```
236
+
237
+ Verify that the model was registered:
238
+
239
+ ```bash
240
+ curl http://<modalix-ip>:9998/v1/models
241
+ ```
242
+
243
+ Send an audio transcription request with the tutorial client:
244
+
245
+ ```bash
246
+ python3 share/sima-neat/tutorials/021_serve_genai_models/request_audio_transcription.py \
247
+ --server-ip <modalix-ip> \
248
+ --model asr \
249
+ speech.wav
250
  ```
251
 
252
+ See
253
+ [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
254
+ for the complete C++, Python, and HTTP workflow.
255
 
256
+ ## Recompile
257
 
258
+ Activate a Model Compiler environment with LLiMa at
259
+ [commit `0747a38`](https://github.com/sima-neat/llima/commit/0747a38b99e88ab45786c3141a0f96afb980b3c7)
260
+ or a compatible revision containing the Whisper GELU-tanh change. The included
261
+ `compile.sh` uses the bundled Python helper; no LLiMa repository path is needed.
262
 
263
  ```bash
264
  hf download openai/whisper-medium \
265
+ --revision abdf7c39ab9d0397620ccaea8974cc764cd0953e \
266
+ --local-dir models--openai--whisper-medium
267
+
268
+ bash compile.sh
269
+ llima-deploy whisper-medium-a16w8 whisper-medium-a16w8-deployed
270
+ ```
271
+
272
+ The wrapper compiles all components with A16W8 precision, future-token masking
273
+ and log-probability outputs enabled. Set `NUM_PROCESSES` to change the default
274
+ of 16 workers. Additional arguments such as `--model_path`, `--output`, `--part`
275
+ and `--resume` are forwarded to the Python helper. Resume is opt-in.
276
+
277
+ See `compilation.json` for the recorded build settings.
278
+
279
+ ## Artifact Layout
280
+
281
+ ```text
282
+ devkit/ Runtime configuration, tokenizer, and embeddings
283
+ elf_files/ Compiled Modalix accelerator programs
284
+ compile.sh Compilation entry point
285
+ gen_models--openai--whisper.py Whisper compilation utility
286
+ compilation.json Source revision and compiler settings
287
+ SHA256SUMS File checksums
288
  ```
289
 
290
+ The compilation scripts are included for provenance and reproducibility. They
291
+ are not required when using the precompiled artifacts. Verify a downloaded
292
+ package with `sha256sum -c SHA256SUMS`.
293
+
294
+ ## Evaluation
295
+
296
+ All 98 components compiled successfully, were unpacked with
297
+ `llima-deploy`, and had their ELF inventory, headers and runtime assets checked.
298
+ This new compiled build has not yet been tested on Modalix.
299
 
300
+ No accuracy or performance results specific to this A16W8 compiled artifact
301
+ are currently reported. Results on the
302
+ [source model card](https://huggingface.co/openai/whisper-medium) describe the
303
+ unquantized upstream checkpoint and should not be treated as measurements of
304
+ this build.
305
 
306
+ ## Limitations
 
 
307
 
308
+ - These artifacts run on SiMa.ai Modalix and cannot be loaded with
309
+ `transformers.AutoModel`.
310
+ - The runtime processes an audio window of up to 30 seconds. Segment longer
311
+ recordings before inference.
312
+ - This build produces text without word- or segment-level timestamps.
313
+ - Speaker diarization is not provided.
314
+ - Quantization can cause minor differences from the full-precision source
315
+ model.
316
+ - Whisper can hallucinate text during silence or noisy audio and performs
317
+ unevenly across languages, accents, and domains. Evaluate the model on
318
+ representative data before production deployment.
319
 
320
+ ## Resources
321
 
322
+ - [SiMa.ai Neat](https://developer.sima.ai/software/getting-started/)
323
+ - [GenAI Model API](https://developer.sima.ai/software/develop-apps/development-workflow/genai-model)
324
+ - [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
325
+ - [GenAI with LLiMa](https://developer.sima.ai/software/genai-llima/)
326
+ - [OpenAI Whisper Medium](https://huggingface.co/openai/whisper-medium)
327
+ - [Whisper paper](https://arxiv.org/abs/2212.04356)
328
+
329
+ ## Citation
330
+
331
+ ```bibtex
332
+ @misc{radford2022whisper,
333
+ doi = {10.48550/ARXIV.2212.04356},
334
+ url = {https://arxiv.org/abs/2212.04356},
335
+ author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and
336
+ Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
337
+ title = {Robust Speech Recognition via Large-Scale Weak Supervision},
338
+ publisher = {arXiv},
339
+ year = {2022}
340
+ }
341
+ ```
SHA256SUMS CHANGED
@@ -1,6 +1,6 @@
1
- e1214768e45580ef6235bf451302a79ec8ac0fb1e45015aeec1c8309fd798204 README.md
2
  570abe80d2cb27c2866360089bed608bf8422cc149fd65bebfe130dfe6fa0929 compilation.json
3
- e859f85229bb5804cf30d3f4cee8e21cf10c736e327d0df265d78e966a6e64c3 compile.sh
4
  9f42a3247ee4aa47341f71f9e75c3743718784a40d4abd1370a4684aca13807f devkit/models--openai--whisper-medium_position_embeddings.npy
5
  54e0aa8f295458c56cedc78d839c7f44558047f7c18a7fc3649a1ebe81b536c7 devkit/models--openai--whisper-medium_token_embeddings.npy
6
  a896cec8787fb048db3ff2d396f9c7bea0d79262ba768459f580da8050c7caef devkit/preprocessor_config.json
@@ -104,3 +104,4 @@ b8fee7a74637cb78a181d2566e2cde125e3472bee80b9b6c6b8c96a401f3a511 elf_files/mode
104
  f774c66cf809491818f391bdfbda26ab42e7207385ae1f0e537b35a5e788826a elf_files/models--openai--whisper-medium_encoder_layer7_stage1_mla.elf
105
  e72ee3c853cfd25ad47bbd3ee024c45dc85961104b77ccac603ec981c88182cc elf_files/models--openai--whisper-medium_encoder_layer8_stage1_mla.elf
106
  11e8400f8a39c018a433046e71f2a7cf2acd861c1189ddc1659b1e2387ba8f15 elf_files/models--openai--whisper-medium_encoder_layer9_stage1_mla.elf
 
 
1
+ 264df33dbb2cd777d97a626d7cf9c6f5df8b5860d5eba86542bb4f7f98679a02 README.md
2
  570abe80d2cb27c2866360089bed608bf8422cc149fd65bebfe130dfe6fa0929 compilation.json
3
+ 7d5206a61f3b9e837b21c360f5373ada9f6fee6bf3e90d5ed7c74035d423e9d2 compile.sh
4
  9f42a3247ee4aa47341f71f9e75c3743718784a40d4abd1370a4684aca13807f devkit/models--openai--whisper-medium_position_embeddings.npy
5
  54e0aa8f295458c56cedc78d839c7f44558047f7c18a7fc3649a1ebe81b536c7 devkit/models--openai--whisper-medium_token_embeddings.npy
6
  a896cec8787fb048db3ff2d396f9c7bea0d79262ba768459f580da8050c7caef devkit/preprocessor_config.json
 
104
  f774c66cf809491818f391bdfbda26ab42e7207385ae1f0e537b35a5e788826a elf_files/models--openai--whisper-medium_encoder_layer7_stage1_mla.elf
105
  e72ee3c853cfd25ad47bbd3ee024c45dc85961104b77ccac603ec981c88182cc elf_files/models--openai--whisper-medium_encoder_layer8_stage1_mla.elf
106
  11e8400f8a39c018a433046e71f2a7cf2acd861c1189ddc1659b1e2387ba8f15 elf_files/models--openai--whisper-medium_encoder_layer9_stage1_mla.elf
107
+ 66b01700240a05b3dba00b3ed88942c48b080391841f317def8c959079818e03 gen_models--openai--whisper.py
compile.sh CHANGED
@@ -1,14 +1,27 @@
1
  #!/usr/bin/env bash
2
  set -euo pipefail
3
 
4
- whisper_source="${1:?Usage: LLIMA_REPO=/path/to/llima bash compile.sh MODEL_PATH [OUTPUT_DIR]}"
5
- whisper_output="${2:-CompiledModels/whisper-medium-a16w8}"
6
- whisper_llima="${LLIMA_REPO:?Set LLIMA_REPO to a matching LLiMa checkout}"
7
- whisper_script="$whisper_llima/scripts/gen_models--openai--whisper.py"
8
-
9
- python "$whisper_script" \
10
- --model_path "$whisper_source" \
11
- --output "$whisper_output" \
12
- --num_processes "${JOBS:-16}" \
 
 
 
 
 
 
 
 
 
 
 
 
13
  --part all \
14
- --enable_log_probe
 
 
1
  #!/usr/bin/env bash
2
  set -euo pipefail
3
 
4
+ # Download the source checkpoint at the revision recorded in compilation.json:
5
+ # hf download openai/whisper-medium \
6
+ # --revision abdf7c39ab9d0397620ccaea8974cc764cd0953e \
7
+ # --local-dir models--openai--whisper-medium
8
+
9
+ SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
10
+ MODEL_PATH="models--openai--whisper-medium"
11
+ OUTPUT_PATH="whisper-medium-a16w8"
12
+ NUM_PROCESSES="${NUM_PROCESSES:-16}"
13
+
14
+ whisper_tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/llima-whisper.XXXXXX")"
15
+ export TMPDIR="$whisper_tmp_dir"
16
+ cleanup() {
17
+ rm -rf -- "$whisper_tmp_dir"
18
+ }
19
+ trap cleanup EXIT
20
+
21
+ python3 "${SCRIPT_DIR}/gen_models--openai--whisper.py" \
22
+ --model_path "${MODEL_PATH}" \
23
+ -o "${OUTPUT_PATH}" \
24
+ --num_processes "${NUM_PROCESSES}" \
25
  --part all \
26
+ --enable_log_probe \
27
+ "$@"
gen_models--openai--whisper.py ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Repository-local compiler utility for the existing Whisper ASR path.
2
+
3
+ This script invokes ``WhisperModel`` directly for contributor development. It
4
+ is not a public ``llima-compile`` entry point or a framework for new ASR
5
+ architectures.
6
+ """
7
+
8
+ import logging
9
+
10
+ from pathlib import Path
11
+
12
+ from afe.apis.error_handling_variables import enable_verbose_error_messages
13
+ from sima_lmm.model import FileGenMode, FileGenPrecision, WhisperModel
14
+
15
+
16
+ DEFAULT_MODEL_PATH = Path("models--openai--whisper-medium")
17
+
18
+
19
+ def gen_files(
20
+ model_path: Path,
21
+ output_path: Path,
22
+ num_processes: int,
23
+ resume: bool,
24
+ part: str | None,
25
+ part_idx: int | None,
26
+ enable_log_probe: bool,
27
+ ):
28
+ enable_verbose_error_messages()
29
+
30
+ model = WhisperModel.from_hf_cache(
31
+ hf_cache_path=model_path,
32
+ model_name=model_path.name,
33
+ sima_path=output_path / "sima_files",
34
+ use_future_token_mask=True,
35
+ enable_log_probe=enable_log_probe,
36
+ )
37
+
38
+ precision = {
39
+ "all": {"precision": FileGenPrecision.A_BF16_W_INT8},
40
+ }
41
+ model.gen_files(
42
+ FileGenMode.ALL,
43
+ precision=precision,
44
+ log_level=logging.INFO,
45
+ num_processes=num_processes,
46
+ part=part,
47
+ part_idx=part_idx,
48
+ resume=resume,
49
+ )
50
+
51
+
52
+ if __name__ == "__main__":
53
+ import argparse
54
+
55
+ parser = argparse.ArgumentParser(
56
+ description="Generate Whisper compiler artifacts for LLiMa contributor development"
57
+ )
58
+ parser.add_argument("--model_path", type=Path, default=DEFAULT_MODEL_PATH)
59
+ parser.add_argument(
60
+ "-o",
61
+ "--output",
62
+ "--output_path",
63
+ dest="output_path",
64
+ type=Path,
65
+ default=None,
66
+ help="Directory to write model output into (default: ./<model_name>)",
67
+ )
68
+ parser.add_argument("--num_processes", type=int, default=1)
69
+ parser.add_argument("--resume", action="store_true", default=False)
70
+ parser.add_argument(
71
+ "--part",
72
+ choices=[
73
+ "all", "encoder", "language_detect", "init", "single_pre", "single_post",
74
+ "single_cache",
75
+ ],
76
+ default="all",
77
+ help="Whisper model part to compile.",
78
+ )
79
+ parser.add_argument(
80
+ "--part_idx",
81
+ type=int,
82
+ default=None,
83
+ help=(
84
+ "Optional layer index for encoder/init/pre/post, or token index for cache."
85
+ ),
86
+ )
87
+ parser.add_argument("--enable_log_probe", action="store_true", default=False)
88
+ args = parser.parse_args()
89
+ args.output_path = args.output_path or Path(".") / args.model_path.name
90
+ print("Arguments:", args, flush=True)
91
+ gen_files(
92
+ args.model_path,
93
+ args.output_path,
94
+ args.num_processes,
95
+ args.resume,
96
+ args.part,
97
+ args.part_idx,
98
+ args.enable_log_probe,
99
+ )