alpindale commited on
Commit
d9a2b65
·
1 Parent(s): f1ca006

Increase `sliding_window` to 32k

Browse files

Some backends, such as vLLM, do not dynamically adjust the sliding window param based on the max context length provided. Modifying it here seems to fix that.

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -20,7 +20,7 @@
20
  "rms_norm_eps": 1e-05,
21
  "rope_theta": 1000000.0,
22
  "router_aux_loss_coef": 0.02,
23
- "sliding_window": 4096,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "bfloat16",
26
  "transformers_version": "4.36.0.dev0",
 
20
  "rms_norm_eps": 1e-05,
21
  "rope_theta": 1000000.0,
22
  "router_aux_loss_coef": 0.02,
23
+ "sliding_window": 32768,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "bfloat16",
26
  "transformers_version": "4.36.0.dev0",