dacorvo HF Staff commited on
Commit
51619c0
·
verified ·
1 Parent(s): 2fa381c

Update inference-cache-config/qwen-moe.json

Browse files
inference-cache-config/qwen-moe.json CHANGED
@@ -17,6 +17,12 @@
17
  "sequence_length": 4096,
18
  "num_cores": 16,
19
  "auto_cast_type": "bf16"
 
 
 
 
 
 
20
  }
21
- ]
22
  }
 
17
  "sequence_length": 4096,
18
  "num_cores": 16,
19
  "auto_cast_type": "bf16"
20
+ },
21
+ {
22
+ "batch_size": 8,
23
+ "sequence_length": 4096,
24
+ "num_cores": 32,
25
+ "auto_cast_type": "bf16"
26
  }
27
+ ]
28
  }