starbix/Apertus-8B-Instruct-2509-FP8_dynamic

Apertus-8B-Instruct-2509-FP8-Dynamic

This is an FP8 dynamically quantized version of swiss-ai/Apertus-8B-Instruct-2509 using llm-compressor.

Quantization Details

• Quantization Scheme: FP8_dynamic
• Method: Dynamic quantization of weights and activations to FP8 format
• Targets: All Linear layers
• Ignored Layers: lm_head (kept in higher precision for better output quality)
• Tool: llm-compressor (Neural Magic)