Instructions to use 0xdfi/GLM-5.2-QuantTrio-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 0xdfi/GLM-5.2-QuantTrio-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="0xdfi/GLM-5.2-QuantTrio-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("0xdfi/GLM-5.2-QuantTrio-Abliterated")
model = AutoModelForCausalLM.from_pretrained("0xdfi/GLM-5.2-QuantTrio-Abliterated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 0xdfi/GLM-5.2-QuantTrio-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "0xdfi/GLM-5.2-QuantTrio-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xdfi/GLM-5.2-QuantTrio-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/0xdfi/GLM-5.2-QuantTrio-Abliterated

SGLang

How to use 0xdfi/GLM-5.2-QuantTrio-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "0xdfi/GLM-5.2-QuantTrio-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xdfi/GLM-5.2-QuantTrio-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "0xdfi/GLM-5.2-QuantTrio-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xdfi/GLM-5.2-QuantTrio-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 0xdfi/GLM-5.2-QuantTrio-Abliterated with Docker Model Runner:
```
docker model run hf.co/0xdfi/GLM-5.2-QuantTrio-Abliterated
```

Model Access and Use Terms

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

By checking the required box below and selecting “Agree and Access Model,” you (for yourself and your heirs, successors, and assigns) represent that you are at least 18 years old (or the age of majority in your jurisdiction, if greater), have legal capacity and authority to enter this agreement, and have read and understood these terms. You acknowledge that the model weights, code, configurations, documentation, and related files in this repository (the “Artifact”) are an unofficial research derivative in a model lineage with intentionally reduced refusal or safety behavior, are not sponsored, endorsed, approved by, or affiliated with any upstream author or base-model provider, have not been safety-evaluated by the repository owner, and may generate or enable harmful, illegal, offensive, infringing, private, sensitive, inaccurate, or otherwise unsafe content or outcomes. You accept the Artifact “AS IS” and “WITH ALL FAULTS.” To the fullest extent permitted by applicable law, the repository owner, author, uploader, maintainers, contributors, licensors, upstream authors, and each of their respective affiliates, officers, employees, and agents (collectively, the “Released Parties”) disclaim all warranties and conditions, express, implied, statutory, or arising from course of dealing or usage of trade, including merchantability, fitness for a particular purpose, title, non-infringement, accuracy, reliability, and safety. You knowingly and voluntarily assume all risks arising from access to or any use, misuse, modification, fine-tuning, distribution, deployment, output, or reliance on the Artifact and, to the fullest extent permitted by applicable law, irrevocably release, waive, discharge, and covenant not to sue the Released Parties for any claim, loss, liability, damage, penalty, cost, or expense arising from or related to the Artifact, including claims based on ordinary negligence. The Released Parties will not be liable for indirect, incidental, special, consequential, exemplary, or punitive damages, and their total aggregate liability arising from or related to the Artifact will not exceed US $100. You are solely responsible for reviewing and validating outputs, implementing appropriate safeguards and human oversight, preventing access by minors, complying with all applicable laws, regulations, export and sanctions controls, and license terms, and all consequences of your access and use. You agree to defend, indemnify, and hold harmless the Released Parties from every third-party claim and resulting loss, liability, damage, penalty, cost, or expense, including reasonable attorneys’ fees, arising from or related to your access, use, misuse, modification, distribution, deployment, outputs, or violation of law or these terms. The Released Parties are intended third-party beneficiaries entitled to enforce this agreement. These access and risk-allocation terms supplement and do not alter the copyright permissions granted by any applicable license; if a conflict exists, the applicable license controls copyright permissions and these terms control repository access and risk allocation. This agreement is governed by New York law, excluding conflict-of-laws rules, and you consent to exclusive jurisdiction and venue in the state and federal courts located in New York County, New York. If any provision is held unenforceable, it will be modified and limited to the minimum extent necessary or severed, and the remainder will remain in full force and effect. Terms version 2026-07-23.

GLM-5.2-QuantTrio-Abliterated

Complete weights for direct download; pinned reconstruction and adaptation guidance included

The complete checkpoint is uploaded in this repository: 124 model shards, four MTP shards, the Safetensors index, runtime configuration, tokenizer, and chat templates. After accepting the access terms, users can download the repository and load or serve it directly with a compatible runtime; rebuilding from a separate base model is not required. An agent connects to an inference server that has loaded the checkpoint—not to the raw model files themselves.

The repository also includes a tested deterministic builder for reconstructing this exact checkpoint from the pinned QuantTrio/GLM-5.2-Int4-Int8Mix base, plus technical documentation of the direction-recovery method, transplant formula, quantization codec, and validation criteria. That documentation records how this checkpoint was produced and provides a technical reference for adapting the process to another compatible compressed-tensors W8 group-128 variant with matching architecture and tensor layout. This is not an automated cross-variant workflow: the repository does not include a generic direction-extraction script or a drop-in builder, and the included builder and manifests are pinned to the named base revision.

This repository therefore provides both distribution paths:

Direct use: approximately 405.5 GB of complete, ready-to-load model shards, MTP shards, index, configuration, tokenizer, and chat templates.
Independent reconstruction: the direction vectors, relative magnitudes, and deterministic builder needed to reproduce the checkpoint from the pinned QuantTrio base, together with technical documentation of the full method.

What is this?

Huihui-ai produced an abliterated GGUF of GLM-5.2 by suppressing refusal-related weight directions in the attention output projections. This kit transplants those directions into QuantTrio's compressed-tensors W8 pack-quantized format, which loads natively with the transformers library.

In Huihui's language, this is an uncensored version of GLM-5.2 created with abliteration. This repository adapts that intervention to the pinned QuantTrio checkpoint rather than redistributing Huihui's GGUF weights.

Specifically:

Ablation directions and relative magnitudes were recovered by comparing the Huihui abliterated GGUF (pinned revision 2f0aff2760627f87faf16f8adc81caca7d7b10f6) against the Unsloth stock GGUF (pinned revision abc55e72527792c6e77069c99b4cb7de16fa9f23) via top-2 randomised SVD on the blk.{13..77}.attn_output.weight tensors.
Those directions were transplanted into the QuantTrio W8 base (pinned revision 1d3bcfe5ec549ecd000fd80b37f191183842e983) using fresh group-128 requantization and validated against 65 numerical gates.

The resulting model is not byte-identical to the Huihui GGUF. The edit magnitude uses a uniform scale factor of 1.75, which was the minimum tested value that passed all 65 frozen W8 numerical gates. Factor 1.0 reproduced the original Huihui magnitude but fell below the group-W8 reliable edit resolution; factor 1.6 still failed layer 23.

Credits

Zhipu AI / GLM-5.2: original model architecture and weights. https://huggingface.co/zai-org/GLM-5.2
QuantTrio: W4/W8 compressed-tensors quantization. https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix
Huihui-ai: original abliteration concept and abliterated GGUF artifact. https://huggingface.co/huihui-ai/Huihui-GLM-5.2-abliterated-GGUF
0xdfi: compressed-tensors direction transplant, requantization, and validation.

Usage warning

This model has reduced safety filtering. It can generate sensitive, controversial, inappropriate, harmful, unsafe, or illegal content. It is not suitable for deployment to minors, public-facing services, or high-security contexts without additional controls. Users are solely responsible for all outputs and for compliance with local laws and ethical standards. Research, testing, and controlled internal use with monitoring and manual review are recommended. There are no default safety guarantees. No warranty or liability is provided.

Numerical validation

The builder applies 65 independent frozen W8 gates (one per edited layer). Gates require:

Cosine similarity between requested and actual delta >= 0.99
Relative RMSE <= 0.15
Orthogonal leakage / requested norm <= 0.15

All 65 gates passed. The worst-performing layer was layer 46:

Cosine: 0.9907504192
Relative RMSE: 0.1369091172
Leakage / requested: 0.1367936930

Summary: 65 layers edited, 130 tensor payloads modified (weight_packed and weight_scale), 177,439 tensors byte-identical to the pristine QuantTrio base.

Behavioral validation

These results are specific to this build and the test corpus used. They are not a universal guarantee.

A 32-question refusal corpus was evaluated:

32 / 32 questions bypassed (no refusals)
Capability score: 11 / 12 (the same question was missed by unmodified stock GLM-5.2)

Direct download and serving

Accept the repository access terms on Hugging Face, then download the complete checkpoint:

hf download 0xdfi/GLM-5.2-QuantTrio-Abliterated \
    --local-dir ./GLM-5.2-QuantTrio-Abliterated

The download contains 124 model shards, four MTP shards, model.safetensors.index.json, runtime configuration, tokenizer files, and chat templates. Reserve at least 430 GB of free disk space for the download and additional space required by your runtime.

Point a GLM-5.2 inference runtime that supports QuantTrio's compressed-tensors INT4/INT8Mix checkpoint format at the downloaded directory. After the inference server is healthy, point your agent at that server's OpenAI-compatible endpoint and use the served model name. Raw weights are not themselves an agent endpoint; the inference server is the layer between the model files and the agent.

Runtime flags, tensor/context parallelism, memory limits, and maximum context depend on the target hardware. Do not copy the four-DGX-Spark production profile onto unrelated hardware without validating memory and backend compatibility.

Rebuild instructions

You need approximately 820 GiB free disk space on a single filesystem.

Step 1: Install dependencies

pip install -r requirements.txt

Step 2: Fetch the QuantTrio base

hf download QuantTrio/GLM-5.2-Int4-Int8Mix \
    --revision 1d3bcfe5ec549ecd000fd80b37f191183842e983 \
    --local-dir ./base

Step 3: Build

python build.py \
    --base-dir ./base \
    --output-dir ./output \
    --chunk-rows 512

The builder will:

Parse the base index and identify the 65 target layers (self_attn.o_proj, layers 13-77).
Hash the target shards to verify against the manifest.
Copy the entire base tree byte-by-byte to the output directory.
Patch the 130 target tensor payloads (weight_packed and weight_scale per layer) using fresh group-128 requantization.
Verify all edited and unchanged tensors.
Write output/BUILD_RECEIPT.json.
Print PASS on success.

Alternatively, let the builder download the base automatically:

python build.py --download-base --output-dir ./output --chunk-rows 512

Step 4: Verify

python -c "
import json
r = json.load(open('output/BUILD_RECEIPT.json'))
assert r['passed'], r
print('PASS: edited', r['edited_tensors'], 'tensors')
"

Step 5: Load with transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./output", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("./output", trust_remote_code=True)

The output directory is a complete model tree in compressed-tensors W8 format, identical to the QuantTrio base except for the 130 edited tensor payloads.

Repository contents

model-00001-of-00124.safetensors
...                             # 124 complete model shards
model-00124-of-00124.safetensors
mtp-00001-of-00004.safetensors
...                             # 4 complete MTP shards
mtp-00004-of-00004.safetensors
model.safetensors.index.json
config.json
configuration.json
generation_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
chat_template_original.jinja
build.py                        # Agent-facing wrapper (constructs manifest, runs builder)
repair_builder_v4.py            # Reviewed fail-closed CPU-only builder (do not modify)
requirements.txt
recipe/
    refusal_dirs.safetensors    # FP32 ablation direction vectors, layers 13-77
    huihui_mag.json             # Per-layer relative magnitudes
    manifest.json               # Provenance and numerical contract
docs/
    TECHNICAL_REPRODUCTION.md   # Full acquisition and validation workflow
tests/
    test_repair_builder_v4.py   # Frozen builder tests
    test_public_wrapper.py      # Wrapper tests
LICENSE
NOTICE.md
AGENTS.md

Technical details

The edit formula for each layer L is:

W*_L = W_qt,L - lambda_final_L * d_L (d_L^T W_qt,L)

where:

d_L is the unit-normalised ablation direction recovered from the Huihui/Unsloth diff
qt_mag1_L = ||d_L^T W_qt,L||_2 / ||W_qt,L||_F (projection of QuantTrio matrix onto direction)
lambda_match_L = m_L / qt_mag1_L (scale to match Huihui relative magnitude)
lambda_final_L = 1.75 * lambda_match_L
m_L is the per-layer relative magnitude from recipe/huihui_mag.json

After editing, the float32 result is requantized using the compressed-tensors group-128 W8 codec: fresh amax-based BF16 scales are stored before quantization, then weights are divided by the restored BF16 scale, clamped to [-128, 127], rounded to nearest even, converted to q+128 unsigned bytes, and packed little-endian into int32 lanes.

The builder is CPU-only, uses no network during the build phase, creates no hardlinks or symlinks, writes through fsynced temporary files with atomic rename, and operates under an exclusive fcntl lock with crash-recoverable checkpoint state. See docs/TECHNICAL_REPRODUCTION.md for the full acquisition and validation procedure.

Downloads last month: 109

Safetensors

Model size

785B params

Tensor type

BF16

I64

I32

F32

Model tree for 0xdfi/GLM-5.2-QuantTrio-Abliterated

Base model

zai-org/GLM-5.2

Quantized

QuantTrio/GLM-5.2-Int4-Int8Mix

Quantized

(3)

this model