Tessera-27B-B-CJK-LoRA

This repo holds the PEFT LoRA adapter behind ZelligeAI/Tessera-27B-B-CJK. The main card has all results, limits and usage notes. In short, the model thinks in compressed CJK and uses 0.50-0.64x the base model's thinking tokens per agent turn on held-out coding-agent tasks. The typical turn is 0.496x, 90% CI 0.385-0.624, and the token-weighted ratio is 0.644x, 90% CI 0.208-0.866.

The adapter applies to Qwen/Qwen3.8-27B at revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, with the stock tokenizer and chat template. It has rank 32 and alpha 32 and covers attention, the MLP and lm_head, 1.09 GB in total.

Tinker trains separate q, k and v LoRAs in the linear-attention layers, while the Hugging Face model has one fused in_proj_qkv module. This adapter fuses them with a stacked A and a block-diagonal B, at rank 96 on that module. The maximum relative fusion error is 6.5e-7. A direct conversion without this fusion drops those deltas in 48 of 64 layers without any error message.

We evaluated this LoRA on Tinker. We merged it into the bf16 weights of the main repo and checked the merge by logprob parity, as the main card describes. We never loaded the adapter with PEFT. Its keys follow the multimodal module layout, base_model.model.model.language_model.layers.N..., so load the base with AutoModelForImageTextToText. Check the load result for missing or unexpected keys. A mismatch means the adapter did not apply, and PEFT may not warn. The merged model in the main repo avoids this step.

# Untested. We never loaded this adapter with PEFT.
import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3.8-27B", revision="1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
    dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "ZelligeAI/Tessera-27B-B-CJK-LoRA")

Use reasoning_effort="xhigh" in agent loops. The license is Apache-2.0, the same as the base model's.