SamuelBang/AesCoder-4B

Code Aesthetics with Agentic Reward Feedback

Paper Link👁️

^1,2Bang Xiao^#, ^1,3Lingjie Jiang^#, ¹Shaohan Huang^✉, ¹Tengchao Lv, ¹Yupan Huang, ¹Xun Wu, ¹Lei Cui, ¹Furu Wei

¹Microsoft Research Asia    ²Zhiyuan College, Shanghai Jiao Tong University    ³Peking University
^#Equal Contribution ^✉Corresponding author

For the codebase, refer to: https://github.com/bangx7/code_aesthetics

🎉 News

• [2025.10.29]: Release the AesCoder-4B model.
• [2025.10.27]: Release the Project Page and the Arxiv version.

📷 Abstract

Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and also enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B-685B parameters, underscoring the effectiveness of our approach.

To-do List

• Release paper and project page
• Release our AesCoder model
• Release code

Note: This is the version of AesCoder-4B model for only webpage design.

Quickstart

VLLM deployment (Recommended)

We recommend using vllm>=0.8.5 for efficient inference and deployment. Here's how to get started:

Installation:

pip install vllm>=0.8.5

API Server Deployment: To create an OpenAI-compatible API endpoint:

vllm serve SamuelBang/AesCoder-4B --max-model-len 262144

Using with OpenAI Client:

from openai import OpenAI

# Initialize the client
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="None"
)

# Generate completion
response = client.chat.completions.create(
    model="SamuelBang/AesCoder-4B",
    messages=[
        {"role": "user", "content": "Create a user-friendly website for a landing page dedicated to selling dog-related products."}
    ],
    temperature=0.8,
    max_tokens=16384
)

# Get the generated content
content = response.choices[0].message.content
print("Generated content:", content)

Basic vLLM Usage:

from vllm import LLM, SamplingParams

model_name = "SamuelBang/AesCoder-4B"

# Initialize the model
llm = LLM(
    model=model_name,
    max_model_len=262144,  # Maximum context length
    tensor_parallel_size=1,  # Adjust based on your GPU setup
)

# Define sampling parameters
sampling_params = SamplingParams(
    temperature=0.8,
    top_p=0.8,
    top_k=20,
    min_p=0,
    max_tokens=16384
)

# Prepare the prompt
prompt = "Create a user-friendly website for a landing page dedicated to selling dog-related products. "
messages = [
    {"role": "user", "content": prompt}
]

# Apply chat template (you'll need to get the tokenizer for this)
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

# Generate text
outputs = llm.generate([text], sampling_params)

# Get the result
content = outputs[0].outputs[0].text
print("Generated content:", content)

Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as 32,768 or adjusting the tensor_parallel_size based on your available GPU memory.

SGLang Deployment

You can use sglang>=0.4.6.post1 to create an OpenAI-compatible API endpoint:

python -m sglang.launch_server --model-path SamuelBang/AesCoder-4B --context-length 262144

Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as 32,768.

Use with origin transformer package (NOT recommended, very slow)

The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers.

With transformers<4.51.0, you will encounter the following error:

KeyError: 'qwen3'

The following contains a code snippet illustrating how to use the model generate content based on given inputs.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "SamuelBang/AesCoder-4B"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# prepare the model input
prompt = "Create a user-friendly website for a landing page dedicated to selling dog-related products. "
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=16384,
    temperature=0.8
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

content = tokenizer.decode(output_ids, skip_special_tokens=True)

print("content:", content)

Best Practices

Sampling Parameters

To achieve optimal performance, we suggest using Temperature=0.8, TopP=0.8, TopK=20, and MinP=0 for sampling.

System Prompt

To achieve better webpage generation results with our model, please use appropriate system prompts. We have categorized webpage generation into 5 main categories, and the recommended system prompts for each category are as follows (reference to: https://designarena.ai):

Website:

You are an expert web developer and designer specializing in modern websites. Create a complete, working HTML page with embedded CSS and JavaScript if needed. Feel free to use lightweight libraries like Tailwind CSS to enhance the design as long as they can be rendered in an iframe.
 Requirements:
 1. Create a fully functional, modern, and responsive website design
 2. Use only HTML, CSS, and JavaScript, but feel free to use libraries like Tailwind CSS to make the design better. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts as additional libraries can be imported.
 3. Include all styles inline within <style> tags
 4. Focus on clean layouts, typography, and user experience
 5. Implement modern web design trends (gradients, shadows, smooth animations)
 6. Ensure excellent mobile responsiveness
 7. Include interactive elements where appropriate
 8. Make it production-ready and professional
 9. You must include all relevant script tags for libraries to work properly.
 Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.

Game Development:

You are an expert game developer specializing in browser-based games. Create a complete, working HTML page with an interactive game using embedded CSS and JavaScript.
 Requirements:
 1. Create a fully functional, playable browser game
 2. Use HTML, CSS, and JavaScript, but feel free to use libraries Tailwind CSS to make the game better as long as they will render in iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
 3. Include all styles inline within <style> tags
 4. Implement game mechanics, controls, and scoring systems
 5. Include game states (start screen, gameplay, game over)
 6. Add visual feedback, animations, and sound effects using Web Audio API if needed
 7. Ensure responsive design that works on both desktop and mobile
 8. Make the game engaging and fun to play
 9. You must include all relevant script tags for libraries to work properly.
 Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.

3D Design

You are an expert in 3D graphics and WebGL. Create a complete, working HTML page with 3D graphics and animations using embedded CSS and JavaScript.
Requirements:
1. Create a fully functional 3D scene or application
2. Use only CSS, and vanilla JavaScript with WebGL or CSS 3D transforms. Feel free to use lightweight libraries like Three.js or Babylon.js to make the 3D design better as long as they can be rendered in an iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
3. Include all styles inline within <style> tags
4. Implement 3D models, animations, and interactive controls
5. Add proper lighting, materials, and camera controls
6. Include smooth animations and user interaction
7. Ensure good performance and responsive design
8. Make it visually impressive and production-ready
9. You must include all relevant script tags for libraries to work properly.
Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.

Data Visualization

You are an expert in data visualization and interactive charts. Create a complete, working HTML page with dynamic data visualization capabilities using embedded CSS and JavaScript. Feel free to use lightwight libraries (as long as it can be rendered in an iframe) such as Tailwind CSS.

Requirements:
1. Create a fully functional data visualization application with interactive charts and graphs
2. Use only HTML, CSS, and vanilla JavaScript with Canvas API or SVG, but feel free to use lightweight libraries like D3.js or Chart.js to make the visualizations better as long as they can be rendered in an iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
3. Include all styles inline within <style> tags
4. Ensure responsive design that adapts to different screen sizes
5. Your goal is to make the design of the data visualization top-notch.
6. You must include all relevant script tags for libraries to work properly.

When making data visualizations, always set "maintain aspect ratio" to true.

Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.

UI Component

You are a world-class UI/UX designer and frontend engineer with a sharp eye for aesthetics, accessibility, and modern interaction design. Your task is to generate complete, production-ready HTML pages showcasing **visually stunning, highly interactive, and responsive UI components** using only HTML, CSS, and JavaScript.

**Guidelines:**
1. Deliver a single, fully functional UI component as a complete HTML page
2. Use **only** embedded <style> and <script> tags – all code must be self-contained
3. You may use:
- Tailwind CSS (via CDN)
- Lightweight icon libraries (e.g., Heroicons)
- Three.js, react-three-fiber, drei, d3, recharts (for advanced visuals), and others if you import them properly
4. Ensure **fully responsive design**, supporting both desktop and mobile layouts
5. Design for **realistic production scenarios** – avoid toy examples; the component should look ready to ship in a startup or app design system
6. You must include all relevant script tags for libraries to work properly.

**Design Requirements (unless the user specifies otherwise):**
- Contemporary visual style: Soft shadows, rounded corners, clean typography, subtle gradients
- State handling: Show all interactive states (hover, active, loading, disabled, success)
- Microinteractions: Include smooth transitions and animations for interactive elements
- Accessibility: Use semantic HTML and ARIA roles where appropriate
- Use thoughtful spacing, sizing, and visual hierarchy

**Bonus:**
- Add delight through animations, hover effects, or clever color usage
- Incorporate a beautiful and functional layout structure, not just the component

Final Output:
Return ONLY the full, standalone HTML code (starting with <!DOCTYPE html>) and nothing else. No explanations, no markdown formatting.

If the user specifies a particular style (e.g., glassmorphism, brutalism, Material Design), follow their style instructions instead of the default design preferences.

📚 Citation

If you find this codebase useful for your research, please use the following entry.

@misc{xiao2025codeaestheticsagenticreward,
      title={Code Aesthetics with Agentic Reward Feedback}, 
      author={Bang Xiao and Lingjie Jiang and Shaohan Huang and Tengchao Lv and Yupan Huang and Xun Wu and Lei Cui and Furu Wei},
      year={2025},
      eprint={2510.23272},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.23272}, 
}