xfu20/BEMGPT_1.0.0

1. Model Information

Model Name: BEMGPT (version) 1.0.0

New Features:

• Domain-Specific Fine-Tuning for BEM: BEMGPT is the first large language model fine-tuned specifically for BEM, addressing the lack of domain expertise in general-purpose LLMs.
• Embedded Domain Knowledge: Unlike Retrieval-Augmented Generation (RAG), BEMGPT integrates domain knowledge directly into the model parameters, making it a cost-effective alternative that reduces inference latency and dependency on external databases.
• Foundation for Intelligent, AI-Driven BEM Tools: BEMGPT establishes a new foundation for intelligent automation in BEM workflows, supporting more efficient, scalable, and accessible applications across the building energy sector.

Finetuned From: Meta Llama 3.2 11B Vision Instruct

Developers: Xiaoqin Fu and Liang Zhang @ TensorBuild Lab, Department of Civil and Architectural Engineering and Mechanics (CAEM), University of Arizona

Release Date: October 8, 2025

The corresponding Paper: BEMGPT: A Supervised Fine-Tuned Large Language Model for Building Energy Modeling Education and Training

Overview

BEMGPT is a large language model (LLM) fine-tuned specifically for Building Energy Modeling (BEM). It addresses the complexity of BEM data and the fragmented nature of domain knowledge by incorporating domain-specific expertise through fine-tuning.

By leveraging the generative capabilities of large language models, BEMGPT enhances knowledge representation, reasoning, and automation within the BEM workflow—making energy modeling more intelligent, efficient, and accessible.

How to Get Started with the Model

Use the following code to download and run the model.

The code for downloading and running BEMGPT 1.0.0

The file Load_Fine-tuned_Llama_11B.ipynb can be opened on Jupyter Notebook.

• The user can update the first code snippet according to his/her hardware.
• The user can update the last code snippet according to his/her question.

2. Who should use this model?

Design and construction professionals

• Architects and designers: They use BEMGPT to explore energy-efficient design strategies, generate and interpret building performance simulations, and integrate sustainability insights early in the design process.
• Mechanical, Electrical, and Plumbing (MEP) engineers: They apply BEMGPT to assist in system sizing, load calculations, and optimization of HVAC, lighting, and control systems using AI-enhanced analysis and documentation tools.

Building owners and operators

• Building owners: They benefit from BEMGPT’s ability to translate technical BEM outputs into actionable insights for energy investment decisions, retrofits, and operational improvements.
• Facility and energy managers: They use BEMGPT to automate performance reporting, identify energy-saving opportunities, and support predictive maintenance by integrating modeling knowledge with operational data.

Government agencies and policymakers

• Local, state, and federal agencies: They leverage BEMGPT for policy analysis, compliance evaluation, and large-scale energy benchmarking studies.
• Urban planners: They employ BEMGPT to assess the energy and carbon implications of urban design and zoning decisions.
• Code officials: They use BEMGPT to interpret and verify compliance with building energy codes and standards more efficiently.

Researchers and students

• Building Science Researchers: They utilize BEMGPT as a knowledge engine to extract, synthesize, and reason about BEM-related literature, datasets, and simulation results.
• Building Science students: They learn the principles and workflows of building energy modeling interactively through natural language queries and AI-guided explanations.

Manufacturers

• Equipment manufacturers: They use BEMGPT to query the energy questions of their products within building environments, generate compliance documentation, and optimize product design for specific use cases.

3. Fine-tuning Details

Fine-tuning Data

The fine-tuning data for BEMGPT 1.0.0

The file engineering-reference_quadruple.txt includes "context-instruction-question-answer" (CIQA) quadruplets for fine-tuning BEMGPT 1.0.0.

Fine-tuning Source

EnergyPlus documentation: EnergyPlus engineering reference

PDF document

Latex documents

The document is organized to give users the best possible look into the EnergyPlus calculations. First, the concepts of modeling in EnergyPlus are presented. These include descriptions of the zone heat balance process, air loop/plant loop processes as well as other important processes for the building simulation. The document remainder focuses on individual models.

Fine-tuning Methods

• LoRA (Low-Rank Adaptation)
• QLoRA (Quantized Low-Rank Adaptation)
• DoRA (Weight-Decomposed Low-Rank Adaptation)

......

4. Evaluation

Testing Data

The testing data for BEMGPT 1.0.0

The file engineering-reference_quadruple_test.txt includes "context-instruction-question-answer" (CIQA) quadruplets for testing BEMGPT 1.0.0.

Testing Metrics

• BLEU: Measures token-level overlap between generated and reference texts
• ROUGE: Recall-oriented similarity metrics, including ROUGE-1, ROUGE-2, and ROUGE-L
• BERTScore: Embedding-based semantic similarity
• Score evaluated by the OpenAI model gpt-4o: Human-like correctness score (0.00–100.00)

Evaluation Prompt

The example gpt-4o (2024-11-20 version) Evaluation Prompt:

prompt = f"""

You are an expert evaluator and semantic grader. Grade the correctness of the provided answer as concise as possible on a decimal ranging from 0.00 to 100.00, where:

100.00 = Completely correct, perfectly matches the reference text, or provides necessary information and details

90.00 = Mostly correct, very accurate and comprehensive, minor details may be missing

80.00 = Moderately correct, mostly accurate but may have minor errors or omissions

70.00 = Partially correct, contains some inaccuracies, or misses information

60.00 = Only little correct, contains few accuracies, or misses most information

0.00 = Only repeated question, no answer, or completely incorrect, useless, and meaningless answer

Decimals are acceptable and encouraged to reflect partial correctness.

......

Use the reference text to evaluate the answer. If the reference text is empty, incomprehensible, unsuitable, or useless, please do not use the reference text and directly evaluate the answer.

Ignore any interrogative sentences within the answer. Return only the decimal score (e.g., 97.23), with no additional text.

Instruction: {instruction}

Question: {question}

Answer: {provided_answer}

Reference Text: "{text}"
"""

5. Technical Specifications

Hardware

• CPUs: 2 × Intel Xeon Gold 6438Y+ (32 cores @ 2.0GHz, 60MB cache, 205W)
• Memory: 16 × 64GB DDR5 ECC RDIMM (4800MHz)
• Storage:
  • 2 × 960GB Micron 7450 PRO M.2 NVMe
  • 15.36TB Micron 7450 PRO U.3 NVMe
• GPUs:
  • 3 × NVIDIA H100 (80GB HBM2e, PCIe 5.0)
  • 1 × NVIDIA T400 (4GB GDDR6, PCIe 3.0)
• Network: Broadcom NetXtreme 10Gb Ethernet (2× SFP+)

Software

• OS: Ubuntu 22.04 LTS (64-bit)
• Environment: Anaconda 24.11.3
• Python: 3.12.9
• Libraries: torch, transformers, huggingface_hub, datasets, peft, trl, nltk, rouge, bert_score, numpy, etc.

6. Contact

• [email protected]
• [email protected]