NahOR102/gemma-3-penal-code-model

Model Card: Gemma-3 Indian Penal Code Legal Assistant

Model Details

Model Description

• Developed by: Independent researcher
• Model type: LoRA fine-tuned LLM for Indian legal assistance
• Language(s): English with focus on Indian legal terminology
• License: Apache 2.0 (same as base model)
• Finetuned from model: unsloth/gemma-3-4b-it-unsloth-bnb-4bit

Model Sources

• Base model: Google's Gemma 3
• Fine-tuning framework: Unsloth

Uses

Direct Use This model is designed to provide accurate information about the Indian Penal Code (IPC). It can be used by:

• Legal professionals seeking quick reference to IPC sections
• Law students studying Indian criminal law
• Individuals seeking to understand their legal rights and responsibilities
• Paralegals and legal assistants drafting documents

Downstream Use

• Integration into legal assistance chatbots
• Incorporation into legal research tools
• Development of educational materials on Indian criminal law

Out-of-Scope Use This model should NOT be used for:

• Providing definitive legal advice that replaces professional legal counsel
• Making judicial decisions or determinations
• Predicting case outcomes in specific legal scenarios
• Generating legal documents without professional review

Bias, Risks, and Limitations

• The model may have incomplete knowledge of the most recent amendments to the Indian Penal Code
• It may provide information that is technically correct but lacks important contextual nuance
• The model cannot account for jurisdiction-specific interpretations or case law
• Performance may vary when dealing with complex legal questions involving multiple sections
• The model should not be relied upon for actual legal proceedings

Recommendations

• Always verify information with official legal sources
• Consult with qualified legal professionals before making legal decisions
• Use the model as a supplementary tool, not a replacement for proper legal research
• Be aware that the model may occasionally generate inaccurate or outdated information

How to Get Started with the Model

from unsloth import FastModel
import torch
from unsloth.chat_templates import get_chat_template

# Load the base model first
base_model_name = "unsloth/gemma-3-4b-it-unsloth-bnb-4bit"
model, tokenizer = FastModel.from_pretrained(
    model_name = base_model_name,
    max_seq_length = 2048,
    load_in_4bit = True,
    load_in_8bit = False,
)

# Load the adapter separately
from peft import PeftModel
adapter_path = "username/gemma-3-indian-penal-code-model"  # Replace with actual path
model = PeftModel.from_pretrained(model, adapter_path)

# Set up the chat template
tokenizer = get_chat_template(
    tokenizer,
    chat_template = "gemma-3",
)

# Function to generate responses
def generate_response(question):
    # Include system prompt in user message for Gemma 3
    user_message = "You are an expert legal assistant providing accurate answers based on the Indian Penal Code (IPC). " + question
    
    messages = [
        {"role": "user", "content": user_message}
    ]
    
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    
    inputs = tokenizer([text], return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=512,
            temperature=0.7,
            top_p=0.95,
            top_k=64,
        )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example usage
query = "What is the punishment for theft under Section 379 of the IPC?"
response = generate_response(query)
print(response)

Training Details

Training Data

• The model was fine-tuned on a curated dataset of Indian Penal Code sections, explanations, and legal interpretations
• Training data was structured in a conversational format in the file "IPC_cleaned_fine_tuning.json"
• The dataset covers various sections of the IPC with specific focus on commonly referenced sections

Training Procedure

Preprocessing

• Legal texts were structured into conversational format for instruction fine-tuning
• Data was formatted with appropriate role tags (user/system/model)
• The Gemma 3 chat template was applied to format the conversations

Training Hyperparameters

• Training regime: LoRA (Low-Rank Adaptation) fine-tuning with Unsloth optimization
• Rank (r): 8
• Alpha: 8
• Dropout: 0
• Learning rate: 3e-4
• Per device batch size: 2
• Gradient accumulation steps: 4
• Training steps: 100
• Warmup steps: 2
• Optimizer: AdamW 8-bit
• Weight decay: 0.01
• LR scheduler: Linear
• Seed: 3407
• Model bits: 4-bit quantization (QLoRA)
• Max sequence length: 2048

Training Configuration

• Fine-tuned attention and MLP modules
• Used response-only training to focus on improving the model's outputs
• Utilized 4-bit quantization for memory efficiency

Speeds, Sizes, Times

• Training was completed in Google Colab environment
• Total training time was approximately 2-3 hours on a single GPU
• The adapter size is approximately 20MB
• Memory usage was optimized through 4-bit quantization

Evaluation

Testing Data, Factors & Metrics

Testing Data

• A set of 8 test questions covering various aspects of the Indian Penal Code:
  1. Punishment for theft
  2. Difference between culpable homicide and murder
  3. Concept of 'mens rea' in Indian criminal law
  4. Punishments for different degrees of hurt
  5. Criminal conspiracy under Section 120A
  6. Self-defense as a valid defense
  7. Legal definition of 'dowry death' under Section 304B
  8. Concept of 'abetment' under the IPC

Factors

• Accuracy of legal information provided
• Completeness of responses
• Adherence to the specific sections of the IPC
• Handling of complex legal concepts

Metrics

• Accuracy of section citations
• Correctness of punishment terms
• Completeness of legal explanations
• Consistency with official IPC text

Results

• The model demonstrates strong understanding of core IPC concepts
• It accurately cites relevant sections and their corresponding punishments
• Legal explanations are comprehensive and contextually appropriate
• The model can differentiate between related but distinct legal concepts

Environmental Impact

• Hardware Type: GPU (Google Colab)
• Hours used: Approximately 2-3 hours for fine-tuning
• Cloud Provider: Google Colab
• Carbon Emitted: Minimal due to short training time and efficient fine-tuning approach

Technical Specifications

Model Architecture and Objective

• Base model: Gemma 3 4B instruction-tuned model
• Fine-tuning method: LoRA adapters with Unsloth optimization
• Objective: Provide accurate information about the Indian Penal Code while maintaining the base model's general capabilities

Compute Infrastructure

• Fine-tuning performed on Google Colab
• Used 4-bit quantization for memory efficiency

Software

• Python 3.x
• PyTorch
• Transformers (v4.49.0-Gemma-3)
• PEFT
• Unsloth library for optimization
• TRL for fine-tuning

Model Card Contact

For questions or feedback about this model, please contact the author through the Hugging Face model repository.