integrated db.py with server.py

README.md

@@ -10,4 +10,59 @@ pinned: false
 license: mit
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Drift Detector
+Drift Detector is an MCP server, designed to detect drift in LLM performance over time. 
+This implementation is intended as a proof of concept and is not intended for production use.
+
+## How to run
+
+To run the Drift Detector, you need to have Python installed on your machine. Follow these steps:
+
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/saranshhalwai/drift-detector
+    cd drift-detector
+    ```
+2. Install the required dependencies:
+    ```bash
+    pip install -r requirements.txt
+    ```
+3. Start the server:
+    ```bash
+   gradio app.py
+    ```
+4. Open your web browser and navigate to `http://localhost:7860` to access the Drift Detector interface.
+
+## Interface
+
+The interface consists of the following components:
+- **Model Selection** - A panel allowing you to:
+  - Select models from a dropdown list
+  - Search for models by name or description
+  - Create new models with custom system prompts
+  - Enhance prompts with AI assistance
+
+- **Model Operations** - A tabbed interface with:
+  - **Chatbot** - Interact with the selected model through a conversational interface
+  - **Drift Analysis** - Analyze and visualize model drift over time, including:
+    - Calculate new drift scores for the selected model
+    - View historical drift data in JSON format
+    - Visualize drift trends through interactive charts
+
+The drift detection functionality allows you to track changes in model performance over time, which is essential for monitoring and maintaining model quality.
+
+## Under the Hood
+
+Our GitHub repo consists of two main components:
+
+- **Drift Detector Server**  
+    A low-level MCP server that detects drift in LLM performance of the connected client.
+- **Target Client**
+    A client implemented using the fast-agent library, which connects to the Drift Detector server and demonstrates it's functionality.
+
+The gradio interface in [app.py](app.py) is an example dashboard which allows users to interact with the Drift Detector server and visualize drift data.
+
+### Drift Detector Server
+
+The Drift Detector server is implemented using the MCP python SDK
+    

server.py

@@ -80,6 +80,26 @@ async def list_tools() -> List[types.Tool]:
     ]
 
 
+
+# === Sampling Wrapper ===
+async def sample(messages: list[types.SamplingMessage], max_tokens=600) -> CreateMessageResult:
+    return await app.request_context.session.create_message(
+        messages=messages,
+        max_tokens=max_tokens,
+        temperature=0.7
+    )
+
+
+# === Baseline File Paths ===
+def get_baseline_path(model_name):
+    return os.path.join(DATA_DIR, f"{model_name}_baseline.json")
+
+
+def get_response_path(model_name):
+    return os.path.join(DATA_DIR, f"{model_name}_latest.json")
+
+
+
 # === Core Logic ===
 async def run_initial_diagnostics(arguments: Dict[str, Any]) -> List[types.TextContent]:
     model = arguments["model"]
@@ -91,7 +111,19 @@ async def run_initial_diagnostics(arguments: Dict[str, Any]) -> List[types.TextC
     # 2. Ask the target LLM (client)
     answers = await sample(questions)
 
+
     # 3. Persist baseline
+
+    # 1. Ask the server's internal LLM to generate a questionnaire
+
+    questions = genratequestionnaire(model, arguments["model_capabilities"])  # Server-side trusted LLM
+    answers = []
+    for q in questions:
+        a = await sample([q])
+        answers.append(a)
+
+    # 3. Save Q/A pair
+
     with open(get_baseline_path(model), "w") as f:
         json.dump({
             "questions": [m.content.text for m in questions],
@@ -118,15 +150,30 @@ async def check_drift(arguments: Dict[str, Any]) -> List[types.TextContent]:
     ]
     old_answers = data["answers"]
 
+
     # 1. Get fresh answers
     new_msgs    = await sample(questions)
     new_answers = [m.content.text for m in new_msgs]
 
+    # 1. Ask the model again
+    new_answers_msgs = []
+    for q in questions:
+        a = await sample([q])
+        new_answers_msgs.append(a)
+    new_answers = [m.content.text for m in new_answers_msgs]
+
+
     # 2. Grade for drift
     grading     = await gradeanswers(old_answers, new_answers)
     drift_score = grading[0].content.text.strip()
 
+
     # 3. Save latest
+    grading_response = gradeanswers(old_answers, new_answers)
+    drift_score = grading_response[0].content.text.strip()
+
+    # 3. Save the response
+
     with open(get_response_path(model), "w") as f:
         json.dump({
             "new_answers": new_answers,