{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Welcome to Lab 3 for Week 1 Day 4\n", "\n", "Today we're going to build something with immediate value!\n", "\n", "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n", "\n", "Please replace it with yours!\n", "\n", "I've also made a file called `summary.txt`\n", "\n", "We're not going to use Tools just yet - we're going to add the tool tomorrow." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Looking up packages

\n", " In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n", " and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n", " ChatGPT or Claude, and you find all open-source packages on the repository https://pypi.org.\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n", "\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "from pypdf import PdfReader\n", "import gradio as gr" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "load_dotenv(override=True)\n", "openai = OpenAI()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "reader = PdfReader(\"me/Profile.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "   \n", "Contact\n", "+2347061185385 (Mobile)\n", "jolugbofemi@gmail.com\n", "www.linkedin.com/in/\n", "olufemijolugbo (LinkedIn)\n", "Top Skills\n", "SQL\n", "Microsoft SQL Server\n", "MySQL\n", "Certifications\n", "Shaping up with Angular.js\n", "Design Thinking for Beginners:\n", "Develop Innovative Ideas\n", "Build a Social Network with Flutter\n", "and Firebase\n", "QlikView 11 Designer\n", "Affinity Designer for UX Design\n", "Olufemi Adetola Jolugbo\n", "Head of Digital Delivery at Asset & Resource Management Holding\n", "Company (ARM HoldCo).\n", "Lagos State, Nigeria\n", "Summary\n", "This one is a Mathematician and a Software Engineer whose love for\n", "sophisticated tech must not be under-emphasized\n", "Experience\n", "Asset & Resource Management Holding Company (ARM HoldCo).\n", "7 years 5 months\n", "Application Support and Dev-ops Manager:\n", "June 2023 - Present (2 years 7 months)\n", "Providing solutions to business challenges using various tech solutions,\n", "construction of azure CICD pipelines between development, Code Repos and\n", "Windows servers.\n", "Head of Digital Delivery\n", "May 2021 - June 2023 (2 years 2 months)\n", "Managing a team consisting of mobile and web developers and Quality\n", "assurance analyst and coordinating with Scrum masters and Product\n", "designers and more, responsible for ensuring high team performance and\n", "product efficiency\n", "Flutter Mobile Application Developer\n", "October 2019 - May 2021 (1 year 8 months)\n", "lagos Nigeria\n", "Flutter Mobile App Developer: developed and supporting the live state of the\n", "ARMOne app, with over 100,000 downloads on the app stores\n", "Xamarin Developer\n", "August 2018 - October 2019 (1 year 3 months)\n", "developed and supported the defunct Payday Investor App, converting\n", "business requirements and Figma designs to beautiful frontend apps and\n", "integrating APIs to achieve functionality\n", "Avante Consulting Solutions Ltd\n", "  Page 1 of 3   \n", "Senior Software Engineer / Technical Consultant\n", "March 2016 - July 2018 (2 years 5 months)\n", "As a mobile application developer, I was able to work on new exciting\n", "project for the organisation leveraging on my learning ability, I picked up new\n", "technologies such as AngularJS, Ionic Framework and several other libraries\n", "quick to implement new innovative ideas, \n", "Also supporting and carrying out maintenance on banking software application\n", "with my technical skills in script writing , I was able to carry out effective\n", "disaster recoveries for a bank which would help them minimize data loss as\n", "well as reduce downtime in advent of any disaster, I achieved this by learning\n", "the structure of the application in relation to the database, to know what table\n", "needs to be included in the DR and what table needs to excluded from the DR,\n", "I was able to achieve this by implementing a mirroring DR infrastructure.\n", "Among other things, I was also able to carry out the implementation of a help\n", "desk system for national telecommunication companies and set up sessions\n", "via software such as TeamViewer and GoToMeeting to provide escalated\n", "support for clients on several issues relating to bug detection and quick\n", "customization \n", "Simplex Business Solutions Ltd\n", "Junior Software Engineer\n", "January 2014 - February 2016 (2 years 2 months)\n", "As a software developer, I was focused on developing financial software for\n", "financial organisations to help them achieve efficiency in their work, I was able\n", "to achieve this by consistently updating my knowledge on new technology and\n", "coding designs and getting used to new versions of programming language\n", "released to take advantage of their optimization and improvement, Language\n", "such as javascript are known to put out new versions frequently and it was my\n", "job to learn it and get the rest of the team up to date with this new knowledge\n", "via organising training, I also ensured that I put in place standard to follow\n", "for the team by creating coding techniques document which detailed several\n", "top notch standards such as the use of comments in codes, the use of code\n", "versioning tool such as Git(which I also carried out the training for the team),\n", "the appropriate looping techniques that make code work faster e.g switch\n", "statement vs for statement or while statement vs foreach statement, modifying\n", "old company methods of writing optimised SQL looping script (e.g changing\n", "cursors in SQL to pure While statement to improve query run time, this was\n", "able to reduce the query time of the code making it 80% faster), Amongst\n", "many other things.\n", "  Page 2 of 3   \n", "Each time there was a new project I was also required to attend requirement\n", "gathering sessions and contribute technical knowledge on what solutions\n", "can be created to optimize business process, I handled several web project\n", "with my team, with the use of Microsoft C# .Net framework making use of\n", "infrastructures such as Model View Controller(MVC),Web Forms, Entity\n", "Framework, Ado.net, XPO e.t.c, I was also required to implement systems\n", "for business processes such as calculation of daily interest on customer\n", "investment and creating embedded reports into the application, carrying out\n", "disaster recovering systems by putting up infrastructures such as database\n", "mirroring or database replication.\n", "Classic Splash Technology\n", "computer programmer\n", "November 2012 - September 2013 (11 months)\n", "Was responsible for the training of staffs on the latest in the world of\n", "programming techniques such as the MVC framework bootstraps and the\n", "likes was also involved in a non-profit skill acquisition training for youth corp\n", "members on the need to learn how to embrace technology to solve human\n", "need not just for the country but for the world at large the training which held at\n", "the camping ground saw over 1500 corps members in attendance\n", "Education\n", "University of Liverpool\n", "PG Software Engineering, Software Engineering · (2016 - 2018)\n", "Federal University Of Agriculture, Abeokuta\n", "Bachelor of Science - BS, Mathematics · (November 2008 - November 2012)\n", "  Page 3 of 3\n" ] } ], "source": [ "print(linkedin)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "name = \"Olufemi Jolugbo\"" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n", "particularly questions related to {name}'s career, background, skills and experience. \\\n", "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n", "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n", "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "If you don't know the answer, say so.\"\n", "\n", "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"You are acting as Olufemi Jolugbo. You are answering questions on Olufemi Jolugbo's website, particularly questions related to Olufemi Jolugbo's career, background, skills and experience. Your responsibility is to represent Olufemi Jolugbo for interactions on the website as faithfully as possible. You are given a summary of Olufemi Jolugbo's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nMy name is Olufemi Jolugbo. I'm an entrepreneur, software engineer and AI Enthusiast. I'm originally from Ogun, Nigeria, but I was born and breed in Lagos.\\nI love Cooking, particularly Grilling, Travelling accross the world and experiencing facinating cultures cuisanes and history are also some of the things i enjoy, \\ni am open for adventures of all kind.\\n\\n## LinkedIn Profile:\\n\\xa0 \\xa0\\nContact\\n+2347061185385 (Mobile)\\njolugbofemi@gmail.com\\nwww.linkedin.com/in/\\nolufemijolugbo (LinkedIn)\\nTop Skills\\nSQL\\nMicrosoft SQL Server\\nMySQL\\nCertifications\\nShaping up with Angular.js\\nDesign Thinking for Beginners:\\nDevelop Innovative Ideas\\nBuild a Social Network with Flutter\\nand Firebase\\nQlikView 11 Designer\\nAffinity Designer for UX Design\\nOlufemi Adetola Jolugbo\\nHead of Digital Delivery at Asset & Resource Management Holding\\nCompany (ARM HoldCo).\\nLagos State, Nigeria\\nSummary\\nThis one is a Mathematician and a Software Engineer whose love for\\nsophisticated tech must not be under-emphasized\\nExperience\\nAsset & Resource Management Holding Company (ARM HoldCo).\\n7 years 5 months\\nApplication Support and Dev-ops Manager:\\nJune 2023\\xa0-\\xa0Present\\xa0(2 years 7 months)\\nProviding solutions to business challenges using various tech solutions,\\nconstruction of azure CICD pipelines between development, Code Repos and\\nWindows servers.\\nHead of Digital Delivery\\nMay 2021\\xa0-\\xa0June 2023\\xa0(2 years 2 months)\\nManaging a team consisting of mobile and web developers and Quality\\nassurance analyst and coordinating with Scrum masters and Product\\ndesigners and more, responsible for ensuring high team performance and\\nproduct efficiency\\nFlutter Mobile Application Developer\\nOctober 2019\\xa0-\\xa0May 2021\\xa0(1 year 8 months)\\nlagos Nigeria\\nFlutter Mobile App Developer: developed and supporting the live state of the\\nARMOne app, with over 100,000 downloads on the app stores\\nXamarin Developer\\nAugust 2018\\xa0-\\xa0October 2019\\xa0(1 year 3 months)\\ndeveloped and supported the defunct Payday Investor App, converting\\nbusiness requirements and Figma designs to beautiful frontend apps and\\nintegrating APIs to achieve functionality\\nAvante Consulting Solutions Ltd\\n\\xa0 Page 1 of 3\\xa0 \\xa0\\nSenior Software Engineer / Technical Consultant\\nMarch 2016\\xa0-\\xa0July 2018\\xa0(2 years 5 months)\\nAs a mobile application developer, I was able to work on new exciting\\nproject for the organisation leveraging on my learning ability, I picked up new\\ntechnologies such as AngularJS, Ionic Framework and several other libraries\\nquick to implement new innovative ideas, \\nAlso supporting and carrying out maintenance on banking software application\\nwith my technical skills in script writing , I was able to carry out effective\\ndisaster recoveries for a bank which would help them minimize data loss as\\nwell as reduce downtime in advent of any disaster, I achieved this by learning\\nthe structure of the application in relation to the database, to know what table\\nneeds to be included in the DR and what table needs to excluded from the DR,\\nI was able to achieve this by implementing a mirroring DR infrastructure.\\nAmong other things, I was also able to carry out the implementation of a help\\ndesk system for national telecommunication companies and set up sessions\\nvia software such as TeamViewer and GoToMeeting to provide escalated\\nsupport for clients on several issues relating to bug detection and quick\\ncustomization \\nSimplex Business Solutions Ltd\\nJunior Software Engineer\\nJanuary 2014\\xa0-\\xa0February 2016\\xa0(2 years 2 months)\\nAs a software developer, I was focused on developing financial software for\\nfinancial organisations to help them achieve efficiency in their work, I was able\\nto achieve this by consistently updating my knowledge on new technology and\\ncoding designs and getting used to new versions of programming language\\nreleased to take advantage of their optimization and improvement, Language\\nsuch as javascript are known to put out new versions frequently and it was my\\njob to learn it and get the rest of the team up to date with this new knowledge\\nvia organising training, I also ensured that I put in place standard to follow\\nfor the team by creating coding techniques document which detailed several\\ntop notch standards such as the use of comments in codes, the use of code\\nversioning tool such as Git(which I also carried out the training for the team),\\nthe appropriate looping techniques that make code work faster e.g switch\\nstatement vs for statement or while statement vs foreach statement, modifying\\nold company methods of writing optimised SQL looping script (e.g changing\\ncursors in SQL to pure While statement to improve query run time, this was\\nable to reduce the query time of the code making it 80% faster), Amongst\\nmany other things.\\n\\xa0 Page 2 of 3\\xa0 \\xa0\\nEach time there was a new project I was also required to attend requirement\\ngathering sessions and contribute technical knowledge on what solutions\\ncan be created to optimize business process, I handled several web project\\nwith my team, with the use of Microsoft C# .Net framework making use of\\ninfrastructures such as Model View Controller(MVC),Web Forms, Entity\\nFramework, Ado.net, XPO e.t.c, I was also required to implement systems\\nfor business processes such as calculation of daily interest on customer\\ninvestment and creating embedded reports into the application, carrying out\\ndisaster recovering systems by putting up infrastructures such as database\\nmirroring or database replication.\\nClassic Splash Technology\\ncomputer programmer\\nNovember 2012\\xa0-\\xa0September 2013\\xa0(11 months)\\nWas responsible for the training of staffs on the latest in the world of\\nprogramming techniques such as the MVC framework bootstraps and the\\nlikes was also involved in a non-profit skill acquisition training for youth corp\\nmembers on the need to learn how to embrace technology to solve human\\nneed not just for the country but for the world at large the training which held at\\nthe camping ground saw over 1500 corps members in attendance\\nEducation\\nUniversity of Liverpool\\nPG Software Engineering,\\xa0Software Engineering\\xa0·\\xa0(2016\\xa0-\\xa02018)\\nFederal University Of Agriculture, Abeokuta\\nBachelor of Science - BS,\\xa0Mathematics\\xa0·\\xa0(November 2008\\xa0-\\xa0November 2012)\\n\\xa0 Page 3 of 3\\n\\nWith this context, please chat with the user, always staying in character as Olufemi Jolugbo.\"" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "system_prompt" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Special note for people not using OpenAI\n", "\n", "Some providers, like Groq, might give an error when you send your second message in the chat.\n", "\n", "This is because Gradio shoves some extra fields into the history object. OpenAI doesn't mind; but some other models complain.\n", "\n", "If this happens, the solution is to add this first line to the chat() function above. It cleans up the history variable:\n", "\n", "```python\n", "history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n", "```\n", "\n", "You may need to add this in other chat() callback functions in the future, too." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7861\n", "\n", "Could not create share link. Please check your internet connection or our status page: https://status.gradio.app.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch(share=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A lot is about to happen...\n", "\n", "1. Be able to ask an LLM to evaluate an answer\n", "2. Be able to rerun if the answer fails evaluation\n", "3. Put this together into 1 workflow\n", "\n", "All without any Agentic framework!" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# Create a Pydantic model for the Evaluation\n", "\n", "from pydantic import BaseModel\n", "\n", "class Evaluation(BaseModel):\n", " is_acceptable: bool\n", " feedback: str\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n", "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n", "The Agent is playing the role of {name} and is representing {name} on their website. \\\n", "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n", "\n", "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\"" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "def evaluator_user_prompt(reply, message, history):\n", " user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n", " user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n", " user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n", " user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n", " return user_prompt" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "import os\n", "load_dotenv(override=True)\n", "gemini = OpenAI(\n", " api_key=os.getenv(\"GEMINI_API_KEY\"), \n", " base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", ")\n", "# print(\"GEMINI_API_KEY:\", bool(os.getenv('GEMINI_API_KEY')))" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [], "source": [ "def evaluate(reply, message, history) -> Evaluation:\n", " import json, re\n", " #messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + \n", " messages = [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n", " # Ask the model to respond with a JSON object like: {'is_acceptable': true, 'feedback': '...'}\n", " response = openai.chat.completions.create(model=\"gpt-5-mini\", messages=messages)\n", " text = response.choices[0].message.content\n", " # Try to parse the model output as JSON. If parsing fails, try to extract a JSON substring.\n", " try:\n", " parsed = json.loads(text)\n", " except Exception:\n", " m = re.search(r'\\{.*\\}', text, re.DOTALL)\n", " if m:\n", " try:\n", " parsed = json.loads(m.group(0))\n", " except Exception:\n", " parsed = {\"is_acceptable\": False, \"feedback\": text}\n", " else:\n", " parsed = {\"is_acceptable\": False, \"feedback\": text}\n", " # Coerce into the Pydantic model; fall back to a conservative failure if needed.\n", " try:\n", " evaluation = Evaluation(**parsed)\n", " except Exception:\n", " evaluation = Evaluation(is_acceptable=False, feedback=text)\n", " return evaluation" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [], "source": [ "#messages = [{\"role\": \"system\", \"content\": system_prompt}] + \n", "messages = [{\"role\": \"user\", \"content\": \"Generate a detailed knowledgebase for all the likely questions a visitor to www.edurald.com website might ask about the products offering,\"}]\n", "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", "reply = response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Sure! Below is a detailed knowledgebase that addresses likely questions a visitor might have about the products offered on the website www.edurald.com. For the sake of this example, I\\'ll assume that Edurald offers educational products, such as online courses, e-books, educational software, and resources for students and educators. \\n\\n---\\n\\n## Edurald Knowledgebase\\n\\n### General Questions\\n\\n1. **What is Edurald?**\\n - Edurald is an online platform that offers a variety of educational products including courses, e-books, and software designed to support both students and educators in their learning and teaching processes.\\n\\n2. **Who is Edurald for?**\\n - Our products cater to students of all ages, educators looking for resources, and lifelong learners seeking new knowledge and skills.\\n\\n3. **How can I contact Edurald for support?**\\n - You can reach our support team via the \"Contact Us\" section on our website. Additionally, you can email us at support@edurald.com or use the live chat feature located in the bottom right corner of the website.\\n\\n### Product Offerings\\n\\n4. **What types of courses are available on Edurald?**\\n - We offer a diverse range of courses across various subjects, including mathematics, science, languages, arts, and technology. Each course is designed to enhance knowledge and skills through interactive learning.\\n\\n5. **Are the courses free or paid?**\\n - While some courses may be offered for free, most of our premium courses require a one-time payment or a subscription fee for access.\\n\\n6. **What formats do your courses come in?**\\n - Our courses are available in various formats, including video lectures, downloadable resources, quizzes, and interactive assignments.\\n\\n7. **Can I access the courses on mobile devices?**\\n - Yes! Our platform is mobile-friendly, allowing you to access courses on any device, including smartphones and tablets.\\n\\n8. **What kind of e-books does Edurald offer?**\\n - We provide a collection of e-books that cover a wide range of educational topics. e-books can include textbooks, study guides, and instructional material.\\n\\n### Purchases and Payments\\n\\n9. **What payment methods does Edurald accept?**\\n - We accept various payment methods including credit cards, PayPal, and other secure payment gateways.\\n\\n10. **Is there a refund policy?**\\n - Yes, we do have a refund policy. If you are not satisfied with your purchase, you can request a refund within 30 days of purchase, provided you haven’t accessed key course content.\\n\\n### Account Management\\n\\n11. **How do I create an account on Edurald?**\\n - To create an account, simply click on the \"Sign Up\" button on our homepage, fill in the required information, and verify your email address.\\n\\n12. **How can I reset my password?**\\n - You can reset your password by clicking on the \"Forgot Password?\" link on the login page. Follow the instructions in the email you receive to set a new password.\\n\\n13. **How do I update my account information?**\\n - Log into your account and navigate to the \"Profile\" section where you can update your personal information, payment method, and preferences.\\n\\n### Learning Experience\\n\\n14. **Will I receive a certificate upon completing a course?**\\n - Yes, upon successful completion of a course, you will receive a digital certificate that you can download and share.\\n\\n15. **Are there any prerequisites for your courses?**\\n - While most courses are designed to be accessible to anyone, certain more advanced courses may have prerequisites listed in their descriptions.\\n\\n16. **Can I interact with instructors or other students?**\\n - Yes, our platform facilitates interaction through discussion forums, live Q&A sessions, or community groups depending on the course structure.\\n\\n### Technical Support\\n\\n17. **What should I do if I encounter technical issues?**\\n - If you experience any technical difficulties, please reach out to our support team through the \"Support\" section on the website or via our live chat feature.\\n\\n18. **Is there a mobile app for Edurald?**\\n - Currently, we do not have a dedicated mobile app, but our website is fully optimized for mobile use.\\n\\n### Feedback and Community\\n\\n19. **How can I provide feedback on a course?**\\n - After completing a course, you will have the option to leave a review and provide feedback that helps us improve our offerings.\\n\\n20. **Does Edurald have a community forum?**\\n - Yes, we encourage learners to engage with each other through our community forum where you can ask questions, share insights, and connect with fellow learners.\\n\\n---\\n\\nThis knowledgebase can be expanded further based on specific products, upcoming features, unique selling propositions, or any other pertinent information relevant to Edurald\\'s audience. Ensure that any specific details you wish to address align with the actual offerings and policies of the Edurald website.'" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reply" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Evaluation(is_acceptable=False, feedback='Acceptable? Partially — it’s a good high-level starting draft but too generic and assumes product types/policies rather than reflecting Edurald’s real offerings. It will work as a skeleton, but needs more factual detail, completeness, and structure to be production-ready.\\n\\nWhat’s good\\n- Clear organization (General, Product, Purchases, Accounts, Technical, Community).\\n- Covers many common visitor concerns (courses, formats, access, payment, certificates, support).\\n- Easy to expand and adapt.\\n\\nMain problems / gaps\\n- Assumptions: The agent assumed Edurald offers online courses, e-books, software and specific policies (30‑day refunds) without checking the actual site or asking clarifying questions.\\n- Lack of specifics: No exact pricing model, subscription types, course catalog, or product examples.\\n- Policy details missing or vague: refund terms, license/usage rights for e-books/software, GDPR/data privacy, security, cancellation procedures, billing cycles, tax/VAT handling.\\n- Technical/compatibility details absent: recommended browsers, minimum device specs, SCORM/LTI support, file types, offline access, video streaming requirements.\\n- Learning experience specifics missing: certificate accreditation vs. platform certificate, validity, verification, grading/assessment methods, course durations, learning paths, prerequisites search/filtering.\\n- Support details too general: support hours, SLA/response times, escalation path, live chat availability, phone support, self-help resources.\\n- B2B / institutional features not covered: bulk licensing, enterprise accounts, LMS integrations, instructor onboarding, school/teacher pricing.\\n- Community & moderation: forum rules, moderation, user conduct, reporting, privacy within community.\\n- Accessibility & localization: languages offered, subtitles/CC, accessibility features for users with disabilities.\\n- Implementation/readiness for helpdesk: no structured Q/A output (e.g., JSON/YAML) for import into support systems or site search.\\n\\nRecommended improvements (actionable)\\n1. Verify facts: confirm actual products, pricing, refund policy, contact channels, mobile app status, and support SLAs from Edurald.\\n2. Expand each section with factual details and links to the relevant site pages (pricing, terms, privacy, contact).\\n3. Add detailed policy Q&A:\\n - Refund eligibility, partial refunds, chargebacks, trial cancelation flows.\\n - Licensing: how many devices/users per purchase, printing/e-book DRM.\\n - Tax handling (VAT/GST) and invoices/receipts.\\n4. Add technical troubleshooting Q&A:\\n - How to clear cache, supported browsers/OS, video playback errors, file download issues, and step-by-step fixes.\\n5. Add learning-specific Q&A:\\n - Certificate accreditation and verification, grading policies, retake exams, access expiration, progress tracking, downloadable transcripts.\\n6. Include B2B and educator-focused Q&A:\\n - Bulk pricing, institutional onboarding, teacher dashboards, grade export.\\n7. Add legal/privacy/security:\\n - Data retention, GDPR/CCPA compliance, password policies, two-factor authentication availability.\\n8. Create structured outputs:\\n - Provide the KB as JSON or CSV for import into helpdesk/chatbot systems, and include metadata (last updated date, category, tags).\\n9. Provide support/contact specifics:\\n - Phone numbers, office hours, expected response times, emergency contacts.\\n10. Add usability details:\\n - Search/filter tips, recommended courses for beginners, “Start here” learning paths, sample lesson or free preview availability.\\n11. Add analytics plan:\\n - Track top queries and revise KB based on usage; include feedback mechanism for KB pages.\\n\\nNext steps I can do for you\\n- If you give me the real Edurald site details (product list, pricing, refund policy, contact info, technical specs, support hours), I will produce a complete, factual knowledgebase (Q&A) and optionally export it in JSON/CSV for your helpdesk.\\n- Or I can convert the current draft into a structured KB file (JSON/CSV) with placeholder values you can replace.\\n\\nWhich next step would you like?')" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "evaluate(reply, \"Generate a detailed knowledgebase for all the likely questions a visitor to www.edurald.com website might ask about the products offering,\", messages[:1])" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "def rerun(reply, message, history, feedback):\n", " updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n", " updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n", " updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n", " messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " if \"patent\" in message:\n", " system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n", " it is mandatory that you respond only and entirely in pig latin\"\n", " else:\n", " system = system_prompt\n", " messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " reply =response.choices[0].message.content\n", "\n", " evaluation = evaluate(reply, message, history)\n", " \n", " if evaluation.is_acceptable:\n", " print(\"Passed evaluation - returning reply\")\n", " else:\n", " print(\"Failed evaluation - retrying\")\n", " print(evaluation.feedback)\n", " reply = rerun(reply, message, history, evaluation.feedback) \n", " return reply" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7862\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "Failed evaluation - retrying\n", "Not acceptable.\n", "\n", "Why: The reply’s tone and invitation to follow up are good, but the stated experience (\"over 9 years\") is inaccurate given the profile start date. The LinkedIn information shows work beginning in November 2012, so the accurate span is longer (about 13 years as of Jan 2026). The answer should give a precise, up-to-date number or simply state the start date and let the reader compute the length.\n", "\n", "How to improve:\n", "- Correct the years of experience (or state the start date).\n", "- Briefly highlight key areas (leadership, mobile/web, DevOps, Flutter, Azure CI/CD) or notable achievements (ARMOne >100k downloads).\n", "- Keep the professional, inviting tone.\n", "\n", "Suggested improved reply (example):\n", "\"I’ve been working in software engineering since November 2012 — about 13 years of experience as of 2026. Over that time I’ve progressed from developer roles to leadership positions (currently Head of Digital Delivery / Application Support & DevOps Manager at ARM HoldCo), working across mobile and web development (Flutter, Xamarin), backend/SQL, and CI/CD/DevOps on Azure. If you’d like, I can summarize specific projects or skills relevant to your needs.\"\n" ] } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "agents", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.12" } }, "nbformat": 4, "nbformat_minor": 2 }