"In this notebook, we introduce a new feature for AutoBuild, `build_from_library`, which help users build an automatic task-solving process powered by a multi-agent system from a pre-defined agent library. \n",
"Specifically, in `build_from_library`, we prompt an LLM to explore useful agents from a pre-defined agent library, generating configurations for those agents for a group chat to solve the user's task."
],
"metadata": {
"collapsed": false
},
"id": "6264276d39875995"
},
{
"cell_type": "markdown",
"id": "ec78dda8e3826d8a",
"metadata": {
"collapsed": false
},
"source": [
"## Requirement\n",
"\n",
"AutoBuild require `pyautogen[autobuild]`, which can be installed by the following command:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e8e9ae50658be975",
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%pip install pyautogen[autobuild]"
]
},
{
"cell_type": "markdown",
"source": [
"## Preparation and useful tools\n",
"We need to specify a `config_path`, `default_llm_config` that include backbone LLM configurations."
"Here, we show an example of generating an agent library from a pre-defined list of agents' names by prompting a `gpt-4`. You can also prepare a handcraft library yourself.\n",
"\n",
"A Library contains each agent's name and profile. The profile is a brief introduction about agent's characteristics. As we will put all agents' names and profiles into gpt-4 and let it choose the best agents for us, each agent's profile should be simple and capable. We will further complete the selected agents' system message based on the agents' names and the short profile as in the previous `build`.\n",
"\n",
"First, we define a prompt template and a list of agents' name:"
],
"metadata": {
"collapsed": false
},
"id": "5fb3db8885dd6ee6"
},
{
"cell_type": "code",
"execution_count": 3,
"id": "68315f6ec912c58a",
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-12-23T07:39:03.317527600Z",
"start_time": "2023-12-23T07:39:03.276859600Z"
}
},
"outputs": [],
"source": [
"AGENT_SYS_MSG_PROMPT = \"\"\"Considering the following position:\n",
"\n",
"POSITION: {position}\n",
"\n",
"What requirements should this position be satisfied?\n",
"\n",
"Hint:\n",
"# Your answer should be in one sentence.\n",
"# Your answer should be natural, starting from \"As a ...\".\n",
"# People with the above position need to complete a task given by a leader or colleague.\n",
"# People will work in a group chat, solving tasks with other people with different jobs.\n",
"# The modified requirement should not contain the code interpreter skill.\n",
"The generated profile will have the following format:"
],
"metadata": {
"collapsed": false
},
"id": "9e26c6db4befacc5"
},
{
"cell_type": "code",
"execution_count": 5,
"outputs": [
{
"data": {
"text/plain": "[{'name': 'Environmental_Scientist',\n 'profile': 'As an Environmental Scientist, the candidate should possess a strong background in environmental science, demonstrate the ability to effectively collaborate with a diverse team in a group chat to solve tasks, and have proficiency in Python for data analysis, without the need for code interpretation skills.'},\n {'name': 'Astronomer',\n 'profile': 'As an astronomer required to work collaboratively in a group chat setting, the candidate must possess strong proficiency in Python for data analysis and research purposes, alongside the ability to efficiently complete tasks assigned by leadership or colleagues without the need for code interpretation skills.'},\n {'name': 'Software_Developer',\n 'profile': 'As a Software Developer for this position, you must be able to work collaboratively in a group chat environment to complete tasks assigned by a leader or colleague, primarily using Python programming expertise, excluding the need for code interpretation skills.'},\n {'name': 'Data_Analyst',\n 'profile': 'As a Data Analyst for this position, you must be adept at analyzing data using Python, completing tasks assigned by leaders or colleagues, and collaboratively solving problems in a group chat setting with professionals of various roles.'},\n {'name': 'Journalist',\n 'profile': 'As a journalist in this position, you must possess strong collaboration and communication abilities to efficiently complete tasks assigned by leaders or colleagues within a group chat environment, without the need for code interpretation skills, although a basic understanding of Python is preferred.'},\n {'name': 'Teacher',\n 'profile': \"As a teacher, you need to possess a bachelor's degree in education or a related field, have a valid teaching certificate, be able to complete assignments provided by supervisors or colleagues, work collaboratively in group chats with professionals from various fields, and have a basic understanding of Python for educational purposes, excluding the need to interpret code.\"},\n {'name': 'Lawyer',\n 'profile': 'As a lawyer in this position, you must possess a Juris Doctor degree, be licensed to practice law, have strong analytical and communication skills, be able to complete tasks assigned by leaders or colleagues, and collaborate effectively in group chat environments with professionals across various disciplines, while having a basic understanding of Python for task-related purposes, excluding code interpretation.'},\n {'name': 'Programmer',\n 'profile': 'As a Programmer for this position, you should be proficient in Python, able to effectively collaborate and solve problems within a group chat environment, and complete tasks assigned by leaders or colleagues without requiring expertise in code interpretation.'},\n {'name': 'Accountant',\n 'profile': 'As an accountant in this position, one should possess a strong proficiency in accounting principles, the ability to effectively collaborate within team environments, such as group chats, to solve tasks, and have a basic understanding of Python for limited coding tasks, all while being able to follow directives from leaders and colleagues.'},\n {'name': 'Mathematician',\n 'profile': 'As a mathematician in this position, you should possess an advanced degree in mathematics, excel at collaborating and communicating within a group chat to solve complex tasks alongside professionals from various disciplines, and have proficiency in Python for any required computational work.'},\n {'name': 'Physicist',\n 'profile': \"As a physicist for this position, one must hold a strong foundation in physics principles, possess a minimum of a master's degree in physics or related fields, demonstrate proficiency in Python for task-specific computations, be willing to collaborate and solve problems within a multidisciplinary group chat, and not be required to interpret code from languages other than Python.\"},\n {'name': 'Biologist',\n 'profile': 'As a biologist for this position, one must hold a degree in
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sys_msg_list"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-12-23T07:40:01.712399300Z",
"start_time": "2023-12-23T07:40:01.707400200Z"
}
},
"id": "8ede1d7088eb183d"
},
{
"cell_type": "markdown",
"source": [
"We can save the generated agents' information into a json file."
"Here, we introduce how to build agents from the generated library. As in the previous `build`, we also need to specify a `building_task` that lets the build manager know which agents should be selected from the library according to the task. \n",
"\n",
"We also need to specify a `library_path_or_json`, which can be a path of library or a JSON string with agents' configs. Here, we use the previously saved path as the library path."
"building_task = \"Find a paper on arxiv by programming, and analyze its application in some domain. For example, find a recent paper about gpt-4 on arxiv and find its potential applications in software.\""
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-12-23T07:40:01.752918500Z",
"start_time": "2023-12-23T07:40:01.735461Z"
}
},
"id": "8963a8709c8e92e2"
},
{
"cell_type": "markdown",
"source": [
"Then, we can call the `build_from_library` from the AgentBuilder to generate a list of agents from the library and let them complete the user's `execution_task` in a group chat."
],
"metadata": {
"collapsed": false
},
"id": "72656a8d0c1a9b12"
},
{
"cell_type": "code",
"execution_count": 10,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Looking for suitable agents in ./agent_library_example.json...\n",
"['Software_Developer', 'Programmer', 'Artificial_Intelligence_Engineer'] are selected.\n",
"Preparing configuration for Software_Developer...\n",
"Preparing configuration for Programmer...\n",
"Preparing configuration for Artificial_Intelligence_Engineer...\n",
"Creating agent Software_Developer with backbone gpt-4-1106-preview...\n",
"Creating agent Programmer with backbone gpt-4-1106-preview...\n",
"Creating agent Artificial_Intelligence_Engineer with backbone gpt-4-1106-preview...\n",
"To find a recent paper about explainable AI on arXiv and explore its potential applications in the medical field, I will perform the following steps:\n",
"\n",
"1. Write a Python script to interact with the arXiv API to search for recent papers related to explainable AI.\n",
"2. Analyze the abstract and content of the retrieved paper to understand its implications and potential applications in the medical domain.\n",
"3. Discuss the findings with the team.\n",
"\n",
"Let's start with step 1. Below is a Python script that uses the `arxiv` library to search for papers related to explainable AI. If you don't have the `arxiv` library installed, you can install it using `pip install arxiv`.\n",
"\n",
"```python\n",
"import arxiv\n",
"\n",
"# Define the search query and parameters\n",
"search_query = 'cat:cs.AI AND ti:explainable'\n",
"max_results = 5\n",
"sort_by = arxiv.SortCriterion.SubmittedDate\n",
"\n",
"# Search for papers on arXiv\n",
"search = arxiv.Search(\n",
" query=search_query,\n",
" max_results=max_results,\n",
" sort_by=sort_by,\n",
" sort_order=arxiv.SortOrder.Descending\n",
")\n",
"\n",
"# Fetch the results\n",
"papers = list(search.results())\n",
"\n",
"# Print out the title and summary of the most recent paper\n",
"if papers:\n",
" recent_paper = papers[0]\n",
" print(f\"Title: {recent_paper.title}\\n\")\n",
" print(f\"Authors: {', '.join(author.name for author in recent_paper.authors)}\\n\")\n",
" print(\"No papers found on the topic of explainable AI.\")\n",
"```\n",
"\n",
"Please note that this script is meant to be run in a Python environment where you have the necessary permissions and capabilities to install and use external libraries. If you are ready to proceed, you can run this script in your Python environment to retrieve the most recent papers on explainable AI from arXiv.\n",
"\n",
"Once we have the paper, we can move on to step 2 and analyze its content for potential medical applications. Since I cannot execute Python code directly, you would need to run the script on your local machine or development environment. After running the script, you can share the paper's title and abstract here, and we can discuss its potential applications in the medical field.\n",
"The paper you've found, titled \"Transparency and Privacy: The Role of Explainable AI and Federated Learning in Financial Fraud Detection,\" discusses the use of Explainable AI (XAI) and Federated Learning (FL) in the context of financial fraud detection. While the paper's primary focus is on the financial industry, the concepts of XAI and FL can be applied to the medical field as well.\n",
"\n",
"Potential applications of XAI and FL in the medical field include:\n",
"\n",
"1. **Patient Data Privacy**: Similar to financial institutions, healthcare providers must adhere to strict privacy regulations like HIPAA in the United States. FL can enable different healthcare institutions to collaboratively train machine learning models on patient data without sharing the data itself, thus preserving patient privacy.\n",
"\n",
"2. **Disease Diagnosis**: XAI can help in developing transparent AI systems that assist doctors in diagnosing diseases by providing interpretable predictions. This transparency is crucial for gaining the trust of medical professionals and patients when AI is used to support decision-making in diagnoses.\n",
"\n",
"3. **Personalized Medicine**: By using FL, medical researchers can develop more generalized and robust models for personalized medicine, as they can learn from a wide range of patient data across different institutions without compromising privacy.\n",
"\n",
"4. **Fraud Detection in Healthcare**: The approach discussed in the paper can be adapted to detect fraudulent activities within healthcare, such as insurance fraud or prescription fraud, by training models across various healthcare providers.\n",
"\n",
"5. **Clinical Trial Research**: FL can facilitate the analysis of clinical trial data from multiple sources, enhancing the development of new drugs and treatments while maintaining the confidentiality of trial participants.\n",
"\n",
"6. **Predictive Analytics**: XAI can improve predictive analytics in healthcare by providing insights into the risk factors and predictors of patient outcomes, making it easier for clinicians to understand and trust the predictions made by AI models.\n",
"\n",
"7. **Medical Imaging**: In medical imaging, XAI can help radiologists and other specialists understand the reasoning behind AI-generated insights, which can be critical for early detection and treatment planning.\n",
"\n",
"The integration of XAI ensures that the AI's decision-making process in these applications is transparent, which is essential for clinical acceptance. Meanwhile, FL addresses the challenge of leveraging large-scale, diverse datasets while respecting privacy concerns, which is particularly relevant in the medical field due to the sensitive nature of health data.\n",
"\n",
"To discuss these findings with your team, you can highlight the parallels between the financial and medical domains in terms of data privacy and the need for transparency in AI systems. The paper's approach can be a starting point for developing similar systems in healthcare that benefit from the privacy-preserving and explainable nature of the technologies discussed.\n",
"## Build agents from library (by profile-task similarity)\n",
"We also support using embedding similarity to select agents. You can use a [Sentence-Transformers model](https://www.sbert.net/docs/pretrained_models.html) as an embedding extractor, and AgentBuilder will select agents with profiles that are the most similar to the building task from the library by comparing their embedding similarity. This will reduce the use of LLMs but may have less accuracy."
],
"metadata": {
"collapsed": false
},
"id": "c7a10e6fa00a5a0d"
},
{
"cell_type": "code",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Looking for suitable agents in ./agent_library_example.json...\n",
"['Programmer', 'Mathematician', 'Software_Developer', 'Physicist', 'Data_Analyst'] are selected.\n",
"Preparing configuration for Programmer...\n",
"Preparing configuration for Mathematician...\n",
"Preparing configuration for Software_Developer...\n",
"Preparing configuration for Physicist...\n",
"Preparing configuration for Data_Analyst...\n",
"Creating agent Programmer with backbone gpt-4-1106-preview...\n",
"Creating agent Mathematician with backbone gpt-4-1106-preview...\n",
"Creating agent Software_Developer with backbone gpt-4-1106-preview...\n",
"Creating agent Physicist with backbone gpt-4-1106-preview...\n",
"Creating agent Data_Analyst with backbone gpt-4-1106-preview...\n",
"To find a recent paper about GPT-4 on arXiv and analyze its potential applications in software, I would typically write a Python script that uses the arXiv API to search for papers related to GPT-4. However, as an AI, I can't execute scripts or make live API calls. Instead, I can guide you through the process of writing such a script.\n",
"\n",
"Here's a Python script template that you can use to search for papers on arXiv:\n",
"# Note: To analyze the potential applications in software, you would need to\n",
"# read the abstracts and possibly the full papers to determine their relevance\n",
"# and applications. This part cannot be fully automated and requires human expertise.\n",
"```\n",
"\n",
"To run this script, you'll need Python installed on your machine along with the `requests` and `feedparser` libraries, which you can install using pip:\n",
"\n",
"```bash\n",
"pip install requests feedparser\n",
"```\n",
"\n",
"After running the script, you'll get a list of recent papers related to GPT-4. You would then need to manually read through the abstracts and potentially the full papers to understand their potential applications in software.\n",
"\n",
"Remember, the actual applications will depend on the content of the papers, which might include but are not limited to natural language processing, content generation, automation, decision support systems, and more.\n",
"It appears that the code has been executed and the output lists several papers related to GPT-4 from arXiv. Let's analyze the potential applications in software based on the abstracts provided:\n",
"\n",
"1. **Dementia Diagnosis**: The first paper discusses the potential of GPT-4 in dementia diagnosis, comparing it with traditional AI tools. While GPT-4 shows promise, it currently does not outperform traditional methods. This suggests potential applications in healthcare software for diagnostic assistance.\n",
"\n",
"2. **Reasoning Ability**: The second paper criticizes GPT-4's reasoning ability, indicating that while it has improved over its predecessors, it still lacks true reasoning capabilities. This suggests that applications requiring deep reasoning, such as complex decision-making systems, may still be out of reach.\n",
"\n",
"3. **Logical Reasoning**: The third paper evaluates GPT-4's logical reasoning ability and finds that it performs well on known datasets but struggles with out-of-distribution and natural language inference datasets. This implies potential applications in enhancing logical reasoning in software systems, with a focus on improving robustness.\n",
"\n",
"4. **Behavior Over Time**: The fourth paper examines how GPT-3.5 and GPT-4's behavior changes over time, which is crucial for applications that require stability and predictability, such as automated monitoring systems.\n",
"\n",
"5. **Advancements in NLP**: The fifth paper reviews GPT-4's advancements and opportunities in NLP, suggesting applications in chatbots, personal assistants, language translation, text summarization, and question-answering systems.\n",
"\n",
"6. **Data Analysis**: The sixth paper explores whether GPT-4 can replace human data analysts. The results show that GPT-4 can perform comparably to humans, indicating potential applications in data analysis software tools.\n",
"\n",
"7. **Graph Neural Architecture Search**: The seventh paper introduces a method to use GPT-4 for graph neural architecture search, which could be applied in software for designing more accurate graph neural networks.\n",
"\n",
"8. **Math Word Problems**: The eighth paper discusses using GPT-4 for solving math word problems with self-verification, suggesting applications in educational software and tools that require mathematical problem-solving capabilities.\n",
"\n",
"9. **Tax Calculation**: The ninth paper questions GPT-4's ability to handle tax calculations, which is relevant for financial software that requires accurate and reliable tax computation.\n",
"\n",
"10. **Mathematical Understanding**: The tenth paper critically evaluates GPT-4's mathematical understanding, indicating that while it can reproduce known proofs, it struggles with novel problems. This suggests that while GPT-4 can assist in mathematical software, it may not yet be suitable for generating new mathematical insights.\n",
"\n",
"In summary, the potential applications of GPT-4 in software are vast, ranging from healthcare diagnostics to educational tools, data analysis, and NLP applications. However, limitations in reasoning, robustness, and novel problem-solving must be considered when integrating GPT-4 into software solutions.\n",
