wangtao
/
autogen
mirror of https://github.com/microsoft/autogen.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update assistant agent (#1121) 

* update colab link

* typo

* upload file instruction

* update system message and notebooks

* update notebooks

* notebook test

* aoai api version and exclusion

* gpt-3.5-turbo

* dict check

* change model for test

* endpoints, cache_path and func description update

* model list

* gitter -> discord
This commit is contained in:
Chi Wang 2023-07-14 15:52:45 -07:00 committed by GitHub
parent 8ac9a393b8
commit 297a1ea9e0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 746 additions and 1389 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.github/workflows/openai.yml vendored
View File

@ -56,9 +56,9 @@ jobs:
WOLFRAM_ALPHA_APPID: ${{ secrets.WOLFRAM_ALPHA_APPID }}
run: |
pip install nbconvert nbformat ipykernel
coverage run -a -m pytest test/autogen/oai/test_notebook.py
coverage run -a -m pytest test/autogen/test_notebook.py
coverage xml
cat "$(pwd)/test/autogen/oai/executed_openai_notebook_output.txt"
cat "$(pwd)/test/autogen/executed_openai_notebook_output.txt"
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:

10
flaml/autogen/agent/assistant_agent.py
View File

@ -5,15 +5,15 @@ from typing import Dict, Union
class AssistantAgent(Agent):
"""(Experimental) Assistant agent, able to suggest code blocks."""
"""(Experimental) Assistant agent, able to suggest code blocks with default system message."""
DEFAULT_SYSTEM_MESSAGE = """You are a helpful AI assistant.
In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. You must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code.
1. When you need to ask the user for some info, use the code to output the info you need, for example, browse or search the web, download/read a file.
In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. You must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user.
1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file.
2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to.
If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user.
If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes.
Verify your answer carefully. If a function for planning is provided, call the function to make plans and verify the execution.
If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.
When you find an answer, verify the answer carefully. If a function for planning is provided, call the function to make plans and verify the execution.
Reply "TERMINATE" in the end when everything is done.
"""

2
flaml/autogen/oai/completion.py
View File

@ -211,7 +211,7 @@ class Completion(openai_Completion):
logger.info(f"retrying in {cls.retry_time} seconds...", exc_info=1)
sleep(cls.retry_time)
except APIError as err:
error_code = err and err.json_body and err.json_body.get("error")
error_code = err and err.json_body and isinstance(err.json_body, dict) and err.json_body.get("error")
error_code = error_code and error_code.get("code")
if error_code == "content_filter":
raise

2
flaml/autogen/oai/openai_utils.py
View File

@ -107,7 +107,7 @@ def config_list_openai_aoai(
# Assuming Azure OpenAI api bases in os.environ["AZURE_OPENAI_API_BASE"], in separated lines
api_bases=os.environ.get("AZURE_OPENAI_API_BASE", "").split("\n"),
api_type="azure",
api_version="2023-03-15-preview", # change if necessary
api_version="2023-06-01-preview", # change if necessary
)
if exclude != "aoai"
else []

2
flaml/version.py
View File

@ -1 +1 @@
__version__ = "2.0.0rc3"
__version__ = "2.0.0rc4"

550
notebook/autogen_agent_auto_feedback_from_code_execution.ipynb
View File

File diff suppressed because one or more lines are too long

10
notebook/autogen_agent_function_call.ipynb
View File

@ -54,8 +54,9 @@
"- Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
"\n",
"It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base.\n",
"If you open this notebook in google colab, you can upload your files by click the file icon on the left panel and then choose \"upload file\" icon.\n",
"\n",
"If you open this notebook in google colab, you can upload your files by click the file icon on the left panel and then choose \"upload file\" icon.\n"
"The following code excludes Azure OpenAI endpoints from the config list because they don't support functions yet. Remove the `exclude` argument after they do."
]
},
{
@ -67,7 +68,7 @@
"source": [
"from flaml import oai\n",
"\n",
"config_list = oai.config_list_from_models(model_list=[\"gpt-4-0613\"])"
"config_list = oai.config_list_from_models(model_list=[\"gpt-4\", \"gpt-3.5-turbo\", \"gpt-3.5-turbo-16k\"], exclude=\"aoai\")"
]
},
{
@ -212,7 +213,7 @@
" \"functions\": [\n",
" {\n",
" \"name\": \"execute_code\",\n",
" \"description\": \"Receive a list of python code or shell script and return the execution result.\",\n",
" \"description\": \"Receive a python code or shell script and return the execution result.\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
@ -237,6 +238,7 @@
"# for simplicity, we don't pass in `exec_func` directly to UserProxyAgent because it requires a list of tuple as parameter\n",
"# instead, we define a wrapper function to call `exec_func`\n",
"exec_func = UserProxyAgent(name=\"execute_code\", work_dir=\"coding\", use_docker=False)._execute_code\n",
"\n",
"def execute_code(code_type, code):\n",
" return exec_func([(code_type, code)])\n",
"\n",
@ -420,7 +422,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
"version": "3.9.15"
}
},
"nbformat": 4,

571
notebook/autogen_agent_human_feedback.ipynb
View File

File diff suppressed because one or more lines are too long

252
notebook/autogen_agent_planning.ipynb
View File

@ -99,7 +99,7 @@
" config_list=config_list,\n",
" model=\"gpt-4\",\n",
" # the default system message of the AssistantAgent is overwritten here\n",
" system_message=\"You are a helpful AI assistant. You suggest coding and reasoning steps for another AI assistant to accomplish a task. Do not suggest concrete code. For any action beyond writing code or reasoning, convert it to a step which can be implemented by writing code. For example, the action of browsing the web can be implemented by writing code which reads and prints the content of a web page. Finally, inspect the execution result. If the plan is not good, suggest a better plan. If the execution is wrong, point out the error and suggest a fix.\"\n",
" system_message=\"You are a helpful AI assistant. You suggest coding and reasoning steps for another AI assistant to accomplish a task. Do not suggest concrete code. For any action beyond writing code or reasoning, convert it to a step which can be implemented by writing code. For example, the action of browsing the web can be implemented by writing code which reads and prints the content of a web page. Finally, inspect the execution result. If the plan is not good, suggest a better plan. If the execution is wrong, analyze the error and suggest a fix.\"\n",
")\n",
"planner_user = UserProxyAgent(\n",
" name=\"planner_user\",\n",
@ -135,6 +135,7 @@
" # Change to `exclude=\"openai\"` to exclude openai endpoints, or remove the `exclude` argument to include both.\n",
" config_list=oai.config_list_openai_aoai(exclude=\"aoai\"),\n",
" model=\"gpt-4-0613\", # make sure the endpoint you use supports the model\n",
" temperature=0,\n",
" functions=[\n",
" {\n",
" \"name\": \"ask_planner\",\n",
@ -190,52 +191,56 @@
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"To fix a good first issue in an open source project like FLAML (Fast, Lightweight AutoML), you need to first find an issue. I can assist in finding an issue and provide code examples for how you might address it.\n",
"To suggest a fix to an open good first issue of FLAML, we first need to fetch the list of open issues labeled as \"good first issue\" from the FLAML GitHub repository. We can do this using the GitHub API. Here is a Python script that fetches and prints the list of such issues.\n",
"\n",
"Below is a shell script (Bash) that performs these steps:\n",
"1. Uses `curl` to fetch the list of issues labelled as 'good first issue' from the GitHub API.\n",
"2. Prints out the title and URL of the first issue in the list.\n",
"```python\n",
"# filename: fetch_issues.py\n",
"\n",
"Here is the script, you may run this in a linux terminal:\n",
"import requests\n",
"import json\n",
"\n",
"```shell\n",
"# filename: get_issues.sh\n",
"def fetch_issues():\n",
" url = \"https://api.github.com/repos/microsoft/FLAML/issues\"\n",
" response = requests.get(url, params={\"state\": \"open\", \"labels\": \"good first issue\"})\n",
" if response.status_code == 200:\n",
" issues = json.loads(response.text)\n",
" for issue in issues:\n",
" print(f\"Issue #{issue['number']}: {issue['title']}\")\n",
" else:\n",
" print(f\"Failed to fetch issues. HTTP status code: {response.status_code}\")\n",
"\n",
"#!/bin/bash\n",
"\n",
"# Replace 'username:token' with your GitHub username and personal access token\n",
"auth='username:token'\n",
"\n",
"# GitHub API URL for 'good first issue' labeled issues in the 'microsoft/FLAML' repository\n",
"api_url='https://api.github.com/repos/microsoft/FLAML/issues?labels=good%20first%20issue'\n",
"\n",
"# Fetch issues and filter to get the title and URL of the first one\n",
"issue=$(curl -u $auth -s $api_url | jq -r '.[0] | \"\\(.title): \\(.html_url)\"')\n",
"\n",
"# Print the title and URL of the first issue\n",
"echo \"Good First Issue: $issue\"\n",
"fetch_issues()\n",
"```\n",
"\n",
"Please replace 'username:token' with your actual GitHub username and a [personal access token](https://github.com/settings/tokens) you have created. If you do not have a personal access token, please follow the provided link to create one.\n",
"\n",
"Please note that you will need the `jq` tool to run this script. If you don't have it installed, you can install it with:\n",
"\n",
"```shell\n",
"sudo apt-get install jq\n",
"```\n",
"\n",
"Before proceeding with a fix, inspect the issue, understand it and plan an approach for the fix. Once you have a rough idea for a fix, fork the FLAML repository, create a new branch, implement the fix and finally open a pull request to the main FLAML repository. Please check the project guidelines for contributing if they exist. Also keep the conversation with the repository maintainers active for effective contribution. Good luck!\n",
"Please save this script as `fetch_issues.py` and run it. It will print the list of open issues labeled as \"good first issue\" in the FLAML repository.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 100 (execution failed)\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Good First Issue: Do not print data split method when user provides X_val in AutoML: https://github.com/microsoft/FLAML/issues/1099\n",
"\n",
"E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).\n",
"Issue #1120: use_label_encoder warning with xgboost\n",
"Issue #1099: Do not print data split method when user provides X_val in AutoML\n",
"Issue #1034: Use appropriate wait time for retry based on the error message. \n",
"Issue #1029: Issues with Adding Custom APIs in Auto Generation\n",
"Issue #981: Running flaml[tune] using \"-O\" flag for python interpreter (optimization - disables assertions) crashes\n",
"Issue #903: Conditional parameter flow2 crash\n",
"Issue #884: indentation space\n",
"Issue #882: Check if openml version is required\n",
"Issue #834: Adjust the indent\n",
"Issue #821: pip install flaml FAIL\n",
"Issue #807: Isolate the ensemble part and expose it to users\n",
"Issue #805: how to pass categorical features names or indices to learner\n",
"Issue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\n",
"Issue #764: Add an announcement of the discord channel\n",
"Issue #748: Documentation about small budget\n",
"Issue #737: Make zero-shot automl more discoverable\n",
"Issue #509: New HCrystalBall release\n",
"Issue #429: samples about conversion to ONNX\n",
"Issue #413: support anomaly detection\n",
"Issue #304: CatBoost Fails with Keyword 'groups'\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
@ -244,7 +249,7 @@
"***** Suggested function Call: ask_planner *****\n",
"Arguments: \n",
"{\n",
"\"message\": \"The user would like a fix for a good first issue from the FLAML project. The issue retrieved from GitHub is 'Do not print data split method when user provides X_val in AutoML: https://github.com/microsoft/FLAML/issues/1099'. Assume the flaml software is installed and ready to use. What would be the steps for a fix?\"\n",
" \"message\": \"Here are the open good first issues of FLAML: \\nIssue #1120: use_label_encoder warning with xgboost\\nIssue #1099: Do not print data split method when user provides X_val in AutoML\\nIssue #1034: Use appropriate wait time for retry based on the error message.\\nIssue #1029: Issues with Adding Custom APIs in Auto Generation\\nIssue #981: Running flaml[tune] using \\\"-O\\\" flag for python interpreter (optimization - disables assertions) crashes\\nIssue #903: Conditional parameter flow2 crash\\nIssue #884: indentation space\\nIssue #882: Check if openml version is required\\nIssue #834: Adjust the indent\\nIssue #821: pip install flaml FAIL\\nIssue #807: Isolate the ensemble part and expose it to users\\nIssue #805: how to pass categorical features names or indices to learner\\nIssue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\\nIssue #764: Add an announcement of the discord channel\\nIssue #748: Documentation about small budget\\nIssue #737: Make zero-shot automl more discoverable\\nIssue #509: New HCrystalBall release\\nIssue #429: samples about conversion to ONNX\\nIssue #413: support anomaly detection\\nIssue #304: CatBoost Fails with Keyword 'groups'\\n\\nPlease suggest a fix for one of these issues.\"\n",
"}\n",
"************************************************\n",
"\n",
@ -253,89 +258,81 @@
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"planner_user (to planner):\n",
"\n",
"The user would like a fix for a good first issue from the FLAML project. The issue retrieved from GitHub is 'Do not print data split method when user provides X_val in AutoML: https://github.com/microsoft/FLAML/issues/1099'. Assume the flaml software is installed and ready to use. What would be the steps for a fix?\n",
"Here are the open good first issues of FLAML: \n",
"Issue #1120: use_label_encoder warning with xgboost\n",
"Issue #1099: Do not print data split method when user provides X_val in AutoML\n",
"Issue #1034: Use appropriate wait time for retry based on the error message.\n",
"Issue #1029: Issues with Adding Custom APIs in Auto Generation\n",
"Issue #981: Running flaml[tune] using \"-O\" flag for python interpreter (optimization - disables assertions) crashes\n",
"Issue #903: Conditional parameter flow2 crash\n",
"Issue #884: indentation space\n",
"Issue #882: Check if openml version is required\n",
"Issue #834: Adjust the indent\n",
"Issue #821: pip install flaml FAIL\n",
"Issue #807: Isolate the ensemble part and expose it to users\n",
"Issue #805: how to pass categorical features names or indices to learner\n",
"Issue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\n",
"Issue #764: Add an announcement of the discord channel\n",
"Issue #748: Documentation about small budget\n",
"Issue #737: Make zero-shot automl more discoverable\n",
"Issue #509: New HCrystalBall release\n",
"Issue #429: samples about conversion to ONNX\n",
"Issue #413: support anomaly detection\n",
"Issue #304: CatBoost Fails with Keyword 'groups'\n",
"\n",
"Please suggest a fix for one of these issues.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"planner (to planner_user):\n",
"\n",
"To fix this issue in FLAML project, you can follow these steps:\n",
"To provide appropriate suggestions for fixes, we first need to isolate a problem, understand it, and then propose a solution. Let's consider this issue as an example: \n",
"\n",
"1. Understand the problem: Read the issue description to get a clear understanding of the problem. In this case, it is about not printing the data split method when the user provides the `X_val` parameter during the AutoML process.\n",
"Issue #1099: Do not print data split method when user provides X_val in AutoML\n",
"\n",
"2. Locate the relevant code: Identify the relevant code section in the FLAML project where the data split message is printed. This might require browsing through the source code, specifically focusing on the AutoML and data splitting components.\n",
"\n",
"3. Analyze the current behavior: Check the conditions under which the data split message is printed and understand how the provided `X_val` parameter affects the code execution.\n",
"\n",
"4. Implement the fix: Modify the code, adding a condition that checks whether the `X_val` parameter has been provided by the user. If it has been provided, do not print the data split message.\n",
"\n",
"5. Test the fix: Create a test case or use an existing one that covers the scenario when the user provides the `X_val` parameter. Run the test to ensure that the data split message is not printed in this case.\n",
"\n",
"6. Verify other scenarios: Test other scenarios that include different combinations of input parameters and data split methods to ensure that your fix doesn't introduce any new issues or side effects.\n",
"\n",
"7. Document the changes: Update the documentation, including comments in the code and any relevant user guides, to accurately reflect the changes made.\n",
"\n",
"8. Create a pull request: Once you have tested and verified the fix, create a pull request to merge your changes into the main branch of the FLAML project. In the pull request description, link to the original issue to provide context for the reviewers.\n",
"\n",
"9. Address review comments: If the maintainers or other contributors provide feedback on your pull request, address their comments, and make necessary changes to the code until your pull request is approved.\n",
"\n",
"10. Merge the pull request: Once the maintainers approve your pull request, merge it into the main branch to resolve the issue.\n",
"Plan: \n",
"1. Understand the Source of the Problem: Study the part of the code where the issue originates. From the issue title, it seems like in the FLAML AutoML component, a data split method is being printed even when a user has provided a pre-split validation dataset.\n",
"2. Analyze the Code: If the user has already provided a validation data set (X_val), there would be no requirement for another data split. Therefore, it involves understanding why, nonetheless, a data split is printed. To solve this, the AI might have to go through the process where the software splits the data or loads the validation dataset, if available.\n",
"3. Propose a Solution: The solution could involve incorporating a conditional statement that checks if X_val is not null. If it isn't, then avoid printing the data split method. \n",
"4. Test the Solution: Once the code is implemented, it is important to test the solution by providing a validation dataset and ensuring that the data split information is no longer printed.\n",
"5. Inspect the result: If the proposed plan doesn't work as expected, analyze the issue and try to develop a better plan. If the execution is wrong, try to understand the error and adapt the approach accordingly.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"user (to assistant):\n",
"\n",
"***** Response from calling function \"ask_planner\" *****\n",
"To fix this issue in FLAML project, you can follow these steps:\n",
"To provide appropriate suggestions for fixes, we first need to isolate a problem, understand it, and then propose a solution. Let's consider this issue as an example: \n",
"\n",
"1. Understand the problem: Read the issue description to get a clear understanding of the problem. In this case, it is about not printing the data split method when the user provides the `X_val` parameter during the AutoML process.\n",
"Issue #1099: Do not print data split method when user provides X_val in AutoML\n",
"\n",
"2. Locate the relevant code: Identify the relevant code section in the FLAML project where the data split message is printed. This might require browsing through the source code, specifically focusing on the AutoML and data splitting components.\n",
"\n",
"3. Analyze the current behavior: Check the conditions under which the data split message is printed and understand how the provided `X_val` parameter affects the code execution.\n",
"\n",
"4. Implement the fix: Modify the code, adding a condition that checks whether the `X_val` parameter has been provided by the user. If it has been provided, do not print the data split message.\n",
"\n",
"5. Test the fix: Create a test case or use an existing one that covers the scenario when the user provides the `X_val` parameter. Run the test to ensure that the data split message is not printed in this case.\n",
"\n",
"6. Verify other scenarios: Test other scenarios that include different combinations of input parameters and data split methods to ensure that your fix doesn't introduce any new issues or side effects.\n",
"\n",
"7. Document the changes: Update the documentation, including comments in the code and any relevant user guides, to accurately reflect the changes made.\n",
"\n",
"8. Create a pull request: Once you have tested and verified the fix, create a pull request to merge your changes into the main branch of the FLAML project. In the pull request description, link to the original issue to provide context for the reviewers.\n",
"\n",
"9. Address review comments: If the maintainers or other contributors provide feedback on your pull request, address their comments, and make necessary changes to the code until your pull request is approved.\n",
"\n",
"10. Merge the pull request: Once the maintainers approve your pull request, merge it into the main branch to resolve the issue.\n",
"Plan: \n",
"1. Understand the Source of the Problem: Study the part of the code where the issue originates. From the issue title, it seems like in the FLAML AutoML component, a data split method is being printed even when a user has provided a pre-split validation dataset.\n",
"2. Analyze the Code: If the user has already provided a validation data set (X_val), there would be no requirement for another data split. Therefore, it involves understanding why, nonetheless, a data split is printed. To solve this, the AI might have to go through the process where the software splits the data or loads the validation dataset, if available.\n",
"3. Propose a Solution: The solution could involve incorporating a conditional statement that checks if X_val is not null. If it isn't, then avoid printing the data split method. \n",
"4. Test the Solution: Once the code is implemented, it is important to test the solution by providing a validation dataset and ensuring that the data split information is no longer printed.\n",
"5. Inspect the result: If the proposed plan doesn't work as expected, analyze the issue and try to develop a better plan. If the execution is wrong, try to understand the error and adapt the approach accordingly.\n",
"********************************************************\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Here is a Python script that may fix the issue in the FLAML project.\n",
"\n",
"Let's assume that the message is printed from the `AutoML.fit()` method. Please adapt the script to your actual project layout.\n",
"Based on the plan, here is a Python script that you can use to fix the issue #1099. This script assumes that the data split method is printed in a function called `split_data` in a file called `automl.py` (you need to replace these with the actual function and file names). \n",
"\n",
"```python\n",
"# filename: fix_issue_1099.py\n",
"\n",
"def fit(self, X_train, y_train, X_val=None, y_val=None, **kwargs):\n",
" # Other code...\n",
"\n",
" # Check if the user provided X_val and if so, avoid printing the data split method\n",
" if X_val is None:\n",
" print(\"\\nData split method: ...\")\n",
" \n",
" # Remaining code...\n",
"\n",
"# You should replace ... with the actual method details or variable holding it\n",
"def split_data(X, y, X_val=None):\n",
" if X_val is not None:\n",
" # If validation data is provided, use it without splitting\n",
" print(\"Using provided validation data\")\n",
" return X, y, X_val\n",
" else:\n",
" # If validation data is not provided, split the data and print the split method\n",
" print(\"Splitting data using method X\")\n",
" # Code to split the data goes here\n",
" return X_train, y_train, X_val\n",
"```\n",
"\n",
"This script modifies the `fit()` method of the `AutoML` class in FLAML. Before this script can be correctly executed, it should replace the actual code in the FLAML project.\n",
"\n",
"Please be sure to thoroughly test the fix and to re-run all unit tests for the `AutoML` class and integration tests involving the `AutoML` class to make sure everything still works as expected.\n",
"\n",
"After you have tested and confirmed that everything is working as expected, you can commit your change to your local git repository and push it to your remote repository. Then you can create a pull request on the FLAML repository for your changes to be reviewed and possibly merged into the main branch.\n",
"\n",
"This is only a hypothetical fix. In the actual FLAML project, the relevant code section may be in different files or classes/functions or need more complex modifications. Always carefully examine the specific codebase and the issue details before making modifications.\n",
"Please replace the function name and file name with the actual ones, save this script as `fix_issue_1099.py`, and run it. This script will fix the issue by not printing the data split method when validation data is provided.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
@ -349,26 +346,67 @@
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Great! It seems you have successfully implemented the hypothetical fix based on my suggestion. \n",
"\n",
"The next steps would be to:\n",
"1. Test the fix in different scenarios to ensure it works as expected and doesn't introduce new issues. \n",
"2. Once you validate the solution, commit the changes to your local git repository and push the updates to your forked repository on GitHub.\n",
"3. Then revisit the FLAML repository on GitHub and open a new Pull Request, proposing your changes to be merged into the main project.\n",
"4. Describe your changes in the Pull Request and link it back to the issue it solves.\n",
"\n",
"Keep in mind that this was a hypothetical fix and it may not directly apply to the real FLAML project. Always make sure to read the project guidelines for contributing, understand the issue properly and inspect the codebase thoroughly before making changes.\n",
"\n",
"Keep contributing to open source projects, every contribution counts! \n",
"\n",
"TERMINATE\n",
"***** Suggested function Call: ask_planner *****\n",
"Arguments: \n",
"{\n",
" \"message\": \"The user has executed the suggested code for fixing issue #1099 in FLAML. The code executed successfully without any output. Please suggest the next steps.\"\n",
"}\n",
"************************************************\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"planner_user (to planner):\n",
"\n",
"The user has executed the suggested code for fixing issue #1099 in FLAML. The code executed successfully without any output. Please suggest the next steps.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"planner (to planner_user):\n",
"\n",
"Great, if the code executed successfully, we can continue with the following steps:\n",
"\n",
"1. Code Review: Ensure the code is conforming to the overall structure, style, and standards of the FLAML project. Check if the changes are minimal, only related to the specific issue, and not affecting other parts of the codebase. \n",
"\n",
"2. Testing: Now, it's time to test the code under different scenarios. Here are some test cases you might consider:\n",
"\n",
" A. Provide a test when X_val is not provided. The code should behave in the same way as before your changes, including printing data split information.\n",
"\n",
" B. Provide a test where X_val is provided. The updated code should no longer print data split information.\n",
"\n",
" C. Test with different types of data (real numbers, categorical variables, missing values, etc.) for robustness. \n",
" \n",
"3. Review Error Messages or Unexpected Behavior: If anything doesn't work as expected, understand the cause and modify the code as needed.\n",
"\n",
"4. Documentation: If changes made to the code are substantial or might affect the way users interact with FLAML, properly document these changes. Make sure to clarify that if 'X_val' is provided, the data split method will not be printed.\n",
"\n",
"5. Submit the Code: When you're sure the changes fix the issue and don't introduce new ones, it would be the time to submit your code for review. Depending on the project guidelines, you may need to fork the repository, commit your changes, push the commit(s) to your fork, and then open a pull request.\n",
"\n",
"6. Wait for Feedback: Once the code is submitted, wait for the maintainers' feedback. They might suggest some changes, enhancements, or approve the changes right away. Keep an open communication channel and respond to the comments, making the suggested changes. You might need to go through a few iterations of this process until the code is accepted and the issue is closed.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"user (to assistant):\n",
"\n",
"***** Response from calling function \"ask_planner\" *****\n",
"Great, if the code executed successfully, we can continue with the following steps:\n",
"\n",
"1. Code Review: Ensure the code is conforming to the overall structure, style, and standards of the FLAML project. Check if the changes are minimal, only related to the specific issue, and not affecting other parts of the codebase. \n",
"\n",
"2. Testing: Now, it's time to test the code under different scenarios. Here are some test cases you might consider:\n",
"\n",
" A. Provide a test when X_val is not provided. The code should behave in the same way as before your changes, including printing data split information.\n",
"\n",
" B. Provide a test where X_val is provided. The updated code should no longer print data split information.\n",
"\n",
" C. Test with different types of data (real numbers, categorical variables, missing values, etc.) for robustness. \n",
" \n",
"3. Review Error Messages or Unexpected Behavior: If anything doesn't work as expected, understand the cause and modify the code as needed.\n",
"\n",
"4. Documentation: If changes made to the code are substantial or might affect the way users interact with FLAML, properly document these changes. Make sure to clarify that if 'X_val' is provided, the data split method will not be printed.\n",
"\n",
"5. Submit the Code: When you're sure the changes fix the issue and don't introduce new ones, it would be the time to submit your code for review. Depending on the project guidelines, you may need to fork the repository, commit your changes, push the commit(s) to your fork, and then open a pull request.\n",
"\n",
"6. Wait for Feedback: Once the code is submitted, wait for the maintainers' feedback. They might suggest some changes, enhancements, or approve the changes right away. Keep an open communication channel and respond to the comments, making the suggested changes. You might need to go through a few iterations of this process until the code is accepted and the issue is closed.\n",
"********************************************************\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",

633
notebook/autogen_agent_web_info.ipynb
View File

@ -106,6 +106,7 @@
" seed=42,\n",
" config_list=config_list,\n",
" model=\"gpt-4-32k\", # modify if the endpoint you use doesn't support this model\n",
" temperature=0,\n",
")\n",
"# create a UserProxyAgent instance named \"user\"\n",
"user = UserProxyAgent(\n",
@ -145,7 +146,7 @@
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I will need to check the abstract of the paper to determine who should read it. Let me do that for you.\n",
"To determine who should read this paper, I will first extract the abstract and analyze its content. Here's a Python code to fetch the abstract from the given URL:\n",
"\n",
"```python\n",
"import requests\n",
@ -155,11 +156,10 @@
"response = requests.get(url)\n",
"soup = BeautifulSoup(response.text, \"html.parser\")\n",
"abstract = soup.find(\"blockquote\", {\"class\": \"abstract\"}).text.strip()\n",
"\n",
"print(\"Abstract:\", abstract)\n",
"print(abstract)\n",
"```\n",
"\n",
"Please run the above Python code to fetch the abstract of the paper. Based on the abstract, we can determine who should read it.\n",
"Please execute the above code to get the abstract of the paper.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
@ -168,7 +168,7 @@
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Abstract: Abstract: Employing Large Language Models (LLMs) to address mathematical problems is an\n",
"Abstract: Employing Large Language Models (LLMs) to address mathematical problems is an\n",
"intriguing research endeavor, considering the abundance of math problems\n",
"expressed in natural language across numerous science and engineering fields.\n",
"While several prior works have investigated solving elementary mathematics\n",
@ -184,13 +184,15 @@
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Based on the abstract, the paper is about using Large Language Models, specifically GPT-4, to solve complex mathematical problems. The paper introduces a conversational problem-solving framework called MathChat and evaluates its performance on challenging high school competition problems from the MATH dataset.\n",
"Based on the abstract, the paper is about using Large Language Models (LLMs), specifically GPT-4, to solve complex mathematical problems. The paper introduces a conversational problem-solving framework called MathChat and evaluates its performance on difficult high school competition problems from the MATH dataset.\n",
"\n",
"The paper should be of interest to the following groups:\n",
"The target audience for this paper includes:\n",
"\n",
"1. Researchers and practitioners working in the field of artificial intelligence, specifically natural language processing and language models.\n",
"2. Professionals interested in the application of AI for solving mathematical problems.\n",
"3. Educators and instructors focused on teaching mathematics and incorporating AI-based solutions into teaching methodologies.\n",
"1. Researchers and practitioners in the field of artificial intelligence, natural language processing, and machine learning, particularly those interested in large language models like GPT-4.\n",
"2. Mathematicians and educators who are interested in exploring the potential of AI for solving complex mathematical problems and improving educational tools.\n",
"3. Developers working on AI-based applications for problem-solving, tutoring, or educational purposes.\n",
"\n",
"If you belong to any of these categories or have a general interest in AI and its applications in mathematics, you should consider reading this paper.\n",
"\n",
"TERMINATE\n",
"\n",
@ -218,7 +220,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"metadata": {},
"outputs": [
{
@ -227,48 +229,42 @@
"text": [
"user (to assistant):\n",
"\n",
"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?\n",
"Show me the YTD gain of 10 largest technology companies as of today.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"First, let's find out today's date. Please execute the following Python code:\n",
"To get the YTD gain of the 10 largest technology companies, we can use a Python script to fetch the data from a financial API. We'll use the free Alpha Vantage API for this purpose. First, you need to obtain an API key from the Alpha Vantage website. Please visit https://www.alphavantage.co/support/#api-key and get your API key.\n",
"\n",
"Once you have the API key, please replace `YOUR_API_KEY` in the code below with your actual API key and execute the code:\n",
"\n",
"```python\n",
"import datetime\n",
"import requests\n",
"import json\n",
"\n",
"today = datetime.date.today()\n",
"print(\"Today's date:\", today)\n",
"api_key = \"YOUR_API_KEY\"\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"FB\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"def get_ytd_gain(symbol):\n",
" url = f\"https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol={symbol}&apikey={api_key}\"\n",
" response = requests.get(url)\n",
" data = json.loads(response.text)\n",
" daily_data = data[\"Time Series (Daily)\"]\n",
" sorted_dates = sorted(daily_data.keys())\n",
" first_day_of_year = next(date for date in sorted_dates if date.startswith(\"2022-01\"))\n",
" last_day = sorted_dates[-1]\n",
" open_price = float(daily_data[first_day_of_year][\"1. open\"])\n",
" close_price = float(daily_data[last_day][\"4. close\"])\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
"```\n",
"\n",
"Once we know today's date, let's find out which big tech stock has the largest year-to-date gain. I will provide a code to search for this information. This task requires the `yfinance` package. Install it by running:\n",
"\n",
"```sh\n",
"pip install yfinance\n",
"```\n",
"\n",
"Next, execute the following Python code:\n",
"\n",
"```python\n",
"import yfinance as yf\n",
"\n",
"big_tech_stocks = [\"AAPL\", \"GOOGL\", \"AMZN\", \"MSFT\", \"FB\"]\n",
"max_gain = 0\n",
"max_gain_stock = \"\"\n",
"\n",
"for stock_symbol in big_tech_stocks:\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" current_price = stock_data[\"Close\"][-1]\n",
" gain_percentage = (current_price - year_start_price) / year_start_price * 100\n",
"\n",
" if gain_percentage > max_gain:\n",
" max_gain = gain_percentage\n",
" max_gain_stock = stock_symbol\n",
"\n",
"print(\"The big tech stock with the largest year-to-date gain is\", max_gain_stock)\n",
"print(f\"The gain is {max_gain:.2f}%\")\n",
"```\n",
"This script fetches the daily adjusted stock data for the 10 largest technology companies and calculates the YTD gain for each of them. The results will be printed in the format \"Symbol: YTD Gain%\".\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
@ -277,8 +273,249 @@
"\n",
"exitcode: 1 (execution failed)\n",
"Code output: \n",
"Today's date: 2023-07-10\n",
"Traceback (most recent call last):\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_0fb74c6e622b29ddd2ed9f0415582b98.py\", line 20, in <module>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_0fb74c6e622b29ddd2ed9f0415582b98.py\", line 20, in <dictcomp>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_0fb74c6e622b29ddd2ed9f0415582b98.py\", line 13, in get_ytd_gain\n",
" first_day_of_year = next(date for date in sorted_dates if date.startswith(\"2022-01\"))\n",
"StopIteration\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the error. It seems that the code failed to find the first trading day of the year. Let's modify the code to handle this issue:\n",
"\n",
"```python\n",
"import requests\n",
"import json\n",
"\n",
"api_key = \"YOUR_API_KEY\"\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"FB\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"def get_ytd_gain(symbol):\n",
" url = f\"https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol={symbol}&apikey={api_key}\"\n",
" response = requests.get(url)\n",
" data = json.loads(response.text)\n",
" daily_data = data[\"Time Series (Daily)\"]\n",
" sorted_dates = sorted(daily_data.keys())\n",
" first_day_of_year = None\n",
" for date in sorted_dates:\n",
" if date.startswith(\"2022-01\"):\n",
" first_day_of_year = date\n",
" break\n",
" if first_day_of_year is None:\n",
" return None\n",
" last_day = sorted_dates[-1]\n",
" open_price = float(daily_data[first_day_of_year][\"1. open\"])\n",
" close_price = float(daily_data[last_day][\"4. close\"])\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"Please replace `YOUR_API_KEY` with your actual API key and execute the code again. This should resolve the issue and display the YTD gains for the 10 largest technology companies.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 1 (execution failed)\n",
"Code output: \n",
"Traceback (most recent call last):\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_a050ea892378b99f57347f4e08c4a294.py\", line 26, in <module>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_a050ea892378b99f57347f4e08c4a294.py\", line 26, in <dictcomp>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_a050ea892378b99f57347f4e08c4a294.py\", line 11, in get_ytd_gain\n",
" daily_data = data[\"Time Series (Daily)\"]\n",
"KeyError: 'Time Series (Daily)'\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the error. It seems that the Alpha Vantage API is not returning the expected data. This might be due to the API call limit or an issue with the API itself. To resolve this, let's add a delay between API calls and handle the KeyError exception:\n",
"\n",
"```python\n",
"import requests\n",
"import json\n",
"import time\n",
"\n",
"api_key = \"YOUR_API_KEY\"\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"FB\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"def get_ytd_gain(symbol):\n",
" url = f\"https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol={symbol}&apikey={api_key}\"\n",
" response = requests.get(url)\n",
" data = json.loads(response.text)\n",
" if \"Time Series (Daily)\" not in data:\n",
" return None\n",
" daily_data = data[\"Time Series (Daily)\"]\n",
" sorted_dates = sorted(daily_data.keys())\n",
" first_day_of_year = None\n",
" for date in sorted_dates:\n",
" if date.startswith(\"2022-01\"):\n",
" first_day_of_year = date\n",
" break\n",
" if first_day_of_year is None:\n",
" return None\n",
" last_day = sorted_dates[-1]\n",
" open_price = float(daily_data[first_day_of_year][\"1. open\"])\n",
" close_price = float(daily_data[last_day][\"4. close\"])\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"ytd_gains = {}\n",
"for symbol in symbols:\n",
" ytd_gains[symbol] = get_ytd_gain(symbol)\n",
" time.sleep(15) # Add a delay between API calls\n",
"\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"Please replace `YOUR_API_KEY` with your actual API key and execute the code again. This should resolve the issue and display the YTD gains for the 10 largest technology companies. Note that the script will take some time to execute due to the added delay between API calls.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"YTD Gains of 10 Largest Technology Companies:\n",
"AAPL: Data not available\n",
"MSFT: Data not available\n",
"GOOGL: Data not available\n",
"GOOG: Data not available\n",
"AMZN: Data not available\n",
"FB: Data not available\n",
"TSLA: Data not available\n",
"NVDA: Data not available\n",
"ADBE: Data not available\n",
"ASML: Data not available\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the inconvenience. It seems that the Alpha Vantage API is not providing the required data. In this case, let's switch to another API, the FRED (Federal Reserve Economic Data) API, which provides historical stock data. First, you need to obtain an API key from the FRED website. Please visit https://research.stlouisfed.org/docs/api/api_key.html and get your API key.\n",
"\n",
"Once you have the API key, please replace `YOUR_API_KEY` in the code below with your actual API key and execute the code:\n",
"\n",
"```python\n",
"import requests\n",
"import json\n",
"import time\n",
"\n",
"api_key = \"YOUR_API_KEY\"\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"FB\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"def get_ytd_gain(symbol):\n",
" url = f\"https://fred.stlouisfed.org/graph/fredgraph.csv?bgcolor=%23e1e9f0&chart_type=line&drp=0&fo=open%20sans&graph_bgcolor=%23ffffff&height=450&mode=fred&recession_bars=on&txtcolor=%23444444&ts=12&tts=12&width=1168&nt=0&thu=0&trc=0&show_legend=yes&show_axis_titles=yes&show_tooltip=yes&id={symbol}&scale=left&cosd=2022-01-01&coed=2022-12-31&line_color=%234572a7&link_values=false&line_style=solid&mark_type=none&mw=3&lw=2&ost=-99999&oet=99999&mma=0&fml=a&fq=Daily&fam=avg&fgst=lin&fgsnd=2020-02-01&line_index=1&transformation=lin&vintage_date=2022-12-31&revision_date=2022-12-31&nd=2022-01-01\"\n",
" response = requests.get(url)\n",
" data = response.text.splitlines()\n",
" if len(data) < 2:\n",
" return None\n",
" first_day_of_year = data[1].split(\",\")[1]\n",
" last_day = data[-1].split(\",\")[1]\n",
" open_price = float(first_day_of_year)\n",
" close_price = float(last_day)\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"ytd_gains = {}\n",
"for symbol in symbols:\n",
" ytd_gains[symbol] = get_ytd_gain(symbol)\n",
" time.sleep(5) # Add a delay between API calls\n",
"\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"This script fetches the daily stock data for the 10 largest technology companies from the FRED API and calculates the YTD gain for each of them. The results will be printed in the format \"Symbol: YTD Gain%\". Note that the script will take some time to execute due to the added delay between API calls.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 1 (execution failed)\n",
"Code output: \n",
"Traceback (most recent call last):\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_ba93be30359073e751a640cd13b3ef7e.py\", line 23, in <module>\n",
" ytd_gains[symbol] = get_ytd_gain(symbol)\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_ba93be30359073e751a640cd13b3ef7e.py\", line 14, in get_ytd_gain\n",
" first_day_of_year = data[1].split(\",\")[1]\n",
"IndexError: list index out of range\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the confusion. It seems that the FRED API does not provide the required stock data. In this case, let's use the Yahoo Finance API, which is a popular choice for fetching stock data. We will use the `yfinance` library to fetch the data. First, you need to install the library by executing the following command:\n",
"\n",
"```sh\n",
"pip install yfinance\n",
"```\n",
"\n",
"Once the library is installed, please execute the following code:\n",
"\n",
"```python\n",
"import yfinance as yf\n",
"import datetime\n",
"\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"FB\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"def get_ytd_gain(symbol):\n",
" now = datetime.datetime.now()\n",
" start_date = datetime.datetime(now.year, 1, 1)\n",
" end_date = now\n",
" stock_data = yf.download(symbol, start=start_date, end=end_date)\n",
" if stock_data.empty:\n",
" return None\n",
" open_price = stock_data.iloc[0][\"Open\"]\n",
" close_price = stock_data.iloc[-1][\"Close\"]\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"This script fetches the daily stock data for the 10 largest technology companies from the Yahoo Finance API using the `yfinance` library and calculates the YTD gain for each of them. The results will be printed in the format \"Symbol: YTD Gain%\".\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Defaulting to user installation because normal site-packages is not writeable\n",
"Requirement already satisfied: yfinance in /home/vscode/.local/lib/python3.9/site-packages (0.2.18)\n",
"Requirement already satisfied: pandas>=1.3.0 in /usr/local/lib/python3.9/site-packages (from yfinance) (1.5.2)\n",
@ -303,42 +540,76 @@
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests>=2.26->yfinance) (2022.9.24)\n",
"Requirement already satisfied: pycparser in /home/vscode/.local/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=3.3.2->yfinance) (2.21)\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_dad672c8a5e8e3b25c27c3bd8085848d.py\", line 8, in <module>\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
"NameError: name 'today' is not defined\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"\n",
"1 Failed download:\n",
"- FB: No timezone found, symbol may be delisted\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"YTD Gains of 10 Largest Technology Companies:\n",
"AAPL: 44.37%\n",
"MSFT: 36.77%\n",
"GOOGL: 30.75%\n",
"GOOG: 31.04%\n",
"AMZN: 50.69%\n",
"FB: Data not available\n",
"TSLA: 127.73%\n",
"NVDA: 185.54%\n",
"ADBE: 48.38%\n",
"ASML: 25.61%\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the oversight. It seems that the `today` variable from the previous code block is not accessible in the current code block. Let's modify the code to include fetching today's date in the same code block. Please execute the following Python code:\n",
"Great! The code executed successfully, and the YTD gains of the 10 largest technology companies are as follows:\n",
"\n",
"- AAPL: 44.37%\n",
"- MSFT: 36.77%\n",
"- GOOGL: 30.75%\n",
"- GOOG: 31.04%\n",
"- AMZN: 50.69%\n",
"- FB: Data not available (The symbol may be delisted or not available in the Yahoo Finance API)\n",
"- TSLA: 127.73%\n",
"- NVDA: 185.54%\n",
"- ADBE: 48.38%\n",
"- ASML: 25.61%\n",
"\n",
"Please note that these values are subject to change as the stock market fluctuates. If you need to update the values, simply re-run the code.\n",
"\n",
"TERMINATE\n",
"\n",
"--------------------------------------------------------------------------------\n",
"user (to assistant):\n",
"\n",
"continue\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the confusion regarding the FB symbol. It seems that the symbol for Meta Platforms, Inc. (formerly Facebook, Inc.) has changed to \"META\" on the Yahoo Finance API. Let's update the symbol and re-run the code to get the YTD gain for Meta Platforms, Inc.\n",
"\n",
"```python\n",
"import datetime\n",
"import yfinance as yf\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"META\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"today = datetime.date.today()\n",
"print(\"Today's date:\", today)\n",
"\n",
"big_tech_stocks = [\"AAPL\", \"GOOGL\", \"AMZN\", \"MSFT\", \"FB\"]\n",
"max_gain = 0\n",
"max_gain_stock = \"\"\n",
"\n",
"for stock_symbol in big_tech_stocks:\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" current_price = stock_data[\"Close\"][-1]\n",
" gain_percentage = (current_price - year_start_price) / year_start_price * 100\n",
"\n",
" if gain_percentage > max_gain:\n",
" max_gain = gain_percentage\n",
" max_gain_stock = stock_symbol\n",
"\n",
"print(\"The big tech stock with the largest year-to-date gain is\", max_gain_stock)\n",
"print(f\"The gain is {max_gain:.2f}%\")\n",
"ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"Please execute the code above to get the updated YTD gains for the 10 largest technology companies, including Meta Platforms, Inc. (formerly Facebook, Inc.).\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
@ -347,47 +618,47 @@
"exitcode: 1 (execution failed)\n",
"Code output: \n",
"Traceback (most recent call last):\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_0d5c40281daaf2e5e5430a5b89413be7.py\", line 13, in <module>\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" File \"/usr/local/lib/python3.9/site-packages/pandas/core/series.py\", line 978, in __getitem__\n",
" return self._values[key]\n",
"IndexError: index 0 is out of bounds for axis 0 with size 0\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_cfc7665767b48828cfef36e1a2a3dbc6.py\", line 3, in <module>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
" File \"/workspaces/FLAML/notebook/web/tmp_code_cfc7665767b48828cfef36e1a2a3dbc6.py\", line 3, in <dictcomp>\n",
" ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"NameError: name 'get_ytd_gain' is not defined\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"I apologize for the error. It seems that there is an issue with the stock data retrieved. Let's add a check to ensure that the stock data is available before processing it. Please execute the following modified Python code:\n",
"I apologize for the error. It seems I didn't include the `get_ytd_gain` function in the code snippet. Here's the complete code with the updated symbol for Meta Platforms, Inc. (formerly Facebook, Inc.):\n",
"\n",
"```python\n",
"import datetime\n",
"import yfinance as yf\n",
"import datetime\n",
"\n",
"today = datetime.date.today()\n",
"print(\"Today's date:\", today)\n",
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"GOOG\", \"AMZN\", \"META\", \"TSLA\", \"NVDA\", \"ADBE\", \"ASML\"]\n",
"\n",
"big_tech_stocks = [\"AAPL\", \"GOOGL\", \"AMZN\", \"MSFT\", \"FB\"]\n",
"max_gain = 0\n",
"max_gain_stock = \"\"\n",
"def get_ytd_gain(symbol):\n",
" now = datetime.datetime.now()\n",
" start_date = datetime.datetime(now.year, 1, 1)\n",
" end_date = now\n",
" stock_data = yf.download(symbol, start=start_date, end=end_date)\n",
" if stock_data.empty:\n",
" return None\n",
" open_price = stock_data.iloc[0][\"Open\"]\n",
" close_price = stock_data.iloc[-1][\"Close\"]\n",
" ytd_gain = (close_price - open_price) / open_price * 100\n",
" return ytd_gain\n",
"\n",
"for stock_symbol in big_tech_stocks:\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
" \n",
" if not stock_data.empty:\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" current_price = stock_data[\"Close\"][-1]\n",
" gain_percentage = (current_price - year_start_price) / year_start_price * 100\n",
"\n",
" if gain_percentage > max_gain:\n",
" max_gain = gain_percentage\n",
" max_gain_stock = stock_symbol\n",
"ytd_gains = {symbol: get_ytd_gain(symbol) for symbol in symbols}\n",
"print(\"YTD Gains of 10 Largest Technology Companies:\")\n",
"for symbol, gain in ytd_gains.items():\n",
" if gain is not None:\n",
" print(f\"{symbol}: {gain:.2f}%\")\n",
" else:\n",
" print(f\"Unable to retrieve data for stock {stock_symbol}\")\n",
"\n",
"print(\"The big tech stock with the largest year-to-date gain is\", max_gain_stock)\n",
"print(f\"The gain is {max_gain:.2f}%\")\n",
" print(f\"{symbol}: Data not available\")\n",
"```\n",
"\n",
"Please execute the code above to get the updated YTD gains for the 10 largest technology companies, including Meta Platforms, Inc. (formerly Facebook, Inc.).\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
@ -395,154 +666,46 @@
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Today's date: 2023-07-10\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"\n",
"1 Failed download:\n",
"- FB: No timezone found, symbol may be delisted\n",
"Unable to retrieve data for stock FB\n",
"The big tech stock with the largest year-to-date gain is AAPL\n",
"The gain is 52.46%\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"YTD Gains of 10 Largest Technology Companies:\n",
"AAPL: 44.37%\n",
"MSFT: 36.77%\n",
"GOOGL: 30.75%\n",
"GOOG: 31.04%\n",
"AMZN: 50.69%\n",
"META: 142.87%\n",
"TSLA: 127.73%\n",
"NVDA: 185.54%\n",
"ADBE: 48.38%\n",
"ASML: 25.61%\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Today's date is 2023-07-10. The big tech stock with the largest year-to-date gain is AAPL (Apple Inc.) with a gain of 52.46%. Please note that there was an issue retrieving the data for the stock FB (Facebook). It is possible that the symbol has been delisted or there is a temporary issue with the data source.\n",
"Great! The code executed successfully, and the updated YTD gains of the 10 largest technology companies are as follows:\n",
"\n",
"If you have any other questions or need further assistance, please let me know.\n",
"- AAPL: 44.37%\n",
"- MSFT: 36.77%\n",
"- GOOGL: 30.75%\n",
"- GOOG: 31.04%\n",
"- AMZN: 50.69%\n",
"- META (formerly FB): 142.87%\n",
"- TSLA: 127.73%\n",
"- NVDA: 185.54%\n",
"- ADBE: 48.38%\n",
"- ASML: 25.61%\n",
"\n",
"TERMINATE\n",
"\n",
"--------------------------------------------------------------------------------\n",
"user (to assistant):\n",
"\n",
"Replace FB with Meta\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Sure, let's replace \"FB\" with \"META\" in the stock list and run the modified code again to find the largest year-to-date gain.\n",
"\n",
"```python\n",
"import datetime\n",
"import yfinance as yf\n",
"\n",
"today = datetime.date.today()\n",
"print(\"Today's date:\", today)\n",
"\n",
"big_tech_stocks = [\"AAPL\", \"GOOGL\", \"AMZN\", \"MSFT\", \"META\"]\n",
"max_gain = 0\n",
"max_gain_stock = \"\"\n",
"\n",
"for stock_symbol in big_tech_stocks:\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
" \n",
" if not stock_data.empty:\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" current_price = stock_data[\"Close\"][-1]\n",
" gain_percentage = (current_price - year_start_price) / year_start_price * 100\n",
"\n",
" if gain_percentage > max_gain:\n",
" max_gain = gain_percentage\n",
" max_gain_stock = stock_symbol\n",
" else:\n",
" print(f\"Unable to retrieve data for stock {stock_symbol}\")\n",
"\n",
"print(\"The big tech stock with the largest year-to-date gain is\", max_gain_stock)\n",
"print(f\"The gain is {max_gain:.2f}%\")\n",
"```\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Today's date: 2023-07-10\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"The big tech stock with the largest year-to-date gain is META\n",
"The gain is 132.91%\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Today's date is 2023-07-10. The big tech stock with the largest year-to-date gain is META (Meta Platforms Inc., formerly known as Facebook) with a gain of 132.91%. If you have any other questions or need further assistance, please let me know.\n",
"\n",
"TERMINATE\n",
"\n",
"--------------------------------------------------------------------------------\n",
"user (to assistant):\n",
"\n",
"Add NVDA and TESLA\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Let's add \"NVDA\" (Nvidia Corporation) and \"TSLA\" (Tesla, Inc.) to the stock list and run the modified code again to find the largest year-to-date gain.\n",
"\n",
"```python\n",
"import datetime\n",
"import yfinance as yf\n",
"\n",
"today = datetime.date.today()\n",
"print(\"Today's date:\", today)\n",
"\n",
"big_tech_stocks = [\"AAPL\", \"GOOGL\", \"AMZN\", \"MSFT\", \"META\", \"NVDA\", \"TSLA\"]\n",
"max_gain = 0\n",
"max_gain_stock = \"\"\n",
"\n",
"for stock_symbol in big_tech_stocks:\n",
" stock_data = yf.download(stock_symbol, start=f\"{today.year}-01-01\", end=today)\n",
" \n",
" if not stock_data.empty:\n",
" year_start_price = stock_data[\"Close\"][0]\n",
" current_price = stock_data[\"Close\"][-1]\n",
" gain_percentage = (current_price - year_start_price) / year_start_price * 100\n",
"\n",
" if gain_percentage > max_gain:\n",
" max_gain = gain_percentage\n",
" max_gain_stock = stock_symbol\n",
" else:\n",
" print(f\"Unable to retrieve data for stock {stock_symbol}\")\n",
"\n",
"print(\"The big tech stock with the largest year-to-date gain is\", max_gain_stock)\n",
"print(f\"The gain is {max_gain:.2f}%\")\n",
"```\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"user (to assistant):\n",
"\n",
"exitcode: 0 (execution succeeded)\n",
"Code output: \n",
"Today's date: 2023-07-10\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"[*********************100%***********************] 1 of 1 completed\n",
"The big tech stock with the largest year-to-date gain is NVDA\n",
"The gain is 196.91%\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"assistant (to user):\n",
"\n",
"Today's date is 2023-07-10. The big tech stock with the largest year-to-date gain, including \"NVDA\" (Nvidia Corporation) and \"TSLA\" (Tesla, Inc.), is NVDA with a gain of 196.91%. If you have any other questions or need further assistance, please let me know.\n",
"Please note that these values are subject to change as the stock market fluctuates. If you need to update the values, simply re-run the code.\n",
"\n",
"TERMINATE\n",
"\n",
@ -554,7 +717,7 @@
"# it is suggested to reset the assistant to clear the state if the new task is not related to the previous one.\n",
"assistant.reset()\n",
"assistant.receive(\n",
" \"\"\"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?\"\"\",\n",
" \"\"\"Show me the YTD gain of 10 largest technology companies as of today.\"\"\",\n",
" user,\n",
")"
]

4
notebook/autogen_chatgpt_gpt4.ipynb
View File

@ -341,7 +341,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"For (local) reproducibility and cost efficiency, we cache responses from OpenAI."
"For (local) reproducibility and cost efficiency, we cache responses from OpenAI with a controllable seed."
]
},
{
@ -368,7 +368,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This will create a disk cache in \".cache/{seed}\". You can change `cache_path` in `set_cache()`. The cache for different seeds are stored separately.\n",
"This will create a disk cache in \".cache/{seed}\". You can change `cache_path_root` from \".cache\" to a different path in `set_cache()`. The cache for different seeds are stored separately.\n",
"\n",
"### Perform tuning\n",
"\n",

75
notebook/autogen_openai_completion.ipynb
View File

@ -56,7 +56,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Set your OpenAI key:"
"## Set your API Endpoint\n",
"\n",
"The [`config_list_openai_aoai`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_openai_aoai) function tries to create a list of configurations using Azure OpenAI endpoints and OpenAI endpoints. It assumes the api keys and api bases are stored in the corresponding environment variables or local txt files:\n",
"\n",
"- OpenAI API key: os.environ[\"OPENAI_API_KEY\"] or `openai_api_key_file=\"key_openai.txt\"`.\n",
"- Azure OpenAI API key: os.environ[\"AZURE_OPENAI_API_KEY\"] or `aoai_api_key_file=\"key_aoai.txt\"`. Multiple keys can be stored, one per line.\n",
"- Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
"\n",
"It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base.\n"
]
},
{
@ -72,37 +80,9 @@
},
"outputs": [],
"source": [
"import os\n",
"from flaml import oai\n",
"\n",
"if \"OPENAI_API_KEY\" not in os.environ:\n",
" os.environ[\"OPENAI_API_KEY\"] = \"<your OpenAI API key here>\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"If you use Azure OpenAI, uncomment the following:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"execution": {
"iopub.execute_input": "2023-02-24T23:25:36.925804Z",
"iopub.status.busy": "2023-02-24T23:25:36.925423Z",
"iopub.status.idle": "2023-02-24T23:25:36.928191Z",
"shell.execute_reply": "2023-02-24T23:25:36.927673Z"
}
},
"outputs": [],
"source": [
"# import openai\n",
"# openai.api_type = \"azure\"\n",
"# openai.api_base = \"https://<your_endpoint>.openai.azure.com/\"\n",
"# openai.api_version = \"2023-03-15-preview\" # change if necessary"
"config_list = oai.config_list_openai_aoai(exclude=\"openai\")"
]
},
{
@ -329,34 +309,9 @@
"\n",
"### Import the oai and tune subpackages from flaml.\n",
"\n",
"FLAML has provided an API for hyperparameter optimization of OpenAI models: `oai.Completion.tune` and to make a request with the tuned config: `oai.Completion.create`. First, we import oai from flaml:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"execution": {
"iopub.execute_input": "2023-02-24T23:25:39.179030Z",
"iopub.status.busy": "2023-02-24T23:25:39.178624Z",
"iopub.status.idle": "2023-02-24T23:25:40.584410Z",
"shell.execute_reply": "2023-02-24T23:25:40.583802Z"
},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"from flaml import oai"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"For (local) reproducibility and cost efficiency, we cache responses from OpenAI."
"FLAML has provided an API for hyperparameter optimization of OpenAI models: `oai.Completion.tune` and to make a request with the tuned config: `oai.Completion.create`.\n",
"\n",
"For (local) reproducibility and cost efficiency, we cache responses from OpenAI with a controllable seed."
]
},
{
@ -383,7 +338,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This will create a disk cache in \".cache/{seed}\". You can change `cache_path` in `set_cache()`. The cache for different seeds are stored separately.\n",
"This will create a disk cache in \".cache/{seed}\". You can change `cache_path_root` from \".cache\" to a different path in `set_cache()`. The cache for different seeds are stored separately.\n",
"\n",
"### Perform tuning\n",
"\n",

4
test/autogen/oai/test_completion.py
View File

@ -214,7 +214,7 @@ def test_humaneval(num_samples=1):
mode="max",
eval_func=eval_with_generated_assertions,
n=1,
model="gpt-3.5-turbo",
model="text-davinci-003",
prompt="{definition}",
)
responses = oai.Completion.create(context=test_data[0], **config)
@ -313,7 +313,7 @@ def test_math(num_samples=-1):
oai.ChatCompletion.set_cache(seed)
vanilla_config = {
"model": "gpt-3.5-turbo",
"model": "text-davinci-003",
"temperature": 1,
"max_tokens": 2048,
"n": 1,

14
test/autogen/oai/test_notebook.py → test/autogen/test_notebook.py
View File

@ -19,11 +19,12 @@ def run_notebook(input_nb, output_nb="executed_openai_notebook.ipynb", save=Fals
from nbconvert.preprocessors import CellExecutionError
try:
file_path = os.path.join(here, os.pardir, os.pardir, os.pardir, "notebook", input_nb)
nb_loc = os.path.join(here, os.pardir, os.pardir, "notebook")
file_path = os.path.join(nb_loc, input_nb)
with open(file_path) as nb_file:
nb = nbformat.read(nb_file, as_version=4)
preprocessor = ExecutePreprocessor(timeout=4800, kernel_name="python3")
preprocessor.preprocess(nb, {"metadata": {"path": here}})
preprocessor.preprocess(nb, {"metadata": {"path": nb_loc}})
output_file_name = "executed_openai_notebook_output.txt"
output_file = os.path.join(here, output_file_name)
@ -84,7 +85,8 @@ def test_autogen_chatgpt_gpt4(save=False):
if __name__ == "__main__":
test_autogen_chatgpt_gpt4(save=True)
test_autogen_openai_completion(save=True)
test_autogen_agent_MathChat(save=True)
test_autogen_agent_function_call(save=True)
test_autogen_agent_auto_feedback_from_code(save=True)
# test_autogen_chatgpt_gpt4(save=True)
# test_autogen_openai_completion(save=True)
# test_autogen_agent_MathChat(save=True)
# test_autogen_agent_function_call(save=True)

2
website/docs/Contribute.md
View File

@ -53,7 +53,7 @@ print(flaml.__version__)
## Becoming a Reviewer
There is currently no formal reviewer solicitation process. Current reviewers identify reviewers from active contributors. If you are willing to become a reviewer, you are welcome to let us know on gitter.
There is currently no formal reviewer solicitation process. Current reviewers identify reviewers from active contributors. If you are willing to become a reviewer, you are welcome to let us know on discord.
## Developing