Commit Graph

344 Commits

Author SHA1 Message Date
Chi Wang 99f2f82fa3 Merge remote-tracking branch 'upstream/autogen-links' into code 2023-09-16 16:34:04 +00:00
Chi Wang bc4473fe8a cleanup 2023-09-16 10:57:57 +00:00
Chi Wang 4886cb5689
Rename Responsive -> Conversable (#1202)
* responsive -> conversable

* preview

* rename

* register reply

* rename and version

* bump version to 2.1.0

* notebook

* bug fix
2023-09-12 00:07:35 +00:00
Chi Wang 599731cb22
rename human to user_proxy (#1215)
* rename human to user_proxy

* notebook update and bug fix
2023-09-11 14:33:47 +00:00
Qingyun Wu 3bfc5f21d2 Merge remote-tracking branch 'origin/main' into autogen-links 2023-09-11 09:39:55 -04:00
Chi Wang 0cb79dfdff
group chat for visualization (#1213)
* group chat for visualization

* show figure

* webpage update

* link update

* example 2

* example 2

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-09-10 23:20:45 +00:00
Qingyun Wu faee51ca9c update links 2023-09-08 10:07:19 -04:00
Chi Wang 93b9e09166
admin takeover in group chat (#1209)
* admin takeover in group chat

* comments

* add comments
2023-09-07 02:17:53 +00:00
Chi Wang 5f9b514be7
suffix in model name (#1206)
* suffix in model name

* bump version to 2.0.3
2023-09-04 02:32:51 +00:00
Chi Wang 44932712c4
Prompt improvement (#1203)
* prompt improvement

* image None for unsupported lang

* notebook update

* prompt improvement
2023-08-30 00:54:09 +00:00
Chi Wang 57a2bea95a
prompt improvement (#1188)
* test

* chart

* plan

* separate

* bump version to 2.0.1

* explain plan and code/lang

* notebook update

* notebook update

* typo

* plan

* match can be None
2023-08-27 22:16:52 +00:00
Chi Wang 7ab4d114d7
silent; code_execution_config; exit; version (#1179)
* silent; code_execution_config; exit; version

* url

* url

* readme

* preview

* doc

* url

* endpoints

* timeout

* chess

* Fix retrieve chat

* config

* mathchat

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-08-14 07:09:45 +00:00
Li Jiang 700ff05874
Add RetrieveChat (#1158)
* Add RetrieveChat notebook, RetrieveAssistantAgent and RetrieveUserProxyAgent

* Update according to comments

* Add output

* Add tests, merge main, address comments

* Fix tests

* Merge main

* Remove unnecessary code

* Update test

* Update notebook, some functions

* Fix print issue

* Update notebook

* Update notebook

* Update notebook

* Improve retrieve utils and update notebook

* Update vector db creation method

* Update notebook

* Update notebook

* Add terminate if no more context

* Update prompt and notebook, add example for update context

* Update results

* Update results

* Update results of update context

* Fix typo

* Add table of contents

* Update table of contents
2023-08-13 12:51:54 +00:00
Chi Wang c44d2f4a01
support async in agents (#1178)
* Make auto reply method pluggable

* support async

* async

* allow richer trigger types

* test list

* rename key
2023-08-08 01:34:47 +00:00
Chi Wang a603e6dddc
Make auto reply method pluggable (#1177)
* Make auto reply method pluggable

* allow richer trigger types

* test list
2023-08-07 18:41:58 +00:00
Chi Wang 2208dfb79e
Improve auto reply registration (#1170)
* Improve auto reply registration

* object key

* fix test error

* bug fix in math user proxy agent

* allow send/receive without reply

* reset -> stop
2023-08-04 14:26:58 +00:00
Chi Wang 45b7d908e4
consecutive auto reply, history, template, group chat, class-specific reply (#1165)
* max consecutive auto reply

* chess notebook

* link to notebook

* clear history

* filter

* **context -> context

* format str template

* groupchat

* register class specific reply

* groupchat notebook

* move human reply into generate_reply

* arg in config

* colab link

* remove room

* rename
2023-08-03 09:17:20 +00:00
Chi Wang c48babd02f
raise error when msg is invalid; fix docstr; improve ResponsiveAgent; update doc and packaging; capture ipython output; find code blocks with llm when regex fails. (#1154)
* autogen.agent -> autogen.agentchat

* bug fix in portfolio

* notebook

* timeout

* timeout

* infer lang; close #1150

* timeout

* message context

* context handling

* add sender to generate_reply

* clean up the receive function

* move mathchat to contrib

* contrib

* last_message

* Add OptiGuide: agent and notebook

* Optiguide notebook: add figures and URL
1. figures and code points to remote URL
2. simplify the prompt for the interpreter, because
all information is already in the chat history.

* Update name: Agent -> GenericAgent

* Update notebook

* Rename: GenericAgent -> ResponsiveAgent

* Rebase to autogen.agentchat

* OptiGuide: Comment, sytle, and notebook updates

* simplify optiguide

* raise error when msg is invalid; fix docstr

* allow return None for generate_reply()

* update_system_message

* test update_system_message

* simplify optiguide

* simplify optiguide

* simplify optiguide

* simplify optiguide

* move test

* add test and fix bug

* doc update

* doc update

* doc update

* color

* optiguide

* prompt

* test danger case

* packaging

* docker

* remove path in traceback

* capture ipython output

* simplify

* find code blocks with llm

* find code with llm

* order

* order

* fix bug in context handling

* print executing msg

* print executing msg

* test find code

* test find code

* disable find_code

* default_auto_reply

* default auto reply

* remove optiguide

* remove -e

---------

Co-authored-by: Beibin Li <beibin79@gmail.com>
2023-08-01 02:22:30 +00:00
Chi Wang 7ddb171cd9
autogen.agent -> autogen.agentchat (#1148)
* autogen.agent -> autogen.agentchat

* bug fix in portfolio

* notebook

* timeout

* timeout

* infer lang; close #1150

* timeout

* message context

* context handling

* add sender to generate_reply

* clean up the receive function

* move mathchat to contrib

* contrib

* last_message
2023-07-29 04:17:51 +00:00
Chi Wang 3e7aac6e8b
unify auto_reply; bug fix in UserProxyAgent; reorg agent hierarchy (#1142)
* simplify the initiation of chat

* version update

* include openai

* completion

* load config list from json

* initiate_chat

* oai config list

* oai config list

* config list

* config_list

* raise_error

* retry_time

* raise condition

* oai config list

* catch file not found

* catch openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* close #1139

* use property

* termination msg

* AIUserProxyAgent

* smaller dev container

* update notebooks

* match

* document code execution and AIUserProxyAgent

* gpt 3.5 config list

* rate limit

* variable visibility

* remove unnecessary import

* quote

* notebook comments

* remove mathchat from init import

* two users

* import location

* expose config

* return str not tuple

* rate limit

* ipython user proxy

* message

* None result

* rate limit

* rate limit

* rate limit

* rate limit

* make auto_reply a common method for all agents

* abs path

* refactor and doc

* set mathchat_termination

* code format

* modified

* emove import

* code quality

* sender -> messages

* system message

* clean agent hierarchy

* dict check

* invalid oai msg

* return

* openml error

* docstr

---------

Co-authored-by: kevin666aa <yrwu000627@gmail.com>
2023-07-25 23:46:11 +00:00
Chi Wang 2406e69496
Json config list, agent refactoring and new notebooks (#1133)
* simplify the initiation of chat

* version update

* include openai

* completion

* load config list from json

* initiate_chat

* oai config list

* oai config list

* config list

* config_list

* raise_error

* retry_time

* raise condition

* oai config list

* catch file not found

* catch openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* handle openml error

* close #1139

* use property

* termination msg

* AIUserProxyAgent

* smaller dev container

* update notebooks

* match

* document code execution and AIUserProxyAgent

* gpt 3.5 config list

* rate limit

* variable visibility

* remove unnecessary import

* quote

* notebook comments

* remove mathchat from init import

* two users

* import location

* expose config

* return str not tuple

* rate limit

* ipython user proxy

* message

* None result

* rate limit

* rate limit

* rate limit

* rate limit
2023-07-23 13:23:09 +00:00
Chi Wang 16f0fcd6f8
simplify the initiation of chat (#1131)
* simplify the initiation of chat

* version update

* include openai

* completion
2023-07-18 03:40:41 +00:00
Chi Wang 297a1ea9e0
Update assistant agent (#1121)
* update colab link

* typo

* upload file instruction

* update system message and notebooks

* update notebooks

* notebook test

* aoai api version and exclusion

* gpt-3.5-turbo

* dict check

* change model for test

* endpoints, cache_path and func description update

* model list

* gitter -> discord
2023-07-14 22:52:45 +00:00
Chi Wang 3527e8a14d
update colab link (#1118)
* update colab link

* typo

* upload file instruction
2023-07-11 17:45:04 +00:00
Chi Wang 784c4c4f86
Try to prevent the default AssistantAgent from asking users to modify the code (#1114)
* inform assistant agent:
user proxy agent can't modify code

* update notebooks

* configure timeout in UserProxyAgent

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-07-10 16:07:48 +00:00
Yiran Wu 255faf97e2
Add Funccall notebook and document (#1110)
* add funccall example and doc

* revise to comments

* Update website/docs/Use-Cases/Auto-Generation.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* revise

* update

* minor update

* add test notebook

* update

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-07-10 04:56:07 +00:00
Chi Wang 9c09090bf9
update notebook with new models (#1112)
* rerun notebook with new models

* planning agent

* link to notebook
2023-07-09 05:25:43 +00:00
Yiran Wu ca10b286cc
Support function_call in `autogen/agent` (#1091)
* update funccall

* code format

* update to comments

* update notebook

* remove test for py3.7

* allow funccall to class functions

* add test and clean up notebook

* revise notebook and test

* update

* update mathagent

* Update flaml/autogen/agent/agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* revise to comments

* revise function call design, notebook and test. add doc

* code format

* ad message_to_dict function

* update mathproxyagent

* revise docstr

* update

* Update flaml/autogen/agent/math_user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/autogen/agent/math_user_proxy_agent.py

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>

* simply funccall in userproxyagent, rewind auto-gen.md, revise to comments

* code format

* update

* remove notebook for another pr

* revise oai_conversation part  in agent, revise function exec in user_proxy_agent

* update test_funccall

* update

* update

* fix pydantic version

* Update test/autogen/test_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* fix bug

* fix bug

* update

* update is_termination_msg to accept dict

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-07-05 22:08:44 +00:00
Chi Wang 4f1dfe6676
doc update (#1089)
* doc update

* add link to mathchat notebook

* use_docker property

* function name

* version update

---------

Co-authored-by: kevin666aa <yrwu000627@gmail.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-07-04 20:29:32 +00:00
Yiran Wu 8236533faf
update math notebook (#1095)
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-07-02 15:26:30 +00:00
Yiran Wu e3ca95bf8a
An agent implementation of MathChat (#1090)
* mathcaht implementation

* code forrmat

* update readme

* update openai.yml

* update openai.yml

* update openai.yml
2023-06-25 13:49:34 +00:00
Chi Wang a30d198530
Fix documentation (#1075)
* Fix indentation in documentation

* newline

* version
2023-06-11 01:03:49 +00:00
Chi Wang 5387a0a607
Agent notebook example with human feedback; Support shell command and multiple code blocks; Improve the system message for assistant agent; Improve utility functions for config lists; reuse docker image (#1056)
* add agent notebook and documentation

* fix bug

* set flush to True when printing msg in agent

* add a math problem in agent notebook

* remove

* header

* improve notebook doc

* notebook update

* improve notebook example

* improve doc

* agent notebook example with user feedback

* log

* log

* improve notebook doc

* improve print

* doc

* human_input_mode

* human_input_mode str

* indent

* indent

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* shell command and multiple code blocks

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* coding agent

* math notebook

* renaming and doc format

* typo

* infer lang

* sh

* docker

* docker

* reset consecutive autoreply counter

* fix explanation

* paper talk

* human feedback

* web info

* rename test

* config list explanation

* link to blogpost

* installation

* homepage features

* features

* features

* rename agent

* remove notebook

* notebook test

* docker command

* notebook update

* lang -> cmd

* notebook

* make it work for gpt-3.5

* return full log

* quote

* docker

* docker

* docker

* docker

* docker

* docker image list

* notebook

* notebook

* use_docker

* use_docker

* use_docker

* doc

* agent

* doc

* abs path

* pandas

* docker

* reuse docker image

* context window

* news

* print format

* pyspark version in py3.8

* pyspark in py3.8

* pyspark and ray

* quote

* pyspark

* pyspark

* pyspark

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-06-09 18:40:04 +00:00
Qingyun Wu fed28e700b
add agent notebook and documentation (#1052)
* add agent notebook and documentation

* fix bug

* set flush to True when printing msg in agent

* add a math problem in agent notebook

* remove

* header

* improve notebook doc

* notebook update

* improve notebook example

* improve doc

* improve notebook doc

* improve print

* doc

* human_input_mode

* human_input_mode str

* indent

* indent

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* renaming and doc format

* typo

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-05-28 03:17:23 +00:00
Qingyun Wu 3e6e834bbb
remove redundant doc and add tutorial (#1004)
* remove redundant doc and add tutorial

* add demos for pydata2023

* Update pydata23 docs

* remove redundant notebooks

* Move tutorial notebooks to notebook folder

* update readme and notebook links

* update notebook links

* update links

* update readme

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-05-27 02:59:51 +00:00
Chi Wang a0b318b12e
create an automl option to remove unnecessary dependency for autogen and tune (#1007)
* version update post release v1.2.2

* automl option

* import pandas

* remove automl.utils

* default

* test

* type hint and version update

* dependency update

* link to open in colab

* use packging.version to close #725

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-05-24 23:55:04 +00:00
Chi Wang 9977a7aae1
Blogpost for adaptation in HumanEval (#1048)
* Blogpost for adaptation in HumanEval

* doc

* fix link

* fix link

* explain

* model

* interface

* link

* typo

* doc
2023-05-23 04:22:15 +00:00
Chi Wang e463146cb8
response filter (#1039)
* response filter

* rewrite implement based on the filter

* multi responses

* abs path

* code handling

* option to not use docker

* context

* eval_only -> raise_error

* notebook

* utils

* utils

* separate tests

* test

* test

* test

* test

* test

* test

* test

* test

* **config in test()

* test

* test

* filename
2023-05-21 22:22:29 +00:00
Chi Wang f097c20f86
version update post release v1.2.2 (#1005) 2023-04-25 04:48:17 +00:00
Chi Wang fa5ccea862
extract code from text; solve_problem; request_timeout in config; improve code (#999)
* extract code from text

* solve_problem; request_timeout in config

* improve

* move import statement

* improve code

* generate assertions

* constant

* configs for implement; voting

* doc

* execute code in docker

* success indicator of code executation in docker

* success indicator

* execute code

* strip n

* add cost in generate_code

* add docstr

* filename

* bytes

* check docker version

* print log

* python test

* remove api key address

* rename exit code

* success exit code

* datasets

* exit code

* recover openai tests

* cache and pattern match

* wait

* wait

* cache and test

* timeout test

* python image name and skip macos

* windows image

* docker images

* volume path and yaml

* win path -> posix

* extensions

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* skip windows

* path

* timeout in windows

* use_docker

* use_docker

* hot fix from #1000

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-04-23 11:50:29 +00:00
Susan Xueqing Liu 99bb0a8425
update nlp notebook (#940)
* update nlp notebook

* rerun

* rerun

* removing redundant in notebook

* remove redundant content in nlp notebook

* update notebook

* update plot

* update plot

* update plot

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
2023-04-17 17:29:36 +00:00
Chi Wang 82f0a4309d
autogen subpackage (#968)
* math utils in autogen

* cleanup

* code utils

* remove check function from code response

* comment out test

* GPT-4

* increase request timeout

* name

* logging and error handling

* better doc

* doc

* codegen optimized

* GPT series

* text

* no demo example

* math

* import openai

* import openai

* azure model name

* azure model name

* openai version

* generate assertion if necessary

* condition to generate assertions

* init region key

* rename

* comments about budget

* prompt

---------

Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-04-08 03:04:01 +00:00
Qingyun Wu 45641000c0
Adding a test function for OpenAI completion in flaml (#951)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* add support for chatgpt

* notebook

* newline at end of file

* chatgpt notebook

* ChatGPT in Azure

* doc

* math

* warning, timeout, log file name

* handle import error

* doc update; default value

* paper

* doc

* docstr

* eval_func

* add a test func in completion

* update notebook

* update math notebook

* improve notebok

* lint and handle exception

* flake8

* exception in test

* add agg_method

* NameError

* refactor

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add example

* merge files from oai_eval_test

* Revert "merge files from oai_eval_test"
This reverts commit 1e6a550f913bb94df6e9680934ccb7175d00702e.

* merge

* save results to notebook_output

* update version and cache

* update doc

* save nb cell results to file

* fix typo in model name

* code improvements

* improve docstr

* docstr

* docstr on the Returns of test

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-04-02 16:14:11 +00:00
Chi Wang 595f5a8025
gpt-4 support; openai workflow fix; model str; timeout; voting (#958)
* workflow; model str; timeout

* voting

* notebook

* pull request

* recover workflow

* voted answer

* aoai

* ignore None answer

* default config

* note

* gpt-4

* n=5

* cleanup

* config name

* introduction

* readme

* avoid None

* add output/ to gitignore

* openai version

* invalid var

* comment long running cells
2023-03-26 17:13:06 +00:00
Li Jiang 50334f2c52
Support spark dataframe as input dataset and spark models as estimators (#934)
* add basic support to Spark dataframe

add support to SynapseML LightGBM model

update to pyspark>=3.2.0 to leverage pandas_on_Spark API

* clean code, add TODOs

* add sample_train_data for pyspark.pandas dataframe, fix bugs

* improve some functions, fix bugs

* fix dict change size during iteration

* update model predict

* update LightGBM model, update test

* update SynapseML LightGBM params

* update synapseML and tests

* update TODOs

* Added support to roc_auc for spark models

* Added support to score of spark estimator

* Added test for automl score of spark estimator

* Added cv support to pyspark.pandas dataframe

* Update test, fix bugs

* Added tests

* Updated docs, tests, added a notebook

* Fix bugs in non-spark env

* Fix bugs and improve tests

* Fix uninstall pyspark

* Fix tests error

* Fix java.lang.OutOfMemoryError: Java heap space

* Fix test_performance

* Update test_sparkml to test_0sparkml to use the expected spark conf

* Remove unnecessary widgets in notebook

* Fix iloc java.lang.StackOverflowError

* fix pre-commit

* Added params check for spark dataframes

* Refactor code for train_test_split to a function

* Update train_test_split_pyspark

* Refactor if-else, remove unnecessary code

* Remove y from predict, remove mem control from n_iter compute

* Update workflow

* Improve _split_pyspark

* Fix test failure of too short training time

* Fix typos, improve docstrings

* Fix index errors of pandas_on_spark, add spark loss metric

* Fix typo of ndcgAtK

* Update NDCG metrics and tests

* Remove unuseful logger

* Use cache and count to ensure consistent indexes

* refactor for merge maain

* fix errors of refactor

* Updated SparkLightGBMEstimator and cache

* Updated config2params

* Remove unused import

* Fix unknown parameters

* Update default_estimator_list

* Add unit tests for spark metrics
2023-03-25 19:59:46 +00:00
Mark Harley 27b2712016
Extract task class from automl (#857)
* Refactor into automl subpackage

Moved some of the packages into an automl subpackage to tidy before the
task-based refactor. This is in response to discussions with the group
and a comment on the first task-based PR.

Only changes here are moving subpackages and modules into the new
automl, fixing imports to work with this structure and fixing some
dependencies in setup.py.

* Fix doc building post automl subpackage refactor

* Fix broken links in website post automl subpackage refactor

* Fix broken links in website post automl subpackage refactor

* Remove vw from test deps as this is breaking the build

* Move default back to the top-level

I'd moved this to automl as that's where it's used internally, but had
missed that this is actually part of the public interface so makes sense
to live where it was.

* Re-add top level modules with deprecation warnings

flaml.data, flaml.ml and flaml.model are re-added to the top level,
being re-exported from flaml.automl for backwards compatability. Adding
a deprecation warning so that we can have a planned removal later.

* Fix model.py line-endings

* WIP

* WIP - Notes below

Got to the point where the methods from AutoML are pulled to
GenericTask. Started removing private markers and removing the passing
of automl to these methods. Done with decide_split_type, started on
prepare_data. Need to do the others after

* Re-add generic_task

* Fix tests: add Task.__str__

* Fix tests: test for ray.ObjectRef

* Hotwire TS_Sklearn wrapper to fix test fail

* Remove unused data size field from Task

* Fix import for CLASSIFICATION in notebook

* Update flaml/automl/data.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Fix review comments

* Fix task -> str in custom learner constructor

* Remove unused CLASSIFICATION imports

* Hotwire TS_Sklearn wrapper to fix test fail by setting
optimizer_for_horizon == False

* Revert changes to the automl_classification and pin FLAML version

* Fix imports in reverted notebook

* Fix FLAML version in automl notebooks

* Fix ml.py line endings

* Fix CLASSIFICATION task import in automl_classification notebook

* Uncomment pip install in notebook and revert import

Not convinced this will work because of installing an older version of
the package into the environment in which we're running the tests, but
let's see.

* Revert c6a5dd1a0

* Revert "Revert c6a5dd1a0"

This reverts commit e55e35adea03993de87b23f092b14c6af623d487.

* Black format model.py

* Bump version to 1.1.2 in automl_xgboost

* Add docstrings to the Task ABC

* Fix import in custom_learner

* fix 'optimize_for_horizon' for ts_sklearn

* remove debugging print statements

* Check for is_forecast() before is_classification() in decide_split_type

* Attempt to fix formatting fail

* Another attempt to fix formatting fail

* And another attempt to fix formatting fail

* Add type annotations for task arg in signatures and docstrings

* Fix formatting

* Fix linting

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: EgorKraevTransferwise <egor.kraev@transferwise.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Kevin Chen <chenkevin.8787@gmail.com>
2023-03-11 02:39:08 +00:00
Chi Wang 169012f3e7
ChatGPT support (#942)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* add support for chatgpt

* notebook

* newline at end of file

* chatgpt notebook

* ChatGPT in Azure

* doc

* math

* warning, timeout, log file name

* handle import error

* doc update; default value

* paper

* doc

* docstr

* eval_func

* prompt and messages

* remove confusing words

* notebook name

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-03-10 19:35:36 +00:00
Shaokun 3a606930d1
Update doc and notebook (#946)
* update

* update doc

---------

Co-authored-by: “skzhang1” <“shaokunzhang529@gmail.com”>
2023-03-10 13:54:48 +00:00
Chi Wang 1ec77b58b4
improve max_valid_n and doc (#933)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* newline at end of file

* doc

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-03-05 16:40:57 +00:00
Susan Xueqing Liu 97928609ba
adding openai documentation (#938)
* adding openai documentation

* adding openai documentation

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* addressing comments

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - OpenAI.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* adding Azure

* rerun

* addressing comments about Azure

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-03-05 01:15:49 +00:00
Jirka Borovec 2ff1035733
precommit: end-of-file-fixer (#929)
* precommit: end-of-file-fixer

* exclude .gitignore

* apply

---------

Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
2023-02-28 16:27:14 +00:00
Chi Wang a3e368d2ca
minor change to notebook (#928) 2023-02-24 10:43:18 +00:00
Chi Wang 35ce9b79e8
azure oai (#920)
* azure oai

* price update in notebook

* text Davinci

* pytorch-lightning version

* trigger action in merge queue

* types

* doc check in mege group
2023-02-16 23:38:50 +00:00
Chi Wang 63d350d4c8
Openai (#905)
* add cost budget; move loc of make_dir

* support openai completion

* install pytest in workflow

* skip openai test

* test openai

* path for docs rebuild

* install datasets

* signal

* notebook

* notebook in workflow

* optional arguments and special params

* key -> k

* improve readability

* assumption

* optimize for model selection

* larger range of max_tokens

* notebook

* python package workflow

* skip on win
2023-02-05 20:13:08 -08:00
Qingyun Wu d0f7b30b61 update doc for research papers 2023-02-03 16:57:16 -08:00
Shaokun 7644265d1f
Merge branch 'main' into first_contribution 2023-01-29 10:48:16 -05:00
Chi Wang d46532efda
display data head in notebook; exclude None (#885) 2023-01-28 15:42:49 -08:00
Andrea W 297fc5ad7c annotate state, add cv notebook 2023-01-26 14:18:55 +00:00
Chi Wang 75e3454120
notebook test; spark warning message; reproducibility bug; sequential tuning stop condition (#869)
* notebook test

* add ipykernel, remove except

* only create dir if not empty

* Stop sequential tuning when result is None

* fix reproducibility of global search

* save gs seed

* use get to avoid KeyError

* test
2023-01-07 18:39:29 -08:00
Li Jiang da2cd7ca89
Add supporting using Spark as the backend of parallel training (#846)
* Added spark support for parallel training.

* Added tests and fixed a bug

* Added more tests and updated docs

* Updated setup.py and docs

* Added customize_learner and tests

* Update spark tests and setup.py

* Update docs and verbose

* Update logging, fix issue in cloud notebook

* Update github workflow for spark tests

* Update github workflow

* Remove hack of handling _choice_

* Allow for failures

* Fix tests, update docs

* Update setup.py

* Update Dockerfile for Spark

* Update tests, remove some warnings

* Add test for notebooks, update utils

* Add performance test for Spark

* Fix lru_cache maxsize

* Fix test failures on some platforms

* Fix coverage report failure

* resovle PR comments

* resovle PR comments 2nd round

* resovle PR comments 3rd round

* fix lint and rename test class

* resovle PR comments 4th round

* refactor customize_learner to broadcast_code
2022-12-23 08:18:49 -08:00
Jing Dong 3a194d047b fix checkpoint.value in the notebook and test 2022-12-19 09:22:16 -08:00
Chi Wang 92b79221b6
make performance test reproducible (#837)
* make performance test reproducible

* fix test error

* Doc update and disable logging

* document random_state and version

* remove hardcoded budget

* fix test error and dependency; close #777

* iloc
2022-12-06 10:13:39 -08:00
Shreyas 3b3b0bfa8e
roc_auc_weighted metric addition (#827)
* Pending changes exported from your codespace

* Update flaml/automl.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/automl.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update website/docs/Examples/Integrate - Scikit-learn Pipeline.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* added documentation for new metric

* Update flaml/ml.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* minor notebook changes

* Update Integrate - Scikit-learn Pipeline.md

* Update notebook/automl_classification.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update integrate_azureml.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-12-02 19:27:32 -08:00
Chi Wang 595af7a04f
install editable package in codespace (#826)
* install editable package in codespace

* fix test error in test_forecast

* fix test error in test_space

* openml version

* break tests; pre-commit

* skip on py10+win32

* install mlflow in test

* install mlflow in [test]

* skip test in windows

* import

* handle PermissionError

* skip test in windows

* skip test in windows

* skip test in windows

* skip test in windows

* remove ts_forecast_panel from doc
2022-11-27 14:22:54 -05:00
shreyas36 cb3402052f
added the models used for forecasting in documentation (#811) 2022-11-14 15:29:36 -05:00
Jing Dong 9a7a9ef1c6
Added a link to documentation webpage in notebook time_series_forcast (#791)
* added a link to documentation webpage in notebook time_series_forcast

* added a link to documentation webpage in notebook time_series_forcast

* Update notebook/automl_time_series_forecast.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* complete output

* added all cell output

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: zsk <shaokunzhang529@gmail.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-11-12 18:36:06 -08:00
Anonymous-submission-repo 2d18c49cdd update 2022-10-12 04:31:51 +00:00
Anonymous-submission-repo ff995d08d8 notebook 2022-10-10 13:37:01 +00:00
Xueqing Liu 2ead296676
updating nlp notebook (#693)
* updating nlp notebook
2022-08-22 07:20:48 -04:00
Chi Wang 47e034d203
LightGBM notebook update (#690)
* version update in notebook

* comment about optuna install

* monotone constraints
2022-08-20 07:43:06 -07:00
Xueqing Liu 6c7d37374d
updating nlp notebook (#683) 2022-08-15 18:09:08 -07:00
Kevin Chen 2e8e3937ef
update time series forecast notebook (#682)
* update forecasting with exogeneous variables example

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update forecasting with exogeneous variables example on website

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* rerun automl_time_series_forecast with new predict function for tft

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* correct spelling error

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2022-08-13 18:58:45 -07:00
Kevin Chen f718d18b5e
time series forecasting with panel datasets (#541)
* time series forecasting with panel datasets
- integrate Temporal Fusion Transformer as a learner based on pytorchforecasting

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py and test_forecast.py
- remove blank lines

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py to prevent errors

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py and data.py
- change forecast task name
- update documentation for fit() method

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py
- add performance test
- use 'fit_kwargs_by_estimator'

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* add time index function

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py performance test

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update data.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update data.py to prevent type error

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update setup.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update for pytorch forecasting tft on panel datasets

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* - rename estimator
- add 'gpu_per_trial' for tft estimator

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update test_forecast.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* include ts panel forecasting as an example

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl_time_series_forecast.ipynb

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update documentations

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* "weights_summary" argument deprecated and removed for pl.Trainer()

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py tft estimator prediction method

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update model.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update `fit_kwargs` documentation

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* update automl.py

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-08-12 08:39:22 -07:00
Xueqing Liu 5eb5d43d7f
Fix HPO evaluation bug (#645)
* fix eval automl metric bug on val_loss inconsistency

* updating starting point search space to continuous

* shortening notebok
2022-07-28 23:08:42 -04:00
Chi Wang e14e909af9
Feature names and importances (#621)
* feature names and importances

* None check

* StackingClassifier has no feature_importances_

* StackingClassifier has no feature_names_in_
2022-07-10 12:25:59 -07:00
Zvi Baratz 127e482ac8
Replaced !pip calls with %pip magic command. (#604)
Closes #603.
2022-06-23 18:45:42 -07:00
Zvi Baratz fefdb48ef3
Fix automl settings in scikit-learn pipeline integration example (#602)
* Added test directory and core file to gitignore.

Closes #601.

* Fixed pipeline fit parameters.

Closes #600.

* Reverted changes to gitignore.
2022-06-22 19:44:14 -07:00
Chi Wang c45741a67b
support latest xgboost version (#599)
* support latest xgboost version

* Update test_classification.py

* Update 

Exists problems when installing xgb1.6.1 in py3.6

* cleanup

* xgboost version

* remove time_budget_s in test

* remove redundancy

* stop support of python 3.6

Co-authored-by: zsk <shaokunzhang529@gmail.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-06-21 18:59:07 -07:00
Chi Wang 0642b6e7bb
init value type match (#575)
* init value type match

* bump version to 1.0.6

* add a note about flaml version in notebook

* add note about mismatched ITER_HP

* catch SSLError when accessing OpenML data

* catch errors in autovw test

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-06-09 08:11:15 -07:00
Chi Wang 619107edf5 install openml for notebook example 2022-06-05 22:52:20 -07:00
Chi Wang c9bac02ea4
add zeroshot notebook (#569)
* add zeroshot notebook
2022-06-04 12:00:58 -07:00
Xueqing Liu ca35fa969f
refactoring TransformersEstimator to support default and custom_hp (#511)
* refactoring TransformersEstimator to support default and custom_hp

* handling starting_points not in search space

* addressing starting point more than max_iter

* fixing upper < lower bug
2022-04-28 14:06:29 -04:00
Chi Wang e877de6414 use ffill in forecasting example 2022-04-01 09:23:23 -07:00
Qingyun Wu 2cdc08a75a update notebook and test 2022-03-30 19:11:10 -07:00
liususan091219 0bcf618fea fixing bug 2022-03-28 11:53:58 -07:00
Qingyun Wu 6c16e47e42
Bug fix and add documentation for metric_constraints (#498)
* metric constraint documentation

* update link

* update notebook

* fix a bug in adding 'time_total_s' to result

* use the default multiple factor from config file

* update notebook

* format

* improve test

* revise test budget for macos

* bug fix in adding time_total_s

* increase performance check budget

* revise test

* update notebook

* uncomment test

* remove redundancy

* clear output

* remove n_jobs

* remove constraint in notebook

* increase budget

* revise test

* add python version

* use getattr

* improve code robustness

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2022-03-26 21:11:45 -04:00
Xueqing Liu 5f97532986
adding evaluation (#495)
* adding automl.score

* fixing the metric name in train_with_config

* adding pickle after score

* fixing a bug in automl.pickle
2022-03-25 17:00:08 -04:00
Xueqing Liu af423463c3
fixing bug for ner (#463)
* fixing bug for ner

* removing global var

* adding class for trial counter

* adding notebook

* adding use_ray dict

* updating documentation for nlp
2022-03-20 22:03:02 -04:00
Kevin Chen f9eda0cc40
update documentation for time series forecasting (#472)
* update automl.py
- documentation update

* update test_forecast.py

* update model.py

* update automl_time_series_forecast.ipynb

* update time series forecast website examples

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2022-03-08 11:21:18 -08:00
Chi Wang 6095be039f
update regression metrics in notebooks (#454) 2022-02-15 13:17:43 -08:00
Chi Wang 569908fbe6
fix issues in logging, bug in space.py, constraint sign, and improve code coverage (#388)
* console log handler

* version update

* doc

* skippable steps

* notebook update

* constraint sign

* doc for constraints

* bug fix: define-by-run and unflatten_hierarchical

* const

* handle nested space in indexof()

* test grid search

* test suggestion

* model test

* >1 ckpts

* always increase iter count

* log total # iterations

* security patch

* make iter_per_learner consistent
2022-01-14 13:39:09 -08:00
Chi Wang 8602def1c4
logging (#371)
* query logged runs

* mlflow log when using ray

* key check for newer version of ray #363

* catch importerror

* log and load AutoML model

* retrain if necessary when ensemble fails
2022-01-02 21:37:19 -08:00
Chi Wang efd85b4c86
Deploy a new doc website (#338)
A new documentation website. And:

* add actions for doc

* update docstr

* installation instructions for doc dev

* unify README and Getting Started

* rename notebook

* doc about best_model_for_estimator #340

* docstr for keep_search_state #340

* DNN

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: Z.sk <shaokunzhang@psu.edu>
2021-12-16 17:11:33 -08:00
Chi Wang b773e2898f Update flaml_pytorch_cifar10.ipynb
training_function -> train_cifar
2021-12-07 15:17:15 -08:00
Qingyun Wu dd60dbc5eb
rename training_function (#327)
* rename training_function

* add docstr

* update docstr

* update docstr and comments

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2021-12-06 17:03:43 -05:00
Qingyun Wu 17b17d084f
tune api for schedulers (#322)
* revise api and tests

* rename prune_attr

* update finetune notebook

* add scheduler test and notebook

* update tune api for scheduler

* remove scheduler notebook

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* docstr

* fix imports

* clear notebook output

* fix ray import

* Update flaml/tune/tune.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* improve docstr

* Update flaml/searcher/blendsearch.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* remove redundant import

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-12-04 21:52:20 -05:00
Chi Wang c57954fbbd
include default value in rf search space (#317)
* include default value in rf search space

* init _mem_per_iter with -1

* bump version to 0.8.2

* docstr for search space's arguments
2021-12-03 09:15:21 -08:00
Chi Wang 85e21864ce
test -> val; docstr (#300)
* rename test -> val in custom metric function
* add an example in docstr
resolve #299
2021-11-22 22:17:29 -08:00
Chi Wang ea6d28d7bd
add max_depth to xgboost search space (#282)
* add max_depth to xgboost search space

* notebook update

* two learners for xgboost (max_depth or max_leaves)
2021-11-22 21:17:48 -08:00
Chi Wang 72caa2172d
model_history, ITER_HP, settings in AutoML(), checkpoint bug fix (#283)
if save_best_model_per_estimator is False and retrain_final is True, unfit the model after evaluation in HPO.
retrain if using ray.
update ITER_HP in config after a trial is finished.
change prophet logging level.
example and notebook update.
allow settings to be passed to AutoML constructor. Are you planning to add multi-output-regression capability to FLAML #192 Is multi-tasking allowed? #277 can pass the auotml setting to the constructor instead of requiring a derived class.
remove model_history.
checkpoint bug fix.

* model_history meaning save_best_model_per_estimator

* ITER_HP

* example update

* prophet logging level

* comment update in forecast notebook

* print format improvement

* allow settings to be passed to AutoML constructor

* checkpoint bug fix

* time limit for autohf regression test

* skip slow test on macos

* cleanup before del
2021-11-18 09:39:45 -08:00
Xueqing Liu 42de3075e9
Make NLP tasks available from AutoML.fit() (#210)
Sequence classification and regression: "seq-classification" and "seq-regression"

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-11-16 11:06:20 -08:00
Chi Wang 59083fbdcb
example update (#281)
* example update

* bump version to 0.7.2

* notebook update
2021-11-12 22:29:33 -08:00
Chi Wang 92ebd1f7f9
when max_iter=1, skip search only if retrain_final (#280)
* when max_iter=1, skip search only if retrain_final

* remove nlp
redesign in #210

* minor change in readme example
2021-11-09 21:51:23 -08:00
Chi Wang 549a0dfb53
limit time and memory consumption (#264)
* limit time and memory

* separate tests

* lrl1 can't be limited by limit_resource

* free memory when possible

* passthrough=False when ensemble fails;
retrain when trained_estimator is None

* use callback to for resource limit

* handle lower version of xgb with no callback

* free mem ratio

* reduce verbosity

* retrain_final when max_iter==1

* remove trained_estimator from result

* model_history

* wheel

* retrain time as best_config_train_time

* ci: libomp version for xgboost on macos

* limit_resource not working in windows

* test pickle load

* mute forecaster

* notebook update

* check hard

* preventive callback

* add use_ray
2021-11-03 19:08:23 -07:00
Kevin Chen 519bfc2a18
Integrate multivariate time series forecasting (#254)
* Integrate multivariate time series forecasting, now supports
continuous and categorical variables

- update data.py to transform time series data
- update search space
- update documentations to reflect changes
- update test_forecast.py
- rename 'forecast' task to 'ts_forecast' task

* update automl.py and test_forecast.py

* update forecast notebook

* update README.md and setup.py

* update ml.py and test_forecast.py

- make "ds" and "y" constant variables

* replace constants with constant variables

* bump version to 0.7.0

* update setup.py
- support 'forecast' and 'ts_forecast'

* update automl.py and data.py
- support 'forecast' and 'ts_forecast' tasks
2021-10-30 09:48:57 -07:00
Chi Wang 524f22bcc5
fix bug in hierarchical search space (#248); optional dependency on lgbm and xgb (#250)
* close #249

* admissible region

* best_config can be None

* optional dependency on lgbm and xgb
resolve #252
2021-10-15 21:36:42 -07:00
Chi Wang f48ca2618f
warning -> info for low cost partial config (#231)
* warning -> info for low cost partial config
#195, #110

* when n_estimators < 0, use trained_estimator's

* log debug info

* test random seed

* remove "objective"; avoid ZeroDivisionError

* hp config to estimator params

* check type of searcher

* default n_jobs

* try import

* Update searchalgo_auto.py

* CLASSIFICATION

* auto_augment flag

* min_sample_size

* make catboost optional
2021-10-08 16:09:43 -07:00
Chi Wang ea6c6ded2f
clean up forecast notebook (#202) 2021-09-13 21:16:42 -07:00
Chi Wang 71219df6c6
notebook example (#189)
* config in result

* value can be float

* pytorch notebook example

* docker, pre-commit

* max_failure (#192); early_stop

* extend starting_points (#196)

Co-authored-by: Chi Wang (MSR) <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qw2ky@virginia.edu>
2021-09-10 16:39:16 -07:00
Chi Wang e46573a01d
warmstart blendsearch (#186)
* increase test coverage

* use define by run only when needed

* warmstart bs

* classification -> binary, multi

* warm start with evaluated rewards

* data transformer; resource attr for gs

* BlendSearchTuner bug fix and unittest

* bug fix

* docstr and import

* task type
2021-09-04 01:42:21 -07:00
Kevin Chen ec34427ca8
Forecast v2 (#182)
* update flaml_forecast.ipynb

* visualize predictions for comparison

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2021-09-01 18:06:37 -07:00
Chi Wang 6ab0730793
remove catboost training dir; ensemble api; blendsearch for hierarchical space; ranking task; forecast improvement (#178)
* remove catboost training dir

* close #48

* bs for hierarchical space. close #85

* retrain for hierarchical space

* clean ml (#180)

Co-authored-by: Qingyun Wu <qxw5138@psu.edu>

* support ranking task

* examples

* cv shuffle

* forecast api and implementation cleaner

* period constraints

* delete groups after fit
2021-09-01 16:25:04 -07:00
Qingyun Wu a229a6112a
Support parallel and add random search (#167)
* non hashable value out of signature

* parallel trials

* add random in _search_parallel

* fix bug in retraining

* check memory constraint before training

* retrain_full

* log custom metric

* retraining budget check

* sample size check before retrain

* remove 'time2eval' from result

* report 'total_search_time' in result

* rename total_search_time to wall_clock_time

* rename train_loss boolean to log_training_metric

* set default train_loss to None

* exclude oom result

* log retrained model

* no subsample

* doc str

* notebook

* predicted value is NaN for sarimax

* version

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qxw5138@psu.edu>
2021-08-23 16:36:51 -07:00
Kevin Chen 3d0a3d26a2
Forecast (#162)
* added 'forecast' task with estimators ['fbprophet', 'arima', 'sarimax']

* update setup.py

* add TimeSeriesSplit to 'regression' and 'classification' task

* add 'time' split_type for 'classification' and 'regression' task

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* feature importance

* variable name

* Update test/test_split.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update test/test_forecast.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* prophet installation fail in windows

* upload flaml_forecast.ipynb

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2021-08-23 13:26:46 -07:00
すずまる 6270353458
support ROC and AUC for multi-class classification (#170)
* support ROC and AUC for multi-class classification

* add a test case to cover ROC and AUC for multi-class classification
2021-08-22 15:16:10 -07:00
Naga Budigam 2fb888e64e
Tutorial on using AutoML in sklearn pipeline (#157)
* tutorial on using AutoML in sklearn pipeline
2021-08-11 17:46:46 -07:00
Xueqing Liu eeaf5b5963
space -> main (#148)
* subspace in flow2

* search space and trainable from AutoML

* experimental features: multivariate TPE, grouping, add_evaluated_points

* test experimental features

* readme

* define by run

* set time_budget_s for bs

Co-authored-by: liususan091219 <Xqq630517>

* version

* acl

* test define_by_run_func

* size

* constraints

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-08-02 16:10:26 -07:00
Qingyun Wu 58c0ec959d
Update readme for flaml.tune (#137)
* add time_budget_s for bs in readme

* version update

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-07-24 17:10:43 -07:00
Chi Wang b3bb00966d
coverage (#135)
* coverage

* readme

* timeout
2021-07-20 17:00:44 -07:00
Qingyun Wu b04b00dc9d
V0.5.6 (#128)
* recover ConcurrencyLimiter

* cost attribute

* update notebooks

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-07-06 08:32:20 -07:00
Qingyun Wu a291abfab9
Cha cha (#127)
* unordered categorical

* allow cost attribute to be None

* tensorboardX version

* quote

* cfo cat

* trunc

* Update version.py

* incumbent is normalized

* python 3.9

* remove ConcurrencyLimiter

* seed

* estimator

* update autovw notebook

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-07-05 18:17:26 -07:00
Xueqing Liu da9fc51e62
update notebook 2021-06-22 22:20:16 -07:00
Xueqing Liu 54fbf09322
fixing jupyter notebook (#118) 2021-06-21 19:08:24 -07:00
Xueqing Liu cd4be9c0e5
add notebook (#109)
* added support for transformers==3.4.0

* updating error message

* adding arxiv
2021-06-17 21:42:26 -07:00
Xueqing Liu 926589bdda
exception, coverage for autohf (#106)
* increase coverage

* fixing exception messages

* fixing import
2021-06-14 14:11:40 -07:00
Chi Wang c26720c299
api doc for chacha (#105)
* api doc for chacha

* update params

* link to paper

* update dataset id

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-06-11 10:25:45 -07:00
Xueqing Liu a4049ad9b6
autohf (#43)
automate huggingface transformer
2021-06-09 08:37:03 -07:00
Qingyun Wu c0be2eefe8
typo (#101) 2021-06-04 11:13:28 -07:00
Qingyun Wu c8da829e4d
ChaCha documentation (#100)
* update readme

* naming
2021-06-04 10:52:52 -07:00
Qingyun Wu 0d3a0bfab6
Add ChaCha (#92)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* prevent divide by zero

* test roberta

* BlendSearchTuner

* sync

* version number

* update gitignore

* delta time

* reindex columns when dropping int-indexed columns

* add seed

* add seed in Args

* merge

* init upload of ChaCha

* remove redundancy

* add back catboost

* improve AutoVW API

* set min_resource_lease in VWOnlineTrial

* docstr

* rename

* docstr

* add docstr

* improve API and documentation

* fix name

* docstr

* naming

* remove max_resource in scheduler

* add TODO in flow2

* remove redundancy in rearcher

* add input type

* adapt code from ray.tune

* move files

* naming

* documentation

* fix import error

* fix format issues

* remove cb in worse than test

* improve _generate_all_comb

* remove ray tune

* naming

* VowpalWabbitTrial

* import error

* import error

* merge test code

* scheduler import

* fix import

* remove

* import, minor bug and version

* Float or Categorical

* fix default

* add test_autovw.py

* add vowpalwabbit and openml

* lint

* reorg

* lint

* indent

* add autovw notebook

* update notebook

* update log msg and autovw notebook

* update autovw notebook

* update autovw notebook

* add available strings for model_select_policy

* string for metric

* Update vw format in flaml/onlineml/trial.py

Co-authored-by: olgavrou <olgavrou@gmail.com>

* make init_config optional

* add _setup_trial_runner and update notebook

* space

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
Co-authored-by: olgavrou <olgavrou@gmail.com>
2021-06-02 22:08:24 -04:00
Chi Wang 3083229e40
Notebook (#87)
* notebook update
2021-05-07 19:50:50 -07:00
Qingyun Wu 06045703bf
Lgbm w customized obj (#64)
* add customized lgbm learner

* add comments

* fix format issue

* format

* OpenMLError

* add test

* add notebook

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2021-04-10 21:14:28 -04:00
Chi Wang 97a7c114ee
Issue58 (#59)
* iter per learner

* code cleanup
2021-04-08 09:29:55 -07:00
Chi Wang b7a91e0385
V0.3.0 (#55)
* flaml v0.3

* low cost partial config
2021-04-06 11:37:52 -07:00
Chi Wang 4a8110c87b
pickle the AutoML object (#37)
* pickle the AutoML object

* get best model per estimator

* test deberta

* stateless API

* Add Gitter badge (#41)

* prevent divide by zero

* test roberta

* BlendSearchTuner

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: The Gitter Badger <badger@gitter.im>
2021-03-16 22:13:35 -07:00
Chi Wang 1560a6e52a
V0.2.7 (#35)
* bug fix

* admissible region

* use CFO's init point as backup

* step lower bound

* test electra
2021-03-05 23:39:14 -08:00
Chi Wang 7bd231e497
v0.2.6 (#32)
* xgboost notebook

* finetuning notebook

* finetuning test

* experimental nni support

* support nested search space

* log file name

* record training_iteration

* eps

* reset times

* std set to default step size if 0
2021-02-28 12:43:43 -08:00
Chi Wang 6ff0ed434b
v0.2.5 (#30)
* test distillbert

* import check

* complete partial config

* None check

* init config is not suggested by bo

* badge

* notebook for lightgbm
2021-02-22 22:10:41 -08:00
Chi Wang (MSR) 79a851e408 step curve 2021-02-17 14:03:19 -08:00
Chi Wang (MSR) bd16eeee69 sample_weight; dependency; notebook 2021-02-13 10:43:11 -08:00
Chi Wang (MSR) d659079a5d transformer example 2021-02-06 16:24:38 -08:00
Chi Wang 776aa55189
V0.2.2 (#19)
* v0.2.2

separate the HPO part into the module flaml.tune
enhanced implementation of FLOW^2, CFO and BlendSearch
support parallel tuning using ray tune
add support for sample_weight and generic fit arguments
enable mlflow logging

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: qingyun-wu <qw2ky@virginia.edu>
2021-02-05 21:41:14 -08:00
Chi Wang cb5ce4e3a6
v0.1.3 Set default logging level to INFO (#14)
* set default logging level to INFO

* remove unnecessary import

* API future compatibility

* add test for customized learner

* test dependency

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
2020-12-15 08:10:43 -08:00
Chi Wang (MSR) 492990655d v0.1.0 2020-12-04 09:40:27 -08:00