Commit Graph

58 Commits

Author SHA1 Message Date
Yiran Wu e3ca95bf8a
An agent implementation of MathChat (#1090)
* mathcaht implementation

* code forrmat

* update readme

* update openai.yml

* update openai.yml

* update openai.yml
2023-06-25 13:49:34 +00:00
Li Jiang 3874a429cf
fix workflow (#1071) 2023-06-14 06:44:30 +00:00
Chi Wang 5387a0a607
Agent notebook example with human feedback; Support shell command and multiple code blocks; Improve the system message for assistant agent; Improve utility functions for config lists; reuse docker image (#1056)
* add agent notebook and documentation

* fix bug

* set flush to True when printing msg in agent

* add a math problem in agent notebook

* remove

* header

* improve notebook doc

* notebook update

* improve notebook example

* improve doc

* agent notebook example with user feedback

* log

* log

* improve notebook doc

* improve print

* doc

* human_input_mode

* human_input_mode str

* indent

* indent

* Update flaml/autogen/agent/user_proxy_agent.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* shell command and multiple code blocks

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update notebook/autogen_agent.ipynb

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* coding agent

* math notebook

* renaming and doc format

* typo

* infer lang

* sh

* docker

* docker

* reset consecutive autoreply counter

* fix explanation

* paper talk

* human feedback

* web info

* rename test

* config list explanation

* link to blogpost

* installation

* homepage features

* features

* features

* rename agent

* remove notebook

* notebook test

* docker command

* notebook update

* lang -> cmd

* notebook

* make it work for gpt-3.5

* return full log

* quote

* docker

* docker

* docker

* docker

* docker

* docker image list

* notebook

* notebook

* use_docker

* use_docker

* use_docker

* doc

* agent

* doc

* abs path

* pandas

* docker

* reuse docker image

* context window

* news

* print format

* pyspark version in py3.8

* pyspark in py3.8

* pyspark and ray

* quote

* pyspark

* pyspark

* pyspark

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-06-09 18:40:04 +00:00
Chi Wang e463146cb8
response filter (#1039)
* response filter

* rewrite implement based on the filter

* multi responses

* abs path

* code handling

* option to not use docker

* context

* eval_only -> raise_error

* notebook

* utils

* utils

* separate tests

* test

* test

* test

* test

* test

* test

* test

* test

* **config in test()

* test

* test

* filename
2023-05-21 22:22:29 +00:00
Li Jiang 7de4eb347d
Fix PULL_REQUEST_TEMPLATE and improve test by removing unnecessary environment variable (#1043)
* Improve test by removing unnecessary environment variable

* Fix PULL_REQUEST_TEMPLATE

* Hide pre-commit check

* remove the checkbox for pre-commit

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2023-05-19 20:05:14 +00:00
Jirka Borovec 73bb6e7667
pyproject.toml & switch to Ruff (#976)
* unify config to pyproject.toml
replace flake8 with Ruff

* drop configs

* update

* fixing

* Apply suggestions from code review

Co-authored-by: Zvi Baratz <z.baratz@gmail.com>

* setup

* ci

* pr template

* reword

---------

Co-authored-by: Zvi Baratz <z.baratz@gmail.com>
Co-authored-by: Li Jiang <lijiang1@microsoft.com>
2023-04-28 01:54:55 +00:00
Chi Wang fa5ccea862
extract code from text; solve_problem; request_timeout in config; improve code (#999)
* extract code from text

* solve_problem; request_timeout in config

* improve

* move import statement

* improve code

* generate assertions

* constant

* configs for implement; voting

* doc

* execute code in docker

* success indicator of code executation in docker

* success indicator

* execute code

* strip n

* add cost in generate_code

* add docstr

* filename

* bytes

* check docker version

* print log

* python test

* remove api key address

* rename exit code

* success exit code

* datasets

* exit code

* recover openai tests

* cache and pattern match

* wait

* wait

* cache and test

* timeout test

* python image name and skip macos

* windows image

* docker images

* volume path and yaml

* win path -> posix

* extensions

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* path

* skip windows

* path

* timeout in windows

* use_docker

* use_docker

* hot fix from #1000

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-04-23 11:50:29 +00:00
Jirka Borovec f404b66a4c
ci: concurrency (#931)
* ci: concurrency

* no fail-fast

* Apply suggestions from code review

---------

Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
2023-04-04 04:44:23 +00:00
Qingyun Wu 45641000c0
Adding a test function for OpenAI completion in flaml (#951)
* improve max_valid_n and doc

* Update README.md

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* add support for chatgpt

* notebook

* newline at end of file

* chatgpt notebook

* ChatGPT in Azure

* doc

* math

* warning, timeout, log file name

* handle import error

* doc update; default value

* paper

* doc

* docstr

* eval_func

* add a test func in completion

* update notebook

* update math notebook

* improve notebok

* lint and handle exception

* flake8

* exception in test

* add agg_method

* NameError

* refactor

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update flaml/integrations/oai/completion.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* add example

* merge files from oai_eval_test

* Revert "merge files from oai_eval_test"
This reverts commit 1e6a550f913bb94df6e9680934ccb7175d00702e.

* merge

* save results to notebook_output

* update version and cache

* update doc

* save nb cell results to file

* fix typo in model name

* code improvements

* improve docstr

* docstr

* docstr on the Returns of test

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Susan Xueqing Liu <liususan091219@users.noreply.github.com>
2023-04-02 16:14:11 +00:00
Chi Wang 595f5a8025
gpt-4 support; openai workflow fix; model str; timeout; voting (#958)
* workflow; model str; timeout

* voting

* notebook

* pull request

* recover workflow

* voted answer

* aoai

* ignore None answer

* default config

* note

* gpt-4

* n=5

* cleanup

* config name

* introduction

* readme

* avoid None

* add output/ to gitignore

* openai version

* invalid var

* comment long running cells
2023-03-26 17:13:06 +00:00
Li Jiang 50334f2c52
Support spark dataframe as input dataset and spark models as estimators (#934)
* add basic support to Spark dataframe

add support to SynapseML LightGBM model

update to pyspark>=3.2.0 to leverage pandas_on_Spark API

* clean code, add TODOs

* add sample_train_data for pyspark.pandas dataframe, fix bugs

* improve some functions, fix bugs

* fix dict change size during iteration

* update model predict

* update LightGBM model, update test

* update SynapseML LightGBM params

* update synapseML and tests

* update TODOs

* Added support to roc_auc for spark models

* Added support to score of spark estimator

* Added test for automl score of spark estimator

* Added cv support to pyspark.pandas dataframe

* Update test, fix bugs

* Added tests

* Updated docs, tests, added a notebook

* Fix bugs in non-spark env

* Fix bugs and improve tests

* Fix uninstall pyspark

* Fix tests error

* Fix java.lang.OutOfMemoryError: Java heap space

* Fix test_performance

* Update test_sparkml to test_0sparkml to use the expected spark conf

* Remove unnecessary widgets in notebook

* Fix iloc java.lang.StackOverflowError

* fix pre-commit

* Added params check for spark dataframes

* Refactor code for train_test_split to a function

* Update train_test_split_pyspark

* Refactor if-else, remove unnecessary code

* Remove y from predict, remove mem control from n_iter compute

* Update workflow

* Improve _split_pyspark

* Fix test failure of too short training time

* Fix typos, improve docstrings

* Fix index errors of pandas_on_spark, add spark loss metric

* Fix typo of ndcgAtK

* Update NDCG metrics and tests

* Remove unuseful logger

* Use cache and count to ensure consistent indexes

* refactor for merge maain

* fix errors of refactor

* Updated SparkLightGBMEstimator and cache

* Updated config2params

* Remove unused import

* Fix unknown parameters

* Update default_estimator_list

* Add unit tests for spark metrics
2023-03-25 19:59:46 +00:00
Jirka Borovec a340919458
adding precommit check (#930)
* adding precommit check

* run precommit

* Apply suggestions from code review

Co-authored-by: Zvi Baratz <z.baratz@gmail.com>

* apply precommit

---------

Co-authored-by: Zvi Baratz <z.baratz@gmail.com>
2023-02-28 16:29:07 +00:00
Jirka Borovec 2ff1035733
precommit: end-of-file-fixer (#929)
* precommit: end-of-file-fixer

* exclude .gitignore

* apply

---------

Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
2023-02-28 16:27:14 +00:00
Chi Wang 501a0b8eb8
bump version to 1.1.3; workflow trigger (#924) 2023-02-18 16:07:27 +00:00
Chi Wang 35ce9b79e8
azure oai (#920)
* azure oai

* price update in notebook

* text Davinci

* pytorch-lightning version

* trigger action in merge queue

* types

* doc check in mege group
2023-02-16 23:38:50 +00:00
Chi Wang 671146146a
workflow path->paths (#915)
* workflow path->paths

* Apply suggestions from code review

Co-authored-by: Li Jiang <bnujli@gmail.com>

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-02-08 20:47:07 -08:00
Chi Wang 63d350d4c8
Openai (#905)
* add cost budget; move loc of make_dir

* support openai completion

* install pytest in workflow

* skip openai test

* test openai

* path for docs rebuild

* install datasets

* signal

* notebook

* notebook in workflow

* optional arguments and special params

* key -> k

* improve readability

* assumption

* optimize for model selection

* larger range of max_tokens

* notebook

* python package workflow

* skip on win
2023-02-05 20:13:08 -08:00
Li Jiang da2cd7ca89
Add supporting using Spark as the backend of parallel training (#846)
* Added spark support for parallel training.

* Added tests and fixed a bug

* Added more tests and updated docs

* Updated setup.py and docs

* Added customize_learner and tests

* Update spark tests and setup.py

* Update docs and verbose

* Update logging, fix issue in cloud notebook

* Update github workflow for spark tests

* Update github workflow

* Remove hack of handling _choice_

* Allow for failures

* Fix tests, update docs

* Update setup.py

* Update Dockerfile for Spark

* Update tests, remove some warnings

* Add test for notebooks, update utils

* Add performance test for Spark

* Fix lru_cache maxsize

* Fix test failures on some platforms

* Fix coverage report failure

* resovle PR comments

* resovle PR comments 2nd round

* resovle PR comments 3rd round

* fix lint and rename test class

* resovle PR comments 4th round

* refactor customize_learner to broadcast_code
2022-12-23 08:18:49 -08:00
Chi Wang 232c356a4b
fix bug related to _choice_ (#848)
* fix bug related to _choice_

* remove py 3.6

* sanitize config

* optimize test
2022-12-13 15:48:32 -05:00
dependabot[bot] b58151b3fa
Bump actions/checkout from 2 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-04 23:23:27 +00:00
zsk d6fd89a80a
Merge branch 'main' into dependabot/github_actions/codecov/codecov-action-3 2022-11-01 20:13:22 -04:00
zsk dfdef8db32
Merge branch 'main' into dependabot/github_actions/codecov/codecov-action-3 2022-11-01 11:07:47 -04:00
zsk 06cff076f0
Merge branch 'main' into dependabot/github_actions/actions/setup-python-4 2022-11-01 11:07:39 -04:00
Chi Wang b7a010e657
Move import location for Ray 2 (#721)
* ray version check when importing

* display learner_class when starting_points removed

* test ray 2
2022-09-13 19:13:06 -07:00
zsk c224694f17
Merge branch 'main' into dependabot/github_actions/actions/setup-python-4 2022-08-24 13:51:58 -04:00
zsk d7f5a54525
Merge branch 'main' into dependabot/github_actions/codecov/codecov-action-3 2022-08-24 13:51:51 -04:00
zsk 8a4a401097
Merge branch 'main' into dependabot/github_actions/actions/cache-3 2022-08-23 23:06:54 -04:00
zsk 71a2d9fd89
Merge branch 'main' into dependabot/github_actions/actions/setup-node-3 2022-08-22 10:30:02 -04:00
zsk af85eb3b25
Merge branch 'main' into dependabot/github_actions/actions/cache-3 2022-08-22 10:29:37 -04:00
Xueqing Liu 2ead296676
updating nlp notebook (#693)
* updating nlp notebook
2022-08-22 07:20:48 -04:00
dependabot[bot] 8064ecf0e7
Bump actions/setup-python from 2 to 4
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-21 22:46:02 +00:00
dependabot[bot] 07656e4af0
Bump actions/setup-node from 2 to 3
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2 to 3.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-21 22:45:54 +00:00
dependabot[bot] 822ca59ed2
Bump codecov/codecov-action from 1 to 3
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-21 22:45:50 +00:00
dependabot[bot] 066842cfcb
Bump actions/cache from 1 to 3
Bumps [actions/cache](https://github.com/actions/cache) from 1 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v1...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-21 22:45:46 +00:00
Ismaël Mejía 40ce15cbe3 chore: Auto update github actions with dependabot 2022-08-17 21:58:13 +02:00
Qingyun Wu 8b3c6e4d7b
VW version requirement and documentation on config_constraints vs metric_constraints (#686)
* add vw version requirement

* vw version

* version range

* add documentation

* vw version range

* skip test on py3.10

* vw version

* rephrase

* don't install vw on py 3.10

* move import location

* remove inherit

* 3.10 in version

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2022-08-15 20:16:11 -07:00
Chi Wang 524e5e2140
pull request template (#668)
* pull request template

* update instructions about tests
2022-08-11 17:35:41 -07:00
skzhang1 d88c7390ce ignore engine 2022-07-21 14:48:21 -04:00
Chi Wang cbb85e2aab
Py36 (#614)
* allow installation in py 3.6

* test py 3.6
2022-06-26 08:32:28 -07:00
Chi Wang c45741a67b
support latest xgboost version (#599)
* support latest xgboost version

* Update test_classification.py

* Update 

Exists problems when installing xgb1.6.1 in py3.6

* cleanup

* xgboost version

* remove time_budget_s in test

* remove redundancy

* stop support of python 3.6

Co-authored-by: zsk <shaokunzhang529@gmail.com>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2022-06-21 18:59:07 -07:00
Chi Wang 1b40b4b3a6
set_search_properties (#595)
* update the signature of set_search_properties
2022-06-16 16:30:50 -07:00
Chi Wang 3a7ebe6896
Add python 3.10 in the CI (#591)
* fix resource limit issue

* add python 3.10 in CI

* reinstall libomp in macos
2022-06-15 20:13:33 -07:00
Chi Wang f0b0cae682
zero-shot AutoML in readme (#474)
* zero-shot AutoML in readme

* use pydoc-markdown 4.5.0 to avoid error in 4.6.0
2022-03-05 11:49:39 -08:00
Chi Wang efd85b4c86
Deploy a new doc website (#338)
A new documentation website. And:

* add actions for doc

* update docstr

* installation instructions for doc dev

* unify README and Getting Started

* rename notebook

* doc about best_model_for_estimator #340

* docstr for keep_search_state #340

* DNN

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: Z.sk <shaokunzhang@psu.edu>
2021-12-16 17:11:33 -08:00
Xueqing Liu fd136b02d1
bug fix for TransformerEstimator (#293)
* fix checkpoint naming + trial id for non-ray mode, fix the bug in running test mode, delete all the checkpoints in non-ray mode

* finished testing for checkpoint naming, delete checkpoint, ray, max iter = 1

* adding predict_proba, address PR 293's comments

close #293 #291
2021-11-23 11:26:39 -08:00
Chi Wang 549a0dfb53
limit time and memory consumption (#264)
* limit time and memory

* separate tests

* lrl1 can't be limited by limit_resource

* free memory when possible

* passthrough=False when ensemble fails;
retrain when trained_estimator is None

* use callback to for resource limit

* handle lower version of xgb with no callback

* free mem ratio

* reduce verbosity

* retrain_final when max_iter==1

* remove trained_estimator from result

* model_history

* wheel

* retrain time as best_config_train_time

* ci: libomp version for xgboost on macos

* limit_resource not working in windows

* test pickle load

* mute forecaster

* notebook update

* check hard

* preventive callback

* add use_ray
2021-11-03 19:08:23 -07:00
Chi Wang ddc1a63a76
Package (#244)
* build and upload pypi package

* pandas in dependency
2021-10-10 22:57:22 -07:00
Kevin Chen 3d0a3d26a2
Forecast (#162)
* added 'forecast' task with estimators ['fbprophet', 'arima', 'sarimax']

* update setup.py

* add TimeSeriesSplit to 'regression' and 'classification' task

* add 'time' split_type for 'classification' and 'regression' task

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>

* feature importance

* variable name

* Update test/test_split.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update test/test_forecast.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* prophet installation fail in windows

* upload flaml_forecast.ipynb

Signed-off-by: Kevin Chen <chenkevin.8787@gmail.com>
2021-08-23 13:26:46 -07:00
Qingyun Wu a291abfab9
Cha cha (#127)
* unordered categorical

* allow cost attribute to be None

* tensorboardX version

* quote

* cfo cat

* trunc

* Update version.py

* incumbent is normalized

* python 3.9

* remove ConcurrencyLimiter

* seed

* estimator

* update autovw notebook

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qiw@microsoft.com>
2021-07-05 18:17:26 -07:00
Qingyun Wu 3e991e4352
code coverage (#79)
* add codecov.yml

* change tests to test

* condition

* windows

* install coverage

* remove conditions

* revert remove conditions

* check build failure

* code coverage

* Revert "check build failure"

This reverts commit bc06b3f961d05c923c2974451961f5c252c9e042.

* del codecov.yml

Co-authored-by: Chi Wang (MSR) <chiw@microsoft.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2021-04-26 20:04:57 -04:00