Commit Graph

76 Commits

Author SHA1 Message Date
Nicolas Hug 0a5af0d2a1
FEA Successive halving for faster parameter search (#13900)
* More flexible grid search interface

* added info dict parameter

* Put back removed test

* renamed info into more_results

* Passed grroups as well since we need n_to use get_n_splits(X, y, groups)

* port

* pep8

* dabl -> sklearn

* add _required_parameters

* skipping check in rst file if pandas not installed

* Update sklearn/model_selection/_search_successive_halving.py

Co-Authored-By: Joel Nothman <joel.nothman@gmail.com>

* renamed into GridHalvingSearchCV and RandomHalvingSearchCV

* Addressed thomas' comments

* repr

* removed passing group as a parameter to evaluate_candidates

* Joels comments

* pep8

* reorganized user user guide

* renaming

* update user guide

* remove groups support + pass fit_params

* parameter renaming

* pep8

* r_i -> resource_iter

* fixed r_i issues

* examples + removed use of word budget

* Added inpute checking tests

* added cv_resutlts_ user guide

* minor title change

* fixed doc layout

* Addressed some comments

* properly pass down fit_params

* change default value of force_exhaust_resources and update doc

* should fix doc

* Used check_fit_params

* Update section about min_resources and number of candidates

* Clarified ratio section

* Use ~ to refer to classes

* fixed doc checks

* Apply suggestions from code review

Co-authored-by: Joel Nothman <joel.nothman@gmail.com>

* Addressed easy comments from Joel

* missed some

* updated docstring of run_search

* Used f strings instead of format

* remove candidate duplication checks

* fix example

* Addressed easy comments

* rotate ticks labels

* Added discussion in the intro as suggested by Joel

* Split examples into sections

* minor changes

* remove force_exhaust_budget and introduce min_resources=exhaust

* some minor validation

* Added a n_resources_ attribute

* update examples

* Addressed comments

* passing CV instead of X,y

* minor revert for handling fit_params

* updated docs

* fix len

* whatsnew

* Add test for sampling when all_list

* minor change to top-k

* Force CV splits to be consistent across calls

* reorder parameters

* reduced diff

* added tests for top_k

* put back doc for groups

* not sure what went wrong

* put import at its place

* some comment

* Addressed comments

* Added tests for cv_results_ and base estimator inputs

* pep8

* avoid monkeypatching

* rename df

* use Joel's suggestions for testing masks

* Made it experimental

* Should fix docs

* whats new entry

* Apply suggestions from code review

Co-authored-by: Andreas Mueller <t3kcit@gmail.com>

* Addressed comments to docs

* Addressed comments in examples

* minor doc update

* minor renaming in UG

* forgot some

* some sad note about splitter statefulness :'(

* Addressed comments

* ratio -> factor

Co-authored-by: Joel Nothman <joel.nothman@gmail.com>
Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
2020-09-09 17:12:35 +02:00
Thomas J. Fan 9acfaab966
ENH Adds support for multimetric callable return a dictionary (#15126) 2020-07-16 17:55:40 +02:00
Nicolas Hug ac8cbb3799
[MRG] More deprecation for 0.24 (#17142)
* normalize_components in sparsePCA

* changed default strategy of Dummy to prior and removed outputs_2d ttribute

* removed usage of None to drop estimator in ensemble and behaviour param of IsolationForest

* remove support for drop=None in Voting

* removed some warning decorators

* remove feature_extraction.extract_patches

* removed VectorizerMixin and copy parameter from TFIDFVectorizer

* kernel.set_params now raises attributeerror

* removed fig from plot_partial_dependence

* removed iid parameter of search estimators

* removed brier_scorer

* raise error in split when shuffle is False and random_state is not None

* removed MultiOutputEstimator

* removed base classes of NaiveBayes

* removed drop from pipeline

* removed utils in random_projection

* removed presort and classes_ in trees

* flake8

* fixed some tests

* flake

* fixed docstring

* fixed other one

* some left

* mmmm
2020-05-20 16:04:14 -04:00
Kyle Kosic b4e215c01c
FEA Additional `TimeSeriesSplit` Functionality (#13204)
Co-authored-by: Kyle Kosic <kylekosic@Kyles-MacBook-Pro.local>
Co-authored-by: Thomas J Fan <thomasjpfan@gmail.com>
2020-05-11 21:48:01 -04:00
Adrin Jalali 02309ffbda
API kwonly args in manifold, metrics, mixture, model_selection, multclass, multioutput (#16982) 2020-04-24 17:24:24 -04:00
Roman Yurchak 75d3f29e72
MAINT Remove outdated numpy and scipy backports (#16725)
* Remove unecessary numpy & scipy backports

* More fixes

* Remove unused imports
2020-03-20 13:57:18 +01:00
Roman Feldbauer 9b39c4c4d2
TST Fix unreachable code in tests (#16110) 2020-02-16 14:41:26 +01:00
scibol 3a5aced32b TST Fixes integer test for train and test indices (#15941) 2019-12-20 12:43:30 -05:00
Hanmin Qin 6c582888ab
TST Specify random_state in test_cv_iterable_wrapper (#15829) 2019-12-11 18:33:53 +08:00
Nicolas Hug 19ad136223 MNT Replace DeprecationWarning with FutureWarning (#15080)
* bruteforce renaming

* WIP

* WIP

* some more

* removed weird line

* update -Werror

* testiforest

* again

* Fixed some tests

* fixed some tests

* removed -Werror

* fixed test_docstring_param issue

* fixed some tests

* some more

* renamed to SklearnDeprecationWarning

* pep8

* updated docs

* pep8

* merge

* changed to FutureWarning

* fixes

* Update doc/developers/tips.rst

Co-Authored-By: Adrin Jalali <adrin.jalali@gmail.com>

* avoid duplicates

* fixed warning for deprecations

* Still make CI break if DeprecationWarning isn't caught

* updated one warning

* fixed test

* fixed some renamings

* updated new dep warnings

* fixed bad import

* ignore warnings

* update again

* ignore futurewarning when walking packages

* Fixed test

* pep8

* Added whatsnew
2019-10-29 15:39:26 +01:00
Nicolas Hug b92455a6b2 MAINT Deprecate all of utils.testing except all_estimators (#15367) 2019-10-28 17:28:56 +01:00
Nicolas Hug 7effc37174
MNT Warn when shuffle is False but random_state is not None (#15353) 2019-10-24 16:33:12 -04:00
Thomas J Fan dba753a200 API Auto generates deprecation for sklearn.utils.mocking (#15071) 2019-10-03 00:09:28 +10:00
DrGFreeman ca9ceba558 FIX implement repr for RepeatedKFold and RepeatedStratifiedKFold (#14983) 2019-09-16 13:58:02 +02:00
Joel Nothman 3de368d405 FIX reimplement StratifiedKFold to avoid variation in test size (#14704) 2019-08-24 20:48:05 +08:00
Adrin Jalali 19c068a2ec MNT towards removing assert_equal, etc (#14222) 2019-07-01 09:13:32 -04:00
Roman Yurchak 5674122c97 MAINT More test runtime optimizations (#14136)
* Feature extraction / feature selection

* Metrics, manifold, impute, GP optimization

* Optimize mixture

* Optimize model_selection

* Fix tests

* Lint
2019-06-22 10:44:06 +03:00
Roman Yurchak ccd3331f7e MNT remove unused imports (#14021) 2019-06-04 23:15:19 +10:00
Guillaume Lemaitre 9adba491a2 [MRG] DEP change the default of cv and n_splits (#13839) 2019-05-29 23:39:20 +10:00
Guillaume Lemaitre 69eb4d4678 [MRG] DEP change default and deprecate iid in SearchCV (#13834) 2019-05-09 14:26:05 +02:00
jeremiedbb 358c6922f6 API/MNT Update default test_size of ShuffleSplit for 0.21 (#13483)
completing deprecation
2019-03-26 09:33:19 +11:00
Nicolas Hug 2718d6212f ENH Raise ValueError when trainset is empty in CVSplitters (#12861) 2019-03-22 23:04:10 -04:00
Andreas Mueller d879b5cdbe MNT be more friendly in the deprecation warning of cv=3 (#13395)
* be more friendly in the deprecation of cv=3

* add hint on specifying cv

* catch all the right warnings
2019-03-05 19:28:34 +01:00
Hanmin Qin afc6cc58da FIX Shuffle each class's samples with different random_state in StratifiedKFold (#13124)
* Enable StratifiedKFold to produce different splits

* what's new

* redundant statement

* update what's new

* redundant comment

* add a test

* move what's new entry

* review comment

* review comment
2019-02-27 14:31:21 +01:00
Christos Aridas c3415b8bd1 TST change name of test such that they are included by pytest in test_split (#13299) 2019-02-27 18:39:16 +08:00
Roman Yurchak 0e3bb17e62 MAINT Run pyupgrade following Python2 deprecation (#12997) 2019-02-08 20:13:34 +08:00
surgan12 62d2059804 MNT remove __future__ imports (#12791) 2019-02-02 22:05:06 +08:00
Roman Yurchak acb8106472 MNT Use list and dict comprehension (#12668) 2019-01-08 09:32:34 +11:00
Hanmin Qin 6dd6af78eb TST Use match instead of message in pytest.raises (#12933) 2019-01-07 15:20:11 +01:00
Andreas Mueller 952ef6637a MRG Drop legacy python / remove six dependencies (#12639) 2019-01-03 15:50:05 +02:00
Dror Atariah 440c08684f FIX Add more validation for size parameters in train_test_split (#12733)
* Fixed typo in an example

* Removed un-needed lines from example

* Added tests on the validity of parameters

* Fixed error msg for train_size issue

* Added tests to check the validation of test and train sizes

* Switched to parameterized test

* Fixed typo in error msg

* Swithced to pytest.raises

* Swithced to pytest.raises also when checking msg

* Improved the validity tests and their unit testing
2018-12-19 12:36:53 +01:00
Bartosz Michałowski fa98a72dcc MNT Replaced all occurrences of assert_true and assert_false with assert (#12588) 2018-11-28 09:16:26 +08:00
Hanmin Qin 69c5dd9d85 TST Specify random seed in test_shuffle_kfold_stratifiedkfold_reproducibility (#12674) 2018-11-27 21:17:49 +01:00
Yaroslav Halchenko 362cb3bcab TST autoreplace assert_true(...==...) with plain assert (#12547) 2018-11-11 09:05:34 +08:00
Andreas Mueller 0f94f2962b
MNT simple deprecations and removals for 0.21 (#12238)
Part of #11992.
These were all the things that seemed pretty straight-forward. It's actually a bit bulky but should still be easy to review, hopefully.
2018-10-11 14:56:37 -04:00
Roman Yurchak ec69171d52 TST Use pytest.raises instead of legacy constructions (#12057) 2018-09-13 13:57:43 +02:00
Hanmin Qin 5533deb8e0 [MRG] FIX Support cv=None in *CV (#11761) 2018-08-07 11:53:59 +10:00
ZJ Poh adddf00433 [MRG] np.ones -> np.full (#11628) 2018-07-23 09:49:01 +02:00
Alexandre Boucaud f158e2dfe2 [MRG+1] Change CV defaults to 5 (#11557)
* add FutureWarning for methods with defaults=3

* add explicit cv values to fix assertion errors

* add tests for catching the FutureWarning

* Write current deprecation version

* Add deprecation in docstring

* change default cv value to None

* change cv from 3 to 5 in the examples

* upgrade doctests

* update doctest in tutorial

* update doctest in cross-validation doc

* fix tests

* add entry to whats new

* address Gael comments

* address Gael comments 2

* fix wrong indentation

* update doc

* add docstring deprecation warning in CV subclasses

* address Andy's comments

* fix PR number

* fix flake8

* add filterwarnings in tests

* fix doctests

* cv=None mendatory in Ridge

* fix warning related errors

* skip some doctests warnings

* make travis happy

* change from deprecated to versionchanged

* fix doctests and remove skipping

* address comments
2018-07-19 14:46:11 +02:00
Andreas Mueller 8e2c2aa35d
raise DeprecationWarnings and FutureWarnings as errors (#11570)
Towards #11252.
In the end we'd like to make these errors so we can keep this cleaner in the future.
2018-07-17 15:31:45 -05:00
Andreas Mueller eec7649236 MAINT Complete 0.20 deprecations (#9570) 2018-06-24 23:06:26 +10:00
Charlie Brummitt d074e403f1 [MRG+1] Fix bug in StratifiedShuffleSplit for multi-label data with targets having > 1000 labels (#9922)
* Use ' '.join(row) for multi-label targets in StratifiedShuffleSplit because str(row) uses an ellipsis when len(row) > 1000
* Add a new test for multilabel problems with more than a thousand labels
2017-10-17 09:44:35 +02:00
Andreas Mueller 94fa697356 [MRG+1] check that splitters handle 2d y and give reasonable errors on multilabel y (#9744) 2017-10-09 11:22:15 +02:00
Loïc Estève 4e6b403796 FIX test broken in numpy 1.14.dev due to array str changes 2017-09-29 16:34:52 +02:00
(Venkat) Raghav, Rajagopalan 75d6005fea [MRG] Add few more tests + Documentation for re-entrant cross-validation estimators (#7823)
* DOC Add NOTE that unless random_state is set, split will not be identical

* TST use np.testing.assert_equal for nested lists/arrays

* TST Make sure cv param can be a generator

* DOC rank_ becomes a link when rendered

* Use test_...

* Remove blank line; Add if shuffle is True

* Fix tests

* Explicitly test for GeneratorType

* TST Add the else clause

* TST Add comment on usage of np.testing.assert_array_equal

* TYPO

* MNT Remove if ;

* Address Joel's comments

* merge the identical points in doc

* DOC address Andy's comments

* Move comment to before the check for generator type
2017-07-17 08:58:59 +10:00
Nelson Liu 1f781e64a2 [MRG+3] CV splitters: train/test_size default behavior will change in 0.21 (#7459) 2017-06-14 21:28:52 +10:00
Vlad Niculae 93563b0ac1 [MRG+1] fix StratifiedShuffleSplit with 2d y (#9044)
* regression test and fix for 2d stratified shuffle split

* strengthen non-overlap sss tests

* clarify test and comment

* remove iter from tests, use str instead of hash
2017-06-13 17:40:45 +05:30
Naoya Kanai 95aa2952e1 Add logsumexp and comb to utils.fixes (#9046) 2017-06-10 15:01:02 +02:00
Alexandre Gramfort 32f2418a07 use ValueError 2017-06-08 11:44:54 +02:00
themrmax eaebfe0535 [MRG+1] add shuffle paramater to train_test_split (#8845)
* add shuffle paramater to train_test_split

* fix syntax error

* fix variable name

* fix formatting in doctest output

* fix doctest output

* refactor shuffle paramater into ShuffleSplit and StratifiedShuffleSplit

* include shuffle option in tests

* rollback refactor

* revert to simpler version of unshuffled split

* fix flake8 errors

* revert changes to ShuffleSplit

* revert BaseShuffleSplit

* more reversions

* fix indentation

* remove shuffle parameter from CVclass

* add text to NotImplementedError

* change indexing to use numpy.arange rather than range

* specify precondition for stratify to be None in docstring
2017-06-08 11:43:22 +02:00