* Adapted the number of splits
* Update plot_learning_curve.py
* Update plot_learning_curve.py
Added ``random_state=0`` to call of ``learning_curve``
* Update plot_learning_curve.py
Added ``shuffle=True`` for ``random_state`` to make an impact
* Update examples/model_selection/plot_learning_curve.py
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
* resolve ambiguity of the nested cross-val example
* Update examples/model_selection/plot_nested_cross_validation_iris.py
make it more explict
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
* Initial implementation
* Forgot to add to second __add__ list
* Update split method parameter doc
* Added example; changed default test_size to 0.1; added to author list
* StratifiedGroupKFold impl and other improvements
* Add class to __all__ spec
* Remove random_state when no shuffle
* Tighter formatting
* Update the implementation of StratifiedGroupKFold
* Add StratifiedGroupKFold to __init__
* Add y checks to StartifiedGroupKFold
* Raise error if n_splits > max num samples in class
* Warn if n_splits > mn num samples in class
* Add SGKfold to general repr test
* Add SGKFold to 2d_y test case
* Add SGKfold to value erros test case
Parameters are the same as for StratifiedKFold
to ensure similar behavior given n_groups == n_samples
* Add SGKFold to StratifiedKFold test cases
The idea is to ensure similar behavior when groups are trivial
(n_groups == n_samples)
* Add SGKFold to reproducibility test case
* Add SGKFold to GroupKFold test case
* Add SGKFold to nested cv test case
* Add SGKFold to random_state with shuffle=False test case
* Add SGKFold to constant splits test case
* Fix repr test case
* Fix formatting issues
* Add samples to a fold with least num samples
Required to produce balanced size folds when the distribution of y is
more or less the same
* Remove GroupShuffleSplit impl
* Add notes to StratifiedGroupKFold
* Fix doctest
* Added stratified group kfold tests
* Better variable naming
* Add section to documentation
* Remove leftover StratifiedGroupShuffleSplit import
* Add changelist and reference to original kernel
* Better naming for least populated class check
* Better expression for number of labels
* Remove use of Counter
We already have this data in output of np.unique
* Add tests for homogeneous groups
* Add StratifiedGroupKFold test against GroupKFold
* Add changes to changelist in docstring
* Add StratifiedGroupKFold to classes.rst
* Fix description of StratifiedGroupKFold
* Move license notice out of docstring
* Disambiguate labels to classes in doc
* Add changelog entry
* Fix changelog author entry
* Fix StratifiedGroupKFold docstring
* Better variable names
* Remove defaultdict in favor of numpy indexing
* Extracted best_fold search into a separate method
* Make use of numpy broadcasting instead of for loop
* Encode groups and use arrays instead of dicts
* Use numpy sort instead of python
* Clarify shuffling behavior of StratifiedGroupKF in docs
* Switch name from label_idx to class_idx
* Remove accidentally leftover comment
* Fix np.sort keyword to support numpy < 1.15
* Fix typo in docstring
* Add StratifiedGroupKFold to visualization doc
* Add visualization for uneven group as an example
* Fix image numbers to match updated example
* Add author
* Add SGKF visualization to docs
* Add comments for groups in stratified CV tests
Co-authored-by: Leandro Hermida <hermidal@cs.umd.edu>
Co-authored-by: marrodion <rodion_martynov@epam.com>
* More flexible grid search interface
* added info dict parameter
* Put back removed test
* renamed info into more_results
* Passed grroups as well since we need n_to use get_n_splits(X, y, groups)
* port
* pep8
* dabl -> sklearn
* add _required_parameters
* skipping check in rst file if pandas not installed
* Update sklearn/model_selection/_search_successive_halving.py
Co-Authored-By: Joel Nothman <joel.nothman@gmail.com>
* renamed into GridHalvingSearchCV and RandomHalvingSearchCV
* Addressed thomas' comments
* repr
* removed passing group as a parameter to evaluate_candidates
* Joels comments
* pep8
* reorganized user user guide
* renaming
* update user guide
* remove groups support + pass fit_params
* parameter renaming
* pep8
* r_i -> resource_iter
* fixed r_i issues
* examples + removed use of word budget
* Added inpute checking tests
* added cv_resutlts_ user guide
* minor title change
* fixed doc layout
* Addressed some comments
* properly pass down fit_params
* change default value of force_exhaust_resources and update doc
* should fix doc
* Used check_fit_params
* Update section about min_resources and number of candidates
* Clarified ratio section
* Use ~ to refer to classes
* fixed doc checks
* Apply suggestions from code review
Co-authored-by: Joel Nothman <joel.nothman@gmail.com>
* Addressed easy comments from Joel
* missed some
* updated docstring of run_search
* Used f strings instead of format
* remove candidate duplication checks
* fix example
* Addressed easy comments
* rotate ticks labels
* Added discussion in the intro as suggested by Joel
* Split examples into sections
* minor changes
* remove force_exhaust_budget and introduce min_resources=exhaust
* some minor validation
* Added a n_resources_ attribute
* update examples
* Addressed comments
* passing CV instead of X,y
* minor revert for handling fit_params
* updated docs
* fix len
* whatsnew
* Add test for sampling when all_list
* minor change to top-k
* Force CV splits to be consistent across calls
* reorder parameters
* reduced diff
* added tests for top_k
* put back doc for groups
* not sure what went wrong
* put import at its place
* some comment
* Addressed comments
* Added tests for cv_results_ and base estimator inputs
* pep8
* avoid monkeypatching
* rename df
* use Joel's suggestions for testing masks
* Made it experimental
* Should fix docs
* whats new entry
* Apply suggestions from code review
Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
* Addressed comments to docs
* Addressed comments in examples
* minor doc update
* minor renaming in UG
* forgot some
* some sad note about splitter statefulness :'(
* Addressed comments
* ratio -> factor
Co-authored-by: Joel Nothman <joel.nothman@gmail.com>
Co-authored-by: Andreas Mueller <t3kcit@gmail.com>
* Initial add DET curve to classification metrics
* Add DET to exports
* Fix DET-curve doctest errors
- Sample snippet in model_evaluation documentation was outdated.
* Clarify wording in DET-curve computation
- Align to the wording of ranking module to make it consistent.
- Add correct describtion of input and outputs.
- Update and fix non-existent links
* Beautify DET curve documentation source
- Limit line length to 80 characters.
* Expand DET curve documentation
- Add an example plot to show difference between ROC and DET curves.
- Expand Usage Note section with background information and properties
of DET curves.
* Update DET-curve documentation
- Fix typos and some grammar improvements.
- Use named references to avoid potential conflicts with other sections.
- Remove unneeded references and improved existing ones by using e.g.
using versioned links.
* Select relevant DET points using slice object
* Remove some dubiety from DET curve doc-string
* Add DET curve contributors
* Add tests for DET curves
* Streamline DET test by using parametrization
* Increase verbosity of DET curve error handling
- Explicitly sanity check input before computing a DET curve.
- Add test for perfect scores.
- Adapt indentation style to match the test module.
* Add reference for DET curves in invariance test
* Add automated invariance checks for DET curves
* Resolve merge artifacts
* Make doctest happy
* Fix whitespaces for doctest
* Revert unintended whitespace changes
* Revert unintended white space changes #2
* Fix typos and grammar
* Fix white space in doc
* Streamline test code
* Remove rebase artifacts
* Fix PR link in doc
* Fix test_ranking
* Fix rebase errors
* Fix import
* Bring back newlines
- Swallowed by copy/paste
* Remove uncited ref link
* Remove matplotlib deprecation warning
* Bring back hidden reference
* Add motivation to DET example
* Fix lint
* Add citation
* Use modern matplotlib API
Co-authored-by: Jeremy Karnowski <jeremy.karnowski@gmail.com>
Co-authored-by: Julien Cornebise <julien@cornebise.com>
Co-authored-by: Daniel Mohns <daniel.mohns@zenguard.org>
cdrig