<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md#pull-request-checklist
-->
#### Reference Issues/PRs
<!--
Example: Fixes#1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Closes#12798
#### What does this implement/fix? Explain your changes.
This PR deprecates the use of `logistic_regression_path` and makes it private.
#### Any other comments?
<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.
Thanks for contributing!
-->
* First draft on elasticnet penaly for LogisticRegression
* Some basic tests
* Doc update
* First draft for LogisticRegressionCV.
It seems to be working for binary classification and for multiclass when
multi_class='ovr'. I'm having a hard time figuring out the intricacies
of multi_class='multinomial'.
* Changed default to None for l1_ratio.
added warning message is user sets l1_ratio while penalty is not
elastic-net
* Some more doc
* Updated example to plot elastic net sparsity
* Fixed flake8
* Fixed test by not modifying attribute in fit
* Fixed doc issues
* WIP
* Partially fixed logistic_reg_CV for multinomial.
Also added some comments that are hopefully clear.
Still need to fix refit=False
* Fixed doc issue
* WIP
* Fixed test for refit=False in LogisticRegressionCV
* Fixed Python 2 numpy version issue
* minor doc updates
* Weird doc error...
* Added test to ensure that elastic net is at least as good as L1 or L2
once l1_ratio has been optimized with grid search
Also addressed minor reviews
* Fixed test
* addressed comments
* Added back ignore warning on tests
* Added a functional test
* Scale data in test... Now failing
* elastic-net --> elasticnet
* Updated doc for some attributes and checked their shape in tests
* Added l1_ratio dimension to coefs_paths and scores attr
* improve example + fix test
* FIX incorrect lagged_update in SAGA
* Add non-regression test for SAGA's bug
* FIX flake8 and warning
* Re fixed warning
* Updated some tests
* Addressed comments
* more comments and added dimension to LogisticRegressionCV.n_iter_ attribute
* Updated whatsnew for 0.21
* better doc shape looks
* Fixed whatnew entry after merges
* Added dot
* Addressed comments + standardized optional default param docstrings
* Addessed comments
* use swapaxes instead of unsupported moveaxis (hopefully fixes tests)
* Change default solver in LogisticRegression
* This is an API change, not a feature
* Decrease numerical precision in LogisticRegression doctest
* ENH add multi_class='auto' for LR, default from 0.22
* No warning when binary
* add FutureWarning for methods with defaults=3
* add explicit cv values to fix assertion errors
* add tests for catching the FutureWarning
* Write current deprecation version
* Add deprecation in docstring
* change default cv value to None
* change cv from 3 to 5 in the examples
* upgrade doctests
* update doctest in tutorial
* update doctest in cross-validation doc
* fix tests
* add entry to whats new
* address Gael comments
* address Gael comments 2
* fix wrong indentation
* update doc
* add docstring deprecation warning in CV subclasses
* address Andy's comments
* fix PR number
* fix flake8
* add filterwarnings in tests
* fix doctests
* cv=None mendatory in Ridge
* fix warning related errors
* skip some doctests warnings
* make travis happy
* change from deprecated to versionchanged
* fix doctests and remove skipping
* address comments
* improved docstring for the `solver` parameter of LogisticRegression
* further improve docstring on `n_jobs` and `solver`
* added warning when self.solver == 'liblinear' & self.n_jobs != -1
in LogisticRegression
* corrected typo: warning => warnings
* TST/DOC reverse doc and add test
* Add a test to ensure not changing the input's data type
Test that np.float32 input data is not cast to np.float64 when using LR + newton-cg
* [WIP] Force X to remain float32. (self.coef_ remains float64 even if X is not)
* [WIP] ensure self.coef_ same type as X
* keep the np.float32 when multi_class='multinomial'
* Avoid hardcoded type for multinomial
* pass flake8
* Ensure that the results in 32bits are the same as in 64
* Address Gael's comments for multi_class=='ovr'
* Add multi_class=='multinominal' to test
* Add support for multi_class=='multinominal'
* prefer float64 to float32
* Force X and y to have the same type
* Revert "Add support for multi_class=='multinominal'"
This reverts commit 4ac33e8c02.
* remvert more stuff
* clean up some commmented code
* allow class_weight to take advantage of float32
* Add a test where X.dtype is different of y.dtype
* Address @raghavrv comments
* address the rest of @raghavrv's comments
* Revert class_weight
* Avoid copying if dtype matches
* Address alex comment to the cast from inside _multinomial_loss_grad
* address alex comment
* add sparsity test
* Addressed Tom comment of checking that we keep the 64 aswell
* remove stuff to be removed 0.19
* more changes
* remove classes from 0.19 whatsnew
* remove _LearntSelectorMixin
* remove ProjectedGradientNMF, load_lwf_*
* minor fixes
* remove more copy from logistic regression path
* remove lda, qda from __init__.__all__
* remove pg solver in nmf from tests etc
* remove class_weight="auto" from tests
* doctest change for decision_function_shape="ovr"
* remove transfrom from tree test, minor fixes to tree tests
* some fixes in the tests
* undo changes in functions which still allow 1d input...
* also allow 1d in scale
* more test fixes...
* last test fixes in forest and tree
* svm default value change doctest failures
* pep8
* remove more class_weight="auto" stuff
* minor cosmetics in docstrings deprecated / removed behavior.
* say that store_covariance has been moved to __init__ in discriminant_analysis
* Remove cdivision=True Cython compiler directive causing incorrect int division.
* Decrease tol for LogisticRegression fit test, perhaps needed due to test not controlling `random_state`.
--------------------
* ENH Reogranize classes/fn from grid_search into search.py
* ENH Reogranize classes/fn from cross_validation into split.py
* ENH Reogranize cls/fn from cross_validation/learning_curve into validate.py
* MAINT Merge _check_cv into check_cv inside the model_selection module
* MAINT Update all the imports to point to the model_selection module
* FIX use iter_cv to iterate throught the new style/old style cv objs
* TST Add tests for the new model_selection members
* ENH Wrap the old-style cv obj/iterables instead of using iter_cv
* ENH Use scipy's binomial coefficient function comb for calucation of nCk
* ENH Few enhancements to the split module
* ENH Improve check_cv input validation and docstring
* MAINT _get_test_folds(X, y, labels) --> _get_test_folds(labels)
* TST if 1d arrays for X introduce any errors
* ENH use 1d X arrays for all tests;
* ENH X_10 --> X (global var)
Minor
-----
* ENH _PartitionIterator --> _BaseCrossValidator;
* ENH CVIterator --> CVIterableWrapper
* TST Import the old SKF locally
* FIX/TST Clean up the split module's tests.
* DOC Improve documentation of the cv parameter
* COSMIT consistently hyphenate cross-validation/cross-validator
* TST Calculate n_samples from X
* COSMIT Use separate lines for each import.
* COSMIT cross_validation_generator --> cross_validator
Commits merged manually
-----------------------
* FIX Document the random_state attribute in RandomSearchCV
* MAINT Use check_cv instead of _check_cv
* ENH refactor OVO decision function, use it in SVC for sklearn-like
decision_function shape
* FIX avoid memory cost when sampling from large parameter grids
ENH Major to Minor incremental enhancements to the model_selection
Squashed commit messages - (For reference)
Major
-----
* ENH p --> n_labels
* FIX *ShuffleSplit: all float/invalid type errors at init and int error at split
* FIX make PredefinedSplit accept test_folds in constructor; Cleanup docstrings
* ENH+TST KFold: make rng to be generated at every split call for reproducibility
* FIX/MAINT KFold: make shuffle a public attr
* FIX Make CVIterableWrapper private.
* FIX reuse len_cv instead of recalculating it
* FIX Prevent adding *SearchCV estimators from the old grid_search module
* re-FIX In all_estimators: the sorting to use only the 1st item (name)
To avoid collision between the old and the new GridSearch classes.
* FIX test_validate.py: Use 2D X (1D X is being detected as a single sample)
* MAINT validate.py --> validation.py
* MAINT make the submodules private
* MAINT Support old cv/gs/lc until 0.19
* FIX/MAINT n_splits --> get_n_splits
* FIX/TST test_logistic.py/test_ovr_multinomial_iris:
pass predefined folds as an iterable
* MAINT expose BaseCrossValidator
* Update the model_selection module with changes from master
- From #5161
- - MAINT remove redundant p variable
- - Add check for sparse prediction in cross_val_predict
- From #5201 - DOC improve random_state param doc
- From #5190 - LabelKFold and test
- From #4583 - LabelShuffleSplit and tests
- From #5300 - shuffle the `labels` not the `indxs` in LabelKFold + tests
- From #5378 - Make the GridSearchCV docs more accurate.
- From #5458 - Remove shuffle from LabelKFold
- From #5466(#4270) - Gaussian Process by Jan Metzen
- From #4826 - Move custom error / warnings into sklearn.exception
Minor
-----
* ENH Make the KFold shuffling test stronger
* FIX/DOC Use the higher level model_selection module as ref
* DOC in check_cv "y : array-like, optional"
* DOC a supervised learning problem --> supervised learning problems
* DOC cross-validators --> cross-validation strategies
* DOC Correct Olivier Grisel's name ;)
* MINOR/FIX cv_indices --> kfold
* FIX/DOC Align the 'See also' section of the new KFold, LeaveOneOut
* TST/FIX imports on separate lines
* FIX use __class__ instead of classmethod
* TST/FIX import directly from model_selection
* COSMIT Relocate the random_state documentation
* COSMIT remove pass
* MAINT Remove deprecation warnings from old tests
* FIX correct import at test_split
* FIX/MAINT Move P_sparse, X, y defns to top; rm unused W_sparse, X_sparse
* FIX random state to avoid doctest failure
* TST n_splits and split wrapping of _CVIterableWrapper
* FIX/MAINT Use multilabel indicator matrix directly
* TST/DOC clarify why we conflate classes 0 and 1
* DOC add comment that this was taken from BaseEstimator
* FIX use of labels is not needed in stratified k fold
* Fix cross_validation reference
* Fix the labels param doc
FIX/DOC/MAINT Addressing the review comments by Arnaud and Andy
COSMIT Sort the members alphabetically
COSMIT len_cv --> n_splits
COSMIT Merge 2 if; FIX Use kwargs
DOC Add my name to the authors :D
DOC make labels parameter consistent
FIX Remove hack for boolean indices; + COSMIT idx --> indices; DOC Add Returns
COSMIT preds --> predictions
DOC Add Returns and neatly arrange X, y, labels
FIX idx(s)/ind(s)--> indice(s)
COSMIT Merge if and else to elif
COSMIT n --> n_samples
COSMIT Use bincount only once
COSMIT cls --> class_i / class_i (ith class indices) -->
perm_indices_class_i
FIX/ENH/TST Addressing the final reviews
COSMIT c --> count
FIX/TST make check_cv raise ValueError for string cv value
TST nested cv (gs inside cross_val_score) works for diff cvs
FIX/ENH Raise ValueError when labels is None for label based cvs;
TST if labels is being passed correctly to the cv and that the
ValueError is being propagated to the cross_val_score/predict and grid
search
FIX pass labels to cross_val_score
FIX use make_classification
DOC Add Returns; COSMIT Remove scaffolding
TST add a test to check the _build_repr helper
REVERT the old GS/RS should also be tested by the common tests.
ENH Add a tuple of all/label based CVS
FIX raise VE even at get_n_splits if labels is None
FIX Fabian's comments
PEP8
ENH NonBLASDotWarning -> EfficiencyWarning; Improve error message
DOC Add exceptions module to modules/classes.rst
MAINT Move ConvergenceWarning, UndefinedMetricWarning et al into exceptions
MAINT Remove ChangedBehaviorWarning from base
DOC/FIX Improve DataConversionWarning's docstring
* Updated _check_solver_option to include sample_weight check
* Updated all calls to _check_solver_option()
* Updated documentation of class_weight throughout logistic.py
* Added sample_weight parameter to logistic_regression_path.
* Added handling of sample weights to logistic_regression_path.
* Added sample_weight parameter to _log_reg_scoring_path.
* Added handling of sample weights to _log_reg_scoring_path.
* Added sample_weight parameter to fit() in the LogisticRegression class.
* Added handling of sample sample weights in LogisticRegression.fit()
* Added sample_weight parameter to fit() in the LogisticRegressionCV class.
* Added handling of sample weights in LogisticRegressionCV.fit()
* Added test_logistic_regressioncv_sample_weights, which:
* tests that a ValueError is raised if liblinear is used with
sample weights
* tests that passing sample weights as np.ones(y.shape[0]) is
the same as not passing them (default None)
* tests that using both lbfgs and newton-cg solvers with
sample weights yields the same results
* tests that passing class weights to scale one class is the
same as passing sample weights for the training data of just
that class
* Fixed bug with *= in logistic_regression_path.
* Fixed bug in test_logistic_regressioncv_sample_weights where
no data was created prior to fitting.
* Changes to accepted sample_weight type.
* Fixed bug in naming of sample_weight when passed from
_log_reg_scoring_path to logistic_regression_path.
* Fixed issue of sample_weight=None being converted to np.array()
and then not being reconigzed as None.
* Added tests for LogisticRegression
* Attempting to fix same issue as 9d3becf by instead implementing
if statement in bagging.py.
* Added TODO to eliminate check for liblinear w/ sample weights
in bagging.py
Nicolas Hug
Hanmin Qin