Commit Graph

141 Commits

Author SHA1 Message Date
Charlie Newey c554aad456 [MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform on unseen labels (#9816) 2017-09-21 19:21:55 +02:00
Bastian Venthur 7400775633 [MRG+1] MAINT Replace assert_array_equal with -assert_array_almost_equal where necessary. (#9774) 2017-09-18 19:55:23 +10:00
Guillaume Lemaitre 89b02af431 [MRG+1] EHN Accept 1D array for preprocessing functions and update doc (#9596)
* EHN/TST robust_scale accepts 1D array

* DOC update doc for preprocessing functions
2017-08-22 19:08:47 +03:00
Sebastin Santy dc43486806 Remove unused imports (#9235) 2017-07-01 05:57:54 -07:00
Taehoon Lee ebf2bf8107 Fix typos (#9205) 2017-06-23 11:43:46 +02:00
(Venkat) Raghav, Rajagopalan 763d93b72e [MRG+1] Do not transform y (#9180)
* we do not transform y

* more

* added Deprecation Warning to transform() to remove Y parameter

* more

* ENH ensure FunctionTransformer's transform/inverse_transform doesn't permit y

* Undo changes to pls_. It will be done in a separate PR (see #9160)

* flake8

* Update whatsnew

* Fully undo PLS changes
2017-06-22 23:24:12 +02:00
Guillaume Lemaitre 26a1027a83 [MRG+1] QuantileTransformer (#8363)
* resurrect quantile scaler

* move the code in the pre-processing module

* first draft

* Add tests.

* Fix bug in QuantileNormalizer.

* Add quantile_normalizer.

* Implement pickling

* create a specific function for dense transform

* Create a fit function for the dense case

* Create a toy examples

* First draft with sparse matrices

* remove useless functions and non-negative sparse compatibility

* fix slice call

* Fix tests of QuantileNormalizer.

* Fix estimator compatibility

* List of functions became tuple of functions
* Check X consistency at transform and inverse transform time

* fix doc

* Add negative ValueError tests for QuantileNormalizer.

* Fix cosmetics

* Fix compatibility numpy <= 1.8

* Add n_features tests and correct ValueError.

* PEP8

* fix fill_value for early scipy compatibility

* simplify sampling

* Fix tests.

* removing last pring

* Change choice for permutation

* cosmetics

* fix remove remaining choice

* DOC

* Fix inconsistencies

* pep8

* Add checker for init parameters.

* hack bounds and make a test

* FIX/TST bounds are provided by the fitting and not X at transform

* PEP8

* FIX/TST axis should be <= 1

* PEP8

* ENH Add parameter ignore_implicit_zeros

* ENH match output distribution

* ENH clip the data to avoid infinity due to output PDF

* FIX ENH restraint to uniform and norm

* [MRG] ENH Add example comparing the distribution of all scaling preprocessor (#2)

* ENH Add example comparing the distribution of all scaling preprocessor

* Remove Jupyter notebook convert

* FIX/ENH Select feat before not after; Plot interquantile data range for all

* Add heatmap legend

* Remove comment maybe?

* Move doc from robust_scaling to plot_all_scaling; Need to update doc

* Update the doc

* Better aesthetics; Better spacing and plot colormap only at end

* Shameless author re-ordering ;P

* Use env python for she-bang

* TST Validity of output_pdf

* EXA Use OrderedDict; Make it easier to add more transformations

* FIX PEP8 and replace scipy.stats by str in example

* FIX remove useless import

* COSMET change variable names

* FIX change output_pdf occurence to output_distribution

* FIX partial fixies from comments

* COMIT change class name and code structure

* COSMIT change direction to inverse

* FIX factorize transform in _transform_col

* PEP8

* FIX change the magic 10

* FIX add interp1d to fixes

* FIX/TST allow negative entries when ignore_implicit_zeros is True

* FIX use np.interp instead of sp.interpolate.interp1d

* FIX/TST fix tests

* DOC start checking doc

* TST add test to check the behaviour of interp numpy

* TST/EHN Add the possibility to add noise to compute quantile

* FIX factorize quantile computation

* FIX fixes issues

* PEP8

* FIX/DOC correct doc

* TST/DOC improve doc and add random state

* EXA add examples to illustrate the use of smoothing_noise

* FIX/DOC fix some grammar

* DOC fix example

* DOC/EXA make plot titles more succint

* EXA improve explanation

* EXA improve the docstring

* DOC add a bit more documentation

* FIX advance review

* TST add subsampling test

* DOC/TST better example for the docstring

* DOC add ellipsis to docstring

* FIX address olivier comments

* FIX remove random_state in sparse.rand

* FIX spelling doc

* FIX cite example in user guide and docstring

* FIX olivier comments

* EHN improve the example comparing all the pre-processing methods

* FIX/DOC remove title

* FIX change the scaling of the figure

* FIX plotting layout

* FIX ratio w/h

* Reorder and reword the plot_all_scaling example

* Fix aspect ratio and better explanations in the plot_all_scaling.py example

* Fix broken link and remove useless sentence

* FIX fix couples of spelling

* FIX comments joel

* FIX/DOC address documentation comments

* FIX address comments joel

* FIX inline sparse and dense transform

* PEP8

* TST/DOC temporary skipping test

* FIX raise an error if n_quantiles > subsample

* FIX wording in smoothing_noise example

* EXA Denis comments

* FIX rephrasing

* FIX make smoothing_noise to be a boolearn and change doc

* FIX address comments

* FIX verbose the doc slightly more

* PEP8/DOC

* ENH: 2-ways interpolation to avoid smoothing_noise

Simplifies also the code, examples, and documentation
2017-06-10 01:15:46 +02:00
Andreas Mueller 5c4b1bb231 [MRG+1] Housekeeping Deprecations for v0.19 (#7927)
* remove stuff to be removed 0.19

* more changes

* remove classes from 0.19 whatsnew

* remove _LearntSelectorMixin

* remove ProjectedGradientNMF, load_lwf_*

* minor fixes

* remove more copy from logistic regression path

* remove lda, qda from __init__.__all__

* remove pg solver in nmf from tests etc

* remove class_weight="auto" from tests

* doctest change for decision_function_shape="ovr"

* remove transfrom from tree test, minor fixes to tree tests

* some fixes in the tests

* undo changes in functions which still allow 1d input...

* also allow 1d in scale

* more test fixes...

* last test fixes in forest and tree

* svm default value change doctest failures

* pep8

* remove more class_weight="auto" stuff

* minor cosmetics in docstrings deprecated / removed behavior.

* say that store_covariance has been moved to __init__ in discriminant_analysis
2016-12-09 12:43:38 -05:00
Ang Lu c17156106c [MRG+1] Fix return_norm bug in preprocessing.normalize (#7789)
Fixes #7771
2016-11-23 09:45:32 +11:00
CJ Carey 94c2094f3e [MRG+1] BUG: MultiLabelBinarizer.fit_transform sometimes returns an invalid CSR matrix (#7750)
* BUG: MultiLabelBinarizer makes invalid CSR matrix

See https://github.com/scipy/scipy/issues/6719 for context.

The gist is that the `inverse` array may have a different dtype than `yt.indices`, which causes trouble down the line because, in those cases, `yt.indices` and `yt.indptr` have different dtypes.

Alternately, we could insert `yt.check_format(full_check=False)` after modifying the sparse matrix members.

* Fixing for old numpy

Older versions don't support kwargs for `astype`

* Adding tests

* line-wrapping

* adding comment to tests

[ci skip]

* added rationale comment

[ci skip]
2016-10-26 17:43:35 -04:00
Andreas Mueller 4da44c8541 [MRG+1] replaced some assert_true(np.allclose(x, y)) with assert_almost_equal (#7742)
* replaced some assert_true(np.allclose(x, y)) with assert_almost_equal for better error messages.

also some pep8.

* typo fixes
2016-10-25 09:15:58 -04:00
Konstantin Podshumok 9b2aac9e5c [MRG + 1] [TST] (half-cosmetic) use less nose.tools import to simplify future transition to py.test (#7384)
* use less nose.tools import to simplify future transition to activly developing test suites/runners

* assert_equal -> assert_array_equal in test_feature_hasher_pairs_with_string_values

and one missed ImportError that should be replaced with AttributeError

* test for py2.6 compat with except AttributeError

* fix importing of SkipTest

* force using nose in python2.6 for now

* there was no assert_dict_equal in py2.6. but we can use assert_equal

although failed test will look a little bit ugly

* remove nose imports from doc/datasets
2016-10-07 12:46:52 -04:00
Andreas Mueller 1dff0aec21 clean up deprecation warning stuff in common tests
minor fixes in preprocessing tests
2016-09-22 19:35:31 +05:30
Joel Nothman 0a0bf2478e Revert "ENH: Add indicator features to imputer output (#6607)" (#7292)
This reverts commit 18396be8cb as it was
merged when incomplete.
2016-08-30 21:28:01 +10:00
Andreas Mueller 002ff1237a fix some warnings in test outputs. 2016-08-27 17:02:22 +02:00
Caio Oliveira 19d6d925ed [MRG+1] Fix for "_transform_selected" not copying when "selected='all'" (#7201)
* fixes issue scikit-learn/scikit-learn#7194

* Added test

* Making `selected='all'` explicit on test

* Updated whats_new.rst

* Fixed typo on `whats_new.rst`
2016-08-18 11:47:00 +02:00
Konstantin Podshumok f893565773 [MRG+2] ENH: (minor) add quantile_low/high parameters to robust scaler (#5929)
* add quantile_low/high parameters to robust scaler

add versionadded tags

* robust scaler: add parameter for quantile_range, check its validity

* robust scaler: simplify params validations

fix typo in docstrings of RobustScaler

* preprocessing.data fix some lines > 80 columns

* add whatsnew for robust scaler quantile_range param

add missing url link in whatsnew
2016-08-13 10:32:16 +02:00
Gael Varoquaux 1dbc069cab TST: Speed up: cv=2
This is a smoke test. Hence there is no point having cv=4
2016-06-22 15:39:53 +02:00
John Moeller 055bc4c004 pep8 2016-06-22 02:10:41 -06:00
John Moeller 13f68c93b8 Simplifying imports and test 2016-06-17 17:44:51 -06:00
John Moeller cc9dbac37f Adding test for PR #6900 2016-06-17 17:28:36 -06:00
Thierry Guillemot 78a674875e Correct the deprecation of the random_integers numpy function. (#6712) 2016-04-26 16:08:48 +02:00
Maniteja Nandana 18396be8cb ENH: Add indicator features to imputer output 2016-04-14 12:21:16 +05:30
ningchi 1d487fb550 [MRG+1] issue #6532 Add `inverse_transform` function (#6570)
* [MRG+1] #6532 Add inverse_func argument to FunctionTransformer


* modify test:inverse_func is not true inverse
2016-04-12 01:26:30 -04:00
YenChenLin 08e1db4cd3 Test normalize function in data.py 2016-03-26 10:38:29 +08:00
dsquareindia e9492b7ec2 LabelBinarizer single label case now works for sparse and dense case 2016-03-21 10:43:56 +05:30
Andreas Mueller 8fb928d644 fixed doc for powers, added test 2016-02-24 17:08:15 -05:00
Andreas Mueller 897f8b6cc6 don't do ^1 2016-02-22 15:09:47 -05:00
Andreas Mueller 3fa684db0e add get_feature_names to PolynomialFeatures 2016-02-19 15:52:55 -05:00
dsquareindia 74475bc929 LabelEncoder now raises error for 0-D arrays 2016-01-26 01:34:26 +05:30
Gael Varoquaux 317dea8a05 Merge pull request #6005 from seales/SpellingFix
[MRG+1] General spelling fixes
2016-01-04 13:42:20 +01:00
Brian McFee 318b93d2ae Implemented keyword arguments in FunctionTransformer 2015-12-19 23:00:35 -05:00
seales 0485ada58b General spelling fixes 2015-12-16 09:46:42 -08:00
MechCoder 6e87813550 Scaling a sparse matrix along axis 0 should accept a csc by default 2015-11-11 14:21:49 -05:00
Andreas Mueller de3a527905 Merge pull request #5692 from amueller/skip_32_bit_tests
[MRG] Skip 32 bit tests that fail, skip doctests on 32bit
2015-11-05 10:33:33 -05:00
Andreas Mueller c55dc89902 Merge pull request #5695 from amueller/doc_fixes
[MRG] DOC some fixes to the doc build.
2015-11-03 15:24:56 -05:00
Andreas Mueller e2eba1ffbc Merge pull request #5688 from amueller/robust_scaler_1column_fix
[MRG+2] fix 1 sparse row scaling in robust scaler
2015-11-03 14:06:26 -05:00
Andreas Mueller 61df16e0e9 fix 1 sparse row scaling in robust scaler 2015-11-03 14:03:09 -05:00
Andreas Mueller 2653833a07 DOC some fixes to the doc build. 2015-11-03 12:23:23 -05:00
Andreas Mueller 776e53b127 skip unstable tests on 32bit platform 2015-11-02 15:21:22 -05:00
trevorstephens 89f8a514c3 OneHotEncoder warn fix
add regression test
2015-11-02 11:04:00 -08:00
Graham Clenaghan f69f895eee remove _balance_weights 2015-10-23 23:18:29 -07:00
Raghav R V 3f8743f47b Main Commits - Major
--------------------

* ENH Reogranize classes/fn from grid_search into search.py
* ENH Reogranize classes/fn from cross_validation into split.py
* ENH Reogranize cls/fn from cross_validation/learning_curve into validate.py

* MAINT Merge _check_cv into check_cv inside the model_selection module
* MAINT Update all the imports to point to the model_selection module
* FIX use iter_cv to iterate throught the new style/old style cv objs
* TST Add tests for the new model_selection members
* ENH Wrap the old-style cv obj/iterables instead of using iter_cv

* ENH Use scipy's binomial coefficient function comb for calucation of nCk
* ENH Few enhancements to the split module
* ENH Improve check_cv input validation and docstring
* MAINT _get_test_folds(X, y, labels) --> _get_test_folds(labels)
* TST if 1d arrays for X introduce any errors
* ENH use 1d X arrays for all tests;
* ENH X_10 --> X (global var)

Minor
-----

* ENH _PartitionIterator --> _BaseCrossValidator;
* ENH CVIterator --> CVIterableWrapper
* TST Import the old SKF locally
* FIX/TST Clean up the split module's tests.
* DOC Improve documentation of the cv parameter
* COSMIT consistently hyphenate cross-validation/cross-validator
* TST Calculate n_samples from X
* COSMIT Use separate lines for each import.
* COSMIT cross_validation_generator --> cross_validator

Commits merged manually
-----------------------

* FIX Document the random_state attribute in RandomSearchCV
* MAINT Use check_cv instead of _check_cv
* ENH refactor OVO decision function, use it in SVC for sklearn-like
  decision_function shape
* FIX avoid memory cost when sampling from large parameter grids

ENH Major to Minor incremental enhancements to the model_selection

Squashed commit messages - (For reference)

Major
-----

* ENH p --> n_labels
* FIX *ShuffleSplit: all float/invalid type errors at init and int error at split
* FIX make PredefinedSplit accept test_folds in constructor; Cleanup docstrings
* ENH+TST KFold: make rng to be generated at every split call for reproducibility
* FIX/MAINT KFold: make shuffle a public attr
* FIX Make CVIterableWrapper private.
* FIX reuse len_cv instead of recalculating it
* FIX Prevent adding *SearchCV estimators from the old grid_search module
* re-FIX In all_estimators: the sorting to use only the 1st item (name)
    To avoid collision between the old and the new GridSearch classes.
* FIX test_validate.py: Use 2D X (1D X is being detected as a single sample)
* MAINT validate.py --> validation.py
* MAINT make the submodules private
* MAINT Support old cv/gs/lc until 0.19
* FIX/MAINT n_splits --> get_n_splits
* FIX/TST test_logistic.py/test_ovr_multinomial_iris:
    pass predefined folds as an iterable
* MAINT expose BaseCrossValidator
* Update the model_selection module with changes from master
  - From #5161
  -  - MAINT remove redundant p variable
  -  - Add check for sparse prediction in cross_val_predict
  - From #5201 - DOC improve random_state param doc
  - From #5190 - LabelKFold and test
  - From #4583 - LabelShuffleSplit and tests
  - From #5300 - shuffle the `labels` not the `indxs` in LabelKFold + tests
  - From #5378 - Make the GridSearchCV docs more accurate.
  - From #5458 - Remove shuffle from LabelKFold
  - From #5466(#4270) - Gaussian Process by Jan Metzen
  - From #4826 - Move custom error / warnings into sklearn.exception

Minor
-----

* ENH Make the KFold shuffling test stronger
* FIX/DOC Use the higher level model_selection module as ref
* DOC in check_cv "y : array-like, optional"
* DOC a supervised learning problem --> supervised learning problems
* DOC cross-validators --> cross-validation strategies
* DOC Correct Olivier Grisel's name ;)
* MINOR/FIX cv_indices --> kfold
* FIX/DOC Align the 'See also' section of the new KFold, LeaveOneOut
* TST/FIX imports on separate lines
* FIX use __class__ instead of classmethod
* TST/FIX import directly from model_selection
* COSMIT Relocate the random_state documentation
* COSMIT remove pass
* MAINT Remove deprecation warnings from old tests
* FIX correct import at test_split
* FIX/MAINT Move P_sparse, X, y defns to top; rm unused W_sparse, X_sparse
* FIX random state to avoid doctest failure
* TST n_splits and split wrapping of _CVIterableWrapper
* FIX/MAINT Use multilabel indicator matrix directly
* TST/DOC clarify why we conflate classes 0 and 1
* DOC add comment that this was taken from BaseEstimator
* FIX use of labels is not needed in stratified k fold
* Fix cross_validation reference
* Fix the labels param doc

FIX/DOC/MAINT Addressing the review comments by Arnaud and Andy

COSMIT Sort the members alphabetically
COSMIT len_cv --> n_splits
COSMIT Merge 2 if; FIX Use kwargs
DOC Add my name to the authors :D
DOC make labels parameter consistent
FIX Remove hack for boolean indices; + COSMIT idx --> indices; DOC Add Returns
COSMIT preds --> predictions
DOC Add Returns and neatly arrange X, y, labels
FIX idx(s)/ind(s)--> indice(s)
COSMIT Merge if and else to elif
COSMIT n --> n_samples
COSMIT Use bincount only once
COSMIT cls --> class_i / class_i (ith class indices) -->
perm_indices_class_i

FIX/ENH/TST Addressing the final reviews

COSMIT c --> count
FIX/TST make check_cv raise ValueError for string cv value
TST nested cv (gs inside cross_val_score) works for diff cvs
FIX/ENH Raise ValueError when labels is None for label based cvs;
TST if labels is being passed correctly to the cv and that the
ValueError is being propagated to the cross_val_score/predict and grid
search
FIX pass labels to cross_val_score
FIX use make_classification
DOC Add Returns; COSMIT Remove scaffolding
TST add a test to check the _build_repr helper
REVERT the old GS/RS should also be tested by the common tests.
ENH Add a tuple of all/label based CVS
FIX raise VE even at get_n_splits if labels is None
FIX Fabian's comments
PEP8
2015-10-23 17:28:08 +02:00
Olivier Grisel 718a7df4c5 Merge pull request #5449 from Jeffrey04/5433-MaxAbsScaler-1-row-csr-fix
[MRG + 1] max abs scaler 1 row csr fix
2015-10-21 11:42:27 +02:00
Raghav R V e3afc0e8c9 MAINT move custom error/warning classes into sklearn.exceptions
ENH NonBLASDotWarning -> EfficiencyWarning; Improve error message
DOC Add exceptions module to modules/classes.rst
MAINT Move ConvergenceWarning, UndefinedMetricWarning et al into exceptions
MAINT Remove ChangedBehaviorWarning from base
DOC/FIX Improve DataConversionWarning's docstring
2015-10-19 22:35:35 +02:00
Jeffrey04 2571158168 add test for inverse_transform 2015-10-19 23:29:09 +08:00
Jeffrey04 658129ac5b updated test with tips from @giorgiop 2015-10-19 23:10:41 +08:00
Jeffrey04 e7a3675eaf the test to reflect issue #5433 2015-10-19 19:29:00 +08:00
giorgiop c7b1a6ebc3 BUG: reset internal state of scaler before fitting 2015-10-16 17:28:40 +02:00
Loic Esteve 47f19e9692 MAINT add safe_{median|mean} for np 1.10.1 2015-10-15 18:21:52 +02:00