Commit Graph

72 Commits

Author SHA1 Message Date
Guillaume Lemaitre 89b02af431 [MRG+1] EHN Accept 1D array for preprocessing functions and update doc (#9596)
* EHN/TST robust_scale accepts 1D array

* DOC update doc for preprocessing functions
2017-08-22 19:08:47 +03:00
Guillaume Lemaitre 26a1027a83 [MRG+1] QuantileTransformer (#8363)
* resurrect quantile scaler

* move the code in the pre-processing module

* first draft

* Add tests.

* Fix bug in QuantileNormalizer.

* Add quantile_normalizer.

* Implement pickling

* create a specific function for dense transform

* Create a fit function for the dense case

* Create a toy examples

* First draft with sparse matrices

* remove useless functions and non-negative sparse compatibility

* fix slice call

* Fix tests of QuantileNormalizer.

* Fix estimator compatibility

* List of functions became tuple of functions
* Check X consistency at transform and inverse transform time

* fix doc

* Add negative ValueError tests for QuantileNormalizer.

* Fix cosmetics

* Fix compatibility numpy <= 1.8

* Add n_features tests and correct ValueError.

* PEP8

* fix fill_value for early scipy compatibility

* simplify sampling

* Fix tests.

* removing last pring

* Change choice for permutation

* cosmetics

* fix remove remaining choice

* DOC

* Fix inconsistencies

* pep8

* Add checker for init parameters.

* hack bounds and make a test

* FIX/TST bounds are provided by the fitting and not X at transform

* PEP8

* FIX/TST axis should be <= 1

* PEP8

* ENH Add parameter ignore_implicit_zeros

* ENH match output distribution

* ENH clip the data to avoid infinity due to output PDF

* FIX ENH restraint to uniform and norm

* [MRG] ENH Add example comparing the distribution of all scaling preprocessor (#2)

* ENH Add example comparing the distribution of all scaling preprocessor

* Remove Jupyter notebook convert

* FIX/ENH Select feat before not after; Plot interquantile data range for all

* Add heatmap legend

* Remove comment maybe?

* Move doc from robust_scaling to plot_all_scaling; Need to update doc

* Update the doc

* Better aesthetics; Better spacing and plot colormap only at end

* Shameless author re-ordering ;P

* Use env python for she-bang

* TST Validity of output_pdf

* EXA Use OrderedDict; Make it easier to add more transformations

* FIX PEP8 and replace scipy.stats by str in example

* FIX remove useless import

* COSMET change variable names

* FIX change output_pdf occurence to output_distribution

* FIX partial fixies from comments

* COMIT change class name and code structure

* COSMIT change direction to inverse

* FIX factorize transform in _transform_col

* PEP8

* FIX change the magic 10

* FIX add interp1d to fixes

* FIX/TST allow negative entries when ignore_implicit_zeros is True

* FIX use np.interp instead of sp.interpolate.interp1d

* FIX/TST fix tests

* DOC start checking doc

* TST add test to check the behaviour of interp numpy

* TST/EHN Add the possibility to add noise to compute quantile

* FIX factorize quantile computation

* FIX fixes issues

* PEP8

* FIX/DOC correct doc

* TST/DOC improve doc and add random state

* EXA add examples to illustrate the use of smoothing_noise

* FIX/DOC fix some grammar

* DOC fix example

* DOC/EXA make plot titles more succint

* EXA improve explanation

* EXA improve the docstring

* DOC add a bit more documentation

* FIX advance review

* TST add subsampling test

* DOC/TST better example for the docstring

* DOC add ellipsis to docstring

* FIX address olivier comments

* FIX remove random_state in sparse.rand

* FIX spelling doc

* FIX cite example in user guide and docstring

* FIX olivier comments

* EHN improve the example comparing all the pre-processing methods

* FIX/DOC remove title

* FIX change the scaling of the figure

* FIX plotting layout

* FIX ratio w/h

* Reorder and reword the plot_all_scaling example

* Fix aspect ratio and better explanations in the plot_all_scaling.py example

* Fix broken link and remove useless sentence

* FIX fix couples of spelling

* FIX comments joel

* FIX/DOC address documentation comments

* FIX address comments joel

* FIX inline sparse and dense transform

* PEP8

* TST/DOC temporary skipping test

* FIX raise an error if n_quantiles > subsample

* FIX wording in smoothing_noise example

* EXA Denis comments

* FIX rephrasing

* FIX make smoothing_noise to be a boolearn and change doc

* FIX address comments

* FIX verbose the doc slightly more

* PEP8/DOC

* ENH: 2-ways interpolation to avoid smoothing_noise

Simplifies also the code, examples, and documentation
2017-06-10 01:15:46 +02:00
Andreas Mueller 5c4b1bb231 [MRG+1] Housekeeping Deprecations for v0.19 (#7927)
* remove stuff to be removed 0.19

* more changes

* remove classes from 0.19 whatsnew

* remove _LearntSelectorMixin

* remove ProjectedGradientNMF, load_lwf_*

* minor fixes

* remove more copy from logistic regression path

* remove lda, qda from __init__.__all__

* remove pg solver in nmf from tests etc

* remove class_weight="auto" from tests

* doctest change for decision_function_shape="ovr"

* remove transfrom from tree test, minor fixes to tree tests

* some fixes in the tests

* undo changes in functions which still allow 1d input...

* also allow 1d in scale

* more test fixes...

* last test fixes in forest and tree

* svm default value change doctest failures

* pep8

* remove more class_weight="auto" stuff

* minor cosmetics in docstrings deprecated / removed behavior.

* say that store_covariance has been moved to __init__ in discriminant_analysis
2016-12-09 12:43:38 -05:00
Ang Lu c17156106c [MRG+1] Fix return_norm bug in preprocessing.normalize (#7789)
Fixes #7771
2016-11-23 09:45:32 +11:00
Andreas Mueller 1dff0aec21 clean up deprecation warning stuff in common tests
minor fixes in preprocessing tests
2016-09-22 19:35:31 +05:30
Andreas Mueller 002ff1237a fix some warnings in test outputs. 2016-08-27 17:02:22 +02:00
Caio Oliveira 19d6d925ed [MRG+1] Fix for "_transform_selected" not copying when "selected='all'" (#7201)
* fixes issue scikit-learn/scikit-learn#7194

* Added test

* Making `selected='all'` explicit on test

* Updated whats_new.rst

* Fixed typo on `whats_new.rst`
2016-08-18 11:47:00 +02:00
Konstantin Podshumok f893565773 [MRG+2] ENH: (minor) add quantile_low/high parameters to robust scaler (#5929)
* add quantile_low/high parameters to robust scaler

add versionadded tags

* robust scaler: add parameter for quantile_range, check its validity

* robust scaler: simplify params validations

fix typo in docstrings of RobustScaler

* preprocessing.data fix some lines > 80 columns

* add whatsnew for robust scaler quantile_range param

add missing url link in whatsnew
2016-08-13 10:32:16 +02:00
Gael Varoquaux 1dbc069cab TST: Speed up: cv=2
This is a smoke test. Hence there is no point having cv=4
2016-06-22 15:39:53 +02:00
John Moeller 055bc4c004 pep8 2016-06-22 02:10:41 -06:00
John Moeller 13f68c93b8 Simplifying imports and test 2016-06-17 17:44:51 -06:00
John Moeller cc9dbac37f Adding test for PR #6900 2016-06-17 17:28:36 -06:00
Thierry Guillemot 78a674875e Correct the deprecation of the random_integers numpy function. (#6712) 2016-04-26 16:08:48 +02:00
YenChenLin 08e1db4cd3 Test normalize function in data.py 2016-03-26 10:38:29 +08:00
Andreas Mueller 8fb928d644 fixed doc for powers, added test 2016-02-24 17:08:15 -05:00
Andreas Mueller 897f8b6cc6 don't do ^1 2016-02-22 15:09:47 -05:00
Andreas Mueller 3fa684db0e add get_feature_names to PolynomialFeatures 2016-02-19 15:52:55 -05:00
MechCoder 6e87813550 Scaling a sparse matrix along axis 0 should accept a csc by default 2015-11-11 14:21:49 -05:00
Andreas Mueller de3a527905 Merge pull request #5692 from amueller/skip_32_bit_tests
[MRG] Skip 32 bit tests that fail, skip doctests on 32bit
2015-11-05 10:33:33 -05:00
Andreas Mueller c55dc89902 Merge pull request #5695 from amueller/doc_fixes
[MRG] DOC some fixes to the doc build.
2015-11-03 15:24:56 -05:00
Andreas Mueller e2eba1ffbc Merge pull request #5688 from amueller/robust_scaler_1column_fix
[MRG+2] fix 1 sparse row scaling in robust scaler
2015-11-03 14:06:26 -05:00
Andreas Mueller 61df16e0e9 fix 1 sparse row scaling in robust scaler 2015-11-03 14:03:09 -05:00
Andreas Mueller 2653833a07 DOC some fixes to the doc build. 2015-11-03 12:23:23 -05:00
Andreas Mueller 776e53b127 skip unstable tests on 32bit platform 2015-11-02 15:21:22 -05:00
trevorstephens 89f8a514c3 OneHotEncoder warn fix
add regression test
2015-11-02 11:04:00 -08:00
Olivier Grisel 718a7df4c5 Merge pull request #5449 from Jeffrey04/5433-MaxAbsScaler-1-row-csr-fix
[MRG + 1] max abs scaler 1 row csr fix
2015-10-21 11:42:27 +02:00
Raghav R V e3afc0e8c9 MAINT move custom error/warning classes into sklearn.exceptions
ENH NonBLASDotWarning -> EfficiencyWarning; Improve error message
DOC Add exceptions module to modules/classes.rst
MAINT Move ConvergenceWarning, UndefinedMetricWarning et al into exceptions
MAINT Remove ChangedBehaviorWarning from base
DOC/FIX Improve DataConversionWarning's docstring
2015-10-19 22:35:35 +02:00
Jeffrey04 2571158168 add test for inverse_transform 2015-10-19 23:29:09 +08:00
Jeffrey04 658129ac5b updated test with tips from @giorgiop 2015-10-19 23:10:41 +08:00
Jeffrey04 e7a3675eaf the test to reflect issue #5433 2015-10-19 19:29:00 +08:00
giorgiop c7b1a6ebc3 BUG: reset internal state of scaler before fitting 2015-10-16 17:28:40 +02:00
Raghav R V 4f4c3598e6 FIX dtypes to conform to the stricter type cast rules of numpy 1.10
FIX set copy to (copy & whiten).

FIX/DOC Use float outputs for doctest
2015-10-15 18:21:51 +02:00
giorgiop 6a5a2f7960 partial_fit for scalers 2015-10-13 11:37:00 +02:00
Vighnesh Birodkar 2f099334f9 MAINT deprecate 1d input arrays for all estimators
Passing 1D arrays to check_array, without setting `ensure_2d` to false now
raises a deprecation warning before reshaping it. This will later throw an
error.

All Scaler classes also throw warnings when 1D arrays are passed.

All unit tests/doctests are modified to ensure that no 1D arrays are passed,
except in explicit 1D array tests where the warnings have been silenced.

Additional tests are also included which check for different 1D array cases.

2D array tests with one samples and one features are also added and where
they failed, `check_array` call has been modified to give a more useful error
message
2015-09-09 15:49:58 +02:00
Thomas Unterthiner 26fbfe6f2e ENH add minmax_scale 2015-07-01 23:27:35 +02:00
Thomas Unterthiner ab734b7a39 ENH add MaxAbsScaler 2015-06-11 16:08:17 +02:00
Andreas Mueller 0b07536990 Merge pull request #4125 from untom/RobustScaler
[MRG + 2] ENH RobustScaler
2015-05-26 18:02:41 -04:00
TomDLT 89c1018c64 ENH improve check_array
ENH improve check_array to warn on dtype conversions

ENH make check_array accept several dtypes

ENH change validation with improved check_array

ENH change astype to avoid copy if possible

ENH remove warn_if_not_float
2015-05-19 16:13:42 +02:00
Thomas Unterthiner d86a629ac7 Added robust scaling example 2015-05-16 11:15:32 +02:00
Thomas Unterthiner dd2fdbb18c Removed superfluous testcase for robust scaler 2015-05-16 01:50:56 +02:00
Thomas Unterthiner e5fb9dd5a4 Changed outdated test 2015-05-16 00:50:25 +02:00
Thomas Unterthiner 4ff47d9efa Fix documentation in test case 2015-05-15 23:51:17 +02:00
CJ Carey 007ae76cb3 TST: covering norm='max' branches of normalize() 2015-05-08 14:20:54 -04:00
Andreas Mueller 574ebfda85 Merge pull request #4436 from ogrisel/rebased-pr-3747
[MRG+2] FIX make StandardScaler & scale more numerically stable
2015-03-23 11:12:32 -05:00
Olivier Grisel b596dfcfef FIX test for older versions of numpy 2015-03-23 16:41:18 +01:00
Nicolas 04078be7da FIX make StandardScaler & scale more numerically stable
Detect common numerical stability problems in the standardization
routines and try simple mitigation strategies while alos issuing
a warning.
2015-03-23 14:32:33 +01:00
Raghav R V cd2ee7e454 MAINT docstring --> comments to prevent nose from using doc in verbose mode 2015-03-21 11:16:49 +05:30
Eric Martin ed47b5b744 Directly compute polynomial features.
Polynomial features are computed by iterating over all combinations
of features. For each combination of features, the product of the
columns indexed by the combination is computed.

The fit method is now a no-op, and the transform method works with any
number of features (regardless of what fit was called with).
2015-03-18 13:07:15 +11:00
Thomas Unterthiner 22f718dcce ENH Added RobustScaler 2015-01-19 21:13:31 +01:00
Raghav R V 48e72b5559 TST Add test to check if ValueError is raised if input contains NaN 2015-01-16 19:13:31 +05:30