Commit Graph

229 Commits

Author SHA1 Message Date
Fibinse Xavier` 9928713ae5 DOC on parallelisation of custom scorer (#12813) 2019-01-15 18:48:45 +11:00
Nicolas Hug a50c03f975 [MRG] Add pprint for estimators - continued (#11705)
* add pprint for estimators

* strip color from length, add color option

* Minor cleaning, fixes, factoring and docs

* Added some basic tests

* Fixed line length issue

* fixed flake8 and added visual test for review

* Fixed test

* Fixed Python 2 issues (inspect.signature import)

* Trying to fix flake8 again

* Added special repr for functions

* Added some other visual tests

* Changed _format_function in to _format_callable

because callable() returns True also for class objects (which we want to
reprensent with their name as well anyway)

* Consistent output in Python 2 and 3

* WIP

* Now using the builtin pprint module

* pep8

* Added changed_only param

* Fixed printing when string would fit in less than line width

* Fixed printing of steps parameter

* Fixed changed_only param for short estimators

* fixed pep8

* Added some more description in docstring

* changed_only is now an option from set_config()

* Put _pprint.py into sklearn/utils, added tests

* Added doctest NORMALIZE_WHITESPACE where needed

* Fixed tests

* fix test-doc

* fixing test that passed before....

* Fixed tests

* Added test for changed_only and long lines

* typo

* Added authors names

* Added license file

* Added ellipsis based on number of elements in sequence + added increasinly aggressive repr strategies

* Updated whatsnew

* dont use increaingly aggressive strategy

* Fixed tests

* Removed LICENSE file and put license text in _pprint.py

* fixed test_base

* Sorted parameters dictionary for consistent output in 3.5

* Actually using OrderedDict...

* Addressed comments

* Added test for NaN changed parameter

* Update whatsnew

* Added example to set_config()

* Removed example

* Added example in gallery

* Spelling
2018-12-19 23:48:21 -05:00
adanhawth ff64eae031 DOC Fix LaTeX text{} block escaping of underscores in doc-modules-model_evaluation.rst (#12788) 2018-12-16 09:07:03 +08:00
Emmanuel Arias 72ba30060a ENH Use accuracy instead of micro-average in classification_report (#12758) 2018-12-13 21:00:40 +11:00
Hanmin Qin 7fe3413475 Revert "DOC Fix math typo in model_evaluation's explained_variance (#12747)"
This reverts commit bf2f648639.
2018-12-10 19:53:03 +08:00
Adrin Jalali bf2f648639 DOC Fix math typo in model_evaluation's explained_variance (#12747) 2018-12-10 10:21:44 +08:00
Joel Nothman 654c452291 DOC Revert "DOC Use defined notation for precision and recall (#12726)" (#12743) 2018-12-09 19:42:35 +08:00
Greg Gandenberger 1cb56ba882 DOC Use defined notation for precision and recall (#12726) 2018-12-06 09:10:23 +08:00
Shangwu Yao 65556317c9 FEA (0.21) multilabel confusion matrix (#11179) 2018-10-30 19:26:37 +08:00
Krishna Sangeeth 831c760ccf ENH (0.21) Add max_error to the existing set of metrics for regression (#12232) 2018-10-11 15:33:06 +08:00
TakingItCasual f4e7d2b19a Converting http to https (3)... (#12302) 2018-10-05 18:50:31 +02:00
Denis Kataev b020f62250 MNT Apply pep8 to docs code (#12275) 2018-10-05 22:51:42 +08:00
TakingItCasual 1e052e9da9 Converting http to https (2)... (#12292) 2018-10-04 23:06:14 +02:00
TakingItCasual 74b56dbc57 MNT Converting http to https (#12277) 2018-10-04 22:37:57 +08:00
Hanmin Qin 0efaf2d040
ENH Adding various averages to classification_report (#11679) 2018-08-07 19:54:07 +08:00
Hanmin Qin 825cf09a1e ENH Simplify error message of get_scorer (#11738) 2018-08-06 12:30:35 +03:00
Hanmin Qin 9c306c070f Revert "DOC n_classes -> #classes (#11702)"
This reverts commit 89b25a586e.
2018-07-29 22:18:12 +08:00
Joel Nothman 89b25a586e
DOC n_classes -> #classes (#11702) 2018-07-29 23:05:44 +10:00
Joel Nothman 54dc0d730f DOC escape underscores in TeX 2018-07-29 13:48:58 +10:00
Joel Nothman e888c0d65c ENH multiclass balanced accuracy (#10587) 2018-07-27 15:21:46 +08:00
Alexandre Boucaud f158e2dfe2 [MRG+1] Change CV defaults to 5 (#11557)
* add FutureWarning for methods with defaults=3

* add explicit cv values to fix assertion errors

* add tests for catching the FutureWarning

* Write current deprecation version

* Add deprecation in docstring

* change default cv value to None

* change cv from 3 to 5 in the examples

* upgrade doctests

* update doctest in tutorial

* update doctest in cross-validation doc

* fix tests

* add entry to whats new

* address Gael comments

* address Gael comments 2

* fix wrong indentation

* update doc

* add docstring deprecation warning in CV subclasses

* address Andy's comments

* fix PR number

* fix flake8

* add filterwarnings in tests

* fix doctests

* cv=None mendatory in Ridge

* fix warning related errors

* skip some doctests warnings

* make travis happy

* change from deprecated to versionchanged

* fix doctests and remove skipping

* address comments
2018-07-19 14:46:11 +02:00
Georgi Peev 91bfca6938 DOC: reduce the tolerance in SV examples to get cross-platform stability (#11289) 2018-07-14 21:19:36 +02:00
Narendran Santhanam 6f9a8dfd54 MNT replace log with log1p (#11424) 2018-07-05 09:13:48 +10:00
Max Copeland 8083ea4a21 DOC Clarified variation in confusion_matrix axes (#11334)
fixes #11326
2018-06-26 10:56:01 +10:00
Loïc Estève 1c61b8ac1b DOC Fix some example refs due to renaming examples (#11214) 2018-06-07 10:40:30 +08:00
Andreas Mueller f97b515431 DOC fix sentence about micro being equal, add accuracy (#10705) 2018-05-30 21:57:13 +08:00
Hanmin Qin 399f1b2761
FIX Correct iris dataset (#11082) 2018-05-22 12:56:57 +08:00
Loïc Estève 20661b5018
TST: only run doctests on numpy 1.14. (#10835)
Fix doctests due to numpy 1.14 formatting changes.
2018-03-27 07:44:40 +02:00
Gaurav Dhingra eed83790b0 ENH Change default gamma from 'auto' to 'scale' in SVC (#10331) 2018-03-10 10:39:21 +08:00
Alexander-N 02ddc70ee6 ENH Partial AUC (#3840) 2018-03-02 09:33:05 +11:00
Albert Thomas 28a2ad265f [MRG] fix bad naming in make scorer example (#10651) 2018-02-18 22:56:27 +11:00
Joel Nothman 6919a22c80 DOC ReST markup 2018-02-05 16:04:50 +11:00
martin-kokos 764abb6bc0 DOC Fix confusion_matrix documentation example (#10519) 2018-01-25 09:31:03 +08:00
Franz Wompner 1e1417cab2 DOC Fix typo in model_evaluation.rst (#10376) 2017-12-27 23:48:08 +08:00
Nihar Sheth cec3458b03 DOC Fix broken links in docs (#10353) 2017-12-22 18:23:29 +01:00
Charlie Brummitt 6b2dc04d22 [MRG + 1] DOC Fix cardinality vs. l0 norm in multilabel ranking user guide (#10219) 2017-11-30 15:10:18 +01:00
Mohamed Maskani 653de6ca54 Balanced accuracy doc - 2 (#10040)
* Add references for multiclass balanced-accuracy

* Add precision not implemented

* Add another reference

* Adjust note

* Add succint definitions

* Add macro-average recall implementation

* Add class balance accuracy reference and definition

* Fix typo

* Add precision/recall comparison

* Fix typo

* Make descriptions less verbose and merge definitions

* Add reference for averaging accuracy

* Move macro-average recall example to corresponding section
2017-11-15 09:42:47 -08:00
Hanmin Qin 3e85359f59 [MRG] Ensure that ROC curve starts at (0, 0) (#10093) 2017-11-10 14:14:03 +11:00
Mohamed Maskani 9172a5905d DOC Add references for multiclass balanced-accuracy definitions (#9982) 2017-10-30 10:06:21 +11:00
Aman Dalmia 8daad062f1 [MRG+1] Adding support for balanced accuracy (#8066)
* add function computing balanced accuracy

* documentation for the balanced_accuracy_score

* apply common tests to balanced_accuracy_score

* constrained to binary classification problems only

* add balanced_accuracy_score for CLF test

* add scorer for balanced_accuracy

* reorder the place of importing balanced_accuracy_score to be consistent with others

* eliminate an accidentally added non-ascii character

* remove balanced_accuracy_score from METRICS_WITH_LABELS

* eliminate all non-ascii charaters in the doc of balanced_accuracy_score

* fix doctest for nonexistent scoring function

* fix documentation, clarify linkages to recall and auc

* FIX: added changes as per last review See #6752, fixes #6747

* FIX: fix typo

* FIX: remove flake8 errors

* DOC: merge fixes

* DOC: remove unwanted files

* DOC update what's new
2017-10-17 11:12:27 -04:00
Joel Nothman f05a95b103 [MRG] FIX Revert the addition of ndcg_score and dcg_score (#9932) 2017-10-17 09:34:25 +11:00
Anthony Gitter 59bb32ff71 DOC Add average precision definitions and cross references (#9583) 2017-09-25 20:04:21 +10:00
Hanmin Qin ee2025f282 [MRG+1] Add scorer based on brier_score_loss (#9521) 2017-08-22 08:27:41 +10:00
Hanmin Qin 9f91ec7ffd [MRG+1] add scorer based on explained_variance_score (#9259) 2017-08-08 10:02:15 +02:00
(Venkat) Raghav, Rajagopalan 02a31f0a89 [MRG] DOC use def instead of lambda in the multimetric example at model_evaluation.rst (#9442) 2017-07-26 09:32:35 +02:00
Andreas Mueller c71e1275ea minor sphinx fixes (#9370) 2017-07-15 16:09:06 -07:00
Tom Dupré la Tour e55d0bcba5 FIX broken link in gallery and bad title rendering 2017-07-12 16:37:49 +02:00
(Venkat) Raghav, Rajagopalan a08555a238 [MRG + 2] ENH Allow `cross_val_score`, `GridSearchCV` et al. to evaluate on multiple metrics (#7388)
* ENH cross_val_score now supports multiple metrics

* DOCFIX permutation_test_score

* ENH validate multiple metric scorers

* ENH Move validation of multimetric scoring param out

* ENH GridSearchCV and RandomizedSearchCV now support multiple metrics

* EXA Add an example demonstrating the multiple metric in GridSearchCV

* ENH Let check_multimetric_scoring tell if its multimetric or not

* FIX For single metric name of scorer should remain 'score'

* ENH validation_curve and learning_curve now support multiple metrics

* MNT move _aggregate_score_dicts helper into _validation.py

* TST More testing/ Fixing scores to the correct values

* EXA Add cross_val_score to multimetric example

* Rename to multiple_metric_evaluation.py

* MNT Remove scaffolding

* FIX doctest imports

* FIX wrap the scorer and unwrap the score when using _score() in rfe

* TST Cleanup the tests. Test for is_multimetric too

* TST Make sure it registers as single metric when scoring is of that type

* PEP8

* Don't use dict comprehension to make it work in python2.6

* ENH/FIX/TST grid_scores_ should not be available for multimetric evaluation

* FIX+TST delegated methods NA when multimetric is enabled...

TST Add general tests to GridSearchCV and RandomizedSearchCV

* ENH add option to disable delegation on multimetric scoring

* Remove old function from __all__

* flake8

* FIX revert disable_on_multimetric

* stash

* Fix incorrect rebase

* [ci skip]

* Make sure refit works as expected and remove irrelevant tests

* Allow passing standard scorers by name in multimetric scorers

* Fix example

* flake8

* Address reviews

* Fix indentation

* Ensure {'acc': 'accuracy'} and ['precision'] are valid inputs

* Test that for single metric, 'score' is a key

* Typos

* Fix incorrect rebase

* Compare multimetric grid search with multiple single metric searches

* Test X, y list and pandas input; Test multimetric for unsupervised grid search

* Fix tests; Unsupervised multimetric gs will not pass until #8117 is merged

* Make a plot of Precision vs ROC AUC for RandomForest varying the n_estimators

* Add example to grid_search.rst

* Use the classic tuning of C param in SVM instead of estimators in RF

* FIX Remove scoring arg in deafult scorer test

* flake8

* Search for min_samples_split in DTC; Also show f-score

* REVIEW Make check_multimetric_scoring private

* FIX Add more samples to see if 3% mismatch on 32 bit systems gets fixed

* REVIEW Plot best score; Shorten legends

* REVIEW/COSMIT multimetric --> multi-metric

* REVIEW Mark the best scores of P/R scores too

* Revert "FIX Add more samples to see if 3% mismatch on 32 bit systems gets fixed"

This reverts commit ba766d98353380a186fbc3dade211670ee72726d.

* ENH Use looping for iid testing

* FIX use param grid as scipy's stats dist in 0.12 do not accept seed

* ENH more looping less code; Use small non-noisy dataset

* FIX Use named arg after expanded args

* TST More testing of the refit parameter

* Test that in multimetric search refit to single metric, the delegated methods
  work as expected.
* Test that setting probability=False works with multimetric too
* Test refit=False gives sensible error

* COSMIT multimetric --> multi-metric

* REV Correct example doc

* COSMIT

* REVIEW Make tests stronger; Fix bugs in _check_multimetric_scorer

* REVIEW refit param: Raise for empty strings

* TST Invalid refit params

* REVIEW Use <scorer_name> alone; recall --> Recall

* REV specify when we expect scorers to not be None

* FLAKE8

* REVERT multimetrics in learning_curve and validation_curve

* REVIEW Simpler coding style

* COSMIT

* COSMIT

* REV Compress example a bit. Move comment to top

* FIX fit_grid_point's previous API must be preserved

* Flake8

* TST Use loop; Compare with single-metric

* REVIEW Use dict-comprehension instead of helper

* REVIEW Remove redundant test

* Fix tests incorrect braces

* COSMIT

* REVIEW Use regexp

* REV Simplify aggregation of score dicts

* FIX precision and accuracy test

* FIX doctest and flake8

* TST the best_* attributes multimetric with single metric

* Address @jnothman's review

* Address more comments \o/

* DOCFIXES

* Fix use the validated fit_param from fit's arguments

* Revert alpha to a lower value as before

* Using def instead of lambda

* Address @jnothman's review batch 1: Fix tests / Doc fixes

* Remove superfluous tests

* Remove more superfluous testing

* TST/FIX loop over refit and check found n_clusters

* Cosmetic touches

* Use zip instead of manually listing the keys

* Fix inverse_transform

* FIX bug in fit_grid_point; Allow only single score

TST if fit_grid_point works as intended

* ENH Use only ROC-AUC and F1-score

* Fix typos and flake8; Address Andy's reviews

MNT Add a comment on why we do such a transpose + some fixes

* ENH Better error messages for incorrect multimetric scoring values +...

ENH Avoid exception traceback while using incorrect scoring string

* Dict keys must be of string type only

* 1. Better error message for invalid scoring 2...
Internal functions return single score for single metric scoring

* Fix test failures and shuffle tests

* Avoid wrapping scorer as dict in learning_curve

* Remove doc example as asked for

* Some leftover ones

* Don't wrap scorer in validation_curve either

* Add a doc example and skip it as dict order fails doctest

* Import zip from six for python2.7 compat

* Make cross_val_score return a cv_results-like dict

* Add relevant sections to userguide

* Flake8 fixes

* Add whatsnew and fix broken links

* Use AUC and accuracy instead of f1

* Fix failing doctests cross_validation.rst

* DOC add the wrapper example for metrics that return multiple return values

* Address andy's comments

* Be less weird

* Address more of andy's comments

* Make a separate cross_validate function to return dict and a cross_val_score

* Update the docs to reflect the new cross_validate function

* Add cross_validate to toc-tree

* Add more tests on type of cross_validate return and time limits

* FIX failing doctests

* FIX ensure keys are not plural

* DOC fix

* Address some pending comments

* Remove the comment as it is irrelevant now

* Remove excess blank line

* Fix flake8 inconsistencies

* Allow fit_times to be 0 to conform with windows precision

* DOC specify how refit param is to be set in multiple metric case

* TST ensure cross_validate works for string single metrics + address @jnothman's reviews

* Doc fixes

* Remove the shape and transform parameter of _aggregate_score_dicts

* Address Joel's doc comments

* Fix broken doctest

* Fix the spurious file

* Address Andy's comments

* MNT Remove erroneous entry

* Address Andy's comments

* FIX broken links

* Update whats_new.rst

missing newline
2017-07-07 11:12:31 -04:00
Taehoon Lee 5d7df621a1 Fix typos (#9265) 2017-07-03 12:27:53 +02:00
Joel Nothman 9996491e1a DOC wording 2017-07-01 21:05:55 +10:00