1383 lines
54 KiB
ReStructuredText
1383 lines
54 KiB
ReStructuredText
.. _contributing:
|
|
|
|
============
|
|
Contributing
|
|
============
|
|
|
|
This project is a community effort, and everyone is welcome to
|
|
contribute.
|
|
|
|
The project is hosted on https://github.com/scikit-learn/scikit-learn
|
|
|
|
Scikit-learn is somewhat :ref:`selective <selectiveness>` when it comes to
|
|
adding new algorithms, and the best way to contribute and to help the project
|
|
is to start working on known issues.
|
|
See :ref:`new_contributors` to get started.
|
|
|
|
.. topic:: **Our community, our values**
|
|
|
|
We are a community based on openness and friendly, didactic,
|
|
discussions.
|
|
|
|
We aspire to treat everybody equally, and value their contributions.
|
|
|
|
Decisions are made based on technical merit and consensus.
|
|
|
|
Code is not the only way to help the project. Reviewing pull
|
|
requests, answering questions to help others on mailing lists or
|
|
issues, organizing and teaching tutorials, working on the website,
|
|
improving the documentation, are all priceless contributions.
|
|
|
|
We abide by the principles of openness, respect, and consideration of
|
|
others of the Python Software Foundation:
|
|
https://www.python.org/psf/codeofconduct/
|
|
|
|
|
|
|
|
|
|
|
In case you experience issues using this package, do not hesitate to submit a
|
|
ticket to the
|
|
`GitHub issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_. You are
|
|
also welcome to post feature requests or pull requests.
|
|
|
|
|
|
==================
|
|
Ways to contribute
|
|
==================
|
|
|
|
There are many ways to contribute to scikit-learn, with the most common ones
|
|
being contribution of code or documentation to the project. Improving the
|
|
documentation is no less important than improving the library itself. If you
|
|
find a typo in the documentation, or have made improvements, do not hesitate to
|
|
send an email to the mailing list or preferably submit a GitHub pull request.
|
|
Full documentation can be found under the doc/ directory.
|
|
|
|
But there are many other ways to help. In particular answering queries on the
|
|
`issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_,
|
|
investigating bugs, and :ref:`reviewing other developers' pull requests
|
|
<code_review>` are very valuable contributions that decrease the burden on the
|
|
project maintainers.
|
|
|
|
Another way to contribute is to report issues you're facing, and give a "thumbs
|
|
up" on issues that others reported and that are relevant to you. It also helps
|
|
us if you spread the word: reference the project from your blog and articles,
|
|
link to it from your website, or simply star to say "I use it":
|
|
|
|
.. raw:: html
|
|
|
|
<a class="github-button" href="https://github.com/scikit-learn/scikit-learn"
|
|
data-icon="octicon-star" data-size="large" data-show-count="true" aria-label="Star
|
|
scikit-learn/scikit-learn on GitHub">Star</a>
|
|
<script async defer src="https://buttons.github.io/buttons.js"></script>
|
|
|
|
.. topic:: Contributing to related projects
|
|
|
|
Scikit-learn thrives in an ecosystem of several related projects, which also
|
|
may have relevant issues to work on, including smaller projects such as:
|
|
|
|
* `scikit-learn-contrib <https://github.com/search?q=org%3Ascikit-learn-contrib+is%3Aissue+is%3Aopen+sort%3Aupdated-desc&type=Issues>`__
|
|
* `joblib <https://github.com/joblib/joblib/issues>`__
|
|
* `sphinx-gallery <https://github.com/sphinx-gallery/sphinx-gallery/issues>`__
|
|
* `numpydoc <https://github.com/numpy/numpydoc/issues>`__
|
|
|
|
and larger projects:
|
|
|
|
* `numpy <https://github.com/numpy/numpy/issues>`__
|
|
* `scipy <https://github.com/scipy/scipy/issues>`__
|
|
* `matplotlib <https://github.com/matplotlib/matplotlib/issues>`__
|
|
* and so on.
|
|
|
|
Look for issues marked "help wanted" or similar.
|
|
Helping these projects may help Scikit-learn too.
|
|
See also :ref:`related_projects`.
|
|
|
|
|
|
Submitting a bug report or a feature request
|
|
============================================
|
|
|
|
In case you experience issues using this package, do not hesitate to submit a
|
|
ticket to the
|
|
`Bug Tracker <https://github.com/scikit-learn/scikit-learn/issues>`_. You are
|
|
also welcome to post feature requests or pull requests.
|
|
|
|
It is recommended to check that your issue complies with the
|
|
following rules before submitting:
|
|
|
|
- Verify that your issue is not being currently addressed by other
|
|
`issues <https://github.com/scikit-learn/scikit-learn/issues?q=>`_
|
|
or `pull requests <https://github.com/scikit-learn/scikit-learn/pulls?q=>`_.
|
|
|
|
- If you are submitting an algorithm or feature request, please verify that
|
|
the algorithm fulfills our
|
|
`new algorithm requirements
|
|
<http://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms>`_.
|
|
|
|
- If you are submitting a bug report, we strongly encourage you to follow the guidelines in
|
|
:ref:`filing_bugs`.
|
|
|
|
.. _filing_bugs:
|
|
|
|
How to make a good bug report
|
|
-----------------------------
|
|
|
|
When you submit an issue to `Github
|
|
<https://github.com/scikit-learn/scikit-learn/issues>`__, please do your best to
|
|
follow these guidelines! This will make it a lot easier to provide you with good
|
|
feedback:
|
|
|
|
- The ideal bug report contains a **short reproducible code snippet**, this way
|
|
anyone can try to reproduce the bug easily (see `this
|
|
<http://stackoverflow.com/help/mcve>`_ for more details). If your snippet is
|
|
longer than around 50 lines, please link to a `gist
|
|
<https://gist.github.com>`_ or a github repo.
|
|
|
|
- If not feasible to include a reproducible snippet, please be specific about
|
|
what **estimators and/or functions are involved and the shape of the data**.
|
|
|
|
- If an exception is raised, please **provide the full traceback**.
|
|
|
|
- Please include your **operating system type and version number**, as well as
|
|
your **Python, scikit-learn, numpy, and scipy versions**. This information
|
|
can be found by running the following code snippet::
|
|
|
|
>>> import sklearn
|
|
>>> sklearn.show_versions() # doctest: +SKIP
|
|
|
|
.. note::
|
|
|
|
This utility function is only available in scikit-learn v0.20+.
|
|
For previous versions, one has to explicitly run::
|
|
|
|
import platform; print(platform.platform())
|
|
import sys; print("Python", sys.version)
|
|
import numpy; print("NumPy", numpy.__version__)
|
|
import scipy; print("SciPy", scipy.__version__)
|
|
import sklearn; print("Scikit-Learn", sklearn.__version__)
|
|
|
|
- Please ensure all **code snippets and error messages are formatted in
|
|
appropriate code blocks**. See `Creating and highlighting code blocks
|
|
<https://help.github.com/articles/creating-and-highlighting-code-blocks>`_
|
|
for more details.
|
|
|
|
|
|
|
|
Contributing code
|
|
=================
|
|
|
|
.. note::
|
|
|
|
To avoid duplicating work, it is highly advised that you contact the
|
|
developers on the mailing list before starting work on a non-trivial feature.
|
|
|
|
https://mail.python.org/mailman/listinfo/scikit-learn
|
|
|
|
How to contribute
|
|
-----------------
|
|
|
|
The preferred way to contribute to scikit-learn is to fork the `main
|
|
repository <https://github.com/scikit-learn/scikit-learn/>`__ on GitHub,
|
|
then submit a "pull request" (PR):
|
|
|
|
1. `Create an account <https://github.com/join>`_ on
|
|
GitHub if you do not already have one.
|
|
|
|
2. Fork the `project repository
|
|
<https://github.com/scikit-learn/scikit-learn>`__: click on the 'Fork'
|
|
button near the top of the page. This creates a copy of the code under your
|
|
account on the GitHub server. For more details on how to fork a
|
|
repository see `this guide <https://help.github.com/articles/fork-a-repo/>`_.
|
|
|
|
3. Clone this copy to your local disk::
|
|
|
|
$ git clone git@github.com:YourLogin/scikit-learn.git
|
|
|
|
4. Create a branch to hold your changes::
|
|
|
|
$ git checkout -b my-feature
|
|
|
|
and start making changes. Never work in the ``master`` branch!
|
|
|
|
5. Work on this copy, on your computer, using Git to do the version
|
|
control. When you're done editing, do::
|
|
|
|
$ git add modified_files
|
|
$ git commit
|
|
|
|
to record your changes in Git, then push them to GitHub with::
|
|
|
|
$ git push -u origin my-feature
|
|
|
|
Finally, follow `these
|
|
<https://help.github.com/articles/creating-a-pull-request-from-a-fork>`_
|
|
instructions to create a pull request from your fork. This will send an
|
|
email to the committers. You may want to consider sending an email to the
|
|
mailing list for more visibility.
|
|
|
|
.. note::
|
|
|
|
In the above setup, your ``origin`` remote repository points to
|
|
YourLogin/scikit-learn.git. If you wish to fetch/merge from the main
|
|
repository instead of your forked one, you will need to add another remote
|
|
to use instead of ``origin``. If we choose the name ``upstream`` for it, the
|
|
command will be::
|
|
|
|
$ git remote add upstream https://github.com/scikit-learn/scikit-learn.git
|
|
|
|
If any of the above seems like magic to you, then look up the `Git documentation
|
|
<https://git-scm.com/documentation>`_ and the `Git development workflow
|
|
<http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html>`_ on the
|
|
web.
|
|
|
|
If some conflicts arise between your branch and the ``master`` branch, you need
|
|
to merge ``master``. The command will be::
|
|
|
|
$ git merge master
|
|
|
|
with ``master`` being synchronized with the ``upstream``.
|
|
|
|
Subsequently, you need to solve the conflicts. You can refer to the `Git
|
|
documentation related to resolving merge conflict using the command line
|
|
<https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/>`_.
|
|
|
|
.. note::
|
|
|
|
In the past, the policy to resolve conflicts was to rebase your branch on
|
|
``master``. GitHub interface deals with merging ``master`` better than in
|
|
the past.
|
|
|
|
|
|
Contributing pull requests
|
|
--------------------------
|
|
|
|
It is recommended to check that your contribution complies with the following
|
|
rules before submitting a pull request:
|
|
|
|
* Follow the `coding-guidelines`_ (see below). To make sure that
|
|
your PR does not add PEP8 violations you can run
|
|
`./build_tools/travis/flake8_diff.sh` or `make flake8-diff` on a
|
|
Unix-like system.
|
|
|
|
* When applicable, use the validation tools and other code in the
|
|
``sklearn.utils`` submodule. A list of utility routines available
|
|
for developers can be found in the :ref:`developers-utils` page.
|
|
|
|
* Give your pull request a helpful title that summarises what your
|
|
contribution does. In some cases "Fix <ISSUE TITLE>" is enough.
|
|
"Fix #<ISSUE NUMBER>" is not enough.
|
|
|
|
* Often pull requests resolve one or more other issues (or pull requests).
|
|
If merging your pull request means that some other issues/PRs should
|
|
be closed, you should `use keywords to create link to them
|
|
<https://github.com/blog/1506-closing-issues-via-pull-requests/>`_
|
|
(e.g., ``Fixes #1234``; multiple issues/PRs are allowed as long as each
|
|
one is preceded by a keyword). Upon merging, those issues/PRs will
|
|
automatically be closed by GitHub. If your pull request is simply
|
|
related to some other issues/PRs, create a link to them without using
|
|
the keywords (e.g., ``See also #1234``).
|
|
|
|
* All public methods should have informative docstrings with sample
|
|
usage presented as doctests when appropriate.
|
|
|
|
* Please prefix the title of your pull request with ``[MRG]`` if the
|
|
contribution is complete and should be subjected to a detailed review.
|
|
Two core developers will review your code and change the prefix of the pull
|
|
request to ``[MRG + 1]`` and ``[MRG + 2]`` on approval, making it eligible
|
|
for merging. An incomplete contribution -- where you expect to do more
|
|
work before receiving a full review -- should be prefixed ``[WIP]`` (to
|
|
indicate a work in progress) and changed to ``[MRG]`` when it matures.
|
|
WIPs may be useful to: indicate you are working on something to avoid
|
|
duplicated work, request broad review of functionality or API, or seek
|
|
collaborators. WIPs often benefit from the inclusion of a
|
|
`task list
|
|
<https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments>`_
|
|
in the PR description.
|
|
|
|
* All other tests pass when everything is rebuilt from scratch. On
|
|
Unix-like systems, check with (from the toplevel source folder)::
|
|
|
|
$ make
|
|
|
|
* When adding additional functionality, provide at least one example script
|
|
in the ``examples/`` folder. Have a look at other examples for reference.
|
|
Examples should demonstrate why the new functionality is useful in
|
|
practice and, if possible, compare it to other methods available in
|
|
scikit-learn.
|
|
|
|
* Documentation and high-coverage tests are necessary for enhancements to be
|
|
accepted. Bug-fixes or new features should be provided with
|
|
`non-regression tests
|
|
<https://en.wikipedia.org/wiki/Non-regression_testing>`_. These tests
|
|
verify the correct behavior of the fix or feature. In this manner, further
|
|
modifications on the code base are granted to be consistent with the
|
|
desired behavior. For the case of bug fixes, at the time of the PR, the
|
|
non-regression tests should fail for the code base in the master branch
|
|
and pass for the PR code.
|
|
|
|
* At least one paragraph of narrative documentation with links to
|
|
references in the literature (with PDF links when possible) and
|
|
the example. For more details on writing and building the
|
|
documentation, see the :ref:`contribute_documentation` section.
|
|
|
|
You can also check for common programming errors with the following tools:
|
|
|
|
* Code with a good unittest coverage (at least 90%, better 100%), check
|
|
with::
|
|
|
|
$ pip install pytest pytest-cov
|
|
$ pytest --cov sklearn path/to/tests_for_package
|
|
|
|
see also :ref:`testing_coverage`
|
|
|
|
* No flake8 warnings, check with::
|
|
|
|
$ pip install flake8
|
|
$ flake8 path/to/module.py
|
|
|
|
Bonus points for contributions that include a performance analysis with
|
|
a benchmark script and profiling output (please report on the mailing
|
|
list or on the GitHub wiki).
|
|
|
|
Also check out the :ref:`performance-howto` guide for more details on profiling
|
|
and Cython optimizations.
|
|
|
|
.. note::
|
|
|
|
The current state of the scikit-learn code base is not compliant with
|
|
all of those guidelines, but we expect that enforcing those constraints
|
|
on all new contributions will get the overall code base quality in the
|
|
right direction.
|
|
|
|
.. note::
|
|
|
|
For two very well documented and more detailed guides on development
|
|
workflow, please pay a visit to the `Scipy Development Workflow
|
|
<http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html>`_ -
|
|
and the `Astropy Workflow for Developers
|
|
<http://astropy.readthedocs.io/en/latest/development/workflow/development_workflow.html>`_
|
|
sections.
|
|
|
|
.. topic:: Continuous Integration (CI)
|
|
|
|
* Travis is used for testing on Linux platforms
|
|
* Appveyor is used for testing on Windows platforms
|
|
* CircleCI is used to build the docs for viewing and for testing with PyPy on Linux
|
|
|
|
Please note that if one of the following markers appear in the latest commit
|
|
message, the following actions are taken.
|
|
|
|
====================== ===================
|
|
Commit Message Marker Action Taken by CI
|
|
---------------------- -------------------
|
|
[scipy-dev] Add a Travis build with our dependencies (numpy, scipy, etc ...) development builds
|
|
[ci skip] CI is skipped completely
|
|
[doc skip] Docs are not built
|
|
[doc quick] Docs built, but excludes example gallery plots
|
|
[doc build] Docs built including example gallery plots
|
|
====================== ===================
|
|
|
|
.. _new_contributors:
|
|
|
|
Issues for New Contributors
|
|
---------------------------
|
|
|
|
New contributors should look for the following tags when looking for issues. We
|
|
strongly recommend that new contributors tackle "easy" issues first: this helps
|
|
the contributor become familiar with the contribution workflow, and for the core
|
|
devs to become acquainted with the contributor; besides which, we frequently
|
|
underestimate how easy an issue is to solve!
|
|
|
|
.. topic:: good first issue tag
|
|
|
|
A great way to start contributing to scikit-learn is to pick an item from
|
|
the list of `good first issues
|
|
<https://github.com/scikit-learn/scikit-learn/labels/good%20first%20issue>`_
|
|
in the issue tracker. Resolving these issues allow you to start contributing
|
|
to the project without much prior knowledge. If you have already contributed
|
|
to scikit-learn, you should look at Easy issues instead.
|
|
|
|
.. topic:: Easy tag
|
|
|
|
Another great way to contribute to scikit-learn is to pick an item from the
|
|
list of `Easy issues
|
|
<https://github.com/scikit-learn/scikit-learn/labels/Easy>`_ in the issue
|
|
tracker. Your assistance in this area will be greatly appreciated by the
|
|
more experienced developers as it helps free up their time to concentrate on
|
|
other issues.
|
|
|
|
.. topic:: help wanted tag
|
|
|
|
We often use the help wanted tag to mark issues regardless of difficulty. Additionally,
|
|
we use the help wanted tag to mark Pull Requests which have been abandoned
|
|
by their original contributor and are available for someone to pick up where the original
|
|
contributor left off. The list of issues with the help wanted tag can be found
|
|
`here <https://github.com/scikit-learn/scikit-learn/labels/help%20wanted>`__ .
|
|
|
|
Note that not all issues which need contributors will have this tag.
|
|
|
|
.. _contribute_documentation:
|
|
|
|
Documentation
|
|
-------------
|
|
|
|
We are glad to accept any sort of documentation: function docstrings,
|
|
reStructuredText documents (like this one), tutorials, etc. reStructuredText
|
|
documents live in the source code repository under the ``doc/`` directory.
|
|
You can edit the documentation using any text editor, and then generate the
|
|
HTML output by building the documentation website.
|
|
|
|
Building the documentation
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Building the documentation requires installing some additional packages::
|
|
|
|
pip install sphinx sphinx-gallery numpydoc matplotlib Pillow pandas scikit-image
|
|
|
|
To build the documentation, you need to be in the ``doc`` folder::
|
|
|
|
cd doc
|
|
|
|
It also requires having the version of scikit-learn installed that corresponds
|
|
to the documentation, e.g.::
|
|
|
|
pip install --editable ..
|
|
|
|
To generate the full web site, including the example gallery::
|
|
|
|
make html
|
|
|
|
Generating the example gallery will run all our examples which takes a
|
|
while. To save some time, you can use:
|
|
|
|
- ``make html-noplot``: this will generate the documentation without the
|
|
example gallery. This is useful when changing a docstring for example.
|
|
- ``EXAMPLES_PATTERN=your_regex_goes_here make html``: only the examples
|
|
matching ``your_regex_goes_here`` will be run. This is particularly
|
|
useful if you are modifying a few examples.
|
|
|
|
That should create all the documentation in the ``_build/html/stable``
|
|
directory. Set the environment variable `NO_MATHJAX=1` if you intend to view
|
|
the documentation in an offline setting.
|
|
|
|
To build the PDF manual, run::
|
|
|
|
make latexpdf
|
|
|
|
.. warning:: **Sphinx version**
|
|
|
|
While we do our best to have the documentation build under as many
|
|
versions of Sphinx as possible, the different versions tend to
|
|
behave slightly differently. To get the best results, you should
|
|
use the same version as the one we used on CircleCI. Look at this
|
|
`github search <https://github.com/search?utf8=%E2%9C%93&q=sphinx+repo%3Ascikit-learn%2Fscikit-learn+extension%3Ash+path%3Abuild_tools%2Fcircle&type=Code>`_
|
|
to know the exact version.
|
|
|
|
Guidelines for writing documentation
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
It is important to keep a good compromise between mathematical and algorithmic
|
|
details, and give intuition to the reader on what the algorithm does.
|
|
|
|
Basically, to elaborate on the above, it is best to always
|
|
start with a small paragraph with a hand-waving explanation of what the
|
|
method does to the data. Then, it is very helpful
|
|
to point out why the feature is useful and when it should be used -
|
|
the latter also including "big O"
|
|
(:math:`O\left(g\left(n\right)\right)`)
|
|
complexities of the algorithm, as opposed to just *rules of thumb*,
|
|
as the latter can be very machine-dependent.
|
|
If those complexities are not available, then rules of thumb
|
|
may be provided instead.
|
|
|
|
Secondly, a generated figure from an example (as mentioned in the previous
|
|
paragraph) should then be included to further provide some
|
|
intuition.
|
|
|
|
Next, one or two small code examples to show its use can be added.
|
|
|
|
Next, any math and equations, followed by references,
|
|
can be added to further the documentation. Not starting the
|
|
documentation with the maths makes it more friendly towards
|
|
users that are just interested in what the feature will do, as
|
|
opposed to how it works "under the hood".
|
|
|
|
Finally, follow the formatting rules below to make it consistently good:
|
|
|
|
* Add "See also" in docstrings for related classes/functions.
|
|
|
|
* "See also" in docstrings should be one line per reference,
|
|
with a colon and an explanation, for example::
|
|
|
|
See also
|
|
--------
|
|
SelectKBest : Select features based on the k highest scores.
|
|
SelectFpr : Select features based on a false positive rate test.
|
|
|
|
* For unwritten formatting rules, try to follow existing good works:
|
|
|
|
* For "References" in docstrings, see the Silhouette Coefficient
|
|
(:func:`sklearn.metrics.silhouette_score`).
|
|
|
|
Generated documentation on CircleCI
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When you change the documentation in a pull request, CircleCI automatically
|
|
builds it. To view the documentation generated by CircleCI:
|
|
|
|
* navigate to the bottom of your pull request page to see the CI
|
|
statuses. You may need to click on "Show all checks" to see all the CI
|
|
statuses.
|
|
* click on the CircleCI status with "python3" in the title.
|
|
* add ``#artifacts`` at the end of the URL. Note: you need to wait for the
|
|
CircleCI build to finish before being able to look at the artifacts.
|
|
* once the artifacts are visible, navigate to ``doc/_changed.html`` to see a
|
|
list of documentation pages that are likely to be affected by your pull
|
|
request. Navigate to ``doc/index.html`` to see the full generated html
|
|
documentation.
|
|
|
|
If you often need to look at the documentation generated by CircleCI, e.g. when
|
|
reviewing pull requests, you may find :ref:`this tip
|
|
<viewing_rendered_html_documentation>` very handy.
|
|
|
|
.. _testing_coverage:
|
|
|
|
Testing and improving test coverage
|
|
------------------------------------
|
|
|
|
High-quality `unit testing <https://en.wikipedia.org/wiki/Unit_testing>`_
|
|
is a corner-stone of the scikit-learn development process. For this
|
|
purpose, we use the `pytest <https://docs.pytest.org>`_
|
|
package. The tests are functions appropriately named, located in `tests`
|
|
subdirectories, that check the validity of the algorithms and the
|
|
different options of the code.
|
|
|
|
The full scikit-learn tests can be run using 'make' in the root folder.
|
|
Alternatively, running 'pytest' in a folder will run all the tests of
|
|
the corresponding subpackages.
|
|
|
|
We expect code coverage of new features to be at least around 90%.
|
|
|
|
.. note:: **Workflow to improve test coverage**
|
|
|
|
To test code coverage, you need to install the `coverage
|
|
<https://pypi.python.org/pypi/coverage>`_ package in addition to pytest.
|
|
|
|
1. Run 'make test-coverage'. The output lists for each file the line
|
|
numbers that are not tested.
|
|
|
|
2. Find a low hanging fruit, looking at which lines are not tested,
|
|
write or adapt a test specifically for these lines.
|
|
|
|
3. Loop.
|
|
|
|
|
|
|
|
Developers web site
|
|
-------------------
|
|
|
|
More information can be found on the `developer's wiki
|
|
<https://github.com/scikit-learn/scikit-learn/wiki>`_.
|
|
|
|
|
|
Issue Tracker Tags
|
|
------------------
|
|
All issues and pull requests on the
|
|
`GitHub issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_
|
|
should have (at least) one of the following tags:
|
|
|
|
:Bug / Crash:
|
|
Something is happening that clearly shouldn't happen.
|
|
Wrong results as well as unexpected errors from estimators go here.
|
|
|
|
:Cleanup / Enhancement:
|
|
Improving performance, usability, consistency.
|
|
|
|
:Documentation:
|
|
Missing, incorrect or sub-standard documentations and examples.
|
|
|
|
:New Feature:
|
|
Feature requests and pull requests implementing a new feature.
|
|
|
|
There are four other tags to help new contributors:
|
|
|
|
:good first issue:
|
|
This issue is ideal for a first contribution to scikit-learn. Ask for help
|
|
if the formulation is unclear. If you have already contributed to
|
|
scikit-learn, look at Easy issues instead.
|
|
|
|
:Easy:
|
|
This issue can be tackled without much prior experience.
|
|
|
|
:Moderate:
|
|
Might need some knowledge of machine learning or the package,
|
|
but is still approachable for someone new to the project.
|
|
|
|
:help wanted:
|
|
This tag marks an issue which currently lacks a contributor or a
|
|
PR that needs another contributor to take over the work. These
|
|
issues can range in difficulty, and may not be approachable
|
|
for new contributors. Note that not all issues which need
|
|
contributors will have this tag.
|
|
|
|
|
|
.. _coding-guidelines:
|
|
|
|
Coding guidelines
|
|
=================
|
|
|
|
The following are some guidelines on how new code should be written. Of
|
|
course, there are special cases and there will be exceptions to these
|
|
rules. However, following these rules when submitting new code makes
|
|
the review easier so new code can be integrated in less time.
|
|
|
|
Uniformly formatted code makes it easier to share code ownership. The
|
|
scikit-learn project tries to closely follow the official Python guidelines
|
|
detailed in `PEP8 <https://www.python.org/dev/peps/pep-0008>`_ that
|
|
detail how code should be formatted and indented. Please read it and
|
|
follow it.
|
|
|
|
In addition, we add the following guidelines:
|
|
|
|
* Use underscores to separate words in non class names: ``n_samples``
|
|
rather than ``nsamples``.
|
|
|
|
* Avoid multiple statements on one line. Prefer a line return after
|
|
a control flow statement (``if``/``for``).
|
|
|
|
* Use relative imports for references inside scikit-learn.
|
|
|
|
* Unit tests are an exception to the previous rule;
|
|
they should use absolute imports, exactly as client code would.
|
|
A corollary is that, if ``sklearn.foo`` exports a class or function
|
|
that is implemented in ``sklearn.foo.bar.baz``,
|
|
the test should import it from ``sklearn.foo``.
|
|
|
|
* **Please don't use** ``import *`` **in any case**. It is considered harmful
|
|
by the `official Python recommendations
|
|
<https://docs.python.org/2/howto/doanddont.html#from-module-import>`_.
|
|
It makes the code harder to read as the origin of symbols is no
|
|
longer explicitly referenced, but most important, it prevents
|
|
using a static analysis tool like `pyflakes
|
|
<https://divmod.readthedocs.io/en/latest/products/pyflakes.html>`_ to automatically
|
|
find bugs in scikit-learn.
|
|
|
|
* Use the `numpy docstring standard
|
|
<https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt>`_
|
|
in all your docstrings.
|
|
|
|
|
|
A good example of code that we like can be found `here
|
|
<https://gist.github.com/nateGeorge/5455d2c57fb33c1ae04706f2dc4fee01>`_.
|
|
|
|
Input validation
|
|
----------------
|
|
|
|
.. currentmodule:: sklearn.utils
|
|
|
|
The module :mod:`sklearn.utils` contains various functions for doing input
|
|
validation and conversion. Sometimes, ``np.asarray`` suffices for validation;
|
|
do *not* use ``np.asanyarray`` or ``np.atleast_2d``, since those let NumPy's
|
|
``np.matrix`` through, which has a different API
|
|
(e.g., ``*`` means dot product on ``np.matrix``,
|
|
but Hadamard product on ``np.ndarray``).
|
|
|
|
In other cases, be sure to call :func:`check_array` on any array-like argument
|
|
passed to a scikit-learn API function. The exact parameters to use depends
|
|
mainly on whether and which ``scipy.sparse`` matrices must be accepted.
|
|
|
|
For more information, refer to the :ref:`developers-utils` page.
|
|
|
|
Random Numbers
|
|
--------------
|
|
|
|
If your code depends on a random number generator, do not use
|
|
``numpy.random.random()`` or similar routines. To ensure
|
|
repeatability in error checking, the routine should accept a keyword
|
|
``random_state`` and use this to construct a
|
|
``numpy.random.RandomState`` object.
|
|
See :func:`sklearn.utils.check_random_state` in :ref:`developers-utils`.
|
|
|
|
Here's a simple example of code using some of the above guidelines::
|
|
|
|
from sklearn.utils import check_array, check_random_state
|
|
|
|
def choose_random_sample(X, random_state=0):
|
|
"""
|
|
Choose a random point from X
|
|
|
|
Parameters
|
|
----------
|
|
X : array-like, shape (n_samples, n_features)
|
|
array representing the data
|
|
random_state : RandomState or an int seed (0 by default)
|
|
A random number generator instance to define the state of the
|
|
random permutations generator.
|
|
|
|
Returns
|
|
-------
|
|
x : numpy array, shape (n_features,)
|
|
A random point selected from X
|
|
"""
|
|
X = check_array(X)
|
|
random_state = check_random_state(random_state)
|
|
i = random_state.randint(X.shape[0])
|
|
return X[i]
|
|
|
|
If you use randomness in an estimator instead of a freestanding function,
|
|
some additional guidelines apply.
|
|
|
|
First off, the estimator should take a ``random_state`` argument to its
|
|
``__init__`` with a default value of ``None``.
|
|
It should store that argument's value, **unmodified**,
|
|
in an attribute ``random_state``.
|
|
``fit`` can call ``check_random_state`` on that attribute
|
|
to get an actual random number generator.
|
|
If, for some reason, randomness is needed after ``fit``,
|
|
the RNG should be stored in an attribute ``random_state_``.
|
|
The following example should make this clear::
|
|
|
|
class GaussianNoise(BaseEstimator, TransformerMixin):
|
|
"""This estimator ignores its input and returns random Gaussian noise.
|
|
|
|
It also does not adhere to all scikit-learn conventions,
|
|
but showcases how to handle randomness.
|
|
"""
|
|
|
|
def __init__(self, n_components=100, random_state=None):
|
|
self.random_state = random_state
|
|
|
|
# the arguments are ignored anyway, so we make them optional
|
|
def fit(self, X=None, y=None):
|
|
self.random_state_ = check_random_state(self.random_state)
|
|
|
|
def transform(self, X):
|
|
n_samples = X.shape[0]
|
|
return self.random_state_.randn(n_samples, n_components)
|
|
|
|
The reason for this setup is reproducibility:
|
|
when an estimator is ``fit`` twice to the same data,
|
|
it should produce an identical model both times,
|
|
hence the validation in ``fit``, not ``__init__``.
|
|
|
|
.. _contributing_deprecation:
|
|
|
|
Deprecation
|
|
-----------
|
|
|
|
If any publicly accessible method, function, attribute or parameter
|
|
is renamed, we still support the old one for two releases and issue
|
|
a deprecation warning when it is called/passed/accessed.
|
|
E.g., if the function ``zero_one`` is renamed to ``zero_one_loss``,
|
|
we add the decorator ``deprecated`` (from ``sklearn.utils``)
|
|
to ``zero_one`` and call ``zero_one_loss`` from that function::
|
|
|
|
from ..utils import deprecated
|
|
|
|
def zero_one_loss(y_true, y_pred, normalize=True):
|
|
# actual implementation
|
|
pass
|
|
|
|
@deprecated("Function 'zero_one' was renamed to 'zero_one_loss' "
|
|
"in version 0.13 and will be removed in release 0.15. "
|
|
"Default behavior is changed from 'normalize=False' to "
|
|
"'normalize=True'")
|
|
def zero_one(y_true, y_pred, normalize=False):
|
|
return zero_one_loss(y_true, y_pred, normalize)
|
|
|
|
If an attribute is to be deprecated,
|
|
use the decorator ``deprecated`` on a property.
|
|
E.g., renaming an attribute ``labels_`` to ``classes_`` can be done as::
|
|
|
|
@property
|
|
@deprecated("Attribute labels_ was deprecated in version 0.13 and "
|
|
"will be removed in 0.15. Use 'classes_' instead")
|
|
def labels_(self):
|
|
return self.classes_
|
|
|
|
If a parameter has to be deprecated, use ``DeprecationWarning`` appropriately.
|
|
In the following example, k is deprecated and renamed to n_clusters::
|
|
|
|
import warnings
|
|
|
|
def example_function(n_clusters=8, k='not_used'):
|
|
if k != 'not_used':
|
|
warnings.warn("'k' was renamed to n_clusters in version 0.13 and "
|
|
"will be removed in 0.15.", DeprecationWarning)
|
|
n_clusters = k
|
|
|
|
When the change is in a class, we validate and raise warning in ``fit``::
|
|
|
|
import warnings
|
|
|
|
class ExampleEstimator(BaseEstimator):
|
|
def __init__(self, n_clusters=8, k='not_used'):
|
|
self.n_clusters = n_clusters
|
|
self.k = k
|
|
|
|
def fit(self, X, y):
|
|
if k != 'not_used':
|
|
warnings.warn("'k' was renamed to n_clusters in version 0.13 and "
|
|
"will be removed in 0.15.", DeprecationWarning)
|
|
self._n_clusters = k
|
|
else:
|
|
self._n_clusters = self.n_clusters
|
|
|
|
As in these examples, the warning message should always give both the
|
|
version in which the deprecation happened and the version in which the
|
|
old behavior will be removed. If the deprecation happened in version
|
|
0.x-dev, the message should say deprecation occurred in version 0.x and
|
|
the removal will be in 0.(x+2), so that users will have enough time to
|
|
adapt their code to the new behaviour. For example, if the deprecation happened
|
|
in version 0.18-dev, the message should say it happened in version 0.18
|
|
and the old behavior will be removed in version 0.20.
|
|
|
|
In addition, a deprecation note should be added in the docstring, recalling the
|
|
same information as the deprecation warning as explained above. Use the
|
|
``.. deprecated::`` directive::
|
|
|
|
.. deprecated:: 0.13
|
|
``k`` was renamed to ``n_clusters`` in version 0.13 and will be removed
|
|
in 0.15.
|
|
|
|
What's more, a deprecation requires a test which ensures that the warning is
|
|
raised in relevant cases but not in other cases. The warning should be caught
|
|
in all other tests (using e.g., ``@pytest.mark.filterwarnings``),
|
|
and there should be no warning in the examples.
|
|
|
|
|
|
Change the default value of a parameter
|
|
---------------------------------------
|
|
|
|
If the default value of a parameter needs to be changed, please replace the
|
|
default value with a specific value (e.g., ``warn``) and raise
|
|
``FutureWarning`` when users are using the default value. In the following
|
|
example, we change the default value of ``n_clusters`` from 5 to 10
|
|
(current version is 0.20)::
|
|
|
|
import warnings
|
|
|
|
def example_function(n_clusters='warn'):
|
|
if n_clusters == 'warn':
|
|
warnings.warn("The default value of n_clusters will change from "
|
|
"5 to 10 in 0.22.", FutureWarning)
|
|
n_clusters = 5
|
|
|
|
When the change is in a class, we validate and raise warning in ``fit``::
|
|
|
|
import warnings
|
|
|
|
class ExampleEstimator:
|
|
def __init__(self, n_clusters='warn'):
|
|
self.n_clusters = n_clusters
|
|
|
|
def fit(self, X, y):
|
|
if self.n_clusters == 'warn':
|
|
warnings.warn("The default value of n_clusters will change from "
|
|
"5 to 10 in 0.22.", FutureWarning)
|
|
self._n_clusters = 5
|
|
|
|
Similar to deprecations, the warning message should always give both the
|
|
version in which the change happened and the version in which the old behavior
|
|
will be removed. The docstring needs to be updated accordingly. We need a test
|
|
which ensures that the warning is raised in relevant cases but not in other
|
|
cases. The warning should be caught in all other tests
|
|
(using e.g., ``@pytest.mark.filterwarnings``), and there should be no warning
|
|
in the examples.
|
|
|
|
|
|
.. currentmodule:: sklearn
|
|
|
|
Python versions supported
|
|
-------------------------
|
|
|
|
All scikit-learn code should work unchanged in both Python 2.7 and 3.4 or
|
|
newer. Since Python 3.x is not backwards compatible, that may require changes
|
|
to code and it certainly requires testing on both 2.7 and 3.4 or newer.
|
|
|
|
For most numerical algorithms, Python 3.x support is easy:
|
|
just remember that ``print`` is a function and
|
|
integer division is written ``//``.
|
|
String handling has been overhauled, though, as have parts of
|
|
the Python standard library.
|
|
The `six <http://pythonhosted.org/six/>`_ package helps with
|
|
cross-compatibility and is included in scikit-learn as
|
|
``sklearn.externals.six``.
|
|
|
|
|
|
.. _code_review:
|
|
|
|
Code Review Guidelines
|
|
======================
|
|
Reviewing code contributed to the project as PRs is a crucial component of
|
|
scikit-learn development. We encourage anyone to start reviewing code of other
|
|
developers. The code review process is often highly educational for everybody
|
|
involved. This is particularly appropriate if it is a feature you would like to
|
|
use, and so can respond critically about whether the PR meets your needs. While
|
|
each pull request needs to be signed off by two core developers, you can speed
|
|
up this process by providing your feedback.
|
|
|
|
Here are a few important aspects that need to be covered in any code review,
|
|
from high-level questions to a more detailed check-list.
|
|
|
|
- Do we want this in the library? Is it likely to be used? Do you, as
|
|
a scikit-learn user, like the change and intend to use it? Is it in
|
|
the scope of scikit-learn? Will the cost of maintaining a new
|
|
feature be worth its benefits?
|
|
|
|
- Is the code consistent with the API of scikit-learn? Are public
|
|
functions/classes/parameters well named and intuitively designed?
|
|
|
|
- Are all public functions/classes and their parameters, return types, and
|
|
stored attributes named according to scikit-learn conventions and documented clearly?
|
|
|
|
- Is any new functionality described in the user-guide and illustrated with examples?
|
|
|
|
- Is every public function/class tested? Are a reasonable set of
|
|
parameters, their values, value types, and combinations tested? Do
|
|
the tests validate that the code is correct, i.e. doing what the
|
|
documentation says it does? If the change is a bug-fix, is a
|
|
non-regression test included? Look at `this
|
|
<https://jeffknupp.com/blog/2013/12/09/improve-your-python-understanding-unit-testing>`__
|
|
to get started with testing in Python.
|
|
|
|
- Do the tests pass in the continuous integration build? If
|
|
appropriate, help the contributor understand why tests failed.
|
|
|
|
- Do the tests cover every line of code (see the coverage report in the build
|
|
log)? If not, are the lines missing coverage good exceptions?
|
|
|
|
- Is the code easy to read and low on redundancy? Should variable names be
|
|
improved for clarity or consistency? Should comments be added? Should comments
|
|
be removed as unhelpful or extraneous?
|
|
|
|
- Could the code easily be rewritten to run much more efficiently for
|
|
relevant settings?
|
|
|
|
- Is the code backwards compatible with previous versions? (or is a
|
|
deprecation cycle necessary?)
|
|
|
|
- Will the new code add any dependencies on other libraries? (this is
|
|
unlikely to be accepted)
|
|
|
|
- Does the documentation render properly (see the
|
|
:ref:`contribute_documentation` section for more details), and are the plots
|
|
instructive?
|
|
|
|
:ref:`saved_replies` includes some frequent comments that reviewers may make.
|
|
|
|
.. _api_overview:
|
|
|
|
APIs of scikit-learn objects
|
|
============================
|
|
|
|
To have a uniform API, we try to have a common basic API for all the
|
|
objects. In addition, to avoid the proliferation of framework code, we
|
|
try to adopt simple conventions and limit to a minimum the number of
|
|
methods an object must implement.
|
|
|
|
Elements of the scikit-learn API are described more definitively in the
|
|
:ref:`glossary`.
|
|
|
|
Different objects
|
|
-----------------
|
|
|
|
The main objects in scikit-learn are (one class can implement
|
|
multiple interfaces):
|
|
|
|
:Estimator:
|
|
|
|
The base object, implements a ``fit`` method to learn from data, either::
|
|
|
|
estimator = estimator.fit(data, targets)
|
|
|
|
or::
|
|
|
|
estimator = estimator.fit(data)
|
|
|
|
:Predictor:
|
|
|
|
For supervised learning, or some unsupervised problems, implements::
|
|
|
|
prediction = predictor.predict(data)
|
|
|
|
Classification algorithms usually also offer a way to quantify certainty
|
|
of a prediction, either using ``decision_function`` or ``predict_proba``::
|
|
|
|
probability = predictor.predict_proba(data)
|
|
|
|
:Transformer:
|
|
|
|
For filtering or modifying the data, in a supervised or unsupervised
|
|
way, implements::
|
|
|
|
new_data = transformer.transform(data)
|
|
|
|
When fitting and transforming can be performed much more efficiently
|
|
together than separately, implements::
|
|
|
|
new_data = transformer.fit_transform(data)
|
|
|
|
:Model:
|
|
|
|
A model that can give a `goodness of fit <https://en.wikipedia.org/wiki/Goodness_of_fit>`_
|
|
measure or a likelihood of unseen data, implements (higher is better)::
|
|
|
|
score = model.score(data)
|
|
|
|
Estimators
|
|
----------
|
|
|
|
The API has one predominant object: the estimator. A estimator is an
|
|
object that fits a model based on some training data and is capable of
|
|
inferring some properties on new data. It can be, for instance, a
|
|
classifier or a regressor. All estimators implement the fit method::
|
|
|
|
estimator.fit(X, y)
|
|
|
|
All built-in estimators also have a ``set_params`` method, which sets
|
|
data-independent parameters (overriding previous parameter values passed
|
|
to ``__init__``).
|
|
|
|
All estimators in the main scikit-learn codebase should inherit from
|
|
``sklearn.base.BaseEstimator``.
|
|
|
|
Instantiation
|
|
^^^^^^^^^^^^^
|
|
|
|
This concerns the creation of an object. The object's ``__init__`` method
|
|
might accept constants as arguments that determine the estimator's behavior
|
|
(like the C constant in SVMs). It should not, however, take the actual training
|
|
data as an argument, as this is left to the ``fit()`` method::
|
|
|
|
clf2 = SVC(C=2.3)
|
|
clf3 = SVC([[1, 2], [2, 3]], [-1, 1]) # WRONG!
|
|
|
|
|
|
The arguments accepted by ``__init__`` should all be keyword arguments
|
|
with a default value. In other words, a user should be able to instantiate
|
|
an estimator without passing any arguments to it. The arguments should all
|
|
correspond to hyperparameters describing the model or the optimisation
|
|
problem the estimator tries to solve. These initial arguments (or parameters)
|
|
are always remembered by the estimator.
|
|
Also note that they should not be documented under the "Attributes" section,
|
|
but rather under the "Parameters" section for that estimator.
|
|
|
|
In addition, **every keyword argument accepted by** ``__init__`` **should
|
|
correspond to an attribute on the instance**. Scikit-learn relies on this to
|
|
find the relevant attributes to set on an estimator when doing model selection.
|
|
|
|
To summarize, an ``__init__`` should look like::
|
|
|
|
def __init__(self, param1=1, param2=2):
|
|
self.param1 = param1
|
|
self.param2 = param2
|
|
|
|
There should be no logic, not even input validation,
|
|
and the parameters should not be changed.
|
|
The corresponding logic should be put where the parameters are used,
|
|
typically in ``fit``.
|
|
The following is wrong::
|
|
|
|
def __init__(self, param1=1, param2=2, param3=3):
|
|
# WRONG: parameters should not be modified
|
|
if param1 > 1:
|
|
param2 += 1
|
|
self.param1 = param1
|
|
# WRONG: the object's attributes should have exactly the name of
|
|
# the argument in the constructor
|
|
self.param3 = param2
|
|
|
|
The reason for postponing the validation is that the same validation
|
|
would have to be performed in ``set_params``,
|
|
which is used in algorithms like ``GridSearchCV``.
|
|
|
|
Fitting
|
|
^^^^^^^
|
|
|
|
The next thing you will probably want to do is to estimate some
|
|
parameters in the model. This is implemented in the ``fit()`` method.
|
|
|
|
The ``fit()`` method takes the training data as arguments, which can be one
|
|
array in the case of unsupervised learning, or two arrays in the case
|
|
of supervised learning.
|
|
|
|
Note that the model is fitted using X and y, but the object holds no
|
|
reference to X and y. There are, however, some exceptions to this, as in
|
|
the case of precomputed kernels where this data must be stored for use by
|
|
the predict method.
|
|
|
|
============= ======================================================
|
|
Parameters
|
|
============= ======================================================
|
|
X array-like, shape (n_samples, n_features)
|
|
|
|
y array, shape (n_samples,)
|
|
|
|
kwargs optional data-dependent parameters.
|
|
============= ======================================================
|
|
|
|
``X.shape[0]`` should be the same as ``y.shape[0]``. If this requisite
|
|
is not met, an exception of type ``ValueError`` should be raised.
|
|
|
|
``y`` might be ignored in the case of unsupervised learning. However, to
|
|
make it possible to use the estimator as part of a pipeline that can
|
|
mix both supervised and unsupervised transformers, even unsupervised
|
|
estimators need to accept a ``y=None`` keyword argument in
|
|
the second position that is just ignored by the estimator.
|
|
For the same reason, ``fit_predict``, ``fit_transform``, ``score``
|
|
and ``partial_fit`` methods need to accept a ``y`` argument in
|
|
the second place if they are implemented.
|
|
|
|
The method should return the object (``self``). This pattern is useful
|
|
to be able to implement quick one liners in an IPython session such as::
|
|
|
|
y_predicted = SVC(C=100).fit(X_train, y_train).predict(X_test)
|
|
|
|
Depending on the nature of the algorithm, ``fit`` can sometimes also
|
|
accept additional keywords arguments. However, any parameter that can
|
|
have a value assigned prior to having access to the data should be an
|
|
``__init__`` keyword argument. **fit parameters should be restricted
|
|
to directly data dependent variables**. For instance a Gram matrix or
|
|
an affinity matrix which are precomputed from the data matrix ``X`` are
|
|
data dependent. A tolerance stopping criterion ``tol`` is not directly
|
|
data dependent (although the optimal value according to some scoring
|
|
function probably is).
|
|
|
|
Estimated Attributes
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Attributes that have been estimated from the data must always have a name
|
|
ending with trailing underscore, for example the coefficients of
|
|
some regression estimator would be stored in a ``coef_`` attribute after
|
|
``fit`` has been called.
|
|
|
|
The last-mentioned attributes are expected to be overridden when
|
|
you call ``fit`` a second time without taking any previous value into
|
|
account: **fit should be idempotent**.
|
|
|
|
Optional Arguments
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
In iterative algorithms, the number of iterations should be specified by
|
|
an integer called ``n_iter``.
|
|
|
|
|
|
Rolling your own estimator
|
|
==========================
|
|
If you want to implement a new estimator that is scikit-learn-compatible,
|
|
whether it is just for you or for contributing it to scikit-learn, there are
|
|
several internals of scikit-learn that you should be aware of in addition to
|
|
the scikit-learn API outlined above. You can check whether your estimator
|
|
adheres to the scikit-learn interface and standards by running
|
|
:func:`utils.estimator_checks.check_estimator` on the class::
|
|
|
|
>>> from sklearn.utils.estimator_checks import check_estimator
|
|
>>> from sklearn.svm import LinearSVC
|
|
>>> check_estimator(LinearSVC) # passes
|
|
|
|
The main motivation to make a class compatible to the scikit-learn estimator
|
|
interface might be that you want to use it together with model evaluation and
|
|
selection tools such as :class:`model_selection.GridSearchCV` and
|
|
:class:`pipeline.Pipeline`.
|
|
|
|
Before detailing the required interface below, we describe two ways to achieve
|
|
the correct interface more easily.
|
|
|
|
.. topic:: Project template:
|
|
|
|
We provide a `project template <https://github.com/scikit-learn-contrib/project-template/>`_
|
|
which helps in the creation of Python packages containing scikit-learn compatible estimators.
|
|
It provides:
|
|
|
|
* an initial git repository with Python package directory structure
|
|
* a template of a scikit-learn estimator
|
|
* an initial test suite including use of ``check_estimator``
|
|
* directory structures and scripts to compile documentation and example
|
|
galleries
|
|
* scripts to manage continuous integration (testing on Linux and Windows)
|
|
* instructions from getting started to publishing on `PyPi <https://pypi.python.org/pypi>`_
|
|
|
|
.. topic:: ``BaseEstimator`` and mixins:
|
|
|
|
We tend to use "duck typing", so building an estimator which follows
|
|
the API suffices for compatibility, without needing to inherit from or
|
|
even import any scikit-learn classes.
|
|
|
|
However, if a dependency on scikit-learn is acceptable in your code,
|
|
you can prevent a lot of boilerplate code
|
|
by deriving a class from ``BaseEstimator``
|
|
and optionally the mixin classes in ``sklearn.base``.
|
|
For example, below is a custom classifier, with more examples included
|
|
in the scikit-learn-contrib
|
|
`project template <https://github.com/scikit-learn-contrib/project-template/blob/master/skltemplate/template.py>`__.
|
|
|
|
>>> import numpy as np
|
|
>>> from sklearn.base import BaseEstimator, ClassifierMixin
|
|
>>> from sklearn.utils.validation import check_X_y, check_array, check_is_fitted
|
|
>>> from sklearn.utils.multiclass import unique_labels
|
|
>>> from sklearn.metrics import euclidean_distances
|
|
>>> class TemplateClassifier(BaseEstimator, ClassifierMixin):
|
|
...
|
|
... def __init__(self, demo_param='demo'):
|
|
... self.demo_param = demo_param
|
|
...
|
|
... def fit(self, X, y):
|
|
...
|
|
... # Check that X and y have correct shape
|
|
... X, y = check_X_y(X, y)
|
|
... # Store the classes seen during fit
|
|
... self.classes_ = unique_labels(y)
|
|
...
|
|
... self.X_ = X
|
|
... self.y_ = y
|
|
... # Return the classifier
|
|
... return self
|
|
...
|
|
... def predict(self, X):
|
|
...
|
|
... # Check is fit had been called
|
|
... check_is_fitted(self, ['X_', 'y_'])
|
|
...
|
|
... # Input validation
|
|
... X = check_array(X)
|
|
...
|
|
... closest = np.argmin(euclidean_distances(X, self.X_), axis=1)
|
|
... return self.y_[closest]
|
|
|
|
|
|
get_params and set_params
|
|
-------------------------
|
|
All scikit-learn estimators have ``get_params`` and ``set_params`` functions.
|
|
The ``get_params`` function takes no arguments and returns a dict of the
|
|
``__init__`` parameters of the estimator, together with their values.
|
|
It must take one keyword argument, ``deep``,
|
|
which receives a boolean value that determines
|
|
whether the method should return the parameters of sub-estimators
|
|
(for most estimators, this can be ignored).
|
|
The default value for ``deep`` should be true.
|
|
|
|
The ``set_params`` on the other hand takes as input a dict of the form
|
|
``'parameter': value`` and sets the parameter of the estimator using this dict.
|
|
Return value must be estimator itself.
|
|
|
|
While the ``get_params`` mechanism is not essential (see :ref:`cloning` below),
|
|
the ``set_params`` function is necessary as it is used to set parameters during
|
|
grid searches.
|
|
|
|
The easiest way to implement these functions, and to get a sensible
|
|
``__repr__`` method, is to inherit from ``sklearn.base.BaseEstimator``. If you
|
|
do not want to make your code dependent on scikit-learn, the easiest way to
|
|
implement the interface is::
|
|
|
|
def get_params(self, deep=True):
|
|
# suppose this estimator has parameters "alpha" and "recursive"
|
|
return {"alpha": self.alpha, "recursive": self.recursive}
|
|
|
|
def set_params(self, **parameters):
|
|
for parameter, value in parameters.items():
|
|
setattr(self, parameter, value)
|
|
return self
|
|
|
|
|
|
Parameters and init
|
|
-------------------
|
|
As :class:`model_selection.GridSearchCV` uses ``set_params``
|
|
to apply parameter setting to estimators,
|
|
it is essential that calling ``set_params`` has the same effect
|
|
as setting parameters using the ``__init__`` method.
|
|
The easiest and recommended way to accomplish this is to
|
|
**not do any parameter validation in** ``__init__``.
|
|
All logic behind estimator parameters,
|
|
like translating string arguments into functions, should be done in ``fit``.
|
|
|
|
Also it is expected that parameters with trailing ``_`` are **not to be set
|
|
inside the** ``__init__`` **method**. All and only the public attributes set by
|
|
fit have a trailing ``_``. As a result the existence of parameters with
|
|
trailing ``_`` is used to check if the estimator has been fitted.
|
|
|
|
.. _cloning:
|
|
|
|
Cloning
|
|
-------
|
|
For use with the :mod:`model_selection` module,
|
|
an estimator must support the ``base.clone`` function to replicate an estimator.
|
|
This can be done by providing a ``get_params`` method.
|
|
If ``get_params`` is present, then ``clone(estimator)`` will be an instance of
|
|
``type(estimator)`` on which ``set_params`` has been called with clones of
|
|
the result of ``estimator.get_params()``.
|
|
|
|
Objects that do not provide this method will be deep-copied
|
|
(using the Python standard function ``copy.deepcopy``)
|
|
if ``safe=False`` is passed to ``clone``.
|
|
|
|
Pipeline compatibility
|
|
----------------------
|
|
For an estimator to be usable together with ``pipeline.Pipeline`` in any but the
|
|
last step, it needs to provide a ``fit`` or ``fit_transform`` function.
|
|
To be able to evaluate the pipeline on any data but the training set,
|
|
it also needs to provide a ``transform`` function.
|
|
There are no special requirements for the last step in a pipeline, except that
|
|
it has a ``fit`` function. All ``fit`` and ``fit_transform`` functions must
|
|
take arguments ``X, y``, even if y is not used. Similarly, for ``score`` to be
|
|
usable, the last step of the pipeline needs to have a ``score`` function that
|
|
accepts an optional ``y``.
|
|
|
|
Estimator types
|
|
---------------
|
|
Some common functionality depends on the kind of estimator passed.
|
|
For example, cross-validation in :class:`model_selection.GridSearchCV` and
|
|
:func:`model_selection.cross_val_score` defaults to being stratified when used
|
|
on a classifier, but not otherwise. Similarly, scorers for average precision
|
|
that take a continuous prediction need to call ``decision_function`` for classifiers,
|
|
but ``predict`` for regressors. This distinction between classifiers and regressors
|
|
is implemented using the ``_estimator_type`` attribute, which takes a string value.
|
|
It should be ``"classifier"`` for classifiers and ``"regressor"`` for
|
|
regressors and ``"clusterer"`` for clustering methods, to work as expected.
|
|
Inheriting from ``ClassifierMixin``, ``RegressorMixin`` or ``ClusterMixin``
|
|
will set the attribute automatically. When a meta-estimator needs to distinguish
|
|
among estimator types, instead of checking ``_estimator_type`` directly, helpers
|
|
like :func:`base.is_classifier` should be used.
|
|
|
|
Working notes
|
|
-------------
|
|
|
|
For unresolved issues, TODOs, and remarks on ongoing work, developers are
|
|
advised to maintain notes on the `GitHub wiki
|
|
<https://github.com/scikit-learn/scikit-learn/wiki>`__.
|
|
|
|
Specific models
|
|
---------------
|
|
|
|
Classifiers should accept ``y`` (target) arguments to ``fit``
|
|
that are sequences (lists, arrays) of either strings or integers.
|
|
They should not assume that the class labels
|
|
are a contiguous range of integers;
|
|
instead, they should store a list of classes
|
|
in a ``classes_`` attribute or property.
|
|
The order of class labels in this attribute
|
|
should match the order in which ``predict_proba``, ``predict_log_proba``
|
|
and ``decision_function`` return their values.
|
|
The easiest way to achieve this is to put::
|
|
|
|
self.classes_, y = np.unique(y, return_inverse=True)
|
|
|
|
in ``fit``.
|
|
This returns a new ``y`` that contains class indexes, rather than labels,
|
|
in the range [0, ``n_classes``).
|
|
|
|
A classifier's ``predict`` method should return
|
|
arrays containing class labels from ``classes_``.
|
|
In a classifier that implements ``decision_function``,
|
|
this can be achieved with::
|
|
|
|
def predict(self, X):
|
|
D = self.decision_function(X)
|
|
return self.classes_[np.argmax(D, axis=1)]
|
|
|
|
In linear models, coefficients are stored in an array called ``coef_``,
|
|
and the independent term is stored in ``intercept_``.
|
|
``sklearn.linear_model.base`` contains a few base classes and mixins
|
|
that implement common linear model patterns.
|
|
|
|
The :mod:`sklearn.utils.multiclass` module contains useful functions
|
|
for working with multiclass and multilabel problems.
|