scikit-learn/sklearn/ensemble/_hist_gradient_boosting/predictor.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

126 lines
4.0 KiB
Python
Raw Normal View History

"""
This module contains the TreePredictor class which is used for prediction.
"""
# Author: Nicolas Hug
import numpy as np
ENH Native support for missing values in GBDTs (#13911) * Added NaN support in mapper * pep * WIP * some more * WIP * WIP * bug fix * basic tests * some doc * avoid some interactions * Added tag * better test * decent test + fix bug * add missing_fraction param to benchmark * bin training and validation data separately * shorter test * Map missing values to first bin instead of last * pep8 * Added whats new entry * avoid some python interactions * make predict_binned work * fixed bug due to offset in bin_thresholds_ attribute * more sensible binning strat * typo * user name * Add small test * convert to fortran array in tests * some doc * Added function test * pep8 * Bin validation data using binmaper of training data * Allocate first bin for missing entries based on the whole data, not just training data. * Addressed Thomas' comments * Update sklearn/ensemble/_hist_gradient_boosting/tests/test_grower.py * Addressed Guillaume's comments * always allocate first bin for missing values * reduce diff * minor more consistent test * typo * WIP * some doc * reduce diff * pep8 * minor * remove prints * towards nan only splits * don't check right to left on split_on_nan * cleaups * format and comment * Fixed bug + added more tests * refactor tests * put back n_threads to max value * minor changes * minor cleaning * Add (failing) test that checks equivalence with min max imputation * Decrease the likelihood of ties when training the trees * More robust test * Fix pytest parametrization * Check bin thresholds in test * Try to make the test even easier to see if the Linux 32bit build would pass in this case * Don't check last non-missing bin if there's no nan * Improve min-max imputation test * FIX: _find_best_bin_to_split_right_to_left is still required even when left to right wants to split on nans * comments * remove split_on_nan * ooops deleted useless files * Got rid of individual checks in predictor code +inf thresholds are only allowed in a split on nan situation. Thresholds that are computed as +inf are capped to a very high constant value * can also remove special case in binning code * minor typos + more consistent test * renamed types -> common * 1e300 -> almost inf * added user guide section on missing values * Addressed Olivier's comment + updated whatsnew * addressed comments * Fix doctest formatting * Fix nan predictive doctest
2019-08-21 17:22:00 +08:00
from .common import Y_DTYPE
from ._predictor import _predict_from_raw_data
from ._predictor import _predict_from_binned_data
from ._predictor import _compute_partial_dependence
class TreePredictor:
"""Tree class used for predictions.
Parameters
----------
2019-05-29 06:52:09 +08:00
nodes : ndarray of PREDICTOR_RECORD_DTYPE
The nodes of the tree.
binned_left_cat_bitsets : ndarray of shape (n_categorical_splits, 8), \
dtype=uint32
Array of bitsets for binned categories used in predict_binned when a
split is categorical.
raw_left_cat_bitsets : ndarray of shape (n_categorical_splits, 8), \
dtype=uint32
Array of bitsets for raw categories used in predict when a split is
categorical.
"""
def __init__(self, nodes, binned_left_cat_bitsets, raw_left_cat_bitsets):
self.nodes = nodes
self.binned_left_cat_bitsets = binned_left_cat_bitsets
self.raw_left_cat_bitsets = raw_left_cat_bitsets
def get_n_leaf_nodes(self):
"""Return number of leaves."""
return int(self.nodes["is_leaf"].sum())
def get_max_depth(self):
"""Return maximum depth among all leaves."""
return int(self.nodes["depth"].max())
def predict(self, X, known_cat_bitsets, f_idx_map, n_threads):
"""Predict raw values for non-binned data.
Parameters
----------
X : ndarray, shape (n_samples, n_features)
The input samples.
known_cat_bitsets : ndarray of shape (n_categorical_features, 8)
Array of bitsets of known categories, for each categorical feature.
f_idx_map : ndarray of shape (n_features,)
Map from original feature index to the corresponding index in the
known_cat_bitsets array.
n_threads : int
Number of OpenMP threads to use.
Returns
-------
y : ndarray, shape (n_samples,)
The raw predicted values.
"""
out = np.empty(X.shape[0], dtype=Y_DTYPE)
_predict_from_raw_data(
self.nodes,
X,
self.raw_left_cat_bitsets,
known_cat_bitsets,
f_idx_map,
n_threads,
out,
)
return out
def predict_binned(self, X, missing_values_bin_idx, n_threads):
"""Predict raw values for binned data.
Parameters
----------
X : ndarray, shape (n_samples, n_features)
The input samples.
ENH Native support for missing values in GBDTs (#13911) * Added NaN support in mapper * pep * WIP * some more * WIP * WIP * bug fix * basic tests * some doc * avoid some interactions * Added tag * better test * decent test + fix bug * add missing_fraction param to benchmark * bin training and validation data separately * shorter test * Map missing values to first bin instead of last * pep8 * Added whats new entry * avoid some python interactions * make predict_binned work * fixed bug due to offset in bin_thresholds_ attribute * more sensible binning strat * typo * user name * Add small test * convert to fortran array in tests * some doc * Added function test * pep8 * Bin validation data using binmaper of training data * Allocate first bin for missing entries based on the whole data, not just training data. * Addressed Thomas' comments * Update sklearn/ensemble/_hist_gradient_boosting/tests/test_grower.py * Addressed Guillaume's comments * always allocate first bin for missing values * reduce diff * minor more consistent test * typo * WIP * some doc * reduce diff * pep8 * minor * remove prints * towards nan only splits * don't check right to left on split_on_nan * cleaups * format and comment * Fixed bug + added more tests * refactor tests * put back n_threads to max value * minor changes * minor cleaning * Add (failing) test that checks equivalence with min max imputation * Decrease the likelihood of ties when training the trees * More robust test * Fix pytest parametrization * Check bin thresholds in test * Try to make the test even easier to see if the Linux 32bit build would pass in this case * Don't check last non-missing bin if there's no nan * Improve min-max imputation test * FIX: _find_best_bin_to_split_right_to_left is still required even when left to right wants to split on nans * comments * remove split_on_nan * ooops deleted useless files * Got rid of individual checks in predictor code +inf thresholds are only allowed in a split on nan situation. Thresholds that are computed as +inf are capped to a very high constant value * can also remove special case in binning code * minor typos + more consistent test * renamed types -> common * 1e300 -> almost inf * added user guide section on missing values * Addressed Olivier's comment + updated whatsnew * addressed comments * Fix doctest formatting * Fix nan predictive doctest
2019-08-21 17:22:00 +08:00
missing_values_bin_idx : uint8
Index of the bin that is used for missing values. This is the
index of the last bin and is always equal to max_bins (as passed
to the GBDT classes), or equivalently to n_bins - 1.
n_threads : int
Number of OpenMP threads to use.
Returns
-------
y : ndarray, shape (n_samples,)
The raw predicted values.
"""
out = np.empty(X.shape[0], dtype=Y_DTYPE)
_predict_from_binned_data(
self.nodes,
X,
self.binned_left_cat_bitsets,
missing_values_bin_idx,
n_threads,
out,
)
return out
def compute_partial_dependence(self, grid, target_features, out):
"""Fast partial dependence computation.
Parameters
----------
grid : ndarray, shape (n_samples, n_target_features)
The grid points on which the partial dependence should be
evaluated.
target_features : ndarray, shape (n_target_features)
The set of target features for which the partial dependence
should be evaluated.
out : ndarray, shape (n_samples)
The value of the partial dependence function on each grid
point.
"""
_compute_partial_dependence(self.nodes, grid, target_features, out)