* FIX MAE reg. criterion: Use safe_realloc to avoid memory leak
* Release GIL in safe_realloc and clean up scaffolding
* As gil is released in safe_realloc, no need of a with gil block
* Use except * to propagate error in all cdef functions
* Don't use except * for functions that return python objects
* Don't use except * for the comparison function passed to qsort
* Omissions and Errors
* Use safe_realloc now that gil is released there
* Fix realloc size
* Acquire GIL only if we need to raise
* Use except * more judiciously; Release gil only when raising; Add comments to clarify
* Actually that was unneeded; realloc will also allocate for the first time
* StackRecord*, PriorityHeapRecord* to fused type realloc_ptr; Use safe_realloc
* Use except -1 to propagate exceptions. This should avoid overheads
* Fix docstrings and add return 0 to reset methods
* TYPO
* REVIEW Remove redundant MemoryError raising calls
* feature: add beta-threshold early stopping for decision tree growth
* check if value of beta is greater than or equal to 0
* test if default value of beta is 0 and edit input validation error message
* feature: separately validate beta for reg. and clf., and add tests for it
* feature: add beta to forest-based ensemble methods
* feature: add separate condition to determine that beta is float
* feature: add beta to gradient boosting estimators
* rename parameter to min_impurity_split, edit input validation and associated tests
* chore: fix spacing in forest and force recompilation of grad boosting extension
* remove trivial comment in grad boost and add whats new
* edit wording in test comment / rebuild
* rename constant with the same name as our parameter
* edit line length for what's new
* remove constant and set min_impurity_split to 1e-7 by default
* fix docstrings for new default
* fix defaults in gradientboosting and forest classes
Uses the heapsort version that I'm familiar with: linear-time heapify
followed by n delete-min operations. Also cache-friendlier by copying
active features into a temporary array.
Random forest training time on covertype:
now 34.4123s
before 60.1179s
* avoid segfault (#2726) by setting base on numpy arrays
* A struct array for tree nodes and values array (size indeterminate at
compile time) are the only underlying structure, so each node is locally
grouped memory, and joblib dumps will save only two numpy files per
tree.
* predict() uses the value array's take method, reducing code repetition
All mallocs should be checked now. Some notes for the next person to touch
this code, or do anything with malloc in Cython code:
* with gil: raise MemoryError() didn't work as expected, as a surrounding
nogil block catches and logs the exception, then proceeds as if nothing
happened.
* There were several opportunities for double free in the code. This is
handled by letting the destructor handle the freeing.
* Similarly, constructors failed to set pointers to NULL so __dealloc__
would try to free unitialized pointers.
* Out of memory might leave trees in an inconsistent state, but will not
cause a double free. A subsequent successful fit should produce a
consistent state, although I didn't test this.
* Replaced a free/calloc with a realloc; realloc(NULL, n) = malloc(n).
* free(NULL) is a harmless no-op per ISO C89.
Also performed an optimization: the pre-sort splitter was using size_t for
an array of booleans. unsigned char is faster and smaller.
Finally, fixed some cosmetic issues (long lines).
Jérémie du Boisberranger