scikit-learn/examples/linear_model/plot_sgd_loss_functions.py

"""
==========================
SGD: Convex Loss Functions
==========================

Plot the convex loss functions supported by
`sklearn.linear_model.stochastic_gradient`.
"""
print __doc__

import numpy as np
import pylab as pl
from sklearn.linear_model.sgd_fast import Hinge, \
     ModifiedHuber, SquaredLoss

###############################################################################
# Define loss funcitons
xmin, xmax = -3, 3
hinge = Hinge(1)
log_loss = lambda z, p: np.log2(1.0 + np.exp(-z))
modified_huber = ModifiedHuber()
squared_loss = SquaredLoss()

###############################################################################
# Plot loss funcitons
xx = np.linspace(xmin, xmax, 100)
pl.plot([xmin, 0, 0, xmax], [1, 1, 0, 0], 'k-',
        label="Zero-one loss")
pl.plot(xx, [hinge.loss(x, 1) for x in xx], 'g-',
        label="Hinge loss")
pl.plot(xx, [log_loss(x, 1) for x in xx], 'r-',
        label="Log loss")
pl.plot(xx, [modified_huber.loss(x, 1) for x in xx], 'y-',
        label="Modified huber loss")
#pl.plot(xx, [2.0*squared_loss.loss(x, 1) for x in xx], 'c-',
#        label="Squared loss")
pl.ylim((0, 5))
pl.legend(loc="upper right")
pl.xlabel(r"$y \cdot f(x)$")
pl.ylabel("$L(y, f(x))$")
pl.show()
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`"""`
			`==========================`
			`SGD: Convex Loss Functions`
			`==========================`

pep8 2011-12-24 02:12:26 +08:00			`Plot the convex loss functions supported by`
			`sklearn.linear_model.stochastic_gradient`.
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`"""`
ENH : pyflakes on examples to avoid useless imports + addint print __doc__ 2010-11-01 23:46:40 +08:00			`print __doc__`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00
			`import numpy as np`
			`import pylab as pl`
Move project directory from scikits.learn to sklearn 2011-09-02 17:00:02 +08:00			`from sklearn.linear_model.sgd_fast import Hinge, \`
finalized sgd module renaming. renamed ClassifierSGD to SGDClassifier (same for Regressor) updated setup.py files updated examples updated docs 2010-11-30 19:14:30 +08:00			`ModifiedHuber, SquaredLoss`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00
ENH : pyflakes on examples to avoid useless imports + addint print __doc__ 2010-11-01 23:46:40 +08:00			`###############################################################################`
			`# Define loss funcitons`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`xmin, xmax = -3, 3`
FIX SGD loss example for new hinge loss. 2012-02-02 05:52:37 +08:00			`hinge = Hinge(1)`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`log_loss = lambda z, p: np.log2(1.0 + np.exp(-z))`
			`modified_huber = ModifiedHuber()`
changend "squarederror" to "squaredloss". fixed weighted class example. 2010-11-23 03:12:33 +08:00			`squared_loss = SquaredLoss()`
ENH : pyflakes on examples to avoid useless imports + addint print __doc__ 2010-11-01 23:46:40 +08:00
			`###############################################################################`
			`# Plot loss funcitons`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`xx = np.linspace(xmin, xmax, 100)`
bugfix in plot_loss_functions (import loss functions). added dense SGD to documentation. 2010-11-06 00:30:15 +08:00			`pl.plot([xmin, 0, 0, xmax], [1, 1, 0, 0], 'k-',`
			`label="Zero-one loss")`
pep8 2011-12-24 02:12:26 +08:00			`pl.plot(xx, [hinge.loss(x, 1) for x in xx], 'g-',`
bugfix in plot_loss_functions (import loss functions). added dense SGD to documentation. 2010-11-06 00:30:15 +08:00			`label="Hinge loss")`
pep8 2011-12-24 02:12:26 +08:00			`pl.plot(xx, [log_loss(x, 1) for x in xx], 'r-',`
bugfix in plot_loss_functions (import loss functions). added dense SGD to documentation. 2010-11-06 00:30:15 +08:00			`label="Log loss")`
pep8 2011-12-24 02:12:26 +08:00			`pl.plot(xx, [modified_huber.loss(x, 1) for x in xx], 'y-',`
bugfix in plot_loss_functions (import loss functions). added dense SGD to documentation. 2010-11-06 00:30:15 +08:00			`label="Modified huber loss")`
pep8 2011-12-24 02:12:26 +08:00			`#pl.plot(xx, [2.0*squared_loss.loss(x, 1) for x in xx], 'c-',`
changend "squarederror" to "squaredloss". fixed weighted class example. 2010-11-23 03:12:33 +08:00			`# label="Squared loss")`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`pl.ylim((0, 5))`
			`pl.legend(loc="upper right")`
more SGD documentation added: example, math formulation , implementation details. adopted the seperating HP example from the svm package for sgd. 2010-10-27 03:11:18 +08:00			`pl.xlabel(r"$y \cdot f(x)$")`
			`pl.ylabel("$L(y, f(x))$")`
additional documentation for sgd (plot of various convex loss functions). TODOs in sgd_fast_sparse.pyx done. 2010-10-26 04:37:57 +08:00			`pl.show()`