It is easy to overfit on the 20newsgroups dataset, by letting
classifiers learn from metadata that commonly appears in newsgroup
texts, but would be useless for identifying topics outside of this set
of newsgroups in 1993.
For example, many classifiers will tell you that three of the most
informative features are "nntp", "posting", and "host", because the
NNTP-Posting-Host header appears with different frequency in different
groups.
The fetch_20newsgroups function now allows you to ask for any of the
following kinds of text to be removed:
- Newsgroup headers (which contain lots of NNTP metadata that can
identify the group)
- Signature blocks (which often contain multiple terms that uniquely
identify the person posting, which in turn identifies the group)
- Quote blocks (which contain people's e-mail addresses and large
amounts of text from another post in the same newsgroup)
The 20newsgroups classification example takes the "--filtered" flag,
which will remove all of these. This noticeably decreases the accuracy
of all classifiers, leaving room for a better method to improve the
accuracy.
- The size of the test set should be labeled (test set), not (training
set).
- The generated figure is so tall that it doesn't fit on a Mac laptop
screen; shrink it by 20%.
Replaced all BSD style licenses with "BSD 3 clause"
Not checked yet!
Removed duplicate "3 clause, 3 clause"
Removed trailing period if exists
Fixed some missed licences, still about 50 to do, but those can be automated
Think I got the last of them.
Apparently me and sed have different ideas of regex.
Found a few more
- the output will be dense and the models expect sparse input.
- the hyperparameters would need to be re-tuned for the new
dimensionality. f1 scores without re-tuning is misleading
Better have dedicated examples for random projections.
Rob Speer