-
Now we're implementing something called spam classfier.
-
Boosting is one of the famous ensemble learning
-
First let's introduce a problem of spam email
-
we can think of being positive as one indication of spam
-
and negative as non spam
-
This is the core algorithm of boosting algorithm
-
First learn a simple spam indicator. set that as a rule
-
Then for every other spam, combine more and more until whole one rule
-
But it's not be the case to include directly whole data as a rule. That would be called overfiting.
-
This is just how overall boosting should look like.
-
One of the simplest method of 1 is just taking subset(email) and learn from it.
-
Then for combine, just average of it
-
So if we take it to the learning algorithm
-
We have n points, and the attribute is all 0th order poloynomial.
-
We take one data point and take to the learner, the results is the mean for one particuular data points.
-
The ensemble outputs are just mean with n-numbers.
-
First we take randomly 5 subsets; each subset takes randomly 5 example from it.
-
the red points is the all available data points, and the green one is cross validation points
-
Then we apply third order polynomial, and average each of the subsets.
-
The curve will then produced 5 lines.
-
We can see that all line perform polynomial regression.
-
Some of the lines match point 1 to 4, but last two points maybe missed
-
The red line is the average of all subsets with 3rd degree polynomial
-
The blue line is
the average of all subsets with 4th degree polynomial
-
These overall called bagging, takes bag of data, but not necessarily called boosting.
