# Applies the AdaBoost.M1 and SAMME algorithms to a data set Fits the AdaBoost.M1 (Freund and Schapire, 1996) and SAMME (Zhu et al., 2009) algorithms using classification trees as single classifiers. ```r boosting(formula, data, boos = TRUE, mfinal = 100, coeflearn = 'Breiman', control,...) ``` ## Arguments - `formula`: a formula, as in the `lm` function. - `data`: a data frame in which to interpret the variables named in `formula`. - `boos`: if `TRUE` (by default), a bootstrap sample of the training set is drawn using the weights for each observation on that iteration. If `FALSE`, every observation is used with its weights. - `mfinal`: an integer, the number of iterations for which boosting is run or the number of trees to use. Defaults to `mfinal=100` iterations. - `coeflearn`: if 'Breiman'(by default), `alpha=1/2ln((1-err)/err)` is used. If 'Freund' `alpha=ln((1-err)/err)` is used. In both cases the AdaBoost.M1 algorithm is used and `alpha` is the weight updating coefficient. On the other hand, if coeflearn is 'Zhu' the SAMME algorithm is implemented with `alpha=ln((1-err)/err)+` `ln(nclasses-1)`. - `control`: options that control details of the rpart algorithm. See rpart.control for more details. - `...`: further arguments passed to or from other methods. ## Details AdaBoost.M1 and SAMME are simple generalizations of AdaBoost for more than two classes. In AdaBoost-SAMME the individual trees are required to have an error lower than 1-1/nclasses instead of 1/2 of the AdaBoost.M1 ## Returns An object of class `boosting`, which is a list with the following components: - **formula**: the formula used. - **trees**: the trees grown along the iterations. - **weights**: a vector with the weighting of the trees of all iterations. - **votes**: a matrix describing, for each observation, the number of trees that assigned it to each class, weighting each tree by its `alpha` coefficient. - **prob**: a matrix describing, for each observation, the posterior probability or degree of support of each class. These probabilities are calculated using the proportion of votes in the final ensemble. - **class**: the class predicted by the ensemble classifier. - **importance**: returns the relative importance of each variable in the classification task. This measure takes into account the gain of the Gini index given by a variable in a tree and the weight of this tree. ## References Alfaro, E., Gamez, M. and Garcia, N. (2013): ``adabag: An R Package for Classification with Boosting and Bagging''. Journal of Statistical Software, Vol 54, 2, pp. 1--35. Alfaro, E., Garcia, N., Gamez, M. and Elizondo, D. (2008): ``Bankruptcy forecasting: An empirical comparison of AdaBoost and neural networks''. Decision Support Systems, 45, pp. 110--122. Breiman, L. (1998): ``Arcing classifiers''. The Annals of Statistics, Vol 26, 3, pp. 801--849. Freund, Y. and Schapire, R.E. (1996): ``Experiments with a new boosting algorithm''. In Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148--156, Morgan Kaufmann. Zhu, J., Zou, H., Rosset, S. and Hastie, T. (2009): ``Multi-class AdaBoost''. Statistics and Its Interface, 2, pp. 349--360. ## Author(s) Esteban Alfaro-Cortes Esteban.Alfaro@uclm.es , Matias Gamez-Martinez Matias.Gamez@uclm.es and Noelia Garcia-Rubio Noelia.Garcia@uclm.es ## See Also `predict.boosting`, `boosting.cv` ## Examples ```r ## rpart library should be loaded data(iris) iris.adaboost <- boosting(Species~., data=iris, boos=TRUE, mfinal=3) iris.adaboost ## Data Vehicle (four classes) library(mlbench) data(Vehicle) l <- length(Vehicle[,1]) sub <- sample(1:l,2*l/3) mfinal <- 3 maxdepth <- 5 Vehicle.rpart <- rpart(Class~.,data=Vehicle[sub,],maxdepth=maxdepth) Vehicle.rpart.pred <- predict(Vehicle.rpart,newdata=Vehicle[-sub, ],type="class") tb <- table(Vehicle.rpart.pred,Vehicle$Class[-sub]) error.rpart <- 1-(sum(diag(tb))/sum(tb)) tb error.rpart Vehicle.adaboost <- boosting(Class ~.,data=Vehicle[sub, ],mfinal=mfinal, coeflearn="Zhu", control=rpart.control(maxdepth=maxdepth)) Vehicle.adaboost.pred <- predict.boosting(Vehicle.adaboost,newdata=Vehicle[-sub, ]) Vehicle.adaboost.pred$confusion Vehicle.adaboost.pred$error #comparing error evolution in training and test set errorevol(Vehicle.adaboost,newdata=Vehicle[sub, ])->evol.train errorevol(Vehicle.adaboost,newdata=Vehicle[-sub, ])->evol.test plot.errorevol(evol.test,evol.train) ```

boosting function