multinomial_naive_bayes function

Multinomial Naive Bayes Classifier

multinomial_naive_bayes is used to fit the Multinomial Naive Bayes model.

multinomial_naive_bayes(x, y, prior = NULL, laplace = 0.5, ...)

Arguments

• x: numeric matrix with integer predictors (matrix or dgCMatrix from Matrix package).
• y: class vector (character/factor/logical).
• prior: vector with prior probabilities of the classes. If unspecified, the class proportions for the training set are used. If present, the probabilities should be specified in the order of the factor levels.
• laplace: value used for Laplace smoothing (additive smoothing). Defaults to 0.5.
• ...: not used.

Returns

multinomial_naive_bayes returns an object of class "multinomial_naive_bayes" which is a list with following components: - data: list with two components: x (matrix with predictors) and y (class variable).

• levels: character vector with values of the class variable.

• laplace: amount of Laplace smoothing (additive smoothing).

• params: matrix with class conditional parameter estimates.

• prior: numeric vector with prior probabilities.

• call: the call that produced this object.

Details

This is a specialized version of the Naive Bayes classifier, where the features represent frequencies generated by a multinomial distribution.

Sparse matrices of class "dgCMatrix" (Matrix package) are supported in order to speed up calculation times.

Please note that the Multinomial Naive Bayes is not available through the naive_bayes function.

References

Manning, C.D., Raghavan, P., & Schütze, H. (2008). An Introduction to Information Retrieval. Cambridge: Cambridge University Press (Chapter 13). Available at https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf

Examples

# library(naivebayes)

### Simulate the data:
set.seed(1)
cols <- 3 # words
rows <- 10000 # all documents
rows_spam <- 100 # spam documents

prob_word_non_spam <- prop.table(runif(cols))
prob_word_spam <- prop.table(runif(cols))

M1 <- t(rmultinom(rows_spam, size = cols, prob = prob_word_spam))
M2 <- t(rmultinom(rows - rows_spam, size = cols, prob = prob_word_non_spam))
M <- rbind(M1, M2)
colnames(M) <- paste0("word", 1:cols) ; rownames(M) <- paste0("doc", 1:rows)
head(M)
y <- c(rep("spam", rows_spam), rep("non-spam", rows - rows_spam))

### Train the Multinomial Naive Bayes
laplace <- 1
mnb <- multinomial_naive_bayes(x = M, y = y, laplace = laplace)
summary(mnb)

# Classification
head(predict(mnb, newdata = M, type = "class")) # head(mnb %class% M)

# Posterior probabilities
head(predict(mnb, newdata = M, type = "prob")) # head(mnb %prob% M)

# Parameter estimates
coef(mnb)

# Compare
round(cbind(non_spam = prob_word_non_spam, spam = prob_word_spam), 3)


### Sparse data: train the Multinomial Naive Bayes
library(Matrix)
M_sparse <- Matrix(M, sparse = TRUE)
class(M_sparse) # dgCMatrix

# Fit the model with sparse data
mnb_sparse <- multinomial_naive_bayes(M_sparse, y, laplace = laplace)

# Classification
head(predict(mnb_sparse, newdata = M_sparse, type = "class"))

# Posterior probabilities
head(predict(mnb_sparse, newdata = M_sparse, type = "prob"))

# Parameter estimates
coef(mnb_sparse)

Author(s)

Michal Majka, michalmajka@hotmail.com

See Also

predict.multinomial_naive_bayes, tables, get_cond_dist, %class%, coef.multinomial_naive_bayes