check_CH() R function from [ulrb]

Check Calinski-Harabasz index

Calculates Calinski-Harabasz pseudo F-statistic (CH) for a given sample


check_CH(
  data,
  sample_id,
  samples_col = "Sample",
  abundance_col = "Abundance",
  range = 3:10,
  with_plot = FALSE,
  ...
)

Arguments

data: A data.frame with, at least, a column for Abundance and Sample. Additional columns are allowed.
sample_id: String with name of the sample to apply this function.
samples_col: String with name of column with sample names.
abundance_col: String with name of column with abundance values.
range: The range of values of k to test, default is from 3 to 10.
with_plot: If FALSE (default) returns a vector, but if TRUE will return a plot with the scores.
...: Extra arguments.

Returns

Vector or plot with Calinski-Harabasz index for each pre-specified k.

Details

CH is an index used to decide the number of clusters in a clustering algorithm. This function, check_CH(), calculates the CH index for every k in a pre-specified range of values. Thus providing a score for each number of clusters tested (k). The default range of cluster values (k) is range = 3:10 (see why this is in Pascoal et al., 2024, in peer review). However, this function may calculate the CH index for all possible k's.

Note that CH index is not an absolute value that indicates the quality of a single clustering. Instead, it allows the comparison of clustering results. Thus, if you have several clusterings, the best one will be the one with higher CH index.

Data input

This function takes a data.frame with a column for samples and a column for abundance (minimum), but can take any number of other columns. It will then filter the specific sample that you want to analyze. You can also pre-filter for your specific sample, but you still need to provide the sample ID (sample_id) and the table always needs a column for Sample and another for Abundance (indicate how you name them with the arguments samples_col and abundance_col).

Output options

The default option returns a vector with CH scores for each k. This is a simple output that can then be used for other analysis. However, we also provide the option to show a plot (set with_plot = TRUE) with the CH score for each k.

Explanation of Calinski-Harabasz index

The CH index is a variance ratio criterion , it measures both separation and density of the clusters. The higher, the better, because it means that the points within the same cluster are close to each other; and the different clusters are well separated.

You can see CH index as:

CH = \frac{\text{inter cluster dispersion}}{\text{intra cluster dispersion}}

To calculate inter-cluster:

Let $k$ be the number of clusters and BGSS be the Between-group sum of squares,

inter-cluster dispersion is

\frac{BGSS}{(k-1)}

To calculate BGSS:

Let $n_k$ be the number of observations in a cluster, $C$ be the centroid of the dataset (barycenter) and $C_k$ the centroid of a cluster,

BGSS = \sum_{k = 1}^{k}{n_k * \left\lvert C_k-C \right\rvert^2}

Thus, the BGSS multiplies the distance between the cluster centroid and the centroid of the whole dataset, by all observations in a given cluster, for all clusters.

To calculate intra-cluster dispersion:

Let $WGSS$ be the Within Group Sum of Squares and $N$ be the total number of observations in the dataset.

intra-cluster dispersion

\frac{WGSS}{(N-1)}

Let $X_ik$ be i'th observation of a cluster and $n_k$ be the number of observations in a cluster.

WGSS = \sum_{k=1}^{k}\sum_{i=1}^{n_k}\left\lvert X_ik - C_k \right\rvert

Thus, WGSS measures the distance between observations and their cluster center; if divided by the total number of observations, then gives a sense of intra-dispersion.

Finally, the CH index can be given by:

CH = \frac{\sum_{k = 1}^{k}{n_k * \left\lvert C_k-C \right\rvert^2}}{\sum_{k=1}^{k}\sum_{i=1}^{n_k}\left\lvert X_ik - C_k \right\rvert}\frac{(N-k)}{(k-1)}

Examples


library(dplyr)
# Just scores
check_CH(nice_tidy, sample_id = "ERR2044662")

# To change range
check_CH(nice_tidy, sample_id = "ERR2044662", range = 4:11)

# To see a simple plot
check_CH(nice_tidy, sample_id = "ERR2044662", range = 4:11, with_plot=TRUE)

References

Calinski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics - Theory and Methods, 3(1), 1–27. Pascoal et al. (2025). Definition of the microbial rare biosphere through unsupervised machine learning. Communications Biology.

check_CH function