# Calculation of Pairwise Distances for Continuous Data Computes a distance matrix for continuous data with support for multiple distance metrics, scaling methods, dimensionality reduction, and validation data. The function implements various distance calculation approaches as described in van de Velden et al. (2024), including options for commensurable distances and variable weighting. ```r ndist(x, validate_x = NULL, commensurable = TRUE, method = "manhattan", sig = NULL, scaling = "none", ncomp = ncol(x), threshold = NULL, weights = rep(1, ncol(x))) ``` ## Arguments - `x`: A data frame or matrix of continuous input variables. - `validate_x`: Optional data frame or matrix for validation data. If provided, distances are computed between observations in `validate_x` and `x`. Default is `NULL`. - `commensurable`: Logical. If `TRUE`, standardizes each variable's distance matrix by dividing by its mean distance, making distances comparable across variables. Default is `FALSE`. - `method`: Character string specifying the distance metric. Options include `"manhattan"`, `"euclidean"`, and `"mahalanobis"`. Default is `"manhattan"`. - `sig`: Covariance matrix to be used when `method = "mahalanobis"`. If `NULL`, computed from the data. Default is `NULL`. - `scaling`: Character string specifying the scaling method. Options: * `"none"`: No scaling * `"std"`: Standardization (zero mean, unit variance) * `"range"`: Min-max scaling to [0,1] * `"pc_scores"`: PCA-based dimensionality reduction * `"robust"`: Robust scaling using median and IQR Default is `"none"`. - `ncomp`: Number of principal components to retain when `scaling = "pc_scores"`. Default is the number of columns in `x`. - `threshold`: Proportion of variance to retain when `scaling = "pc_scores"`. If specified, overrides `ncomp`. Default is `NULL`. - `weights`: Numeric vector of weights for each variable. Must have length equal to the number of variables in `x`. Default is a vector of ones. ## Details The `ndist` function provides a comprehensive framework for distance calculations in continuous data: * When `validate_x` is provided, computes distances between observations in `validate_x` and `x`. * Supports multiple scaling methods that can be applied before distance calculation. * PCA-based dimensionality reduction can be controlled either by number of components or variance threshold. * For Mahalanobis distance, handles singular covariance matrices with appropriate error messages. * Implements commensurable distances for better comparability across variables. Warning: The function validates: * Weight vector length must match the number of variables * Covariance matrix singularity for Mahalanobis distance * Compatibility of `x` and `validate_x` dimensions ## Returns A distance matrix where element [i,j] represents the distance between: * observation i and j of `x` if `validate_x` is `NULL` * observation i of `validate_x` and observation j of `x` if `validate_x` is provided ## Examples ```r library(palmerpenguins) library(rsample) penguins_cont <- palmerpenguins::penguins[, c("bill_length_mm", "bill_depth_mm", "flipper_length_mm", "body_mass_g")] penguins_cont <- penguins_cont[complete.cases(penguins_cont), ] # Basic usage dist_matrix <- ndist(penguins_cont) # Commensurable distances with standardization dist_matrix <- ndist(penguins_cont, commensurable = TRUE, scaling = "std") # PCA-based dimensionality reduction dist_matrix <- ndist(penguins_cont, scaling = "pc_scores", threshold = 0.95) # Mahalanobis distance dist_matrix <- ndist(penguins_cont, method = "mahalanobis") # Weighted Euclidean distance dist_matrix <- ndist(penguins_cont, method = "euclidean", weights = c(1, 0.5, 2, 1)) # Training-test split example with validation data set.seed(123) # Create training-test split using rsample penguins_split <- initial_split(penguins_cont, prop = 0.8) tr_penguins <- training(penguins_split) ts_penguins <- testing(penguins_split) # Basic usage with training data only dist_matrix <- ndist(tr_penguins) # Computing distances between test and training sets val_dist_matrix <- ndist(x = tr_penguins, validate_x = ts_penguins, method = "euclidean") # Using validation data with standardization val_dist_matrix_std <- ndist(x = tr_penguins, validate_x = ts_penguins, scaling = "std", method = "manhattan") # Validation with PCA and commensurability val_dist_matrix_pca <- ndist(x = tr_penguins, validate_x = ts_penguins, scaling = "pc_scores", ncomp = 2, commensurable = TRUE) # Validation with robust scaling and custom weights val_dist_matrix_robust <- ndist(x = tr_penguins, validate_x = ts_penguins, scaling = "robust", weights = c(1, 0.5, 2, 1)) # Mahalanobis distance with validation data val_dist_matrix_mahal <- ndist(x = tr_penguins, validate_x = ts_penguins, method = "mahalanobis") ``` ## References van de Velden, M., Iodice D'Enza, A., Markos, A., Cavicchia, C. (2024). (Un)biased distances for mixed-type data. **arXiv preprint**. Retrieved from [https://arxiv.org/abs/2411.00429](https://arxiv.org/abs/2411.00429). ## See Also `mdist` for mixed-type data distances, `cdist` for categorical data distances.

ndist function