# Adaptive Gibbs Sampler for Nonparametric Model-based Clustering using models from the IMIFA family Carries out Gibbs sampling for all models from the IMIFA family, facilitating model-based clustering with dimensionally reduced factor-analytic covariance structures, with automatic estimation of the number of clusters and cluster-specific factors as appropriate to the method employed. Factor analysis with one group (FA/IFA), finite mixtures (MFA/MIFA), overfitted mixtures (OMFA/OMIFA), infinite factor models which employ the multiplicative gamma process (MGP) shrinkage prior (IFA/MIFA/OMIFA/IMIFA), and infinite mixtures which employ Pitman-Yor / Dirichlet Process Mixture Models (IMFA/IMIFA) are all provided. ```r mcmc_IMIFA(dat, method = c("IMIFA", "IMFA", "OMIFA", "OMFA", "MIFA", "MFA", "IFA", "FA", "classify"), range.G = NULL, range.Q = NULL, MGP = mgpControl(...), BNP = bnpControl(...), mixFA = mixfaControl(...), alpha = NULL, storage = storeControl(...), ...) ## S3 method for class 'IMIFA' print(x, ...) ## S3 method for class 'IMIFA' summary(object, ...) ``` ## Arguments - `dat`: A matrix or data frame such that rows correspond to observations (`N`) and columns correspond to variables (`P`). Non-numeric variables will be discarded if they are explicitly coded as factors or ordinal factors; otherwise they will be treated as though they were continuous. Rows with missing entries will be also be automatically removed. - `method`: An acronym for the type of model to fit where: - **`"FA"`**: Factor Analysis - **`"IFA"`**: Infinite Factor Analysis - **`"MFA"`**: Mixtures of Factor Analysers - **`"MIFA"`**: Mixtures of Infinite Factor Analysers - **`"OMFA"`**: Overfitted Mixtures of Factor Analysers - **`"OMIFA"`**: Overfitted Mixtures of Infinite Factor Analysers - **`"IMFA"`**: Infinite Mixtures of Factor Analysers - **`"IMIFA"`**: Infinite Mixtures of Infinite Factor Analysers In principle, of course, one could overfit the `"MFA"` or `"MIFA"` models, but it is recommend to use the corresponding model options which begin with `O' instead. Note that the `"classify"` method is not yet implemented. - `range.G`: Depending on the method employed, either the range of values for the number of clusters, or the conservatively high starting value for the number of clusters. Defaults to (and must be!) `1` for the `"FA"` and `"IFA"` methods. For the `"MFA"` and `"MIFA"` models this is to be given as a range of candidate models to explore. For the `"OMFA"`, `"OMIFA"`, `"IMFA"`, and `"IMIFA"` models, this is the conservatively high number of clusters with which the chain is to be initialised (default = `max(25, ceiling(3 * log(N)))` for large N, or `min(N-1, ceiling(3 * log(N)))` for small N<=50). For the `"OMFA"`, and `"OMIFA"` models this upper limit remains fixed for the entire length of the chain; the upper limit for the for the `"IMFA"` and `"IMIFA"` models can be specified via `trunc.G` (see `bnpControl`), which must satisfy `range.G <= trunc.G < N`. If `length(range.G) * length(range.Q)` is large, consider not storing unnecessary parameters (via `storeControl`), or breaking up the range of models to be explored into chunks and sending each chunk to `get_IMIFA_results` separately. - `range.Q`: Depending on the method employed, either the range of values for the number of latent factors or, for methods ending in IFA, the conservatively high starting value for the number of cluster-specific factors, in which case the default starting value is `round(3 * log(P))`. For methods ending in IFA, different clusters can be modelled using different numbers of latent factors (incl. zero); for methods not ending in IFA it is possible to fit zero-factor models, corresponding to simple diagonal covariance structures. For instance, fitting the `"IMFA"` model with `range.Q=0` corresponds to a vanilla Pitman-Yor / Dirichlet Process Mixture Model. If `length(range.G) * length(range.Q)` is large, consider not storing unnecessary parameters (via `storeControl`), or breaking up the range of models to be explored into chunks and sending each chunk to `get_IMIFA_results`. See `Ledermann` for bounds on `range.Q`; this is useful in both the finite factor and infinite factor settings, as one may wish to ensure the fixed number of factors, or upper limits on the number of factors, respectively, respects this bound to yield indentifiable solutions, particularly in low-dimensional settings. It has also been argued that `range.Q` should not exceed `floor((P - 1)/2)`. In both cases, a warning is returned. - `MGP`: A list of arguments pertaining to the multiplicative gamma process (MGP) shrinkage prior and adaptive Gibbs sampler (AGS). For use with the infinite factor models `"IFA"`, `"MIFA"`, `"OMIFA"`, and `"IMIFA"` only. Defaults are set by a call to `mgpControl`, with further checking of validity by `MGP_check` (though arguments can also be supplied here directly). - `BNP`: A list of arguments pertaining to the Bayesian Nonparametric Pitman-Yor / Dirichlet process priors, for use with the infinite mixture models `"IMFA"` and `"IMIFA"`, or select arguments related to the Dirichlet concentration parameter for the overfitted mixtures `"OMFA"` and `"OMIFA"`. Defaults are set by a call to `bnpControl` (though arguments can also be supplied here directly). - `mixFA`: A list of arguments pertaining to **all other** aspects of model fitting, e.g. MCMC settings, cluster initialisation, and hyperparameters common to every `method` in the `IMIFA` family. Defaults are set by a call to `mixfaControl` (though arguments can also be supplied here directly). - `alpha`: Depending on the method employed, either the hyperparameter of the Dirichlet prior for the cluster mixing proportions, or the Pitman-Yor / Dirichlet process concentration parameter. Defaults to `1` for the finite mixture models `"MFA"` and `"MIFA"`, and must be a strictly positive scalar. Not relevant for the `"FA"` and `"IFA"` methods. - **Under the `"IMFA"` and `"IMIFA"` models:**: `alpha` defaults to a simulation from the prior if `learn.alpha` is `TRUE`, otherwise `alpha` **must** be specified. Must be positive, unless non-zero `discount` is supplied or `learn.d=TRUE` (the default), in which case it must be greater than `-discount`. Under certain conditions, `alpha` can remain fixed at `0` (see `bnpControl`). Additionally, when `discount` is negative, `alpha` must be a positive integer multiple of `abs(discount)` (default=`range.G * abs(discount)`). - **Under the `"OMFA"` and `"OMIFA"` models:**: `alpha` defaults to a simulation from the prior if `learn.alpha` is `TRUE`, otherwise `alpha` defaults to `0.5/range.G`. If supplied, `alpha` must be positive, and you are supplying the numerator of `alpha/range.G`. If `alpha` remains fixed (i.e. `learn.alpha=FALSE`), `alpha` should be less than half the dimension (per cluster!) of the free parameters of the smallest model considered in order to ensure superfluous clusters are emptied (for `"OMFA"`, this corresponds to the smallest `range.Q`; for `"OMIFA"`, this corresponds to a zero-factor model) [see: `PGMM_dfree` and Rousseau and Mengersen (2011)]. See `bnpControl` for further details of specifying `alpha` or specifying a prior for `alpha` under the `"IMFA"`, `"IMIFA"`, `"OMFA"`, or `"OMIFA"` methods. - `storage`: A vector of named logical indicators governing storage of parameters of interest for all models in the IMIFA family. Defaults are set by a call to `storeControl`. It may be useful not to store certain parameters if memory is an issue. - `...`: An alternative means of passing control parameters directly via the named arguments of `mixfaControl`, `mgpControl`, `bnpControl`, and `storeControl`. Do not pass the output from calls to those functions here! - `x, object`: Object of class `"IMIFA"`, for the `print.IMIFA` and `summary.IMIFA` functions, respectively. ## Returns A list of lists of lists of class `"IMIFA"` to be passed to `get_IMIFA_results`. If the returned object is `x`, candidate models are accessible via subsetting, where `x` is of the following form: `x[[1:length(range.G)]][[1:length(range.Q)]]`. However, these objects of class "IMIFA" should rarely if ever be manipulated by hand - use of the `get_IMIFA_results` function is **strongly** advised. ## Details Creates a raw object of class `"IMIFA"` from which the optimal/modal model can be extracted by `get_IMIFA_results`. Dedicated `print` and `summary` functions exist for objects of class `"IMIFA"`. ## Note Further control over the specification of advanced function arguments can be obtained with recourse to the following functions: - **‘mgpControl’**: Supply arguments (with defaults) pertaining to the multiplicative gamma process (MGP) shrinkage prior and adaptive Gibbs sampler (AGS). For use with the infinite factor models `"IFA"`, `"MIFA"`, `"OMIFA"`, and `"IMIFA"` only. - **‘bnpControl’**: Supply arguments (with defaults) pertaining to the Bayesian Nonparametric Pitman-Yor / Dirichlet process priors, for use with the infinite mixture models `"IMFA"` and `"IMIFA"`. Certain arguments related to the Dirichlet concentration parameter for the overfitted mixtures `"OMFA"` and `"OMIFA"` can be supplied in this manner also. - **‘mixfaControl’**: Supply arguments (with defaults) pertaining to **all other** aspects of model fitting (e.g. MCMC settings, cluster initialisation, and hyperparameters common to every `method` in the `IMIFA` family. - **‘storeControl’**: Supply logical indicators governing storage of parameters of interest for all models in the IMIFA family. It may be useful not to store certain parameters if memory is an issue (e.g. for large data sets or for a large number of MCMC iterations after burnin and thinning). Note however that the named arguments of these functions can also be supplied directly. Parameter starting values are obtained by simulation from the relevant prior distribution specified in these control functions, though initial means and mixing proportions are computed empirically. ## Examples ```r # data(olive) # data(coffee) # Fit an IMIFA model to the olive data. Accept all defaults. # simIMIFA <- mcmc_IMIFA(olive, method="IMIFA") # summary(simIMIFA) # Fit an IMIFA model assuming a Pitman-Yor prior. # Control the balance between the DP and PY priors using the kappa parameter. # simPY <- mcmc_IMIFA(olive, method="IMIFA", kappa=0.75) # summary(simPY) # Fit a MFA model to the scaled olive data, with isotropic uniquenesses (i.e. MPPCA). # Allow diagonal covariance as a special case where range.Q = 0. # Don't store the scores. Accept all other defaults. # simMFA <- mcmc_IMIFA(olive, method="MFA", n.iters=10000, range.G=3:6, range.Q=0:3, # score.switch=FALSE, centering=FALSE, uni.type="isotropic") # Fit a MIFA model to the centered & scaled coffee data, w/ cluster labels initialised by K-Means. # Note that range.Q doesn't need to be specified. Allow IFA as a special case where range.G=1. # simMIFA <- mcmc_IMIFA(coffee, method="MIFA", n.iters=10000, range.G=1:3, z.init="kmeans") # Fit an IFA model to the centered and pareto scaled olive data. # Note that range.G doesn't need to be specified. We can optionally supply a range.Q starting value. # Enforce additional shrinkage using alpha.d1, alpha.d2, prop, and eps (via mgpControl()). # simIFA <- mcmc_IMIFA(olive, method="IFA", n.iters=10000, range.Q=4, scaling="pareto", # alpha.d1=2.5, alpha.d2=4, prop=0.6, eps=0.12) # Fit an OMIFA model to the centered & scaled coffee data. # Supply a sufficiently small alpha value. Try varying other hyperparameters. # Accept the default value for the starting number of factors, # but supply a value for the starting number of clusters. # Try constraining uniquenesses to be common across both variables and clusters. # simOMIFA <- mcmc_IMIFA(coffee, method="OMIFA", range.G=10, psi.alpha=3, # phi.hyper=c(2, 1), alpha=0.8, uni.type="single") ``` ## References Murphy, K., Viroli, C., and Gormley, I. C. (2020) Infinite mixtures of infinite factor analysers, **Bayesian Analysis**, 15(3): 937-963. . Bhattacharya, A. and Dunson, D. B. (2011) Sparse Bayesian infinite factor models, **Biometrika**, 98(2): 291-306. Kalli, M., Griffin, J. E. and Walker, S. G. (2011) Slice sampling mixture models, **Statistics and Computing**, 21(1): 93-105. Rousseau, J. and Mengersen, K. (2011) Asymptotic Behaviour of the posterior distribution in overfitted mixture models, **Journal of the Royal Statistical Society: Series B (Statistical Methodology)**, 73(5): 689-710. McNicholas, P. D. and Murphy, T. B. (2008) Parsimonious Gaussian mixture models, **Statistics and Computing**, 18(3): 285-296. ## See Also `get_IMIFA_results`, `mixfaControl`, `mgpControl`, `bnpControl`, `storeControl`, `Ledermann` ## Author(s) Keefe Murphy -

mcmc_IMIFA function