# Extract results, conduct posterior inference and compute performance metrics for MCMC samples of models from the IMIFA family This function post-processes simulations generated by `mcmc_IMIFA` for any of the IMIFA family of models. This includes accounting for label switching, and accounting for rotational invariance via Procrustean methods. It can be re-ran at little computational cost in order to extract different models explored by the sampler used for `sims`, without having to re-run the model itself. New results objects using different numbers of clusters and different numbers of factors (if visited by the model in question), or using different model selection criteria (if necessary) can be generated with ease. Posterior predictive checking of the appropriateness of the fitted model is also facilitated. ```r get_IMIFA_results(sims = NULL, burnin = 0L, thinning = 1L, G = NULL, Q = NULL, criterion = c("bicm", "aicm", "dic", "bic.mcmc", "aic.mcmc"), adapt = FALSE, G.meth = c("mode", "median"), Q.meth = c("mode", "median"), conf.level = 0.95, error.metrics = TRUE, vari.rot = FALSE, z.avgsim = FALSE, zlabels = NULL, nonempty = TRUE, ...) ## S3 method for class 'Results_IMIFA' print(x, ...) ## S3 method for class 'Results_IMIFA' summary(object, MAP = TRUE, ...) ``` ## Arguments - `sims`: An object of class `"IMIFA"` generated by `mcmc_IMIFA`. - `burnin`: Optional additional number of iterations to discard. Defaults to 0, corresponding to no additional burnin. See `mixfaControl` for the default `burnin` settings used previously by `mcmc_IMIFA`. - `thinning`: Optional interval for extra thinning to be applied. Defaults to 1, corresponding to no additional thinning. See `mixfaControl` for the default `thinning` settings used previously by `mcmc_IMIFA`. - `G`: If this argument is not specified, results will be returned with the optimal number of clusters. If different numbers of clusters were explored in `sims` for the `"MFA"` or `"MIFA"` methods, supplying an integer value allows pulling out a specific solution with `G` clusters, even if the solution is sub-optimal. Similarly, this allows retrieval of samples corresponding to a solution, if visited, with `G` clusters for the `"OMFA"`, `"OMIFA"`, `"IMFA"` and `"IMIFA"` methods. - `Q`: If this argument is not specified, results will be returned with the optimal number of factors. If different numbers of factors were explored in `sims` for the `"FA"`, `"MFA"`, `"OMFA"` or `"IMFA"` methods, this allows pulling out a specific solution with `Q` factors, even if the solution is sub-optimal. Similarly, this allows retrieval of samples corresponding to a solution, if visited, with `Q` factors for the `"IFA"`, `"MIFA"`, `"OMIFA"` and `"IMIFA"` methods if adaptation has already taken place. Can be supplied as a scalar or a vector of values for each cluster. If adaptation didn't take place during model-fitting, `adapt=TRUE` can be supplied here to determine the optimal cluster-specific numbers of non-redundant factors (see below). - `criterion`: The criterion to use for model selection, where model selection is only required if more than one model was run under the `"FA"`, `"MFA"`, `"MIFA"`, `"OMFA"` or `"IMFA"` methods when `sims` was created via `mcmc_IMIFA`. Defaults to `bicm`, but note that these are **all** calculated; this argument merely indicates which one will form the basis of the construction of the output. Note that the first three options here might exhibit bias in favour of zero-factor models for the finite factor `"FA"`, `"MFA"`, `"OMFA"` and `"IMFA"` methods and might exhibit bias in favour of one-cluster models for the `"MFA"` and `"MIFA"` methods. The `aic.mcmc` and `bic.mcmc` criteria will only be returned for finite factor models. - `adapt`: A logical indicating if adaptation should be applied to the stored loadings and scores matrices to truncate the cluster-specific number(s) of non-redundant factors. This argument is only relevant if `adapt=FALSE` was used when initially fitting the model (otherwise, adaptation has already taken place during sampling, by default), and hence defaults to `FALSE` here. Relevant parameters from `mgpControl` (namely `active.crit`, `eps`, `prop`, and `forceQg`) can be passed via the `...` construct, but will default to their values under `mgpControl` if not specified. Note that `adapt=TRUE` is only invoked if the relevant parameters were stored via `storeControl` when running the model: loadings only for `active.crit="BD"`, loadings and scores for `active.crit="SC"` for `"IFA"` models. - `G.meth`: If the object in `sims` arises from the `"OMFA"`, `"OMIFA"`, `"IMFA"` or `"IMIFA"` methods, this argument determines whether the optimal number of clusters is given by the mode or median of the posterior distribution of `G`. Defaults to `"mode"`. Often the mode and median will agree in any case. - `Q.meth`: If the object in `sims` arises from the `"IFA"`, `"MIFA"`, `"OMIFA"` or `"IMIFA"` methods, this argument determines whether the optimal number of latent factors is given by the mode or median of the posterior distribution of `Q`. Defaults to `"mode"`. Often the mode and median will agree in any case. - `conf.level`: The confidence level to be used throughout for credible intervals for all parameters of inferential interest, and error metrics if `error.metrics=TRUE`. Defaults to `0.95`. - `error.metrics`: A logical activating or deactivating posterior predictive checking: i.e. controlling whether metrics quantifying a) the posterior predictive reconstruction error (PPRE) between bin counts of the data and bin counts of replicate draws from the posterior distribution & and b) the error between the empirical and estimated covariance matrices should be computed. These are computed for every **valid** retained iteration (see `Details`). Defaults to `TRUE`, but can be time-consuming for models which achieve clustering. These error metrics, and the uncertainty associated with them, can be visualised via `plot.Results_IMIFA`. Depending on what parameters were stored when calling `mcmc_IMIFA`, potentially not all error metrics will be available to compute. The Frobenius norm is used in the computation of the PPRE, by default, but the `type` of `norm` can be changed via the `...` construct below. So too can the breakpoints (`dbreaks`) used to bin the data and the posterior predictive replicate data sets. Some caution is advised in the latter case. - `vari.rot`: Logical indicating whether the loadings matrix/matrices template(s) should be `varimax` rotated first, prior to the Procrustes rotation steps. Defaults to `FALSE`. Not necessary at all for clustering purposes, or inference on the covariance matrix, but useful if interpretable inferences on the loadings matrix/matrices are desired. Arguments to `varimax` can be passed via the `...` construct, but note that the argument `normalize` here defaults to `FALSE`. - `z.avgsim`: Logical (defaults to `FALSE`) indicating whether the clustering should also be summarised with a call to `Zsimilarity` by the clustering with minimum mean squared error to the similarity matrix obtained by averaging the stored adjacency matrices, in addition to the MAP estimate. Note that the MAP clustering is computed **conditional** on the estimate of the number of clusters (whether that be the modal estimate or the estimate according to `criterion`) and other parameters are extracted conditional on this estimate of `G`: however, in contrast, the number of distinct clusters in the summarised labels obtained by specifying `z.avgsim=TRUE` may not necessarily coincide with the MAP estimate of `G`, but it may provide a useful alternative summary of the partitions explored during the chain, and the user is free to call `get_IMIFA_results` again with the new suggested `G` value. Please be warned that this feature requires loading the `mcclust` package. This is liable to take considerable time to compute, and may not even be possible if the number of observations &/or number of stored iterations is large and the resulting matrix isn't sufficiently sparse. When `z.avgsim=TRUE`, both the summarised clustering and the similarity matrix are stored: the latter can be visualised as part of a call to `plot.Results_IMIFA`. - `zlabels`: For any method that performs clustering, the true labels can be supplied if they are known in order to compute clustering performance metrics. This also has the effect of ordering the MAP labels (and thus the ordering of cluster-specific parameters) to most closely correspond to the true labels if supplied. - `nonempty`: For `"MFA"` and `"MIFA"` models ONLY: a logical indicating whether only iterations with non-empty components should be retained. Defaults to `TRUE`, but may lead to empty chains - conversely, `FALSE` may lead to empty components and related errors. - `x, object, MAP, ...`: Arguments required for the `print.Results_IMIFA` and `summary.Results_IMIFA` functions: `x` and `object` are objects of class `"Results_IMIFA"` resulting from a call to `get_IMIFA_results`. `MAP` is a logical which governs whether a table of the MAP classification is printed, while `...` gathers additional arguments to those functions. Users can also pass the `type` argument to the `norm` function when `isTRUE(error.metrics)` and the posterior predictive reconstruction error (PPRE) is calculated. By default the Frobenius norm (`type="F"`) is employed. When `adapt=TRUE` here, relevant arguments from `mgpControl` (namely `active.crit`, `eps`, `prop`, and `forceQg`) can be supplied here too, though they retain their default values from `mgpControl` otherwise. Finally, the `...` construct also allows arguments to `varimax` to be passed to `get_IMIFA_results` itself, when `isTRUE(vari.rot)`, or arguments to `hist` when `isTRUE(error.metrics)`, in order to guide construction of the bins. Additionally, by passing the argument `dbreaks` via the `...` construct, the bins can be specified directly. However, caution is advised in doing so; in particular, the bins must be constructed on data which has been standardised in the same way as the data modelled within `mcmc_IMIFA`. ## Returns An object of class `"Results_IMIFA"` to be passed to `plot.Results_IMIFA` for visualising results. Dedicated `print` and `summary` functions also exist for objects of this class. The object, say `x`, is a list of lists, the most important components of which are: - **`Clust`**: Everything pertaining to clustering performance can be found here for all but the `"FA"` and `"IFA"` methods (or models where the estimate number of clusters is 1), in particular `x$Clust$MAP`, the MAP summary of the posterior clustering, the last valid sample of cluster labels `x$Clust$last.z`, the matrix of posterior cluster membership probabilities `x$Clust$post.prob`, and the posterior confusion matrix `x$Clust$PCM`. More detail is given if known `zlabels` are supplied: performance is always evaluated against the MAP clustering, with additional evaluation against the alternative clustering computed if `z.avgsim=TRUE`. Posterior summaries of the mixing proportions, and the concentration/discount parameters, if necessary, are also included here, as well as the last valid samples of each. - **`Error`**: Everything pertaining the model fit assessment can be found here, incl. the distribution of the PPRE values and associated bin counts for the replicate draws, as well as average error metrics (e.g. MSE, RMSE), and credible intervals quantifying the associated uncertainty, between the empirical and estimated covariance matrix/matrices, both of which are also included. - **`GQ.results`**: Everything pertaining to model choice can be found here, incl. posterior summaries for the estimated number of clusters and estimated number of factors, if applicable to the method employed. Model selection criterion values are also accessible here. - **`Means`**: Posterior summaries for the means, after conditioning on `G`. - **`Loadings`**: Posterior summaries for the factor loadings matrix/matrices, after conditioning on `G` and `Q`. Posterior mean loadings given by `x$Loadings$post.load` are given the `loadings` class for printing purposes and thus the manner in which they are displayed can be modified. The number of iterations retained for posterior summaries of the loadings may vary for different clusters for the infinite factor methods, corresponding to iterations where the cluster-specific number of factors was greater than or equal to the modal estimate of the cluster-specific number of factors. - **`Scores`**: Posterior summaries for the latent factor scores, after conditioning on the maximum of the estimated number of cluster-specific factors. Summaries are given for the **single** matrix of factor scores. See `scores_MAP` to decompose these summaries into sub-matrices according to the MAP partition (for models which achieve clustering). For the infinite factor methods, iterations where the maximum number of factors was greater than or equal to the maximum of the estimated cluster-specific factors are retained for posterior summaries of the scores, in order to preserve the estimated dimension of the scores matrices. - **`Uniquenesses`**: Posterior summaries for the uniquenesses, after conditioning on `G`. The objects `Means`, `Loadings`, `Scores` and `Uniquenesses` (if stored when calling `mcmc_IMIFA`!) also contain, as well as the posterior summaries, the entire chain of valid samples of each, as well as, for convenience, the last valid samples of each (after conditioning on the modal `G` and `Q` values, and accounting for label switching, and rotational invariance via Procrustes rotation). ## Details The function also performs post-hoc corrections for label switching, as well as post-hoc Procrustes rotation of loadings matrices and scores, in order to ensure sensible posterior parameter estimates, computes error metrics, constructs credible intervals, and generally transforms the raw `sims` object into an object of class `"Results_IMIFA"` in order to prepare the results for plotting via `plot.Results_IMIFA`. For the infinite factor methods, iterations where the maximum number of factors was greater than or equal to the maximum of the estimated cluster-specific factors are retained for posterior summaries of the scores, in order to preserve the estimated dimension of the scores matrices. Similarly, these are also the **valid** iterations used for the computation of the averages and credible intervals for the error metrics. For the finite factor models, **all** retained iterations are used in both instances (i.e. both for the scores and the error metrics). In all cases, only iterations with `G` non-empty components are retained. ## Note For the `"IMIFA"`, `"IMFA"`, `"OMIFA"`, and `"OMFA"` methods, the retained mixing proportions are renormalised after conditioning on the modal `G`. This is especially necessary for the computation of the `error.metrics`, just note that the values on which posterior inference are conducted will ever so slightly differ from the actually sampled values. Due to the way the offline label-switching correction is performed, different runs of this function may give **very slightly** different results in terms of the cluster labellings (and by extension the parameters, which are permuted in the same way), but only if the chain was run for an extremely small number of iterations, well below the number required for convergence, and samples of the cluster labels match poorly across iterations (particularly if the number of clusters suggested by those sampled labels is high). ## Examples ```r # data(coffee) # data(olive) # Run a MFA model on the coffee data over a range of clusters and factors. # simMFAcoffee <- mcmc_IMIFA(coffee, method="MFA", range.G=2:3, range.Q=0:3, n.iters=1000) # Accept all defaults to extract the optimal model. # resMFAcoffee <- get_IMIFA_results(simMFAcoffee) # Instead let's get results for a 3-cluster model, allowing Q be chosen by aic.mcmc. # resMFAcoffee2 <- get_IMIFA_results(simMFAcoffee, G=3, criterion="aic.mcmc") # Run an IMIFA model on the olive data, accepting all defaults. # simIMIFAolive <- mcmc_IMIFA(olive, method="IMIFA", n.iters=10000) # Extract optimum results # Estimate G & Q by the median of their posterior distributions # Construct 90% credible intervals and try to return the similarity matrix. # resIMIFAolive <- get_IMIFA_results(simIMIFAolive, G.meth="median", Q.meth="median", # conf.level=0.9, z.avgsim=TRUE) # summary(resIMIFAolive) # Simulate new data from the above model # newdata <- sim_IMIFA_model(resIMIFAolive) # Fit an IFA model without adaptation and examine results with and without post-hoc adaptation # Use the "SC" criterion for determining the number of active factors # simIFAnoadapt <- mcmc_IMIFA(olive, method="IFA", n.iters=10000, adapt=FALSE) # resIFAnoadapt <- get_IMIFA_results(simIFAnoadapt) # resIFApostadapt <- get_IMIFA_results(simIFAnoadapt, adapt=TRUE, active.crit="SC") # Compare to an IFA model with adaptive Gibbs sampling # simIFAadapt <- mcmc_IMIFA(coffee, method="IFA", n.iters=10000, active.crit="BD") # resIFAadapt <- get_IMIFA_results(simIFAadapt) # plot(resIFAnoadapt, "GQ") # plot(resIFApostadapt, "GQ") # plot(resIFAadapt, "GQ") ``` ## References Murphy, K., Viroli, C., and Gormley, I. C. (2020) Infinite mixtures of infinite factor analysers, **Bayesian Analysis**, 15(3): 937-963. . ## See Also `plot.Results_IMIFA`, `mcmc_IMIFA`, `Zsimilarity`, `scores_MAP`, `sim_IMIFA_model`, `Procrustes`, `varimax`, `norm`, `mgpControl`, `storeControl` ## Author(s) Keefe Murphy -

get_IMIFA_results function