cs_statistical() R function from [clinicalsignificance]

Statistical Analysis of Clinical Significance

cs_statistical() can be used to determine the clinical significance of intervention studies employing the statistical approach. For this, it will be assumed that the functional (non-clinical population) and patient (clinical population) scores form two distinct distributions on a continuum. cs_statistical() calculates a cutoff point between these two populations and counts, how many patients changed from the clinical to the functional population during intervention. Several methods for calculating this cutoff are available.


cs_statistical(
  data,
  id,
  time,
  outcome,
  group = NULL,
  pre = NULL,
  post = NULL,
  m_functional = NULL,
  sd_functional = NULL,
  reliability = NULL,
  better_is = c("lower", "higher"),
  cutoff_method = c("JT", "HA"),
  cutoff_type = c("a", "b", "c"),
  significance_level = 0.05
)

Arguments

data: A tidy data frame
id: Participant ID
time: Time variable
outcome: Outcome variable
group: Grouping variable (optional)
pre: Pre measurement (only needed if the time variable contains more than two measurements)
post: Post measurement (only needed if the time variable contains more than two measurements)
m_functional: Numeric, mean of functional population.
sd_functional: Numeric, standard deviation of functional population
reliability: The instrument's reliability estimate. If you selected the NK method, the here specified reliability will be the instrument's pre measurement reliability. Not needed for the HLM method.
better_is: Which direction means a better outcome for the used instrument? Available are
- "lower" (lower outcome scores are desirable, the default) and
- "higher" (higher outcome scores are desirable)
cutoff_method: Cutoff method, Available are
- "JT" (Jacobson & Truax, 1991, the default)
- "HA" (Hageman & Arrindell, 1999)
cutoff_type: Cutoff type. Available are "a", "b", and "c". Defaults to "a" but "c" is usually recommended. For "b" and "c", summary data from a functional population must be given with arguments m_functional and sd_functional.
significance_level: Significance level alpha, defaults to 0.05. If you choose the "HA" method, this value corresponds to the maximum risk of misclassification

Returns

An S3 object of class cs_analysis and cs_statistical

Computational details

There are three available cutoff types, namely a, b, and c which can be used to "draw a line" or separate the functional and clinical population on a continuum. a as a cutoff is defined as the mean of the clinical population minus two times the standard deviation (SD) of the clinical population. b is defined as the mean of the functional population plus also two times the SD of the clinical population. This is true for "negative" outcomes, where a lower instrument score is desirable. For "positive" outcomes, where higher scores are beneficial, a is the mean of the clinical population plus 2 $\cdot$ SD of the clinical population and b is mean of the functional population minus 2 $\cdot$ SD of the clinical population. The summary statistics for the clinical population are estimated from the provided data at pre measurement.

c is defined as the midpoint between both populations based on their respective mean and SD. In order to calculate b and c, descriptive statistics for the functional population must be provided.

Data preparation

The data set must be tidy, which corresponds to a long data frame in general. It must contain a patient identifier which must be unique per patient. Also, a column containing the different measurements and the outcome must be supplied. Each participant-measurement combination must be unique, so for instance, the data must not contain two "After" measurements for the same patient.

Additionally, if the measurement column contains only two values, the first value based on alphabetical, numerical or factor ordering will be used as the pre measurement. For instance, if the column contains the measurements identifiers "pre" and "post" as strings, then "post"

will be sorted before "pre" and thus be used as the "pre" measurement. The function will throw a warning but generally you may want to explicitly define the "pre" and "post" measurement with arguments pre and post. In case of more than two measurement identifiers, you have to define pre and post manually since the function does not know what your pre and post intervention measurements are.

If your data is grouped, you can specify the group by referencing the grouping variable (see examples below). The analysis is then run for every group to compare group differences.

Examples


# By default, cutoff type "a" is used
cs_results <- claus_2020 |>
  cs_statistical(id, time, hamd, pre = 1, post = 4)

cs_results
summary(cs_results)
plot(cs_results)

# You can choose a different cutoff type but need to provide additional
# population summary statistics for the functional population
cs_results_c <- claus_2020 |>
  cs_statistical(
    id,
    time,
    hamd,
    pre = 1,
    post = 4,
    m_functional = 8,
    sd_functional = 8,
    cutoff_type = "c"
  )

cs_results_c
summary(cs_results_c)
plot(cs_results_c)

# You can use a different method to calculate the cutoff
cs_results_ha <- claus_2020 |>
  cs_statistical(
    id,
    time,
    hamd,
    pre = 1,
    post = 4,
    m_functional = 8,
    sd_functional = 8,
    reliability = 0.80,
    cutoff_type = "c",
    cutoff_method = "HA"
  )

cs_results_ha
summary(cs_results_ha)
plot(cs_results_ha)

# And you can group the analysis by providing a grouping variable from the data
cs_results_grouped <- claus_2020 |>
  cs_statistical(
    id,
    time,
    hamd,
    pre = 1,
    post = 4,
    m_functional = 8,
    sd_functional = 8,
    cutoff_type = "c",
    group = treatment
  )

cs_results_grouped
summary(cs_results_grouped)
plot(cs_results_grouped)

References

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037//0022-006X.59.1.12
Hageman, W. J., & Arrindell, W. A. (1999). Establishing clinically significant change: increment of precision and the distinction between individual and group level analysis. Behaviour Research and Therapy, 37(12), 1169–1193. https://doi.org/10.1016/S0005-7967(99)00032-7

cs_statistical function