DMAR 1.1.0 names the total sample size, the number of predictors, a planner’s planning value, and the desired width one way, and names an argument for the quantity it holds rather than for a symbol. The DMAR 1.0.x names listed here are not accepted: a call that gives one stops with R’s “unused argument” error. The exceptions, all of them, are listed under Old Names That R Still Matches, after the table. There is no deprecation period and no function that maps an old name to a new one. Change the call. Three functions are removed, with no alias; the call that replaces each is given under One Function for Each ANOVA Effect Size.
var_partial_r(), var_semipartial_r(),
expected_partial_r(), ss_aipe_partial_r(),
ss_aipe_semipartial_r(),
ss_aipe_partial_r_sensitivity(), and
ss_aipe_semipartial_r_sensitivity() took J,
the number of variables partialed out. They take p, the
total number of predictors in the regression of Y on X and the variables
partialed out, in the place J held, and
p = J + 1. A DMAR 1.0.x call that gave the count by
position still runs, and now describes one fewer variable partialed
out. var_partial_r(.3, 100, 2) was a partial
correlation given two other variables and is now a partial correlation
given one; ss_aipe_partial_r(.3, 2, .2) planned for two
variables partialed out and now plans for one. Add 1 to the count in
every such call: var_partial_r(.3, 100, 3),
ss_aipe_partial_r(.3, 3, .2). A call that named the count
(J = 2) stops with “unused argument” and becomes
p = 3. p must be a whole number of at least 2;
with nothing partialed out the correlation is the simple correlation,
which var_r(), expected_r(),
ss_aipe_r(), and ss_aipe_r_sensitivity()
cover.
eta_squared_partial(),
omega_squared_partial(), and
ci_eta_squared_partial() are removed.
eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared() take
estimand = c("all", "total", "partial") and report the
total and the partial value of each effect in columns of their own (see
Total and Partial Eta Squared and Omega Squared). There is no alias.
Replace
| DMAR 1.0.x call | DMAR 1.1.0 call |
|---|---|
eta_squared_partial(fit) |
eta_squared(fit, estimand = "partial") |
eta_squared_partial(F_value = , df_effect = , df_error = ) |
eta_squared(F_value = , df_effect = , df_error = ) (the
partial value; estimand = "partial" leaves out the
NA total column) |
omega_squared_partial(fit) |
omega_squared(fit, estimand = "partial") |
omega_squared_partial(F_value = , df_effect = , df_error = , N = ) |
omega_squared(F_value = , df_effect = , df_error = , N = , estimand = "partial") |
ci_eta_squared_partial(fit) |
ci_eta_squared(fit, estimand = "partial") |
ci_eta_squared_partial(F_value = , df_effect = , df_error = , N = ) |
ci_eta_squared(F_value = , df_effect = , df_error = , N = , estimand = "partial") |
The columns of the four tables are named for the
estimand. A script that read $eta_squared,
$omega_squared, $ci_lower, or
$ci_upper from these tables reads the new name. The values
under the old names were the partial values, and the new argument does
not change them.
| function | DMAR 1.0.x column | DMAR 1.1.0 column |
|---|---|---|
eta_squared() |
eta_squared |
eta_squared_partial |
omega_squared() |
omega_squared |
omega_squared_partial |
ci_eta_squared() |
eta_squared, ci_lower,
ci_upper |
eta_squared_partial,
eta_squared_partial_ci_lower,
eta_squared_partial_ci_upper |
ci_omega_squared() |
omega_squared, ci_lower,
ci_upper |
omega_squared_partial,
omega_squared_partial_ci_lower,
omega_squared_partial_ci_upper |
With estimand = "all", the default, the columns of the
total (eta_squared_total, omega_squared_total,
and for the intervals eta_squared_total_ci_lower and so on)
come before those of the partial value;
estimand = "partial" gives the table of 1.0.x with the new
names. The effect, stratum,
F_value, df_effect, df_error, and
N columns are unchanged.
estimand is the second argument of
eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared(),
after object. A call that gave F_value by
position (as the second argument) stops; name it.
tidy() of these tables gives one row per
effect and estimand, the rows of the total before those of the
partial value, with term naming the estimand
("eta_squared_total", "eta_squared_partial")
where it named "eta_squared". glance() of a
table with more than one row of tidy() gives the estimands
in term, separated by a comma.
An argument is named for the quantity it holds and for the role of
the value. A population value that a planner or a sensitivity function
takes begins with true_, as do those of the analysis of
variance, analysis of covariance, and regression data generators; the
value a sensitivity function plans the design from begins with
supposed_ (estimated_ or est_ in
DMAR 1.0.x, which suggested an estimate from data, where the value is a
planning assumption); and a null value begins with null_.
The quantity is named in words: a correlation is cor
(true_cor, true_cor_YX,
cor_lower), a contrast contrast and a
standardized one std_contrast, a regression coefficient
reg_coef and a standardized one std_reg_coef,
a squared multiple correlation R2, a mean
mean, a standard deviation sd
(true_sd, and true_sd_X and
true_sd_Y when the variable is named), a population
variance true_var_ followed by what it is the variance of
(true_var_within, true_var_error,
true_var_trend), a covariance matrix
cov_matrix, a factor loading factor_loading,
and a residual variance residual_var. A function that
computes the same quantity from population values or from estimates
(cohen_f(), ci_srsnr(),
covmat_from_cfa(), var_ete(),
var_indirect_effect()) names its arguments without a role.
A sample size given so that the power or the precision at that size is
returned is specified_N (the total, or the one group) or
specified_n_per_group. The table gives the name each
argument had in DMAR 1.0.x.
| family | function | DMAR 1.0.x | now |
|---|---|---|---|
| the chi square statistic | ci_nc_chisq(), convert_chisq_F() |
chi_square |
chisq (as in sem_fit_measures(),
ecvi(), and equivalence_rmsea(); a row or
column of a table is still chi_square) |
| a per-group sample size given for evaluation | ss_power_contrast(),
ss_aipe_c_sensitivity(),
ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity(),
ss_aipe_smd_sensitivity(),
ss_aipe_equivalence_smd_sensitivity(),
ss_aipe_pcm_sensitivity() |
n_per_group |
specified_n_per_group (also in
ss_power_c(), ss_power_c_ancova(),
ss_power_sc(),
ss_power_composite_ancova_2group(), and
ss_power_split_plot_anova(), listed below) |
| a population standard deviation | ss_seq_c_sensitivity() |
true_sigma |
true_sd |
power_equivalence_md(),
power_density_equivalence_md(),
power_equivalence_md_plot() |
sigma; power_sigma in
power_density_equivalence_md() |
true_sd; sd_values |
|
| a calculator given population values or sample summaries | cohen_f() |
mu, sigma_squared, sigma_m,
sigma |
means, var_within, sd_means,
sd_within |
ci_srsnr() |
sigma_squared |
var_within |
|
covmat_from_cfa() |
lambda, psi_squared |
factor_loading, residual_var |
|
var_ete() |
sigma2, sigma2_Z, beta_1,
beta_2, mu_Z |
error_variance, var_Z (either may be a
population value or an estimate, as type says, so neither
is named true_), reg_coef_1,
reg_coef_2, mean_Z |
|
| a size per cell or per arm given for evaluation | ss_power_factorial_anova(),
ss_power_factorial_ancova(),
ss_power_composite_anova(),
ss_power_composite_ancova(),
ss_power_composite_factorial_anova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_ancova_het() |
n_per_cell |
specified_n_per_cell |
| the estimated contrast and its equivalence bounds | equivalence_c() |
psi_hat, delta_lower,
delta_upper |
contrast, contrast_lower,
contrast_upper |
| the variance of the predictor in a two-level model | ss_aipe_mixed_effects_sensitivity() |
true_sigma2_x, estimated_sigma2_x |
true_var_X, supposed_var_X |
| partial and semipartial correlation | var_partial_r(), var_semipartial_r(),
expected_partial_r() |
n, J |
N, p (p = J + 1) |
expected_partial_r() |
rho |
true_cor |
|
ss_aipe_partial_r() |
rho, J |
true_cor, p (p = J + 1) |
|
ss_aipe_semipartial_r() |
r_sp, J |
true_r_sp, p (p = J + 1) |
|
ss_aipe_partial_r_sensitivity(),
ss_aipe_semipartial_r_sensitivity() |
J |
p (p = J + 1) |
|
ss_aipe_partial_r_sensitivity() |
true_rho, estimated_rho |
true_cor, supposed_cor |
|
ss_aipe_semipartial_r_sensitivity() |
estimated_r_sp |
supposed_r_sp |
|
| correlation | ci_r(), var_r(),
expected_r(), equivalence_r() |
n |
N |
var_r(), expected_r() |
rho |
true_cor |
|
equivalence_r() |
rho_lower, rho_upper |
cor_lower, cor_upper |
|
ss_aipe_r() |
rho |
true_cor |
|
ss_aipe_r_sensitivity() |
true_rho, estimated_rho |
true_cor, supposed_cor |
|
ss_power_r() |
rho, rho_0, N |
true_cor, null_cor,
specified_N |
|
ss_aipe_equivalence_r() |
population_r |
true_cor |
|
ss_aipe_equivalence_r_sensitivity() |
true_r, estimated_r,
rho_lower, rho_upper |
true_cor, supposed_cor,
cor_lower, cor_upper |
|
| coefficient of variation | ci_cv(), var_cv() |
n |
N |
var_cv() |
cv |
true_cv, as in ss_aipe_cv() |
|
ss_aipe_cv() |
C_of_V, mu, sigma |
true_cv, true_mean,
true_sd |
|
ss_aipe_cv_sensitivity() |
mean, estimated_cv |
true_mean, supposed_cv |
|
| squared multiple correlation | ss_aipe_R2() |
population_R2, g |
true_R2, G_initial |
ss_power_R2() |
population_R2, cohen_f2,
print_progress |
true_R2, true_f2,
print_iter |
|
var_R2(), expected_R2() |
population_R2 |
true_R2, as in ss_aipe_R2() and
ss_power_R2() |
|
ss_aipe_R2_sensitivity() |
w, estimated_R2, rho_yx,
rho_xx |
width, supposed_R2,
true_cor_YX, true_cor_XX |
|
ss_power_R2_sensitivity() |
estimated_R2, rho_yx,
rho_xx |
supposed_R2, true_cor_YX,
true_cor_XX |
|
| regression coefficient | ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(),
ss_aipe_src_sensitivity() |
w, estimated_var_Y,
estimated_cov_YX, estimated_cov_XX |
width, supposed_var_Y,
supposed_cov_YX, supposed_cov_XX |
ss_power_reg_coef_sensitivity() |
estimated_var_Y, estimated_cov_YX,
estimated_cov_XX |
supposed_var_Y, supposed_cov_YX,
supposed_cov_XX |
|
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(),
ss_power_reg_coef_sensitivity() |
standardize (default FALSE) |
removed: these report the unstandardized coefficient;
ss_aipe_src_sensitivity() reports the standardized one |
|
ss_aipe_src_sensitivity() |
standardize (default TRUE) |
removed: it reports the standardized coefficient | |
ci_reg_coef(), ci_rc() |
b_j, SE_b_j, s_Y,
s_X |
reg_coef_j, se_reg_coef_j,
sd_Y, sd_X |
|
ci_src() |
beta_j, SE_beta_j, b_j,
SE_b_j, s_Y, s_X |
std_reg_coef_j, se_std_reg_coef_j,
reg_coef_j, se_reg_coef_j, sd_Y,
sd_X |
|
ss_aipe_reg_coef() |
b_j, rho2_Y_X,
rho2_j_X_without_j, sigma_Y,
sigma_X, rho_XX, rho_YX |
true_reg_coef_j, true_R2_Y_X,
true_R2_j_X_without_j, true_sd_Y,
true_sd_X, true_cor_XX,
true_cor_YX |
|
ss_aipe_rc() |
b_j, rho2_Y_X,
Rho2_j_X_without_j, sigma_Y,
sigma_X_j, rho_XX, rho_YX |
true_reg_coef_j, true_R2_Y_X,
true_R2_j_X_without_j, true_sd_Y,
true_sd_X, true_cor_XX,
true_cor_YX |
|
ss_aipe_src() |
beta_j, rho2_Y_X,
Rho2_j_X_without_j, sigma_Y,
sigma_X_j, rho_XX, rho_YX |
true_std_reg_coef_j, true_R2_Y_X,
true_R2_j_X_without_j, true_sd_Y,
true_sd_X, true_cor_XX,
true_cor_YX |
|
ss_power_reg_coef() |
beta_j, rho2_Y_X,
rho2_Y_X_without_j, rho2_j_X_without_j,
sigma_X, sigma_Y, rho_XX,
rho_YX, cohen_f2,
print_progress |
true_reg_coef_j (the coefficient on the scale
true_sd_X and true_sd_Y describe),
true_R2_Y_X, true_R2_Y_X_without_j,
true_R2_j_X_without_j, true_sd_X,
true_sd_Y, true_cor_XX,
true_cor_YX, true_f2,
print_iter |
|
ss_power_rc() |
beta_j, rho2_Y_X,
rho2_Y_X_without_j, Rho2_j_X_without_j,
sigma_X, sigma_Y, rho_XX,
rho_YX, cohen_f2,
print_progress |
true_reg_coef_j, true_R2_Y_X,
true_R2_Y_X_without_j, true_R2_j_X_without_j,
true_sd_X, true_sd_Y,
true_cor_XX, true_cor_YX,
true_f2, print_iter |
|
| contrast among means | ss_aipe_c() |
error_variance, MSwithin,
SD |
true_var_error (the square of SD) |
ss_aipe_c_ancova() |
error_var_ancova, error_var_anova,
rho |
true_var_error_ancova,
true_var_error_anova, true_cor |
|
ss_aipe_c_ancova_sensitivity() |
true_error_var_ancova,
true_error_var_anova, est_error_var_ancova,
est_error_var_anova, rho,
est_rho, mu_y, sigma_y,
mu_x, sigma_x |
true_var_error_ancova,
true_var_error_anova,
supposed_var_error_ancova,
supposed_var_error_anova, true_cor,
supposed_cor, true_mean_Y,
true_sd_Y, true_mean_X,
true_sd_X |
|
ss_aipe_c_sensitivity() |
true_error_variance,
estimated_error_variance, true_psi |
true_var_error, supposed_var_error,
true_contrast |
|
ci_c(), ci_c_ancova(),
ci_sc(), ci_sc_ancova() |
psi |
contrast (the estimate row is contrast in
ci_c() and ci_c_ancova() and
std_contrast in ci_sc() and
ci_sc_ancova()) |
|
ss_power_c() |
psi, sigma, n |
true_contrast, true_sd,
specified_n_per_group |
|
ss_power_contrast() |
psi, mu, sigma_squared,
print_progress |
true_contrast, true_means,
true_var_error, print_iter |
|
ci_c(), ci_c_ancova(),
equivalence_c(), ss_seq_c() |
n |
n_per_group |
|
ss_power_c_ancova() |
psi, sigma, rho,
n |
true_contrast, true_sd,
true_cor, specified_n_per_group |
|
| standardized contrast | ss_aipe_sc() |
psi_standardized |
true_std_contrast |
ss_power_sc() |
psi_standardized, n |
true_std_contrast,
specified_n_per_group |
|
ci_sc(), ci_sc_ancova() |
n |
n_per_group |
|
ss_aipe_sc_ancova() |
psi_standardized, psi, rho,
sigma_anova, sigma_ancova,
ratio |
true_std_contrast, true_contrast,
true_cor, true_sd_anova,
true_sd_ancova, true_sd_ratio |
|
ss_aipe_sc_sensitivity() |
true_psi, estimated_psi,
desired_width |
true_std_contrast, supposed_std_contrast,
width |
|
ss_aipe_sc_ancova_sensitivity() |
true_psi, estimated_psi,
desired_width, rho, mu_x,
sigma_x |
true_std_contrast, supposed_std_contrast,
width, true_cor, true_mean_X,
true_sd_X |
|
| standardized mean difference | ss_aipe_smd() |
delta |
true_smd |
ss_aipe_smd_sensitivity() |
true_delta, estimated_delta,
desired_width |
true_smd, supposed_smd,
width |
|
ss_power_smd() |
smd |
true_smd |
|
ss_aipe_equivalence_smd() |
population_smd |
true_smd |
|
ss_aipe_equivalence_smd_sensitivity() |
estimated_smd, delta_lower,
delta_upper |
supposed_smd, smd_lower,
smd_upper |
|
equivalence_smd() |
delta_lower, delta_upper |
smd_lower, smd_upper |
|
expected_smd(), var_smd() |
delta |
true_smd |
|
var_smd_trimmed() |
population_smd_trimmed |
true_smd_trimmed, as var_smd() takes
true_smd |
|
convert_delta_lambda() |
delta |
smd (a population value or an estimate; the function
name keeps delta) |
|
cles(), nnt_from_smd(),
proportion_of_superiority() |
smd_lower, smd_upper (confidence limits on
d) |
smd_ci_lower, smd_ci_upper |
|
ss_power_composite_ancova_2group() |
smd, rho, sigma,
sd_cov, n |
true_smd, true_cor, true_sd,
true_sd_X, specified_n_per_group |
|
| noncentral t and F | convert_lambda_delta(),
convert_lambda_R2() |
lambda |
ncp (the function names keep lambda) |
ci_nc_t() |
t_value (a second name for ncp) |
ncp |
|
| analysis of variance | ss_aipe_omega_squared() |
population_omega_squared |
true_omega_squared |
ss_aipe_omega_squared_sensitivity() |
estimated_omega_squared |
supposed_omega_squared |
|
var_omega_squared() |
population_omega_squared |
true_omega_squared, as in
ss_aipe_omega_squared() |
|
ss_power_one_way_anova() |
f, eta_squared, N |
true_f, true_eta_squared,
specified_N |
|
ss_power_rm_anova() |
f, eta_squared, rho,
epsilon, n |
true_f, true_eta_squared,
true_cor, true_epsilon,
specified_N |
|
ss_power_split_plot_anova() |
f, partial_eta_squared, rho,
epsilon, n |
true_f, true_eta_squared_partial,
true_cor, true_epsilon,
specified_n_per_group |
|
ss_power_factorial_anova() |
f, partial_eta_squared |
true_f, true_eta_squared_partial |
|
ss_power_factorial_ancova() |
f, partial_eta_squared,
covariate_R2 |
true_f, true_eta_squared_partial,
true_covariate_R2 |
|
| composite power in the analysis of variance and covariance | ss_power_composite_anova(),
ss_power_composite_factorial_anova() |
means, sigma |
true_means, true_sd |
ss_power_composite_factorial_ancova() |
means, sigma,
covariate_R2 |
true_means, true_sd,
true_covariate_R2 |
|
ss_power_composite_factorial_ancova_het() |
means, correlations, sigma,
sd_cov |
true_means, true_cor,
true_sd, true_sd_X |
|
ss_power_composite_ancova() |
means, sigma, covariate_R2,
correlations, sd_cov |
true_means, true_sd,
true_covariate_R2, true_cor,
true_sd_X |
|
cohen_f(), is_orthogonal_set(),
variance_components_mls() |
n |
n_per_group |
|
| equivalence of a contrast and of a mean difference | power_equivalence_c() |
n, sigma, delta_lower,
delta_upper, true_psi |
n_per_group, true_sd,
contrast_lower, contrast_upper,
true_contrast |
ss_power_equivalence_c() |
sigma, delta_lower,
delta_upper, true_psi |
true_sd, contrast_lower,
contrast_upper, true_contrast |
|
power_equivalence_md(),
power_density_equivalence_md(),
power_equivalence_md_plot() |
n |
n_per_treatment |
|
power_equivalence_md() |
ltheta1, ltheta2, ldiff,
nu |
lower_bound, upper_bound,
true_effect, df_error (ratios on the original
scale when logscale = TRUE, differences otherwise, as
before) |
|
power_density_equivalence_md() |
theta1, theta2, diff,
nu |
lower_bound, upper_bound,
true_effect, df_error (on the analysis scale,
as before) |
|
power_equivalence_md_plot() |
theta1, theta2, nu |
lower_bound, upper_bound,
df_error |
|
plot_equivalence() |
delta_lower, delta_upper |
effect_lower, effect_upper (bounds on the
scale of whatever estimates are drawn) |
|
| sequential estimation with minimum risk | mr_smd() |
n |
n_per_group |
| intraclass correlation and coefficient alpha | ss_aipe_icc() |
rho, k |
true_icc, n_raters |
ss_aipe_icc_sensitivity() |
true_rho, estimated_rho,
k |
true_icc, supposed_icc,
n_raters |
|
var_icc() |
rho, n, k |
icc, N, n_raters |
|
var_alpha() |
n, p_items |
N, n_items |
|
| two-level models | ss_aipe_mixed_effects() |
sigma2_y, sigma2_x, icc |
true_total_var, true_var_X,
true_icc |
ss_aipe_mixed_effects_sensitivity() |
true_sigma2_y, estimated_sigma2_y,
estimated_icc, true_beta,
specified_K |
true_total_var, supposed_total_var,
supposed_icc, true_reg_coef,
specified_n_clusters |
|
ss_power_mixed_effects() |
d, n, rho,
J |
true_smd, cluster_size,
true_icc, specified_n_clusters_per_arm |
|
| polynomial change models | ss_aipe_pcm() |
variance_trend, error_variance,
variance_true_minus_estimated_trend |
true_var_trend, true_var_error,
true_var_trend_error |
ss_power_pcm() |
beta, tau, level_1_variance,
N |
true_std_reg_coef (beta under the default
standardized = TRUE) or true_reg_coef
(beta with standardized = FALSE),
true_var_trend, true_var_error,
specified_n_per_group (the size of each group, half of the
total that N held) |
|
ss_power_pcm() |
standardized |
removed: the argument given, true_std_reg_coef or
true_reg_coef, says the scale |
|
ss_aipe_pcm_sensitivity() |
true_variance_trend, true_error_variance,
estimated_variance_trend,
estimated_error_variance |
true_var_trend, true_var_error,
supposed_var_trend, supposed_var_error |
|
| cluster randomized designs | ss_aipe_crd_both_fixed_budget(),
ss_aipe_crd_both_fixed_width(),
ss_aipe_crd_n_clusters_fixed_budget(),
ss_aipe_crd_n_clusters_fixed_width(),
ss_aipe_crd_n_individuals_fixed_budget(),
ss_aipe_crd_n_individuals_fixed_width() |
tau_Y, sigma2_Y, total_var,
icc_Y, R2_between, R2_within,
num_predictors, clus_cost,
indiv_cost |
true_var_between, true_var_within,
true_total_var, true_icc,
true_R2_between, true_R2_within,
n_covariates, cluster_cost,
individual_cost |
ss_aipe_crd_es_both_fixed_budget(),
ss_aipe_crd_es_both_fixed_width(),
ss_aipe_crd_es_n_clusters_fixed_budget(),
ss_aipe_crd_es_n_clusters_fixed_width(),
ss_aipe_crd_es_n_individuals_fixed_budget(),
ss_aipe_crd_es_n_individuals_fixed_width() |
es, icc_Y, icc_Z,
R2_between, R2_within,
num_predictors, nrep, clus_cost,
indiv_cost, multicore,
num_proc |
true_es, true_icc,
true_icc_covariate, true_R2_between,
true_R2_within, n_covariates, G,
cluster_cost, individual_cost,
ncpus (multicore = TRUE, num_proc = 4 is
ncpus = 4) |
|
ss_aipe_crd_n_clusters_fixed_budget(),
ss_aipe_crd_n_clusters_fixed_width(),
ss_aipe_crd_es_n_clusters_fixed_budget(),
ss_aipe_crd_es_n_clusters_fixed_width() |
n_individuals |
cluster_size |
|
| indirect (mediated) effect | ss_aipe_indirect_effect() |
a, b, n_max |
true_std_reg_coef_a, true_std_reg_coef_b,
max_N |
ss_aipe_indirect_effect_sensitivity() |
true_a, true_b, estimated_a,
estimated_b |
true_std_reg_coef_a, true_std_reg_coef_b,
supposed_std_reg_coef_a,
supposed_std_reg_coef_b |
|
ss_power_indirect_effect() |
a, b, c_prime,
N |
true_std_reg_coef_a, true_std_reg_coef_b,
true_std_reg_coef_direct, specified_N |
|
var_indirect_effect() |
a, b, var_a,
var_b, cov_ab |
reg_coef_a, reg_coef_b,
var_reg_coef_a, var_reg_coef_b,
cov_reg_coef_a_b (population values or estimates, raw or
standardized, so neither true_ nor std_) |
|
| structural equation models | ss_aipe_composite_sem() |
Sigma, pop_model, mu,
desired_width, N |
true_cov_matrix, true_model,
true_mean, width,
specified_N |
ss_power_composite_sem() |
Sigma, pop_model, mu,
N |
true_cov_matrix, true_model,
true_mean, specified_N |
|
ss_aipe_sem_path() |
Sigma, desired_width |
true_cov_matrix, width |
|
ss_aipe_sem_path_sensitivity() |
est_Sigma, true_Sigma,
desired_width, N |
supposed_cov_matrix, true_cov_matrix,
width, specified_N |
|
ss_aipe_rmsea() |
RMSEA |
true_rmsea |
|
ss_aipe_rmsea_sensitivity() |
Sigma, N |
true_cov_matrix, specified_N |
|
ss_power_sem() |
RMSEA_null, RMSEA_true, F_ML,
RMSEA_full, RMSEA_res, F_full,
F_res, df_res |
null_rmsea, true_rmsea,
true_F_ML, true_rmsea_full,
true_rmsea_restricted, true_F_full,
true_F_restricted, df_restricted |
|
| reliability | ss_aipe_reliability() |
S, i, cor_est,
lambda, psi_square, initial_iter,
final_iter, start_ss |
true_cov_matrix, n_items,
true_item_covariance, true_factor_loading,
true_residual_var, G_initial, G,
start_N |
ss_aipe_reliability_sensitivity() |
estimated_reliability, i |
supposed_reliability, n_items |
|
| measurement models and item response theory | dmacs() |
lambda_reference, lambda_focal,
nu_reference, nu_focal |
factor_loading_reference,
factor_loading_focal, intercept_reference,
intercept_focal |
simple_structure() |
Lambda |
factor_loading_matrix |
|
irt_information() |
a, b, theta |
discrimination, threshold,
trait_value (item and grm keep
their names, and so do the columns a and b of
an irt_grm() table given as grm) |
|
| standardized mean | ss_aipe_sm() |
sm |
true_sm |
ss_aipe_sm_sensitivity() |
estimated_sm, desired_width |
supposed_sm, width |
|
| Cliff’s delta | ss_aipe_cliff_delta() |
delta |
true_delta |
ss_aipe_cliff_delta_sensitivity() |
estimated_delta |
supposed_delta |
|
| Cohen’s h | cohen_h() |
p1, p2 |
p_1, p_2 |
| consequences of a design | design_consequences() |
w |
width |
| one sample and paired data | bayes_one_sample_t(),
bayes_paired_t() |
n |
N |
ci_proportion() |
n |
N |
|
| the null value of a mean or a mean difference | bayes_one_sample_t(),
ci_mahalanobis() |
mu_0 |
null_mean |
welch_t(), summary_t_test() |
mu |
null_mean_diff |
|
| eigenvalue of a covariance matrix | ci_eigenvalue() |
n, k |
N, which_eigenvalue |
| expected cross-validation index | ecvi() |
n |
multiplier |
| critical values and simultaneous intervals | dbryant_paulson(), pbryant_paulson(),
qbryant_paulson() |
num_covariates, num_groups |
n_covariates, n_groups |
cv_bryant_paulson() |
covariates, groups |
n_covariates, n_groups |
|
cv_tukey_hsd() |
groups |
n_groups |
|
ci_c_ancova_bp() |
n, num_covariates |
n_per_group, n_covariates |
|
| congruence of factor loadings | procrustes_phi() |
n_perm |
n_resamples |
| data generators | simulate_anova_data() |
mu, sigma, n |
true_mean, true_sd,
n_per_group |
simulate_ancova_data() |
mu_y, mu_x, sigma_y,
sigma_x, rho, n |
true_mean_Y, true_mean_X,
true_sd_Y, true_sd_X, true_cor,
n_per_group |
|
simulate_ancova_factorial_data() |
mu_y, mu_x, sigma_y,
sigma_x, rho_y_x, rho_x_x,
n |
true_mean_Y, true_mean_X,
true_sd_Y, true_sd_X,
true_cor_YX, true_cor_XX,
n_per_cell |
|
simulate_regression_data() |
rho_YX, rho_XX, mu_Y,
mu_X, sigma_Y, sigma_X |
true_cor_YX, true_cor_XX,
true_mean_Y, true_mean_X,
true_sd_Y, true_sd_X |
|
simulate_longitudinal_polynomial(),
simulate_longitudinal_logistic(),
simulate_longitudinal_gompertz(),
simulate_longitudinal_negative_exponential(),
simulate_longitudinal_richards() |
n, occasions |
n_per_group, n_occasions |
R matches an argument name by its leading characters, and a function
that passes ... to another program does not refuse a name
itself. The table lists 602 old names, a name counted once for each
function that had it. Of these, 536 stop with “unused argument”. These
66 do not:
w = in
ss_aipe_R2_sensitivity() and
design_consequences() (matched to width);
df_res = in ss_power_sem()
(df_restricted); n = in ci_c(),
ci_c_ancova(), ci_sc_ancova(),
equivalence_c(), cohen_f(),
is_orthogonal_set(),
variance_components_mls(),
simulate_anova_data(), simulate_ancova_data(),
ss_seq_c(), mr_smd(), and
power_equivalence_c() (matched to n_per_group,
the per-group size it always was; in the five power planners that took
n and take specified_n_per_group,
ss_power_c(), ss_power_sc(),
ss_power_c_ancova(),
ss_power_split_plot_anova(), and
ss_power_composite_ancova_2group(), n = now
stops with “unused argument”); and n = in
power_equivalence_md(),
power_density_equivalence_md(), and
power_equivalence_md_plot() (matched to
n_per_treatment, the same quantity, now that the degrees of
freedom are df_error and no other argument begins with
n: a 1.0.x call that gave the degrees of freedom by
position runs, and one that named nu stops for
nu). Write the full name: the match is R’s, it is not a
name these functions have, and it ends the day one of them gains another
argument that begins with the same letters.es = in the six
ss_aipe_crd_es_*() functions is matched to
es_type when es_type is left out, and the call
stops where the effect size is needed, with a message about
es_type or about the missing effect size; a fixed-budget
call that does not ask for the expected width never used the effect size
and still runs. a = in
ss_power_indirect_effect() is matched to
alpha_level, so a 1.0.x call stops for the unused
b. n = in ecvi() is matched to
npar when npar is left out, and the call stops
for the missing multiplier; a call that also names
npar stops with “unused argument”. f = in
ss_power_factorial_anova() and
ss_power_factorial_ancova() is matched to
factor_levels when the levels are given by position, and
the call stops because a value of Cohen’s f is not a vector of
levels; a call that names factor_levels stops with “unused
argument”.w = in ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() stops with “argument matches
multiple formal arguments” (which_predictor,
width), as does n = in
ci_c_ancova_bp() (n_per_group,
n_covariates), in the five
simulate_longitudinal_*() functions
(n_per_group, n_occasions), in
ci_sc() (n_per_group, ncp), and
in simulate_ancova_factorial_data()
(n_per_cell, n_covariates). d =
in ss_power_mixed_effects() stops with the same message
when desired_power and directional are both
left out; when one of them is named, d is matched to the
other and the call stops for the missing true_smd....; the function reads
the name and stops. n = in var_icc()
would be matched to n_raters, and in
var_alpha() to n_items, so a call that named
only the sample size and gave the count by position,
var_alpha(.80, n = 100, 10) or
var_icc(.60, n = 30, 4), would run with the two values
exchanged: 100 items and 10 respondents, or 30 raters and 4 subjects.
n = in bayes_one_sample_t() would be matched
to null_mean, so a call that named the sample size
n would take it as the comparison value for the mean. The
three functions read the names in the call and refuse n
with a condition of class dmar_argument_invalid that says
to write N =. A 1.0.x call of var_icc() or
var_alpha() that named every argument also named
k or p_items (and rho in
var_icc()) and stops with “unused argument”;
n = in bayes_paired_t(), which has no argument
that begins with n, stops with “unused argument” too. Four
names would reach a ...: t_value = in
ci_nc_t(), which was a second name for ncp,
would pass to the root finder; mu_0 = in
ci_mahalanobis() would reach the root search of
ci_nc_F(); psi = in ci_sc() would
reach ci_nc_t() and stop inside its root finder; and
psi = in ci_c_ancova() would be dropped
without a word under procedure = "t" and passed to
ci_c_ancova_bp() under "bryant_paulson". Each
function refuses the name with the same class and says to write
ncp =, null_mean =, or
contrast =.... to lavaan,
beside the new arguments; the function reads the name and
stops. Sigma =, pop_model =,
mu =, desired_width =, and N = in
ss_aipe_composite_sem(); Sigma =,
pop_model =, mu =, and N = in
ss_power_composite_sem(); Sigma = and
N = in ss_aipe_rmsea_sensitivity();
Sigma = and desired_width = in
ss_aipe_sem_path(); and est_Sigma =,
true_Sigma =, desired_width =, and
N = in ss_aipe_sem_path_sensitivity()
(true_Sigma was that function’s 1.0.x name for the
generating matrix, now true_cov_matrix). Each of the five
functions checks the names in ... against the arguments and
options of lavaan before anything is fitted, and a name that is neither
is refused with a condition of class dmar_argument_invalid
that gives the name as it was typed and points to the function’s Renamed
arguments section.... for its root
search; the call stops, but not with “unused argument”.
chi_square = in ci_nc_chisq() goes to
..., and the call stops at once because chisq
is missing (class dmar_argument_invalid).
sigma_squared = in ci_srsnr() with
means stops the same way because var_within is
missing; with F_value, it reaches the noncentral F
search, which stops with “unused argument (sigma_squared = …)” inside a
condition of class dmar_computation_failed.ci_cv(), ss_aipe_c(),
ss_aipe_cv(), ss_aipe_R2(),
ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_sc_ancova_sensitivity(), ss_aipe_sm(),
and ss_power_R2() took ... and used it for
nothing, so a name they did not have was dropped without a word and the
call ran on the default of the argument that was meant:
ss_power_R2(true_R2 = .3, p = 3, N = 50) planned for the
default power and returned 36. They no longer take ..., and
any name they do not have, old or misspelled, is an “unused argument”.
The two counts of the simulation that
ss_aipe_R2(verify_ss = TRUE) runs, which were read from
... as g and G, are the arguments
G_initial (the replications for each candidate N
of the coarse search) and G (the replications of the final
pass); g and G differed only in case.
ci_c_ancova_bp() and the three Bryant-Paulson distribution
functions took a ... they did not use and no longer take
it, for the same reason: ci_c_ancova_bp(num_covariates = 2)
would otherwise have been computed for the default of one covariate
without a word. So did ci_c(), ci_reg_coef(),
plot_randomization_test(), and
R2_mixed_effects(), and ci_rc() and
ci_src(), which passed theirs to
ci_reg_coef(), where nothing read it; none of the six takes
... now. No exported function is left with a
... that its body does not use.
covmat_from_cfa() read only tol_det from its
..., so a misspelled name, or a third value given by
position, was dropped without a word; tol_det is now an
argument of its own, after factor_loading and
residual_var, and the function takes no
....
In ss_power_r() the name N is gone and no
other argument took it: ss_power_r(true_cor = .3, N = 50)
stops, and the call is
ss_power_r(true_cor = .3, specified_N = 50). The null value
was rho_0 and is null_cor, as the other null
values are null_R2 and null_rmsea; the 1.0.x
planning value rho then matches nothing and stops.
ss_aipe_c() takes the error variance one way, as
true_var_error, the name in
ss_aipe_c_sensitivity() and
ss_power_contrast(). MSwithin was a second
name for it and SD was its square root; both are gone, with
the three checks that the values agreed, so a 1.0.x call that gave
SD = s gives true_var_error = s^2. A
population contrast is true_contrast in every planner and
an observed one contrast; the standardized contrast is
true_std_contrast, where the two standardized contrast
sensitivity functions called it true_psi, the 1.0.x name of
the unstandardized contrast elsewhere.
In the planners for the analysis of variance, Cohen’s f is a
planning value and is true_f, beside
true_eta_squared and true_eta_squared_partial,
in ss_power_one_way_anova(),
ss_power_rm_anova(),
ss_power_split_plot_anova(),
ss_power_factorial_anova(), and
ss_power_factorial_ancova(); the row of
ss_power_contrast() that holds the f of the
contrast is true_f too (effect_size_f).
Cohen’s f squared is true_f2 in
ss_power_R2(), ss_power_R2_change(),
ss_power_reg_coef(), and ss_power_rc(), the
argument and the row. The correlation among the repeated measures and
the sphericity value of ss_power_rm_anova() and
ss_power_split_plot_anova() are true_cor and
true_epsilon, and the proportion of within-cell variance
the covariates explain in ss_power_factorial_ancova() is
true_covariate_R2. The five planners report the
noncentrality parameter in the row noncentral_f_parm, the
name in ss_power_R2(), and the degrees of freedom in
df_effect and df_error, the names in
ss_aipe_omega_squared() and ci_eta_squared().
The number of subjects of ss_power_rm_anova(), each
measured on every occasion, is the total sample size, so it is
specified_N when given and the row necessary_N
when planned.
cohen_f(), ci_srsnr(), and
covmat_from_cfa() are calculators, not planners: each
computes the same quantity from population values or from sample
summaries and estimates, so their arguments carry plain quantity names
without true_. cohen_f() takes
means, var_within, n_per_group,
eta_squared, sd_means, and
sd_within (mu, sigma_squared,
n, sigma_m, sigma); given
population values it returns the population f, and given sample
summaries it returns the plug-in value from the same calculation, which
is not an unbiased estimate of f. ci_srsnr() takes
the observed group means, var_within (the pooled
within-group mean square; sigma_squared), and
n_per_group as an alternative to F_value; with
population values in their place, the interval is a hypothetical one,
not an interval from observed data and not the expected interval of a
planned study. covmat_from_cfa() takes the factor loadings
and residual variances, population values or estimates from a fitted
model, as factor_loading and residual_var
(lambda, psi_squared), and returns its matrix
as the element model_implied_cov
(population_cov): the matrix is unchanged, and the new name
says what it is whether the loadings and residual variances are
population values or estimates.
covmat_from_cfa(lambda = l, psi_squared = p)$population_cov
is
covmat_from_cfa(factor_loading = l, residual_var = p)$model_implied_cov.
The planners for composite power in the analysis of variance and
covariance (ss_power_composite_anova(),
ss_power_composite_factorial_anova(),
ss_power_composite_ancova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_ancova_het(), and
ss_power_composite_ancova_2group()) name their population
values as the single-effect planners do: true_means,
true_sd, true_cor (the covariate correlations,
correlations), true_covariate_R2, and
true_sd_X, the standard deviation of the covariate
(sd_cov), which is the name the two ANCOVA contrast
sensitivity functions give the same quantity. The fields of each
element of effects are renamed too: Cohen’s f is
true_f and partial eta squared is
true_eta_squared_partial (f,
partial_eta_squared), so
list(factors = 1, f = 0.25) becomes
list(factors = 1, true_f = 0.25). A field is not an
argument and R does not check it, so the planners now do: an element of
effects with a field they do not read is refused (see Calls
That Are Now Refused). The rows follow:
true_f_<label>,
noncentral_f_parm_<label>, and df_error,
the names the single-effect planners use. The total sample size beside
the per-cell or per-group answer is the row total_N in both
branches, the rule for an implied total everywhere else in the package
(it was necessary_N beside a planned size and
specified_N beside a size that was given); when the
correlations differ in absolute value and the size was planned, it is
approximate_total_N beside
approximate_n_per_cell or
approximate_n_per_group (approximate_N). The
attributes the plot() method reads from the tables of
ss_power_composite_factorial_ancova_het() and of
ss_power_composite_ancova(slopes = "heterogeneous")
(means, correlations, sigma,
sd_cov) are renamed with the arguments:
true_means, true_cor, true_sd,
and true_sd_X. ss_power_c_ancova() takes
true_sd and specified_n_per_group
(sigma, n), as ss_power_c()
does.
A size per group is n_per_group and a size per cell is
n_per_cell, in the estimators as in the planners:
ci_c(), ci_sc(), ci_c_ancova(),
ci_sc_ancova(), equivalence_c(),
cohen_f(), is_orthogonal_set(),
variance_components_mls(), ss_seq_c(),
mr_smd(), simulate_anova_data(), and
simulate_ancova_data() took n and take
n_per_group (one number for equal groups, or one per group
where the function allowed it); ss_power_c(),
ss_power_sc(), and
ss_power_split_plot_anova(), where the size is given so
that the power at that size is returned, took n and take
specified_n_per_group;
simulate_ancova_factorial_data() takes
n_per_cell. ci_c() and ci_sc()
had n and N, a per-group size and the total,
apart only by the case of one letter. In
power_equivalence_md(),
power_density_equivalence_md(), and
power_equivalence_md_plot() the size is
n_per_treatment: the number of subjects who receive each
treatment, which is the size of each group in a parallel design and
every subject in a crossover, the n of the standard error sigma * sqrt(2
/ n) in both. The three functions take the equivalence bounds as
lower_bound and upper_bound, the true
difference or ratio as true_effect, and the error degrees
of freedom as df_error (ltheta1,
ltheta2, and ldiff in
power_equivalence_md(), theta1,
theta2, and diff in the other two, and
nu in all three). The grid attached to the plot keeps its
column labels (n=24); its first column, diff,
is true_effect. power_equivalence_c() takes
n_per_group for independent groups; the number of units of
a paired design, new in this release, is N (see Equivalence
and Noninferiority of a Contrast).
The contrast planners name their population values as the sensitivity
functions do. The error standard deviation is true_sd in
ss_power_c(), ss_power_equivalence_c(), and
power_equivalence_c() (sigma), as in
ss_seq_c_sensitivity(); ss_power_contrast()
takes true_means and true_var_error
(mu, sigma_squared);
ss_aipe_sc_ancova() takes true_sd_anova,
true_sd_ancova, and true_sd_ratio
(ratio alone is an allocation ratio elsewhere); and the
values the two ANCOVA contrast sensitivity functions generate their data
from are true_mean_Y, true_sd_Y,
true_mean_X, and true_sd_X. The data
generators for the analysis of variance and covariance and for
regression name their population values the same way:
simulate_anova_data() takes true_mean and
true_sd (mu, sigma);
simulate_ancova_data() takes true_mean_Y,
true_mean_X, true_sd_Y,
true_sd_X, and true_cor (mu_y,
mu_x, sigma_y, sigma_x,
rho), and simulate_ancova_factorial_data() the
same with true_cor_YX and true_cor_XX
(rho_y_x, rho_x_x); and
simulate_regression_data() takes true_cor_YX,
true_cor_XX, true_mean_Y,
true_mean_X, true_sd_Y, and
true_sd_X (rho_YX, rho_XX,
mu_Y, mu_X, sigma_Y,
sigma_X). mr_smd() and mr_cv()
keep the symbols of the published procedure (A,
epsilon, gamma, m0); their rows
now name the sizes they hold: n_1 and n_2 and
pilot_n_per_group in mr_smd(), N
and pilot_N in mr_cv().
var_omega_squared() takes the value its formula is
evaluated at as true_omega_squared
(population_omega_squared), the name in
ss_aipe_omega_squared(). In the six
ss_aipe_crd_es_*() functions write true_es =,
and in ss_power_mixed_effects() true_smd =;
what R does with es = and d = is under Old
Names That R Still Matches.
ss_power_pcm() and ss_aipe_pcm() give the
variance of the individuals’ change coefficients and the level one error
variance the same names, true_var_trend and
true_var_error (tau and
level_1_variance in ss_power_pcm(),
variance_trend and error_variance in
ss_aipe_pcm()), and ss_aipe_pcm_sensitivity()
plans from supposed_var_trend and
supposed_var_error. ss_aipe_pcm() takes the
sampling-error variance of one individual’s estimated change coefficient
(Kelley & Rausch, 2011, Equation 19) as
true_var_trend_error
(variance_true_minus_estimated_trend), and
ss_power_pcm() reports the same quantity in the row of that
name (error_var_of_slopes). ss_power_pcm()
took the group difference in the change coefficient as beta
and a flag, standardized, that said whether
beta was divided by the between-subject standard deviation
of the change coefficient. It takes the difference under the name of its
scale: true_std_reg_coef for the standardized difference
(beta under the 1.0.x default) and
true_reg_coef for the difference in the units of the change
coefficient (beta with standardized = FALSE).
Exactly one is given; both or neither is refused with a condition of
class dmar_argument_invalid.
ss_power_pcm(beta = -.4, tau = .003, level_1_variance = .0262, frequency = 1, duration = 4, N = 238)
is
ss_power_pcm(true_std_reg_coef = -.4, true_var_trend = .003, true_var_error = .0262, frequency = 1, duration = 4, specified_n_per_group = 119).
The two come first among the arguments, so a 1.0.x call that gave
beta, tau, and level_1_variance
by position gives both and is refused rather than read on the other
scale. The table reports the difference on both scales, in the rows
true_reg_coef and true_std_reg_coef
(unstd_coefficient, std_coefficient),
whichever was given, and the power is unchanged. The help of
ss_power_pcm() keeps the symbols of Raudenbush and Liu
(2001) in its formulas and says which argument each one is.
In the planners for an indirect effect each path is a standardized
regression coefficient of the population the plan is made for, so the
paths are true_std_reg_coef_a,
true_std_reg_coef_b, and
true_std_reg_coef_direct, and the product of the two paths
is the row true_std_indirect_effect. The power of the
indirect effect in ss_power_indirect_effect() is the row
power_ab (actual_power), followed by
power_a and power_b, and tidy()
and glance() read it as the power of the design.
var_indirect_effect() computes the same variances from
population values or from estimates, raw or standardized, so its
arguments carry neither true_ nor std_;
mediate() keeps its rows a and b,
which are estimates.
In the planners for structural equation models a population
covariance matrix is true_cov_matrix and the matrix a
sensitivity study is planned from is supposed_cov_matrix;
the rows of ss_power_composite_sem() that echo the
population value of each parameter are true_<label>,
as in ss_aipe_composite_sem(). In
ss_power_sem() df and df_full
keep their names.
In the functions for one sample and for paired data, the size of the
one sample, the number of pairs, and the number of trials are each the
total sample size, so bayes_one_sample_t(),
bayes_paired_t(), ci_proportion(), and
ci_eigenvalue() take N; a call that gave the
size by position (ci_proportion(17, 50)) is unchanged. The
eigenvalue that ci_eigenvalue() brackets was chosen with
k and is chosen with which_eigenvalue, as
which_predictor and which_path choose theirs.
The per-group sizes of bayes_independent_t(),
ci_ratio_of_means(), and design_consequences()
keep n_1 and n_2. In
ss_aipe_cliff_delta() ratio keeps its name
(the allocation ratio is a constant of the design, not a planning
value).
A null value is null_ followed by the quantity.
bayes_one_sample_t() and ci_mahalanobis() take
the comparison value of the mean as null_mean
(mu_0), and welch_t() and
summary_t_test() take the null value of the difference
between the means as null_mean_diff (mu),
beside null_cor in ss_power_r(),
null_R2 in ss_power_R2(), and
null_rmsea in ss_power_sem().
The proportion of intervals no wider than the desired width is the
row pct_ci_less_width in every sensitivity function and in
design_consequences() (pct_ci_less_w; every
other name in the package spells the width out), and the rows of
ss_aipe_composite_sem() that hold the same proportion for
each parameter are pct_ci_less_width_<label>
(width_within_desired_<label>).
A count is n_<things>. The number of groups is
n_groups in cv_tukey_hsd(),
cv_bryant_paulson(), and the three Bryant-Paulson
distribution functions (groups, num_groups),
and the number of random covariates is n_covariates there
and in ci_c_ancova_bp() (covariates,
num_covariates); covariates in
ancova() and mediate() names columns and keeps
its name. The per-group size of ci_c_ancova_bp() is
n_per_group. The number of random permutations of
procrustes_phi() is n_resamples, the name in
randomization_test(),
randomization_test_paired(), and
vargha_delaney_A(). The five
simulate_longitudinal_*() functions take
n_per_group, the number of units in each population (one
number, or one per population; never a total), and
n_occasions, the number of measurement occasions per unit.
In ecvi() the divisor is multiplier: it is the
multiplier with which the chi square was formed from the fit function,
the total sample size under the normal likelihood and the total less the
number of groups under the Wishart likelihood, so n named
it for one case only.
The number of items is n_items, the number of raters
n_raters, the number of clusters n_clusters,
and the number of level one units in a cluster
cluster_size, in every function that takes or reports one.
ss_aipe_reliability() and
ss_aipe_reliability_sensitivity() took the number of items
as i and var_alpha() as p_items;
the row of reliability(), reliability_alpha(),
reliability_omega(),
reliability_omega_categorical(), and
reliability_kr20() that holds it was J and is
n_items, the name glance() already gave it,
and the row of ss_aipe_reliability_sensitivity() was
items. ss_aipe_icc(),
ss_aipe_icc_sensitivity(), and var_icc() took
the number of raters as k; the help keeps k in the
formulas and in the names of the forms ("ICC(1,k)").
ss_power_mixed_effects() took n,
rho, and J and takes
cluster_size, true_icc, and
specified_n_clusters_per_arm: the number of clusters in
each arm, not the total n_clusters, so its rows are
necessary_n_clusters_per_arm and
specified_n_clusters_per_arm, and its noncentrality
parameter is in the row noncentral_t_parm, as in
ss_power_c() and ss_power_smd().
cluster_size has no default and is checked first.
An intraclass correlation that a planner takes as a planning value is
true_icc: in ss_aipe_icc() (rho)
and ss_aipe_icc_sensitivity() (true_rho; a
correlation is true_cor everywhere else, and the value the
sensitivity study is planned from is supposed_icc, which
was estimated_rho), in
ss_power_mixed_effects() (rho), in the twelve
ss_aipe_crd_*() functions (icc_Y), and in
ss_aipe_mixed_effects() (icc).
var_icc() takes the intraclass correlation its variance is
evaluated at as icc, as in
design_effect().
The total variance of the outcome in a two-level plan is
true_total_var: in ss_aipe_mixed_effects()
(sigma2_y) and
ss_aipe_mixed_effects_sensitivity()
(true_sigma2_y, with supposed_total_var for
estimated_sigma2_y), and in the
ss_aipe_crd_*() functions (total_var), where
true_var_within is the within-cluster variance alone. Two
names that differed by one capital no longer hold two quantities. The
sensitivity function also plans from supposed_var_X and
supposed_icc (estimated_sigma2_x,
estimated_icc) and takes the population slope as
true_reg_coef (true_beta), whose simulated
summaries are the rows mean_reg_coef,
median_reg_coef, and sd_reg_coef.
The twelve planners for a cluster randomized design name every
population value with true_: true_var_between,
true_var_within, true_total_var,
true_icc, true_R2_between, and
true_R2_within, and in the six for the standardized effect
size true_icc_covariate (icc_Z) beside
true_es. The number of covariates beside the treatment
indicator is n_covariates (num_predictors; the
value is unchanged, and the total number of predictors is
p = n_covariates + 1). The number of individuals in a
cluster is the argument cluster_size of the four functions
that take it (n_individuals); the functions that solve for
it report it in the row necessary_cluster_size, beside
necessary_n_clusters in the two that solve for both, and a
size that was given is not repeated in the table. The function names
keep n_individuals. The four functions that plan from a
width echo it in the row width, as the other AIPE planners
do, and the width column of tidy() and
glance() is that desired width for every AIPE planner
(NA for a plan made from a budget and for a planner that
does not echo the request); the expected or assured width is the
glance() column of that name. The number of processes of
the six ss_aipe_crd_es_*() functions is ncpus
(multicore, num_proc), the name in the
bootstrap functions. The two costs are cluster_cost and
individual_cost, and the number of simulated data sets of
the six ss_aipe_crd_es_*() functions is G, as
in every other function that simulates (nrep). The row that
reports the width no longer carries the effect or the assurance in its
name: it is expected_width without an assurance and
assured_width with one, and the assurance is in a row of
its own, assurance. In 1.0.x the row was, for example,
width_of_unstd_conditions_diff_with_0.8_assurance, a name
that changed with the value asked for.
ss_aipe_reliability() names its population as a planner
does: true_cov_matrix (S, which is a sample
covariance matrix in the estimators), true_factor_loading
(lambda), true_residual_var
(psi_square), and true_item_covariance
(cor_est: the help defines it as the common covariance of
every pair of items, which is their correlation only for standardized
items, and it is a planning value, not an estimate). The replications of
the two stages of its assurance search are G_initial and
G (initial_iter, final_iter), the
names in ss_aipe_R2(), and an assurance plan reports the
rows assurance, assurance_attained, and
G (specified_assurance,
empirical_assurance, final_iter), the names in
ss_aipe_partial_r() and
ss_aipe_semipartial_r(). The estimators
(reliability_alpha() and the rest) keep S,
data, and items. The reliability
ss_aipe_reliability_sensitivity() plans from is
supposed_reliability (estimated_reliability):
a planning assumption, not an estimate from data.
The measurement model and item response functions name their
parameters for what they are (see the table).
irt_information() reports the grid in the column
trait_value and carries the attributes
discrimination, threshold, and
trait_value_max_information; the columns a and
b of an irt_grm() table keep their names, and
the help of irt_grm() and irt_information()
says what they are. A call of irt_information() that gives
the values by position is unaffected.
Every population value a planner for a regression coefficient takes
carries true_ and is named for its quantity: the
coefficient is true_reg_coef_j in the units of the data and
true_std_reg_coef_j when it is standardized (DMAR 1.0.x
used b_j for the first and beta_j for both), a
squared multiple correlation is true_R2_ followed by its
variables, and the standard deviations and correlations are
true_sd_Y, true_sd_X,
true_cor_XX, and true_cor_YX. The estimators
name what they are given the same way, without true_
(reg_coef_j, se_reg_coef_j, sd_Y,
sd_X, std_reg_coef_j).
The regression coefficient sensitivity functions lost
standardize: the name of the function says the scale.
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and
ss_power_reg_coef_sensitivity() report the unstandardized
coefficient, as their 1.0.x default did, and
ss_aipe_src_sensitivity() reports the standardized one, as
its 1.0.x default did. A call that gave standardize stops
with “unused argument”. The default of noncentral is
TRUE in ss_aipe_src_sensitivity() and
FALSE in the other two, as in DMAR 1.0.x, so the call that
replaces ss_aipe_reg_coef_sensitivity(standardize = TRUE)
or ss_aipe_rc_sensitivity(standardize = TRUE) is
ss_aipe_src_sensitivity(noncentral = FALSE), and the call
that replaces
ss_aipe_src_sensitivity(standardize = FALSE, noncentral = FALSE)
is ss_aipe_rc_sensitivity().
ss_aipe_src_sensitivity(standardize = FALSE), with its
default noncentral = TRUE, becomes
ss_aipe_rc_sensitivity(noncentral = TRUE), which is refused
when the sample size is planned from supposed variances that are not 1
(see Calls That Are Now Refused). With standardize = TRUE,
ss_power_reg_coef_sensitivity() gave the same power (the
t statistic does not depend on the scale), so such a call drops
the argument.
One name has one meaning, and the bounds of an equivalence region are
named for the quantity they bound: smd_lower and
smd_upper, contrast_lower and
contrast_upper, cor_lower and
cor_upper, and prop_diff_lower and
prop_diff_upper in
equivalence_proportion_paired(), which is new;
plot_equivalence(), which draws whatever estimates it is
given, takes effect_lower and effect_upper.
cles(), nnt_from_smd(), and
proportion_of_superiority() used smd_lower and
smd_upper for confidence limits on d, as arguments
and as rows; those are now smd_ci_lower and
smd_ci_upper, the names DMAR gives the limits of an
interval named for its estimate, so
cles(smd = .5, smd_lower = .2, smd_upper = .8) stops with
“unused argument”. convert_delta_lambda() takes
smd, and convert_lambda_delta() and
convert_lambda_R2() take ncp; the function
names keep their symbols.
A row or column that echoes a renamed argument is renamed with it, a
row that summarizes simulated estimates is named for the quantity
(mean_reg_coef_j, mean_contrast,
mean_std_contrast), and the columns of the per-replication
files that the sensitivity functions write with filename
follow the rows they summarize. Code that looks a row or column up by
its 1.0.x name must be changed, because an old name matches nothing and
gives no error: x$value[x$term == "rho"] returns
numeric(0). The columns of eta_squared(),
omega_squared(), ci_eta_squared(), and
ci_omega_squared() are under One Function for Each ANOVA
Effect Size.
The limits of every confidence interval are
ci_lower and ci_upper. DMAR 1.0.x
named them lower_limit and upper_limit in most
functions and ci_lower and ci_upper in the
coefficient tables and the tidy() results. One pair is now
used everywhere, as a row of a long table, a column of a wide one, and a
suffix of a limit named for its estimate (rmsea_ci_lower,
cles_ci_lower).
x$value[x$term == "lower_limit"] becomes
x$value[x$term == "ci_lower"], and
x$lower_limit becomes x$ci_lower.
| function | DMAR 1.0.x | now |
|---|---|---|
| every function that reports a confidence interval | rows or columns lower_limit,
upper_limit |
ci_lower, ci_upper |
| every function whose limits are named for their estimate | <estimate>_lower_limit,
<estimate>_upper_limit (for example
cles_lower_limit, smd_upper_limit,
shift_lower_limit, loa_upper_lower_limit,
mean_lower_limit) or <estimate>_lower,
<estimate>_upper (for example
tau2_lower, indirect_upper,
sigma2_b_lower,
eta_squared_partial_upper) |
<estimate>_ci_lower,
<estimate>_ci_upper (cles_ci_lower,
smd_ci_upper, shift_ci_lower,
loa_upper_ci_lower, mean_ci_lower,
tau2_ci_lower, indirect_effect_ci_upper,
sigma2_between_ci_lower,
eta_squared_partial_ci_upper) |
combine_p() |
row k |
row n_studies |
welch_t(), summary_t_test() |
row t_statistic |
row t_value |
pairwise_within(), ci_dunnett(),
ci_dunnett_t3() |
column t_statistic |
column t_value |
contrast_test() |
column t |
column t_value |
contrast_adjusted(), bayes_one_sample_t(),
bayes_paired_t(), bayes_independent_t() |
row t |
row t_value |
ci_scheffe() |
columns contrast_value, F_statistic |
columns estimate, F_value |
ci_c_ancova() |
row psi |
row contrast, as in ci_c() |
ci_sc_ancova() |
row psi |
row std_contrast, as in ci_sc() |
equivalence_c() |
rows psi_hat, delta_lower,
delta_upper |
rows contrast, contrast_lower,
contrast_upper (the lower bound is still signed);
plot_equivalence() reads them |
ci_rc() |
rows lower_limit, upper_limit only |
rows ci_lower, reg_coef,
ci_upper: the coefficient sits between its limits, as in
ci_reg_coef() and ci_src() |
var_r() |
row var_r_bonett_wright |
removed with kurtosis_x and kurtosis_y
(see Calls That Are Now Refused) |
ss_aipe_partial_r(),
ss_aipe_semipartial_r(),
ss_aipe_partial_r_sensitivity(),
ss_aipe_semipartial_r_sensitivity() |
row J |
row p, holding p |
expected_partial_r() |
columns n, J, rho |
columns N, p, true_cor |
expected_r() |
columns n, rho |
columns N, true_cor |
equivalence_r() |
rows n, rho_lower,
rho_upper |
rows N, cor_lower, cor_upper
(the lower one still holds the negative bound) |
ss_aipe_r(), ss_power_r(),
ss_aipe_partial_r() |
row rho |
row true_cor |
ss_power_r() |
row rho_0 |
row null_cor |
ss_aipe_r_sensitivity(),
ss_aipe_partial_r_sensitivity() |
rows true_rho, estimated_rho |
rows true_cor, supposed_cor |
ss_aipe_semipartial_r() |
row r_sp |
row true_r_sp |
ss_aipe_semipartial_r_sensitivity() |
row estimated_r_sp |
row supposed_r_sp (beside the new rows
true_R2_full and supposed_R2_full) |
ss_aipe_equivalence_r() |
row population_r |
row true_cor |
ss_aipe_equivalence_r_sensitivity() |
rows true_r, estimated_r,
rho_lower, rho_upper |
rows true_cor, supposed_cor,
cor_lower, cor_upper |
ss_aipe_R2_sensitivity(),
ss_power_R2_sensitivity() |
row estimated_R2 |
row supposed_R2 |
ss_power_R2_sensitivity() |
rows mean_F, median_F,
sd_F |
rows mean_F_value, median_F_value,
sd_F_value |
expected_R2() |
row expected_value_population_R2 |
row expected_R2, as var_R2() reports
var_R2 (the value is unchanged) |
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity() |
rows mean_b_j, median_b_j,
sd_b_j, true_b_j,
estimated_b_j |
rows mean_reg_coef_j, median_reg_coef_j,
sd_reg_coef_j, true_reg_coef_j,
supposed_reg_coef_j |
ss_aipe_src_sensitivity() |
rows mean_b_j, median_b_j,
sd_b_j, true_b_j,
estimated_b_j |
rows mean_std_reg_coef_j,
median_std_reg_coef_j, sd_std_reg_coef_j,
true_std_reg_coef_j, supposed_std_reg_coef_j
(the standardized coefficient, as true_std_reg_coef_j in
ss_aipe_src()) |
ss_power_reg_coef_sensitivity() |
rows mean_b_j, median_b_j,
sd_b_j, mean_se_b_j,
median_se_b_j, sd_se_b_j,
true_b_j, estimated_b_j, mean_t,
median_t, sd_t |
rows mean_reg_coef_j, median_reg_coef_j,
sd_reg_coef_j, mean_se_reg_coef_j,
median_se_reg_coef_j, sd_se_reg_coef_j,
true_reg_coef_j, supposed_reg_coef_j,
mean_t_value, median_t_value,
sd_t_value |
ci_cv() |
rows c_of_v, c_of_v_unbiased |
rows cv, cv_unbiased |
ss_aipe_smd(), ss_power_smd(),
ss_power_composite_ancova_2group() |
row supposed_smd |
row true_smd |
ss_power_composite_ancova_2group() |
rows supposed_rho_group_1,
supposed_rho_group_2 |
rows true_cor_group_1,
true_cor_group_2 |
ss_aipe_equivalence_smd() |
row population_smd |
row true_smd |
ss_aipe_smd_sensitivity() |
rows true_delta, estimated_delta |
rows true_smd, supposed_smd |
ss_aipe_equivalence_smd_sensitivity() |
rows estimated_smd, delta_lower,
delta_upper |
rows supposed_smd, smd_lower,
smd_upper |
equivalence_smd() |
rows delta_lower, delta_upper |
rows smd_lower (still the signed lower bound),
smd_upper |
expected_smd() |
column delta |
column true_smd |
cles() |
rows cl, cl_lower, cl_upper,
smd_lower, smd_upper |
rows cles, cles_ci_lower,
cles_ci_upper, smd_ci_lower,
smd_ci_upper |
nnt_from_smd() |
rows nnt_lower, nnt_upper,
smd_lower, smd_upper |
rows nnt_ci_lower, nnt_ci_upper,
smd_ci_lower, smd_ci_upper |
proportion_of_superiority() |
rows smd_lower, smd_upper |
rows smd_ci_lower, smd_ci_upper |
ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity() |
rows mean_psi, median_psi,
sd_psi, true_psi,
estimated_psi |
rows mean_std_contrast,
median_std_contrast, sd_std_contrast,
true_std_contrast, supposed_std_contrast |
ss_aipe_sc_ancova_sensitivity() |
row rho |
row true_cor |
ss_aipe_c_sensitivity() |
rows mean_psi, median_psi,
sd_psi, true_psi,
true_error_variance,
estimated_error_variance |
rows mean_contrast, median_contrast,
sd_contrast, true_contrast,
true_var_error, supposed_var_error |
ss_aipe_c_ancova_sensitivity() |
rows mean_psi, median_psi,
sd_psi, true_psi,
est_error_var_ancova, rho |
rows mean_contrast, median_contrast,
sd_contrast, true_contrast,
supposed_var_error_ancova, true_cor |
ss_seq_c_sensitivity() |
row true_psi |
row true_contrast |
ss_aipe_omega_squared() |
row population_omega_squared |
row true_omega_squared |
ss_aipe_omega_squared_sensitivity() |
row estimated_omega_squared |
row supposed_omega_squared |
ancova() |
rows eta_squared_partial_lower,
eta_squared_partial_upper,
omega_squared_partial_lower,
omega_squared_partial_upper |
rows eta_squared_partial_ci_lower,
eta_squared_partial_ci_upper,
omega_squared_partial_ci_lower,
omega_squared_partial_ci_upper |
factorial_anova() |
columns eta_squared_partial_lower,
eta_squared_partial_upper,
omega_squared_partial_lower,
omega_squared_partial_upper |
columns eta_squared_partial_ci_lower,
eta_squared_partial_ci_upper,
omega_squared_partial_ci_lower,
omega_squared_partial_ci_upper |
ss_power_rm_anova() |
rows necessary_n_subjects,
specified_n_subjects |
rows necessary_N, specified_N |
ss_power_one_way_anova(),
ss_power_rm_anova(),
ss_power_split_plot_anova(),
ss_power_factorial_anova(),
ss_power_factorial_ancova() |
row noncentrality |
row noncentral_f_parm |
ss_power_rm_anova(),
ss_power_split_plot_anova(),
ss_power_factorial_anova() |
rows effect_df, error_df |
rows df_effect, df_error |
ss_power_factorial_ancova() |
rows f, f_adjusted,
covariate_R2 |
rows true_f, true_f_adjusted,
true_covariate_R2 |
ss_power_contrast() |
row effect_size_f |
row true_f |
variance_components_mls() |
rows sigma2_b_lower, sigma2_b_upper,
sigma2_w_lower, sigma2_w_upper,
icc_lower, icc_upper |
rows sigma2_between_ci_lower,
sigma2_between_ci_upper,
sigma2_within_ci_lower,
sigma2_within_ci_upper, icc_ci_lower,
icc_ci_upper |
mr_smd() |
rows n1, n2, pilot_ss |
rows n_1, n_2,
pilot_n_per_group |
mr_cv() |
rows n, pilot_ss |
rows N, pilot_N |
ss_aipe_icc() |
rows rho, k |
rows true_icc, n_raters |
ss_aipe_icc_sensitivity() |
rows true_rho, estimated_rho,
k |
rows true_icc, supposed_icc,
n_raters |
ss_power_mixed_effects() |
rows necessary_J_per_arm,
specified_J_per_arm, noncentrality |
rows necessary_n_clusters_per_arm,
specified_n_clusters_per_arm,
noncentral_t_parm |
the twelve ss_aipe_crd_*() and
ss_aipe_crd_es_*() functions |
row exp_width_of_unstd_conditions_diff or
exp_width_of_<level>_effect_size |
row expected_width |
the twelve ss_aipe_crd_*() and
ss_aipe_crd_es_*() functions |
row
width_of_unstd_conditions_diff_with_<assurance>_assurance
or
width_of_<level>_effect_size_with_<assurance>_assurance |
rows assured_width and assurance |
the eight ss_aipe_crd_*() and
ss_aipe_crd_es_*() functions that solve for the cluster
size (_n_individuals_ and _both_) |
row cluster_size |
row necessary_cluster_size |
ss_aipe_reliability() |
rows specified_assurance,
empirical_assurance, final_iter |
rows assurance, assurance_attained,
G |
ss_aipe_reliability_sensitivity() |
rows items, estimated_reliability |
rows n_items, supposed_reliability |
reliability(), reliability_alpha(),
reliability_omega(),
reliability_omega_categorical(),
reliability_kr20() |
row J |
row n_items |
ss_aipe_mixed_effects() |
row icc |
row true_icc |
ss_aipe_mixed_effects_sensitivity() |
rows true_sigma2_y, true_sigma2_x,
estimated_sigma2_y, estimated_sigma2_x,
estimated_icc, true_beta,
mean_beta, median_beta, sd_beta;
column beta of the file written with
filename |
rows true_total_var, true_var_X,
supposed_total_var, supposed_var_X,
supposed_icc, true_reg_coef,
mean_reg_coef, median_reg_coef,
sd_reg_coef; column reg_coef |
ss_power_pcm() |
rows unstd_coefficient, std_coefficient,
true_var_of_slopes, l1_error_var,
error_var_of_slopes |
rows true_reg_coef, true_std_reg_coef,
true_var_trend, true_var_error,
true_var_trend_error |
ss_power_pcm() |
row freq |
row frequency |
ss_power_pcm() |
row measurement_occasions |
row n_occasions |
ss_aipe_pcm_sensitivity() |
rows true_variance_trend,
true_error_variance, estimated_variance_trend,
estimated_error_variance |
rows true_var_trend, true_var_error,
supposed_var_trend, supposed_var_error |
ss_aipe_pcm_sensitivity() |
row n_timepoints |
row n_occasions |
ss_aipe_indirect_effect() |
rows a, b, ab |
rows true_std_reg_coef_a,
true_std_reg_coef_b,
true_std_indirect_effect |
ss_power_indirect_effect() |
rows a, b, c_prime,
indirect_effect, actual_power |
rows true_std_reg_coef_a,
true_std_reg_coef_b, true_std_reg_coef_direct,
true_std_indirect_effect, power_ab (the power
to detect the indirect effect, followed by power_a and
power_b as before; tidy() and
glance() read it as the power of the design) |
ss_aipe_indirect_effect_sensitivity() |
rows mean_ab, median_ab,
sd_ab, true_a, true_b,
true_ab, estimated_a,
estimated_b; column ab of the file written
with filename |
rows mean_std_indirect_effect,
median_std_indirect_effect,
sd_std_indirect_effect, true_std_reg_coef_a,
true_std_reg_coef_b, true_std_indirect_effect,
supposed_std_reg_coef_a,
supposed_std_reg_coef_b; column
std_indirect_effect |
mediate() |
rows indirect_lower, indirect_upper,
direct_lower, direct_upper,
total_lower, total_upper |
rows indirect_effect_ci_lower,
indirect_effect_ci_upper,
direct_effect_ci_lower,
direct_effect_ci_upper, total_effect_ci_lower,
total_effect_ci_upper (the estimates are the rows
indirect_effect, direct_effect, and
total_effect) |
ss_aipe_composite_sem() |
rows desired_width_<label>,
population_<label>,
width_within_desired_<label> |
rows width_<label>,
true_<label>,
pct_ci_less_width_<label> |
ss_power_composite_sem() |
rows population_<label> |
rows true_<label> |
ss_power_composite_anova(),
ss_power_composite_factorial_anova(),
ss_power_composite_ancova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_ancova_het() |
rows f_<label>,
noncentral_parm_<label>,
residual_df |
rows true_f_<label>,
noncentral_f_parm_<label>, df_error |
| the same five planners | rows necessary_N, specified_N (beside the
per-cell size); approximate_N |
row total_N; approximate_total_N |
ss_power_composite_ancova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_anova() |
row covariate_R2 |
row true_covariate_R2 |
ss_power_composite_factorial_ancova_het(), and
ss_power_composite_ancova(slopes = "heterogeneous"), which
returns its table |
attributes means, correlations,
sigma, sd_cov |
attributes true_means, true_cor,
true_sd, true_sd_X (read by the
plot() method) |
ss_power_composite_ancova_2group() |
rows sigma, sd_cov,
residual_df |
rows true_sd, true_sd_X,
df_error |
ss_power_composite_ancova_2group() |
rows necessary_N, specified_N (beside the
per-group size); approximate_N |
row total_N; approximate_total_N |
ss_aipe_cliff_delta() |
row delta |
row true_delta |
ss_aipe_cliff_delta_sensitivity() |
row estimated_delta |
row supposed_delta |
ss_aipe_cv_sensitivity() |
row estimated_cv |
row supposed_cv |
ss_aipe_sm_sensitivity() |
row estimated_sm |
row supposed_sm |
design_consequences() |
row target_width |
row width |
ss_aipe_r(), ss_aipe_partial_r(),
ss_aipe_semipartial_r(), ss_aipe_icc(),
ss_aipe_omega_squared(),
ss_aipe_indirect_effect(),
ss_aipe_cliff_delta() |
row width_target |
row width |
ss_aipe_equivalence_r(),
ss_aipe_equivalence_smd(),
ss_aipe_mixed_effects() |
row ci_width_expected |
row expected_width |
ss_power_R2() |
row effect_size |
row true_f2 |
ss_power_reg_coef(), ss_power_rc() |
row effect_size |
row true_f (Cohen’s f, the square root of
true_f2) |
meta_es(), meta_r(),
meta_smd(), meta_contrast() |
row k |
row n_studies |
meta_es(), meta_r(),
meta_smd() |
rows tau2_lower, tau2_upper,
I2_lower, I2_upper |
rows tau2_ci_lower, tau2_ci_upper,
I2_ci_lower, I2_ci_upper
(prediction_lower and prediction_upper, the
limits of the prediction interval, keep their names) |
meta_es(), meta_r(),
meta_smd() |
row t under the Hartung-Knapp adjustment,
z without it |
row test_statistic under either; the reference
distribution is in the attributes test_distribution
("t" or "normal") and test_df
(n_studies - 1, or NA), and a line under the
printed table states it |
meta_contrast() |
row z |
row test_statistic (the attribute
test_distribution is "normal" and
test_df is NA) |
procrustes_phi() |
row n_perm |
row n_resamples |
design_consequences() and the 25
ss_aipe_*_sensitivity() functions of DMAR 1.0.x |
row pct_ci_less_w |
row pct_ci_less_width |
design_effect() |
row n_total |
row N |
obrien_test() |
column n_total |
column N |
contrast_adjusted() |
row p |
row p_value |
anova_within_two_way(),
simple_effects_AB() |
column partial_eta_squared |
column eta_squared_partial |
anova_within() |
attribute partial_eta_squared |
attribute eta_squared_partial |
R2_mixed_effects()
(ci_method = "boot") |
rows R2_marginal_lower, R2_marginal_upper,
R2_conditional_lower,
R2_conditional_upper |
rows R2_marginal_ci_lower,
R2_marginal_ci_upper, R2_conditional_ci_lower,
R2_conditional_ci_upper (its tidy() still
gives ci_lower and ci_upper columns) |
bayes_one_sample_t(),
bayes_paired_t() |
row n |
row N |
ci_proportion() |
row n |
row N |
cohen_kappa() |
column n |
column N |
vargha_delaney_A() |
column z_value |
column test_statistic |
mlmr(), mlmr_mv() (the
coef_table); cfa_k(), cfa_1(),
cfa_2() (the parameter table) |
column z_value |
column test_statistic |
dmacs() |
columns lambda_reference, lambda_focal,
nu_reference, nu_focal (and the same columns
of tidy()) |
columns factor_loading_reference,
factor_loading_focal, intercept_reference,
intercept_focal |
irt_information() |
column theta; attributes a,
b, theta_max_information |
column trait_value; attributes
discrimination, threshold,
trait_value_max_information |
covmat_from_cfa() |
element population_cov of the returned list |
element model_implied_cov |
power_equivalence_md_plot() |
column diff of the power_grid attribute
(and of the plotted data) |
column true_effect (ratios when
logscale = TRUE, as before; the n=<size>
columns keep their labels) |
reliability_H() from loadings |
rows reliability_H, lower_limit,
upper_limit, var_H |
the reliability family table: rows estimate,
se, ci_lower, ci_upper,
conf_level, N, N_complete,
n_items (var_H is the square of
se) |
ss_aipe_rmsea_sensitivity(),
ss_aipe_sem_path_sensitivity() |
row suc_rep |
removed; attr(x, "simulation")$returned |
ss_aipe_R2_sensitivity() |
row num_probs_with_cis |
removed; attr(x, "simulation")$failed |
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity() |
columns b_j, ll_ci_beta_j,
ul_ci_beta_j, r_2, se_b_j,
t_for_b_j of the per-replication file written with
filename |
columns reg_coef_j, ci_lower,
ci_upper, R2, se_reg_coef_j,
t_value |
ss_aipe_src_sensitivity() |
the same six columns of its file | columns std_reg_coef_j, ci_lower,
ci_upper, R2, se_std_reg_coef_j,
t_value |
ss_power_reg_coef_sensitivity() |
columns b_j, se_b_j, t_stat,
r_2 of its file |
columns reg_coef_j, se_reg_coef_j,
t_value, R2 |
ss_power_R2_sensitivity() |
columns r2, f_stat of its file |
columns R2, F_value |
ss_aipe_R2_sensitivity() |
columns r2, lower_width_ci,
upper_width_ci, width_ci of its file |
columns R2, ci_width_lower,
ci_width_upper, ci_width, the names of the
rows that summarize them |
ss_aipe_c_sensitivity() |
column psi_hat of its file |
column contrast |
ss_aipe_c_ancova_sensitivity() |
columns psi_obs, se_psi,
se_psi_restricted of its file |
columns contrast, se_contrast,
se_contrast_restricted |
ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity() |
column psi_obs of the file |
column std_contrast |
ss_aipe_rmsea_sensitivity(),
ss_aipe_sem_path_sensitivity() |
columns ci_low, ci_up of the file |
columns ci_lower, ci_upper; a file begun
by DMAR 1.0.x keeps its header when a run appends to it |
var_cv() |
rows var_cv_mckay, var_cv_vangel |
row var_cv_normal, which holds the value that
var_cv_mckay held; var_cv_vangel is removed
(see Changes to Earlier Results) |
The reasons for each name are in One Notation for the Regression and Correlation Functions, below.
The removed rows suc_rep and
num_probs_with_cis held counts of replications, which every
sensitivity function now reports one way, in the simulation
attribute of its table (see Changes to Earlier Results):
suc_rep was the number of replications that gave an
interval, and num_probs_with_cis the number that did not.
reliability_H() returns the same table for every kind of
input; the table of DMAR 1.0.0 is not returned for any call, and there
is no warning period (see Coefficient H From a Fitted Model).
The column that holds a test statistic is test_statistic
wherever the reference distribution depends on an option: under
test = "t" in vargha_delaney_A(), and under a
finite wald_df in the multilevel and confirmatory factor
analysis tables, the statistic is referred to Student’s t, so
z_value named it wrongly. The meta-analysis functions named
their statistic t under hartung_knapp = TRUE
and z without it (z in
meta_contrast()); it is test_statistic in all
four, with the reference distribution in the attributes
test_distribution and test_df and in a line
under the printed table, and the table of meta_contrast()
now has the class dmar_meta_contrast, a
dmar_tbl. Otherwise a column whose name is always true
keeps it (z_value in cohen_kappa() and
fleiss_kappa(), t_value in
welch_t() and its relatives).
The common language effect size is the row cles wherever
it is reported: cles() named it cl, as
randomization_test() did not, and
results_sentence() on a cles() table begins
“cles =” where it began “cl =”.
ss_power_R2_change() is new in this release, so its rows
true_f2 and true_partial_R2 have no 1.0.x
names; they are named as planning values like every other row that holds
one. The same holds for the other functions new in this release:
ss_aipe_two_phase(), ss_aipe_two_phase_cost(),
and ss_aipe_two_phase_sensitivity() echo
true_cor, and the sensitivity function
supposed_sd_Y and supposed_cor;
equivalence_proportion_paired() reports its bounds in the
rows prop_diff_lower and prop_diff_upper;
ss_aipe_mean_sensitivity() echoes supposed_sd;
ss_aipe_crossed_sensitivity() echoes
supposed_var_unit, supposed_var_rater, and
supposed_var_residual; and
ss_aipe_mixed_effects_sensitivity() echoes
supposed_icc_X.
var_r() no longer takes
kurtosis_x and kurtosis_y. DMAR 1.0.x
returned a row var_r_bonett_wright, the normal-theory
variance times 1 + rho^2 (kurtosis_x + kurtosis_y) / 4, and credited it
to Bonett and Wright (2000). That article has no such formula, no other
source for it was found, and the help’s statement that it is exact for
elliptical distributions was wrong: there the variance is the
normal-theory variance times one factor that does not depend on the
correlation. The large-sample variance under non-normality depends on
every fourth-order cumulant of the joint distribution (Duncan &
Layard, 1973, Equation 1.1), which two marginal kurtoses do not supply.
A call that gives either argument stops with “unused argument”.
desired_power must be strictly between 0 and
1 in every power planner. ss_power_R2() accepted 0
and 1 (and returned a sample size for a power of 1);
ss_power_pcm() made no check. Every ss_power_*
function now applies one rule and gives one refusal, of class
dmar_desired_power_invalid, that carries the value
given.
The unbiased standardized mean difference on one degree
of freedom is refused. smd(unbiased = TRUE) with
n_1 + n_2 = 3 and smd_c(unbiased = TRUE) with
n_C = 2 returned 0, because the correction factor of Hedges
(1981) is 0 at one degree of freedom whatever the data. They now stop
with a refusal of class dmar_hedges_correction_undefined
that says why and what to do (unbiased = FALSE, or more
observations). On no degrees of freedom (smd_c() from
summary statistics with n_C = 1, which returned
Inf, or NaN when the means were equal;
smd() with one standard deviation and
n_1 + n_2 = 2) the refusal has the same class and states
its own reason: the correction factor is not defined there, because
there is no standard deviation.
desired_power is checked when a sample size
is given too. A power planner that is given a sample size
computes the power and does not use desired_power; a value
outside (0, 1) given with it was accepted without a word
(ss_power_r(true_cor = .3, specified_N = 60, desired_power = 1)
returned a table) and is now refused with
dmar_desired_power_invalid.
smd() from summary statistics refuses a
group of one that has its own standard deviation. With
s_1 and s_2, n_1 = 1 and
n_2 = 2 returned a number and two groups of one returned
NaN, where raw data with a group of one were already
refused. Each size must be at least 2; the refusal has class
dmar_argument_invalid.
conf_level, alpha_level,
width, and assurance are checked on entry, one
way. conf_level and alpha_level are
each a single number strictly between 0 and 1; width is a
single positive finite number; assurance is
NULL or a single number strictly between 0 and
NA, a logical, a character string, a vector of two)
reached the computation in many functions. Fifteen planners searched
without end for a width of 0 or below
(ss_aipe_sm(true_sm = 10, width = -1),
ss_aipe_R2(true_R2 = .5, p = 5, width = -1)); others
returned a number for it (ss_aipe_rmsea() 20,000,
ss_aipe_reliability() Inf);
ci_c() returned limits of Inf and
-Inf for conf_level = -1; and
ss_power_reg_coef() did not return for
alpha_level = 0. Each is now refused at the first line of
the function with a condition of class
dmar_argument_invalid (dmar_assurance_invalid
for assurance) that names the argument, carries the value
given, and says what to give. The check is at the head of the 166
functions in which a degenerate value was found to run or to stop with
R’s own message; a function with a narrower range of its own (a width
below 2 for a correlation, a level below .5 for an equivalence test, an
assurance above .5) applies it after. An argument with no default is
checked when it is given.A number that is missing, empty, or not a number is
refused by name. An argument that holds one number and was
given as NA, NaN, a character string, a list,
or a vector of length 0 stopped inside the computation with R’s message
(“missing value where TRUE/FALSE needed”, “non-numeric argument to
binary operator”, “argument is of length zero”), which names neither the
argument nor the function. In 79 functions those arguments are now
checked on entry: ci_smd(smd = .5, n_1 = NA, n_2 = 10)
stops with “‘n_1’ must be a number that is not missing; NA was given”,
class dmar_argument_invalid. No call that returned a result
is affected.
The interval functions check conf_level
before the tail areas are derived from it.
ci_pvaf(), ci_snr(), ci_srsnr(),
ci_eta_squared(), ci_omega_squared(), and
ci_sc() refused conf_level = 1.2 with a
message about alpha_lower, an argument the call did not
give. The refusal now names conf_level.
ss_aipe_R2(), ss_aipe_R2_sensitivity(), and
plot_R2() gave an empty message for
conf_level = NULL; they give the same refusal.
ci_c() refuses one tail area given without
the other.
ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = NULL, alpha_upper = .025)
(n in 1.0.0) stopped in DMAR 1.0.0 and the development
versions of 1.1.0 with R’s own message, “Non-numeric argument to
mathematical function”, and with only alpha_lower given,
“arguments imply differing number of rows: 3, 2”. It is now refused with
class dmar_argument_invalid, arg
c("alpha_lower", "alpha_upper"): “Only ‘alpha_upper’ was
given. ‘alpha_lower’ and ‘alpha_upper’ go together: each is the tail
area beyond one limit, so the interval needs both. Give both (for
example alpha_lower = .025 and alpha_upper = .025, or 0 for a one-sided
interval), or neither and a ‘conf_level’.”
ss_power_c() and ss_power_sc()
refuse a contrast of 0 when a sample size is to be found. A
population contrast of 0 is the null hypothesis; the power is
alpha_level at every sample size. The two functions
searched a million sample sizes and stopped with “Failed to converge
within reasonable sample size.” They now stop at once with a refusal of
class dmar_argument_invalid that says so. With a sample
size given they return the power at 0, as before. The nine power
planners that had that message for a search that reaches its limit
(ss_power_c(), ss_power_sc(),
ss_power_c_ancova(), ss_power_smd(),
ss_power_one_way_anova(),
ss_power_factorial_anova(),
ss_power_rm_anova(),
ss_power_split_plot_anova(),
ss_power_mixed_effects()) now say what it means (the effect
at the planning values is too small for the power asked for at any size
up to 1,000,000) and what to check; the class is
dmar_computation_failed, as before.
ss_aipe_R2_sensitivity() refuses
supposed_R2 together with
specified_N. It ran the simulation at the sample
size planned from the supposed value and ignored
specified_N (a call with specified_N = 6
reported total_N 34). The coefficient of variation, Cliff’s
delta, and reliability siblings refused the pair already; all four
refusals now have class dmar_argument_invalid.
ss_aipe_sc_ancova_sensitivity() asks for
width when it plans. With
supposed_std_contrast and no width the call
stopped inside the planner with “argument is of length zero”; it now
stops with a refusal of class dmar_argument_invalid that
names width.
ss_aipe_reliability() checks its two counts
of replications. G_initial and G
(initial_iter and final_iter in 1.0.x) each
must be a single whole number of at least 1; a fractional number, 0, a
negative number, NA, or more than one value reached the
search unchecked and is now an error of class
dmar_count_invalid, as for every other
G.
ss_power_mixed_effects() refuses a missing
or inadmissible cluster_size, true_icc, or
specified_n_clusters_per_arm by class. Each
refusal has class dmar_argument_invalid, carries the name
of the argument, and says what the value must be and why. A cluster size
or a number of clusters that is not finite was not refused in
1.0.x.
An element of effects with a field the
composite planners do not read is refused. Each element of
effects in ss_power_composite_anova(),
ss_power_composite_factorial_anova(),
ss_power_composite_ancova(),
ss_power_composite_factorial_ancova(), and
ss_power_composite_factorial_ancova_het() is a list of
factors, true_f or
true_eta_squared_partial, and label (and
type under heterogeneous slopes). A field under any other
name, the f and partial_eta_squared of 1.0.x
among them, or an element with no name, was not looked at: with the
sizes read from true_means it was dropped without a word.
It is now an error of class dmar_argument_invalid that
names the effect and the field and gives the fields that are
read.
ss_power_c_ancova() refuses an
n_per_group that is not a single finite number of at least
2 by class (dmar_argument_invalid). A size below 2
was a plain error in 1.0.x, and a size that is not finite was not
refused.
An argument a function does not have is an error in 20
more functions. See the paragraph on ... under Old
Names That R Still Matches.
bifactor_indices() refuses a fit with one
group factor. A general factor beside a single group factor is
not a bifactor model with subscales: its indices divide the common
variance between those two factors alone, every item belongs to the one
subscale, and PUC is 0. DMAR 1.0.0 refused only a fit with no group
factor and described a fit with one when every item was listed under
that factor (some of them with loadings fixed at zero). Such a fit now
stops with a message that names the group factor and says what to fit
instead, as it does in fa_bifactor() and
ci_bifactor_indices(), which are new in this
release.
The six ss_aipe_crd_es_*() planners refuse a
width asked for without the effect size. The width of the
interval is read from data simulated at true_es. Without
true_es the four planners that take or return a width
stopped with a message that did not name the cause
(ss_aipe_crd_es_n_individuals_fixed_width() said that the
width seemed impossible to reach with the number of clusters); they now
stop at once with a refusal of class dmar_argument_invalid
that says why the effect size is needed and what to give. In
ss_aipe_crd_es_n_clusters_fixed_budget() and
ss_aipe_crd_es_n_individuals_fixed_budget() the expected
width is asked for by supplying G: with G and
without true_es the call stopped with “arguments imply
differing number of rows: 660, 0”, and with G and without
pr_treat or true_icc it returned the table
without the width and without a word. Both are now refused with the same
class, in a message that names every population value that is missing
and says that G = NULL returns the size the budget buys
without a width. A true_es that is not a single finite
number is refused the same way. With one covariate
(n_covariates = 1) the same two functions had no default
for true_R2_within and true_R2_between, where
the four planners that take a width have 0, and did not use
true_icc for a missing true_icc_covariate as
the help says: a call with G and a covariate and without
all three stopped with “missing value where TRUE/FALSE needed” or
“arguments imply differing number of rows”. The two proportions now
default to 0 in all six functions, a missing
true_icc_covariate is true_icc, and a value of
any of the three that is not a single number in range is refused with
class dmar_argument_invalid, by name.
The twelve cluster randomized planners
(ss_aipe_crd_*() and ss_aipe_crd_es_*()) check
their sizes, proportions, and allocation on entry.
n_covariates must be a whole number of 0 or more,
cluster_size a whole number of at least 2 (the cluster size
searches start at 2), n_clusters a whole number of at least
3 + n_covariates (the interval has
n_clusters - 2 - n_covariates degrees of freedom),
pr_treat strictly between 0 and 1, and
true_R2_between and true_R2_within in [0, 1),
as the two standardized budget planners already required of a covariate.
Each refusal has class dmar_argument_invalid and names the
argument as arg. DMAR 1.0.0 and the development versions of
1.1.0 planned with whatever they were given (the calls are written with
the 1.1.0 names; 1.0.0 gave the same results under its own):
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8)
returned 17.5 clusters with n_covariates = .5, 5 with
true_R2_between = 1.1, and 19 with
true_R2_within = -.5. Fractional sizes were planned for and
are now refused: cluster_size = 20.5 gave 62 clusters at
width = .5, and
ss_aipe_crd_n_individuals_fixed_width(width = .5, n_clusters = 60.5, ...)
a cluster size of 23; so is cluster_size = 1, which gave
249 clusters. n_clusters = 2 stopped with “missing value
where TRUE/FALSE needed”
(ss_aipe_crd_n_individuals_fixed_width()) or returned an
expected width of NaN
(ss_aipe_crd_n_individuals_fixed_budget()), and
pr_treat = 0 or 1 searched without end. A
fixed number of clusters that leaves a condition without a cluster is
refused, wherever a width is computed, with reason
"arm_without_clusters" and the fewest clusters that would
do (fewest):
ss_aipe_crd_n_individuals_fixed_budget(budget = 1000, n_clusters = 5, pr_treat = .1, ...)
returned a cluster size of 200 with an expected width of
Inf, since round(5 * .1) = 0 clusters are treated, and is
refused with fewest = 6.
ss_aipe_crd_es_n_individuals_fixed_budget() makes the check
only when G asks for the width, since without
G it does not use pr_treat: with
cluster_cost = 0, individual_cost = 1, G = 2, true_icc = .2, true_es = .5
the same design returned a cluster size of 200 with an expected width of
NaN in DMAR 1.0.0 and stopped with class
dmar_computation_failed after simulating it in the
development versions, and without G it still returns
200.
Costs and budgets in the cluster randomized planners must
be single finite numbers of 0 or more, and costs of 0 are refused where
they decide the design. Costs that are both 0 are refused
(reason "costs_all_zero") in the budget
planners and the both planners, and
individual_cost = 0 (reason
"individual_cost_zero") where the costs choose the cluster
size: ss_aipe_crd_n_individuals_fixed_budget(),
ss_aipe_crd_both_fixed_budget(),
ss_aipe_crd_both_fixed_width(), and their
ss_aipe_crd_es_*() counterparts. Each class is
dmar_argument_invalid, and each message says what to give.
A search over designs that all cost the same has no end. In DMAR 1.0.0
and the development versions of 1.1.0 all six budget planners searched
without end with both costs 0 (none returned within 4 seconds), among
them
ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 0, individual_cost = 0);
so did, with cluster_cost = 10 and
individual_cost = 0,
ss_aipe_crd_n_individuals_fixed_budget(budget = 1000, n_clusters = 20, ...),
ss_aipe_crd_es_n_individuals_fixed_budget() with the same
arguments,
ss_aipe_crd_both_fixed_budget(budget = 10000, pr_treat = .5, true_var_between = .2, true_var_within = .8, ...),
and
ss_aipe_crd_es_both_fixed_budget(budget = 10000, true_es = .5, true_icc = .2, pr_treat = .5, G = 2, seed = 113, ...),
which with a budget of 1000 stopped instead with “‘from’ must be a
finite number”. With the same costs
ss_aipe_crd_both_fixed_width(width = .5, ...) returned a
table of NA with a character value column, and
ss_aipe_crd_es_both_fixed_width() stopped with “missing
value where TRUE/FALSE needed”. Clusters of a fixed size cost only their
clusters, so ss_aipe_crd_n_clusters_fixed_budget() with
individual_cost = 0 and a positive
cluster_cost still plans (100 clusters for a budget of 1000
at cluster_cost = 10). In the four planners that plan from
a width and a fixed size the costs only price the plan, and costs of 0
still give a budget of 0. A negative cost, a vector of costs, and a
negative budget were used, and are refused:
ss_aipe_crd_n_clusters_fixed_width(width = .5, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8, cluster_cost = -1, individual_cost = 1)
reported a budget of 1178 (62 clusters at 19 each),
ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, individual_cost = c(1, 2))
stopped with “arguments imply differing number of rows”, and
ss_aipe_crd_n_clusters_fixed_budget(budget = -5, cluster_size = 20)
returned NA, in DMAR 1.0.0 and the development
versions.
A budget that does not buy the smallest design a cluster
randomized planner can return is refused with the cost of that
design. The refusal has class
dmar_argument_invalid, arg
"budget", reason
"budget_infeasible", and min_cost, and the
budget it names buys a design the planner returns:
ss_aipe_crd_n_clusters_fixed_budget(budget = 1, cluster_size = 10, cluster_cost = 10, individual_cost = 1)
(4 clusters of 10 cost 80) and
ss_aipe_crd_n_individuals_fixed_budget(budget = 1, n_clusters = 10, cluster_cost = 10, individual_cost = 1)
(10 clusters of 2 cost
120) returned a table whose size and budget were NA in
DMAR 1.0.0 and the development versions of 1.1.0, as did their
ss_aipe_crd_es_*() counterparts;
ss_aipe_crd_both_fixed_budget(budget = 1, cluster_cost = 10, individual_cost = 1, pr_treat = .5, ...)
and ss_aipe_crd_es_both_fixed_budget() with the same budget
stopped with “invalid ‘type’ (list) of argument” (4 clusters of 2 cost
48). When the width is asked for (pr_treat, or
G in the standardized planner), the smallest design
ss_aipe_crd_n_clusters_fixed_budget() can return has a
cluster in each condition and a degree of freedom:
ss_aipe_crd_n_clusters_fixed_budget(budget = 60, cluster_size = 20, cluster_cost = 0, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8)
returned a table of NA in DMAR 1.0.0 and the development
versions, and is refused with the cost of 6 clusters, 120 (with
n_covariates = 2 and pr_treat = .5, 5
clusters, 100); ss_aipe_crd_es_n_clusters_fixed_budget()
with the same budget and G = 2, true_icc = .2, true_es = .5
stopped with “NA/NaN argument” in both.
A budget that buys clusters of a fixed size with no
finite width is refused when pr_treat asks for the
width.
ss_aipe_crd_n_clusters_fixed_budget(budget = 100, cluster_size = 20, cluster_cost = 0, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8)
returned 5 clusters, none of them treated, with an
expected_width of Inf in DMAR 1.0.0 and the
development versions of 1.1.0. It is refused (class
dmar_argument_invalid, reason
"arm_without_clusters") with the budget, the design it
buys, and the cheapest design that puts a cluster in each condition (6
clusters, cost 120); a design with no degrees of freedom (4 clusters
with n_covariates = 2) is refused with reason
"too_few_clusters".
ss_aipe_crd_es_n_clusters_fixed_budget() makes the same
check before it simulates a width.
The six unstandardized cluster randomized planners refuse
a negative, missing, or contradictory variance before
searching. Any two of true_var_between,
true_var_within, true_total_var, and
true_icc determine the model. Each one given must be a
single finite number (reason
"variance_input_invalid"); a variance must be 0 or more,
the total positive, whether it is given or is the sum of the two
variances, and the intraclass correlation in [0, 1] (reason
"variance_out_of_range"); fewer than two inputs, or a
between variance of 0 with an intraclass correlation of 0, is refused
with reason "underidentified_variances"; and a
third or fourth input that disagrees with the two that determine the
model by more than 64 machine epsilons of its scale is refused with
reason "inconsistent_variances", with the
value they imply in the message. With diff_size a
within-cluster variance of 0 is refused (reason
"within_variance_zero"), where the budget planners reported
a width of NaN and the others stopped with an error from R.
Each refusal has class dmar_argument_invalid, carries the
four inputs as given in supplied, and says what to give. In
DMAR 1.0.0 and the development versions of 1.1.0
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = -.2, true_var_within = .8)
stopped with “missing value where TRUE/FALSE needed” after ten warnings,
as did true_total_var = 1, true_icc = NA without the
warnings; true_var_between = 0, true_var_within = 0
returned 3 clusters with an expected width of 0; and
true_total_var = 1, true_icc = .2, true_var_between = .5
planned 36 clusters with a between variance of .5 and a within variance
of .8, where the pair (total 1, ICC .2) gives 18; it is now refused with
“true_total_var = 1 and true_icc = 0.2 imply true_var_between = 0.2, but
0.5 was given”. A rounded redundant value is refused the same way:
true_var_between = 1, true_var_within = 2, true_icc = .333,
which DMAR 1.0.0 and the development versions planned from the two
variances (71 clusters), says that the variances imply
true_icc = 0.333333333333333; give two of the
four.
G above 999,999 is refused by the six
ss_aipe_crd_es_*() planners. Each simulated data
set takes its own seed, drawn without replacement from 1 to 999,999.
ss_aipe_crd_es_n_clusters_fixed_budget(budget = 1000, cluster_size = 10, cluster_cost = 0, individual_cost = 1, G = 1000000, pr_treat = .5, true_icc = .2, true_es = .5)
stopped with “cannot take a sample larger than the population when
‘replace = FALSE’” in DMAR 1.0.0 (nrep) and the development
versions of 1.1.0; it is refused on entry with class
dmar_count_invalid, max 999999, and a message
that says why. The draw of seeds is unchanged, so every G
that was accepted gives the same seeds and the same result.
The cluster randomized planners check
assurance, and the standardized ones width,
conf_level, and the covariate’s intraclass correlation, on
entry. ss_aipe_crd_es_n_clusters_fixed_width() and
ss_aipe_crd_es_both_fixed_width() check width,
conf_level, and assurance as the other
planners do:
ss_aipe_crd_es_n_clusters_fixed_width(width = -1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, G = 2)
searched without end in DMAR 1.0.0 and the development versions of
1.1.0. ss_aipe_crd_n_clusters_fixed_budget(),
ss_aipe_crd_es_n_clusters_fixed_budget(), and
ss_aipe_crd_es_n_individuals_fixed_budget() refuse an
assurance that is not NULL or a single number
strictly between 0 and 1 (class dmar_assurance_invalid),
whether or not a width is computed, as the other planners do. In DMAR
1.0.0 and the development versions
ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8, assurance = 1.5)
returned 33 clusters with an assured width of NaN, and
without the population values and with assurance = 2 33
clusters and a budget of 990; the two standardized planners with
G = 2 and assurance = 2 stopped with “‘probs’
outside [0,1]”, and without G returned the size the budget
buys (50 clusters of 10 at cluster_cost = 10). The four
standardized planners that take a width or search over both sizes check
true_icc_covariate (a single number in [0, 1]) on entry, as
the two budget planners already did with G:
ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, n_covariates = 1, true_icc_covariate = 1.5, true_R2_within = .1, true_R2_between = .1, G = 2, seed = 113)
stopped with “‘data’ must be of a vector type, was ‘NULL’” in DMAR 1.0.0
and, after simulating three designs, with class
dmar_computation_failed (reason
"no_finite_width") in the development versions; it is
refused with class dmar_argument_invalid and
arg "true_icc_covariate".
The cluster randomized planners refuse a
diff_size, true_icc, or es_type
that makes the width not a number, and a target that no design of up to
1,000,000 clusters meets. diff_size must be
NULL or a vector of one or more finite numbers, in all
twelve planners. Wherever a width is simulated, the six
ss_aipe_crd_es_*() planners check es_type (0,
1, or 2) and true_icc, a single number in [0, 1), and above
0 with es_type = 2: at 1 no variance lies within clusters
and the model fitted to each simulated data set has none to estimate,
and with es_type = 2 an intraclass correlation of 0 leaves
the effect without a scale. Each refusal has class
dmar_argument_invalid. In DMAR 1.0.0 and the development
versions of 1.1.0
ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = 1, pr_treat = .5, G = 2, seed = 113)
stopped with “missing value where TRUE/FALSE needed”, as it did with
true_icc = 1.5, true_icc = -.2, and
true_icc = 0, es_type = 2; with
true_icc = 1, es_type = 0 it stopped with that message in
1.0.0, and the development versions simulated three designs at which
every fit failed and stopped with class
dmar_computation_failed.
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8)
stopped with “missing value where TRUE/FALSE needed” with
diff_size = numeric(0), c(Inf, 0), or
NA, and with “non-numeric argument to mathematical
function” with diff_size = "a". The searches over the
number of clusters consider at most 1,000,000 clusters, the bound the
power planners use for a sample size, and a target that no design within
it meets is refused with arg "width",
reason "n_clusters_beyond_reach", and the
width at the bound as width_at_max: in DMAR 1.0.0 and the
development versions the same call searched without end with
width = 1e-300 and returned 1,639,025 clusters with
width = .003, and
ss_aipe_crd_both_fixed_width(width = .003, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8)
returned 1,391,996 clusters of 209. A target met within the bound is
planned as before (width = .004: 921,953 clusters). The
standardized planners start from this search and refuse such a target
before anything is simulated. A width that is not a number at a design a
search compares stops it with class dmar_computation_failed
and reason "width_not_a_number"; the checks on
entry leave no call that reaches it.
The twelve cluster randomized planners refuse a nonzero
true_R2_between with
n_covariates = 0. true_R2_between is
the share of the between-cluster variance of the outcome that the
cluster-level covariates explain, and n_covariates = 0 says
there are none. Such a call is refused on entry, before any search or
simulation, in all twelve planners, including the budget planners that
compute no width. The refusal has class
dmar_argument_invalid, arg
"true_R2_between", and reason
"R2_between_without_covariates", and its message asks for
n_covariates or true_R2_between = 0.
true_R2_within may still be nonzero without a cluster-level
covariate. The plans these calls returned are under Changes to Earlier
Results.
Summary statistics given with a fitted model are
refused. Through 1.0.x
eta_squared(fit, F_value = 3) and
ci_eta_squared(fit, N = 50) ignored the summary statistics;
eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared() now
stop and say that a fitted model and summary statistics were both given.
An infinite F_value is refused on entry by all four;
ci_eta_squared() and ci_omega_squared() let it
reach the noncentral root search, which failed.
ci_mahalanobis() refuses a name in
... that its root search does not take.
tol, maxiter, check.conv, and
trace are passed to the search of ci_nc_F();
any other name, and an argument with no name, is refused before anything
is computed, with a condition of class
dmar_argument_invalid whose arg names it. DMAR
1.0.x passed every name on, so most stopped inside the search with a
message about the search, and uniroot()’s
lower, upper, f.lower, and
f.upper replaced the bracket the search had set.
ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() refuse
noncentral = TRUE when the sample size is planned from the
supposed values and the supposed variances of Y and of the
targeted predictor are not both 1, with a condition of class
dmar_argument_invalid whose message names
ss_aipe_src_sensitivity(): the noncentral plan of
ss_aipe_reg_coef() is for a standardized coefficient, and
these two report the unstandardized one. With unit variances, or with
specified_N, the call runs.
bifactor_indices() refuses a multiple-group
fit, with the error average_variance_extracted()
gives such a fit (class dmar_argument_invalid, the groups
carried as groups): the factors have indices in each group,
those of the model fitted to that group’s data alone.
ci_bifactor_indices() already refused such a fit.
ss_aipe_cv() refuses a plan that cannot
exist. With alpha_upper = 0 the interval is a
lower confidence bound whose upper limit is infinite at every sample
size; in DMAR 1.0.0 the search ran its cap of 100,000 steps (about a
minute) before stopping with the error “The sample size search did not
converge within 100000 iterations”. Such a call is now refused at once
with a condition of class dmar_argument_invalid naming
alpha_upper. With alpha_lower = 0, an upper
confidence bound, DMAR 1.0.0 returned no plan for any width:
ci_cv() gave that interval a lower limit of
-Inf, so its width was infinite at every sample size and
the search stopped at the same cap. The lower limit is now 0, and the
width is the upper bound itself. With alpha_upper below .5
the bound stays above true_cv at every sample size the
search can reach, so a width at or below
true_cv is refused the same way, naming width
and alpha_lower. A wider target is planned without
assurance (true_cv = .2,
width = .25, conf_level = NULL,
alpha_lower = 0, and alpha_upper = .05 give
37) and is refused with it (see the entry below on
assurance together with
alpha_lower = 0).
ss_aipe_sc() and
ss_aipe_sc_sensitivity() take the contrast weights
ci_sc() takes: weights that sum to zero and whose
positive values sum to 1, with ci_sc()’s messages and class
dmar_argument_invalid.
ss_aipe_sc(true_std_contrast = .5, c_weights = c(.25, .25, -.5), width = .4)
returned 41 for a contrast ci_sc() refuses to estimate.
Integer weights such as -1:1 are accepted, as
before.
ss_aipe_sc_ancova() refuses unequal
alpha_lower and alpha_upper (class
dmar_argument_invalid, naming both). It plans for the
interval of ci_sc_ancova(), which has equal tails only, and
with unequal tails planned for an interval no DMAR function reports
(alpha_lower = .01 and alpha_upper = .04
returned 215 at true_std_contrast = .5,
c_weights = c(1, -1), and
width = .4).
ss_aipe_sc_ancova(divisor = "s_anova")
refuses a ratio of the ANCOVA to the ANOVA error standard deviation
outside (0, 1], which no population has (class
dmar_argument_invalid): true_cor = 1 stopped
with “missing value where TRUE/FALSE needed”, and
true_sd_anova = 1 with true_sd_ancova = 2
returned 508 at true_contrast = 1 and
width = .5.
Every function that takes contrast weights refuses
weights that are all zero (class
dmar_argument_invalid; arg is
"c_weights", or the name the function gives the weights:
"contrast" in contrast_adjusted(),
"weights" in meta_contrast(), and
"contrasts" in ci_scheffe(),
contrast_test(), and is_orthogonal_set()).
Such weights pass the rules that the weights sum to zero and that the
positive ones sum to at most 1, and they define no contrast. The message
is “‘c_weights’ has no nonzero weight, so it defines no contrast; give
weights such as c(1, -1) or c(1/2, 1/2, -1).”, with the function’s own
name for the weights in place of ‘c_weights’ (“‘contrast’ has no nonzero
weight, …” in contrast_adjusted()). Where the weights are a
set of contrasts (the rows of a matrix c_weights in
ci_c_ancova_bp(), the rows of contrasts in
contrast_test(), the columns of contrasts in
ci_scheffe() and is_orthogonal_set()), a set
with one such contrast is refused, and the message names it (“Row 2 of
‘c_weights’ has no nonzero weight”, “The contrast ‘none’ in ‘contrasts’
has no nonzero weight”). The functions are ci_c(),
ci_c_ancova(), ci_c_ancova_bp(),
ci_sc(), ci_sc_ancova(),
ci_scheffe(), contrast_adjusted(),
contrast_test(), equivalence_c(),
is_orthogonal_set(), meta_contrast(),
power_equivalence_c(), ss_aipe_c(),
ss_aipe_c_ancova(),
ss_aipe_c_ancova_sensitivity(),
ss_aipe_c_sensitivity(), ss_aipe_sc(),
ss_aipe_sc_ancova(),
ss_aipe_sc_ancova_sensitivity(),
ss_aipe_sc_sensitivity(), ss_power_c(),
ss_power_c_ancova(), ss_power_contrast(),
ss_power_equivalence_c(), ss_power_sc(),
ss_seq_c(), and ss_seq_c_sensitivity(). With
c_weights = c(0, 0) (or three zeros where the design has
three groups), DMAR 1.0.0:
ci_c(means = c(1, 2), s_anova = 1, c_weights = c(0, 0), n_per_group = c(20, 20)),
from ci_c_ancova() with either procedure, from
ci_c_ancova_bp(), and from
contrast_adjusted(), ci_scheffe(), and
contrast_test(), and [.5, .5] from
ci_c(contrast = .5, s_anova = 1, c_weights = c(0, 0), n_per_group = c(20, 20))
and from ci_c_ancova() with contrast = .5;
ci_scheffe() and contrast_test() reported a
standard error of 0 beside the interval, and
contrast_adjusted(), ci_scheffe(), and
contrast_test() a test statistic and p-value of
NaN;ss_aipe_c(true_var_error = 1, c_weights = c(0, 0), width = .5),
with or without assurance = .9, and from
ss_aipe_c_ancova(), and a summary of zeros (every simulated
contrast and width 0, pct_ci_less_width 1) from
ss_aipe_c_sensitivity() and
ss_aipe_c_ancova_sensitivity();Inf from
ss_power_c(true_contrast = .5, c_weights = c(0, 0), true_sd = 1),
ss_power_c_ancova(), and ss_power_sc(), and a
power of 1 with specified_n_per_group = 10;all_orthogonal 1 from
is_orthogonal_set(cbind(c(1, -1, 0), c(0, 0, 0)));ss_aipe_sc(), ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova(), ci_sc_ancova(), and
ss_aipe_sc_ancova_sensitivity() with
supposed_std_contrast, and with “object ‘k’ not found” in
ss_aipe_sc_ancova_sensitivity() with
specified_n_per_group;ci_sc() (“Please use fractions to specify the contrast
weights”), ss_power_contrast(),
equivalence_c(), power_equivalence_c(),
ss_power_equivalence_c(), ss_seq_c(), and
ss_seq_c_sensitivity(), and with “All contrast weights are
zero after centering.” in meta_contrast().Empty weights, c_weights = numeric(0), have no nonzero
weight either, and they get the same refusal wherever no earlier check
of their length refuses them. DMAR 1.0.0 returned the limits
NaN around the contrast .5 from
ci_c(contrast = .5, s_anova = 1, c_weights = numeric(0), n_per_group = 20)
and from
ci_c_ancova(contrast = .5, s_ancova = .8, c_weights = numeric(0), n_per_group = 20, cov_means = numeric(0), SSwithin_x = 40),
the limits NaN around a contrast of 0 with
means = numeric(0) or adj_means = numeric(0)
in place of contrast, and [.5, .5] from ci_c()
with N = 40 added. It stopped with “missing value where
TRUE/FALSE needed” in
ss_aipe_c(true_var_error = 1, c_weights = numeric(0), width = .5),
ss_aipe_c_ancova(), ss_aipe_c_sensitivity(),
ss_aipe_c_ancova_sensitivity() (with empty
true_mean_Y), ss_aipe_sc_ancova(), and
ss_aipe_sc_ancova_sensitivity(), and a matrix that holds no
contrast stopped with a message from R (“arguments imply differing
number of rows: 1, 0” from
ci_c_ancova_bp(adj_means = c(1, 2), s_ancova = .8, c_weights = matrix(numeric(0), 0, 2), n_per_group = 20),
and “length of ‘dimnames’ [2] not equal to array extent” from
ci_scheffe() with
contrasts = matrix(numeric(0), 3, 0)), as did a set that
holds no contrast in contrast_test() (“length of ‘dimnames’
[1] not equal to array extent” with
contrasts = matrix(numeric(0), 0, 3), and “argument must be
coercible to non-negative integer” with
contrasts = list()). In the other functions DMAR 1.0.0
stopped on empty weights with a message about the length or form of the
weights, the number of groups or contrasts, the positive weights, or the
degrees of freedom; where a check of the length comes first (as in
ss_power_c(), whose weights must have length 2 or more),
DMAR 1.1.0 refuses them with that check’s message.
Logical weights that are all FALSE, which are zeros when
R computes with them, are refused too: with the message above, or, in
the functions that require numeric weights, with that rule’s message.
Those functions refused them with the same message in DMAR 1.0.0,
without a class, except ci_sc(), ss_aipe_sc(),
and ss_aipe_sc_sensitivity(), which require numeric weights
from this release on: there DMAR 1.0.0 stopped with “Please use
fractions to specify the contrast weights” (ci_sc()) and
“missing value where TRUE/FALSE needed” (the two planners). DMAR 1.0.0
returned [0, 0] from
ci_c(means = c(1, 2), s_anova = 1, c_weights = c(FALSE, FALSE), n_per_group = 20)
and a plan of 2 per group from
ss_aipe_c(true_var_error = 1, c_weights = c(FALSE, FALSE), width = .5).
Weights in which every contrast has a nonzero value are accepted or refused as before, and the results for them are unchanged.
ss_aipe_sc() and
ss_aipe_sc_ancova() refuse a width that needs
more than 2^53 per group (about 9.0e15, the largest number up
to which R holds every whole number exactly; class
dmar_argument_invalid, naming width). Past it
neighboring sample sizes can be the same number in R, so no size there
is the smallest that reaches width. DMAR 1.0.0 returned
such numbers at a standardized contrast of 0:
ss_aipe_sc(true_std_contrast = 0, c_weights = c(1, -1), width = 5e-8)
(psi_standardized in 1.0.x) gave 12,292,668,226,221,188,
and width = 1e-9 gave 30,731,670,565,552,979,968. Those
calls are now refused, both when the start of the search is past 2^53
and when the search passes it (as at true_std_contrast = .1
with a start of 9,006,298,534,815,513 per group). Below the limit the
plans are returned at every contrast:
ss_aipe_sc(true_std_contrast = 2, c_weights = c(1, -1), width = 1e-6)
gives 46,097,505,846,480, and true_std_contrast = .1 with
width = 5.9e-8 gives 8,839,438,422,537,546; DMAR 1.0.0
raised the sample size by one from a start about a third below the first
and did not return either within 60 seconds. Past about 1e11 per group
the plans rest on the precision of doubles (Details of
?ss_aipe_sc and ?ss_aipe_sc_ancova give the
sizes). The smallest sample sizes here are for width and
the tail areas as R holds them; with the default
conf_level = .95 each tail area, (1 - .95) / 2, is
0.025000000000000022 in doubles. At a contrast of 0, where the width is
computed from the normal quantiles, width = 6e-7 gives
85,365,751,570,981, the smallest sample size that reaches it, where DMAR
1.0.0 gave 85,365,751,570,976; width = 1e-6 gives
30,731,670,565,553, also the smallest, where DMAR 1.0.0 gave
30,731,670,565,548; and width = 1e-7 gives
3,073,167,056,555,298, 2 below the smallest, where DMAR 1.0.0 gave
3,073,167,056,555,294. At other contrasts the width is the difference of
two limits near the noncentrality: the plan above at a contrast of 2 and
width = 1e-6 is 1,850 below the smallest,
46,097,505,848,330 (see Bug Fixes for the search, and Changes to Earlier
Results for the noncentral t distribution at many degrees of
freedom).
ss_aipe_reg_coef(),
ss_aipe_rc(), and ss_aipe_src() refuse a tail
probability of 0 or above .5, and two tails of .5, with a
condition of class dmar_argument_invalid naming
alpha_lower and alpha_upper. With a tail of 0
the interval has a limit at infinity and is never as narrow as
width, and ci_reg_coef(), which reports the
interval, takes no tail above .5. DMAR 1.0.x planned such a call as if
both tails were their average: for two predictors correlated .3,
correlated .4 and .3 with Y, and a width of .3,
alpha_lower = 0 with alpha_upper = .05 gave
157, the plan for tails of .025, and the noncentral search
(noncentral = TRUE, the default of
ss_aipe_src()) did not end; alpha_lower = .6
with alpha_upper = .01 gave 14; and two tails of .5 gave 5
(2p + 1), or 3 with assurance = .9, which leaves
no residual degrees of freedom.
A nonzero tail area below 1e-9 is refused by
ci_nc_t() and by the intervals and plans of a regression
coefficient, a contrast, and a correlation.
ci_reg_coef(), ci_rc(), ci_src(),
ci_c(), ci_sc(), ci_sc_ancova(),
ci_r(), ci_sm(), ci_cv(),
ss_aipe_reg_coef(), ss_aipe_rc(),
ss_aipe_src(), ss_aipe_c(),
ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_r(), ss_aipe_r_sensitivity(), and
ss_aipe_cv() check the floor once their tails are resolved,
and ci_nc_t() checks it for every interval and plan whose
limits come from it, among them ci_smd(),
ci_smd_c(), smd_trimmed(),
ss_aipe_smd(), ss_aipe_sm(), and the
sensitivity functions of these planners. Functions that form one of
these intervals at their own conf_level refuse the same
levels: cles(), proportion_of_superiority(),
nnt_from_smd(), and plot_smd() through
ci_smd(), ss_aipe_c_sensitivity() through
ci_c(), and
ss_aipe_equivalence_r_sensitivity() through
ci_r(). A tail of 0, a one-sided interval, is accepted
wherever it was. Below 1e-9 the noncentral t limits lose
accuracy: the tail probability a limit is found from carries an absolute
error of up to about 1e-12, a growing share of a smaller tail. The
central intervals of a contrast, a correlation, and a regression
coefficient and the plans for them keep the same floor, so that an
interval and the plan for it accept the same tail areas. DMAR 1.0.0 and
the development versions of 1.1.0 returned what they computed.
ci_reg_coef(reg_coef_j = .5, se_reg_coef_j = .1, N = 50, p = 2, conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20, noncentral = TRUE)
(b_j and SE_b_j in 1.0.0) gave an upper limit
of 1.549 in 1.0.0 and 1.435 in the development versions, where the limit
is 1.554. With noncentral = FALSE the upper limit was
Inf, as it was for
ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20)
(n in 1.0.0).
ci_r(r = .3, N = 50, conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20)
(n in 1.0.0) gave an upper limit of 1, and -1 for the lower
limit with the tails exchanged. ci_sc() with the means and
tails of that ci_c() call gave an upper limit of 4.126 in
1.0.0 and 3.782 in the development versions.
ss_aipe_reg_coef(true_R2_Y_X = .1, true_R2_j_X_without_j = .2, p = 2, true_reg_coef_j = .1, width = .4, conf_level = NULL, alpha_lower = 1e-20, alpha_upper = .05)
(rho2_Y_X, rho2_j_X_without_j, and
b_j in 1.0.0) returned 114 in 1.0.0, the plan for two tails
of .025, and stopped in the development versions with “missing value
where TRUE/FALSE needed”.
ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .5, conf_level = NULL, alpha_lower = 1e-20, alpha_upper = .025),
and ss_aipe_sc_ancova() with the same contrast and width
and both tails 1e-20 (psi_standardized in 1.0.0 for both),
stopped with the same message in 1.0.0 and refused ‘width’ (“needs more
than 2^53 per group”) in the development versions. A level is held to
the floor through its tails, (1 - conf_level) / 2 each, so
conf_level = 1 - 1e-12 is refused by every function above,
and .999999998 is the largest level accepted. At
conf_level = 1 - 1e-12, ss_aipe_reg_coef()
with the planning values above returned 1,459 in 1.0.0 and the
development versions,
ss_aipe_c(true_var_error = 1, c_weights = c(1, -1), width = .5)
(error_variance in 1.0.0) returned 1,640,
ss_aipe_r(true_cor = .3, width = .2) (rho in
1.0.0) returned 4,189, and
ci_smd(smd = .5, n_1 = 50, n_2 = 50) returned [-0.953,
1.948]. At .999999998 the floor leaves these plans as they were;
ss_aipe_reg_coef() gives 1,032, where 1.0.0 gave 1,033,
through the change in its plans without assurance (see
Changes to Earlier Results). The refusal has class
dmar_argument_invalid, reason
"tail_below_floor", arg naming the tail or
tails below the floor (conf_level when the tails came from
it), value, and floor (1e-9). Its message
reads, for example, “‘alpha_upper’ is 1e-20, a tail area above 0 and
below 1e-9, the smallest nonzero tail area accepted. Below 1e-9 the
noncentral t limits lose accuracy (…), so a limit or a sample size could
be off without notice. The central intervals of a contrast, a
correlation, and a regression coefficient, and their plans, keep the
same floor, so that an interval and the plan for it accept the same tail
areas. Give a tail area of at least 1e-9, or 0 for a one-sided
interval.” In ss_aipe_reg_coef(),
ss_aipe_sc(), and ss_aipe_sc_ancova(), which
refuse a tail of 0, and ss_aipe_cv(), which refuses one as
alpha_upper, it ends “Give a tail area of at least 1e-9.”
For a level the message reads “‘conf_level’ of 0.999999999999 leaves a
tail area of 5e-13 on each side, below 1e-9, … Give a conf_level of at
most 0.999999998.” At the floor, the noncentral limits in six cases
(t from -3 to 10, 10 to 1,000 degrees of freedom) at 66 tail
areas from 1e-6 down to 1e-9 were within 1.4e-4 of the exact ones
(6.0e-6 of the width of the interval). “Small tail areas” in
?ci_nc_t gives the numbers.
An assurance above .999999998 is refused by
ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_sm(), and ss_aipe_smd(), and by their
sensitivity functions when they plan with it. The published
assurance step with which DMAR 1.0.x planned them finds noncentral
t limits with tail areas of (1 - assurance) / 2 and 1 -
assurance (in ss_aipe_smd() through ci_smd()),
which then fall below the floor of 1e-9 of the entry above.
ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .5, assurance = 1 - 1e-10)
(psi_standardized in 1.0.0) returned 151 per group in DMAR
1.0.0 and the development versions of 1.1.0, as did
ss_aipe_sc_ancova() at the same values and
ss_aipe_smd(true_smd = .5, width = .5, assurance = 1 - 1e-10)
(delta in 1.0.0), and
ss_aipe_sm(true_sm = .5, width = .5, assurance = 1 - 1e-10)
(sm in 1.0.0) returned 116. Each came from limits below the
floor. The refusal names assurance, not the tails of the
step, which the caller did not give (class
dmar_argument_invalid, reason
"tail_below_floor"): “‘assurance’ of 0.9999999999 is too
close to 1: the published assurance step, with which DMAR 1.0.x planned,
finds noncentral t limits with tail areas of (1 - assurance) / 2 and 1 -
assurance, and (1 - assurance) / 2 is below 1e-9, the smallest nonzero
tail area accepted, below which those limits lose accuracy. Give an
assurance of at most 0.999999998.” The check is made whatever
assurance_method is, so whether a call is refused does not
depend on it.
ss_aipe_reg_coef(),
ss_aipe_rc(), and ss_aipe_src() refuse an
abbreviation of "Lower" or "Upper", as they
refuse those values. They plan only for the full width, but
DMAR 1.0.0 checked which_width with
char.expand() and then tested the value as given, so
"L", "Lo", "U",
"Up", and "" were planned for the full width.
In DMAR 1.0.0,
ss_aipe_reg_coef(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 5, true_reg_coef_j = .25, width = .15, which_width = "L")
(rho2_Y_X, rho2_j_X_without_j, and
b_j in 1.0.x) and ss_aipe_rc() with the same
arguments returned 607, the plan of "Full", and
ss_aipe_src(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 5, true_std_reg_coef_j = .25, width = .15, which_width = "L")
(Rho2_j_X_without_j and beta_j in 1.0.x)
returned 626, also the plan of "Full". These three calls
now stop with a refusal of class dmar_argument_invalid
(arg = "which_width") whose message says that
"L" stands for "Lower". "F" and
"Fu" still give the plan of "Full": 606 from
ss_aipe_reg_coef() and ss_aipe_rc(), 625 from
ss_aipe_src() (a case below 1.0.0; see Changes to Earlier
Results).
ss_aipe_R2(true_R2 = .5, width = .15, which_width = "", p = 5, random_predictors = FALSE)
is refused the same way; DMAR 1.0.0 returned NULL for it
(population_R2 in 1.0.x).
mr_cv() and mr_smd() check
every argument. Each refusal names the argument and has class
dmar_argument_invalid, or dmar_count_invalid
for m0 and n_per_group. In DMAR 1.0.x:
mr_cv() given scores with a negative mean returned the
risk and the decision of the same scores with their signs reversed,
beside a negative coefficient of variation; a data frame stopped with
“is.atomic(x) is not TRUE”; and three observations returned a risk of
NaN and a decision of NA. data
must be a numeric vector of at least 4 finite observations with a
positive mean.sampling_cost of -75 gave a negative criterion and a
stopping rule that was met, and a sampling_cost of 0, an
epsilon of 0, or a gamma of -1 gave a pilot
sample size of Inf. sampling_cost,
A, structural_cost, and epsilon
must each be a single positive finite number, and a call without
sampling_cost is refused by name.gamma = .5 was accepted. gamma must be
strictly between 0 and 1/2, the interval on which the stopping rule is
defined (Chattopadhyay & Kelley, 2016, Equation 23).mr_cv() returned an m0 of 4.5 or of 2 as
the pilot sample size. m0 must be a whole number of at
least 4 in both functions.pilot = 1 and pilot = 0 were read as
TRUE and FALSE, and pilot = "yes"
stopped with “object ‘Outcome’ not found”. pilot and
verbose must be TRUE or
FALSE.mr_smd() given a sample size per group of -5
(n in 1.0.x) returned a negative risk and a decision of
NA, accepted 74.5, and stopped with “arguments imply
differing number of rows” for a vector there or in d.
d must be a single finite number and
n_per_group a whole number of at least 2.mr_cv(pilot = TRUE, A = 1e300, sampling_cost = 1e-10)
returned a pilot sample size of Inf. A divided
by sampling_cost must be a finite number in both
functions.ss_aipe_cv() refuses assurance
together with alpha_lower = 0, a one-sided upper
confidence bound, with a condition of class
dmar_argument_invalid that names both arguments and advises
planning without assurance. The assurance step of Kelley
(2007) was written for a two-sided interval and does not give the
requested probability for a bound; the numbers are on
?ss_aipe_cv under “Edge cases”. DMAR 1.0.0 returned no plan
for a call with alpha_lower = 0 (see the entry above on a
plan that cannot exist).
ss_aipe_cv() refuses a planning value or a
tail area that is not one admissible number, each by name
(class dmar_argument_invalid). With
conf_level = NULL, a tail area that is left out,
NA, or a vector stopped in DMAR 1.0.0 with an error of R’s
own (“missing value where TRUE/FALSE needed” for a tail left out or
NA); the refusal now says that each of
alpha_lower and alpha_upper is a single number
at least 0 and below 1. A vector true_cv
(C_of_V in 1.0.x) stopped with “the condition has length
> 1”. true_mean and true_sd
(mu and sigma in 1.0.x) must each be positive:
DMAR 1.0.0 planned a negative mean with a negative standard deviation as
their positive ratio.
ss_aipe_cv_sensitivity() checks every input
that defines its simulation. DMAR 1.0.0 refused a
true_cv that was left out or not positive and an
estimated_cv (now supposed_cv) that was not
positive. It did not check that either was one finite number, and it did
not check mean (now true_mean) or, when
specified_N was given, specified_N,
width, or assurance. true_cv,
supposed_cv, and true_mean are now each a
single finite positive number; specified_N is a whole
number of at least 4, the smallest sample size ss_aipe_cv()
plans (class dmar_count_invalid), so that a character
string does not turn the value column into text and a
fraction is not echoed as a sample size; width, when given,
is a single finite positive number, also with specified_N;
and assurance is checked on entry (class
dmar_assurance_invalid), also with
specified_N, where it is only echoed.
var_cv() refuses an NA or
infinite true_cv or N. In DMAR 1.0.x,
where the arguments were cv and n, a numeric
NA in either (NA_real_, as a computation
returns it) stopped with R’s “missing value where TRUE/FALSE needed”, an
infinite coefficient returned Inf in both rows, and an
infinite sample size returned 0 and NaN. Each of these
calls now stops with a refusal of class
dmar_argument_invalid that names the argument and states
the value given.
ss_aipe_R2() refuses assurance
with a half-width. With which_width = "Lower" or
"Upper" the function plans for the distance from the
expected sample value to one limit, and assurance given
with either stops with a refusal of class
dmar_argument_invalid
(arg = c("assurance", "which_width")). In DMAR 1.0.0 every
such call with "Upper" stopped with the R error “unused
argument (random_predictors = random_predictors)”, and with
"Lower" the function returned a sample size that did not
deliver the probability asked for.
ss_aipe_R2(true_R2 = .30, width = .10, p = 3, which_width = "Lower", assurance = .99)
(population_R2 in 1.0.x) returned 253, the plan without
assurance, and at 253 the distance from the sample squared
multiple correlation to its lower limit was no larger than .10 in 52
percent of 20,000 simulated samples. With true_R2 = .10 and
width = .05 it returned 486 for an assurance of .80, again
the plan without assurance (52 percent), and 637 for .99
(95 percent); with true_R2 = .70 and
width = .10 it returned 153 for .80 (72 percent).
ss_aipe_R2() and
ss_aipe_R2_sensitivity() refuse a call without
width.
ss_aipe_R2(true_R2 = .5, p = 5) (population_R2
in 1.0.x) stopped in DMAR 1.0.0 with R’s “missing value where TRUE/FALSE
needed” and now stops with a refusal of class
dmar_argument_invalid that names width
(arg = "width"), as a call without p or
true_R2 is refused in the name of the argument left out.
ss_aipe_R2_sensitivity() (where width was
w in 1.0.x) stopped with the same error of R’s when
w or true_R2 was left out, and with “invalid
‘times’ argument” when p was left out beside
specified_N. Each of true_R2,
width, and p left out is now refused in its
name, with supposed_R2 and with
specified_N.
A condition with a class can be caught or muffled by name, whatever
its wording. Every error and every warning DMAR signals on
purpose now has a class. Every error is a
dmar_refusal and carries a code, so
tryCatch(expr, dmar_refusal = function(e) ...) catches
whatever DMAR refused, and an error without that class came from R, from
another package, or from match.arg(). The check of an
argument (its type, length, or range, arguments that cannot be given
together, a column that is not in the data, a model the function does
not cover) is a dmar_argument_invalid, which carries the
names of the arguments at issue as arg (NULL
where the refusal is not about a named argument, and in some checks
whose message names the argument only in running text). Three more
general classes cover the errors that have no class of their own:
dmar_package_required (a package the analysis needs is not
installed; field package),
dmar_computation_failed (the arguments were accepted and a
search, a fit, a refit, or a bootstrap made with them gave no result),
and dmar_internal_error (a state DMAR’s own code should not
reach). Two cover the warnings that have none:
dmar_result_qualified (the result is returned with
something to know before using it) and
dmar_argument_ignored (an argument given has no effect on
the call). The more specific classes (dmar_not_converged,
dmar_no_maximum, dmar_lavaan_error,
dmar_inadmissible, and the others the sections below name)
are dmar_refusals as well;
dmar_bootstrap_cancelled and
dmar_progress_error, which come from the caller’s
progress function, are not. In 1.0.0 nearly every error was
a plain error and no error was a dmar_refusal. Messages are
worded as before, except that some of the oldest checks now name the
argument and state the rule (for example “‘c_weights’ must sum to zero.”
in place of “The sum of the coefficients must be zero”). The help of
dmar_metadata() lists the classes.
ss_aipe_cv() and ss_aipe_sm() refuse an
assurance that is not NULL or a single number
strictly between 0 and 1 with dmar_assurance_invalid (the
condition carries the value given; the message says why neither end is
accepted). dmar_argument_invalid, which carries the name or
names at issue as arg, is the class of the refusals of
ss_power_R2_change() for a p_restricted not
below p_full, for squared multiple correlations that cannot
belong to nested models, and for an effect stated in more than one way;
of the partial and semipartial correlation functions for a
p below 2; and of smd() and
smd_c() for unbiased = TRUE without the sample
sizes. One warning joins a class its siblings already gave:
ss_aipe_reliability_sensitivity() reports replications
without an interval with dmar_replications_without_interval
(fields n_without_interval and G). The count
of noncentral F lower-limit clamps that
ss_aipe_R2() reports after its search has the new class
dmar_nc_F_clamp_search, with the field
n_evaluations. The wording of each message is unchanged,
except that the refusals of smd() and smd_c()
now also say why the sizes are needed. The warnings about the accurate
range of the noncentral t distribution that
ci_nc_t() and the searches built on it gave in DMAR 1.0.x
are gone, since every limit is now computed exactly (see Exact
Noncentral t and Studentized Range Distributions).
which_width that is not a single character
string is refused with a class. In DMAR 1.0.0,
ss_aipe_reg_coef(), ss_aipe_rc(),
ss_aipe_src(), and ss_aipe_R2() with
which_width = 1 stopped with the error of base R “‘input’
and ‘target’ must be character vectors”, and with
which_width = c("Full", "Lower") with “‘input’ must have
length 1”. They now stop with dmar_argument_invalid
(arg = "which_width").Where a check’s message opens with the name of the argument and the
check did not store it, arg is read from the message, so
tryCatch(bayes_one_sample_t(mean = .5, sd = 1, N = 30, prior_location = NULL), dmar_refusal = function(e) e$arg)
returns "prior_location" where it returned
NULL.
ss_aipe_mixed_effects() plans for a predictor
at either level and for three ways of centering it, and
ss_aipe_mixed_effects_sensitivity() simulates what it
plans. In DMAR 1.0.x the planner was for an individual-level
(i.e., Level 1) predictor centered at its cluster mean, and the
sensitivity function generated a cluster-level (i.e., Level 2)
predictor, whose slope has a far larger variance; its intervals were
much wider than the plan and the comparison tested nothing. Both
functions now take predictor_level (1, the default, or 2),
centering ("cluster_mean", the default;
"grand_mean"; "none"), and the intraclass
correlation of an individual-level predictor (true_icc_X,
with supposed_icc_X for the planning value in the
sensitivity function), share one set of expressions, and form one
interval, the estimate plus and minus a t quantile times its
standard error, on K(m - 1) - 1 degrees of freedom for an
individual-level predictor and K - 2 for a cluster-level one.
true_total_var = 1,
true_var_X = 1, true_icc = .10,
width = .20, and clusters of 20; 88 against 70 with
clusters of 5), because it uses the t quantile and the expected
standard error of the interval computed from data, where 1.0.x used the
normal quantile and the total sample size.true_icc_X given for
one.expected_width, predictor_level,
true_icc_X, and supposed_icc_X rows.A minor release, and a large one. DMAR 1.0.1 was prepared but never submitted to CRAN; its changes are part of this release, and its section below is kept as the record of them.
Factor analysis gets its own names and new models.
fa_confirmatory() is the 1.0.0 cfa_k() and
fa_exploratory() its new exploratory partner, with
cfa_k() and plot_cfa_k() kept as aliases,
efa_k() a new alias of fa_exploratory(), and
the 1.0.0 result class kept after the new one;
fa_exploratory() fits incomplete items by FIML, gives
rotation-aware intervals, and rotates to a target;
factor_count() and parallel_analysis() compare
numbers of factors; fa_bifactor() and
fa_second_order() fit the bifactor and second-order models,
with delta method intervals for their indices;
factor_scores() gives per-case standard errors and the
determinacy of each factor, beside 2 rho^2 - 1, the lowest correlation
two equally consistent sets of scores can have; and
factor_congruence() gives Tucker’s congruence between two
solutions. Structural equation models get measures of fit, precision,
and comparison (sem_fit_measures(),
sem_residual_fit(),
sem_information_criteria(),
sem_exact_fit_bootstrap(),
equivalence_rmsea(), compare_sem()) and
sem_sam(), which estimates the structural relations after
the measurement models.
A missing data family states what the missing values cost: the
fraction of missing information for every parameter of a FIML fit, the
patterns and coverage behind it, the maximum likelihood covariance
matrix of incomplete data, and small-sample degrees of freedom with a t
reference distribution for Wald intervals, taught in a new vignette.
Two-stage maximum likelihood is a missing data option for the CFA
family, measurement invariance, and coefficients alpha and omega, and
sample size planning for a reliability coefficient can allow for the
missing values it will be estimated from. The omega of
reliability_omega(),
reliability_omega_categorical(), and
bifactor_indices() can take either the observed or the
model implied variance as its denominator and says which it used;
categorical omega no longer needs mvtnorm, no longer drops a correlation
the data do not determine, and takes incomplete items; coefficient H is
computed from a fitted model; the bifactor indices get bootstrap
intervals; and the reliability bootstrap counts the fate of every
replication, runs on several processes, refuses a solution that leaves
no coefficient, and takes a progress callback that can cancel it.
New intervals, tests, and plans: Dunnett’s T3
(ci_dunnett_t3()); the difference between two proportions
measured on the same units, with tests of noninferiority and equivalence
(ci_proportion_paired(),
equivalence_proportion_paired()); Fieller’s confidence set
for the ratio of two means (ci_ratio_of_means(),
ci_ratio_of_means_paired()); nonsuperiority in
equivalence_c(); the marker variable adjustment for common
method variance with its interval and test
(common_method_adjusted_r()); planning the interval for a
single mean (ss_aipe_mean()) and the assurance of a
contrast’s interval when the errors are not normal
(true_kurtosis in ss_aipe_c()); a crossed
design of units and raters (ci_crossed(),
ss_aipe_crossed()); and two-phase sampling, a costly
measurement with an inexpensive proxy (ci_two_phase(),
ss_aipe_two_phase(), ss_aipe_two_phase_cost(),
ss_seq_two_phase()). A new vignette fits classical tests as
path models in lavaan, and a knitted result table now carries every line
the console prints under it. A result table prints its numbers to three
decimal places, where 1.0.x printed three significant figures (see
Changes to Earlier Results). eta_squared(),
omega_squared(), and their intervals report the total and
the partial value of each effect and take sums of squares from a
published table; ss_aipe_smd(), ss_aipe_sc(),
ss_aipe_sc_ancova(), ss_aipe_reg_coef(),
ss_aipe_rc(), and ss_aipe_src() can find the
sample size for assurance from the probability that the
interval is no wider than asked for, computed exactly or simulated from
the exact sampling distribution; ss_power_indirect_effect()
takes its power from the joint sampling distribution of the two path
tests, with the predictor measured or fixed by design; and the
noncentral t distribution and the studentized range are
computed exactly wherever R’s pt(), qt(),
dt(), qtukey(), and ptukey() are
not accurate.
Some defaults change results. DMAR 1.1.0 gives no warning when a
result or a default differs from 1.0.x; these notes describe every such
change. The changed defaults are described under Changes to Earlier
Results, each with the argument that restores the 1.0.x analysis where
there is one; ss_aipe_rmsea_sensitivity() also corrects how
it forms its intervals, which no argument restores.
fa_confirmatory() (with its alias cfa_k(), and
with cfa_1() and cfa_2()) fits raw data by
full information maximum likelihood and takes the standard errors that
match the estimator (Huber-White under MLR, where 1.0.x took the
standard ones), and reliability_omega(),
reliability_alpha(), and
measurement_invariance() fit incomplete data that way (as
does reliability_H(), which takes data for the
first time in this release); reliability_omega() forms the
default interval of the model implied omega on the logit scale; and
ci_rmsea(), ss_aipe_rmsea(), and
ss_aipe_rmsea_sensitivity() form and plan a 90 percent
RMSEA interval by default, where 1.0.x used 95 percent.
ss_aipe_semipartial_r() without the new
true_R2_full plans for the largest variance over every
full-model R2, which raises the sample size when true_r_sp
squared is above 1/2 (true_R2_full = true_r_sp^2 restores
the 1.0.x plan), and ss_aipe_partial_r_sensitivity() and
ss_aipe_semipartial_r_sensitivity() simulate one more
variable partialed out than 1.0.x did for the same model, which no
argument restores. ss_aipe_partial_r() and
ss_aipe_semipartial_r() find the sample size for
assurance from the sampling distribution of the estimate,
where 1.0.x multiplied by a chi square factor that does not apply to
these intervals, and ss_aipe_partial_r_sensitivity()
simulates the raw-scale interval it plans, where 1.0.x simulated
Fisher’s Z interval; no argument restores the first, and
fisher_z = TRUE gives the Fisher interval, planned and
simulated, for the second. Every function names the total sample size
N, the number of predictors p, a planner’s
planning value true_X, a sensitivity function’s planning
value supposed_X, and the desired width width,
and names an argument for the quantity it holds (true_cor,
true_contrast, true_reg_coef_j,
true_cov_matrix) rather than for a symbol; the 1.0.x names
are not accepted, eta_squared_partial(),
omega_squared_partial(), and
ci_eta_squared_partial() are removed, and the count of the
partial and semipartial correlation functions is the number of
predictors, one more than the 1.0.x count of variables partialed out
(see Breaking Changes in 1.1.0). vargha_delaney_A() refers
its test statistic to Student’s t with Satterthwaite degrees of
freedom, which gives a larger p-value than the standard normal
reference of 1.0.x (test = "z" restores it). With
assurance, ss_aipe_smd(),
ss_aipe_sc(), ss_aipe_rc(),
ss_aipe_reg_coef() with its default
noncentral = FALSE, and their sensitivity functions return
the smallest sample size, not below the plan without
assurance, at which the exact probability that the interval
is no wider than width reaches the request (the new
argument assurance_method, "analytic" by
default), where DMAR 1.0.x returned published approximations that could
attain more or less than the request. ss_aipe_sc_ancova(),
ss_aipe_reg_coef() with noncentral = TRUE,
ss_aipe_src(), and their sensitivity functions return the
published approximations by default, and
assurance_method = "simulation" returns the smallest sample
size at which a simulated estimate of that probability reaches the
request. The four regression and ANCOVA planners without
assurance return the smallest sample size that meets the
width, where 1.0.x could return a case or more above it (a few more per
group in ss_aipe_sc_ancova()), and the approximations,
which start from that plan, move with it. No argument restores the plans
of ss_aipe_r(), ss_aipe_smd(), and
ss_aipe_sc() with assurance, the central plans
of ss_aipe_reg_coef() and ss_aipe_rc() with
assurance or with unequal tails, the plans without
assurance of ss_aipe_reg_coef(),
ss_aipe_rc(), ss_aipe_src(), and
ss_aipe_sc_ancova(), which can be a case or more smaller,
and the approximations that start from them, the plans and true values
of ss_aipe_src_sensitivity() when the variances of
Y and of the targeted predictor differ, the plans of
ss_aipe_equivalence_smd() and its sensitivity function and
of the standardized cluster randomized planners, the power and plans of
ss_power_indirect_effect(), the rows of
var_alpha(), or the simulated contrasts of
ss_aipe_sc_sensitivity(), each corrected in this release.
fa_confirmatory() on complete data gives the same estimates
as 1.0.x, but under ML its standard errors come from the observed
information, the item intercepts join the table, and AIC and BIC count
them; and its omega, average variance extracted, and H rows get limits
formed on the logit scale, as reliability_H()’s already
were, as described under Exploratory Models and Structural Relations.
Its RMSEA interval stays at the 90 percent of 1.0.0, now with an
argument of its own, rmsea_conf_level: every RMSEA interval
in the package, and the SRMR interval of
sem_residual_fit(), defaults to 90 percent, whose limits
decide the tests of close and not close fit at .05, while the other
intervals keep 95 percent (see Every RMSEA Interval at 90 Percent by
Default).
RMSEA and SRMR intervals default to 90 percent, and every
output states the level. A 100(1 - 2 alpha) percent interval
for the RMSEA decides the one-sided tests of close and not close fit at
level alpha (MacCallum, Browne, & Sugawara, 1996, p. 136), and its
lower limit is 0 exactly when the test of exact fit does not reject at
alpha, so the limits of a 90 percent interval decide those tests at .05,
the level conventionally used with them and the default at which
ss_power_sem() plans them; sem_fit_measures()
reports the tests’ p-values beside the interval, and the 90 percent
limits agree with those p-values compared with .05. A 95 percent
interval decides them at .025. Every RMSEA interval in the package
therefore defaults to 90 percent, as lavaan’s does, and so does the SRMR
interval of sem_residual_fit(), whose test of close fit
pairs with its interval in the same way; the package’s other intervals
keep 95 percent. Where a function reports both kinds, the RMSEA interval
has an argument of its own. The functions and their arguments:
ci_rmsea(), ss_aipe_rmsea(), and
ss_aipe_rmsea_sensitivity():
conf_level = 0.90, where 1.0.x defaulted to 0.95.
conf_level = 0.95 restores the 1.0.x result of
ci_rmsea() and ss_aipe_rmsea(), and the 1.0.x
level of ss_aipe_rmsea_sensitivity(), whose intervals were
also corrected, with no argument to restore them (see Changes to Earlier
Results).sem_fit_measures(): conf_level = 0.90,
which also sets the gamma hat and adjusted gamma hat intervals, mapped
from the same noncentrality limits.sem_residual_fit(): conf_level = 0.90 for
the interval of the unbiased SRMR, lavaan’s default too.factor_count(): conf_level = 0.90, the
level of its RMSEA intervals and of nothing else; the Monte Carlo limits
of the parallel analysis reference in its parallel_analysis
attribute keep the 95 percent of parallel_analysis().fa_confirmatory() (and cfa_k()),
cfa_1(), cfa_2(), fa_bifactor(),
and fa_second_order(): the new
rmsea_conf_level = 0.90 sets the RMSEA interval of the fit
rows, which the rmsea_ci_level row and the
glance() column of that name record, while
conf_level = 0.95 sets the intervals of the parameters and
indices and no longer reaches the RMSEA interval. cfa_k(),
cfa_1(), and cfa_2() thereby report the RMSEA
interval of DMAR 1.0.0, which took lavaan’s 90 percent whatever
conf_level was; a development version of 1.1.0 made it
follow conf_level, and that change is withdrawn. In
fa_confirmatory() (and so cfa_k()),
cfa_1(), and cfa_2(), whose arguments DMAR
1.0.0 fixed in order, rmsea_conf_level comes after
..., beside wald_df, so a 1.0.0 call that
gives its arguments by position works as before and the new argument is
given by name.fa_exploratory() (and efa_k()): the new
rmsea_conf_level = 0.90 sets the RMSEA interval of the fit
line and of the fit_measures attribute, while
conf_level sets the intervals of the solution.
factor_count() passes its conf_level to
it.irt_grm(): the new
rmsea_conf_level = 0.90, passed to lavaan, sets the RMSEA
interval of the fit_measures attribute, which held lavaan’s
90 percent interval before with no way to choose it, and is recorded as
the rmsea_conf_level attribute.equivalence_rmsea() keeps alpha_level,
whose 1 - 2 alpha interval is 90 percent at its default of .05.For 95 percent intervals pass conf_level = 0.95 to
ci_rmsea(), ss_aipe_rmsea(),
ss_aipe_rmsea_sensitivity(),
sem_fit_measures(), sem_residual_fit(), and
factor_count(), and rmsea_conf_level = 0.95 to
the factor analysis functions and irt_grm(). The level is
stated wherever such an interval is shown: the line beneath a printed
table (“Confidence level: 90%”, and for the factor analysis tables,
whose parameter intervals are at another level, “Confidence level: 95%
(90% for the RMSEA interval)”), the fit line of
fa_exploratory() (“RMSEA with 90% interval”), the line
beneath factor_count()’s table,
results_sentence(), the conf_level column
tidy() puts on the RMSEA rows of
sem_fit_measures() and beside the RMSEA columns of
factor_count(), and a new rmsea_conf_level
field of dmar_metadata(), which holds the level of a
table’s RMSEA interval beside conf_level.
The help of rmsea_conf_level in
fa_confirmatory(), fa_bifactor(), and
fa_second_order() (and so in cfa_1() and
cfa_2(), which inherit it) now says where the argument
reaches: the table DMAR returns and its glance().
output = "summary" returns the lavaan summary, which shows
lavaan’s 90 percent interval whatever the argument is.
fa_exploratory() and fa_confirmatory()
name the exploratory and the confirmatory factor analysis.
fa_confirmatory() is the function DMAR 1.0.0 released as
cfa_k(); cfa_k() remains as an alias, as
plot_cfa_k() does for plot_fa_confirmatory(),
and its results keep the 1.0.0 class dmar_cfa_k after
dmar_fa_confirmatory, so code written for 1.0.0 still runs,
though on incomplete items its default analysis has changed (FIML in
place of listwise deletion; see Changes to Earlier Results).
efa_k() is the alias of fa_exploratory(),
whose name is DMAR’s own so that attaching lavaan, whose
efa() it calls, masks nothing.
fa_exploratory() fits common-factor ML or MLR models
directly to incomplete raw observations with FIML, or explicitly
requested WLSMV models to ordered items. Rotated loadings, factor
correlations, and communalities carry rotation-aware confidence
intervals. The estimator, missing-data method, and selected rotation
travel with the result. The factor count and rotation are conditioned
on, not treated as known without qualification.
estimator = "ULS" adds the minimum residual (minres)
solution of Harman and Jones (1966): unweighted least squares fitted to
the correlation matrix (lavaan’s correlation = TRUE), whose
loadings and communalities equal those of
psych::fa(fm = "minres") and of a direct minimization of
the squared off-diagonal residuals. With raw data the standard errors
are lavaan’s normal theory sandwich for a least squares estimator
(robust.sem.nt); from a supplied matrix they are
NA, since the sandwich needs the raw data. ULS has no
likelihood, so incomplete data are refused under the default
missing = "fiml" rather than deleted or imputed silently:
missing = "listwise" fits the complete cases, and
S takes a correlation matrix estimated from every case (for
example cov_fiml()’s). The ULS statistic, N - 1 times the
ULS discrepancy of the correlations, has no reference distribution, so
in the fit_measures attribute the chi_square
row holds it without a p-value, the RMSEA rows and the CFI, TLI, RNI,
and IFI are NA, and the NFI and the SRMR are reported;
sem_fit_measures() says why in the terms of a correlation
structure, in a warning of class
dmar_uls_standard_statistic that
fa_exploratory() reads by class. glance()
carries the fitted structure ("correlation" or
"covariance").
fa_exploratory() prints the solution, not
the parameter list. One row per item with its standardized
pattern loading on each factor and the loading’s interval, its
communality with its interval, and Hofmann’s (1978) complexity; above
the table the sample, estimator, missing data method, rotation and its
form, and standard error method; beneath it the variance each factor
accounts for (the diagonal of Phi Lambda’ Lambda, as a sum of squared
loadings and as proportions of the total and of the common variance, in
the new variance_accounted attribute), the factor
correlations with their intervals, and a fit line read from the
fit_measures attribute with its version lines.
knit_print() and as_kable() show the same
display. The table keeps every parameter in both metrics for
tidy(), now with the complexity rows (op
"complexity"), and a subset of it prints as an ordinary
table. The rotation list opens from three criteria to ten, each applied
in the form the help names and refused if lavaan applied it otherwise:
oblique oblimin, quartimin, geomin, simplimax, oblimax, infomax, and
Bentler; orthogonal varimax and quartimax; and the unrotated solution.
The result carries the class dmar_fa_exploratory in front
of dmar_factor_analysis, and the help states the
covariance-first rule (Cudeck, 1989) with its two correlation-structure
exceptions, minres and ordered items.
fa_exploratory() reports each factor’s
determinacy and coefficient H. The determinacy is the
correlation of the regression factor score with its factor, the multiple
correlation of the factor with the items (Grice, 2001; the factor
determinacy index of Ferrando & Lorenzo-Seva, 2018), whose square is
the factor’s maximal reliability under the model; H is Hancock and
Mueller’s (2001) closed form on the factor’s standardized loadings
alone, as Rodriguez, Reise, and Haviland (2016) apply it to one factor
of a multidimensional solution. Each carries a delta method standard
error over lavaan’s covariance matrix of the standardized estimates and
Wald limits formed on the logit scale (Browne, 1982), as
reliability_H() forms them by default. They are rows of the
table (ops "determinacy" and "H"), the new
factor_quality attribute, and a block of the display under
the factor correlations. With one factor H is the squared determinacy,
and its standard error is the one fa_confirmatory()’s H row
carries for the same model fitted the same way. Beside each determinacy
rho the display and the attribute give 2 rho^2 - 1, the lowest
correlation two sets of factor scores equally consistent with the model
can have (Guttman, 1955, as Grice, 2001, presents it), with its delta
method standard error and the determinacy’s limits mapped through it; it
is zero when rho^2 is .5 and negative below, and no line is
drawn.
fa_confirmatory()’s omega, average variance
extracted, and H rows take one interval form. Each keeps
lavaan’s delta method standard error, and its limits are formed on the
logit scale and transformed back to the scale of the coefficient
(Browne, 1982; Kelley, 2007), as reliability_H() and
fa_exploratory() form H’s, so they lie within (0, 1) and
are asymmetric near one; for H with one factor the three functions give
the same interval for the same fit. A finite wald_df puts
its t quantile on the logit scale; under
wald_df = "missing_information" these rows keep the normal
reference, as reliability_omega()’s logistic intervals do,
and only the loadings take the t reference. Through this change the
limits were the estimate plus or minus the normal quantile times the
standard error, which can exceed one.
fa_exploratory() rotates to a partially
specified target. rotation = "target" (oblique;
Browne, 1972b) and rotation = "target_orthogonal" (Browne,
1972a) take the target in the new target argument, either
as fa_confirmatory()’s named list of item keys, which
targets each item at zero on every factor it is not keyed to, or as a
matrix of items by factors with NA for each unspecified
loading; with no items, the items the target names are
analyzed, from raw data or from S alike, and the number of
factors is read from the target. lavaan’s rotation checks neither the
target nor whether it identifies the rotation (an unidentified one comes
back with finite standard errors and no warning), so DMAR checks the
identification counts before fitting and the rank condition on the
standardized unrotated loadings (Algina, 1980; Browne, 2001), and
refuses a target that fails with an error of class
dmar_target_unidentified (a target it cannot read, with
dmar_target_invalid; both inherit
dmar_refusal). The target’s mask is passed explicitly, so
nonzero target values are kept, and no row weights are applied. lavaan
reflects all the factors or none, so when any specified value is nonzero
no factor is reflected: a factor with a nonzero value takes its sign
from the target, and one whose specified values are all zero takes the
sign of the random start that reached the minimum, which
seed fixes (the help says so). Loadings the rotation
attains exactly, which lavaan reports with a standard error of zero and
a meaningless test, are shown as fixed by the target, at the target
value and without a standard error; when every specified loading is
attained the solution is the confirmatory model with those loadings
fixed, and the tests check it against lavaan::cfa(). The
rotation line of the display counts the specified loadings, the target
prints beneath the table with a line mapping lavaan’s factor names to
the target’s, and the method attribute carries the target
and that count. The help summarizes the simulation of Myers, Jin, Ahn,
Celimli, and Zopluoglu (2015) and says that the intervals condition on a
target fixed before the data were seen.
New factor_count() compares factor
counts. It fits the exploratory model with one factor, two, and
so on (by default up to the largest count that keeps a degree of
freedom) and sets beside each count the chi square test, RMSEA with its
interval, CFI, TLI, and SRMR of that model (from
fa_exploratory(), under the same version policy),
lavaan::lavTestLRT()’s test of the previous count against
it, the AIC, BIC, SABIC, and HBIC of
sem_information_criteria() with the comparison
compare_sem() draws by a chosen criterion (BIC by default:
each count’s gap from the lowest, its relative weight, and the log Bayes
factor of the best count over it), the reduced eigenvalue at that rank
with its parallel analysis reference (parallel_analysis(),
Horn, 1965), and Velicer’s (1976) average squared partial correlation
after that many components (the MAP). A count whose model does not
converge or is improper is a row that says so, with its fit withheld and
the counts beyond it still fitted. The lines under the table name the
counts with the lowest criteria, how far the eigenvalues exceed their
reference, and where the MAP is smallest; no cutoff is applied and no
count is chosen. fa_exploratory()’s refusal of a fit
without a proper solution now carries the class
dmar_efa_no_solution and a reason (not
converged, improper, or standard errors unavailable), with a message
that says which.
fa_exploratory() takes its fit measures from
the unrotated solution. lavaan’s FIML-corrected (robust) RMSEA,
CFI, and TLI of a rotated efa object depend on the
rotation, since the rotated parameterization counts more free parameters
than the model has: on six Holzinger and Swineford items the robust 95
percent RMSEA interval of one two-factor model was [0, 0.075] unrotated
and [0, 0.154] under oblimin, with the same chi square statistic; the
standard indices do not depend on it. The fit line and the
fit_measures attribute now come from the unrotated fit of
the same model, so every rotation reports the same fit, and
factor_count() reports those values.
fa_exploratory() can return the point
estimates alone. With the new
point_estimates_only = TRUE (the default FALSE
gives the full analysis, unchanged) the table has the rows and the
estimates of the full analysis, the determinacy, H, variance accounted
for, and every fit measure included, with its standard errors, tests,
and limits NA, because none of them is computed: lavaan is
asked for no standard errors (se = "none"), the rotated fit
is made without the test statistic DMAR never reads from it, and the
delta method intervals of the standardized solution, the determinacy,
and H are skipped. The fit measures, the robust statistic and
FIML-corrected indices of MLR and the scaled and shifted statistic of
WLSMV among them, come from the unrotated fit as before and are
identical, and the RMSEA interval keeps its own level,
rmsea_conf_level. A loading a target rotation attains
exactly is recognized by its value and shown at its target, as in the
full analysis. The input, target, convergence, proper-solution, and no
maximum refusals are unchanged. The refusal of a solution without
rotation-aware standard errors, which guards against a solution that is
not identified at its estimates, is kept by inverting the information of
the unrotated fit, the cheap expected information first (a failure
confined to the rotation’s part of those errors is not detected); a FIML
fit whose errors would need unavailable H1 information is not refused,
since none are computed, and its fit measures are withheld as before.
The display says that standard errors and intervals were not computed,
and states no confidence level beside the RMSEA’s on the fit line, and
the setting travels as the point_estimates_only attribute
and a column of tidy() and glance(). The time
saved is largest for a rotated solution: about a third on the nine
classic Holzinger and Swineford tests under ML and MLR, more than half
under WLSMV, and from 29 s to 17 s for the rotated solution of 18 items
with 410 patterns of missing values under MLR with FIML, where what
remains is the unrotated fit’s robust statistic and FIML-corrected fit
indices; an unrotated analysis saves little or nothing, so a program
that fits both the rotated and the unrotated model saves on the rotated
fit alone (45 s against 34 s there).
New factor_congruence() compares the factors
of two solutions. It takes two loading matrices,
fa_exploratory() tables, or single-group lavaan fits (such
as fa_confirmatory(output = "fit"); the result tables of
fa_confirmatory(), cfa_1(), and
cfa_2() are refused with a pointer to that route), reads
fits in the standardized metric, and aligns the items by name. The
factors of y are paired one to one with those of
x by the assignment that maximizes the sum of the absolute
coefficients (the Hungarian method, with ties kept in the listed order),
and a factor whose match is negative is reflected;
match = "order" pairs them by position instead, signed.
Each row gives the coefficient of a pair and the largest absolute
coefficient of the row’s factor with any other factor of the other
solution, a surplus factor of the larger solution has a row of its own,
and the full matrix, the aligned loadings of y, and its
reflected factor correlations are attributes.
loadings = "structure" compares the structure loadings of
fa_exploratory() tables and lavaan fits, not of a matrix.
The help describes Lorenzo-Seva and ten Berge’s (2006) study of how
analysts judged similarity as a finding about the analysts, and the
table applies no threshold. The coefficients agree with
psych::fa.congruence() to 1e-12, and
procrustes_phi(), the coefficient of one pair of vectors,
links to it.
parallel_analysis() uses the same fixed one-factor
ML communality reduction for observed and reference correlation
matrices. Normal and independently resampled-response references remain
distinct, with ordered margins respected when requested. Reference
quantile limits describe Monte Carlo error, not observed-eigenvalue
sampling uncertainty or confidence in a factor count. The output counts
failed and defined boundary reference fits. A complete-case reference
size alongside a FIML correlation matrix is explicitly a benchmark, with
no effective-sample- size or conservativeness guarantee.
fa_exploratory(), factor_count(), and
parallel_analysis() read items with a
covariance matrix as they read it with raw data: S is
reduced to the named rows and columns, in the order given, before its
entries are checked. Naming six items of a nine-item S
therefore analyzes exactly what S[items, items] would: a
missing entry outside the six does not refuse the call, nor does a whole
matrix that is indefinite (as one from pairwise deletion can be) while
the six-item submatrix is positive definite. A name that is not an item
of S is refused, with a message that lists the names not
found.
factor_count() and
parallel_analysis() fit complete ordered items with
missing left out, as fa_exploratory()
does. Such items were refused with the message meant for
incomplete ordered items (factor_count() passed on its own
default, "fiml"; a direct call of
parallel_analysis() stopped with “Ordered parallel analysis
requires explicit pairwise or listwise handling”), while
missing = "listwise" ran. The three functions now resolve
the default by one rule: with no value missing, leaving
missing out analyzes the items as they stand,
factor_count() uses that method for every fit, correlation,
and parallel analysis, and the method attributes record
"listwise". On six four-category items (400 cases) under
estimator = "WLSMV", leaving missing out gives
the table of missing = "listwise" exactly (chi square
46.972 on 9 df for one factor). Ordered items with a value missing still
require "pairwise" or "listwise", and
"fiml" or "ml" is refused for ordered items;
every call that rule refuses is refused by fa_exploratory()
too, with the class dmar_argument_invalid, before anything
is computed or any random number drawn.
sem_sam() provides local
structural-after-measurement estimation for complete continuous data,
using ML/twostep or MLR/twostep.robust. Independent measurement fits and
derivatives of the corrected latent covariance reproduce the point
estimates and normal or robust uncertainty, including lavaan’s optional
finite-sample Gamma correction. This is numerical agreement for the
documented scope, not a coverage calibration. Incomplete data, derived
cross-stage effects, and other unverified models are refused with a
pointer to joint latent SEM where appropriate.
dmar_metadata() is new, and refusals carry a
class. A program built on DMAR (SIEVER, a simulation runner)
reads a result’s estimator, standard error method, test, missing data
method, interval method, reported version and correction, cases and
groups, two-stage row counts, auxiliary variables (the reliability
family’s aux record and fa_confirmatory()’s
two-stage record included), eligibility (eligible, or qualified by an
unavailable H1 model or by a two-stage fit that could not be computed
again with every case, with the text of the warning the function gave),
and bootstrap counts from one list with fixed names, whatever family the
table came from, instead of from attributes that differ by family.
Refusals a program handles are classed: an unavailable unrestricted
model (dmar_h1_unavailable, now the class of the error as
well as of the qualifying warning; inherits(cnd, "error")
tells them apart) and a two-stage fit that could not be computed again
with every case (dmar_two_stage_rows_unavailable) can be
caught by name, inherit dmar_refusal, and carry a
code and their diagnostic or row counts, so an application
need not match the wording of a message; the wording of these two did
not change. The refusals of the SEM fit family are classed the same way:
an inadmissible solution (dmar_inadmissible), a two-stage
fit conditional on observed exogenous variables
(dmar_two_stage_fixed_x), and the refusals of
sem_exact_fit_bootstrap()
(dmar_exact_fit_bootstrap_scope,
dmar_pattern_too_small, and
dmar_transformation_undefined), each with the fields the
help of dmar_metadata() lists. The message of an
inadmissible solution names what failed (the variable with a negative
variance estimate, or the matrix that is not positive definite, with its
group and level), and the two-stage refusal says on which rows to refit
to keep the fit’s estimates (and, for a fit made with
conditional.x = TRUE, to refit with
conditional.x = FALSE as well). Every other error DMAR
signals is a dmar_refusal with its own class as well; the
help of dmar_metadata() lists them.
A misnamed item is refused with a class by
fa_exploratory(), fa_confirmatory(),
measurement_invariance(),
measurement_alignment(), and the functions that share their
checks. fa_exploratory(),
factor_count(), and parallel_analysis() refuse
items that cannot be read from the input (a name that is
not a column of data or an item of S, a name
given twice, a name that is NA, fewer than three items, or
names that are not a character vector) with an error of class
dmar_items_invalid, a dmar_refusal whose
reason element names the check that failed
("form", "unknown", "duplicate",
"count", or, for raw data whose column names repeat a name
or, with items = NULL, that have a column without a name,
"data_names") and whose items element holds
the names at issue. An item that the factors of
fa_confirmatory() name (and so those of
cfa_k(), cfa_1(), cfa_2(),
fa_bifactor(), and fa_second_order()), or that
the model of sem_sam() names, and that the input lacks is
the same mistake and is refused with the same class and the reason
"unknown" ("form" for a name that is
NA). So is an indicator that the model of
measurement_invariance() names, whether as a named list of
factors or as lavaan syntax, and the items of
measurement_invariance() and
measurement_alignment() are refused with the class and the
reasons of fa_exploratory()’s. The messages and the calls
refused are unchanged, with two exceptions:
measurement_invariance() now refuses an item named twice in
items (see Changes to Earlier Results), with the message
measurement_alignment() gives it (“‘items’ must not repeat
an item.”); and in both functions an items that repeats a
name and names fewer than three columns, such as
c("x1", "x1"), is refused for the repeat, where it had been
refused for the count. sem_sam(), new in this release,
names an indicator its data lack (“Indicators not found in ‘data’: y7.”)
rather than speaking of an items argument it does not have.
The class is new, and a program catches the mistake by it rather than by
the wording, which differs by function.
A rung of measurement_invariance() that
lavaan’s optimizer does not converge on stops with an error that names
it. Through 1.0.x the call ended with lavaan’s error from its
fit measures (“fit measures not available if model did not converge”),
which named no rung. Each rung is now checked before anything is read
from it, and one that did not converge stops with an error of class
dmar_not_converged, the class
fa_confirmatory() gives such a fit, whose message names the
rung and which carries it as level, with the lavaan code
that reproduces the rung’s fit as syntax and
lavaan_call. Three items whose correlations in one group
have a negative product, with the factor scaled by a marker item
(lavaan’s default), give such a rung.
A sufficient affine-collapse diagnostic replaces the ordinary
blanket FIML case-count refusal. It uses every row observing the
candidate support and distinguishes dimension-guaranteed collapse from
numerical affine instability. EM convergence alone cannot establish a
finite global H1 maximum. Analytic alpha, observed-denominator omega,
FMI, and two-stage methods require eligible unrestricted moments. Proper
identified constrained FIML estimates with H0-based errors can remain
available when H1 fails; unavailable fit indices and residual
correlations are withheld, and the reason stays in print,
tidy(), and glance(). A separate distinct-case
bootstrap screen remains an explicit estimability policy. H1-dependent
bootstrap points apply the likelihood screen again to the selected
resample. Fixed/conditional-X model diagnostics screen responses only;
conditional design rank and exact Y-on-X relations remain
unassessed.
The footers, warnings, and help of a fit by
estimator = "ULS" state the multiplier lavaan uses to form
the standard statistic from the ULS fit function, N - G for G groups,
where they said N; the help’s sample size convention already had it. No
value changed. The warning of a ULS fit tested by the standard statistic
has the class dmar_uls_standard_statistic.
Continuous ULS now withholds all ML-derived robust indices, including CFI/TLI/RNI, consistently in fit summaries and invariance ladders. Qualified scaled diagnostics may remain; common-units invariance does not establish validity of an ML correction outside its derivation.
CFA uses estimator-matched SE defaults: standard for ML and
Huber-White for MLR (DMAR 1.0.x passed se = "standard"
under every estimator; see Changes to Earlier Results). Incompatible
SE/test choices cannot silently change FIML to listwise deletion. Its
RMSEA interval keeps lavaan’s 90 percent, as DMAR 1.0.0’s
cfa_k(), cfa_1(), and cfa_2()
reported it whatever conf_level was, and the new argument
rmsea_conf_level sets it (see Every RMSEA Interval at 90
Percent by Default). A development version of 1.1.0 made the interval
follow conf_level, .95 by default; that change, which no
release carried, is withdrawn.
Saturated models retain lavaan’s RMSEA = 0, CI [0, 0], and TLI = 1 conventions. The explanation states that these are saturated-model descriptions without approximate-fit evidence or a fit-index SE; identified parameter SEs and intervals can still be meaningful.
sem_fit_measures() corrects eligible two-stage fits
for auxiliary-only cases, so estimates, SEs, tests, indices, and sample
size use every case the first stage read. The correction is recorded
rather than reporting a smaller sample size beside first-stage moments
based on all rows.
The bootstrap family requires at least half of attempted replications and at least two defined estimates for each target. AVE retains its additional 100-return minimum. Ineligible intervals are withheld while point estimates and counts remain available. Defined improper estimates are retained under the documented policy. These floors do not guarantee nominal coverage. Categorical omega now defaults to 20 imputations inside each bootstrap replication; its separate point-estimate rule already uses at least 20. No new calibration of the 20-imputation default is claimed.
Reliability sensitivity takes the width-target proportion, the
widths, and the coverage over one denominator, the simulated studies
that returned an interval; a study without one is counted in the
simulation attribute (attempted, with an interval,
without), where a binomial interval describes Monte Carlo uncertainty in
the width-target proportion. (DMAR 1.0.0 took the proportions of
intervals that missed the true value over all attempted studies; see the
entry on the record of the simulation.)
Simultaneous-interval sentences use only a matching
multiplicity-adjusted p-value and explicitly name it, or explain that no
matching p-value is available. Equivalence sentences say TOST
p. Correlation and coefficient- of-variation intervals accept supplied
tail areas when conf_level was omitted and refuse an
explicitly inconsistent level; one-sided CV bounds respect the
parameter’s zero boundary.
The eight score columns of factor_scores() carry the
prefix .derived_ (.derived_regression,
.derived_regression_se,
.derived_regression_ci_lower,
.derived_regression_ci_upper, and the same four for
.derived_bartlett), so that a score joined to the observed
data cannot be taken for an observed variable or collide with one. The
attribute derived_columns names them by what each holds.
Valid regression conditional means are retained when a
factor-disturbance relation with an observed conditioning variable
invalidates Bartlett’s conditional unbiasedness; the Bartlett quantities
are then unavailable with a reason. Ordered models outside the verified
score likelihood retain their explicit refusals.
Every indefinite supplied covariance matrix is refused after unit scaling. A general loading fixed or constrained to zero is treated like an omitted loading by the bifactor-index eligibility rule. General CFA/SEM can fit such a model, but the current bifactor indices do not apply.
Welch simple effects retain their valid F tests and withhold the inherited partial-eta-squared interval: its estimand does not match the reported ordinary variance-ratio estimate. Parallel population plans require the supplied population to satisfy parallel-model restrictions. Congeneric complete-data plans below items + 1 are raised to that feasibility minimum before missing-information inflation, with an assurance-planning warning.
A factor analysis or structural equation model fit whose
likelihood has no maximum stops. lavaan lets an error variance
fall below zero, and for some models and data the likelihood then has no
maximum: it keeps rising, by less and less, as one item’s error variance
falls without bound and its loading grows to match, and lavaan’s
optimizer stops where its steps stop gaining and reports convergence, at
a point its starting values and tolerance set rather than the data. When
an error variance is negative, DMAR now fits the model again with that
variance held further out (at ten times its estimate, or one model
implied variance below it, whichever is further; with several, all of
them together and then each alone), and when the log likelihood there is
higher, or lower by no more than 1e-6 per case (a likelihood that flat
leaves the estimate as much to where the optimizer stopped as a ridge
does), it stops, for every output (output = "fit"
included), with an error of class dmar_no_maximum (a
dmar_refusal). The message names the function and the item,
says what the fit further out found, says that DMAR fits the model with
lavaan, the estimation engine, and to see lavaan for more details, where
bounds = "pos.var" or a different model are the choices,
and prints the lavaan code that reproduces the fit (the model syntax
DMAR built, less the lines that define its indices, and the call, with a
placeholder for the data that selects the items where lavaan would
otherwise read every variable it is given); a model whose syntax would
carry the message past the 8,170 bytes R prints of an error has the call
printed and the syntax left in the condition. The condition carries the
syntax, the call, the items, their error variances, the two log
likelihoods, the tolerance, and the number of cases. A finite negative
estimate, at a maximum of the likelihood, is reported as before: the
nine-test bifactor model of fa_bifactor()‘s help gives
visual perception an error variance of -41, and the likelihood falls on
either side of it, while a fourth group factor on the Woody-McCall,
revised paper form board, and flags tests sends the error variance of
paper form board to -1163 under FIML, where the likelihood is still
rising, and that model now stops. The check covers
fa_confirmatory() (with cfa_k(),
cfa_1(), and cfa_2()),
fa_bifactor(), fa_second_order(),
fa_exploratory() (whose refusal is also a
dmar_efa_no_solution, with the reason
"no maximum", which factor_count() reports as
that count’s status), measurement_invariance() (naming the
rung and the group), the configural fit of
measurement_alignment(), and the single-factor fits of
reliability_omega() and reliability_H() (the
tau-equivalent fit of reliability_alpha() has a maximum,
since its equal loadings keep one item’s error variance from running
off). It applies to maximum likelihood fits; least squares and WLSMV
fits have no likelihood and are not checked. A solution with no negative
error variance costs nothing more; one with a negative error variance
costs a refit, or, with several, one with all of them held and then one
for each until one shows a ridge (0.03 s for the nine-test model above,
and 0.08 s for the model with the fourth group factor, whose own fit
takes 5 s). Inside a bootstrap (the reliability family’s,
ci_bifactor_indices()’s, and that of
reliability_H() on a fit), a replication with no maximum is
set aside and counted among the replications that raised an error, under
a message that names the item, and the run goes on. The engine fits each
such replication a second time to recover its message, with the second
fits spread over the same ncpus processes as the
replications, and a fit that runs off is among the slowest lavaan makes:
on one process, 100 replications of the 15-test bifactor model of
fa_bifactor()’s help, 8 of them with no maximum, took 143
s, against 99 s without the check. The warning of
fa_bifactor() about an improper solution names sampling
error, a model that does not describe the items, and a group factor the
data barely separate from the general factor as the possible causes,
where it named the last alone. The structural equation models DMAR fits
with lavaan are checked the same way. The fit of the analysis model to
the population moments, from which ss_aipe_composite_sem(),
ss_power_composite_sem(), ss_aipe_sem_path(),
and ss_aipe_sem_path_sensitivity() (to
supposed_cov_matrix, and to true_cov_matrix,
now fit before any replication is drawn) read their population values,
stops the planner when it has no maximum, with the code that fits the
model to the matrix given (or to
cov_sem(true_model)$sigma_theta) and the model printed
whole, its := lines included, since they define the user’s
own quantities. So does the lavaan fit from which
mediation_mbco() starts its OpenMx fits. Its code fits the
summary statistics directly when they are one group’s (a covariance
matrix given as a data frame converted by as.matrix()); for
several groups, or a model whose product terms the function writes into
columns, it fits the data the function builds from them, which the
message describes exactly, seed included, since the moments of a product
column depend on the rows drawn; and a lavaan fit given as
model is refused without code. The message names a
different model as the choice, since the OpenMx model carries no bounds.
A replication of those planners’ Monte Carlo with no maximum, which
lavaan reports as converged where its optimizer stopped, is set aside as
one that did not converge, and a fresh sample is drawn in its place.
ss_aipe_rmsea_sensitivity() and
sem_exact_fit_bootstrap() read only the chi square of a
fit, and are not checked: along a ridge the likelihood levels off, so
the chi square, unlike the estimates, does not depend on where the
optimizer stopped (in 60 bootstrap samples of a factor with four weak
indicators, the 7 that ran off had statistics of 0.37 to 3.96 that would
move by 0.00005 to 0.0011 further out), and setting such a fit aside
would change the reference distribution, or refuse a population RMSEA
that is determined. A bootstrap sample with no maximum therefore stays
among the improper ones, counted or set aside as improper
says. In a structural model, lavaan carries an observed variable that
enters a regression as a latent variable of its own: an outcome, a
covariate, or an indicator that also enters a regression (with a direct
effect on it, for example). The variance of such a variable is not held
and its loading is not moved, so a ridge on such an indicator, or on an
outcome regressed on a latent variable, is not looked for; on the ridges
of that kind tried, lavaan reported no convergence, where the check does
not apply. A ridge on which lavaan reports no convergence, as it often
does with a marker item fixing the scale, was already a replication that
did not converge. The examples of these pages, none of whose fits has a
negative error variance, take no longer: the check reads each fit’s
parameter table, 5 to 15 microseconds a fit (21 fits in each composite
planner’s examples, a fraction of a millisecond), and refits only a fit
with a negative error variance, which in a Monte Carlo at a small N
costs one refit for each replication that has one.
reliability_omega_categorical() gains a
denominator argument. "observed" (the
default, and the only coefficient the function computed before) builds
the variance of the summed categories from the unrestricted polychoric
correlations, as Kelley and Pornprasertmanit (2016, Eq. 18) did; it is
the categorical counterpart of robust omega.
"model_implied" builds it from the latent-response
correlations the fitted single-factor model reproduces, which is
categorical omega as Green and Yang (2009) defined it: their appendix
program takes “the reproduced (model-based) correlation matrix” as the
input to the denominator. The help page had attributed the unrestricted
denominator to Green and Yang’s Eq. 21; it now states which paper used
which. Every interval method applies to both settings, and the bootstrap
recomputes the coefficient with the chosen denominator on each
replication. reliability() forwards
denominator for type = "omega_categorical" as
it already did for type = "omega". The model implied
setting needs no unrestricted polychoric correlation, but its loadings
rest on the correlations the data do determine. Those identify the
loadings only when they connect every item to the others, directly or
through other items, and include at least one cycle of an odd number of
items (three items whose three correlations are all determined, for
example). When they do not, the loadings are arbitrary. In 13 of 60
samples of 300 from Green and Yang’s mixed-skew design (the help page’s
example with N <- 300 and seeds 1 to 60 in place of 113)
they did not, and in 2 of those lavaan’s start values gave loadings of
opposite signs and a categorical omega of 0.05 and 0.09, against a
population value of 0.716. Both settings then stop with an error naming
the undetermined pairs and either the blocks of items no determined
correlation joins or, within one block, the two sets of items between
which every determined correlation runs. A bootstrap replication or a
completed data set in that state is set aside and counted as
undetermined. The argument changes no existing result; what the observed
setting now does with a correlation the data do not determine is under
Changes to Earlier Results.
bifactor_indices() gains a
denominator argument. The default,
"model_implied", divides by the variances the fitted
bifactor model reproduces (Rodriguez, Reise, & Haviland, 2016), now
including any residual covariances the model specifies (see Changes to
Earlier Results); "observed" divides omega, omega_H, and
omega_HS by the composite variances estimated from the data, the
denominator of robust omega carried to the bifactor coefficients. ECV,
PUC, and H do not depend on the setting. The observed setting requires a
single-group fit. For a fit to ordered items both settings describe the
sum of the latent responses behind the categories, not the summed
categories; a new metric attribute ("observed"
or "latent_response") records which, and the printed line
names it.
Every omega now names its denominator and reports its
local dependence. The printed tables of
reliability_omega(),
reliability_omega_categorical(), and
bifactor_indices() state in one line how the total variance
in the denominator was estimated, so a reader need not know the default
to know which coefficient is on the page.
reliability_omega() and
reliability_omega_categorical() also carry a
residual_cor attribute (the residual correlations of the
single-factor fit, the largest of them, the item pair it belongs to, and
whether the fit is saturated) and print the largest under the table. The
residual correlations come from a maximum likelihood fit whatever the
interval method (the weighted least squares fit behind
ci_method = "adf" has residuals of its own and is not
used). With auxiliary variables, under FIML or two-stage estimation, the
residual correlations are those of the items alone, under the items’ own
names. With three items the single-factor model is saturated and a
proper solution reproduces every correlation, so the attribute’s
saturated element is TRUE, no pair is named,
and the printed line says so. An observed denominator protects the
denominator, not the numerator: a doublet or a method factor shared by a
few same-signed items raises their loadings, and omega with them, under
either setting while leaving the two denominators close. The residual
correlations are where such structure shows, although when it sits on
the items that dominate the factor the single factor absorbs it and the
misfit appears among the other items. No row of any table changes; the
additions are attributes and printed lines.
In a bootstrap, the model implied setting sets aside only
the replications whose loadings are not identified or whose solution is
improper. A resample often leaves out the few respondents who
determine a polychoric correlation, so a bootstrap meets undetermined
correlations far more often than the sample does (see Changes to Earlier
Results for the observed setting).
denominator = "model_implied" needs no unrestricted
polychoric correlation, so it sets aside a replication only when the
correlations left determined do not identify the loadings or, as the
observed setting does, when its solution is improper. That was 1 of 200
in a percentile bootstrap of the help page’s complete data, of 1,000
respondents (seed = 1). In the 45 of the 60 samples of 300
above whose point estimate is identified and proper, a median of 42% of
the replications was set aside (B = 50,
seed = 1), and more than half in 15 of them.
Help pages name the denominator each paper used.
fa_confirmatory() and cfa_1() described the
categorical omega that fa_confirmatory() reports for
ordered factors as Green and Yang’s (2009); it is their assembly with
the unrestricted polychoric denominator of Kelley and Pornprasertmanit
(2016, Eq. 18), the default of
reliability_omega_categorical(), and the pages and the
message now say so. The example of
reliability_omega_categorical() now reproduces Green and
Yang’s mixed-skew design and sets their coefficient beside the one
Kelley and Pornprasertmanit recommended.
**The categorical omega assembly is checked against Green and Yang’s
(2009) population values.** A new test reproduces ten entries of the nonlinear SEM column of their Table 1. Nine agree to the third decimal; the tenth (binary items, two negatively and two positively skewed, loadings of .8; published .591) computes as 0.5904.
The reliability bootstrap refits by maximum
likelihood. Every replication of robust omega, of a bootstrap
interval for the model implied omega, and of the model implied
reliability_alpha() refit the single-factor model by robust
maximum likelihood, whose estimates are those of maximum likelihood;
only the standard errors and the test statistic differ, and a
replication reads neither. The replications now refit by ML, with the
same estimates to the last bit, so every interval is unchanged for a
given seed, and the bootstrap is faster: the scaled test statistic had
cost 1.3, 10, and 37 times an ML fit at 9, 24, and 40 items, and a
40-replication bootstrap of robust omega on 24 items now takes 1.5 s
rather than 7.1 s. Robust omega also fit the model twice on every call,
once for the estimate and once for the residual correlations, and now
fits it once.
Every replication is counted where parallel workers keep
the count, and the draws are returned. The
bootstrap attribute gains improper_kept,
not_converged, too_few_cases,
improper_rule, caution,
require_half, ncpus, t (the
estimate of every replication), and status (its fate:
proper, improper, error, non_finite, undetermined, or not_converged),
and the result gains the B_used attribute that the
package’s other bootstraps carry. No element changes meaning;
not_converged is zero for the reliability coefficients,
whose replications still count a fit that did not converge among the
non-finite ones, and is used by ci_bifactor_indices().
Through 1.0.1 the counts were kept where a forked worker could not
report them, so a session with
options(boot.parallel = "multicore") lost every
count.
ncpus spreads the replications over several
processes. reliability_omega(),
reliability_alpha(), reliability_kr20(),
reliability_omega_categorical(),
reliability_H(), and reliability() gain
ncpus = 1L: forked workers where the platform allows them
and a socket cluster on Windows, with the jackknife behind a BCa
acceleration on the same workers. boot draws every resample before any
replication runs, so a given seed returns the same interval, counts, and
draws whatever ncpus is.
progress follows a bootstrap and can stop
it. reliability_omega(),
reliability_alpha(), reliability_kr20(),
reliability_omega_categorical(),
reliability_H(), reliability(), and
ci_bifactor_indices() gain progress = NULL,
their last argument: a function that the bootstrap calls as
progress(done, total), with the replications attempted so
far and B, after each replication when
ncpus = 1 and after each block of
ncpus * ceiling(B / (50 * ncpus)) replications (about a
fiftieth of B) on several processes. An application that
runs a long bootstrap, a Shiny app above all, can show how far it has
gone and let its user stop it: when the callback returns
FALSE, the bootstrap stops before the next replication or
block with an error of class dmar_bootstrap_cancelled that
carries done and total, and any other value is
ignored. A BCa acceleration comes from the jackknife when fewer
replications than cases return a value or the regression on the
resampling frequencies fails, and while the jackknife runs the callback
is called after each leave-one-out refit, or each block of them on
several processes, with done and total both
B, so a stop is honored there too, and done
equal to total does not mean that the bootstrap has
finished; the bootstrap then impute replications of categorical omega
under missing = "mi" report the same way. An error inside
the callback stops the call with an error of class
dmar_progress_error that carries done,
total, and the callback’s own condition, its class intact,
as parent. The callback runs outside the random number
stream, so a given seed returns the same interval, counts, and draws
with or without it, and the default NULL runs the bootstrap
exactly as before. ?dmar_metadata lists both classes beside
the refusals. sem_exact_fit_bootstrap() and
average_variance_extracted() have no progress
argument in 1.1.0: their bootstraps do not run through this engine, so
an application that calls them cannot follow or stop them this
way.
improper = c("keep", "drop") for the
single-factor coefficients. reliability_omega(),
reliability_alpha(), and reliability() gain
improper. A bootstrap replication whose solution was
improper (a negative error variance) but still yielded a coefficient was
kept and not counted; it is now counted, kept by default, and set aside
under "drop". Keeping is the default because setting such
replications aside conditions the bootstrap distribution on
admissibility and trims its upper tail; in a small coverage check (model
implied one-factor omega, five items with loadings .95, .70, .60, .50,
and .40, N = 60, 120 data sets per condition, B = 300), percentile
intervals covered at .958 when they were kept and at .933 when they were
dropped, a difference near the Monte Carlo standard error of about .02.
Under "drop" an improper point estimate stops when a
bootstrap is requested. When the replications set aside and the improper
ones kept reach the tail area (1 - C)/2 of B, the warning and the
printed line add a caution that the limits depend on how they were
treated. Categorical omega and H are not defined for an improper
solution, so they set such replications aside, and both record
improper_rule = "drop".
Replications of three items with no single-factor
solution are counted on their own. The bootstrap
attribute carries no_solution: the replications set aside
because the correlations of three items had a product that was not
positive, which no loadings reproduce. It is a subset of
improper, reported under the status
"improper", and the warning and the printed line name these
replications apart from the Heywood cases (“had no single-factor
solution”). reliability_H() from data counts them the same
way, and its warning says that setting aside a Heywood case can trim the
upper tail of the bootstrap distribution while one with no single-factor
solution belongs on neither side. The element is zero for categorical
omega, the bifactor intervals, and H from a fitted model, whose refits
are not checked for it, as the help of
reliability_omega_categorical(),
reliability_H(), and ci_bifactor_indices()
says; the categorical page defines returned and
non_finite as reliability_omega()‘s page does.
The warnings describe an improper replication by its criterion, an error
variance below zero (a standardized loading past one; on the items’ own
metric a loading above one is routine in a proper solution), and the
bootstrap then impute warnings of categorical omega describe an improper
replication or completed data set the same way. A replication with no
single-factor solution is set aside under either rule: on the
estimator’s scale it continues the top of the bootstrap distribution,
since one loading, and omega with it, grows without bound toward that
boundary, while its correlations point to low reliability, so it belongs
on neither side. When some are set aside, the caution adds that the
lower limit is likely too high. In a coverage check of the three-item
estimator (three items of unit variance with loadings .70, .60, .30 or
.775, .645, .155; 500 samples of N = 60 drawn after
set.seed(1); percentile intervals from B = 999 resamples),
the lower limit exceeded the population omega in 11.6 and 13.1 percent
of the samples that yielded an estimate (413 and 352 of the 500),
against the nominal 2.5.
The analytic reliability_alpha() records the
improper rule it was given in its
bootstrap attribute rather than the engine’s default. It
fits no model, so no replication can be improper.
The bootstrap reports no interval when most replications
fail the distinct-case rule. The standard error and limits are
NA when more than half of the replications have too few
distinct cases for the coefficient. Near that rule’s bound, the
resamples kept are selected for repeating the fewest rows; at the bound
they are only reorderings of the data and give an interval with no
width. The bootstrap attribute counts these refusals in
too_few_cases, including in its jackknife
record. This rule applies throughout the engine. The separate
require_half rule withholds an interval whenever fewer than
half return a value, whatever the reason; it holds for every caller (the
element require_half of the bootstrap
attribute is always TRUE), as the entry on the bootstrap
family’s half-return requirement states. The number of replications a
request must ask for is set by the rule of the next section, One Rule
for Bootstrap Replications. A bias estimate for H is reported only when
its interval is formed.
A bootstrap warning describes limits only when they exist. With fewer than two returned values, or when either rule above withholds the interval, the warning and printed line say that no interval is reported and give no caution about its limits. The caution names an improper solution only when one was kept. A refusal quoted with its count loses its trailing period inside the parentheses.
An unobserved column in a replication is named separately
from a constant one. No observed value leaves its variance and
covariances unidentified; a single value leaves its variance zero. Both
are counted and quoted by name. A bootstrap of
reliability_H(fit = ) follows the fit’s own missing data
method: complete-data and listwise fits use the rank of the sample
covariance matrix, FIML and two-stage fits use the saturated model’s
case rule, and pairwise fits use neither case-count rule. Its messages
describe the resample without claiming that the supplied fit’s data were
refused.
B replications and is located by the
replications that lie beyond it. An interval read from quantiles is now
run only when B puts at least 10 replications beyond each
limit, and never with fewer than 100: 200, 400, and 2000 replications at
the 90, 95, and 99 percent levels. A BCa interval, whose limits the bias
correction and the acceleration move farther into the tails, is run only
with 50 beyond each nominal limit: 1000, 2000, and 10000. An interval
built from the standard deviation of the replications
("bootstrap_se", "bootstrap_se_logistic")
needs 100, and so does a Monte Carlo p-value. A one-sided bound
and a reference quantile need 10 beyond their one quantile (200 at the
95th percentile). A B below the number required is refused
before anything is computed, with an error of class
dmar_count_invalid whose message states the number and the
count behind it: mediate(d, "x", "m", "y", B = 200), the
call of the 1.0.x help example, returned 0.1816 [0.1163, 0.2684] and now
stops with “‘B’ must be a single whole number of at least 400. A 95%
interval read from the quantiles of the replications has 2.5% of them
beyond each limit, and an interval is reported only when at least 10
replications lie beyond each limit (10 / 0.025 = 400). To run fewer for
exploration or a demonstration, see ?dmar_bootstrap.”; with
B = 400 it returns 0.1816 [0.1165, 0.2592], the numbers
DMAR 1.0.0 returned for B = 400. The numbers of the rule
were computed for DMAR (the tail area a limit cuts off has a relative
standard deviation of about 30 percent with 10 replications beyond it,
whatever the shape of the bootstrap distribution; at 400 replications a
95 percent limit of a bell-shaped distribution varies from run to run by
about 0.13 bootstrap standard errors); no published source is cited for
them, and the new help page ?dmar_bootstrap gives the
reasons.mediate() (percentile, BCa, and Monte Carlo
intervals; the Sobel interval draws nothing and B is not
read for it), mediation_mbco() and
plot_mediation_mbco() (Monte Carlo),
reliability_alpha(), reliability_kr20(),
reliability_omega(),
reliability_omega_categorical(),
reliability(), reliability_H(),
ci_bifactor_indices(),
average_variance_extracted(),
fraction_missing_information(), cohen_kappa(),
fleiss_kappa(), krippendorff_alpha(),
htmt(), ci_ratio_of_means(),
ci_ratio_of_means_paired(),
R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(),
mlmr_mv(), parallel_analysis(),
factor_count(), and sem_exact_fit_bootstrap().
Before, a B was accepted from 2 in the reliability family,
ci_bifactor_indices(), parallel_analysis(),
and sem_exact_fit_bootstrap(); from 100 in
mediate(), mediation_mbco(), the kappa
functions, krippendorff_alpha(), htmt(), the
ratio functions, average_variance_extracted(), and
fraction_missing_information(); and was not checked at all
in R2_mixed_effects(), mlmr(), and
mlmr_mv(). A fractional B was accepted and
truncated by cohen_kappa(), fleiss_kappa(),
and average_variance_extracted()
(cohen_kappa(B = 250.5) ran 250 replications) and is now
refused. mediation_mbco() required a B of at
least 100 whatever the interval and now reads B only for
ci_method = "monte_carlo".
ci_eta_squared_generalized() keeps its own minimum of 1000,
which the rule raises to 2000 at the 99 percent level.
htmt() needs 200 at the default level (100 to 199 were
accepted), parallel_analysis() and
factor_count() 200 at the 95th percentile
(factor_count(B = 0) still omits the reference), and
sem_exact_fit_bootstrap() 100. The planners that run an
interval inside each simulated study follow the rule too:
ss_aipe_indirect_effect() and
ss_aipe_indirect_effect_sensitivity() hold B
to it under method = "monte_carlo" (400 at the 95 percent
level, where 100 was accepted by the planner and the sensitivity
function made no check; under method = "closed_form", which
draws nothing, B need only be a whole number), and the
default B of ss_aipe_reliability_sensitivity()
is now NULL, resolved to the larger of 200 and one and a
quarter times the number the interval method and the level require: 500
for "percentile", 2500 for "bca", and 200 for
the two bootstrap standard error methods at the 95 percent level, where
the default was 200 for all four. The quarter is room for replications
that return no value: the number required is held to the request, so a
simulated study that loses a few replications still reports its
interval, and with the quarter its limits still rest on about the number
required. Its run time is proportional to G times
B, so a percentile check takes about two and a half times
as long by default and a BCa check twelve and a half times. With the
exploratory option the three planners give one warning for the call, not
one for each simulated study, and their results carry the attributes
B, B_used, n_dropped,
B_required, and bootstrap_status, which record
the request each simulated study ran.B = 1000 of
average_variance_extracted(),
fraction_missing_information(),
krippendorff_alpha(), ci_bifactor_indices(),
R2_mixed_effects(), mlmr(),
mlmr_mv(), and of
reliability_omega_categorical() under
missing = "mi" serves the 90 and 95 percent levels and not
conf_level = 0.99, which needs 2000; the default of
ci_bifactor_indices() does not serve
ci_method = "bca" (2000 at 95 percent); and the default
B = 2000 of mediate() serves a 99 percent
percentile interval exactly and not a 99 percent BCa interval (10000).
In each case the refusal names the number to pass.mediation_mbco() (a Monte
Carlo draw at which an effect is not a finite number),
R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(), and
mlmr_mv() (a refit that failed or a value that is not
finite). They now give one warning of class
dmar_bootstrap_replications_dropped, as the others do.
R2_mixed_effects(), mlmr(), and
mlmr_mv() removed a missing value for one statistic only,
so the intervals of one call could rest on different replications; a
replication without a value for any statistic is now dropped for all of
them (see Changes to Earlier Results). The warnings that were unclassed
now carry the class, and every one carries the fields B,
B_used, n_dropped, B_required,
share_dropped, caution, and
status. When the dropped replications number at least the
share of B beyond one limit (2.5 percent at the 95 percent
level) the warning adds a caution, the rule the reliability family
already applied, because they could then decide by themselves where that
limit falls.B is held to the count
before anything is computed, and that is the whole of the count rule. A
request that meets it and then loses replications still reports its
interval: the warning states how many were dropped, and when those that
remain are fewer than the number required it says so and says to request
more (“The 400 required are counted on the number requested (400), not
on the number that remained; the 397 that remained are fewer, so request
more bootstrap replications where some can fail, enough that at least
400 remain.”). The two rules about shares then decide: more than half
dropped withholds the interval, and dropped replications that reach the
share of B beyond one limit bring the caution. A
constrained bifactor model of six tests of
holzinger_swineford with
ci_bifactor_indices(B = 400, seed = 63) and one improper
replication set aside reports the intervals of its 15 indices from the
399 that return a value. Where replications can fail, ask for more than
the minimum; the defaults do at the 95 percent level.reliability_alpha(),
reliability_kr20(), reliability_omega(),
reliability_omega_categorical() (with and without
missing = "mi"), reliability_H(), and
ci_bifactor_indices() share one bootstrap engine, whose
warnings began with the count (“3 of 400 bootstrap replications were set
aside …”) and now begin with the function (“reliability_omega(): 3 of
400 bootstrap replications were set aside …”), as the warnings of the
other functions under the rule do. The condition carries the name in the
field fun. reliability() passes its call on,
so its warning names the function that ran the bootstrap.average_variance_extracted(),
parallel_analysis(), and
sem_exact_fit_bootstrap() name the function too.
These three compose their own account of a run (the factors, the
reference data sets, the reasons a sample was set aside), and it began
with the count (“2 of 200 bootstrap replications were set aside …”). It
now begins with the function (“average_variance_extracted(): 2 of 200
…”), so every warning under the rule says where it came from. A note
that factor_count() prints from its parallel analysis
begins “parallel_analysis():”.mediation_mbco() and
plot_mediation_mbco() draw the estimates from their
estimated sampling distribution. Their caution and their note on an
exploratory run spoke of “replications” and of “the bootstrap
distribution”; they now speak of draws and of the distribution of the
draws, as the rest of each message does.NA, the estimate and every
other row are returned, and the warning has the new class
dmar_bootstrap_insufficient. mediate(),
cohen_kappa(), fleiss_kappa(),
krippendorff_alpha(), htmt(), and the two
ratio functions stopped with an error when fewer than 100 replications
remained, and fraction_missing_information() when fewer
than two did. With a predictor that is nonzero in one of 20 cases,
mediate(B = 150, seed = 113) stopped with “Only 91 of 150
bootstrap replications returned a finite indirect effect”; the same call
with B = 400 now returns the indirect effect 0.5959 with
limits from the 240 replications that remain and the warning that 240 is
fewer than the 400 required, and with B = 2000 it returns
[0.1319, 0.9613] from 1267 replications, as before.B (requested),
B_used (the number the interval rests on),
n_dropped, B_required, and
bootstrap_status ("complete",
"dropped", "exploratory",
"insufficient_count", or "insufficient_share")
are on every such result; cohen_kappa(),
fleiss_kappa(), htmt(),
krippendorff_alpha(), and
average_variance_extracted() had no B
attribute, and mediation_mbco(),
R2_mixed_effects(),
ci_eta_squared_generalized(),
parallel_analysis(), and factor_count()
recorded no counts under these names. mlmr() and
mlmr_mv() carry them as elements of the fitted object, and
print() and summary() state them in one line.
The rows B and B_used of
mediate() and the row B_used of
krippendorff_alpha() are kept. The bootstrap
attribute of the reliability family keeps every element and gains
required, bootstrap_status, and
B_used; sem_exact_fit_bootstrap() gains
B_required and bootstrap_status in its
sem_fit list. dmar_metadata() returns
required and status in its
bootstrap element. The line under a table of the ratio
functions reads “Bootstrap: 2000 replications, 1283 used (717 dropped;
at least 400 must be requested). A dropped replication had a denominator
of zero”; mediation_mbco() prints the corresponding line
for its Monte Carlo interval, and the tables of mediate(),
cohen_kappa(), fleiss_kappa(),
krippendorff_alpha(), htmt(),
R2_mixed_effects(), and
ci_eta_squared_generalized() print “Bootstrap: 400
replications, 400 used (0 dropped; at least 400 must be requested)”
beneath them when a bootstrap was run.B puts fewer than 10 replications beyond
one of them, no BCa interval is reported (limits NA, class
dmar_bootstrap_insufficient, with the positions as
positions and a suggested B as
B_suggest). This applies to mediate(), the
kappa functions, the ratio functions, and the reliability family. In the
probe behind the rule this happened in at most 14 percent of data sets
at B = 2000 (an indirect effect with 50 cases and small
paths) and in 0 to 3 percent of the other conditions; the positions
asked for more than 10,000 replications in about 1 percent of the data
sets of that hardest condition (4 of 400, the largest count 32,909) and
in none of the others. The count the positions take is what the result
records as B_required and what the warning carries, in
every family; the reliability family recorded the 2,000 of the request
beside a message that stated the larger count, and printed no reason
under the table, and now records and prints the count at the positions
(“5105 are required where the bias correction and the acceleration place
the limits, so no BCa interval is reported”).ci_ratio_of_means()
or ci_ratio_of_means_paired() that the rule withholds
prints “Percentile bootstrap interval: no interval is reported.” in
place of a line that called the missing interval “the whole line”, and
its confidence_set attribute and tidy() column
are NA. The messages call a one-sided bound a bound and the
reference quantile of parallel_analysis() and
factor_count() a reference quantile; a BCa interval with
unequal tail areas is refused with a sentence that names the smaller
nominal tail area, where it named two positions the request did not
have; factor_count() gives the exploratory mark under its
own name; parallel_analysis(percentile = 50) and
factor_count(percentile = 50) stop with a message that
names percentile, where it named conf_level. A
warning about a run in which nothing was dropped (in the reliability
family, a BCa interval replaced by the percentile interval, or
replicates that are all the same value; in
average_variance_extracted() and
parallel_analysis(), the function’s account of improper
replications kept or of reference fits that warned) was unclassed and
has the class dmar_bootstrap_note. The B
attribute of the reliability family is an integer, as the other counts
are (it was a double when B was given as one). In an
exploratory run below the required number the reliability family no
longer passes on “extreme order statistics used as endpoints” from
boot::boot.ci() beside its own warning, which already says
how few replications each limit rests on, and mlmr() and
mlmr_mv() no longer pass on lavaan’s count of bootstrap
runs that failed beside the classed warning that reports the same
replications with the counts. plot_mediation_mbco() returns
a plot, which carries no attributes, and its warnings no longer say that
the counts are recorded in the result.cohen_kappa() and fleiss_kappa()
refuse a BCa interval that is not defined. The acceleration
needs kappa with each case left out. A leave-one-out kappa that is
undefined (chance agreement 1) was removed without a word and the
acceleration computed from the rest. The call now stops with an error of
class dmar_bca_undefined (a dmar_refusal) that
names the cases, as mediate() does; the same class is given
when every bootstrap estimate falls on one side of the sample estimate
or the acceleration is not finite. ci_method = "percentile"
does not need these quantities.options(dmar.bootstrap_exploratory = TRUE) runs
fewer replications, and marks the result. A first look, a help
page example, and a simulation over many data sets do not need a
reportable interval from each run. Under the option a B
from 2 up is accepted; the result records B_required and
has bootstrap_status "exploratory", the line
under the table says that the interval is below the number required for
a reported interval, and every call gives a warning of class
dmar_bootstrap_exploratory. The share rule still applies.
The examples of R2_mixed_effects(), mlmr(),
mlmr_mv(), ci_bifactor_indices(),
reliability_omega(), and
sem_exact_fit_bootstrap(), whose replications each refit a
model, set the option, run their small B, and restore it;
the examples of mediate() and
krippendorff_alpha() (now B = 400), of
parallel_analysis() and factor_count()
(B = 200), and the BCa example of
reliability_alpha() (B = 2000) meet the
requirement. The vignettes missing_data (six intervals,
B = 200 before) and DMAR
(factor_count(), B = 50 before) run the
required numbers.sem_exact_fit_bootstrap() gives one warning
about the samples it sets aside. It gave two when no
p-value was formed, one for the samples set aside and one for
the missing p-value; it gives one, with the class that says
which case it is.?dmar_bootstrap. It
states the rule and its reasons, what is recorded and where, the
treatment of BCa intervals, the exploratory option, the conditions with
their fields, and a table of the functions with their kind, default
B, and the number required at the 95 percent level.reliability_H() computes coefficient H from a
fitted model, raw data, or a covariance matrix, and its interval now
uses the full covariance matrix of the estimates. In 1.0.0 the
function took standardized loadings and, for an interval, their marginal
standard errors, the diagonal of the covariance matrix of the loading
estimates. Those estimates are correlated in any fitted factor model, so
that interval could be too narrow or too wide: on one-factor ML and MLR
fits (std.lv = TRUE) of seven blocks of
holzinger_swineford tests (1, 2, 4; 6, 7, 9; 10, 12, 13;
1–4; 5–9; 14–19; and 20–24) the standard error from the full matrix is
0.75 to 1.29 times the one from the diagonal under ML and 0.69 to 1.16
under MLR. H is now computed from the fitted loadings, residual
covariances, and factor variance. It does not depend on how the factor
was identified, and when residuals inside the block are correlated it is
the maximal reliability of Gabler and Raykov (2017). Its delta method
variance runs over every free parameter with the fit’s own covariance
matrix of the estimates (standard, robust, FIML, or WLSMV), under either
ceq.simple setting of a fit whose equality constraints come
from labels or group.equal, and it reproduces the
H_<factor> row of fa_confirmatory() and
lavaan’s own delta method for H defined with :=. The new
arguments come after the 1.0.0 ones: fit (any factor of a
lavaan confirmatory or structural model measured by observed items,
named with factor, and any group, named with
group); data (the single-factor model fitted
as reliability_omega() fits it, with missing
and aux); S and N;
vcov_loadings; ci_method; B;
seed; and ncpus. The default interval is the
Wald interval on the logit scale, with the robust maximum likelihood
standard error for data, the maximum likelihood one for
S, the fit’s own for fit, and for
loadings the one their vcov_loadings (or
se_loadings) gives; standardized loadings with neither
carry nothing an interval can be formed from and give the estimate alone
(ci_method = "none"), with a line under the table saying
what an interval needs. The four bootstrap methods refit the model on
every replication, and their bootstrap attribute carries
the estimated bias of the sample H, which runs upward (Aguirre-Urreta,
Rönkkö, & McIntosh, 2019). On the fit path the model is refitted to
the fit’s own rows before any resample is drawn, and the call stops
unless that refit reproduces the estimate; every refit starts from the
fit’s estimates. A fit with sampling weights or a cluster variable,
which a resample of rows cannot carry, is refused, and the delta method
interval, which uses the fit’s own standard error, is named in its
place. The refits of a fitted model are not checked for three items
whose correlations admit no single-factor solution, so such a resample
is counted as improper when lavaan stops at a solution that has run away
and as non-finite when it does not converge; from data it is counted in
improper and in no_solution, as
reliability_omega() counts it. A fit for which H is not
defined stops with the remedy named: an item that also loads on another
factor or is regressed on another variable, a higher-order factor, an
observed covariate predicting the factor, an improper solution, or three
items whose correlations admit no single-factor solution. A loading or
regression fixed or constrained to zero is not one: a model written with
0*item, or with a*item and a == 0
(directly or through a chain such as a == b with
b == 0), gets the H of the model written without it. A
bifactor fit, whose items load on more than one factor, is refused with
a message that names the index bifactor_indices() reports,
its maximal_reliability attribute, and
ci_bifactor_indices(). Every call but the 1.0.0 form (see
Changes to Earlier Results) returns the reliability family table
(estimate, se, ci_lower,
ci_upper, conf_level, N,
N_complete, n_items; class
dmar_reliability, coefficient "H"). Attributes
name the factor, the group, the metric ("latent_response"
for ordered items), the basis of the standard error, what came with
standardized loadings (vcov_supplied), and the standardized
loadings, and the printed table says which factor of a model it
describes. reliability(type = "H") dispatches to it;
auto-detection never chooses it. From data, a fit that lavaan stops with
an error is reported with lavaan’s message, as
reliability_omega() reports it.fa_bifactor() is new. It fits the
bifactor model, a general factor on every item and two or more group
factors, each on its own items, with every factor uncorrelated with the
others and its variance fixed at 1, described as
fa_confirmatory() describes a model: the items of each
group factor are named in factors, the general factor’s
name in general, and items on the general factor alone in
general_only. It shares fa_confirmatory()’s
engine, interface, and fit rows: raw data (by full information maximum
likelihood, the default, or after listwise deletion) or a covariance
matrix with its N, ML or MLR, equal_loading,
equal_error, and equal_intercept within a
factor, and the five outputs, "fit" returning the lavaan
fit that bifactor_indices(),
ci_bifactor_indices(), and factor_scores()
take. Beside the loadings, error variances, and intercepts the table
reports, with a delta method standard error and a confidence interval
each, omega and omega_H for the total score; omega and omega_HS for each
subscale; ECV, H, and the maximal reliability for every factor; and PUC,
which depends on the structure alone and has no interval. Omega,
omega_H, and omega_HS are the reliabilities of sums of the items as
scored, the coefficients fa_confirmatory() and
semTools::compRelSEM() compute, where
bifactor_indices() keeps the standardized metric (on the
help page’s 15 Holzinger and Swineford tests, omega_H is 0.552 for the
sum of the scores and 0.577 for the sum of the standardized scores);
ECV, H, the maximal reliability, and PUC are
bifactor_indices()’s. The standard errors of the indices
lavaan can define are its own; the maximal reliability’s is the delta
method’s with numerical derivatives, and every limit is formed on the
logit scale and lies within (0, 1). A structure outside the model, or
one that is not identified, is refused before anything is fit, with an
error of class dmar_bifactor_unidentified (a
dmar_refusal) that names the factors or items and says in
its reason element which kind of refusal it is: fewer than
two group factors and an item on two group factors are outside the model
("scope"; a general factor beside one group factor, with
items on the general factor alone, is identified, but the subscale
indices need two or more); an item listed twice, under one group factor
or under a group factor and in general_only, and names that
clash are "names"; and a group factor of one item or of two
with free loadings, and two group factors whose general loadings the
data fix only up to a scale (two groups of three items each with nothing
on the general factor alone, as on six of the Holzinger and Swineford
tests, where lavaan cannot invert the information matrix), are
"unidentified". Every model with two group factors that is
fit gives a warning of class dmar_bifactor_empirical on
empirical underidentification (Green & Yang, 2018), once the
arguments and the data have passed their checks, so a call that then
fails does not get it. When lavaan’s information matrix is singular,
whether the observed information of a FIML fit or the expected
information of a listwise fit or of a covariance matrix, every standard
error, test, and interval is NA, the maximal reliability’s
included, with a warning; lavaan inverts such a matrix without
complaint, and the maximal reliability’s delta method would otherwise
give a standard error of 0. Ordered items are refused with a pointer to
bifactor_indices() for the latent response metric, and
wald_df = "missing_information", calibrated on one factor
models, is refused as well. An improper solution leaves the maximal
reliabilities, and the H and omega rows it makes undefined,
NA with a warning.
bifactor_indices() and
ci_bifactor_indices() take items on the general factor
alone. An item that loads on the general factor and on no group
factor, which DMAR 1.0.x refused, now counts in the general factor’s
ECV, omega, omega_H, H, and maximal reliability and in PUC, whose pairs
with it are uncontaminated, and in no subscale, when the fit has two or
more group factors. A fit with one group factor is refused by both (see
Changes to Earlier Results).
fa_second_order() is new. It fits the
second-order factor model, first-order factors on their own items and
one or more second-order factors on their own first-order factors, named
in factors and second_order; several
second-order factors are always correlated (uncorrelated second-order
factors are not offered). The model is identified as lavaan’s
std.lv = TRUE identifies it, every second-order factor’s
variance and every first-order factor’s disturbance variance fixed at 1,
and it shares fa_confirmatory()’s engine, interface, and
fit rows: raw data (by full information maximum likelihood, the default,
or after listwise deletion) or a covariance matrix with its N, ML or
MLR, equal_loading, equal_error, and
equal_intercept within a first-order factor and
equal_second_order_loading within a second-order factor,
and the five outputs. The table reports the first-order and second-order
loadings, the error variances, and the intercepts; then, for each
second-order factor, omega and omega_H for the sum of its items and its
ECV, H, maximal reliability, and PUC, and for each first-order factor
omega and omega_HS for its subscale score and the ECV, H, and maximal
reliability of its disturbance, each with a delta method interval whose
limits are formed on the logit scale (PUC has none); then the fit rows.
The indices are those of fa_bifactor() computed from the
Schmid-Leiman loadings the estimates imply (Schmid & Leiman, 1957;
Yung, Thissen, & McLeod, 1999), which the schmid_leiman
attribute gives in the standardized metric, item by item. Structures
outside the model or not identified, and names that clash, are refused
before anything is fit with an error of class
dmar_second_order_unidentified (a
dmar_refusal) whose reason element says which,
as for fa_bifactor(): a first-order factor of one item, a
second-order factor of one first-order factor, and one second-order
factor over two first-order factors without equal loadings are
"unidentified"; an item on two first-order factors and a
first-order factor under two second-order factors are
"scope"; and an item or factor listed twice under one
factor, a first-order factor under none, and clashing names are
"names". With exactly three first-order factors under one
second-order factor (or two with equal loadings) the second-order
structure is just identified, and a line under the table states that
condition in words and says that the model then fits as the correlated
factors model of fa_confirmatory() does whenever the factor
correlations have a positive product and imply standardized loadings
below one (on nine Holzinger and Swineford tests, a chi square of 85.31
on 24 degrees of freedom under both). Incomplete data under FIML in
which no case observes two first-order factors together are refused when
the pairs observed together do not identify the second-order loadings
(three first-order factors under one second-order factor with one such
pair), and fit when they do (four). In this parameterization a
standardized second-order loading reaches one only as its loading runs
off toward infinity, and lavaan’s optimizer can stop on the way short of
the maximum, with other second-order loadings drifted far off, or not
converge; so the model is fit a second time in a parameterization whose
boundary is a point (each first-order factor on the scale of its
second-order factor, the variance of its disturbance there bounded below
at zero, every equality constraint of the call imposed exactly), and the
larger likelihood is reported, two log likelihoods within 1e-6 per case
being taken as the same maximum whatever the number of cases. On one
second-order factor over spatial, verbal, and speed tests and t24, t25,
and t26 of holzinger_swineford, the first fit stopped at a
chi square of 251.77, where the maximum is 169.77. A standardized
second-order loading of one at the maximum, a Heywood case at the second
order, leaves that factor’s second-order loading and its loadings on its
disturbance without a finite estimate and the rows of its disturbance
undefined, all NA, while the other rows and the fit rows
are those of the maximum (the standard errors holding the loading at
one, and absent, with a warning, under se = "bootstrap");
and a correlation of two second-order factors beyond one leaves every
maximal reliability NA. Each gives a warning that names it,
with the values in the improper_second_order attribute, as
a negative error variance is handled in fa_bifactor().
Under FIML, output = "summary" of a fit with a loading at
one shows lavaan’s robust CFI, TLI, RMSEA, and GFI as NA,
since lavaan cannot form them for a fit computed without the optimizer,
with one warning in place of lavaan’s three. Each first-order factor is
reported with loadings that sum to a positive value, and then each
second-order factor with second-order loadings that do. The help shows
the likelihood ratio test of the second-order model against the bifactor
model in which it is nested (Yung et al., 1999; Chen, West, & Sousa,
2006), through lavaan::lavTestLRT() on the
output = "fit" objects of fa_second_order()
and fa_bifactor(). The function is named for the structure
it fits because the sources call the orthogonal layered model of the
bifactor family the hierarchical factor model.ci_bifactor_indices() is new. It gives
nonparametric bootstrap confidence intervals for the indices
bifactor_indices() reports: ECV, omega, omega_H, omega_HS,
and coefficient H for each factor they apply to, and the maximal
reliability of each factor, which bifactor_indices()
returns beside its table. PUC depends only on which items share a group
factor and has no interval. Each replication resamples the rows the fit
used (under full information maximum likelihood, every incomplete row
with its missing data pattern), refits the model with the fit’s own
parameter table and options, and reads every index from
bifactor_indices() on the refit, so the intervals rest on
the same replications (under improper = "keep", those of
the maximal reliability on the proper ones among them); before any
resample is drawn, the call checks that such a refit reproduces the
point estimates. The percentile interval is the default (Kelley &
Pornprasertmanit, 2016) and the BCa interval is available.
B = 1000, seed, and ncpus work as
in the reliability family, and the draws and the fate of every
replication travel in the bootstrap attribute. Improper
replications are set aside by default (improper = "drop"),
unlike the reliability family’s default. A solution, the fit’s or a
replication’s, is improper when bifactor_indices() flags it
or lavaan’s post-estimation check fails, a check made on the model’s own
parameters for a fit with auxiliary variables (lavaan’s
aux), as sem_fit_measures() makes it: lavaan’s
own check also tests the residual covariances of an auxiliary variable,
which are not positive definite once it is related to the general factor
strongly enough, however proper the bifactor solution. On a bifactor
model of 15 Holzinger and Swineford tests (a general factor on tests 1
to 9 and 14 to 19, with spatial, verbal, and memory group factors on
tests 1 to 4, 5 to 9, and 14 to 19; seed = 113), 212 of
1,000 replications were improper and their omega ran as high as 26.5;
keeping every one of them would have put the upper limit of the 95
percent interval for omega at 13.7 rather than 0.902.
improper = "keep" uses an improper replication only when
every index it gives but the maximal reliability is defined and a
proportion. The maximal reliability, which is not defined for an
improper solution, rests on the proper replications under either rule,
so its interval is the same under both, and an improper point estimate
reported under "keep" has none. Coefficient H is not
defined when a standardized loading reaches one (see Changes to Earlier
Results), and such a replication is set aside under either rule. Every
replication set aside is counted and named (improper, not converged, an
error, an ordered item that lost a category in the resample), and a
caution is printed when they reach the tail area of an interval. As
throughout the bootstrap family, an index’s interval requires at least
half of the B replications, and at least two, to return a
value for it; otherwise its se and limits are
NA, and the warning and the printed line say so.
Multiple-group, clustered, and weighted fits, fits to a covariance
matrix, and fits that did not converge are refused. So are the fits
bifactor_indices() refuses, among them a fit with an item
on two group factors. The H it bootstraps is the index of Rodriguez,
Reise, and Haviland (2016), not the maximal reliability of the factor,
which has rows of its own (on the help page’s model run with
B = 50 in place of its 10 and seed = 113, the
general factor’s maximal reliability is 0.678 [0.646, 0.734] beside the
index’s 0.769 [0.741, 0.808]). The sample values of both run upward
(Aguirre-Urreta, Rönkkö, & McIntosh, 2019), so the
bootstrap attribute carries the bootstrap estimate of each
index’s bias (the mean of the replicates its interval uses minus the
estimate), and the printed table gives it for H and the maximal
reliability. On the same example with B = 200 (the same
seed) it is 0.031 for the first group factor’s maximal reliability and
0.019 for its H, against 0.005 for its omega. The printed table says
what ECV and omega mean for a group factor. The table has one row per
index and factor, with estimate, se,
ci_lower, and ci_upper; tidy()
and glance() give one row per index and one row of counts
and settings.Categorical omega no longer needs
mvtnorm.
reliability_omega_categorical() and the per-factor
categorical omega of fa_confirmatory() sum a bivariate
normal probability over every pair of thresholds of every pair of items
(Green & Yang, 2009, Eq. 21). Through 1.0.1 each probability was its
own call to mvtnorm::pmvnorm(), so categorical omega
stopped for a user without mvtnorm, a suggested package,
and those calls took most of the time of a bootstrap replication. DMAR
now computes the probabilities itself, all at once, by the method of
Drezner and Wesolowsky (1990) as refined by Genz (2004), the algorithm
mvtnorm applies in two dimensions. The two agree to within
4e-16 on a grid of 2,873 points. Every categorical omega checked is
unchanged to the rounding of its sums, by at most 9e-16: Green and
Yang’s ten population designs, six to 24 Holzinger and Swineford tests
cut into two to seven categories under both denominators,
fa_confirmatory()’s per-factor omega, and a seeded
bootstrap. Within about 1e-7 of a correlation of one
mvtnorm loses accuracy, and within about 1e-10 it returns
the limit at one (an error of up to 2.2e-6, at both limits zero); DMAR’s
computation matches the exact values there. Only a communality or a
polychoric correlation that close to one reaches that region. At 12
items with 7 categories the assembly takes 0.002 s rather than 0.37 s,
so a bootstrap replication takes about 0.07 s rather than 0.43 s, and a
BCa interval at the default B = 10000 about 12 minutes
rather than 70 on the machine where this was measured. No argument,
default, or output shape changes, and mvtnorm leaves
Suggests.
reliability_omega_categorical() analyzes
incomplete items by pairwise deletion or by multiple
imputation. Through 1.0.1 it analyzed only the complete rows.
It gains missing = c("listwise", "pairwise", "mi"),
aux, impute = c("pmm", "polr"), and
imputations = 5, appended after ncpus.
"listwise" stays the default and returns the same results
as before.
"pairwise" estimates each threshold from the rows that
observe its item and each polychoric correlation from the rows that
observe both items (lavaan’s pairwise deletion). With no covariates in
the model it is consistent only when values are missing completely at
random, as listwise deletion is (Asparouhov & Muthén, 2010). Under
missingness at random it is biased (Shi, Lee, Fairchild, &
Maydeu-Olivares, 2020), and the printed table says so.
"mi" imputes the items by chained equations with mice
(now in Suggests): predictive mean matching by default, or the
proportional odds model. Auxiliary variables named in aux
are predictors in the imputation model only, and a row with no observed
item is imputed when an auxiliary variable is observed on it, and
N counts it. The estimate is the mean of categorical omega
over the imputations of the observed data. Its interval is the bootstrap
then impute interval of Schomaker and Heumann (2018): each of
B bootstrap samples of the incomplete rows is imputed
imputations times, and the percentile interval of the
replicate means is the confidence interval. BCa is refused. The items
reach mice as plain integer codes, so a column read with haven, which
carries value labels, and a factor whose labels are whole numbers are
imputed as those numbers. Every incomplete item is imputed, including
one observed together with another on only two or three rows, as in a
planned missing design with a small overlap; the check by which mice
sets aside a variable almost perfectly correlated with another would
otherwise leave it with its missing values. An item with a single
observed value is refused, and an item mice still does not impute stops
the call by name. Turning that check off would also keep an auxiliary
variable that duplicates another, so the auxiliary variables are
screened first. One whose multiple correlation with the columns observed
on every row that observes it (the items and the auxiliary variables
kept before it) reaches .999, mice’s own threshold for a pair, carries
no information beyond the predictors already in the model and is left
out, with a message. Kept together, two or more such variables would
make mice fall back to a ridge penalty and impute unstably. When mice
still applies its ridge penalty, the warning says how often
(ridge in the imputation attribute).
The point estimate is formed from at least 20 imputations, and more when von Hippel’s (2020) quadratic rule asks for more, and averages those that return a value. The rule is applied to the upper limit of the fraction of missing information that the replications themselves estimate. A call with a bootstrap makes the most imputations the rule can ask for (201) and fits every one before the bootstrap runs, so nothing found in a completed data set of the observed data can discard a finished bootstrap.
A completed data set with no categorical omega, an improper (Heywood)
solution above all, is set aside and counted, as a bootstrap replication
is, and the estimate averages the rest. The call stops only when fewer
than half return a value, the criterion of Lim and Cheung (2022) for a
failed analysis. Stopping on any one would fail with probability 1 - (1
- p)^M, for a share p of improper sets among M imputations. Imputation
produces such solutions when the observed data have none, if two items
are skewed in opposite directions and values are missing among the few
respondents who determine their polychoric correlation. On the help
page’s incomplete items under the model implied denominator
(missing = "mi", ci_method = "percentile",
B = 200, seed = 1), 26 of the 201 completed
data sets of the observed data were improper, while the complete data
and pairwise fits were proper (the complete rows leave two polychoric
correlations undetermined and the loadings unidentified, so listwise
deletion stops). An improper set has the largest loadings, so setting it
aside lowers the estimate a little. The warning, the printed lines, and
the imputation attribute
(imputations_returned, imputations_set_aside,
status) say how many were set aside and why.
A bootstrap replication computes the same statistic as the point
estimate: it averages those of its completed data sets that return a
value, and it is set aside only when fewer than half do. A rule
requiring every completed data set to return a value would set a
replication aside with probability 1 - (1 - p)^M for a share p of sets
returning none, which grows with imputations. Under this
rule the share goes to zero as imputations grows when fewer
than half of the sets fail. It does not fall steadily, though: an odd
number of imputations needs one more set to return a value than the even
number below it, so it sets more replications aside. The warning
therefore names an even number when any set was set aside. In the
bootstrap of the help page’s incomplete items above, a quarter of the
replications’ completed data sets returned no value. Applied to the
first two, three, four, and five imputations of each replication (the
replicate_estimates element of the imputation
attribute), the rule set aside 12.5, 21, 14.5, and 20% of the
replications, where requiring every set would set aside 35.5, 45.5,
48.5, and 52.5%. The replicate by imputation estimates form an
unbalanced one-way analysis of variance (Searle, Casella, &
McCulloch, 1992). The warning and the imputation attribute
(replicate_sets_set_aside) count the completed data sets
set aside within the replications kept. As for the other treatments, no
interval is reported when fewer than half of the replications return a
value.
The result’s imputation attribute carries the
imputations, the fraction of missing information, and how much
imputation noise widens the interval. A replication whose imputation
fails is set aside and counted under a new reason,
imputation, and a seed gives the same result for any
ncpus. A completed data set whose fit lavaan stops with an
error is set aside under "error", and the warning quotes
lavaan’s message.
In a Monte Carlo simulation described on the help page (10 items at n = 300, about a fifth of the values missing at random), predictive mean matching came closest to the complete data estimate, ahead of the proportional odds model, pairwise deletion, and listwise deletion.
B defaults to NULL in
reliability_omega_categorical() and
reliability(). For every existing call it means
10000, the default written before. Under missing = "mi",
where each replication imputes the data and refits on every completed
data set, it means 1000. When no item value is missing,
missing = "mi" has nothing to impute and returns the
complete data result, whose bootstrap takes that path’s default of
10000; the message says so.
reliability() forwards each family
function’s missing data treatment. missing is now
NULL by default and forwarded as given, so each family
function checks it against its own choices (listwise, FIML, or two-stage
for alpha and omega; listwise or FIML for H; listwise, pairwise, or
multiple imputation for categorical omega). missing,
aux, and the new impute and
imputations reach
reliability_omega_categorical(). KR-20 still refuses
missing and aux.
CFA help states the standard errors each estimator
receives. se = "standard" supplies
inverse-information standard errors: normal theory for ML, MLR, and GLS,
and Browne’s asymptotically distribution free ones for WLS. Ordered
items request robust.sem in its place. MLR otherwise needs
se = "robust.huber.white" for sandwich standard errors
under FIML. From a covariance matrix, robust.sem with ML
needs the asymptotic covariance of the moments in NACOV.
This corrects the description; no result changes.
sem_exact_fit_bootstrap() is new.
It gives the chi square test of exact fit a reference distribution that
does not assume normal data: the model based bootstrap of Bollen and
Stine (1992), which transforms the data so that the fitted model holds
in them exactly, resamples the transformed rows B times,
refits the model to each sample, and reports the share of bootstrap
statistics at least the observed one as p_bootstrap, with
the Wilson interval for its Monte Carlo error (its limits print to the
four decimal places of the p-value) and the counts of samples
used, improper, and failed. With complete data it is lavaan’s
bootstrapLavaan(type = "bollen.stine"), resample for
resample when the seed given here is passed to lavaan as its
iseed (with lavaan run serially; lavaan called without
iseed draws its own seed from the caller’s random number
stream). With incomplete data fitted by FIML it transforms the rows so
that the saturated FIML estimates on the transformed data are the model
implied moments, by the two transformations of Savalei and Yuan (2009):
one matrix for every missing data pattern (the default) or pattern by
pattern with each pattern’s own moments. Both are solved on the
variables divided by their saturated standard deviations, so the result
does not depend on the units of the variables; it differs element by
element from the same transformations solved in raw units (as semTools’
bsBootMiss() solves the common one), though both make the
model hold exactly. Every bootstrap sample keeps the missing values of
the rows it draws, so the model is refitted by FIML. The transformed
data are checked: the largest saturated estimating equation and the
largest mean residual at the model implied moments are recorded, and
cov_fiml() on the transformed data returns the fitted
moments. Replications are drawn before any refit, so a seed gives the
same result whatever ncpus is. A sample’s solution is
improper when lavaan’s post-estimation check finds a negative variance
estimate or a covariance matrix that is not positive definite; for a fit
with auxiliary variables (lavaan’s aux) the check is made
on the model’s own parameters, as for the fit itself (see the
sem_fit_measures() entry). Fits to summary statistics,
ordered indicators, non-ML estimators, two-level and clustered models,
sampling weights, two-stage fits, fits conditional on observed exogenous
variables, fits whose analysis data keep missing values without FIML
(pairwise deletion), and fits without a standard test statistic are
refused with the reason, as an error of class
dmar_exact_fit_bootstrap_scope (a
dmar_refusal) whose reason element names it (a
fit whose cases lavaan removed by listwise deletion is bootstrapped on
its complete cases, as it was fitted, with a warning); a missing data
pattern too small for the pattern transformation is refused with class
dmar_pattern_too_small, which carries the pattern, its
counts, and its group, and a transformation that is undefined for the
data with class dmar_transformation_undefined.
sem_fit_measures() is new. It
reports the fit of a structural equation model in one table of
twenty-four rows: the chi square test of exact fit; the RMSEA with its
confidence interval and the tests of close fit (Browne & Cudeck,
1993) and not close fit (MacCallum, Browne, & Sugawara, 1996); CFI,
TLI, NFI, IFI, and RNI; the covariance SRMR; and gamma hat and adjusted
gamma hat with their intervals, with NA where the input
does not determine a row. It takes a fitted lavaan model or the summary
statistics a paper reports. With a fitted model the default
version = "auto" gives lavaan’s robust measures for a fit
with a corrected test statistic (MLM, MLR, WLSMV, a sampling weighted
fit; Brosseau-Liard, Savalei, & Li, 2012; Brosseau-Liard &
Savalei, 2014; Savalei, 2018, 2021), the FIML corrected indices of Zhang
and Savalei (2023) for a FIML fit (which approximate the complete data
RMSEA and CFI and are not corrections for nonnormality), and the
standard measures for a normal theory ML fit of complete data; with
auxiliary variables (lavaan’s aux under FIML) the SRMR is
over the model’s own variables, since lavaan’s SRMR averages in the
auxiliary variables, whose residuals of about zero make it lower for the
same model; the printed footer names the version and why it was chosen.
A fitted model keeps lavaan’s sample size multiplier, so its values
equal lavaan’s (for the scaled version, those lavaan gives under
rmsea.scaled.mplus = TRUE); summary input takes
sample_size_convention = "N-1" by default, the convention
of ci_rmsea() (pass "N" for a statistic from
lavaan or Mplus), and n_groups for Steiger’s (1998)
multiple group RMSEA. The default conf_level is 0.90, the
conventional level of the RMSEA interval, whose two limits decide the
tests of close and not close fit at .05 that the table reports beside
it; the help page explains why, and conf_level = 0.95 gives
a 95 percent interval. A fitted model that did not converge is refused
with an error of class dmar_not_converged, and one whose
solution is inadmissible with a refusal of class
dmar_inadmissible that names the negative variance
estimate, or the covariance matrix that is not positive definite (that
of the latent variables, of the residuals of the observed variables, or
of the residuals of the structural part of the model), as lavaan’s
post-estimation check finds them. With auxiliary variables (lavaan’s
aux under FIML) the check is made on the model’s own
parameters: lavaan’s own check also tests the residual covariances of
the auxiliary variables, which it models as saturated correlates, and
that block is not positive definite once an auxiliary variable is
related to a factor strongly enough (for one factor whose items have
uncorrelated residuals, and an auxiliary variable related to them only
through the factor, a correlation above .33 between the two when the
maximal reliability of the items is .9), however proper the model’s
solution. Auxiliary variables are recognized when they are given through
lavaan’s aux; saturated correlates written into the model
by hand are judged with the rest of theta.
sem_residual_fit(),
sem_information_criteria(), compare_sem(),
ecvi(), and sem_exact_fit_bootstrap() share
the check, and ci_bifactor_indices() judges a fit with
auxiliary variables by it.
sem_fit_measures() reports the robust
measures of a mean and variance adjusted statistic, a CFI of 1 for a
saturated model, and no measure of a ULS fit of continuous data that
changes with the units of the variables. In the examples, the
three-factor model of nine Holzinger and Swineford tests is visual
perception, cubes, and lozenges; paragraph comprehension, sentence
completion, and word meaning; and addition, counting groups of dots, and
straight and curved capitals, fit to holzinger_swineford by
lavaan::cfa() with std.lv = TRUE.
estimator = "MLMVS", or
test = "mean.var.adjusted"). Savalei (2018) shows that the
robust RMSEA and CFI of Brosseau-Liard, Savalei, and Li (2012) and
Brosseau-Liard and Savalei (2014) do not depend on which corrected
statistic accompanies them. Each subtracts c df from the ML statistic,
with c = tr(U Gamma)/df, which is that statistic’s scaling constant
times its degrees of freedom over df. So "auto" reports the
robust version of such a fit, computed by DMAR, with the interval and
close fit tests of Brosseau-Liard et al. (2012, Eq. 10) and a robust
gamma hat. The values equal lavaan’s for the same model fitted by MLM.
For the three-factor model by MLMVS the RMSEA is .091 with 90 percent
interval (the default) [.070, .113], the CFI .932, and gamma hat .958,
where the scaled measures version = "scaled" gives are .089
[.066, .113] and .734. A saturated fit with this statistic is reported
in the robust version as well. Nothing is computed from its undefined
scaling constant there, and the footer attributes the corrections to
Brosseau-Liard et al. without saying DMAR computed them."auto" reports the standard version, with a warning; a
bootstrap test, which carries no scaled statistic, gets the standard
version without one. The standard version estimates the population
values of the ML measures, as the robust one does and the scaled one
does not (Savalei, 2018).NA, or a value only by rounding.NA. So are
those of the standard version, whose CFI, TLI, RNI, and IFI set the
degrees of freedom against the same statistic and are NA
too (lavaan’s standard CFI is .977 for the raw scores and .979 for the
scores divided by 10); its NFI, a ratio of two such statistics, is
reported. When the model restricts the means, the ULS fit function adds
the squared mean residuals to the squared covariance residuals, which
are in other units, and every RMSEA row and incremental index of every
version is NA: with the intercepts of paragraph
comprehension and word meaning constrained equal in the same model
(meanstructure = TRUE), lavaan’s robust CFI is .979 for the
raw scores and .972 for the scores divided by 10, and its scaled RMSEA
.160 and .181. The chi_square row and the SRMR are kept.
The robust CFI, TLI, and RNI and the scaled measures of a model with
free means do not depend on the units and are reported, though no
cutoffs have been developed for them (Savalei, 2018). So are the
measures of a fit by estimator = "ULS" alone, tested by
Browne’s residual statistic, when its estimates do not change with the
units. When restrictions link the means and covariances through shared
parameters, as equal loadings and intercepts do through latent means,
ULS estimates change with the units. Browne’s statistic at those
estimates and its indices change too; RMSEA and incremental indices are
withheld. The valid test statistic and SRMR remain. A warning, the
uls element of the sem_fit attribute, and a
line of the printed footer say which rows are NA and
why.The scaled version of sem_fit_measures()
puts the scaled statistic in the standard formulas. Its RMSEA
interval and its tests of close and not close fit are computed from the
chi_square and df rows of the same table
(Brosseau-Liard, Savalei, & Li, 2012, Eqs. 11 and 13; Savalei, 2018,
Table 1). These are the values EQS and Mplus report, and the ones lavaan
gives under rmsea.scaled.mplus = TRUE. lavaan’s default for
a Satorra-Bentler, Yuan-Bentler, or mean and variance adjusted statistic
refers the unscaled statistic to the noncentral chi square on tr(U
Gamma) degrees of freedom, c df with c = tr(U Gamma)/df as in the
previous entry (for a Satorra-Bentler or Yuan-Bentler statistic, whose
degrees of freedom are df, c is the scaling constant itself): the same
point estimate, but an interval and p-values that the rows beside them
do not imply. The two differ little when c is near 1: for the
three-factor model by MLR, c = 0.98, and the 90 percent interval is
[0.0728, 0.1150] against lavaan’s default [0.0726, 0.1152]. They differ
much when c is not near 1: for the same model on multivariate t
data with 4 df (each pupil’s centered scores multiplied by
sqrt(4 / rchisq(301, 4)) after set.seed(1)), c
= 1.80, the interval is [0.0506, 0.0950] against [0.0562, 0.0892], and
the p-value of close fit .046 against .013. A scaled and shifted
statistic (MLMV, WLSMV) gets lavaan’s own values. For an MLMVS fit,
whose scaled measures version = "scaled" reports, the
interval is [0.0659, 0.1125] against lavaan’s default [0.0684, 0.1098];
"auto" reports the robust measures of such a fit, which
DMAR computes.
Gamma hat and adjusted gamma hat come with intervals, and
gamma hat has a robust version. The rows
gamma_hat_ci_lower, gamma_hat_ci_upper,
adjusted_gamma_hat_ci_lower, and
adjusted_gamma_hat_ci_upper of
sem_fit_measures() are the noncentrality limits of the
RMSEA interval mapped to each index (Steiger & Fouladi, 1997,
pp. 251–252, whose 90 percent interval of .972 to .997 for the adjusted
index in their worked example the tests reproduce). For a robust ML fit
of complete continuous data (MLM, MLR, MLMV, and MLMVS, whose robust
measures DMAR computes since lavaan does not), the robust version
reports gamma hat from the robust noncentrality, with limits from the
robust RMSEA interval: 0.956, with the 90 percent interval [0.935,
0.973] that the default conf_level gives ([0.931, 0.976] at
conf_level = 0.95), for the three-factor model by MLR. The
help page says that gamma hat is lavaan 0.7-3’s gfi_lrt,
which lavaan reports only when asked by name, and not the
gfi lavaan prints, which is computed from Browne’s residual
based statistic (for the same fit, the standard gamma hat, lavaan’s
gfi_lrt, is 0.957 and its gfi 0.959). Only NFI
and IFI have no robust version.
Gamma hat is NA where Steiger’s result does
not reach. The result is for covariance structures. A fitted
model that restricts the means gets NA, with the reason
recorded in the sem_fit attribute. Examples are a growth
model, or intercepts constrained equal (under either
ceq.simple setting), constrained to a constant, entered in
a linear constraint, tied to a loading, or held at the bound of an
inequality constraint, which lavaan’s degrees of freedom do not count.
Whether the means are restricted is read from the derivatives of the
model’s moments, with the constraints that bind at the estimates
imposed. Free means change nothing. Summary input whose df
exceeds p(p + 1)/2, which only restricted means can produce, gives
NA with a warning, where the formula would give an adjusted
gamma hat above gamma hat. Below that bound summary input carries
nothing that shows restricted means, and supplying p states
that the model is a covariance structure. A sampling weighted fit gives
NA too, since its statistic is a weighted pseudo likelihood
ratio statistic.
A fit without a corrected statistic is described from
lavaan’s options. "auto" reports its standard
measures under a reason that names the estimator and the data. Some
standard statistics have no chi square reference distribution: an
ordered fit by DWLS or ULS with the standard test alone (the default
test of those estimators, which WLSMV and ULSMV extend with a corrected
one) and a DWLS or ULS fit of continuous data whose test was set to
"standard" (lavaan’s default for continuous data adds
Browne’s residual based statistic, which is chi square under normality),
to which lavaan gives no p-value, and a sampling weighted fit whose test
was set to "standard", a weighted pseudo likelihood ratio
statistic whose p-value lavaan reports all the same. For such a fit a
warning says that the statistic is not chi square distributed and names
the remedy (a corrected test, or for continuous data leaving out
test = "standard"), and version = "robust"
refuses the fit with the same advice. Full WLS for ordered data is the
exception. Its statistic is asymptotically chi square (Browne, 1984;
Muthén, 1984) and lavaan reports its p-value, so its standard measures
are reported without a warning, and a request for the robust version
says that they are the ones to report. A continuous fit tested by a chi
square statistic is described by its estimator (a normal theory ML or
GLS fit, an asymptotically distribution free WLS fit, or a least squares
fit tested by Browne’s residual based statistic) rather than as a normal
theory fit.
equivalence_rmsea() is new. It
tests whether the population RMSEA is below a bound chosen in advance:
the test of not close fit (MacCallum et al., 1996) read as an
equivalence test by Yuan, Chan, Marcoulides, and Bentler (2016). It
reports the 1 - 2 alpha interval, whose upper limit is their T-size
RMSEA, with the p-value and a verdict, and takes n_groups
and sample_size_convention (default
"N-1").
sem_residual_fit() is new. It
reports the unbiased SRMR of Maydeu-Olivares (2017) with its standard
error and confidence interval (90 percent by default, as lavaan’s, which
pairs with the test of close fit at .05), the test of exact fit (with
the standard error that holds under exact fit only), an optional test of
close fit, the SRMR of the mean residuals, and the largest standardized
residual (Maydeu-Olivares, Shi, & Rosseel, 2018). The residual
covariance it used (normal theory, or distribution free for MLM and MLR)
is recorded on the result. A FIML or pairwise fit of complete data gives
the residual inference of a listwise fit with the same mean structure
and information setting; missing values, two-stage fits, and fits with
auxiliary variables (which change nothing in the model’s fit when no
value is missing, but lower lavaan’s SRMR) are refused, and the message
says which.
sem_information_criteria() is new.
It reports AIC (Akaike, 1974), BIC (Schwarz, 1978), the sample size
adjusted BIC (SABIC; Sclove, 1987; lavaan’s bic2),
Haughton’s (1988) HBIC, and the IBIC and SPBIC that Bollen, Ray,
Zavisca, and Harden (2012) proposed and Bollen, Harden, Ray, and Zavisca
(2014) evaluated for structural equation models, then the log
likelihood, the number of parameters, and N, in one table in which a
criterion not requested is NA. It takes a fitted lavaan
model, or as summary input the log likelihood, the parameter count, N,
and (for IBIC and SPBIC) the information matrices, so that a published
comparison can be reproduced; all four rows of Table 6 of Bollen et
al. (2014) are reproduced to within 0.015. A fit whose fit function is
not the normal log likelihood is refused with the reason (ordered
indicators, an estimator other than ML, the Wishart likelihood, or
sampling weights, whose pseudo log likelihood has no justification for
these penalties; Asparouhov, 2005), a two-level model gives AIC only,
and automatic IBIC and SPBIC are refused for fits whose information
matrices are not the ones the criteria are defined on. The help page
explains that IBIC depends on the units of the observed variables and on
how the latent variables are identified, and that SPBIC under its
default prior depends on the identification; it also notes that the
worked example of Bollen et al. (2012, Table 1) counts log N twice in
IBIC.
compare_sem() is new. It ranks
models fitted to the same data by one criterion (BIC by default) and
reports each model’s difference from the best, its relative weight
within the set (Burnham & Anderson, 2004), and, for the BIC family,
the approximate log Bayes factor of the best model over it. It first
checks that the log likelihoods are comparable: the same cases and data
values (or the same summary statistics), the same observed variables in
any order, the same groups, missing data method, and mean structure, and
the same exogenous observed variables (with fixed.x = TRUE,
the default of lavaan::sem(), the log likelihood is
conditional on them, and two saturated models of the same three
variables that condition on different variables differ by hundreds of
BIC points). A refusal names the two models and what differs. The two
two-stage settings count as one missing data method, since they fit the
same EM moments by the same likelihood and differ only in their standard
errors and test, which no criterion uses; the comparison
attribute records each setting used. For IBIC, and SPBIC under its
default prior, a refusal states the rule: one identification for every
latent variable of every model, which two models that each mix the two
identifications in the same way do not meet.
?sem_information_criteria says the same. Weighted fits are
refused; multiple group models (configural against metric invariance,
for example) are compared by AIC, BIC, SABIC, and HBIC.
tidy() and glance() methods come with
it.
results_sentence() reads an interval held in
<estimate>_ci_lower and
<estimate>_ci_upper rows, and gains
term. A sem_fit_measures() table
holds the RMSEA’s interval in rows named for the estimate, and gamma
hat’s and adjusted gamma hat’s the same way.
results_sentence() writes the one term names
or, when the table holds no interval in another form, the first such
estimate, with no p-value: the table’s p-values are
the tests of exact, close, and not close fit. The level is the
interval’s own when the table records it, and otherwise the table’s. For
the three-factor model of nine Holzinger and Swineford tests (visual
perception, cubes, and lozenges; paragraph comprehension, sentence
completion, and word meaning; addition, counting groups of dots, and
straight and curved capitals) fit by lavaan::cfa() with its
defaults,
results_sentence(sem_fit_measures(fit), digits = 3) is
“rmsea = 0.092, 90% CI [0.071, 0.114]”, at the default level.
term = "rmsea" also reaches the RMSEA of a
fa_confirmatory() table, whose rmsea_ci_level
row records the level of its RMSEA interval (its
rmsea_conf_level, .90 by default), and gives the same
sentence for the same model. The level is also read from the table’s
rmsea_conf_level attribute, so a selection of the
rmsea, rmsea_ci_lower, and
rmsea_ci_upper rows, which leaves the
rmsea_ci_level row out, keeps the level of its interval:
for the model above the selection reads “rmsea = 0.092, 90% CI [0.071,
0.114]”, as the full table does. The same holds for the tables of
fa_bifactor() and fa_second_order(). A
term that names no estimate with such rows is refused with
the names of those that have them. term is appended after
digits, and reading such rows changes no sentence written
before.
A vignette, “SEM Fit and Model Comparison”
(vignette("sem_fit_measures")), teaches these measures on
one running example, the nine tests of Jöreskog (1969) in
holzinger_swineford: the chi square test of exact fit; the
RMSEA with its interval, the tests of close and not close fit, and why
the RMSEA is reported with a 90 percent interval; the equivalence test
and the T-size RMSEA; the sample size convention and the multiple group
RMSEA; the incremental indices and gamma hat; the standard, scaled, and
robust version of each measure and which one a robust or FIML fit
reports; the unbiased SRMR and reading the residuals; the information
criteria, including a worked example in which rescaling one variable
reverses an IBIC verdict while BIC, HBIC, and SPBIC do not move; ranking
candidate models with compare_sem(); and the ECVI. Every
number in its text is computed from the fits.
fraction_missing_information() is
new. For each free and each defined parameter of a full
information maximum likelihood fit (a lavaan fit with
missing = "ml" or "ml.x", for example from
fa_confirmatory(..., missing = "fiml", output = "fit"), or
an mlmr() or mlmr_mv() fit) it reports the
fraction of missing information, 1 - SE_C^2 / SE_O^2 (Savalei &
Rhemtulla, 2012), the proportion of the sampling variance of the
estimate that is due to the missing values, with the effective N, N(1 -
FMI), the target N, N / (1 - FMI), and the interval width inflation
factor, 1 / sqrt(1 - FMI), which state in AIPE terms the precision the
missing values cost. Both standard errors come from refits at the fit’s
estimates with the optimizer off and one information type for both, the
complete data side fit to the EM (unrestricted) moments with the same N
and no (N - 1)/N rescaling of the covariance matrix, as lavaan computes
them. information chooses among the three estimators of
Chen and Savalei (2021): "unstructured", the default, which
cannot be negative and broke down least in their simulations;
"structured"; and "hessian", lavaan’s default,
which agrees with lavaan’s parameterEstimates(fmi = TRUE)
to about 1e-7 when the fit runs lavaan’s EM as far
(em_h1_args = list(tol = 1e-10)), and otherwise only to the
precision of lavaan’s EM (see the first example below). The default
therefore differs from lavaan’s. An MLR fit gets the normal theory FMI
at its estimates, which is the ML fit’s. Multiple groups are supported,
each row’s N following its group and its equality constraints, whether a
constraint names labels or defined parameters (ind1 == ind2
with ind1 := a1*b1 and ind2 := a2*b2 gives the
rows it ties the summed N, as a1*b1 == a2*b2 does), and the
omega, H, and AVE that fa_confirmatory() defines get their
FMI through their delta method variances. A percentile bootstrap
interval is opt-in (ci_method = "percentile", with
B and seed), and the printed table says that a
sample FMI is imprecise unless N is several hundred (Chen & Savalei,
2021). Fits with ordered items, an estimator outside the ML family,
listwise, pairwise, or two-stage missing data handling,
conditional.x = TRUE, more than one level, sampling
weights, or no convergence are refused with the reason, as are
lavaanList objects and rotated EFA solutions. tidy() and
glance() methods come with it. The examples below use the
data sets of
tests/testthat/test-fraction_missing_information.R in the
source package.
set.seed(11), then
x5[runif(301) < plogis(-1 + x4)] and
x9[runif(301) < 0.25] set to NA). lavaan’s
own FMIs move by up to 5e-6 when its EM is run to 1e-14 rather than
1e-5, and those of the same model by school, whose solution has a
negative residual variance, by up to .0014. With the fit’s estimates
held (the fit to the rescaled data made at the rescaled estimates with
optim.method = "none"), the FMIs of the three estimators
agree with those in the original units to within 1e-9 in one group and
5e-8 by school when every variable is multiplied by .001, .01, 10, 1000,
10,000, or 1,000,000. Take next a regression of y on three predictors
with values missing at random (200 cases of four standard normal
variables correlated .5, drawn after set.seed(3), with
x3[runif(200) < plogis(1 + 2 * x1)] and
x2[sample(200, 3)] set to NA), fit by
lavaan::sem("y ~ x1 + x2 + x3", missing = "ml", fixed.x = FALSE).
Fit anew to the rescaled data, its FMIs agree with those at a standard
deviation of 1 to within 1.3e-7 when y, x1, or x3 alone, or every
variable, is multiplied by .001, 10,000, or 100,000, when x2 alone is
multiplied by 10,000 or 100,000 (by .001 its standard deviation lies
just above the .001 at which lavaan rescales a fit, and lavaan’s fit
does not converge), and when the outcome is multiplied by 100,000,000,
where mlmr(wald_df = "missing_information") gives the
degrees of freedom it gives at 1. lavaan’s own
parameterEstimates(fmi = TRUE), which refits in the
variables’ own metric, is off by up to .93 (by .04 to .07 for the
slopes) with every standard deviation near 100,000, and returns NA with
every standard deviation near 300,000 or with that of y near
10,000,000.ov_order = "data" and
rescaled by lavaan is not the maximum likelihood fit when the columns of
the data are not in the model’s order: with x3, y, and x1 of the
regression multiplied by .004, .03, and .5 and the columns ordered x3,
x2, x1, y, the slopes’ FMIs are .28, .22, and .70 in place of .06, .19,
and .70. The computation takes one Newton step in the refit metric. When
an estimate moves by more than 1e-4 of the larger of 1 and its size, the
FMIs are computed again one step on. A warning is given only when an FMI
changes by more than 1e-4. It names the parameter, likelihood gain, and
largest FMI change. Units and rescale_data = TRUE are named
only for a fit lavaan did not rescale whose standard deviations indicate
that cause; variable order is named only for a fit lavaan rescaled. A
flat small- sample likelihood can trigger the warning without either
cause, and the warning does not prescribe a remedy already applied. The
FIML refit estimates the unrestricted moments in its own order of the
variables, so a fit made with ov_order = "data" that lavaan
does not rescale gets the FMIs of the same fit in the model’s
order.mlmr() or mlmr_mv() fit as its formula
does.aux, which fa_confirmatory(..., aux = ) passes
on) is supported. Every refit keeps the fit’s estimates whatever
lavaan’s checks of a solution say of them, except for the run-away
Heywood case described next. This covers an auxiliary fit, whose
augmented residual covariance matrix is not positive definite by
construction, two factors correlated above 1, and a residual variance a
little below zero. lavaan’s own
parameterEstimates(fmi = TRUE) shrinks such a covariance to
a correlation below 1 before it refits, and so reports the FMIs of other
estimates: -1.62 against .25 for the covariance of two factors
correlated 1.08 in a sample of 60. The computation stops with a message
that says why when the refit at the estimates moves them. Besides a
rotated solution, this happens with an improper one: lavaan (0.7-3) will
not take as a starting point the estimates of a Heywood case whose
negative residual variance exceeds the variable’s observed variance in
size, and restarts from its simple starting values, and lavaan’s own
parameterEstimates(fmi = TRUE) then reports FMIs computed
at those values (NA for some rows, negative for the intercepts), warning
only that the covariance matrix of the estimates is not positive
definite. The computation also refuses data with no unrestricted
estimate of their means and covariances: a pair no case observes
together, or a saturated likelihood without a maximum. The pair or
singular subset is named, and the checks precede every refit. An EM
iteration limit where the maximum exists instead gives a warning in
DMAR’s words. The conditions have classes
dmar_fmi_never_together and
dmar_fmi_saturated_singular;
wald_df = "missing_information" gives the same reason and
names numeric degrees of freedom as the alternative. Refits do not
repeat lavaan’s notice of low pairwise coverage.missing_patterns() and
missing_coverage() are new. They describe the
missing values before any model is fit: the distinct patterns of
observed and missing items with their counts (the pattern with nothing
observed kept as a row, since lavaan sets those cases aside), and the
number and proportion of cases that observed each pair of items, with
lavaan’s denominator, the cases with at least one observed item. Both
return tables of the same shape whatever the number of items.
df_missing() is new. A Wald
interval from full information maximum likelihood refers the estimate to
the normal distribution; von Hippel (2016) showed that in small samples
with missing values this reference distribution, not the estimate or its
standard error, is what makes the interval too short, and that a
t reference on degrees of freedom that shrink with the
parameter’s fraction of missing information brought his intervals for
regression coefficients and moments near nominal coverage.
df_missing() computes both of his rules from a vector of
FMIs or from the table fraction_missing_information()
returns: method = "effective_n", N(1 - FMI) - k (his Eq.
31), and method = "complete_df", df_complete(1 -
FMI)(df_complete + 1)/(df_complete + 3) (his Eq. 32), each bounded below
at 3, as he recommends (floor). Neither k nor
df_complete has a default, because the right value depends
on the parameter: for a regression coefficient k counts the slopes and
the intercept, and for a factor loading or omega DMAR’s calibration
study chose k = 1 (see the wald_df items below). A clearly
negative FMI, which the structured and Hessian estimators can return, is
refused. Its example takes the FMIs from
fraction_missing_information() on an mlmr()
fit and sets the t interval beside the Wald interval it
replaces.
df_barnard_rubin() is new. It gives
Barnard and Rubin’s (1999) degrees of freedom for an estimate pooled
over imputations, with the total variance, the imputations’ estimate of
the FMI, the relative increase in variance, and both components of the
formula, vectorized over parameters. floor = 3 is von
Hippel’s bound; floor = 0 reproduces mice’s
barnard.rubin(), and df_complete = Inf gives
Rubin’s (1987) degrees of freedom. The between-imputation variance is
passed as between, not B, which DMAR keeps for
bootstrap counts. Both df functions return fixed-column tables with
tidy() and glance() methods, and print beneath
the table a line naming the rule that made the df column
(with the m and df_complete used, for
df_barnard_rubin()) and the value it is bounded below at,
which a knitted table carries as well.
wald_df refers Wald intervals to a
t distribution. reliability_omega(),
reliability_alpha(), reliability(),
fa_confirmatory() (with cfa_1() and
cfa_2()), mlmr(), and mlmr_mv()
gain wald_df = Inf, appended after their existing arguments
(after the ... that fa_confirmatory(), its
wrappers, and the mlmr pair forward to lavaan, so it must be named in
full): the degrees of freedom of the t distribution their Wald
intervals refer to. The estimate and its standard error are unchanged
and the normal quantile is replaced by the t quantile (von
Hippel, 2016); in fa_confirmatory() and the mlmr pair the
p-values refer to the same t distribution, so the
interval and the test agree, and the ratio is reported in the column
test_statistic (z_value in 1.0.x; see the
breaking changes). The default reproduces every interval and
p-value of 1.0.x exactly (qt() and
pt() at infinite degrees of freedom are
qnorm() and pnorm()): results gain a
wald_df attribute and their tidy() rows a
wald_df column, Inf by default; a finite value
adds a line under the printed table. summary() of an
mlmr() or mlmr_mv() fit then heads its
coefficient table t value and Pr(>|t|), as
summary.lm() does, with that line beneath the table’s
significance codes. A number applies to every Wald interval of the call:
the "ml", "mlr", and "adf"
intervals of alpha and of the model implied omega and their
_logistic variants (on the logit scale), every parameter
row of the fa_confirmatory() tables including the
standardized solution, and every coefficient of the mlmr pair. A finite
value where the interval is not a Wald interval (the closed forms, the
profile likelihood, a bootstrap, robust omega,
fa_confirmatory(output = "fit"), se = "none")
is an error. mlmr() and mlmr_mv() also accept
wald_df = "missing_information", von Hippel’s (2016) rule
for regression coefficients, max(3, N(1 - FMI) -
k) for each coefficient, with the FMI estimated from the fit as
fraction_missing_information() estimates it and k the
slopes and intercept of the coefficient’s outcome; with complete data it
gives the N - k of lm(). Its computation does not depend on
the units of the variables. It warns when the fit’s estimates are not at
the maximum of the likelihood, as lavaan’s can be for variables measured
in small units: with every standard deviation of the regression of the
fraction_missing_information() entry near .003
(mlmr(y ~ x1 + x2 + x3, ci_method = "wald")), the
intercept’s degrees of freedom move by .15, and
rescale_data = TRUE, passed through ..., is
the remedy. A coefficient whose fraction of missing information cannot
be computed stops the call with the reason rather than receive NA
degrees of freedom. The degrees of freedom and the
df_missing() table they came from are attributes of the
fit, and summary() lists each coefficient’s FMI and degrees
of freedom. As von Hippel published it, the interval pairs the maximum
likelihood standard error with the t quantile, so on complete
data it is shorter than the interval of lm() by sqrt((N -
k)/N). fa_confirmatory() and
reliability_omega() accept the keyword as well, for the
loadings and omega, on the rule DMAR’s calibration study chose (next
item); reliability_alpha() takes a number only.
wald_df = "missing_information" for factor
loadings and omega. Von Hippel’s (2016) evidence for his
degrees of freedom covers regression coefficients and moments, not
factor loadings or reliability coefficients, so DMAR ran a calibration
study of its own. It lives in the source repository,
methodology/missing_data, with the plan (the design and the
decision rule, fixed before the run), the script, the results, and the
per parameter summary from which every verdict can be recomputed; it is
not in the package.
fa_confirmatory() writes them: 6 items at N from 30 to 200
and 12 items at N = 50 and 100, normal data fit by ML and nonnormal data
by MLR with Huber-White standard errors, and no missing values or 15 or
30 percent of the values of half the items missing completely at random
or at random, with 2000 replications in each of 60 cells. It compared
the normal interval with t intervals on four of von Hippel’s
rules under a decision rule fixed before the run, and a repetition of
his own design reproduced his Tables 4 and 5 to within .012.fa_confirmatory()
(with cfa_1() and cfa_2()) and
reliability_omega() accept the keyword. In
fa_confirmatory() the loadings of the
"verbose" table take the t reference, each on its
own degrees of freedom, and every other row keeps the normal reference,
since the study evaluated neither factor correlations, intercepts, error
variances, nor the AVE and H; omega passed on its own scale,
but fa_confirmatory() now forms omega’s limits on the logit
scale, where the t reference overcovered (.9645), so omega
keeps the normal reference there, and the "measurement"
table, which has no loading, refuses the keyword. In
reliability_omega(denominator = "model_implied") it applies
to the "ml" and "mlr" intervals. With complete
data, listwise deletion, or a covariance matrix every FMI is 0 and the
degrees of freedom are N - 1.holzinger_swineford (visual perception, cubes, lozenges,
the second paper form board, and flags), two of which only the
Grant-White pupils took (156 of the 301 values missing),
fa_confirmatory(missing = "fiml", wald_df = "missing_information")
gives the loadings of those two tests 139.8 and 143.7 degrees of freedom
(FMI .53 and .52), the other three 246.0 to 274.9, and omega 214.6 (FMI
.28); omega’s 95% interval moves from [.6946, .8061] to [.6942, .8064].
In a pilot of 60 of the pupils
(set.seed(113); sample(301, 60)), the model implied omega’s
FMI under
reliability_omega(missing = "fiml", ci_method = "ml", denominator = "model_implied")
is .23, its degrees of freedom are 45.5, and its interval [.665, .888]
becomes [.662, .891], 2.7% wider._logistic
intervals (on the logit scale the t reference covered up to
.9645 with normal data, where the normal interval was already within
.0044 of .95 on average), the ADF intervals, robust omega, the
standardized solution, estimators other than ML and MLR, ordered items,
pairwise deletion and two-stage estimation, standard errors other than
the normal theory and Huber-White ones (robust.sem is
accepted only under listwise deletion, or from a covariance matrix with
the required NACOV; it and Satorra-Bentler tests passed
through ... are refused under FIML whether values are
missing or not, because lavaan falls back to listwise deletion), and,
under full information maximum likelihood, standard errors from any
information but the observed information computed from the Hessian,
lavaan’s default there and the study’s.reliability_alpha() refuses it
too, since no rule passed the study’s decision rule for alpha: the
t reference brought alpha’s FIML interval barely closer to .95
than the normal one, which was already close (.0042 against .0043), and
covered .966 in one cell, the one where the study’s fit of the saturated
model converged in only 53 percent of the replications. Without that
cell the rule would have passed (.0035 against .0042), but the rule was
fixed before the run and its verdict stands; a number of degrees of
freedom remains available.wald_df attribute, the df_missing() table they
came from is the wald_df_table attribute, and a line under
the printed table names the rule, N, and the rows it applied to.cov_fiml() returns the maximum likelihood
covariance matrix and means of incomplete continuous data.
These are the estimates of the saturated model fitted by full
information maximum likelihood, the moments the EM algorithm converges
to, computed with lavaan’s EM, run on the variables centered at their
means and divided by their standard deviations, with the estimates
transformed back. They are the unrestricted moments
lavaan::lavInspect(fit, "sampstat.h1") returns for any FIML
fit to the same variables. They are consistent when values are missing
at random given the variables included, where the complete case matrix
is consistent in general only when values are missing completely at
random. lavaan’s EM stops when no mean or covariance changes by more
than 1e-5, a criterion on the absolute change, so run in the raw metric
its estimates depend on the units: on the regression data of the
fraction_missing_information() entry with every variable
multiplied by .001, lavaan::lavCor(missing = "ml") returns
correlations off by up to .16, with no warning that its EM stopped short
(it warns only that the matrix is near singular, its variances being
near zero). Multiplying a variable by a constant multiplies the mean
cov_fiml() returns, and its row and column of
S, by that constant, and the estimates agree with lavaan’s
own to about seven digits. A warning lavaan raises about the variables
names them as data does. Columns named in aux
enter the saturated model as auxiliary variables and are left out of
what is returned, and a case observed only on its auxiliaries is used,
as the reliability family uses it under FIML (see Changes to Earlier
Results). The result is a list, like that of cov_sem(),
because every use of it takes the matrix: S (divisor N),
mu, N, N_complete,
patterns, and converged. A non-numeric column,
a column with fewer than two distinct observed values, a pair of
variables never observed together, and a pair observed together only in
rows where one of them takes a single value (or in one row only), whose
rows in common say nothing about how the two vary together, are refused
by name. cov_fiml() and the reliability family refuse such
pairs in the same words, from shared checks: the message names the
method that needs the covariance, says what an estimate of it would rest
on (lavaan only warns about a pair never observed together, and its
estimate of that covariance is arbitrary), and advises leaving out one
variable of each pair, an auxiliary variable where the pair has one. The
help page explains why N is not an effective sample size
for a method that treats the matrix as complete data (Enders &
Peugh, 2004; Savalei & Bentler, 2009). The matrix is the basis for
an exploratory factor analysis or a parallel analysis of incomplete
items.
A vignette, “Missing Data: What the Missing Values
Cost” (vignette("missing_data")), teaches the
missing data family on the six spatial tests of
holzinger_swineford, two of which were given only at the
Grant-White school. It covers the patterns of missing values and the
cases behind each covariance; why listwise deletion needs values missing
completely at random and FIML needs them missing at random given the
model’s variables, shown on the real data (where listwise deletion
analyzes one school) and on a deletion made missing at random whose
complete data answer is known; auxiliary variables as saturated
correlates through fa_confirmatory(..., aux = ), and what
lavaan’s warning about the residual covariance matrix means there; the
fraction of missing information of every parameter, with the effective
sample size, the target sample size, the width inflation factor, the
three estimators of Chen and Savalei (2021), and a bootstrap interval
for the FMI; the EM covariance matrix of cov_fiml() as the
basis of an exploratory factor analysis, and why its N is not an
effective sample size; t reference distributions in small
samples through wald_df = "missing_information",
df_missing(), and df_barnard_rubin();
two-stage maximum likelihood with its residual based statistic and
corrected fit indices; omega, alpha, and categorical omega (pairwise
deletion and multiple imputation) from incomplete items; factor scores
and their determinacy for each missing data pattern; and a sample size
plan for omega made with the FMI and checked by simulation. Every number
in its text is computed from the analyses.
fa_confirmatory(), cfa_1(),
cfa_2(), and measurement_invariance() gain
missing = "two_stage" and
missing = "robust_two_stage" (lavaan’s spellings
"two.stage" and "robust.two.stage" are
accepted too). The model is fit to the EM estimates of the means and
covariances, computed from every partially observed case and from any
auxiliary variables passed as lavaan’s aux, which enter
that first stage only (Yuan & Bentler, 2000; Savalei & Bentler,
2009). A case observed on an auxiliary variable but on no variable of
the model is used too. The first stage estimates the items and the
auxiliaries jointly and sums its likelihood over every case (Savalei
& Bentler, 2009, Eq. 10), which is what makes the estimates
consistent when the items go missing together at random given the
auxiliaries.
lavaan’s two-stage estimator reads such a case in its first stage but leaves it out of its N. That N turns the per case quantities of the two stages into:
The fit is therefore computed again with the sample size set to the
cases the first stage read, group by group. lavaan’s own formulas then
give every value on that N, in the table and in the lavaan fits of
output = "fit" and of the "fits"
attribute.
twostage() gives.summary() of such a fit counts the cases as
used, and lavTestLRT() warns that a difference test between
it and a fit lavaan made of the same data on its own N is invalid.Example: six Holzinger and Swineford tests, a spatial factor of
visual perception, cubes, and lozenges and a verbal factor of paragraph
comprehension, sentence completion, and word meaning, with the whole
block missing for 77 of 301 pupils at random given general information,
the auxiliary (set.seed(113), then the six tests set to
NA in the rows
sample(301, 77, prob = plogis(-1.5 + 1.2 * z)), with z the
standardized general information score).
set.seed(2026)), each with the block deleted in the same
way, the intercept’s mean error against the resample’s complete data
estimate is -0.49 with the cases set aside and -0.01 with them
kept.The standard errors are the two-stage sandwich, normal theory or
robust to nonnormality. After normal theory two-stage estimation the
chi_square row is the residual based statistic of Savalei
and Bentler (2009). DMAR computes it from the observed information of
the saturated model over every case the first stage read, and their
simulations found it to hold its rejection rate near the nominal level.
After robust two-stage estimation the row is lavaan’s scaled statistic
(Savalei & Falk, 2014). Simple equality constraints that lavaan
keeps under ceq.simple = TRUE, passed through
..., are read: the residual based statistic of such a model
is that of the same model without ceq.simple.
The cfi and rmsea rows are lavaan’s robust
indices, the corrected indices of Zhang and Savalei (2023). They
subtract from the naive statistic an estimate of its expectation, built
from the asymptotic covariance matrix of the EM moments.
set.seed(1), then sample(301, 90) for each in
turn), the RMSEA is .0869 after normal theory and .0859 after robust
two-stage estimation.tli and nnfi rows and the RMSEA interval are
lavaan’s extension of the correction. A line under the table says which
version it holds.The rung-to-rung tests of measurement_invariance() are
the scaled difference tests (Satorra & Bentler, 2001).
AIC and BIC are the two-stage criteria of Lai
(2021), the values sem_information_criteria() reports. The
printed table says which statistic, indices, and criteria it reports.
fa_confirmatory() frees the item intercepts under two-stage
estimation as it does under FIML; through bare lavaan()
lavaan would fix them at zero. An estimator other than ML, a robust
se, ordered items, and covariance input are errors that
name the option to use. fa_confirmatory()’s table gains a
test attribute naming the statistic in its
chi_square row, and measurement_invariance() a
model_test attribute. No existing value changes.
reliability_omega(),
reliability_alpha(), and reliability() gain
missing = "two_stage", appended to the choices;
the default is full information maximum likelihood when a value is
missing (see Changes to Earlier Results). The model implied coefficients
take the normal theory two-stage standard error under
ci_method = "ml" and the robust one under
"mlr". Their fits go through
fa_confirmatory(), so a case observed on an auxiliary
variable alone is counted as above.
Robust omega divides the squared sum of the two-stage loadings by the sum of the EM covariance matrix, with a bootstrap interval that refits both stages. No delta method interval is offered for it, since none has been studied. Each bootstrap replication fits the second stage to the EM moments of the resample, without the two-stage standard errors and scaled statistic that lavaan computes for every two-stage fit and no replication reads. The bootstrap replications of the model implied coefficients do the same. The estimates are those of a full two-stage fit, to rounding error, wherever the likelihood has a well defined maximum. On a small, heavily incomplete resample, whose likelihood is nearly flat or has no proper maximum, the two optimizers can stop at different points. A replication’s value can then differ at the optimizer’s tolerance, and a solution that runs away can be counted as not converged rather than improper, or the reverse; under either verdict it is among the replications the bootstrap counts as set aside or improper.
Analytic alpha, a function of the first stage alone, equals its FIML
value and interval. Two-stage estimation uses the cases FIML uses, every
case with an observed item or an observed auxiliary variable, since its
first stage reads a case observed on an auxiliary variable alone. A
message counts such cases, once per call rather than once per bootstrap
replication. When the fit of a model implied coefficient cannot be
computed again with every case, a warning says that its analytic
standard error is too large and that a bootstrap interval counts every
case. On visual perception, paragraph comprehension, sentence
completion, and word meaning, with the block deleted as in the example
above and general information as the auxiliary, alpha
(ci_method = "ml") is .773 (SE .0235) on 301 cases, the
FIML value. Setting the 77 aside would give .769 (SE .0242) on 224.
aux names auxiliary variables of the first stage, and
aux without missing still means FIML. The
intervals refused under FIML are refused here too, the profile
likelihood also because no likelihood of the observed data is
maximized.
sem_fit_measures() reports a two-stage fit
with the statistic and indices fa_confirmatory() reports,
and compare_sem() compares two-stage fits by their
two-stage criteria. sem_fit_measures() reports the
robust version, the corrected indices of Zhang and Savalei (2023). After
robust two-stage estimation these are their TS V2. After normal theory
two-stage estimation they are its normal theory analog, which lavaan
forms with the inverse of the observed information of the saturated
model where TS V2 takes the sandwich of the observed and first order
information, and which they did not study. The chi_square
row is the residual based statistic, or the scaled statistic after
robust two-stage estimation, and the printed footer names the version of
the indices, as fa_confirmatory()’s does. It refuses
version = "standard" (the naive statistic, which is not chi
square distributed) and "scaled".
The residual based statistic is refused, with a message that says what to do instead, for a fit with:
fixed.x = TRUE or
conditional.x = TRUE. Refit with
fixed.x = FALSE (and with
conditional.x = FALSE, since lavaan does not take
fixed.x = FALSE beside conditional.x = TRUE),
which leaves the estimates as they are when the exogenous variables are
complete and otherwise also keeps the cases lavaan deletes for a missing
exogenous variable when it conditions on them (robust two-stage
estimation included), so the estimates then differ; to keep the fit’s
estimates, refit on the rows the fit used, those on which every
exogenous variable is observed, which
lavInspect(fit, "case.idx") lists. The refusal is an error
of class dmar_two_stage_fixed_x (a
dmar_refusal) that carries the exogenous variables;Its listwise deletion warning counts only the cases lavaan deletes listwise; the empty cases lavaan drops under FIML or two-stage estimation, which carry no information, draw none.
It warns when a two-stage fit made with lavaan has cases observed on
an auxiliary variable but on none of the model’s variables. lavaan’s
first stage reads such cases and its N leaves them out. The naive
statistic, the scaled statistic and corrected indices built on it, the
RMSEA multiplier, and the N row then rest on fewer cases
than the EM moments, while the residual based statistic uses every case.
The table reports lavaan’s values. The warning points to
fa_confirmatory() and
measurement_invariance(), whose fits keep and count such
cases, and to FIML. On the six tests above
(lavaan::cfa(model, data, std.lv = TRUE, missing = "two.stage", aux = "t5_general_information")),
lavaan’s N is 224 and its robust RMSEA .095; the fit
fa_confirmatory(missing = "two_stage") returns for the same
model counts 301 cases and gives .103.
compare_sem() reports no log evidence for two-stage
criteria, which approximate no marginal likelihood, and refuses
two-stage fits whose auxiliary variables differ.
sem_information_criteria() and
compare_sem() count every case the first stage of a
two-stage fit read. lavaan forms the two-stage log likelihood
as its N times the complete data log likelihood per case at the EM
estimates, and BIC, SABIC, and HBIC take the same N into their
penalties. A case observed on an auxiliary variable but on none of the
model’s variables moves the EM estimates and is left out of that N. The
log likelihood and N are taken instead from the fit computed again with
every case the first stage read.
sem_information attribute records the counts.On the six tests above, fit by lavaan as in the previous entry,
lavaan’s N for the two-factor model is 224 and its log likelihood
-4040.25. That is 224/301 of the -5429.09 reported with N = 301, and BIC
is 10966.61 (8183.32 from lavaan’s values). With the two schools as
groups and equal loadings and intercepts
(group = "school", meanstructure = TRUE, group.equal = c("loadings", "intercepts")),
the log likelihood is -5393.34 and BIC 10957.89, against -5393.52 and
10958.25 at lavaan’s estimates and -4020.79 and 8203.92 from lavaan’s
values. A fit fa_confirmatory() or
measurement_invariance() returns already counts the cases
and gives the same criteria, and compare_sem() compares it
with a lavaan fit of the same data. A warning names what rests on
lavaan’s N only when the fit cannot be computed again.
factor_scores() is new. It returns
both standard factor scores of a fitted lavaan model for every row and
factor: the regression score (the posterior mean) and the Bartlett
score, or for ordered items lavaan’s empirical Bayes modal and maximum
likelihood scores. Each comes with a per-case standard error and a
conf_level interval, beside the number of the factor’s
items the case answered, its missing data pattern, and, for continuous
items in a fit not conditional on exogenous covariates, the determinacy
of each score for that pattern, in one long table with one row per row
of the data and factor. An item listed under a factor with its loading
fixed or constrained to zero (0*item, or
a*item with a == 0) is not one of that
factor’s items. print(), tidy(), and
glance() methods come with it.
Under FIML each case is scored from its own observed items, and the
determinacy for each pattern shows what a missing item costs; for one
factor its square is coefficient H. Beside the regression score’s
determinacy rho, the display and the determinacy and
patterns attributes give 2 rho^2 - 1
(min_score_correlation), the lowest correlation two sets of
scores equally consistent with the model can have (Guttman, 1955),
reported without a line drawn. Rows the fit did not score (deleted
listwise, with no observed item, without a group) stay in the table with
the reason when data is supplied, and a frame that is not
the rows given to the fit is refused. A row deleted listwise from an
ordered fit is told that missing = "pairwise" scores it,
since lavaan has no FIML for ordered items. A row deleted for a missing
exogenous covariate is told, whatever the fit’s treatment of missing
values, that missing = "ml.x" scores it (with
conditional.x = FALSE for a fit conditional on the
covariates), or with ordered items that no fit lavaan can score keeps
it. Without data, where the two cannot be told apart, the
reason in a one-group fit by listwise deletion names what scores each;
in any other fit that deletes such rows it names
missing = "ml.x" and says that supplying data
identifies the reason. A case that answered none of a factor’s items has
no Bartlett score and keeps a regression score that comes only from its
other observed variables (the items of correlated factors, a covariate
that predicts the factor, an outcome it predicts). A factor that a
pattern’s observed items do not separate from another (a case that
answered only an item two factors share) has no Bartlett score, rather
than the minimum norm solution of lavaan’s generalized inverse; a factor
the pattern does identify keeps its score.
A fit of continuous items that keeps the incomplete cases without
FIML is scored as a FIML fit is, each case from its own observed items,
with the fit’s estimates. That covers a two-stage fit (lavaan’s
missing = "two.stage" or "robust.two.stage",
or fa_confirmatory(missing = "two_stage")), whose estimates
are consistent when the values are missing at random, as the FIML
estimates are (Savalei & Bentler, 2009). It also covers a fit by
pairwise deletion (missing = "pairwise" or
"available.cases"), whose estimates require values missing
completely at random. lavaan’s lavPredict() scores such a
fit as complete data and leaves every case with a missing value without
a score, so factor_scores() computes these scores itself,
by the closed forms lavPredict() applies pattern by pattern
under FIML. Such a fit conditional on exogenous covariates is
refused.
Without a mean structure (the default for one group under listwise or
pairwise deletion), a factor that a covariate predicts is centered where
lavaan and FIML center it, at the slope times the covariate’s mean. In
such a fit without an exogenous covariate, lavPredict()
counts twice the mean of an observed variable that lavaan holds among
the latent variables, which shifts by a constant every score whose
weights involve it (the regression score of every factor related to it,
the Bartlett score of a factor that predicts it);
factor_scores() computes those scores itself, and a higher
order factor keeps lavaan’s Bartlett score less that constant. An
observed variable regressed on others, such as an item with a direct
effect of a covariate (the MIMIC model of differential item functioning)
or an outcome the factor predicts, is held by lavaan among the latent
variables, with its loadings among the regressions. The scores, their
standard errors and determinacies, the check of the ML score, and the
check of the solution take its loadings and residual variance from
there, as lavPredict() does, so an item with a direct
effect enters its factor’s Bartlett score. An observed variable that
lavaan holds the same way but regresses on nothing, such as a criterion
that covaries with a factor (f ~~ z) or a covariate of a
factor that also covaries with another variable, is conditioned on, as
an exogenous covariate is.
Without regressions among the variables lavaan holds as latent, a
variable regressed only on exogenous covariates is conditioned on as
well, also when it covaries with a factor. With such regressions,
lavPredict() gives an observed variable regressed on
nothing a row of zeros among the loadings, leaving its covariances with
the factors out of the scores. factor_scores() scores these
fits itself and refuses one conditional on the covariates. A fit with
such regressions is refused when the disturbance of an observed variable
regressed on others covaries with a factor or a variable the scores
condition on: lavaan’s Bartlett score is then not conditionally
unbiased, and where a factor is involved its regression score is not the
conditional mean. The remedy carries the variable as a single indicator
latent variable without fixing its loading, with its observed residual
variance fixed at zero. This preserves the model under
std.lv = TRUE as well. In multiple groups the refusal also
names the intercept and residual variance and covariance equalities
needed to preserve the constraints.
For ordered items a factor has an ML score only where the likelihood
of the case’s responses has a finite and unique maximum in it, which an
exact check decides for any loading structure, cross-loadings, bifactor
models, and observed variables regressed on the factors included (by
Farkas’ lemma, through nonnegative least squares). Each of these has an
NA ML score with the reason and keeps its EBM score: a
factor whose observed items all sit at the end of their scales that
their loadings point to (all in the lowest category, or all in the
highest, for positive loadings), one whose likelihood rises without
bound together with others, one its items cannot separate from another,
and a higher order factor. Every other factor keeps its ML score, also
one that shares items with a factor whose likelihood rises without
bound: its maximum is then that of the case’s other items, which
lavaan’s optimizer reaches within its tolerance. When lavaan’s default
optimizer stops on the other factor and returns no score, the score
comes from lavaan’s nlminb optimizer. With ordered items,
lavaan’s EBM and ML scores omit terms from each case’s likelihood or
prior. A fit is scored when the omitted term is constant in the factors:
a covariate’s effect on an outcome that depends on no factor directly,
or a fully mediated chain of continuous observed variables. A fit is
refused for a direct effect of a covariate on a variable that depends on
a factor directly, a regression of such a variable on an observed
variable, or a regression on an ordered item. It is also refused when an
observed variable lavaan holds among the loadings has a disturbance
covarying with a factor, or a row lacks a predictor whose path reaches a
factor without an observed mediator cutting it. A fit with exogenous
covariates that is not conditional on them is refused because
lavPredict() cannot score it. A fit of ordered items with
residual covariances is scored with a warning, since lavaan’s EBM and ML
scores leave those covariances out of each case’s likelihood; the help
page gives the size of these errors in an example. Ordered items require
lavaan 0.7-3 or newer: lavaan 0.7-2 counted the intercepts of ordered
indicators twice in its EBM and ML scores, which were then wrong in any
model where those intercepts are not zero (a multiple group model with
equal thresholds, for one), so factor_scores() refuses an
ordered fit under an older lavaan.
Auxiliary variables carried as saturated correlates are set aside,
and the scores are computed from the indicators with the fit’s
estimates, since conditioning on them removes true score variance. A fit
that carries them with ordered items is refused (refit without them), as
is one that carries them and is conditional on exogenous covariates
(refit with conditional.x = FALSE). A higher order factor
has no Bartlett score, and its reason says so, where the scores are
computed from the indicators (auxiliary variables set aside), where the
data have missing values and the scores are computed here for a
two-stage fit or one by pairwise deletion ("pairwise" or
"available.cases"), or in a fit in which lavaan gives an
observed variable regressed on nothing a row of zeros among the
loadings, since lavaan’s construction of that score is not repeated
there. Elsewhere it keeps lavaan’s score (less the constant above where
lavaan counts a mean twice), and its Bartlett standard error from lavaan
is in the pattern table.
An improper solution is scored with a warning: a negative variance (the residual variance of an observed variable regressed on others included), a covariance matrix of the factors that is not positive definite (a factor correlation beyond 1, for one), a residual covariance matrix of the continuous items that is not, or, when none of these holds, a residual covariance matrix of the structural part of the model that is not, the conditions lavaan’s post-estimation check flags. For ordered items lavaan computes no score for a solution with a negative residual variance, and the warning and the reasons say so.
The help page says which score belongs where: regression scores for a
predictor and Bartlett scores for an outcome, each from its own
measurement model (Skrondal & Laake, 2001), and one-step SEM or
structural after measurement (lavaan::sam(); Rosseel &
Loh, 2024, which includes Croon’s correction) as the route to relations
among factors. Scores under missing data are discussed with Estabrook
and Neale (2013). The help says that the Bartlett score’s conditional
unbiasedness and the coverage of its interval hold under missing data
only when whether an item is observed does not depend on the residuals
of the items scored (Rubin, 1976), so not when it depends on the
factor’s own observed items, with an example. It also says that the
regression score’s posterior interval keeps its coverage when the
missingness depends on the items scored, but not when it runs through an
auxiliary variable set aside.
A loading or regression held at zero, directly or through a chain of plain equalities, is read as absent. It makes no item, does not keep a saturated correlate from being set aside as an auxiliary variable, and gives the scores and reasons of the model written without it.
ss_aipe_reliability() gains fmi
and seed. A reliability coefficient estimated by
full information maximum likelihood from incomplete data has a wider
interval at a given N than a complete data plan assumes.
fmi (default 0) is the fraction of missing information
anticipated for the coefficient, supplied by a pilot study or a
comparable data set through fraction_missing_information();
it is not the proportion of missing values. Missing information
multiplies the sampling variance by 1 / (1 - fmi) (Savalei
& Rhemtulla, 2012). The planner’s variance is V / (N - 1), so the
plan divides the complete data N - 1 by 1 - fmi:
N = (N_c - 1) / (1 - fmi) + 1, and an FMI of .5 doubles N -
1, not N. With an assurance, the Monte Carlo search runs at the complete
data (effective) size and its answer is mapped back the same way, a
first-order approximation that the sensitivity function checks. The
table keeps its rows; the attributes fmi and
necessary_N_complete record the plan, and a line under the
table names both. The help page explains where a planning FMI comes
from, why a pilot’s FMI is imprecise (Chen & Savalei, 2021), and why
the upper limit of its interval is the conservative planning value. The
plan maps the unrounded complete data solution, so the formula applied
to the printed complete data N can give a case or two more than the plan
(the first example of the help page, six parallel items that correlate
.6 planned for a width of .10, needs 38 complete data cases and 42 at an
FMI of .1, where the formula on 38 gives 43); the help says so. It also
says that a pilot FMI from fraction_missing_information()
refers to the cases the fit used, and that when the planned N counts
every case recruited, those who will answer no item are folded in as
1 - (N_used / N)(1 - fmi). seed (default
NULL) makes the assurance search reproducible and restores
the caller’s random number stream on exit; without it the search draws
from and advances that stream, as before. With fmi = 0 the
plan is the complete data plan and no attribute is added; the planner’s
other changes are under Changes to Earlier Results.
ss_aipe_reliability_sensitivity() gains
fmi, missing_prop, and
B. fmi is the planning value passed
to the planner; missing_prop deletes each simulated item
score completely at random with that probability and analyzes every
replication by full information maximum likelihood, so a plan made with
fmi is judged by the widths that incomplete data actually
deliver. For four parallel items with reliability .80, a width of .15,
and missing_prop = .2, the FMI of the coefficient is .184
relative to every case and .182 relative to the cases a fit keeps, since
a share .2^4 = .0016 of the cases loses every item (the realized FMI,
described below, of one FIML fit to 200,000 simulated cases:
ss_aipe_reliability_sensitivity(true_reliability = .8, n_items = 4, width = .15, specified_N = 200000, missing_prop = .2, G = 1)
after set.seed(113)). In 1,000 replications, with
set.seed(113) before each call, the complete data plan (N =
74) gave "ml" intervals averaging .170 wide, and the plan
made with fmi = .184 (N = 91) gave .152, against .153 for
the complete data plan with complete data. Under missing data the
default interval is "ml", as with complete data, and
intervals whose standard errors ignore the missing values are refused.
With missing_prop above 0, each replication that returns an
interval also estimates the fraction of missing information its
simulated data realized for the coefficient (the unstructured FMI of
fraction_missing_information(), from alpha’s saturated
model or omega’s single-factor model). The mean is reported beside the
planning fmi as the attribute
mean_realized_fmi, with the number of replications behind
it in n_realized_fmi, a second line under the table, and a
realized_fmi column in the per-replication file. A single
sample FMI is too imprecise at the sizes a plan resolves to check a plan
by (Chen & Savalei, 2021), and the mean over replications is the
figure to carry into fmi. Computing it draws no random
numbers, so the table’s rows are those of a run without it; it costs a
fit and two refits per replication. The realized FMI is relative to
every simulated case, the footing of the planner’s fmi:
lavaan sets aside a case that loses every item, and the FMI relative to
the cases it kept is folded back as
1 - (n_used / n)(1 - fmi_used). With three items at
missing_prop = .5, a share .5^3 = .125 of the cases loses
every item, so the FMI relative to the cases lavaan keeps understates
the FMI the plan needs. B is the number of bootstrap
resamples behind each replication’s interval, which was fixed at 200
whatever the interval; its default is the larger of 200 and the number
the interval method and the level require, and a smaller value is
refused (see One Rule for Bootstrap Replications). For alpha, with
fmi and missing_prop at 0, B at
its default, no assurance, and ci_method = "bonett", the
results are those of 1.0.x, except for replications whose interval lies
wholly below zero (see the entry on clamped limits under Changes to
Earlier Results); the default interval is now "ml" (see the
entry there on the interval its plan was made for).
ss_aipe_rmsea() does not take
fmi. RMSEA is a function of the test statistic,
not a model parameter, so no parameter’s FMI governs the noncentrality
its interval is built on, and Savalei and Rhemtulla (2012) leave global
measures of missing information to future research.
ss_aipe_mean() plans the sample size for the
confidence interval for a single mean, the mean of one sample
or the mean of paired differences, which DMAR could not plan before:
ss_aipe_sm() plans the standardized mean, and
ss_aipe_c() requires contrast weights that sum to zero.
Without an assurance the plan makes the expected width no
wider than width, and the expected width is exact under
normality: it uses E(S) = c4(N) sigma, where ss_aipe_c()
sets S equal to sigma, and so needs the same N as that rule or one
fewer. With an assurance the realized width is no wider
than width with that probability.
true_kurtosis, on the excess scale of
kurtosis() so that a pilot’s value can usually be passed as
it is, adjusts the assurance through a scaled chi square with the exact
mean and variance of S^2; the expected width plan does not use it and
warns when one is supplied. A pilot whose scores take nearly two values
can give a kurtosis() of -2 or less, which is refused, and
the help page says what to pass instead. With a large kurtosis the
assurance can be met at a very small N, missed over a stretch above it,
and met again, so the plan is the smallest N from which the assurance
holds at every larger N, and the search finds that N exactly. The answer
row is necessary_N, beside the expected width at that size
and the inputs.
ss_aipe_mean_sensitivity() checks such a
plan by Monte Carlo, under a normal population or by resampling
a vector of observed scores (population), which is how a
plan meets the tails and skewness of real data, such as a pilot’s
difference scores. It follows the sensitivity family’s arguments and
rows, reports mean_mean, median_mean, and
sd_mean for the realized means, and takes seed
(default NULL), which makes a run reproducible and restores
the caller’s random number stream on exit. For the help page’s pilot
(the IQ gains of the control children in pygmalion, excess
kurtosis 1.2), a plan for 90 percent assurance made under normal theory
gave intervals no wider than the target of 4 points in .870 of 1,000
replications, and the plan made with the pilot’s kurtosis, six children
larger, in .913.
ss_aipe_c() gains
true_kurtosis. With an assurance, the
planner takes the pooled variance to follow the chi square law of normal
theory, which holds only for normal errors. Errors lighter tailed than
the normal, as bounded outcomes such as proportions and rating scale
scores often are, make the pooled variance less variable than that law
says, so a plan (for any assurance above one half) delivers more
assurance than was requested; heavy tails make it deliver less.
true_kurtosis (default 0) is the population excess kurtosis
of the errors, a planning value on the scale kurtosis()
reports, and it replaces the error degrees of freedom df = J(n - 1) in
the chi square quantile with the effective degrees of freedom 2 / (2 /
df + true_kurtosis / (n J)), which match the variance of the pooled
variance (Box, 1953). The default reproduces the plans of earlier
releases exactly, and the expected width plan (no
assurance) does not use the argument. An assurance plan
made with a nonzero true_kurtosis keeps its one row,
carries the value as the attribute true_kurtosis, says in a
line under the table that its assurance allows for that excess kurtosis,
and reports it as the column true_kurtosis of
glance(); with true_kurtosis = 0 no attribute
or line is added. In a Monte Carlo check with two groups and an
assurance of .80 (10,000 replications per condition), the plans made
with the kurtosis supplied met the width target in .811 of the
replications under uniform errors (excess kurtosis -1.2) and .811 under
t errors on 10 degrees of freedom (excess kurtosis 1), where
the normal theory plans met it in .915 and .776. The help page describes
the approximation and its limits, and says that
ss_aipe_c_sensitivity(), which is unchanged, simulates
normal errors.common_method_adjusted_r() gives every
marker adjusted correlation with a confidence interval and the t test of
Lindell and Whitney (2001), over one or more marker
correlations. common_method_marker() works from
the correlation matrix alone and does not take N, so its page left the
test to the user. The new function takes N and returns one
row per marker value and variable pair: the correlation, the adjusted
correlation, its interval, and the t test on N - 3 degrees of freedom.
The interval is the Fisher’s Z interval of ci_r() carried
through the adjustment, conditional on the marker correlation. A grid of
values for marker_r is how the dependence of a conclusion
on that correlation is read. With marker_r = NULL the
smallest positive correlation serves as the proxy marker, as in
common_method_marker(). common_method_marker()
is unchanged, and its page now points to the new function. A lower limit
that the adjustment carries below -1 is set to -1; that happens exactly
when the Fisher’s Z lower limit for the correlation falls below 2 r_M -
1, which with a positive marker correlation a small sample can produce
whatever the sign of the correlation.
The missing-information t reference checks the fitted
method. With missing values, mlmr() and
mlmr_mv() refuse the keyword when lavaan fell back to
listwise deletion, including under a Satorra-Bentler test. Complete data
retain N minus the regression parameter count.
fa_confirmatory() refuses robust.sem and
Satorra-Bentler tests under FIML whether values are missing or not, and
checks the fitted method for other fallbacks. The help correctly states
that its default se = "standard" is normal theory for ML
and MLR; an MLR test needs se = "robust.huber.white" for
sandwich standard errors under FIML. With a covariance matrix,
robust.sem needs the asymptotic moment covariance in
NACOV. Alpha keeps its calibrated refusal of the keyword.
The loading coverage maximum is qualified by whether values are missing;
complete-data coverage uses t on N minus one.
Reliability’s missing-information diagnostics use the caller’s item and auxiliary names. The warning about a fit short of its likelihood maximum names the parameter in words. It recommends dividing every item by one common constant, which preserves omega, and each auxiliary by its own, rather than an unavailable refit option.
Categorical MI diagnostics describe the interval actually formed. Under bootstrap then impute, no widening or imputation-count advice is given for an interval that was not formed. An FMI estimated from the minority of replications that returned a value is labeled as such. An even imputation count is named only for completed data sets that failed, rather than an imputation failure alone.
ci_dunnett_t3() gives Dunnett’s T3, the
pairwise comparisons SPSS reports when the variances are not assumed
equal. Each pair has its own separate-variance standard error
and Welch-Satterthwaite degrees of freedom, as in
ci_games_howell(), and its critical value is the
studentized maximum modulus for all a(a - 1)/2 comparisons at those
degrees of freedom (Dunnett, 1980), the value cv_smm()
returns. The adjusted p-value is that distribution’s tail beyond |t|, so
it is below alpha when the interval excludes zero and not otherwise, to
numerical precision; neither is simulated, and the degrees of freedom
are not rounded. The table has the columns and class of the other post
hoc intervals (contrast, mean_difference,
se, df, t_value, the limits, and
p_adjusted, class dmar_post_hoc_ci), and its
display names the adjustment. It reproduces the Dunnett’s T3 output of
SPSS for Chapter 12, Exercise 25 of Maxwell, Delaney, and Kelley (2027)
and agrees with PMCMRplus::dunnettT3Test(). The help
explains why the formula that raises the single t probability to the
power C is Tamhane’s T2 (Tamhane, 1979), a different procedure whose
intervals are never narrower, and when T3 is preferred to Games-Howell.
ci_games_howell() and cv_smm() link to
it.ci_proportion_paired() gives a confidence
interval for the difference between two proportions measured on the same
units, and equivalence_proportion_paired() the score tests
of noninferiority and equivalence on that difference. Two
diagnostic procedures applied to the same patients, or a pass rate
before and after instruction for the same students, give correlated
outcomes, and only the discordant pairs inform the difference, so the
methods for independent proportions do not apply.
ci_proportion_paired() takes either the two 0/1 vectors or
the counts n_10, n_01, and N, and
returns the score interval of Tango (1998), which inverts the score
statistic of Nam (1997). The interval stays inside [-1, 1], is defined
when no pair is discordant, and excludes 0 exactly when McNemar’s test
without a continuity correction rejects. The Wald interval is available
for comparison; the help shows, from the exact coverage of every
possible table, how far short of its level it can fall when discordant
pairs are rare. equivalence_proportion_paired() reports in
the shape of equivalence_c(), with the flag
nonsuperior (the upper limit is below
prop_diff_upper) as the last row and the verdict “Not
relevantly superior only” among its outcomes.
prop_diff_upper alone gives symmetric bounds;
prop_diff_lower alone is a noninferiority test, whose table
leaves the rows that would need an upper bound NA rather
than reading them against a mirrored one. The tests reproduce Nam’s
restricted estimates and score statistic (2.795) for the liver scan data
he takes from McNeil, Keeler, and Adelstein (1975), and the limits match
PropCIs::scoreci.mp(), against which a live comparison
joins tools/oracle_checks.R.ci_ratio_of_means() and
ci_ratio_of_means_paired() give a confidence interval for
the ratio of two means, for independent groups and for two
variables measured on the same units, from the raw data or from summary
statistics. The default is Fieller’s confidence set, which inverts the
t test of mu_1 - rho mu_2 = 0 and is reported as what it is: a
bounded interval, the complement of an interval (whose gap is in the
excluded_lower and excluded_upper rows), or
the whole line, with the denominator’s t that decides which
beside it. Independent groups default to unequal variances, inverting
the Welch-type test with degrees of freedom that change with the ratio,
so that the interval agrees with the test; var_equal = TRUE
gives Fieller’s pooled interval. The paired function takes
cluster for the cluster-robust covariance on Q - 1 degrees
of freedom. A baseline argument gives the ratio of two
advantages over a baseline, the retained fraction of a three-arm
noninferiority design, and mean_baseline,
sd_baseline, and n_baseline give it from the
three arms’ summary statistics; noninferiority at a fraction f holds
exactly when f is below the lower limit of a bounded interval with a
positive denominator, and the help says so rather than testing chosen
fractions. The log-scale interval of the response ratio, and percentile
and BCa bootstrap intervals (B = 10000,
seed = NULL, written in base R), are offered for
comparison; the line under a log or bootstrap interval says when the
Fieller set at the same level is unbounded, where a bounded interval can
undercover. tidy() carries the gap and the form of the set,
so a complement is not read as the whole line, and
results_sentence() writes such a set as what it is (“99.5%
confidence set: every value except those between 0.90 and 147.05”)
rather than as “99.5% CI [-Inf, Inf]”; the sentence of every other table
is unchanged. The log interval of a ratio whose terms are not both
positive, and a bootstrap from summary statistics, are refused with
conditions of class dmar_log_ratio_undefined and
dmar_bootstrap_needs_data, which inherit
dmar_refusal. The paired page works Fieller’s (1954) own
example, whose set takes all three forms at three confidence
levels.equivalence_c() reports
nonsuperiority. A nonsuperior row, the last of the
table, is 1 when the upper confidence limit is below
contrast_upper: the contrast is then shown to be smaller
than the smallest advantage that would matter, the mirror of
noninferior and the decision of the upper one-sided test
(p_upper, which reproduces emmeans::test()
with side = "nonsuperiority"). The verdict gains a sixth
outcome, “Not relevantly superior only”, checked after “Noninferior
only” and before “Inconclusive”, which now means an interval that
contains both bounds. This is the claim a study needs when
contrast_upper is the improvement that would justify a
change. plot_equivalence() applies the same rule through
one shared helper, draws the sixth verdict in the palette’s sixth color,
and keeps the colors of the other five; its legend sets each one-sided
verdict beside the stronger one that implies it.
Noninferiority is planned without an upper bound, and
paired designs are planned. power_equivalence_c()
and ss_power_equivalence_c() require
contrast_upper only for side = "equivalence".
For side = "noninferiority" they take the margin,
contrast_lower, and a call that gives only
contrast_upper, as 1.0.x required of its
delta_upper, still uses it as the margin, so no earlier
call changes but for the names of the bounds. Both take a new last
argument, design = c("independent", "paired"), and
c_weights now defaults to NULL (it is still
required for independent groups). With "paired",
true_sd is the standard deviation of the per-unit contrast
scores (for two conditions, the difference scores), and the standard
error is sigma / sqrt(N) on N - 1 degrees of freedom, where N is the
number of units. Each design has its own size, because the two are
different quantities: power_equivalence_c() takes
n_per_group for independent groups and N, the
number of units and so the total sample size, for a paired design, and
refuses the size of the other design with a condition of class
dmar_argument_invalid. The planner answers in the same
terms: the rows of an independent plan are
necessary_n_per_group, total_N, and
actual_power, as in 1.0.x, and the rows of a paired plan
are necessary_N and actual_power;
tidy() reports the planned size and the power in the same
columns for both. At 50 units, a difference score SD of 1, and a margin
of .5, the noninferiority power is .9672, and 36 units reach .90.
Independent designs give the results they did.
ss_aipe_crossed() plans the numbers of units
and raters for a fully crossed design, every rater scoring every unit,
so that the interval for the mean is narrow enough at the least
cost. Units and raters are both random, so the variance of the
grand mean is var_unit / n_u + var_rater / n_r + var_residual / (n_u
n_r), and with n_r raters no number of units brings the half-width below
z sqrt(var_rater / n_r); the plan reports that rater floor. A design
costs unit_cost per unit, rater_cost per
rater, and judgment_cost per score (by default the number
of scores), and the search prices the smallest feasible number of units
for every number of raters up to max_raters (or for a panel
fixed by n_raters). quantile = "t", the
default, plans the interval ci_crossed() computes, with
Satterthwaite (1946) degrees of freedom from the planning values;
quantile = "normal" reproduces planning that uses the
normal quantile. With the t quantile the planned width is not monotone
in the number of units (the degrees of freedom eventually fall toward
n_r - 1, and past a point the larger quantile outweighs the smaller
variance), and the search takes the first feasible design. An
assurance rests on the chi square approximation to the
plug-in variance, and a plan with an assurance is never smaller than the
plan without one, which already meets the width somewhat more than half
the time; an assurance at or below one half returns that plan. A target
no design within the limits meets is refused with a message naming the
limit that binds, max_units or the number of raters. With
judgments as the only cost the plan lies near the ratio n_u / n_r =
var_unit / var_rater of Marcoulides and Goldstein (1990). The help gives
the mapping to an information retrieval evaluation, in which the units
are topics and the raters are relevance assessors.ss_aipe_crossed_sensitivity() draws
complete arrays from the true components at a planned or specified
design and reports the realized widths, the proportion at or below the
target, and the coverage of the true mean, in the return schema of the
sensitivity family, with seed = NULL and
filename = NULL. The planner’s arguments, its search limits
included, pass through to it.ci_crossed() gives the interval for
the mean of a complete units by raters array: the ANOVA estimates of the
three components, a negative one set to zero, plugged into the variance
of the grand mean, with Satterthwaite degrees of freedom. Its
var_unit, var_rater, and
var_residual rows are the planning values
ss_aipe_crossed() takes. On the four judges and six targets
of Shrout and Fleiss (1979) it reproduces their mean squares and shows
how much wider the interval for the mean over a population of judges is
than one that treats the four judges as fixed.tidy() and glance() read
necessary_n_units as the planned size of a crossed design,
with necessary_n_raters (or
specified_n_raters) kept beside it.A family for the mean of a target measurement that is
costly to obtain, when an inexpensive proxy is available on a large pool
of units. The design is two-phase, or double, sampling (Neyman,
1938): the proxy is measured on the whole pool and the target on a
random subsample of it, the labeled units. ci_two_phase()
gives the interval for the population mean of the target from the
regression estimator, with standard error from the residual variation
among the labeled units and the proxy’s variation over the pool,
optional cluster standard errors whose degrees of freedom count the
clusters among the labeled units, and units with a missing value set
aside one by one; it refuses a target that is constant on the labeled
units rather than report an interval of zero width.
method = "prediction_powered" gives the prediction-powered
estimate of Angelopoulos et al. (2023), which fixes the proxy’s weight
at 1 where the regression estimator learns it. The table reports the
estimate as mean_Y, named for its estimand, the population
mean of the target, with its standard error, degrees of freedom, and
limits, and the weight the estimate gives the proxy mean of the
unlabeled units as proxy_weight (the slope times N / M for
the regression estimate, 1 for the prediction-powered one), beside
slope, correlation, proxy_bias,
se_ratio_target_only, n_labeled, and
n_pool. ss_aipe_two_phase() plans the number
of labeled units for a given pool, for the expected width or with
assurance, always leaving at least one unit with the proxy alone; it
reports beside the plan the number needed without the proxy, which is
the plan ss_aipe_mean() makes, and refuses a width the pool
cannot deliver rather than returning a missing value.
ss_aipe_two_phase_cost() chooses both sizes: given the cost
of a proxy measurement relative to a target measurement, it returns the
least costly design in whole numbers whose ci_two_phase()
interval has an expected width no larger than the one asked for. The
width is that of the interval as it is reported, with the t
quantile on n - 2 degrees of freedom, which is the criterion
ss_aipe_two_phase() uses for a given pool, so that planner,
given the returned pool, returns the returned number of labeled units;
the alternative of measuring the target alone is the
necessary_n_target_only plan of that function, and a
two-phase design is returned only when it costs less. The search is
exact: for each number of labeled units the smallest pool follows from
the width, a convex lower bound on the cost (from the normal quantile,
which the t quantile is never below) says which numbers of
labeled units can still beat the best design found, and those are all
examined. In 2,200 planning sets drawn at random the result equaled an
exhaustive search over pairs of sizes every time. The rows are
necessary_n_labeled, necessary_n_pool (0 when
the design measures the target alone), relative_cost (the
cost of the returned design over that of the target alone, both in whole
numbers), relative_cost_continuous (the same ratio for
fractional sizes under the normal quantile, which is a lower bound and
is 1 at or below threshold_cor),
expected_width, necessary_n_target_only,
threshold_cor, and the inputs. threshold_cor
is exact for fractional sizes only; above it the least costly design in
whole numbers can still be the target alone, and is then returned.
Planning values whose sizes cannot be represented as whole numbers, or
whose search would examine more than 1e8 numbers of labeled units, stop
with an error of class dmar_two_phase_no_design. For a
width of 0.20 with a standard deviation of 1, a correlation of .90, and
a cost ratio of .05, the design is 109 labeled units in a pool of 1,011,
at a cost of 159.55 target measurements against 387 for the target
alone, with an expected width of 0.19999.
ss_aipe_two_phase_sensitivity() evaluates a design by
simulation under the family’s return schema, from a bivariate normal
population or by resampling observed pairs; it reports the population
target mean as true_mean_Y, the simulated estimates of it
as mean_mean_Y, median_mean_Y, and
sd_mean_Y, and the estimate in its per-replication file as
mean_Y. ss_seq_two_phase() evaluates the
sequential stopping rule, which needs no planning values, in the form of
ss_seq_c(); it asks for more labeled units while the target
is constant on them or is an exact linear function of the proxy on them,
where the rule could stop on no evidence. The vocabulary is that of
measurement (target, proxy, labeled units, pool); each help page says in
one sentence how the terms map onto the evaluation of an AI system’s
outputs.
ss_seq_c()’s help page says when to use it and how
it differs from the balanced procedure of Kelley, Darku, and
Chattopadhyay (2018).
ss_aipe_two_phase_cost() refuses a
cost_ratio that is not a single number above 0 and below 1
with a condition of class dmar_cost_ratio_invalid,
which carries the value given.
The regression, correlation, and coefficient of variation functions now name the total sample size, the number of predictors, a planner’s planning values, and the desired width one way. No function is renamed and no result changes for a given model. The DMAR 1.0.x names are not accepted; Breaking Changes in 1.1.0, at the top of this section, lists every old and new name.
p is the number of predictors in the partial
and semipartial correlation family, as it is in
ss_power_R2(), ci_R2(), and
ci_reg_coef(). var_partial_r(),
var_semipartial_r(), expected_partial_r(),
ss_aipe_partial_r(), ss_aipe_semipartial_r(),
ss_aipe_partial_r_sensitivity(), and
ss_aipe_semipartial_r_sensitivity() took J,
the number of variables partialed out. They now take p, the
number of predictors in the regression of Y on X and the variables
partialed out, so p - 1 variables are partialed out,
p is at least 2, and p = J + 1: a partial
correlation of Y and X given two other variables was J = 2
and is p = 3. A categorical variable with a levels
counts as a - 1 predictors. The degrees of freedom are then the
regression ones: the raw-scale variance of a partial correlation divides
by N - p, Fisher’s Z variance is 1 / (N - p - 2), the squared
standard error of the t test of a semipartial correlation is (1
- R2_full) / (N - p - 1), and the planners return N = p + ceiling(z^2 V
/ h^2) for a half-width h. p holds the place J
held, so a 1.0.x call that gave the count by position must add 1 to it
(see Breaking Changes in 1.1.0). In the notation of a model comparison,
q is the number of predictors removed from the full model
to give the restricted model (q = p_full - p_restricted, the predictors
being tested) and the number of variables partialed out is p - q; a
partial or semipartial correlation concerns one predictor, so q = 1 and
p - 1 variables are partialed out. The help pages of the family and of
ss_power_R2_change() say so, and
ss_power_R2_change() reports a row q. No
result changes for a given model.
N is the total sample size.
var_partial_r(), var_semipartial_r(),
expected_partial_r(), ci_r(),
var_r(), expected_r(),
equivalence_r(), ci_cv(), and
var_cv() took n and take N, as
ci_R2(), ci_R(), cv(), and the
regression functions already did; ci_r() and
ci_R(), which share a help page, now document one
N. A size per group is n_per_group and a size
per cell is n_per_cell; n alone remains only
for one size per study or per plotted effect (meta_r(n = )
is a vector of study sizes). The second position is unchanged, so
ci_r(.3, 100) runs as it did;
ci_r(r = .3, n = 100) stops with “unused argument”. The
other functions that took a total sample size as n take
N too (var_alpha(), var_icc(),
ci_proportion(), ci_eigenvalue(), and the rest
of the table in Breaking Changes in 1.1.0). The divisor of
ecvi() was n and is multiplier,
because it is the multiplier of the fit function (N, or N less the
number of groups), not always the total.
A planner names its planning value as its sensitivity
function names the data generating value, true_X.
The planner treats the value as the truth; the sensitivity function
shows what the plan delivers when the truth (true_X)
differs from the value planned with (supposed_X, which was
estimated_X).
The planners are ss_aipe_cv(),
ss_aipe_R2(), ss_power_R2(),
ss_aipe_r(), ss_power_r(),
ss_aipe_partial_r(), ss_aipe_semipartial_r(),
and ss_aipe_equivalence_r(); the sensitivity function
ss_aipe_equivalence_r_sensitivity(), which took
true_r and estimated_r, takes
true_cor and supposed_cor. The planners for a
regression coefficient take true_reg_coef_j
(ss_aipe_reg_coef(), ss_aipe_rc(),
ss_power_reg_coef(), ss_power_rc()) or, for
the standardized coefficient, true_std_reg_coef_j
(ss_aipe_src()) for the coefficient the plan is made for,
and ss_aipe_cv() takes true_mean and
true_sd where it took mu and
sigma; the mean that ss_aipe_cv_sensitivity()
generates data from is true_mean, as in
ss_aipe_mean_sensitivity().
ss_aipe_cv(true_cv = .25, width = .10, conf_level = .99) is
the worked example of Kelley (2007, pp. 759–760; the call is printed on
p. 765), which gives 99; with the exact noncentral t
distribution the function returns 101 (see Changes to Earlier Results).
The full-model planning value that ss_aipe_semipartial_r()
gains in this release is true_R2_full.
var_r(), expected_r(), and
expected_partial_r(), which evaluate a formula at a
population correlation, take it as true_cor
(rho), the name in the planners, and var_R2()
and expected_R2() take the population squared multiple
correlation as true_R2 (population_R2), as
ss_aipe_R2() and ss_power_R2() do;
expected_R2() reports its value in the row
expected_R2 (expected_value_population_R2), as
var_R2() reports var_R2. The null value of
ss_power_r() was rho_0 and is
null_cor, as null_R2 and
null_rmsea are the null values of their planners.
width is the desired width.
ss_aipe_R2_sensitivity(),
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() took w and take
width, as their planners do.
true_cor_YX and
true_cor_XX. ss_aipe_R2_sensitivity()
and ss_power_R2_sensitivity() took rho_yx and
rho_xx, the correlations their data are generated from, and
take true_cor_YX and true_cor_XX, the names in
the coefficient planners.
Every population value of a coefficient planner carries
true_. ss_aipe_reg_coef(),
ss_aipe_rc(), ss_aipe_src(),
ss_power_reg_coef(), and ss_power_rc() took
rho2_Y_X, rho2_j_X_without_j,
rho2_Y_X_without_j, sigma_Y,
sigma_X, rho_XX, and rho_YX
beside b_j or beta_j; each now has the prefix
the coefficient has and names its quantity: true_R2_Y_X,
true_R2_j_X_without_j, true_R2_Y_X_without_j,
true_sd_Y, true_sd_X,
true_cor_XX, and true_cor_YX, beside
true_reg_coef_j or true_std_reg_coef_j.
Cohen’s f squared was cohen_f2 in
ss_power_R2(), ss_power_reg_coef(), and
ss_power_rc() and is true_f2, the name of the
row that reports it in ss_power_R2() and
ss_power_R2_change(); the row of the two coefficient
planners that holds its square root, Cohen’s f, was
effect_size and is true_f.
specified_N in
ss_power_r(). The sample size at which power is
wanted was N and is specified_N, the name in
ss_power_R2() and ss_power_reg_coef() and the
name of the row the function returns.
The wrappers spell their arguments as the functions they
call do. ss_aipe_rc() and
ss_aipe_src() took Rho2_j_X_without_j and
sigma_X_j and take true_R2_j_X_without_j and
true_sd_X, the names in ss_aipe_reg_coef();
ss_power_rc() took Rho2_j_X_without_j and
takes true_R2_j_X_without_j, the name in
ss_power_reg_coef().
The row for the full-model planning value of
ss_aipe_semipartial_r(), new in this release, is
true_R2_full. A row that echoes an argument has the
argument’s name, and one quantity has one row name: the desired width is
the row width (seven planners and
design_consequences() called it width_target
or target_width), the expected width is
expected_width (three planners called it
ci_width_expected), and the null value of
ss_power_r() is null_cor.
necessary_N, specified_N, and
total_N keep their names.
ss_power_R2_change() plans the sample size
for the test that predictors added to a regression model contribute
nothing, the model comparison F test of a restricted
model with p_restricted predictors against a full model
with p_full, and returns the power at a
specified_N. DMAR could plan the omnibus test of all
predictors (ss_power_R2()) and, with fixed predictors, the
test of one coefficient (ss_power_reg_coef()), and had no
planner for a set of added predictors or for one added predictor when
the predictors are random. The effect is given as any two of
true_R2_full, true_R2_restricted, and
true_R2_change, or as true_f2 alone. The
change in R^2 is the squared semipartial correlation of the added
predictor when one is added and the squared multiple semipartial
correlation of the added set when several are; divided by one minus the
restricted model’s R^2 it is the squared multiple partial correlation,
and Cohen’s f^2 is that partial value divided by one minus itself,
(rho^2 full - rho^2 restricted) / (1 - rho^2 full). The help page sets
these relations out, with the error degrees of freedom of the two models
in the notation of Maxwell, Delaney, and Kelley (2027).
With fixed predictors (random_predictors = FALSE) the
test statistic is noncentral F with
p_full - p_restricted and N - p_full - 1
degrees of freedom and noncentrality N f^2. With one added predictor
that is the calculation of ss_power_reg_coef() (the two
agree to 1e-8 on 81 combinations, and both give 50 for a change from .30
to .40 with five predictors and power .80), and with no predictors in
the restricted model it is ss_power_R2() (agreement to
1e-12 on 54 combinations, both predictor models). For five predictors
that explain .25 and four added predictors that raise that to .30, at an
alpha of .01, the power at N = 90 is .2413 and the sample size for power
.80 is 242, the values the G*Power 3.1 manual prints for that design
(.241297 and 242).
With random predictors, the default, the power comes from the
distribution of the sample squared multiple partial correlation, which
is that of a squared multiple correlation with the added predictors
alone and a sample size reduced by the number of predictors in the
restricted model (Gatsonis & Sampson, 1989, p. 518), computed with
Lee’s (1971, Section 5.1) approximation as in
ss_power_R2(). The same design needs 250, and the power at
N = 90 is .2202. Against the exact distribution, the power by Lee’s
approximation differed by at most .0043 over the combinations examined
(squared partial correlations up to .95, an alpha from .001 to .10, and
one or more error degrees of freedom) and by at most .0004 with 25 or
more error degrees of freedom; the necessary sample size differed from
the one the exact distribution gives, by one case, in 20 of 1,500
searches, and in those the power by the exact distribution at the size
returned was within .0011 of desired_power. The same holds
for ss_power_R2(), which shares the search. In a Monte
Carlo with raw data (20,000 replications in each of twelve conditions,
six with random and six with fixed predictors) the rejection rate under
the null hypothesis was within 2.0 standard errors of
alpha_level and the power within 1.9 standard errors of the
value returned.
Cohen’s (1988) noncentrality, f^2 (u + v + 1), is f^2 (N - w) for an
increment with w predictors in the restricted model, where this function
uses N f^2 for fixed predictors; Cohen’s value is the fixed-predictor
call with no restricted model at a sample size of N - w, which the last
example of the help page shows (noncentrality 6.071 and power .2225
where N f^2 gives 6.429 and .2413), and the necessary sample size by
Cohen’s form is at most w above the fixed-predictor answer (247 against
242 in that design). The result has nine rows: necessary_N
or specified_N, actual_power,
noncentral_f_parm (NA with random predictors),
true_f2 (the planning value of Cohen’s f^2),
true_partial_R2, true_R2_change,
q (the number of predictors removed from the full model to
give the restricted model, p_full - p_restricted),
df_1, and df_2.
A change of zero is refused with a classed
condition. ss_power_R2_change() and
ss_power_R2() stop with a refusal of class
dmar_effect_zero when the planning values give an effect of
exactly zero (no sample size gives a test of a true null hypothesis
power above alpha_level); the message says what to give
instead.
var_semipartial_r() gives the first-order
variance of a semipartial correlation, which depends on the full-model
R2. Under multivariate normality, N times the variance of the
sample semipartial correlation tends to (1 - rho_sp2)2 - q (1
- 2 rho_sp^2), where q is the full-model R2 minus rho_sp^2, the squared
multiple correlation of the outcome with the variables partialed out.
DMAR 1.0.x gave (1 - r_sp2)2 / (N - p), which is that
variance only when q = 0. When the variables partialed out explain some
of the outcome, the 1.0.x value is too large if rho_sp^2 is below 1/2
and too small if it is above: at r_sp = .3 with a full-model R2 of .59
the numerator is .4181 where 1.0.x used .8281, and at r_sp = .8 with a
full-model R2 of .94 it is .2136 where 1.0.x used .1296. At r_sp = 0 the
1.0.x value was 1 / (N - p) whatever the R2, while the same function
with R2_full = .64 returned .36 / (N - p - 1). The new
argument under_null (default TRUE) chooses
between the two quantities that need R2_full:
under_null = FALSE returns the first-order variance with
the divisor N - p, the quantity for a Wald interval and for planning,
and the default returns (1 - R2_full) / (N - p - 1) as before. The help
page now describes that default as what it is, the squared standard
error in the t test of X’s coefficient, which equals the
first-order variance at rho_sp = 0; it is not a variance for an interval
about a nonzero r_sp. No value an existing call returned has changed. A
call without R2_full returns what it did and now warns
(class dmar_semipartial_r_no_R2_full) that the value holds
only when the variables partialed out explain none of the outcome. An
R2_full smaller than r_sp squared, which no
data can produce and which 1.0.x accepted (the 1.0.x call
var_semipartial_r(.9, 100, 1, R2_full = .10) returned
.0093), is refused with an error of class
dmar_semipartial_r_incoherent, a dmar_refusal,
that says what to pass when the two values were rounded. The help page
states the limits of the approximation found in simulation: it
understates the variance when |rho_sp| is large and N - p is small, and
the sample semipartial correlation is biased toward zero by roughly
rho_sp (p - 1) / (2N).
The help pages no longer credit (1 - r_sp2)2
to Olkin and Finn (1995) or to Algina and Olejnik (2003). Olkin
and Finn (1995) do not treat the semipartial correlation; their Model A
is the difference between a squared multiple correlation with two
predictors and a squared simple correlation, from whose delta method
variance the result above follows for one variable partialed out (Alf
and Graf, 1999, give the difference between two squared multiple
correlations with any number of predictors), and the pages now say so.
The Yuan and Chan (2011) entry leaves the references of
var_semipartial_r(), since the sentence that cited it is
withdrawn. Algina and Olejnik (2003, p. 317) are cited for what they
state, that for the squared semipartial correlation the required sample
size is largest when the squared multiple correlation of the restricted
model is zero. The sentences that described the 1.0.x form as Fisher’s
(1924) result applied by analogy, and as matching the leading 1/N
behavior of the variance in Aloe and Becker (2012), are withdrawn; Aloe
and Becker (2012) are cited for the effect size.
ss_aipe_semipartial_r() takes
true_R2_full, a planning value of the full-model
R2, and plans from the first-order variance at that value: N =
p + ceiling(z^2 V / w^2), with V as above and w the half-width. For a
planning value of .2, three variables partialed out
(p = 4), and a width of .10, necessary_N is
714 with true_R2_full = .54, where the plan without it (and
DMAR 1.0.x) gives
1421. The result gains a last row, true_R2_full, which
is NA when none was given. Without true_R2_full the plan
uses the largest V over every full-model R2 that is possible with
true_r_sp, so the sample size is sufficient whatever that
R2 is (see Changes to Earlier Results for the cases in which this
differs from 1.0.x). The count of predictors must be a whole number,
where the 1.0.x call
ss_aipe_semipartial_r(r_sp = .3, J = 1.5, width = .2) had
returned a necessary_N of 321.5. A plan with
true_r_sp = 0 and true_R2_full = 1 is refused:
the first-order variance is zero there.
ss_aipe_semipartial_r_sensitivity()
simulates a population in which the variables partialed out can explain
part of the outcome, through the new arguments
true_R2_full (the population’s full-model R2; by default
true_r_sp squared, the population 1.0.x simulated) and
supposed_R2_full (the planning value passed to the
planner). Under normality the distribution of the sample semipartial
correlation depends on the population only through the semipartial
correlation, the full-model R2, p, and N, so the two values describe
every normal population. Its result gains the rows
true_R2_full and supposed_R2_full. Each
interval is now the Wald interval the planner plans for (see Changes to
Earlier Results). In 20,000 replications of a plan for a width of .10,
the intervals missed the true value in .053 of samples at r_sp = .8 with
a full-model R2 of .94 (N = 331, mean width .0995) and in .051 at r_sp =
0 with a full-model R2 of .64 (N = 556, mean width .0998); the Monte
Carlo standard error of each rate is .002. The pages of
ss_aipe_semipartial_r() and
var_semipartial_r() name two further conditions in which
the interval misses too often: a full-model R2 near 1 with a small N, at
any r_sp (.13 at r_sp = .1, a full-model R2 of .99, p = 4, and the
planned N of 12), and |r_sp| above about .9 with variables partialed out
that explain some of the outcome, where the misses fall mostly on one
side even with several hundred observations (.064, .051 of it on one
side, at r_sp = .95, a full-model R2 of .97, p = 2, and N =
395).
ss_aipe_partial_r() with
assurance returns the smallest sample size that attains it,
computed from the distribution of the sample partial
correlation. At a given sample size the width of either
interval the function plans depends on the sample only through r and
narrows as |r| grows, so the interval is no wider than the target
exactly when |r| is at least a value r* that the function solves for.
The probability of that event is taken from the distribution function of
the sample partial correlation, which under multivariate normality is
that of a simple correlation from N - (p - 1) observations, written as
an integral of a central t probability against a beta density
(R’s noncentral t is an approximation above a noncentrality of
37.62, which plans for narrow intervals reach). The function returns the
smallest sample size, at or above the one planned without
assurance, at which the probability is at least
assurance, and two rows are appended to the table:
assurance and assurance_attained, the
probability at the returned sample size. The search does not go below
the plan without assurance, because below it the
probability is not monotone in the sample size. DMAR 1.0.x multiplied
the sample size by qchisq(assurance, df) / df, the step for an interval
whose width is proportional to a standard deviation; here it gave too
few cases when the correlation is large in absolute value and too many
when it is small. With one variable partialed out (p = 2),
a width of .10, and an assurance of .80:
| rho | 1.0.x | now | attained now |
|---|---|---|---|
| 0 | 1586 | 1539 | 1.000 |
| .30 | 1318 | 1310 | .807 |
| .50 | 902 | 915 | .803 |
| .70 | 426 | 448 | .804 |
| .85 | 135 | 151 | .807 |
| .95 | 23 | 28 | .805 |
The page’s example (true_cor = .30, p = 3,
width = .20, assurance = .80) returns 338,
where it returned 344. In 16 further conditions simulated from
regression data (20,000 data sets each, the variables partialed out by
least squares; Monte Carlo standard error .0015 to .0025), the 1.0.x
sample sizes delivered from .139 below the request to .145 above it
(.711 for a requested .85 at rho = .80, and 1.000 for a requested .95 at
rho = .20), and the sample sizes now returned from .002 to .012 above
it, each within .004 of assurance_attained. At rho = 0 the
plan without assurance already meets the width in every
sample, and assurance adds nothing. No argument restores
the earlier sample sizes.
ss_aipe_semipartial_r() with
assurance finds the sample size by simulation, and takes
G and seed. The width of this
interval depends on two statistics, the sample semipartial correlation
and the sample full-model R2, so the function simulates them: for each
sample size tried it draws G (default 20,000) samples from
a multivariate normal population with the planning values, through the
entries of the triangular factor of the Wishart matrix that the two
statistics depend on, and takes the share of intervals no wider than the
target. It returns the smallest sample size found, at or above the one
planned without assurance, at which that share is at least
assurance. Three rows are appended: assurance,
assurance_attained (the share in a fresh set of
G draws at the returned sample size, which had no part in
choosing it), and G; the Monte Carlo standard error of the
attained share is the attribute assurance_mc_se. The answer
has Monte Carlo error: without seed two calls can differ by
a few cases (eight unseeded plans for one condition gave 252 to 254; 40
seeded plans of each of two other conditions gave 470 to 473 and 306 to
310, and the probability at the smallest of those sizes was .897 and
.895 for a requested .90), and assurance_attained can fall
below the request by a few of its standard errors (the standard error is
.003 at the default G and an assurance of .80; in 200
seeded plans the lowest value was .009 below a request of .90). The
sample size returned is therefore the one at which the probability is
about the request, and assurance_attained is an unbiased
estimate of the probability at it. Without true_R2_full the
simulated population is the one the plan without assurance
is made for (variables partialed out that explain none of the outcome
when true_r_sp squared is at most 1/2, a full-model R2 of 1
otherwise). G and seed are checked and used
only with assurance; G must be at least 100,
because the step estimates a probability from those samples
(G = 1 returned an attained assurance of 0 or 1). DMAR
1.0.x multiplied by the same chi square factor as the partial
correlation planner. With an assurance of .80 (now with
seed = 113):
| true_r_sp | true_R2_full | p | width | 1.0.x | now | attained now (fresh draws) |
|---|---|---|---|---|---|---|
| .25 | not given | 4 | .15 | 635 | 621 | .802 |
| .30 | .09 | 3 | .20 | 344 | 337 | .814 |
| .70 | .49 | 3 | .15 | 197 | 214 | .808 |
| .85 | .7225 | 3 | .15 | 65 | 89 | .798 |
| .80 | not given | 2 | .10 | 219 | 370 | .824 |
(DMAR 1.0.x had no true_R2_full; it planned every row
with variables partialed out that explain none of the outcome. In the
last row, where true_r_sp squared is above 1/2, this
release plans without true_R2_full for the full-model R2 at
which the variance is largest, which accounts for most of the change
from 219.) In 11 further conditions checked from data with
ss_aipe_semipartial_r_sensitivity() (5,000 data sets each;
Monte Carlo standard error .003 to .006), the earlier sample sizes
delivered from .37 below the request to .02 above it (.530 for a
requested .90 at a semipartial correlation of .90 and
true_R2_full = .81; .574 and .623 for a requested .80 at
.80 and -.70 with true_R2_full equal to
true_r_sp squared), and the sample sizes now returned
delivered .798 to .819 for .80, .854 to .858 for .85, .906 to .908 for
.90, and .953 for .95. No argument restores the earlier sample sizes.
The page still says that a plan whose sample size is small (a large
|r_sp| with true_R2_full near true_r_sp
squared) should be checked with the sensitivity function, because the
first-order variance is too small there.
The count of variables partialed out must be a whole
number in the seven functions of the partial and semipartial correlation
family (var_partial_r(),
var_semipartial_r(), expected_partial_r(),
ss_aipe_partial_r(), ss_aipe_semipartial_r(),
and the two sensitivity functions). A fractional count
(J = 1.5 in 1.0.x) had been accepted and carried into the
result. The count is now given as the number of predictors,
p, a whole number of at least 2 (see “One Notation for the
Regression and Correlation Functions”). The two sensitivity functions
also check G (a whole number of at least 1) and
specified_N (a whole number of at least p + 3 for the
partial correlation on Fisher’s Z scale, whose interval needs N
- p - 2 > 0, and p + 2 for the raw-scale interval and for the
semipartial).
ss_aipe_partial_r_sensitivity() simulates
the interval that was planned, and takes fisher_z.
The planner plans the raw-scale interval, r plus or minus z (1 - r^2) /
sqrt(N - p), unless fisher_z = TRUE; the sensitivity
function planned that interval and then formed Fisher’s Z
interval in every replication, so its pct_ci_less_width did
not describe the plan. It now takes fisher_z (appended
after filename), passes it to the planner, and forms the
same interval in each replication; the raw-scale limits are not
truncated, and the smallest sample size is p + 2 for the raw-scale
interval and p + 3 for Fisher’s. The table carries the attribute
interval ("raw" or "fisher_z")
and prints a line that names the interval. For
true_cor = supposed_cor = .70, p = 3,
width = .20 (N = 103), 2,000 replications under
set.seed(7): mean width .203,
pct_ci_less_width .470, and non-coverage .046 in DMAR 1.0.0
(Fisher’s interval at the raw-scale plan, with J = 2), and
.200, .516, and .062 now; with fisher_z = TRUE the plan is
107 and the three are .199, .521, and .050. No argument restores the
1.0.x results of this function or of
ss_aipe_semipartial_r_sensitivity().
A full-model R2 that falls below the squared semipartial
correlation only by rounding is accepted, with a warning. A
semipartial correlation reported as .46 has a square of .2116, and the
full-model R2 beside it may be reported as .21. The 1.0.x call
var_semipartial_r(.46, 100, 1, R2_full = .21) (one variable
partialed out, J = 1 by position) returned .00814433 in
DMAR 1.0.0. The pair is checked in this release, since a full-model R2
cannot be below the squared semipartial correlation. A pair that
two-decimal rounding can produce,
R2_full + .005 >= max(0, |r_sp| - .005)^2, passes with a
warning of class dmar_semipartial_r_rounded (fields
R2_full and r_sp):
var_semipartial_r(.46, 100, p = 2, R2_full = .21) returns
.00814433, as it did (p = 2 is the 1.0.x count of 1 plus
1). What the warning says follows what is computed. The squared standard
error of the t test (the default,
under_null = TRUE) is (1 - R2_full) / (N - p -
1), which does not involve r_sp, so the warning says that
the value uses R2_full as given. The first-order variance
(under_null = FALSE), the planner, and the sensitivity
function compute from the part of the full-model R2 that the partialed
variables explain, which is then taken as zero, and their warning says
that the variables partialed out are taken to explain none of the
outcome. A pair further apart (R2_full = .15 with
r_sp = .46) is refused as before, with the class
dmar_semipartial_r_incoherent. The same rule applies in
ss_aipe_semipartial_r() and
ss_aipe_semipartial_r_sensitivity().
The variance on ?var_partial_r and
?ss_aipe_partial_r is named for what it is, and the
variance of the semipartial correlation is credited to the article that
prints it. The pages called (1 - rho2)2 / (N - p)
the “Olkin-Finn (1995) asymptotic variance”. Olkin and Finn (1995,
Equation 3, p. 157) give the first-order variance of a simple
correlation, (1 - rho2)2 / N; the article has no variance of
a partial correlation and no N - p form. The pages now say “first-order
variance”, state that the sample partial correlation given p - 1
variables is distributed as a simple correlation from N - (p - 1)
observations, and say that the divisor N - p is the package’s and
differs from that at order 1/N^2; the message of
var_partial_r() for too small an N reads “Need N > p for
the first-order variance of the partial correlation.”
?var_semipartial_r and ?ss_aipe_semipartial_r
name the first-order variance of Aloe and Becker (2012, Equation 4,
p. 284), which they obtain from Alf and Graf (1999), with N - p in place
of their n, and show that its numerator is the one the functions
compute. No number changes.
vargha_delaney_A(),
cliff_delta(), and
probability_of_superiority_paired() read their observations
through one check, and say what they will not order. The three
statistics depend on the data only through the order of the
observations. They accept numeric and logical vectors, and now ordered
factors with the same levels in the same order, which are read through
their order: a five-category item supplied as an ordered factor gives
the value its integer codes give, and the value any increasing rescoring
of the categories gives. A one-column matrix (the result of
scale()) and a one-dimensional array (the result of
tapply()) are read as the vectors they are, and
Date and POSIXct vectors are ordered by their
numeric values. vargha_delaney_A() did so before;
cliff_delta() and
probability_of_superiority_paired() refused logical vectors
and dates (“must both be numeric vectors”) and now accept them. Infinite
values have their place in the order in all three functions. When an
infinite value was compared with an equal infinite value,
cliff_delta() stopped (“missing value where TRUE/FALSE
needed”) and the paired function returned a table of NA; a
single infinite value was already ordered correctly. Input with no
common order is refused with an error of class
dmar_scores_not_ordered, a dmar_refusal (see
?dmar_metadata): an unordered factor, an ordered factor
beside anything else or beside an ordered factor with other levels, two
difftime objects in different units or one beside anything
else, a Date beside a POSIXct, and character,
complex, list, and matrix input.
Missing values are removed and counted in all
three. vargha_delaney_A() has always removed them
group by group. cliff_delta() stopped on a missing value
with “missing value where TRUE/FALSE needed”, and
probability_of_superiority_paired() returned a table in
which every row was NA, with no message.
cliff_delta() now removes them group by group and the
paired function removes every pair with a missing member. Each result
carries the numbers removed in its n_missing attribute:
c(group_1 = , group_2 = ) for the two-group functions
(through the formula interface of vargha_delaney_A() as
well), one count of pairs for the paired function. A
conf_level that is missing or not a single number gets the
worded error in all three;
cliff_delta(x, y, conf_level = c(.90, .95)) stopped with
“‘length = 2’ in coercion to ‘logical(1)’”.
A grouping factor with an unused level is accepted by the
formula interface of vargha_delaney_A(). The two
groups are the two levels observed with an outcome;
vargha_delaney_A(y ~ g, data = d) with
levels(d$g) of "a", "b",
"c" and no row in "c" stopped with “Grouping
variable must have exactly two levels.”
An estimated variance of 0 is reported the same way by
all three, with a classed warning. The estimated variance is 0
exactly when the groups are completely separated (every observation in
one group exceeds every observation in the other; for pairs, every pair
favors the same member) or every observation has the same value.
cliff_delta() already reported both limits at the estimate
with a warning under complete separation.
vargha_delaney_A(c(4, 5, 6), c(1, 2, 3)) returned
A = 1 with limits [1, 1] and no warning, and
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7))
returned NaN limits and no warning. All three now report
both limits at the estimate and warn, with class
dmar_interval_degenerate and a reason element
("complete_separation" or "no_variation"),
that the limits are degenerate and are not a confidence interval with
the stated coverage. The warning of cliff_delta() keeps its
1.0.x opening words through “completely separated (|delta| = 1)” and
gains the class; cliff_delta() and the other two also warn
when every observation has the same value, where the limits are both at
0 (0.5 for A and the paired probability). Each help page has a
paragraph on the case.
The pairwise comparisons are computed from ranks, not
from a matrix of every pair. vargha_delaney_A()
and cliff_delta() built the matrix of all pairwise
comparisons with outer(), about 36 bytes a pair. The
placements and the proportions of pairs won, lost, and tied now come
from the mid-ranks, minimum ranks, and maximum ranks of the pooled and
the separate samples. The estimators, the variance estimators, and the
tie conventions are the same, and the results agree with the earlier
ones to rounding (see “Changes to Earlier Results”). With 6,000
observations per group, vargha_delaney_A() took 0.52 s and
about 1,240 MB, and now takes 0.03 s and under 30 MB;
cliff_delta() took 0.57 s and about 960 MB, and now takes
0.02 s and under 30 MB.
vargha_delaney_A() refers its test to
Student’s t by default, and offers a studentized permutation
test. The new argument
test = c("t", "permutation", "z") chooses the reference
distribution of the statistic W = (A - 1/2) / SE,
which is the same under all three and is reported in the column
test_statistic (z_value in 1.0.x, a name that
is wrong under the t and permutation references; see the
breaking changes). "t", the default, uses Student’s
t with the Satterthwaite degrees of freedom of the two
placement variances, the Brunner-Munzel test as Schüürhuis, Konietschke,
and Brunner (2025, Equations 7 and 8) give it. "z" uses the
standard normal distribution, as DMAR 1.0.x did.
"permutation" recomputes W over reassignments of
the pooled scores to the two groups, with the arguments
exact, n_resamples, and seed of
randomization_test() and its rules: every reassignment when
there are at most 50,000, random ones otherwise, and a p-value
that is the share of reassignments at least as extreme in absolute
value. The three arguments are read and checked only under
"permutation". Two columns are added after the nine of
1.0.x, df and test; tidy() and
glance() give them after p_value; the table
records the reassignments in a resampling attribute and
prints “Permutation test: all 252 reassignments” or “Permutation test:
10000 random reassignments”. The function reproduces the statistic
3.1375 on 17.683 degrees of freedom, p = 0.005786, of the pain
score example in the help of
lawstat::brunner.munzel.test(), and the estimate 0.8 and
p = 0.0239 of Table 2 of Schüürhuis et al. The permutation
p-value equals a brute-force enumeration in 400 small data sets
with and without ties. The normal reference rejects a true A =
1/2 too often in small samples. At a nominal .05 with normal data
(10,000 samples a condition for z and t, Monte Carlo
standard error .002; 2,000 for the permutation test, standard error
.005) the rates for z, t, and the permutation test
were .086, .071, and .047 with 5 per group; .073, .056, and .047 with 10
per group; .090, .062, and .063 with 8 and 20 and the larger standard
deviation, three times the other, in the smaller group; and .060, .052,
and .046 with 20 per group. The permutation test is exact when the two
distributions are equal and approximate when they differ in spread with
A = 1/2. With completely separated groups it is the only one of
the three that gives a p-value (2/252 for 5 and 5 untied
scores). The help has the formulas, the table, and a paragraph on why
the test and the logit interval can disagree. See “Changes to Earlier
Results”.
Each refusal of input with no common order says why, what
would have been computed, and what to supply. The class is
still dmar_scores_not_ordered. For an unordered factor the
message names the argument and shows the levels of the object and the
codes R stores for them (“here the levels are high, low, mid, stored as
1, 2, 3”), says how the statistic would have ranked them, and gives the
call that states the order. A character vector has a message of its own
(“10” sorts before “9”); it was told only that it was of class
character. An ordered factor beside anything else is told what the other
group is and that the comparison would be of positions in a list of
levels with scores. Two ordered factors with different levels are shown
both lists and, where the two share a label under two codes, that label
and its codes. Every message says “Nothing was computed.” The condition
of an unordered factor or a character vector carries the argument at
fault as arg. Each help page has a paragraph “What is
refused, and why”, and ?vargha_delaney_A and
?cliff_delta an example: for the ratings low, mid, high,
high against low, low, mid, high, the codes of the unordered factors
give A = 0.406 where the ordered levels give 0.656.
The warning of an estimated variance of 0 explains the
case, and states the limit Feng and Cliff (2004) give for it.
The class is still dmar_interval_degenerate, and the
reported limits do not change. The message now says what the two
reported limits are, why a sample without overlap gives a variance
estimate of 0, that this describes the sample and not the population,
how large the samples are, what to report, and that the randomization
test remains valid. For completely separated groups
cliff_delta() and vargha_delaney_A() state the
limit of Feng and Cliff (2004, p. 324, Equations 6 to 8), (n_b - z^2) /
(n_b + z^2) for a delta of 1 with n_b the smaller group size: 0.131 at
the 95 percent level with five in the smaller group, and 0.566 on the
scale of A. That limit is not the one the functions report. The
condition carries estimate, n, and
limit_feng_cliff beside reason. The help of
each function has a section “When the estimated variance is zero” that
names every case and what is returned: complete separation, no
variation, separation produced by infinite values, two constant groups,
groups that only touch (an ordinary, wide interval), and, for pairs,
wins with ties and fewer than three pairs.
A table whose limits are not a confidence interval
carries the mark with it. When the estimated variance is 0 the
three functions set the attribute interval_degenerate to
"complete_separation" or "no_variation";
dmar_metadata() returns it in a field of that name; the
printed table adds “Limits are not a confidence interval: the estimated
variance is 0.”; and tidy() and glance() have
a logical column interval_degenerate, so that a stack of
rows keeps the mark. cliff_delta() and
probability_of_superiority_paired() return tables of the
classes dmar_cliff_delta and
dmar_probability_of_superiority_paired (before
dmar_tbl) for this. results_sentence() reads
the mark; see “Changes to Earlier Results”.
The help of vargha_delaney_A(),
cliff_delta(), and
probability_of_superiority_paired() is rewritten in
full. Each page states the estimand, the sampling unit, and the
kind of interval; the assumptions; the estimator and the variance
estimator with their formulas and sources; how the interval is formed;
what each column or row is; what is refused; every case of a zero
variance; and a reading of an example in words.
?cliff_delta states that the variance is the sample version
of Feng and Cliff’s (2004) Equation 2 (p. 323), whose printed
coefficients (n_1 - 1) and (n_2 - 1) are transposed, with the argument
that shows it, and that their Equation 4 is not used; it no longer cites
Cliff (1996) or says what other R implementations do.
?vargha_delaney_A no longer calls the DeLong, DeLong, and
Clarke-Pearson (1988) estimator “the standard nonparametric variance
(Brunner & Munzel, 2000)”; it says that Schüürhuis, Konietschke, and
Brunner (2025) state the two estimators to be identical. The two pages
show that their intervals are the same construction on two scales with
different variance estimates: for 1:5 against
3:7, A = 0.18 with limits 0.034 and 0.576, which
are -0.931 and 0.151 on the scale of delta, where
cliff_delta() gives -0.64 with limits -0.935 and 0.177.
?probability_of_superiority_paired derives its variance as
that of a mean of independent scores and no longer attributes it to
Brunner and Munzel (2000).
randomization_test() gives the
degenerate-interval warning under its own name. When the
Cliff’s delta rows have no variation or the groups are completely
separated, the warning of class dmar_interval_degenerate
began “cliff_delta():”, a function the user did not call. It now begins
“randomization_test(): for the Cliff’s delta rows (computed by
cliff_delta()),”, with the same class and elements.
The examples of vargha_delaney_A(),
cliff_delta(), randomization_test(), and
randomization_test_paired() show the accepted
call. The call that is refused is described in a comment and is
no longer run under try().
mediate() codes its covariates once and
decides in each resample whether the indirect effect is
defined. The two regressions were refit in every bootstrap
resample from a formula, so the columns of a factor covariate were
recoded from the rows drawn. Three things followed, and each is
repaired. A covariate category with few cases stopped the call with R’s
contrasts can be applied only to factors with 2 or more levels
whenever a resample held no case of it; with 2 of 60 cases in a category
and B = 400 the call failed as a factor or a character
covariate and returned 0.1857 [0.0644, 0.3222] as a 0/1 column. All four
codings now return that table. A resample of this kind leaves the
covariate column redundant, the paths a and b keep
unique values, and the resample is kept; the new attribute
"n_covariate_redundant" counts them. A resample in which
a or b is not estimable (the predictor constant, or
the predictor or the mediator a linear combination of the other
predictors in the rows drawn) has no indirect effect. It is dropped and
counted: the table has a new row B_used after
B (the number of replications the interval rests on;
B for the Monte Carlo interval, NA for Sobel),
the attributes "B", "B_used", and
"n_dropped" hold the counts, dmar_metadata()
now records the bootstrap of a mediate() result, and the
one warning has the class
dmar_bootstrap_replications_dropped and carries
B, B_used, and n_dropped. When
too few replications remain the limits are withheld (see One Rule for
Bootstrap Replications); through 1.0.x the call stopped when fewer than
100 remained. The interval then describes resamples in which the effect
is defined; that is the package’s rule for a replication that returns no
value, not a result that gives the interval its nominal coverage, and
the help page says so. The decision does not depend on the units or the
origin of a variable: a predictor with a large mean relative to its
spread (x + 1e9) stopped with
missing value where TRUE/FALSE needed and now returns the
table of the unshifted predictor.
mediate(ci_method = "boot_bca") refuses the
interval when it is not defined. The acceleration is estimated
by the jackknife, which needs the indirect effect with each case left
out in turn. When one of those was not estimable the value was removed
without a word and the acceleration computed from the rest. The call now
stops with an error of class dmar_bca_undefined (a
dmar_refusal) that names the cases, and likewise when the
bias correction or the acceleration is not finite;
ci_method = "boot_percentile" does not need these
quantities. This is DMAR’s rule for its jackknife acceleration.
mediate() takes column names as they
are. The names given as x, m,
y, and covariates were pasted into a formula,
so a name was read as formula syntax. A column named
predictor score stopped the call with a parse error and one
named outcome$score with
object 'outcome' not found; a covariate column named
w^2 beside w was read as w a
second time and the square left out of the model with no message (in the
data of the independent check, N = 120, the path a was 0.831
where the model with the square gives 0.451); a column named
w:x, I(w), or log(w) was
evaluated as an expression and could use an object w that
was not in data; and a column named . put
every analysis column, the outcome included, among the predictors of the
mediator. No name reaches a formula now, and every such column is the
column it names.
mediate() says what it cannot use.
A factor or character x, m, or y,
and a logical x or m, stopped with
subscript out of bounds; the message now says the column
must be numeric and that a predictor with two categories is supplied
coded 0 and 1. A logical y, which was read as 0 and 1, is
refused with the same message (see Changes to Earlier Results). A
one-column matrix, such as scale() leaves in a data frame,
is read as its column, as before. A column named in two roles
(x and m the same column, y among
the covariates, a covariate named twice) stops with the name. An
infinite value among the analysis variables stops with a message, as
does a predictor that is constant or a linear combination of the
covariates, and a model that leaves the outcome regression no residual
degrees of freedom. A covariate with a single value in the complete
cases, which stopped the call when it was a factor or a character
column, is left out with a message. A Date, date-time, or time
difference predictor is read as a number, as before; a mediator or an
outcome of these classes, which stopped the call (“‘^’ not defined
for”difftime” objects”), is now read the same way. B must
be finite: B = Inf was accepted with
ci_method = "sobel".
mediation_mbco() gives at most one warning
per call for its Monte Carlo intervals. A call in which several
effects lost draws gave one warning for each effect. It now gives one:
of class dmar_bootstrap_insufficient if the interval of any
effect is withheld, otherwise
dmar_bootstrap_replications_dropped if any effect lost
draws, otherwise dmar_bootstrap_exploratory. The message
has one line for each affected effect and then the full account of the
effect with the fewest usable draws, and the condition carries the data
frame effects (term, B_used,
n_dropped, formed,
status).
eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared() take
estimand = c("all", "total", "partial"). The
default, "all", reports every estimand;
"total" or "partial" selects one. These four
functions replace eta_squared_partial(),
omega_squared_partial(), and
ci_eta_squared_partial(), which are removed (see One
Function for Each ANOVA Effect Size for the call that replaces each and
the columns of the tables).
Total eta squared and total omega squared.
eta_squared() reports, beside the partial value SS_effect /
(SS_effect + SS_error), the total value SS_effect / SS_total, with the
corrected total sum of squares, and omega_squared() reports
the total omega squared (SS_effect - df_effect MS_error) / (SS_total +
MS_error) beside the partial value (SS_effect - df_effect MS_error) /
(SS_effect + (N - df_effect) MS_error) (Olejnik & Algina, 2000,
Table 10; Maxwell, Delaney, & Kelley, 2027, Eqs. 7.40 to 7.47). In a
one-way design the two are the same number. For a fitted model the total
uses the sums of squares of anova(); for an
aovlist fit the total eta squared divides by the corrected
total over every stratum, and the total omega squared, which is defined
for designs whose factors are all between subjects, is
NA.
Sums of squares from a published table.
eta_squared() takes SS_effect with
SS_total (the total) and SS_error (the
partial); omega_squared() takes SS_effect,
df_effect, and the error mean square
(MS_error, or SS_error with
df_error), with SS_total (the total) and
N (the partial). SS_effect,
df_effect, and F_value may hold several
effects of one analysis, named for the effects, and the table has a row
for each: the 2 by 4 table of Kim (2016, Table 3) is
eta_squared(SS_effect = c(LIGHT = 34.716, MATERIAL = 1996.998, "LIGHT:MATERIAL" = 1570.389), SS_error = 2261.612, SS_total = 5863.715, df_effect = c(1, 3, 3), df_error = 72).
Each interface takes exactly its own inputs: sums of squares with
F_value, MS_error with SS_error,
and a SS_total smaller than
SS_effect + SS_error are refused, and so are summary
statistics given with a fitted model (see Calls That Are Now
Refused).
An estimand the inputs do not determine is
NA, with the reason. An F statistic and
its degrees of freedom determine the partial value only, since the total
also depends on the design’s other effects. With
estimand = "all" such an estimand is NA, the
estimand_notes attribute gives the reason, and the console
prints it beneath the table. Asked for by name, it is refused with an
error of class dmar_estimand_undetermined, which carries
estimand and reason.
The interval functions report the total where an interval
exists. The noncentral F interval from an effect’s
F is an interval for its partial value, which in a one-way
design is the total, so ci_eta_squared() and
ci_omega_squared() of a fitted one-way model report it for
both. For an effect of a design with other effects, and for an
aovlist fit, the total is reported with NA
limits, and estimand = "total" is refused.
ci_omega_squared() takes an
aovlist fit, computing the partial value of each
effect and converting its limits with the size of its stratum, as
ci_eta_squared() and omega_squared() do;
through 1.0.x it took only a single-stratum fit.
results_sentence() reads an interval held in
columns named for its estimate, such as
eta_squared_partial_ci_lower and
eta_squared_partial_ci_upper in a row of
ci_eta_squared(). A row with one interval gives its
sentence; for a row with two (a one-way model reports the same interval
for the total and the partial value), term names the one to
write. plot_ci() draws each estimate of such a table that
has limits, labeling the rows with the effect and the estimand when
there are two.
The estimand changes no value: every partial eta squared, partial omega squared, and interval is reported as the other entries of this release describe, under the column names of One Function for Each ANOVA Effect Size; the total values are new.
assurance_method in
ss_aipe_smd(), ss_aipe_sc(),
ss_aipe_rc(), ss_aipe_reg_coef(),
ss_aipe_src(), ss_aipe_sc_ancova(), and their
sensitivity functions. With assurance it says how
the planner finds the probability that the realized interval is no wider
than width; without assurance it changes
nothing. "analytic", the default, computes without
simulation.
ss_aipe_smd(), ss_aipe_sc(),
ss_aipe_rc(), and ss_aipe_reg_coef() with its
default noncentral = FALSE, the plan is the smallest sample
size, not below the plan without assurance, at which the
exact probability is at least assurance, and the row
assurance_attained reports that probability. These plans
differ from those of DMAR 1.0.x (see Changes to Earlier Results).ss_aipe_sc_ancova(),
ss_aipe_reg_coef() with noncentral = TRUE, and
both intervals of ss_aipe_src(), the plan is the published
approximation: that of Lai and Kelley (2012) for the ANCOVA contrast,
that of DMAR 1.0.x and MBESS::ss.aipe.reg.coef() for the
noncentral interval, and the chi square plan of Kelley and Maxwell
(2003, Equation 3) for the central interval for the standardized
coefficient. It starts from this release’s plan without
assurance (see Changes to Earlier Results) and draws no
random numbers. It does not compute the probability it attains, which
can be less than the request, so assurance_attained is
NA, and the printed footer says so."simulation", offered only where the probability has no
exact form, returns the smallest sample size, not below the plan without
assurance, at which the probability estimated from
G simulated samples, with seed, is at least
assurance. It reports the estimate as
assurance_attained, with the row G and the
Monte Carlo standard error of the estimate as the attribute
assurance_mc_se. The same samples serve every sample size
tried, so the estimate changes smoothly with the sample size, and the
plan can be a case or two from the one the probability itself would
give."simulation" is refused, with or
without assurance, with a condition of class
dmar_argument_invalid
(arg = "assurance_method") whose message names where it is
offered: ss_aipe_sc_ancova(),
ss_aipe_reg_coef() with noncentral = TRUE,
ss_aipe_src(), and their sensitivity functions, among them
ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() with
noncentral = TRUE. Any other value is refused the same way,
and an abbreviation is not matched.assurance_method and
assurance_probability ("exact",
"simulation", or "not computed") and a line of
the printed footer record how assurance_attained was found.
The sensitivity functions pass the argument to the planner they plan
through. In the planners G and seed are used
and checked only with assurance and
assurance_method = "simulation".The exact probability in ss_aipe_smd() and
ss_aipe_sc(). The interval is no wider than
width when the noncentral t statistic falls
between the two values at which the width equals the target. In
ss_aipe_smd() the probability of that event is an integral
of the normal distribution function over the chi square, which does not
depend on R’s noncentral t; in ss_aipe_sc() it is
the noncentral t probability, computed exactly (see Exact
Noncentral t and Studentized Range Distributions). With equal
tails it depends on the effect only through its magnitude. The plan is
found by doubling a step upward from the plan without
assurance and then bisecting. At true_smd = 0,
width = .8, and assurance = .95 it is 50 per
group (.994636), where the published plan of DMAR 1.0.x, 49, attains
.949878, and at true_smd = 1, width = .5, and
assurance = .99 it is 149 (.990260), where the published
plan, 150, attains .994302. ss_aipe_sc() at
true_std_contrast = 0 and c_weights = c(1, -1)
gives the same 50. In 45 planning sets of ss_aipe_smd()
(|true_smd| of 0 to 2, widths of .25 to 1, assurances of
.80 to .99) and 96 of ss_aipe_sc() (contrasts of -1 to 2,
weights for two, three, and five groups, widths of .5 and 1, equal and
unequal tails) the plan is the smallest sample size found by raising the
sample size one case at a time from the plan without
assurance, and the probability does not fall at any sample
size from that plan to ten past the returned one. With
assurance, ss_aipe_sc() adds the rows
assurance and assurance_attained to the one
row DMAR 1.0.x returned.
The simulated step of
ss_aipe_sc_ancova(),
assurance_method = "simulation" with G
(default 10000) and seed, returns the smallest sample size
per group, not below the plan without assurance, at which
the ci_sc_ancova() interval is no wider than
width with at least that probability, for normal errors and
a normal covariate distributed alike in every group. The probability is
simulated from the exact sampling distribution of the five statistics
the width depends on, so the covariate means term of the standard error
and, with divisor = "s_anova", the sampling of
s_ancova / s_anova enter it, which the published plan of
Lai and Kelley (2012, pp. 358–359) leaves out. This is the a priori
simulation Lai and Kelley (2012, pp. 360–361) gave for the ANOVA
divisor. A contrast and its negative get the same plan. Sample sizes per
group, with the probability that the interval is no wider than the
width:
| standardized contrast, correlation, weights, width | divisor | assurance | 1.0.x and the default of 1.1.0 | "simulation", seed = 1 |
|---|---|---|---|---|
.5, .5, c(1, -1), .5 |
s_ancova |
.80 | 129 (.772) | 130 (.872) |
| the same | s_ancova |
.90 | 130 (.872) | 131 (.933) |
| the same | s_anova |
.80 | 98 (.547) | 104 (.842) |
| the same | s_anova |
.90 | 100 (.658) | 106 (.906) |
.8, .6, c(.5, .5, 0, -1), .6 |
s_anova |
.90 | 47 (.691) | 51 (.937) |
The simulated plan can differ by one from the smallest sample size at
which the probability itself reaches the request (in the third row 103
already gives .803). With assurance,
ss_aipe_sc_ancova() returns the rows assurance
and assurance_attained (NA for the published
plan) and, for the simulated plan, G; DMAR 1.0.x returned
one row. In ss_aipe_sc_ancova_sensitivity() the draws of
the simulated step come from the stream before the replications, so
under set.seed() the replications differ from those of the
default even where the two plans agree.
The plans of the regression coefficient planners with
assurance. For the central interval for the
coefficient in the units of the data, the plan of
ss_aipe_reg_coef() with noncentral = FALSE and
of ss_aipe_rc(), the probability is an exact F
probability for random predictors from a multivariate normal population.
For the standardized coefficient, the noncentral interval of
ss_aipe_reg_coef(noncentral = TRUE) and
ss_aipe_src() and the central one of
ss_aipe_src(noncentral = FALSE),
assurance_method = "simulation" estimates it from
G (default 20000) samples of the exact Wishart sampling
distribution, with seed. Given squared multiple
correlations, which leave the correlation of Y with the
predictor open, the simulated step uses the least favorable value. The
plans, and what they attain, are in Changes to Earlier Results. In
ss_aipe_src_sensitivity(), which now plans for the
standardized coefficient at unit standard deviations, the default plan
is the 1.0.x step only when the supposed variances of Y and of
the targeted predictor are equal (for the noncentral interval, when both
are 1; DMAR 1.0.x refused the noncentral plan otherwise): for
true_var_Y = 4, true_cov_YX = c(.8, .6), and
unit predictor variances correlated .3, supposed correctly,
width = .3, noncentral = FALSE, and
assurance = .9, it gives 181 where DMAR 1.0.x planned 657
(see Changes to Earlier Results). DMAR 1.0.x stopped with an error on
every plan of the three sensitivity functions with one predictor, so
there is no 1.0.x plan for such a call to return.
G and seed in
ss_aipe_reg_coef() and ss_aipe_src(),
for assurance_method = "simulation"; G must be
a whole number of at least 100.
noncentral in
ss_aipe_src(). TRUE by default, the
noncentral interval for the standardized coefficient that
ss_aipe_src() planned in DMAR 1.0.x; FALSE
plans the central interval for the standardized coefficient of Kelley
and Maxwell (2003, Equation 1), whose standard error is formed from the
two sample squared multiple correlations: their plan for the expected
width without assurance (Equation 2), with
assurance their chi square plan (Equation 3) by default,
and with assurance_method = "simulation" the smallest
sample size that attains it, simulated as the noncentral step
is.
New rows in the result of the three regression
coefficient planners: assurance (the request) and
assurance_attained (the probability at the returned sample
size, NA for a published approximation) whenever
assurance is given, and G when the step
simulated.
cohen_f(), ci_srsnr(), and
covmat_from_cfa() say what they compute from each kind of
input. Each computes the same quantity from population values
or from sample summaries and estimates, its help page says what the
result is in each case, and its arguments carry quantity names without
true_ (see Renamed Arguments and the paragraph on these
calculators under Old Names That R Still Matches). The values are
unchanged: every call that still runs, with the arguments renamed,
returns what it returned, and ss_aipe_reliability(), which
builds its population covariance matrix with
covmat_from_cfa(), plans the same sample sizes.The noncentral t distribution is computed
exactly wherever R’s pt(), qt(), and
dt() are not accurate. R’s pt() sums
a series up to a noncentrality of 37.62 and returns a normal
approximation beyond it (and beyond 400,000 degrees of freedom), which
was off by as much as .14 with one degree of freedom, .008 with 38, and
.001 with 1,000; below one degree of freedom it is not accurate even
inside 37.62. DMAR now takes the probability, quantile, or density from
pt(), qt(), or dt() where they
are accurate (within 1e-9 of the exact probability up to a noncentrality
of 37.62 with at most 2,000 degrees of freedom, and up to 33 with more)
and otherwise from an integral of the normal distribution function
against the distribution of the chi square (below one degree of freedom,
against the standard deviation raised to the power df).
ci_nc_t() and every interval and planner built on it, the
critical values of cv_t(), the individual intervals of
limits_of_agreement(), the power of the
ss_power_*() planners whose test is a t test (of
ss_power_indirect_effect(), only the power of the test of
a with X fixed; its other powers average the noncentral
t power over the sums of squares of its predictors) and of
design_consequences(), and the likelihood of the Bayesian
t tests use these routines. Inside the range of
pt() the results are those of pt() and
qt() (to 1e-13 for probabilities and powers). No warning
about the range is given, since no limit is approximate. A limit outside
the range takes about ten times as long to compute as one inside it:
ss_aipe_sm(true_sm = 3, width = .3, assurance = .9) took
0.5 seconds and takes 2.1. Below one degree of freedom, should the
integral or a search fail, the call stops with a condition of class
dmar_computation_failed that names df. The
values that change are in Changes to Earlier Results.
The studentized range is computed exactly wherever R’s
qtukey() and ptukey() are not
accurate. There cv_tukey_hsd(),
ci_tukey_kramer(), ci_games_howell(),
contrast_test(adjust = "tukey"),
qbryant_paulson(), pbryant_paulson(),
cv_bryant_paulson(), and ci_c_ancova_bp()
integrate the probability integral of the range against the chi square
distribution of the error variance, take each critical value as its
root, and take the adjusted p-values from the same distribution
function; elsewhere the values are those of qtukey() and
ptukey(), as before. Where the exact computation is used
and the values that change are in Changes to Earlier Results.
vignette("anova_as_path_model")), fits the single sample
and paired t tests, the one way and two way analyses of
variance (balanced and unbalanced), and the analysis of covariance as
path models, and shows exactly where they reproduce the classical
answers: a factor enters as a set of code variables tested jointly, and
the likelihood divides by N where the classical test divides by
the error degrees of freedom. It uses the depression_bdi
and pygmalion data and draws its path diagrams with base
graphics.holzinger_swineford in place of the first
five mtcars columns, which are not a scale. The tests are
scored on different scales, so their raw-score loadings on one factor
are far from equal and the sum of the raw scores carries that
inequality; the vignette prints the test standard deviations, the
loadings, and the fit with the loadings free and constrained to be
equal, then standardizes the tests and shows alpha and omega nearly
agreeing on the sum of the standardized scores, so the reader sees that
tau-equivalence is a property of the composite as scored. A second
example, labeled as hypothetical, simulates six items that each have
variance 1, three loading .9 on the factor and three loading .1; the
vignette computes the population alpha (0.626) and omega (0.718) from
the loadings and then estimates both from a sample of 300, so the reader
sees a gap larger than the spatial tests’ on items that already share
one variance, where standardizing leaves the gap in place. The vignette
no longer draws on a data set from outside the package.vignette("DMAR")) fits its
one-way ANOVA, for omega squared and for the Tukey pairwise comparisons,
to the drinks_trial outcome on its log scale in place of
PlantGrowth, pooling the trial’s two cohorts and saying
that the pooling is a simplification, and screens six spatial and verbal
tests of holzinger_swineford in place of
attitude, the same six its measurement model takes up.
“Reading DMAR Result Tables” (vignette("dmar_output")) and
“Visualizing Effect Sizes and Distributions With DMAR”
(vignette("effect-size-visualization")) show their
two-factor table, and in the second its forest plot, on
pygmalion, expectancy condition crossed with grade, in
place of ToothGrowth and warpbreaks. Both say
that the rows come from sequential (Type I) sums of squares, which with
these unequal cells differ from Type II or III sums of squares for the
earlier terms, and point to the help page that describes how to supply
those statistics. The visualization vignette also shows the bloomers’
advantage grade by grade, concentrated in grades 1 and 2, which the
omnibus interaction spreads over 5 degrees of freedom.vignette("DMAR")) shows
anova_within() on pygmalion in place of
nlme::Orthodont, with the pretest and the two follow-up IQ
scores as three occasions of a within-subjects factor, reports the
partial eta squared for occasion with its confidence interval from
ci_eta_squared(estimand = "partial") on the same analysis
fitted by aov() with an Error() term, and says
that the analysis pools the two expectancy conditions. DESCRIPTION, the
package citation (citation("DMAR")), the README, and the
vignettes “Introduction to DMAR”, “Moving From MBESS to DMAR”
(vignette("mbess_to_dmar")), and “Visualizing Effect Sizes
and Distributions With DMAR” now describe DMAR in one phrase, as a more
general and greatly expanded reimagining of MBESS. DMAR 1.0.x called it,
in different places, a more modern, more general, and greatly expanded
reimagining, the modern, more general reimplementation and expansion,
and a modern and greatly expanded reimagining. The help pages of
ci_eta_squared(), ci_omega_squared(),
eta_squared(), and omega_squared(), which
suggest car::Anova(object, type = 3) for Type III
statistics, now say that Type III main-effect tests compare unweighted
marginal means only when the factors are coded with sum-to-zero
contrasts (contr.sum, set in the fit or with
options(contrasts = c("contr.sum", "contr.poly"))), as the
help page of car::Anova() warns, and that
factorial_anova() applies that coding itself.seed. seed = NULL is now the last
argument of the sensitivity functions that lacked it
(ss_aipe_c_sensitivity(),
ss_aipe_c_ancova_sensitivity(),
ss_aipe_equivalence_r_sensitivity(),
ss_aipe_icc_sensitivity(),
ss_aipe_indirect_effect_sensitivity(),
ss_aipe_mixed_effects_sensitivity(),
ss_aipe_omega_squared_sensitivity(),
ss_aipe_pcm_sensitivity(),
ss_aipe_r_sensitivity(),
ss_aipe_rc_sensitivity(),
ss_aipe_reg_coef_sensitivity(),
ss_aipe_reliability_sensitivity(),
ss_aipe_rmsea_sensitivity(),
ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity(),
ss_aipe_sem_path_sensitivity(),
ss_aipe_sm_sensitivity(),
ss_aipe_smd_sensitivity(),
ss_aipe_src_sensitivity(),
ss_power_R2_sensitivity(),
ss_power_reg_coef_sensitivity(), and, described in their
own entries, ss_aipe_cliff_delta_sensitivity(),
ss_aipe_partial_r_sensitivity(),
ss_aipe_semipartial_r_sensitivity(),
ss_aipe_cv_sensitivity(),
ss_aipe_R2_sensitivity(), and
ss_aipe_equivalence_smd_sensitivity()), of
procrustes_phi(), of ss_aipe_equivalence_r()
(whose assurance step is a simulation), and of the data generators
simulate_ancova_data(),
simulate_ancova_factorial_data(),
simulate_longitudinal_polynomial(),
simulate_longitudinal_gompertz(),
simulate_longitudinal_logistic(),
simulate_longitudinal_negative_exponential(), and
simulate_longitudinal_richards(). Where a function ends in
..., seed follows it and is given by name.
With the default every one of them draws from the caller’s stream
exactly as before: for each of the 30 functions other than the six with
entries of their own, a call under set.seed() returns the
result it returned before the function had the argument, where no other
entry of this release changes that result, and leaves the stream at the
same point. A number makes the call reproducible, gives what
set.seed() with that number followed by the call gives, and
leaves the caller’s stream as it was. A seed that is not a single number
is an error. For those six sensitivity functions the same holds wherever
their results did not change for another reason (1,314 values of four of
them compared under a common set.seed(), largest difference
0, every run of ss_aipe_cliff_delta_sensitivity() without
separated replications, and, for
ss_aipe_equivalence_smd_sensitivity(), whose interval
changed, the mean_smd, median_smd, and
sd_smd rows and the stream after the call in 4 of 4 cases).
In ss_aipe_semipartial_r_sensitivity(),
ss_aipe_sc_ancova_sensitivity(),
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() the seed is set before the plan,
so it also fixes the simulated assurance step of the planner. With
assurance and assurance_method = "simulation",
ss_aipe_sc_ancova_sensitivity(),
ss_aipe_src_sensitivity(), and
ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() with
noncentral = TRUE take that step from the caller’s stream
when seed is NULL, so the stream after the
call is not where 1.0.x left it; the default, "analytic",
draws nothing for the plan. vargha_delaney_A() takes
seed for its permutation test, and
ss_aipe_semipartial_r(), ss_aipe_sc_ancova(),
ss_aipe_src(), and ss_aipe_reg_coef() with
noncentral = TRUE for their simulated assurance steps.tools/release_checks.R fails when an export that can
draw random numbers has no seed = NULL argument.
tools/precompute_vignettes.R optimizes the PNG figures
without loss after a knit, with oxipng or
optipng when one is installed and otherwise, when
python3 can import Pillow, with
tools/optimize_png.py. That script rewrites a figure only
when the new file decodes, with Pillow and with the script’s own
decoder, to the values and channels of the original, and it copies the
iCCP, eXIf, and pHYs chunks byte for byte; it leaves 16-bit images,
APNGs, and files with other chunks (gAMA, cHRM, sRGB, sBIT, and others)
as they are, and prints a line for each figure. The step says so when
none of the three is available. The 42 vignette figures and
man/figures/logo.png were passed through it: the figures
went from 2,112,862 bytes as knitr wrote them to 1,380,358, and the logo
from 90,909 bytes to 75,027. tools/doi_audit.py compares
every DOI in the help with the metadata registered for it;
tools/release_gate.R prints the number of references that
disagree and does not fail on it.
tests/testthat.R runs every test file only when
NOT_CRAN reads as true the way testthat’s
skip_on_cran() reads it
(isTRUE(as.logical(Sys.getenv("NOT_CRAN")))), and the
curated CRAN subset otherwise. In DMAR 1.0.0 any nonempty value selected
every file, so NOT_CRAN = "false" (or "F",
"1", "yes") ran the whole suite while
skip_on_cran() still skipped its Monte Carlo
blocks.
tools/release_gate.R runs R CMD check
on the CRAN path whatever the shell that starts it has set:
NOT_CRAN unset, TESTTHAT_CPUS = 2, and one
thread for OpenMP, OpenBLAS, MKL, and vecLib, for the check’s process
only. In 1.0.0 the check inherited the caller’s values: from a shell
with NOT_CRAN = true and TESTTHAT_CPUS = 8,
the check’s tests ran the full suite on eight workers instead of the
curated subset on two, and the gate timed that run.
The release gate fails the full local test suite when a test was
skipped, when a test file ran no test, when no expectation passed, or
when the result cannot be read. A skip at the top of a file (eleven test
files open with skip_if_not_installed() for lavaan or boot)
leaves no result for that file, so the gate counts files as well as
tests and its line says how many of the test files ran. It fails the
oracle harness when a snippet was skipped, when none ran, or when the
harness printed no summary line. In 1.0.0 only failures, errors, and
warnings failed the suite, so an empty suite, a skipped test, and a file
skipped from its top level all passed. Only a nonzero exit status failed
the harness, and the harness exits 0 when an oracle package is missing
(“oracle checks: 2 ok, 1 skipped, 0 failed” passed).
When every gate passes, the release gate checks the copy it hands
off against the sha256 on its record and writes
DMAR_<version>.tar.gz.sha256 beside it, in the form
shasum -a 256 -c reads (the hash, two spaces, the file
name). A sidecar an earlier run left there is removed first (emptied
where the folder allows no removal), so a failed delivery leaves no
sidecar that vouches for an older tarball. A copy that fails or does not
match, or a sidecar that cannot be written, ends the run with status 1,
prints “DELIVERY FAILED”, and rewrites the record’s Result line as “ALL
GATES PASSED, DELIVERY FAILED; the hand-off is not certified”. In 1.0.0
the result of the copy was not read: with an unwritable or missing
destination the gate printed “ALL GATES PASSED” and exited 0 with no
tarball delivered, and no sidecar was written.
The release gate’s NOTE gate is named for what it allows: “no NOTE beyond incoming feasibility (new submission, spelling in DESCRIPTION, a URL server error that a re-probe clears) and HTML validation skipped by an old local HTML Tidy”. It reads each item of the feasibility NOTE against those three and fails on any other, naming it (an invalid DOI, a Title not in title case, a URL check that could not run, a CRAN URL not in canonical form). It allows the HTML manual NOTE only when its whole text is R’s message that the local HTML Tidy is too old. It lists every NOTE with its text, so a PASS shows what it tolerated; it judges a NOTE whose result R printed on a line of its own (after a network warning, say) with the step it belongs to; and a NOTE the Status line counts but the log does not show fails it. In 1.0.0 the gate was “no NOTE beyond new-submission and spelling” and passed the old-Tidy NOTE with no detail. It failed a feasibility NOTE only on five phrases (invalid file URIs, Non-FOSS, Insufficient package version, Size of tarball, Days since last update) or on a URL that stayed bad, so an invalid DOI, a Title not in title case, and a URL check that could not run all passed. It also passed an HTML NOTE that said math rendering was skipped as well, and, unlisted, a NOTE whose result was on a line of its own.
Both R CMD check gates of the release gate (no ERROR
or WARNING, and the NOTE gate) fail when the check printed no Status
line. Given a tarball that does not exist, R CMD check
warns (“is neither a file nor directory, skipping”), prints no Status
line, and exits 0. In 1.0.0 a check log with no feasibility step, that
one among them, stopped the gate with the R error “‘from’ must be a
finite number” before any record was written.
The timing limits of the release gate are raised. The whole check may now take 720/403 (about 1.79) times the accepted release’s local time, the ratio that projects 720 s on the r-devel Linux flavor of the CRAN farm, and the CRAN-path tests may take 60 s; the limits were 600/403 (about 1.49) and 30 s. The 1.1.0 candidate checked in 316 s locally (ratio 1.71), with the CRAN-path tests at 46 s; in 284 s (ratio 1.54, a projected 619 s) after the first trimming of the CRAN-path tests and examples; and in 250 s (ratio 1.35, a projected 545 s) after the cuts below, which take the CRAN-path tests to 16 files, prebuild the vignettes, and shorten twenty example pages, three of them for the second time. The anchors of the closed-form planners, added after those measurements, bring the subset to 40 files and add under a second to the CRAN-path tests. The gate still prints the 600 s line of the incoming pretest, which returned the package for overall check time on 5 September 2026, and says when the projection is over it.
On CRAN the curated test subset runs in one process and holds 40
of the 362 test files. The promised detectors
(test-rd_hygiene.R and
test-ss_sensitivity_filename.R) run whole. Fourteen files
each carry a published-value or oracle anchor for one family, which pins
the package’s numerics across platforms: the pinned MBESS values,
metafor, multcomp, car, lavaan, Equation 5.14 of Maxwell, Delaney, and
Kelley, the example of Kelley (2007), for which the exact noncentral t
gives the plans of 101 and 142 where the article reports 99 and 141 from
R’s approximation, the exact noncentral t against its oracle, SIEVER’s
recorded numbers, the published studentized maximum modulus, Yuan et
al. (2016), Shrout and Fleiss (1979), Kelley and Rausch (2011), and
Raudenbush and Liu (2001). Twenty-four more files each keep one block of
a closed-form sample size planner (one that runs no Monte Carlo and fits
no model) on CRAN: the G*Power 3.1 manual for
ss_power_R2_change(), Kelley and Maxwell (2003) for
ss_aipe_src() and ss_aipe_reg_coef(), the
pinned plans of the published approximation of Lai and Kelley (2012) in
ss_aipe_sc_ancova(), the exact probability of an
ss_aipe_sc() plan computed apart from the package, the
exact noncentral t past the range of pt() for
ss_power_c(), ss_power_smd(),
ss_power_indirect_effect(), and
ss_power_composite_ancova_2group(), and, for the others, a
closed form, a scan, or an integral written in the test file.
Closed-form planners that no file of the subset tests are tested in the
full local suite only, among them ss_aipe_c() against the
1.0.0 search over a grid (4.5 s) and ss_aipe_R2() against
MBESS (0.2 s), whose blocks take more than 150 ms,
ss_aipe_reliability() against lavaan,
ss_power_equivalence_c() against the noncentral t, and
ss_aipe_rc(), which calls ss_aipe_reg_coef().
In each anchor file but test-numerical-correctness.R, whose
pinned values all run on CRAN except the ss_aipe_R2()
search, the anchor comes at or near the top, and a top-level
skip_on_cran() after it skips the rest of the file there
with one skip; a skip_on_cran() inside a block costs about
8 ms under the check’s reporter, more than most blocks take to run. The
per-family smoke files left the subset, and with them the
published-value anchors of other functions (those of
combine_p(), omega_squared(), and
fleiss_kappa() among them). They, and every block the CRAN
check skips, run in the full local suite and the release gate, which are
unchanged. In the local check the CRAN-path tests take 5 to 6 s with 527
expectations, where they took 23 s with 4,678, and 47 s (93 s of CPU on
two testthat workers) with 13,578 before the first trim. The
\donttest{} detector collects the tags of the manual in
0.25 s rather than 0.7 s, with the same result on every page. The tests
load lme4 only for the full suite. The release gate’s floor on CRAN-path
expectations is the subset’s count, 527 (it was 2000), so a file that
loses its anchor fails the gate.
The vignettes go to CRAN as prebuilt HTML. Every vignette has
been a precomputed twin since 1.0.0, so the check ran no vignette code,
but its step “checking re-building of vignette outputs” still rendered
the 21 twins through rmarkdown and pandoc: 17 to 18 s of the local
check, which win-builder has run at about three times the local time.
tools/release_gate.R now runs
tools/asis_vignettes.R on its archive of the commit before
R CMD build. In that copy only, the script renders each
twin by the call R CMD build makes, writes
<name>.html.asis beside the HTML for the
R.rsp::asis engine, and removes the twin and its figures.
The check then copies the HTML instead of rendering it: the step takes 7
s locally (5 to 6 s of CPU) in place of 17 to 18 s (13 to 14 s of CPU).
The HTML is byte for byte what the check rendered before, and the 21
vignettes keep their titles. Each vignette’s executable source,
vignettes/<name>.Rmd.orig, ships in the tarball
beside its .html.asis entry: it is the source that
tools/precompute_vignettes.R knits to the twin from which
the HTML is rendered. No vignette engine claims a file whose name ends
in .orig, so R CMD build and
R CMD check neither run nor render it, and
vignette(package = "DMAR") lists the same 21 vignettes. The
tarball is 1.5 MB smaller (7.2 MB against 8.7 MB), because the figures,
and the twins’ markdown, no longer ship beside the HTML that holds them.
A vignette’s source link in browseVignettes() now shows its
R.rsp::asis entry; the twins are in the GitHub repository.
The source tree keeps the twins, and its .Rbuildignore
still leaves out the HTML and the .Rmd.orig sources, so
devtools::build() (with R.rsp installed) and pkgdown work
as before, and no rendered output is committed. R.rsp joins Suggests and
VignetteBuilder. The gate fails in each of these cases:
vignettes/ holds anything other than one
.html.asis entry and one .Rmd.orig source for
each twin, or a shipped .Rmd.orig is not the committed one
byte for byte;inst/doc is not the one rendered from its
twin, or does not record the MD5 of its .Rmd.orig as
committed;A tarball built by hand goes through the same script, and the script’s header gives the commands.
Twenty-seven help pages run their examples faster than in the
first builds of 1.1.0, every demonstration kept and each comment still
giving the count a reported analysis deserves. They were trimmed in two
rounds, ten pages in the first and twenty in the second (three in both);
six of the twenty-seven are new in 1.1.0, and for the other twenty-one
the earlier counts below are those of 1.0.0. The bootstrap and Monte
Carlo counts are lower: ci_bifactor_indices() and
reliability_omega() with B = 10 (were 50 and 40),
mlmr() and sem_exact_fit_bootstrap() with B =
10 (were 20), ss_power_composite_sem(),
ss_aipe_composite_sem(),
ss_aipe_pcm_sensitivity(),
ss_aipe_sem_path_sensitivity(), and
ss_aipe_rmsea_sensitivity() with G = 10 (were 20),
ci_eta_squared_generalized() with B = 100 under
options(dmar.bootstrap_exploratory = TRUE) (was 1000, the
fewest it accepts otherwise), htmt() with B = 500 (was
1000), randomization_test() with
n_resamples = 2000 (was the default 10,000),
ss_aipe_reliability() with G_initial = 20 and
G = 100 (were 50 and 200),
ss_aipe_cv_sensitivity() with G = 100 (was 200), and
ss_aipe_c_ancova_sensitivity() with G = 25 (was 50); the
permutation test of vargha_delaney_A() draws 2,000
reassignments, and the FIML check of
ss_aipe_reliability_sensitivity() runs G = 3. Where a page
fitted the same model twice to show two settings, one call now shows
both: the per-factor descriptor call of fa_confirmatory()
carries the equal loadings and its parallel call the tau-equivalent
constraints, fa_exploratory() shows the target rotation on
the ULS solution, the covariance matrix call of cfa_1()
also names its items, the raw data figure of plot_smd()
carries the group labels and the title, and the 90 percent limits of
limits_of_agreement() use the individual method. Fits that
feed another function skip what it does not use:
point_estimates_only = TRUE in the
factor_congruence() example,
effect_sizes = FALSE in the df_missing()
example, and a fit to the covariance matrix of the complete tests in the
covmat_from_cfa() example. plot_trajectories()
panels six children (was twelve), and now() waits a
twentieth of a second (was a fifth). The examples took 36 s of page
time, where they took 44 s in the first builds. The comments of the
ss_aipe_rmsea_sensitivity() and
ss_aipe_pcm_sensitivity() examples no longer say that the
share of intervals narrower than the target is about one half and that
the mean width is at or under the target, which their output did not
show, and the bias figures of ?ci_bifactor_indices say they
are from the example run with B = 200.
The interval and the plans for the coefficient of variation are
tested against values computed apart from the package.
ci_cv() is checked against the interval Kelley (2007,
p. 758) prints, against limits computed to 30 digits, and at each
one-sided bound. The probability that the interval at an
ss_aipe_cv() plan is no wider than the width is computed in
base R from the noncentral t distribution, by integration over
the normal variate: .5307 and .9966 at the two plans of the article’s
worked example (101 and 142), and .7436 at a plan that the assurance
step leaves below a request of .80 (5, where Table 1 prints 6).
tools/kelley2007_table3_row.R reruns the five plans of the
Table 3 row that shows a misprint, which take about six minutes of
processor time (about two and a half minutes on five cores) and are not
in the suite.
Every author-year mention in a help page now has its entry on the
same page. A check of the release tooling parses every built page and
finds the mentions in its text and examples (“Name (1970)”, “(Name,
1970)”, with chains of authors, et al., possessives, and particles such
as von). The pages its first run flagged are fixed: ten gained the
entries they name (cfa_1() and cfa_2(), for
example, carry the seven estimator and interval references their
inherited arguments cite), and on the others the mention was reworded
instead (example comments that named the Holzinger and Swineford study
now name the holzinger_swineford data, whose page carries
the reference).
The five-group example of the noncentral F interval pages
(ci_pvaf(), ci_snr(), ci_srsnr(),
ci_eta_squared(), ci_omega_squared(),
eta_squared(), and omega_squared(), and in
1.0.x omega_squared_partial(), now removed) and of the
one_way_anova_power vignette, which 1.0.0 credited to
Bargmann (1970) with his name spelled as Venables (1975) spells it, with
one n, now spells it Bargmann; Venables, Fleishman (1980), and Steiger
(2004) used the example in turn. Each of the seven pages gains the
Bargmann (1970) entry, with its editors and pages, and the Venables
(1975) entry, neither of which any of them carried in 1.0.0. The
Fleishman (1980) entry gains its DOI, and eta_squared() and
omega_squared() gain the entry itself. In the vignette the
Bargmann entry gains its editors and pages, and Fleishman and Venables
entries are added. A test pins ci_nc_F() to the .90 limits
of Venables (19.381, 71.551) and of Steiger (19.380, 71.549).
The release tooling fails when a citation form corrected for this release appears again. It keeps a list of the wrong forms, each beside the form that replaced it, and first confirms that the pattern for each form finds examples of the wrong text and passes examples of the right. It then reads the R sources (help sources, comments, and messages), the help pages, the vignette sources, the tests, DESCRIPTION, the package citation, the spelling word list, and the README. The list holds the journal and subtitle of Kelley, Maxwell, and Rausch (2003), the title of the Steiger and Fouladi (1997) chapter, and the editions of Rencher and Christensen (2012) and of Anderson (2003) (all four under Changes to Earlier Results), the spelling of Bargmann above, and, on pages new in 1.1.0, the DOI of the Kiers (1994) entry and of the Saunders (1961) entry, the year and pages of Fieller’s (1941) insulin study, the equation number given for the incremental fit index of Bollen (1989), and the description of DMAR as a “reimplementation and expansion” of MBESS, replaced in the package citation, the README, and the MBESS vignette as described under Teaching. The list also holds seven forms corrected on 1.0.0 pages: the DOI of the Owen (1968) entry, the first page of the Phillips (1990) entry, and the pages of the Hofmann (1977) entry (all three under Changes to Earlier Results), the citation of Sokal and Rohlf (1995) for the expected value of the sample coefficient of variation, which is Expression 3 of Sokal and Braumann (1980), and the entries for Kelley, Darku, and Chattopadhyay (2018), for Lachowicz, Preacher, and Kelley (2018), and for Tofighi and Kelley
(2020) without their issue numbers.
?ss_power_R2 credits the random-predictor
approximation as Lee’s (1971, Section 5.1) three-moment noncentral
F approximation. The help called it a two-moment
Patnaik (1949) approximation; the code is Lee’s Equation 25, which fits
three moments, and Lee (1971) does not cite Patnaik. The Patnaik (1949)
reference is removed from the page. The pages of
ss_power_R2(), ci_R2(), and
ci_correlation() said the distribution of the sample R^2
under random predictors “is given by Lee (1971)”; Lee (1971, p. 117)
credits it to Fisher (1928), and the pages now do, with Lee credited for
the approximation they use.
?vargha_delaney_A no longer calls the
DeLong, DeLong, and Clarke-Pearson (1988) variance estimator
unbiased. The estimator is consistent; it is not unbiased in
general. Its expectation exceeds the variance of the estimate of
A by a nonnegative term of order 1 / (n_1 n_2), which is zero
when both variables take only two values; with three continuous
observations per group from one distribution the variance is 7/108 and
the estimator averages 8/108 (a test enumerates the 20 allocations). The
page now says that A = 0.5 is stochastic equality (Vargha &
Delaney, 2000, p. 102), which is weaker than equality of the two
distributions, and that the reported test is of A = 0.5; cites
Equation 14 on p. 107 for the rank formula, where it cited p. 109; says
that the samples must be independent and points to
probability_of_superiority_paired(); and says that the
limits of cliff_delta() and of
vargha_delaney_A() do not map onto each other exactly,
since the two use different variance estimators. The sentence that
credited Newcombe (2006a, 2006b) with showing better coverage for the
logit interval is removed with its two references, as the claim was not
verified against the papers; the page cites Pohar Perme and Manevski
(2019, doi:10.1177/0962280218814556), who conclude that the
DeLong variance estimator is reliable and that the interval should be
formed on the logit scale. The Hanley and McNeil (1982) entry, which no
sentence cited, is removed, and the DeLong et al. reference gains its
DOI. ?cliff_delta says that its analytic interval is “valid
in large samples” where it said “exact in the large-sample
limit”.
The help of cv() describes the
unbiased = TRUE value as Kelley (2007, Equation 4) does, as
a nearly unbiased estimate, and no longer as unbiased or as a
first-order bias correction, and its title is now “Coefficient of
Variation (Sample Value or Nearly Unbiased Estimate)”. The page gives
the expected value of the sample coefficient from Expression 3 of Sokal
and Braumann (1980, p. 51), which Kelley (2007) reproduces, in place of
the expansion with the single term 1/(4N) that 1.0.0 credited to Sokal
and Rohlf (1995), and it bounds the statement that the sample
coefficient is biased downward: that holds when the population
coefficient is below about one half, and above it the sample coefficient
is too large on average. The multiplier (1 + 1/(4N)) is credited to
Haldane (1955), as both papers credit it. The page now says what the
multiplier leaves (a relative bias of about kappa^2/N - 9/(32 N^2), the
second term derived for the page), gives the bias of both values at a
coefficient of 0.1 and of 0.3 with 10 cases, and says that the corrected
value is the more biased of the two once the coefficient exceeds about
0.35 to 0.4. The sentence that called the interval of
ci_cv() “the McKay (1932) noncentral t based
interval” and the comparison with Vangel (1996) that 1.0.0 attributed to
Kelley (2007), which does not cite Vangel, are replaced:
ci_cv() inverts the noncentral t distribution
(Johnson & Welch, 1940; McKay, 1932; Kelley, 2007), and the chi
square approximations of McKay (1932) and Vangel (1996) are ones it does
not use. Where 1.0.0 pointed to “the corresponding asymptotic variances”
in var_cv(), the page names the one variance that function
returns, the first-order variance of the sample coefficient under
normality. The sample coefficient is written k = s / M, as on
?var_cv and ?ss_aipe_cv, and the entry for
Maxwell, Delaney, and Kelley (2027) no longer points to Chapter 3. The
page gains the Haldane (1955), Johnson and Welch (1940), and Sokal and
Braumann (1980) entries and drops the Sokal and Rohlf (1995) entry,
which nothing on it cites. ci_cv() describes its
cv_unbiased row (c_of_v_unbiased in 1.0.x) as
the nearly unbiased estimate, and the table of the
mbess_to_dmar vignette describes ci_cv() as
the noncentral t interval. No value changes.
DESCRIPTION no longer says that every estimation,
inference, and planning function returns one data frame. Most do; the
regression functions for incomplete data (mlmr(),
mlmr_mv()) and a few others
(correlations_test(), cov_fiml()) return a
list suited to their task, as ?DMAR and the README already
said.
?ss_aipe_crd_es opens with what the six planners
take and return, and its width argument is described as the
width of the interval for the standardized difference (the text said
unstandardized).
?var_cv says where its variance comes from
and how accurate it is. The 1.0.x page said the first row was
McKay’s (1932) noncentral t based approximation, that Vangel
(1996) showed the second gave better coverage, and that the first is
what most planning tables use. McKay (1932) and Vangel (1996) give chi
square approximations to the distribution of a function of the sample
coefficient of variation, which ci_cv() does not use, and
the variance the function returns is not taken from them. The page now
gives the value as the first-order (delta method) variance under
normality, kappa^2 (1/2 + kappa^2) / (N - 1), with its derivation.
Expression 5 of Sokal and Braumann (1980, p. 52) without its third-order
term is kappa^2 / (2 (N - 1)) + kappa^4 / N; the function has N - 1
under both terms, so its value is larger by kappa^4 / (N (N - 1)). The
page states how close the value is to the variance of the sample
coefficient under normality, computed with the sample mean kept away
from zero: for a population coefficient of .30 or less and 10 or more
cases it is too large, and the largest excess is 3.0 percent to one
decimal place; at .50 with 10 cases it is 5.7 percent too small. It also
states how far the value can be from the large-sample variance of
Kelley, Darku, and Chattopadhyay (2018, Equation 30) when the population
is not normal: in the limit it is 39 percent too large for a gamma
population with a coefficient of variation of .80 and 56 percent too
small for a Laplace population with a coefficient of .30. The page gains
the Sokal and Braumann
(1980) and Kelley, Darku, and Chattopadhyay (2018) entries, and its entry for Maxwell, Delaney, and Kelley (2027) no longer points to Chapter 3. Its example prints the variance to six decimal places and sets its square root beside the standard deviation of simulated coefficients, for normal data and for gamma data. The argument and the row have new names and the second row is removed (see Renamed Arguments, Renamed Output Rows and Columns, and Changes to Earlier Results).
?ss_aipe_crd has a title and a first paragraph that
say what the six planners take and return (“Sample Sizes for a Cluster
Randomized Design: Accuracy in Parameter Estimation for the Difference
Between Condition Means”), in the words ?ss_aipe_crd_es
uses for the standardized difference. No number changes.
?ss_seq_c credits each part of the procedure
to its source. The stopping rule is that of Chow and Robbins
(1965) applied to a contrast, and the allocation of the groups is the
cost-optimal allocation of Chattopadhyay, Bandyopadhyay, Kelley, and
Padalunkal (2025, Theorem 1). The page credited the whole fixed-width
procedure to the 2025 article, which plans for noninferiority and
equivalence tests under a budget and has no fixed-width interval;
?ss_seq_c_sensitivity is corrected the same way.
?mr_cv gains the entries for Bao (2009) and for
Bao’s erratum to it, which its part on Equation 22 of Chattopadhyay and
Kelley (2016) cites, and it cites Chattopadhyay and Kelley (2017) and
Kelley (2007), which the page listed without citing. ?mr_cv
and ?mr_smd give the issue of Kelley, Darku, and
Chattopadhyay (2018), 23(2); ?ss_aipe_indirect_effect and
?var_indirect_effect that of Lachowicz, Preacher, and
Kelley (2018), also 23(2); and those two pages and
?ss_aipe_indirect_effect_sensitivity that of Tofighi and
Kelley (2020), 25(4). Both minimum risk pages state the range of
m0 and of gamma, and ?mr_smd
describes A as the structural cost divided by the square of
epsilon (it said that the maximum allowable difference was
the square of epsilon).
?ci_cv states that its interval assumes a
normal population. The interval is exact for a random sample
from a normal population with a positive mean and is not robust to
departures from normality, which a larger sample does not repair: in
simulations of the 95 percent interval (coefficients of variation of .3
and .5, sample sizes of 30 and 200) coverage was .81 to .85 for Laplace
data, .90 to .94 for lognormal data, and .96 to .97 for gamma data. The
page says when the upper limit is infinite and why that interval has the
stated coverage, and describes the tail columns as the requested tail
areas. Its references gain Kelley, Darku, and Chattopadhyay (2018),
whose large-sample interval the page mentions, and lose the Journal
of Statistical Software article, which the page does not cite.
?ss_aipe_cv writes Equation 3 of Kelley (2007) as the
approximation it is (Sokal & Braumann, 1980, p. 51, Expression 3;
for a coefficient of .25 it is above the expected value of the sample
coefficient by 3.1 percent at N = 5 and 0.1 percent at N = 20), and it
and ?ss_aipe_cv_sensitivity say what the plans deliver for
gamma and lognormal populations. On the three pages the pointers to
Maxwell, Delaney, and Kelley (2027) name what they point to: Chapter 3
on confidence intervals for effect sizes, and on
?ss_aipe_cv_sensitivity Sections 3.9.1.5 and 4.9 in place
of Section 3.5. No number changes.
Sizes given as R integers no longer overflow. R
multiplies two integers as an integer, and the product becomes NA, with
the warning “NAs produced by integer overflow”, once it passes 2^31 - 1
= 2,147,483,647: with 46,341 or more in each of two groups, and also
with unequal groups such as 40,000 and 60,000. ci_smd(),
ci_smd_c(), ci_mahalanobis(),
convert_d_r(), convert_r_d(),
convert_delta_lambda(),
convert_lambda_delta(), cles(),
nnt_from_smd(), proportion_of_superiority(),
meta_smd(), ss_power_smd(),
var_smd(), and bayes_independent_t()
multiplied the two group sizes this way when they were given as integers
(such as 50000L, or a count from length()),
and plot_smd() and ss_aipe_smd_sensitivity()
pass their sizes to ci_smd(). In DMAR 1.0.0, at 50,000 in
each group and at 40,000 and 60,000,
ci_smd(smd = .2, n_1 = 50000L, n_2 = 50000L),
ci_smd_c(smd_c = .2, n_C = 50000L, n_E = 50000L),
cles(), nnt_from_smd(),
proportion_of_superiority(), and
plot_smd(smd = .2, n_1 = 50000L, n_2 = 50000L) then stopped
with “missing value where TRUE/FALSE needed”,
ci_mahalanobis() stopped with a message that its limits
“could not be computed for the observed F-statistic NA”,
meta_smd() with “Supply a positive sampling variance for
each effect size.”, and bayes_independent_t() with “‘from’
must be a finite number”, while ci_smd() and
ci_smd_c() given ncp, the four conversion
functions, and var_smd() returned NA.
ss_aipe_smd_sensitivity(true_smd = .1, specified_n_per_group = 50000L, G = 3)
(true_delta and n_per_group in 1.0.x), which
uses its one size for both groups, stopped with “missing value where
TRUE/FALSE needed”, as it did with 46341L (with
46340L it gave its table). ss_power_smd()
overflowed even with sizes given as doubles, since it turned a given
size into an integer, and reported the power as NA.
randomization_test() and plot_smd() count the
group sizes of raw data as integers (randomization_test()
with sum() of a logical vector, plot_smd()
with length()), so they stopped with “missing value where
TRUE/FALSE needed” for any data whose two group sizes have a product
above 2^31 - 1 (46,341 scores in each group, or 100,000 and 21,475),
whatever the type of the scores. The sizes are now multiplied as
doubles, and the results are those for the same sizes given as doubles:
ci_smd_c(smd_c = .2, n_C = 50000L, n_E = 50000L) gives the
limits 0.188 and 0.212,
convert_r_d(.2, n_1 = 50000L, n_2 = 50000L) gives 0.408,
ss_power_smd(.02, n_1 = 50000, n_2 = 50000) gives the power
.885, randomization_test() reports the standardized mean
difference with its interval, plot_smd() draws the interval
that ci_smd() gives for the sizes as doubles, and
ss_aipe_smd_sensitivity(true_smd = .1, specified_n_per_group = 50000L, width = .03, G = 3, seed = 1)
gives the table of the same call with 50000 (mean interval
width 0.0248). The specified-size paths of
ss_power_reg_coef() and ss_power_R2() turned a
size above 2^31 - 1 into NA (“NAs introduced by coercion to integer
range”): in DMAR 1.0.0
ss_power_reg_coef(true_f2 = 1e-9, p = 3, specified_N = 3e9)
(cohen_f2 in 1.0.x) returned NA for the power, and
ss_power_R2(true_R2 = 1e-9, p = 3, specified_N = 3e9)
(population_R2 in 1.0.x) stopped with “missing value where
TRUE/FALSE needed”. Such a size is now used as given: the first call
gives the power .410, and the second .275.
krippendorff_alpha() gives alpha for 46,342
or more pairable values. It formed N (N - 1), where N is the
number of pairable values counted with length(), as a
product of integers, which becomes NA from N = 46,342 on (46,342 times
46,341 is 2,147,534,622). Take 23,171 units rated twice on three nominal
categories, drawn after set.seed(7) as
r1 <- sample(1:3, 23171, TRUE) and
r2 <- ifelse(runif(23171) < .7, r1, sample(1:3, 23171, TRUE)).
In DMAR 1.0.0 krippendorff_alpha(cbind(r1, r2)) summed the
distances over all 1.07e9 pairs of values and then stopped with “missing
value where TRUE/FALSE needed”, after the warning “NAs produced by
integer overflow”. The product is now formed as a double, and the same
call, after the same sum over pairs, gives alpha .698 (D_expected .667),
the values from the frequencies of the ratings.
The Paule-Mandel estimate of tau2 and the
Q-profile interval for tau2 no longer depend on the units
of the effect sizes. meta_es(),
meta_smd(), meta_r(), and
plot_forest() with method = "pm" found the PM
estimate, and every estimator found the limits for tau2 and
I2, by root searches that stopped within 1.5e-8 of the root in the units
of tau2, a distance larger than tau2 for
effect sizes in small units. For y = (-1, 0, 1) with sampling variances
.01 the PM estimate is 0.99 and the interval 0.2611 to 39.49; for the
same effect sizes times 1e-6 (variances times 1e-12) the estimate,
divided by 1e-12, was 10.1 and the interval 101 to 101. The searches are
now made on tau2 divided by the scale of the data, as the
REML search already was, and stop at a distance relative to the root, so
the results for effect sizes multiplied by c are those for the original
ones times c squared (tau2 and its limits) or c (the pooled
estimate, its standard error, and the prediction limits). For the 12
simulated studies of the package’s tests the PM tau2 was
0.0431063198 and is 0.0431063219; metafor at its default tolerance gives
0.04310566, so a comparison with its default output can differ in the
sixth significant digit.
ss_aipe_smd() no longer passes on the
warnings of R’s pt() in its assurance step. For a
negative true_smd it asked pt() for the lower
tail of a probability near 1 (15 warnings
full precision may not have been achieved in 'pnt{final}'
at true_smd = -1, width = .5, and
assurance = .90). Each tail is now computed directly as a
small probability, and those plans give no warning.
equivalence_smd() and
ss_aipe_equivalence_smd_sensitivity() give their intervals
when the product of the two group sizes passes 2^31 - 1. The
product of the two group sizes, stored as integers, overflowed R’s
integer range (“NAs produced by integer overflow”).
equivalence_smd() with integer n_1 and
n_2, or with raw data, whose two sizes had a product above
2,147,483,647 (46,341 or more in each group, or 40,000 and 60,000)
returned NA for the t statistics, the
p-values, the limits, and equivalent; it now
returns them
(equivalence_smd(smd = .1, n_1 = 46341L, n_2 = 46341L, smd_lower = .2, smd_upper = .2)
gives the limits 0.089 and 0.111). The sensitivity function, whose sizes
are integers, stopped there with “missing value where TRUE/FALSE needed”
(at 54,112 per group, the plan for width = .02 at a 90
percent level) and now returns its table.
ss_aipe_equivalence_smd() and
ss_aipe_equivalence_smd_sensitivity() check their numeric
arguments on entry. A missing, not-a-number, or infinite
true_smd or supposed_smd, a missing or
vector-valued equivalence bound, a specified_n_per_group
that is missing, fractional, or below 2, and a missing
width are refused with a condition of class
dmar_argument_invalid that names the argument. DMAR 1.0.0
stopped on most of these with R’s “missing value where TRUE/FALSE
needed”, refused an infinite planning value as “Required n exceeded 1e6
per group; check inputs”, and truncated a per-group size of 10.7 to 10
without a word.
reliability_H() with fit checks
the covariance matrix of the estimates as the factor analysis tables
do. Where that matrix comes from an information matrix that is
singular in the directions the fit’s equality constraints leave free,
the standard error and limits are NA, with the warning of
class dmar_result_qualified that
fa_confirmatory() gives for the same fit; a two-item factor
uncorrelated in the sample with the other factor had H =
.853 with a standard error of 533.8 and the interval [0, 1], with no
warning. Where lavaan computed no standard errors,
reliability_H() stopped with “argument is of length zero”
and now returns the estimate with the same warning (the
vcov_basis attribute is then "none").
The check of the information matrix in
fa_confirmatory(), fa_bifactor(),
fa_second_order(), and reliability_H() reads a
fit with ceq.simple = TRUE correctly. Parameters
that share a label then share one free number and carry no equality
constraint, and the check took the information of the unconstrained
parameters for that of the model, so models identified by their labels
(a two-item factor with equal loadings, two groups with loadings equal
by label, a group.equal fit) were reported
singular.
fa_confirmatory() and
fa_bifactor() with output = "standardized" and
se = "none" return the standardized estimates,
with the se, z, pvalue,
ci.lower, and ci.upper columns
NA; they stopped with “missing value where TRUE/FALSE
needed”. fa_second_order(), which returned the estimates
without those columns, has them too.
bifactor_indices() no longer asks lavaan for
the standard errors of the standardized solution, which it does not
use, so lavaan’s warning “Could not compute standard errors!”
no longer reaches the caller beside the Heywood warning of an improper
fit. The indices are unchanged.
htmt() refuses blocks that are not disjoint
sets of distinct items under distinct names. An item listed
twice in a block, an item placed in two blocks, or two blocks under one
name gave a ratio without a word: for nine simulated items whose blocks
A = x1, x2, x3 and B = x4, x5, x6 give .615,
listing x1 twice in A gave .544, placing
x3 in both blocks .944, and naming both blocks
A 1.533. These, and an item whose name belongs to two or
more columns of data, are refused before anything is
computed, with an error of class dmar_htmt_blocks_invalid
whose reason element names the check
("repeated", "shared",
"construct_names", or "data_names") and whose
blocks and items elements name what is at
issue. Disjoint blocks give the same ratios and bounds as
before.
The standardized cluster randomized planners
(ss_aipe_crd_es_*()) count and report the simulated data
sets that give no usable interval, and pass on the warnings of their
fits. The counts, the reasons, and the first error message are
in the new attribute simulation (also read with
dmar_metadata()) and on the printed line “Replications”; a
warning of class dmar_replications_without_interval gives
the counts when data sets failed, and a design at which no data set gave
an interval, or at which an assurance cannot be reached, is refused with
an error of class dmar_computation_failed, where the width
had been NaN or the search had stopped with “missing value
where TRUE/FALSE needed”. The planners wrapped their search in
suppressWarnings(); each kind of warning a fit gives is now
given once, as a warning of class dmar_result_qualified
that counts the data sets that gave it. A fit of optimizer status 0 or 1
is used, and any other status counts as a failed data set. The models
are built with mxExpectationRAM(),
mxFitFunctionML(), and mxFitFunctionAlgebra()
in place of the deprecated mxRAMObjective() and
mxAlgebraObjective(), with identical limits, and the socket
cluster used on Windows is stopped however the simulation ends.
The searches of the standardized cluster randomized
planners end. The search upward for the cluster size at a fixed
number of clusters had no end when the width leveled off above the
target. It now simulates cluster sizes up to 100 above the size the
unstandardized difference needs (above its own start when that size is
beyond 100,000) and never above 500 (a search that would start above 500
is refused with reason
"cluster_size_beyond_reach", whose field
lower_bound says whether the start is a lower bound on the
size the target needs), and a width still above the target is refused
with reason "width_not_reached", the narrowest
width found, and its Monte Carlo interval. Three designs in a row with
no finite width end either search (reason
"no_finite_width"), and a call simulates at most 500
designs (reason "search_budget"); each is an
error of class dmar_computation_failed. The two planners
over both sizes pick the first of tied widths or costs, where a tie
stopped them with an error from if().
The cluster randomized searches meet a budget or a width
equal to the end of a bracket. A cost or a width equal to the
target at the end of a bracket of the coarse search stopped
ss_aipe_crd_n_clusters_fixed_budget(budget = 20000, cluster_size = 20, cluster_cost = 0, individual_cost = 1),
ss_aipe_crd_n_individuals_fixed_budget(budget = 10000, n_clusters = 10, cluster_cost = 0, individual_cost = 1),
and the two width planners at the width of 1000 clusters of 20
(0.1216020) and of 100 clusters of 1000 (0.3557016) with “‘from’ must be
a finite number” in DMAR 1.0.0 and the development versions of 1.1.0.
Each now returns 1000, at a cost equal to the budget or a width equal to
the target; their ss_aipe_crd_es_*() counterparts share the
budget searches. In a grid of 480 budget searches the 51 (number of
clusters) and 29 (cluster size) that stopped now return a size, and no
other result changed; in 1,152 cluster size searches from a width, at
targets that are not the width of a design, no result changed. A width
equal to the target inside a bracket also meets it now (see “Changes to
Earlier Results”).
ss_aipe_crd_both_fixed_width() returns a
numeric value column. The cluster size search gave
the string “> 100000” for a number of clusters that cannot reach the
width, which made the whole column character whenever one of the numbers
of clusters tried could not:
ss_aipe_crd_both_fixed_width(width = .5, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8)
returned the strings “80”, “7”, “0.499131775130474”, and “1360” in DMAR
1.0.0 and the development versions of 1.1.0 (the development versions
also “0.5”, the width asked for). The search gives Inf
internally, and the plan is the same numbers (80 clusters of 7, budget
1360), now numeric in the table and in tidy() and
glance(): in 720 calls the number of clusters, the cluster
size, and the budget are identical, and the width differs only where the
development versions returned strings, by at most 2.5e-15 relative.
Given back as width, the width a plan reports now gives
that plan:
ss_aipe_crd_both_fixed_width(width = 1, cluster_cost = 30, individual_cost = 1, pr_treat = .5, ...)
plans 20 clusters of 10 for a budget of 800, and the string
“0.994337792651072” that DMAR 1.0.0 and the development versions
reported for its width is below that width and, given back, gave 19
clusters of 13 for 817. Every cluster randomized planner returns a
numeric value column.
ss_aipe_crd_n_individuals_fixed_width() refuses a width no
cluster size up to 100,000 reaches with reason
"width_not_reachable" (class
dmar_argument_invalid, as in the development versions), the
width the interval falls toward as the clusters grow as
width_limit, and whether a larger cluster size would reach
it; the message said only that the target was “impossible to achieve”.
ss_aipe_crd_es_n_individuals_fixed_width() gives its
refusal of a search whose unstandardized start is beyond 100,000 the
reason "cluster_size_beyond_reach", with
start = Inf.
The searches over both sizes no longer stop with an error
from R.
ss_aipe_crd_both_fixed_budget(budget = 20000, cluster_cost = 1000, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8)
stopped with “invalid ‘type’ (list) of argument” in DMAR 1.0.0 and the
development versions of 1.1.0, because no number of clusters it tried
was within the budget, and pr_treat = .005 stopped with
“wrong sign in ‘by’ argument”, because the floor of two clusters in each
condition (400) was above the end of the search (200). Both are refused
with reason "budget_infeasible" (20 clusters
of 2 cost 20,040; 400 cost 400,800). In 720 calls of the budget planner
the 92 that stopped this way are refused, and the 628 plans are
identical.
ss_aipe_crd_es_both_fixed_budget(budget = 1000, cluster_cost = 5, individual_cost = 1, true_es = .5, true_icc = .9, pr_treat = .5, G = 2, seed = 113)
simulated one more cluster than the budget buys, with a cluster size of
NA, and stopped with “invalid ‘times’ argument” in both;
that design is no longer simulated, and the call returns 142 clusters of
2.
The documented variance pairs (between, ICC) and (within,
ICC) plan in the six unstandardized cluster randomized
planners.
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_icc = .2)
and the same call with true_var_within = .8, true_icc = .2
stopped with “missing value where TRUE/FALSE needed” in DMAR 1.0.0 and
the development versions of 1.1.0; both return 18 clusters, as the other
pairs do. The variances are resolved once per call, where they had been
resolved at every number of clusters tried; every call with consistent
inputs gives the same plan (36 calls over the six planners and six
pairs, identical but for the string rounding of
ss_aipe_crd_both_fixed_width() above).
mediation_mbco() refuses a joint hypothesis
one of whose expressions adds no restriction to the others,
before any model is fit, with a condition of class
dmar_argument_invalid (arg = "hypotheses")
whose redundant and kept elements name the
expressions. The degrees of freedom of a joint test were the number of
expressions, so a repeated expression, a multiple or combination of
others, or one that is 0 wherever the others are counted as a
restriction: in the single mediator model fit to 100 simulated cases
(seed 731), hypotheses = list(h = c("a", "a")) gave the
statistic of "a", 4.980, with 2 degrees of freedom
(p = .0829, where "a" gives p = .0256),
and c("a*b", "b"), whose null set is that of
"b", gave p = .0316, where "b" gives
.00858. The rank of the Jacobian of the constraints at the unrestricted
estimates, and points of the null set of the other expressions found
branch by branch, find such an expression.
c("a*b", "a*cp"), whose null set is a = 0 or
b = cp = 0, is still tested with 2 degrees of freedom. See
Constraints that add no restriction in
?mediation_mbco.
mediation_mbco() fits badly scaled variables
in a rescaled metric. When the standard deviations of the
observed variables differ by a factor of 1000 or more (the rule of
lavaan’s rescale_data option), each variable is divided by
a power of 2 near its standard deviation for the OpenMx fits, and every
result is returned in the original metric; the divisors are in the new
"rescale" attribute. The optimizers had stopped far from
the maximum: with the seed-731 data, x multiplied by 1000,
and m divided by 1000, the statistic for "a"
was 1316.2 and is 4.980, the least squares value, and that of the
indirect effect 7.56 and now 4.980.
mediation_mbco() fits the null model of a
conditional indirect effect at a probe value branch by branch
((a0 + a1*w)*b is 0 where b = 0 or where
a0 + a1*w = 0): in a first-stage moderated mediation of 300
simulated cases (seed 3812) the statistic at a moderator value of 1 was
37.294 and is 33.631, the smaller of the two branch statistics. When
every branch fit is valid, the smallest is the constrained minimum and
the constrained searches, which could end just off the null set, are not
run.
mediation_mbco() reports the test of a total
effect (and of other sums and differences of effects) where its null fit
had failed (reported as NA with a warning in 10 of
100 simulated data sets of 150 cases with every coefficient 0). A
constraint without branches is now also started from the full-model
estimates moved onto its null set.
?mediation_mbco states the conditions under
which the likelihood ratio statistic has its chi square
reference, and that the null hypothesis of no indirect effect
is not regular where both coefficients are zero. There, in the single
mediator model with observed variables, the test at the .05 level
rejects with probability .05 squared, .0025, in large samples (between
.0025 and .05 when the estimates are correlated), so it is conservative.
The page gives this limit beside the rejection rates of .025 to .075
that Tofighi and Kelley (2020) report from simulations in which one
coefficient of the indirect effect is zero, and says what each
describes.
ss_power_reg_coef() and
ss_power_rc() check p on every route:
a single whole number of at least 1. p = 2.5 returned
necessary_N = 47.5 (true_f2 = .2),
p = 0 and p = -1 returned 47, and the route
through true_reg_coef_j stopped with “argument is of length
zero” when p was left out. Each is now refused with a
condition of class dmar_argument_invalid
(arg = "p").
ss_power_reg_coef() and
ss_power_rc() plan for a single predictor given through
true_cor_XX and true_cor_YX.
ss_power_reg_coef(true_cor_XX = matrix(1, 1, 1), true_cor_YX = .3, which_predictor = 1)
stopped with 'a' is 0-diml and returns
necessary_N = 93, the result of
true_f2 = .09 / .91 with p = 1.
which_predictor must be a whole number from 1 to
p, true_cor_XX a correlation matrix, and
true_cor_YX possible with it (a squared multiple
correlation below 1); each is checked by name.
ss_power_reg_coef() and
ss_power_rc() refuse at once a sample size for an effect of
0 when desired_power is above
alpha_level, with a condition of class
dmar_effect_zero, since the power is then
alpha_level at every sample size; the search ran to N =
10,000,000 (about 40 seconds) and stopped with class
dmar_computation_failed. An effect too small for any sample
size up to 10,000,000 to reach desired_power is refused at
once with that class. Both inherit dmar_refusal.
ss_aipe_reg_coef() plans for one predictor
on the covariance structure path.
true_cor_XX = matrix(1) with p = 1 stopped
with “the condition has length > 1” (and, with
true_reg_coef_j given,
object 'b_j_tmp' not found); it now returns the plan of the
direct path (327 for a correlation of .4 and width = .2),
and p and which_predictor follow from a 1 x 1
matrix.
ss_aipe_reg_coef() and
ss_aipe_rc() check a supplied true_reg_coef_j
on the covariance structure path against the coefficient on the scale
true_sd_Y and true_sd_X describe.
With true_sd_Y = 2 and true_sd_X = 1 the
correct coefficient .681 was refused and the standardized .341 accepted.
The plan is unchanged; the refusal now gives both values.
ss_aipe_rc(), ss_aipe_src(),
and ci_src() state the conditions of the function they call
in their own argument names (for example
true_std_reg_coef_j, not true_reg_coef_j, in
ss_aipe_src()). ss_aipe_src() refuses
true_sd_Y or true_sd_X other than 1 itself,
saying that it plans for the standardized coefficient and
ss_aipe_rc() for the unstandardized one; the refusal had
spoken of ‘noncentral=TRUE’, which the user had not given.
ci_src() without a coefficient is refused by
name, where it stopped with “argument is of length zero”; it
takes the coefficient from t with its standard error, from
t with R2_Y_X and
R2_j_X_without_j, or from the three squared multiple
correlations. The warning that a standardized coefficient is unusually
large applies beyond 1.1 in either direction, not only to positive
values. ci_rc() and ci_reg_coef() take
reg_coef_j = NULL by default, for the routes on which the
other inputs determine the coefficient, and ci_rc() reports
that coefficient between its limits.
ss_aipe_reg_coef_sensitivity() refuses a
missing true_var_Y (with
noncentral = FALSE it built a covariance matrix of the
wrong size), a which_predictor outside 1 to p,
and a specified_N that is not a whole number of at least
p + 2.
ss_aipe_src_sensitivity() differs from DMAR
1.0.0 when the variances of Y and of the targeted predictor
differ. For true_var_Y = 4,
true_cov_YX = c(.8, .6), unit predictor variances
correlated .3, supposed correctly, width = .3, and
noncentral = FALSE, DMAR 1.0.0 planned 610 (657 with
assurance = .9) and judged the intervals against .681; this
release plans 156 (with assurance = .9, 181 by default and
171 with assurance_method = "simulation" and
seed = 1) and judges them against .341. A call with one
predictor and specified_N, which DMAR 1.0.0 answered,
changes as well: with true_var_Y = 4,
true_cov_YX = .8, true_cov_XX = matrix(1), and
specified_N = 100 it judged the intervals against .8, where
this release judges them against .4. With equal variances of Y
and of the targeted predictor (unit ones, or every variance 4), whatever
the variances of the other predictors, and two or more predictors, the
calls without assurance and those with
assurance and the default assurance_method
return the 1.0.0 table, to rounding error, with the same draws at the
same sample size, plan a case below 1.0.0 (156 and 181 here, 157 and 182
in 1.0.0; see Changes to Earlier Results).
ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() plan with the supposed standard
deviations, as 1.0.x did (609 here without assurance and
672, the exact central plan, with assurance = .9, where
1.0.x planned 610 and 657), and need no such change; for their plans
with one predictor see the next entry.
The regression sensitivity functions plan the calls DMAR
1.0.x refused. In ss_aipe_src_sensitivity() a
noncentral plan (the default) from a supposed variance of Y or
of the targeted predictor other than 1 was refused by DMAR 1.0.0 (“Since
you’ve specified ‘noncentral=TRUE’, all variances should be one”); for
true_var_Y = 4 and assurance = .9 this release
plans 179 (176 with assurance_method = "simulation" and
seed = 1). (DMAR 1.0.0 answered a call in which only a
predictor that is not targeted has a variance other than 1.) DMAR 1.0.0
also stopped on every call of
ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() that planned the sample size with
one predictor, for either interval and with or without
assurance (“argument is of length zero”, or, for a
noncentral plan from a supposed variance of Y other than 1, the
refusal above). This release plans these calls: for
true_var_Y = 1, true_cov_YX = .4,
true_cov_XX = matrix(1), width = .3,
noncentral = FALSE, assurance = .9, and
seed = 1, ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() plan 178 and
ss_aipe_src_sensitivity() 171 (159 with
assurance_method = "simulation"), and the noncentral plans
are 172 (165 with "simulation"). With
true_var_Y = 4 and true_cov_YX = .8,
ss_aipe_src_sensitivity() plans the same sample sizes, and
the central plans of the other two functions are 639.
The upper quantiles of the intervals and plans of a
regression coefficient, a contrast, and a correlation are taken in the
upper tail. ci_reg_coef() with
noncentral = FALSE, ci_c(),
ci_r(), the central plans of
ss_aipe_reg_coef(), ss_aipe_rc(), and
ss_aipe_src() and the exact central assurance step of the
first two, the simulated central step of ss_aipe_src(),
ss_aipe_sc(), and ss_aipe_sc_ancova() formed
an upper quantile as qt(1 - alpha, df) or
qnorm(1 - alpha). 1 - alpha rounds a small tail area away,
and it is exactly 1 for a tail of 2^-54 or less, where the quantile is
Inf. That is how ci_c() and the central
ci_reg_coef() returned an upper limit of Inf,
and ci_r() a limit of 1 or -1, at a tail of 1e-20 in DMAR
1.0.0 and the development versions of 1.1.0. Each quantile is now
qt(alpha, df, lower.tail = FALSE) (or the
qnorm() counterpart), exact at any tail. A tail that small
is now refused (see Calls That Are Now Refused); above the floor the
change is in the last digits at the usual levels and in about the eighth
or ninth significant digit near the floor (see Changes to Earlier
Results).
The help of the regression coefficient planners describes
what they plan for. In ?ss_aipe_reg_coef,
?ss_aipe_src, and ?ss_aipe_rc, the plan
without assurance was called the plan for the expected
width. It is the plan for the width at the expected values of the two
sample squared multiple correlations, which is not the mean width over
samples, as Details already defined it. On the squared multiple
correlations path, the noncentral and standardized assurance step was
said to take the smallest probability over every value of the
correlation of Y with the targeted predictor that the planning
values allow, “so the sample size suffices whatever it is”. It takes the
smallest over 21 equally spaced values across that range, an approximate
minimum: between two of them the probability can be a little lower, and
the plan a little short of the request. With p = 2 the two ends
are the only possible values. The help also said the simulated estimate
rises with N. The search assumes that it does not decrease,
which held in every set checked, although single simulated intervals
were wider at N + 1 in 3,208 of 95 million comparisons. No
result changes.
ss_aipe_R2() plans for an abbreviated
which_width. The function checked
which_width with char.expand() and discarded
the expanded value, so an abbreviation matched none of the three plans
and the function returned NULL. In DMAR 1.0.0,
ss_aipe_R2(true_R2 = .5, width = .15, which_width = "L", p = 5, random_predictors = FALSE)
(population_R2 in 1.0.x) returned NULL, as did
"F", "Fu", "Lo",
"U", and "Up". It now gives 117, the plan of
"Lower"; "F" gives 264, the plan of
"Full", and "Up" gives 13, the plan of
"Upper". An abbreviation is expanded as
match.arg() expands it in the other planners whose
which_width lists the three values.
cohen_f() refuses a specification it cannot
read as one. It computes f from means
with var_within, from eta_squared, or from
sd_means with sd_within, and an argument of
one of these given with an argument of another was ignored: in DMAR
1.0.x, cohen_f(eta_squared = .10, mu = c(1, 2)) returned
.333, the f of eta squared. Such a call, one half of a pair
given alone, n_per_group without means, a
vector where one number is needed, and a missing value among
means are refused with a condition of class
dmar_argument_invalid that names the arguments.
ci_srsnr() with means refuses
df_1, df_2, or N that disagree
with the group sizes, and group sizes that are not whole
numbers. They were replaced without a word:
ci_srsnr(means = c(94, 91, 92, 83), var_within = 67.375, n_per_group = 6, N = 100)
ran with N = 24, and n_per_group = c(7.5, 8, 9) ran with
21.5 denominator degrees of freedom. var_within or
n_per_group without means, which was ignored,
is refused, and the refusal of equal group means (an F of 0)
names means, not F_value.
omega_squared() returns a DMAR
table of class dmar_anova_es, as
eta_squared() does, so it prints with the package’s display
and has tidy() and glance() methods; through
1.0.x it returned a plain data.frame.
omega_squared() and
ci_omega_squared() accept F_value = 0
and report 0; through 1.0.x they refused it.
plot_ci() draws the estimates of
ci_eta_squared_generalized(), where it plotted the
midpoint of each interval.
ci_proportion() keeps every Wilson limit in
[0, 1], with a lower limit of exactly 0 at 0 successes and an upper
limit of exactly 1 at N successes. In floating point those
limits came back a few units in the last place either side of 0 or 1 (as
low as -5.6e-17, as high as 1 + 2.2e-16). Every other limit is
unchanged. The Wald interval is not clamped.
ci_games_howell() gives limits and a
p-value for a pair whose Welch degrees of freedom fall below
2, where R’s qtukey() and ptukey()
return NaN. With groups a = (1.2, 2.9), b = (3.1, 5.5), and
c = (4.0, 7.9, 6.1), the b - a pair (1.80 df) had NA limits
and p-value and has [-7.516, 12.016] and p = .4499.
With two groups the interval and p-value equal those of
t.test(var.equal = FALSE) below 2 df as well.
cv_tukey_hsd(), ci_tukey_kramer(),
contrast_test(adjust = "tukey"), and
qbryant_paulson(n_covariates = 0) likewise return the
critical value below 2 error df
(cv_tukey_hsd(.05, df = 1.5, n_groups = 4) is 10.189), and
a critical value R’s qtukey() fails to find
(qtukey(.25, 51, 32) stops with “convergence failed”) is
computed as the root of the exact distribution function.
contrast_test() no longer returns
NA standard errors when a group has a single
observation, whose variance is NA in R though its
weight in the pooled error variance is zero. For 20 groups of which two
have a second observation (2 error df), the standard error of the “2 -
1” contrast was NA and is 0.2008, from the residual mean
square of the one-way fit.
format_p() refuses a digits_p
that is not a whole number from 1 to 324, with a condition of
class dmar_argument_invalid;
format_p(.123456, digits_p = 3.9) returned “0.123”. Its
help page says that it formats and does not check its input. A refusal
that names the value given shows it to 15 significant digits when 7
would show a different number (a digits_p of 1 + 1e-12 was
named as “1”).
ci_sc() judges contrast weights by their
values. It compared the sum of the positive weights with 1 by
identical(), which also compares the storage type and every
bit: c_weights = c(1L, -1L) or -1:1 stopped
with “Please use fractions to specify the contrast weights”, and
c(.7, .2, .1, -1) was refused while
c(.1, .2, .7, -1) was accepted. ci_sc(),
ss_aipe_sc(), and ss_aipe_sc_sensitivity() now
apply one rule to the values of the weights (the positive values sum to
1 within 1e-8), with one set of messages, and weights given as text are
refused with class dmar_argument_invalid (they stopped with
“invalid ‘type’ (character) of argument”).
ss_aipe_sc() with assurance
plans a negative contrast with unequal tails as its magnitude with
alpha_lower and alpha_upper
exchanged, since the ci_sc() interval at
-t is the reflection of the interval at t with the
tails exchanged. It kept the tails as given, so the interval could be
too wide more often than the request allowed (see Changes to Earlier
Results).
ss_aipe_sc_ancova() plans wide targets and
correlations near 1. Its plan without assurance
let the sample size fall below 2 per group, where the error degrees of
freedom are 0 or below:
ss_aipe_sc_ancova(true_std_contrast = .5, true_cor = .5, divisor = "s_anova", c_weights = c(1, -1), width = 3.6)
did not return, and width = 5 with
divisor = "s_ancova", or true_cor = .999 with
divisor = "s_anova" and width = .5, stopped
with “missing value where TRUE/FALSE needed”. The plan is now at least 2
per group, and these give 2, 2, and 6, the smallest sample sizes that
meet the widths (see Changes to Earlier Results).
ss_aipe_sc() and
ss_aipe_sc_ancova() find the plan without
assurance by doubling a step and bisecting, not by
raising the sample size by one, so a narrow target no longer takes
seconds or minutes, and a plan the 1.0.0 search did not finish, such as
width = 1e-5 (about 3e11 per group), is returned. The plans
of ss_aipe_sc() are unchanged:
ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .01),
316,921 per group, took 3 seconds and takes under a second.
ss_aipe_sc_ancova() starts the search at 2 per group, and
some of its plans are smaller (see Changes to Earlier Results).
ss_power_indirect_effect() returns a sample
size that attains desired_power in data drawn from the
model it plans for. It took the power of each path test with
the noncentrality fixed at the population standard error, as if the sum
of squares of X were N and the residual sum of squares of the mediator
given X were its expectation, and took the joint significance power as
the product of the two path powers. In a study both sums of squares vary
from sample to sample (that of the mediator in every design, since the
mediator is measured, and that of X when X is measured), and the two
tests are not independent, since the residual sum of squares of the
mediator is the error sum of squares of the test of a and the
predictor sum of squares of the test of b. So the reported
power was too high: its plan of 65 for paths of .39 and power .80 has a
power of .7675. The power of each path test is now its noncentral
t power averaged over the chi square distributions of those
sums of squares, and the joint significance power is the probability
that both tests reject, computed given the residual sum of squares of
the mediator, given which the two tests are independent; the plan is 69,
with power .8010. The new argument random_predictors,
TRUE by default as in ss_power_R2_change() and
ci_R(), makes X random (normal), as when it is measured;
FALSE fixes X by design, with variance 1 computed with
divisor N (two groups of equal size, for example). The mediator is
random under either setting, and the setting is recorded in the
"random_predictors" attribute. The help said that the tests
validated the result against raw-data simulation; the test then allowed
a relative error of 6 percent, and it now requires agreement within four
Monte Carlo standard errors, at the plan and at one case fewer, for
either test and either predictor model.
The Sobel power of
ss_power_indirect_effect() is the probability that the
Sobel test rejects. DMAR 1.0.x referred |ab| / se_ab, with
population standard errors, to the normal distribution. The first-order
Sobel statistic that mediate() uses is t_a t_b / sqrt(t_a^2
+ t_b^2), in the t statistics of the two paths; it is not
normal with unit variance, and the approximation erred in either
direction: at N = 96 with paths of .39 it gave .8026 where the power is
.8818, so its plan for .80 was 96 where 84 suffice, and at N = 12 with
paths of .6 and .5 and a direct effect of .2 it gave .3165 where the
power is .1226. The power is now the probability that 1 / t_a^2 + 1 /
t_b^2 is below 1 / z^2 under the joint sampling distribution of the two
statistics.
The probability is an integral over |t_a| that
integrate() computes in pieces, and development versions of
1.1.0, which no release carried, computed it with three faults that are
now fixed. First, they stopped with an error of class
dmar_computation_failed whenever integrate()
returned a piece with a roundoff status, as it does for pieces next to
the critical value z that are worth almost nothing (less than 1e-45 in
this call):
ss_power_indirect_effect(.2, .1, specified_N = 10, alpha_level = .999, method = "sobel")
stopped with “The power of the Sobel test at N = 10 could not be
computed reliably: the numerical integral over the distribution of the
path t statistics failed (roundoff error is detected in the
extrapolation table).” and is .9983 (DMAR 1.0.x gave .9990; see Changes
to Earlier Results). In a grid of 5,760 Sobel powers
(alpha_level from .000001 to .99999, paths from .02 to .95,
N from 10 to 1,000,000, X random and fixed) they refused 664, at
alpha_level .995 (two, both at N = 1,000,000), .998, .999,
.9995, .9999, and .99999, and a path near 1 brought the refusal to
ordinary levels:
ss_power_indirect_effect(.39, .999999, specified_N = 10, alpha_level = .5, method = "sobel")
was refused and is .7457. A piece with either roundoff status (“roundoff
error was detected” or “roundoff error is detected in the extrapolation
table”) is now accepted when its estimated absolute error is finite and
at most its share of 1e-9 and its value is at least minus that share, so
the power carries an estimated quadrature error below 1.2e-9; the
largest such piece in the grid was worth 2.9e-19. A piece returned “OK”
is accepted whatever its error estimate, as before. A roundoff piece
over its share is refused with class
dmar_computation_failed and the message “… failed
(<status>, with an estimated absolute error of
<e> against a budget of <b> for
this piece). The joint significance test (method =”joint_significance”)
does not need this integral.”; every other status of
integrate() is refused as before. Second, at
alpha_level of .99999 and above, the lower end of the
integral can round to just below z, where the integrand was NaN: 40 of
the 664, among them
ss_power_indirect_effect(.95, .3, specified_N = 10, alpha_level = .99999, method = "sobel"),
stopped with “… failed (non-finite function value).” and the warning
“NaNs produced”. The Sobel test cannot reject at |t_a| <= z, so the
integrand is now 0 there, and that call gives .999991. Third, with X
random and |a| near 1, the density of |t_a| has a separate peak at each
node of the distribution of the sum of squares of X, and
integrate() stepped over some of them:
ss_power_indirect_effect(.999999, .5, specified_N = 10, alpha_level = .5, method = "sobel")
gave .3521 where the power is .5216, and the plan
ss_power_indirect_effect(.999999, .5, desired_power = .52, alpha_level = .5, method = "sobel")
stopped at ten million where it is 10. Once two neighboring peaks stand
more than 2 standard deviations apart, which takes |a| above about .9,
each peak gets pieces of its own. No call in the grid is now refused or
warns, and none of 270 plans at alpha_level up to .99999 is
refused (126 were). In the grid of 5,760 the powers that both computed
agree within 3e-15. In a second grid of 2,520 powers with X random (|a|
from .85 to .999999, b from .02 to .999999, N from 10 to 100,000,
alpha_level from .001 to .999), of which they refused 130,
the powers that both computed agree within 5e-14 at |a| up to .999; at
|a| = .9999 they move by up to 2e-7, and at .99999 and .999999 by up to
.23 and .97, and the plan
ss_power_indirect_effect(.99999, .2, desired_power = .80, method = "sobel")
moves from 9,418,685 to 9,418,684. The new values agree with independent
integrals that share no code with the function to 3e-11 or better
(alpha_level up to .999999, paths up to .999999), and with
2,000,000-draw Monte Carlo estimates within 4 standard errors in 360
settings with |a| from .9999 to .999999.
The Bryant-Paulson distribution with covariates is
computed at large error df and with many covariates. From about
4 x 10^7 error df with one covariate (5.6 x 10^7 with six, 10^8 with
30), pbryant_paulson() returned about 0
(pbryant_paulson(3.7, n_covariates = 1, n_groups = 4, df = 1e8)
was 5.6e-25, where the probability is .956), and
qbryant_paulson(), cv_bryant_paulson(), and
ci_c_ancova_bp() stopped with uniroot()’s “f()
values at end points not of opposite sign” or, at the edge of that
range, returned a critical value that was too large
(qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 4.09e7)
was 4.5327, where it is 3.6332). df = Inf stopped with
“non-finite function value”. With 40 or more covariates the same
failures began at smaller df: about 3 x 10^6 with 40 covariates, 10^6
with 150, 1.3 x 10^5 with 200, and 1.6 x 10^4 with 300. At large df the
weight of the integral over the covariate shrinkage factor lies within a
few multiples of sqrt(p / df) of one end of its range, where the
quadrature over the whole range did not find it, and with many
covariates the factors of the weight overflowed and underflowed apart.
The integral now stops where all but 1e-15 of the weight lies, the
weight is computed on the log scale, and with df = Inf the
functions return the studentized range at infinite df, as they do with
no covariates.
The help of seed in the standardized cluster
randomized planners, mediate(),
plot_mediation_mbco(),
ci_eta_squared_generalized(),
mediation_mbco(), ss_aipe_indirect_effect(),
and measurement_alignment() says what
seed = NULL does: it draws from the current random
number stream and advances it. The help said that it left the
generator’s state unchanged.
Help pages state what the code computes and attribute
claims to their sources. Documentation only; no function,
argument, output row, or value changed. ?ci_src and
?ci_reg_coef say that the standard error of a standardized
coefficient treats the sample standard deviations as fixed (Kelley &
Maxwell, 2003, pp. 315–316), so it is approximate, and give the coverage
of the intervals in simulations; ?ci_c_ancova_bp says how
much wider the Bryant-Paulson intervals are than Tukey’s (84 percent
with three covariates and 2 error df at alpha = .05);
?var_alpha attributes the N - 1 form to van Zyl,
Neudecker, and Nel
(2000) and the N - 2 form to Bonett (2003), as the rows now
do; and ?ss_aipe_pcm, ?irt_grm,
?irt_information, ?mr_cv,
?mr_smd, ?skewness, ?kurtosis,
?descriptives, and the pages of four power functions define
their arguments, columns, and formulas. Statements the code, the data,
or the cited source do not bear out are corrected or attributed on the
pages of correlations_test(),
ci_ratio_of_means(),
ci_ratio_of_means_paired(), ci_smd(),
expected_smd(), var_smd(),
var_r(), expected_r(),
ci_proportion(), krippendorff_alpha(),
gwet_ac(), equivalence_c(),
fa_confirmatory(), obrien_test(),
manova_split_plot(), welch_t(),
summary_t_test(), randomization_test_paired(),
teacher_expectancy, prime_time_achievement,
and holzinger_swineford, and ?DMAR and the
pages of more than thirty functions and data sets describe the package
without praise or unsupported claims (“modern”, “easy to use”, “the most
general”, and the like).
Refusal messages state the requirement instead of the
user’s error. ci_reg_coef() said “You have not
properly specified ‘alpha_lower’ correctly.” and says “‘alpha_lower’
must be between 0 and .5.”; convert_R2_f() and
convert_f_R2() say “Specify both ‘df_1’ and ‘df_2’, or omit
both and specify ‘N’ and ‘p’.”; and reliability_alpha() and
reliability_omega() say that DMAR offers no ADF interval
under that treatment of missing data, rather than that none exists. The
condition classes are unchanged.
The vignettes, the README, and DESCRIPTION are
corrected. Documentation only. Rd markup that pandoc left out
of the HTML of four vignettes is Markdown now. The vignettes and the
README no longer say that every ci_*() function inverts a
noncentral distribution, that every plot shows a confidence interval and
the sample size, or that every result is a data frame of
term and value rows; their tables of argument
names give the 1.1.0 names; they and the DESCRIPTION describe the
package and MBESS in factual terms; and the README’s suggested citation
gives the version as 1.1.0.
The refusals of ci_cv() name the argument at
issue. With cv and no N, the
condition’s arg is N, the argument that is
missing. data given together with cv is
refused with “Give either ‘data’, or ‘cv’ with ‘N’, not both.”, where
DMAR 1.0.0 asked for the sample size. A mean or an
sd that is not a single finite number is refused by name;
ci_cv(mean = 5, sd = "a", N = 30) (n in 1.0.x)
stopped with “non-numeric argument to binary operator”. A sample mean
that is not positive has one message, whether it was given as
mean or computed from data, which states the
value and that the interval requires a positive mean and a positive
standard deviation; it no longer asks whether taking the absolute value
of the mean makes sense.
ss_aipe_cv() plans on the expected value of
the sample coefficient of variation again, as Kelley (2007)
does. The method (Kelley, 2007, p. 758) evaluates the width at
E[k | kappa, N] of the paper’s Equation 3 at each candidate sample size.
DMAR 1.0.x evaluated it at the population value multiplied by (1 +
1/(4N)), the nearly unbiased estimate of Equation 4, which is above the
population value where the expected value is below it. The cause is
inherited: MBESS through 3.3.3 computed Equation 3 inside
cv(unbiased = TRUE), which the planner called;
cv() was later changed to the Equation 4 multiplier and the
planner followed it without being edited. Of 34 expected-width
conditions of the paper’s Table 1 the earlier value was the tabled one
in 10 and one larger in 24; of 16 conditions with assurance .99 it was
one to three larger in all 16. The function now returns the tabled value
in 39 of the 50 (24 of the 34 and 15 of the 16), and 101 and 142 for the
paper’s worked example (true_cv = .25,
width = .10, conf_level = .99, without and
with assurance = .99), where the paper gives 99 and 141 and
DMAR 1.0.x returned 100 and 142. The tables were computed with R’s
approximation to the noncentral t distribution, and the
function now plans for the exact interval of ci_cv() (see
the entry on that change below). The help states how the returned sample
size relates to the expected width itself.
ss_aipe_cv_sensitivity(), which plans through
ss_aipe_cv() when no sample size is given, changes with
it.
The population coefficient of variation in
ss_aipe_cv() is the argument true_cv
(it was C_of_V), the name the data generating value has in
ss_aipe_cv_sensitivity(). C_of_V is not
accepted. See Breaking Changes in 1.1.0.
ss_aipe_cv() with assurance is
the published procedure of Kelley (2007), for any assurance strictly
between 0 and 1. Two rows of the tables of that article show a
misprint. Table 1, assurance .80, width .125, coefficient of variation
.40 prints 127, between 69 and 122 in its row; the procedure gives 93.
Table 3, assurance .80, width .010 prints 14,279; 28,321; 50,209;
38,120; and 82,817 for coefficients of variation of .30 to .50, which do
not increase; the procedure gives 14,279; 20,475; 28,321; 38,120; and
50,209 with R’s approximation to the noncentral t distribution,
with which the tables were computed. How the plans of the exact
distribution compare with the printed values is in the entry on the
exact interval below. The help page describes the step as it is
computed: the search is run for the value of the article’s Equation 3 at
the quantile of the sample coefficient, and applies Equation 3 to that
value again at each sample size. With R’s approximation this computation
gives the printed value in 524 of the 528 assurance cells (the other
four are the misprints), where the search for the quantile itself, as
the article’s text words the step (p. 759), gives it in 365. The result
has one row, necessary_N, with or without
assurance, as in 1.0.x. assurance had to be
above .50 and may now be any probability strictly between 0 and 1; a
value such as .50 asks that the realized interval be no wider than
width in about half of the studies that could be run. A
malformed assurance (a vector, a character string,
NA) or one outside (0, 1) stops with one refusal, of class
dmar_assurance_invalid, that states the value given, why
neither 0 nor 1 is accepted, and what to give, where R’s own errors
appeared. The procedure is a close approximation and not an exact
search. With the exact distribution, in the 488 printed cells with an
assurance and at most 5,000 cases, the probability that the realized
interval is no wider than width at the returned sample size
is .744 to .902 where .80 was requested (below .80 in 37 of 244) and
.9916 to .9997 where .99 was requested, and away from the published
tables it can fall below the request (.587 for a requested .60 with
true_cv = .22, width = .09, and a 95 percent
interval, where the function returns 57). The shortfall grows as the
request falls toward .50: in 105 plans at 95 percent confidence the
probability is below the request by at most .038 for .50, .036 for .55,
.058 for .60, .032 for .70, and .001 for .80, and by nothing for .90 and
.99; and a request of .50 returns one fewer than the plan without
assurance in 12 of 15 plans (17 against 18 for
true_cv = .05 and width = .0375, where the
probability at 17 is .462). The help gives these numbers, and
ss_aipe_cv_sensitivity() estimates the probability for any
plan by simulation.
ss_aipe_sm() takes any assurance strictly
between 0 and 1. The assurance step plans for the value that
the sample standardized mean exceeds in magnitude with probability
1 - assurance, the one-sample form of the procedure of
Kelley and Rausch (2006, Equations 18 to 20 and 25 to 31), and that
value is defined for every probability in (0, 1). DMAR 1.0.x refused an
assurance at or below .50. A value such as .50 asks that
the interval be no wider than width in about half of the
studies that could be run, and a value below about one half returns a
smaller sample size than the plan without assurance: for
true_sm = 1, width = 0.5, and a 95 percent
interval the function returns 93 without assurance, 87 at
.20, 93 at .50, 100 at .80, and 105 at .90, where the probability that
the interval is no wider than 0.5, from the noncentral t
distribution, is .228, .519, .812, and .925. An assurance
that is not NULL or a single number strictly between 0 and
1 stops with the refusal of class dmar_assurance_invalid
that ss_aipe_cv() gives. No result for an
assurance above .50 changes for this reason (the exact
interval of ci_nc_t() changes a few plans by one or two;
see the entry on the AIPE planners built on
ci_nc_t()).
ss_aipe_cv() plans for the exact interval of
ci_cv(), so it no longer returns every value printed in
Kelley (2007). The noncentrality of the interval, the square
root of N over the coefficient of variation, passes 37.62 whenever the
interval is narrow, and the assurance step also takes the noncentral
t quantile exactly where R’s qt() inverts
pt() (see Exact Noncentral t and Studentized Range
Distributions). The tables of Kelley (2007) were computed with R’s
approximation. Apart from the four misprinted cells (127 in Table 1;
28,321, 50,209, and 82,817 in Table 3; see the entry above on
ss_aipe_cv() with assurance), the function
returns the printed value in 468 of 788 cells (the procedure computed
with the approximation gives all 788), one to three fewer cases in 232,
and one to three more in 88; it returns every printed value with a width
of .125 or more at 90 and 95 percent confidence and 9 of the 120 with a
width of .075 or less at 99 percent. The worked example of the article
(true_cv = .25, width = .10,
conf_level = .99) gives 101, and 142 with
assurance = .99, where the article gives 99 and 141 and
DMAR 1.0.x gave 100 and 142: at 99 the interval at the expected value of
the sample coefficient is .1006 wide, and .0999 by the approximation.
For true_cv = .10, width = .10, and 99 percent
the plan is 20 (printed 19); for .05, .05, and
99 percent, 19 (printed 21). The help page’s numbers on the assurance
step are recomputed with the exact distribution (see the entry above on
ss_aipe_cv() with assurance). The warning
ss_aipe_cv() gave when its search passed a noncentrality of
37.62 is gone.
ci_cv() no longer warns when its upper limit
is infinite, and the tail columns at that limit hold the requested
areas. DMAR 1.0.0 warned “The computed confidence interval does
not have the same coverage as the specified confidence interval.” and
reported prob_less 1 and prob_greater 0 on the
upper limit. The interval does have the stated coverage. The upper limit
is infinite when the square root of N over the sample coefficient is at
most the 1 - alpha_upper quantile of the central t
distribution on N - 1 degrees of freedom, and the infinite limit is part
of the exact interval: in the 19,975 of 20,000 normal samples with a
positive mean, for a coefficient of variation of .8 and N = 6, the upper
limit of the 95 percent interval was infinite in 31 percent and coverage
was .952, and with .5 and N = 5 it was infinite in 9 percent and
coverage was .948. ci_cv(cv = 1.2, N = 6) returns 0.571 and
Inf without a warning, with prob_less .975 and
prob_greater .025 on the upper limit. No limit changes. The
functions that call ci_cv() are quiet with it:
ss_aipe_cv(true_cv = .45, width = .10, conf_level = .99)
(C_of_V in 1.0.x), whose search starts at a sample size
where the limit is infinite, gave the warning once in DMAR 1.0.0 and
gives none, with the plan unchanged by this; and
ss_aipe_cv_sensitivity() gives one warning of its own for
the replications with an infinite limit (see the entry on the
replications it summarizes).
A fit whose likelihood has no maximum now stops, where
DMAR 1.0.x reported a table or stopped with an unrelated error.
When an error variance has run off below zero along a ridge of the
likelihood, which is flat or still rising there, lavaan’s optimizer
stops where its steps stop gaining and reports convergence, and DMAR
1.0.x reported that point’s estimates, omega, and fit rows, with a
warning about a Heywood case. fa_confirmatory(),
cfa_k(), cfa_1(), cfa_2(),
measurement_invariance(), and
measurement_alignment(), and the single-factor fits behind
reliability_omega(), now stop instead, for every output,
with an error of class dmar_no_maximum that prints the
lavaan code of the fit (see “A factor analysis or structural equation
model fit whose likelihood has no maximum stops” above). On three items
whose correlations have a negative product (the sample of the
reliability tests, population correlations .4, .3, and -.05, N = 300),
DMAR 1.0.0’s cfa_1(), which deleted incomplete cases
listwise by default, returned the table of the point where lavaan
stopped, with an error variance of -1899 and an omega of 442;
cfa_1(missing = "listwise") now stops. With that sample as
one group and a sample of 300 from a population in which three items
correlate .5 as another, DMAR 1.0.0’s
measurement_alignment() aligned a configural fit that had
stopped with a loading of 21.8 and an error variance of -475 in the
first group and gave the second a factor variance of 1282; it now stops.
ss_aipe_reliability() counts a simulated sample with no
maximum as an interval wider than width, and
ss_aipe_reliability_sensitivity() sets such a replication
aside, whatever its error variances sum to; DMAR 1.0.x used the interval
of the point where lavaan stopped. A bootstrap replication with no
maximum, whose estimate DMAR 1.0.x used, is set aside and counted among
the replications that raised an error. A finite negative error variance,
at a maximum of the likelihood, is reported as before.
ss_aipe_composite_sem(),
ss_power_composite_sem(), ss_aipe_sem_path(),
ss_aipe_sem_path_sensitivity(), and
mediation_mbco() stop the same way when the fit they read
their result from has no maximum: a planner’s fit of the analysis model
to the population moments, or the lavaan fit from which
mediation_mbco() starts its OpenMx fits. DMAR 1.0.x went on
from the point where lavaan stopped. Given a lavaan fit
(std.lv = TRUE, meanstructure = TRUE) in which
those three items measure a mediator between two other variables,
mediation_mbco() reported an indirect effect of -0.0000008
with a likelihood ratio statistic of 0; and for a single factor with
free loadings fit to their covariance matrix,
ss_aipe_sem_path() stopped with R’s “incorrect number of
dimensions” and the composite planners said that the loading “gives no
sampling variability”, errors that did not name the cause. A replication
of the Monte Carlo of ss_aipe_composite_sem(),
ss_power_composite_sem(), or
ss_aipe_sem_path_sensitivity() with no maximum, which DMAR
1.0.x kept, is set aside as one that did not converge.
ss_aipe_rmsea_sensitivity(), which reads only the chi
square of its fits, is not checked, and its results change only as its
own entries below describe (its default level and how its intervals are
formed).
ss_aipe_sem_path_sensitivity() passes the
arguments in ... to every fit, and
ss_aipe_sem_path() adjusts its own fit’s plan for
assurance. The arguments in ... go to
lavaan::sem(). DMAR 1.0.x passed them to the planning fit
to supposed_cov_matrix alone, so the fit to
true_cov_matrix, which gives the population value of the
path, and the fit to each simulated sample used lavaan’s defaults: with
std.lv = TRUE, the path of the help’s population was
planned in the metric that argument sets, where it is 0.577, and
simulated and covered in the metric of a marker item, where it is 0.5.
With an assurance, ss_aipe_sem_path() adjusted
a sample size it read from a second fit of the model made without
..., so the plan under std.lv = TRUE was
adjusted from the plan of the default metric; it now adjusts the plan of
the fit it reports, with one population fit fewer. Results change only
when ... is used, and an argument such as
bounds = "pos.var", which the refusal of a fit with no
maximum names as a choice, now reaches every fit.
measurement_invariance() refuses an item
named twice in items. DMAR 1.0.x built the model
f =~ y1 + y1 + y2 + y3 from
items = c("y1", "y1", "y2", "y3"), which lavaan (0.7-3)
reads as the three-item model f =~ y1 + y2 + y3, so the
ladder described one item fewer than the call named. The call now stops
with an error of class dmar_items_invalid (reason
"duplicate"), as measurement_alignment() and
fa_exploratory() refuse such items. Only
items is checked for a repeat, since DMAR writes the model
from it. An indicator repeated in lavaan syntax given as
model is read by lavaan as one indicator, as before, and a
named list given as model that names an item twice is
refused as before, without the class.
fa_confirmatory(), cfa_k(),
cfa_1(), and cfa_2() take the standard errors
that match the estimator by default. se now
defaults to NULL, which gives Huber-White (robust) standard
errors under estimator = "MLR" and
"robust.sem" under "WLSMV"; DMAR 1.0.x
defaulted to se = "standard" and passed it to lavaan under
every estimator but with ordered items, so its MLR standard errors were
the normal theory ones beside a robust test (for the second loading of a
verbal factor of three Holzinger and Swineford tests, 0.249 then and
0.213 now). se = "standard" gives the 1.0.x standard errors
(checked against DMAR 1.0.0’s cfa_k(): identical).
fa_confirmatory() fits by full information
maximum likelihood by default, as fa_exploratory()
does, and cfa_k(), cfa_1(), and
cfa_2() follow it. missing = "listwise" gives
the listwise deletion that DMAR 1.0.x applied by default, and
fa_exploratory() accepts it as well. With incomplete items
the default now uses every case with an observed item, so the estimates,
their standard errors, and N change. With complete data the estimates
are unchanged, and for estimator = "ML" so are the chi
square, CFI, TLI, and RMSEA; the standard errors come from the observed
information, which lavaan uses under FIML and Savalei (2010) recommends
with incomplete data, rather than the expected information, and moved by
at most about 2 percent on the two factors of the examples. The item
intercepts (nu rows) join the table, and AIC and BIC count
them. With estimator = "MLR" FIML on complete data leaves
the standard errors as listwise deletion gives them (DMAR 1.0.x took
other ones under MLR; see the entry on se below), and the
robust CFI and RMSEA are the corrected version 3 indices of Zhang and
Savalei (2023), named under the table. Ordered items, the least squares
estimators, se = "robust.sem", and the Satorra-Bentler kind
of test, which FIML cannot accompany, are fitted as before when no item
value is missing; when values are missing they ask for
missing = "listwise" (or "pairwise" for
ordered items) rather than deleting cases unasked, and
fa_exploratory() now treats ordered items the same way. The
srmr row is the SRMR of the covariance residuals alone
whenever the model has a mean structure (lavaan’s
srmr_bentler_nomean, as sem_fit_measures()
reports it), so the intercepts, whose residuals are zero, do not lower
it. With auxiliary variables (lavaan’s aux under FIML) the
srmr row is over the items alone: lavaan averages the
auxiliary variables in, whose residuals are also about zero, and for the
three-factor model of the nine Holzinger and Swineford tests with 40
values missing per item the SRMR of .068 fell to .062 with one auxiliary
variable and .058 with two. A fit whose observed information matrix is
singular reports no standard errors, with a warning, as a listwise fit
already did; lavaan inverts such a matrix without complaint and gives
finite standard errors for parameters that are not identified.
fa_exploratory() and factor_count() record the
FIML method as "fiml" rather than lavaan’s
"ml".
cv_smm() is right at very small degrees of
freedom. In DMAR 1.0.0, with df below 1 at
conventional levels (below about .8 at an alpha_level of
.05, and below about .6 at .10) or with df of about 2 or
less and an alpha_level of .02 or less, its integral over
the chi squared variate missed a sliver of the integrand by the origin:
the call stopped with “the integral is probably divergent”, or, at
df = 2 and alpha_level = .001, returned a
critical value whose tail area was 1.5 to 5 times
alpha_level (32.80 rather than 40.43 for two comparisons).
Each root is now checked against the upper tail computed by conditioning
on the maximum modulus instead, and found on that tail when the two
disagree. Every value that was right is returned unchanged, bit for
bit.
design_consequences(),
sd_unbiased(), and var_ete() are right at a
very large df or N. Each uses E(S)/sigma
= sqrt(2/df) Gamma((df + 1)/2) / Gamma(df/2), the mean of a standard
deviation on df degrees of freedom relative to sigma, or
the variance 1 - (E(S)/sigma)^2 that goes with it. E(S)/sigma is below 1
at every df and about 1 - 1/(4 df) for a large one, but
each function computed it as a difference of two lgamma()
values that nearly cancel there. design_consequences()
multiplies the width it would have with the standard error known by
E(S)/sigma for mean_ci_width and
expected_half_width, and by the square root of the variance
for sd_ci_width. The factor was off by 4e-11 at
df = 1e5, first reached 1 at df = 10810066,
and was 1 or more at about half of all df above 1e8, which
put mean_ci_width above the fixed width and
sd_ci_width at 0. sd_ci_width lost more,
because one minus the square of a number near 1 keeps only the digits
the two do not share: it was off by 1e-12 at the 118 df of
60 per group, by 8e-6 at df = 1e5, by 0.1 percent at
df = 1e6, and was 57 times its value at
df = 1e9. No other row involves them, and at the
df of an ordinary design the mean rows move only in the
last digits (by 4e-15 at 118 df).
sd_unbiased() divides by the same factor and returned
s unchanged at N = 1e8 and .998 s at
N = 1e12, where the correction is upward;
var_ete() with covariate_value = "sd" uses the
variance and moves by 2e-13 or less at a few hundred cases and by 3e-6
at N = 1e6. Both are now formed from the logarithm of
E(S)/sigma, the mean as its exponential and the variance through
expm1() of twice it. From df = 100 on the
logarithm is its asymptotic expansion, and below that the expansion
carried down two df at a time by an exact recurrence whose
terms all have one sign, so nothing cancels at any df.
Against a 320-bit evaluation, both are right to a few units in the last
place: over some 490,000 df from 0.05 to 1e15, the largest
relative error found is 5.5e-16, about 2.5 times the machine epsilon.
E(S)/sigma is never above 1, and it never decreases as df
grows on grids as fine as 1e-13 apart, including across every even
df from 2 to 100, where the computation changes form; from
one double to the next it can step back by a unit in the last place, as
any value computed with a rounding error can.
ss_aipe_mean() and ss_aipe_two_phase(), both
new in this release, use the same factor.
expected_r() and
expected_partial_r() are right at a very large
N. E[r] = rho 2F1(1/2, 1/2; (N + 1)/2; rho^2)
Gamma(N/2)^2 / {Gamma((N - 1)/2) Gamma((N + 1)/2)}, and the gamma ratio
is the square of E(S)/sigma on N - 1 df (the entry above).
expected_r() formed it from three lgamma()
values that nearly cancel at a large N, and it was off by 1e-11
at N = 1e5 and 1e-7 at N = 1e8. The bias rho - E[r] is
only about rho (1 - rho^2) / (2N), so its error was larger in
proportion: at rho = .5 the bias column was 0.2 percent too
small at N = 1e6, 26 percent too large at N = 1e7, 31
times its value at N = 1e8, and of the wrong sign at N
= 3e8, where E[r] came out above rho. The ratio now comes from the
logarithm in the entry above, and the bias and
relative_bias columns are formed directly rather than from
rho - E[r]: relative_bias is one minus the product of the
series and the ratio, formed from their logarithms, and
bias is rho times it. The difference rho - E[r] keeps only
the digits the two do not share, and with the ratio right to a few units
in the last place it is still off by 3e-7, relative, at N =
1e8. Against a 320-bit evaluation the bias is now right to
2e-14, relative, for |rho| of .9 or less at N from 4 to 1e12.
So that the series carries the digits the bias needs, the option
DMAR.expected_r.tol now stops the sum once a term is below
that fraction of the sum of the terms after the leading 1, rather than
of the whole series, and the sum can run to more terms than in 1.0.x;
the non-convergence warning still compares the last term with the whole
sum, so it is given where it was. At N of 1000 or less, E[r]
moves by less than 2e-12, relative. A rho so small that rho^2 is 0 in
double precision, below about 1.6e-162, gave E[r] = 0 and a
relative_bias of 1; it now gives E[r] = rho c4(N)^2.
expected_partial_r() calls expected_r() at
N - (p - 1) and moves with it.
manova_split_plot() tests the
within-subjects main effect averaged over the groups, as Maxwell,
Delaney, and Kelley (2027, Section 14.6) do. The default Type
III B rows test whether the grand means of the
within-subjects contrasts are zero. DMAR 1.0.x fitted the
between-subjects factor under R’s default treatment coding, in which the
intercept that Type III tests is the first group’s mean of each
contrast, so its B rows were the simple effect of the
within-subjects factor in the first group, tested against the error
matrix pooled over the groups (the book’s Equation 14.66), rather than
the main effect. The factor is now coded with sum-to-zero contrasts
whatever options("contrasts") says, so Type III tests the
unweighted grand means, which the book prefers and notes that SAS and
SPSS test by default (Section 14.6.9 and note 6 of Chapter 14), and
ss_type = 2 tests the weighted grand means, in which each
group counts in proportion to its size. On the reaction times of the
book’s Table 14.9 (10 young and 10 old participants at angles of 0, 4,
and 8 degrees), DMAR 1.0.0 gave F(2, 17) = 36.22 for angle, the
value the book reports for the simple effect of angle among the young
participants (Equation 14.66, p. 810); the book’s angle main effect,
F(2, 17) = 110.65 (Equation 14.54, p. 798), is what the
function now returns. With only the 0 and 8 degree conditions (Table
14.7), DMAR 1.0.0 gave F(1, 18) = 75.77 where the book gives
233.41 (p. 792). The A and interaction rows
are as before, as is every row under ss_type = 2, since
none of them depends on the coding. The B rows were already
the main effect whenever the contrasts in force for the between-subjects
factor summed to zero: for an ordered factor under the default ordered
coding, contr.poly, and for an unordered factor when
options("contrasts") named contr.sum or
contr.helmert. Under contr.SAS they were the
simple effect in the last group. No argument gives the 1.0.x
B rows. They are the Time rows of the call
DMAR 1.0.x made under R’s default contrasts,
car::Anova(lm(Y ~ g), idata = data.frame(Time = factor(within, levels = within)), idesign = ~ Time, type = 3),
with Y the matrix of the within columns and
g the between-subjects factor. The help no longer says that
Types II and III coincide for a single between-subjects factor: they
give the same tests of A and of the interaction, and the
same test of B only when the groups are of equal size. The
model is now fitted to objects of its own, so a between-subjects column
named Time, for which DMAR 1.0.x reported the test of
A in the B rows, is handled like any other, as
are a between-subjects column named Y or with a name that
is not syntactic and a within-subjects column named Y, with
which it stopped with an error. The function no longer passes on car’s
warning HF eps > 1 treated as 1, which comes from the
univariate tests it does not report.
reliability_omega(),
reliability_alpha(), reliability_H(), and
measurement_invariance() take full information maximum
likelihood as the default of missing as well, so
that the omega of reliability_omega() and that of
fa_confirmatory() agree on incomplete data, and
reliability(), which passes missing on only
when it is supplied, follows them. missing = "listwise"
gives the listwise deletion that DMAR 1.0.x applied by default. The
default resolves by one rule in the four functions: the analysis is FIML
whenever an item value is missing and the complete data analysis
otherwise. That differs from fa_confirmatory(), which fits
complete data by FIML too. Data in which no value is missing, and a
covariance matrix, are analyzed as they stand, as DMAR 1.0.x analyzed
them, because the complete-data intervals of alpha and omega (the closed
forms, "adf", and "likelihood", none of which
FIML gives) need that analysis, and so no complete data result changes:
FIML would give the same estimates, but its standard errors come from
the observed information and the Huber-White sandwich, alpha’s bootstrap
would refit a saturated model on every replication, about 70 times as
long, and an MLR ladder’s robust indices would be the corrected ones of
Zhang and Savalei (2023). A row with every item missing (every model
variable, in measurement_invariance()) does not count as a
missing value, since listwise deletion and FIML both drop it, so data
that are complete but for a blank row keep the complete data analysis.
Incomplete items are analyzed by FIML, so the estimates, their standard
errors, and N change; under FIML the analytic alpha’s default interval
is "ml", since Bonett’s needs complete cases. An option
FIML cannot accompany (the complete-data closed form intervals,
"adf", and "likelihood" in alpha and omega;
ordered items, estimators other than ML and MLR,
se = "robust.sem", and the Satorra-Bentler kind of test in
measurement_invariance()) is used as before when no item
value is missing, and with missing values the call stops and names the
options rather than deleting cases unasked; the checks of omega’s
denominator and alpha’s estimator come first, so that neither option
named meets another refusal. Auxiliary variables imply FIML, as before.
reliability_H() takes data for the first time
in this release, and reliability_omega_categorical() keeps
listwise deletion as its default. In
measurement_invariance(), under FIML by default or by name,
a group that never observes a variable of the model, or never observes
two of them together, is refused by name before any fit (an error of
class dmar_group_coverage, a dmar_refusal
carrying the variables and pairs), with what to do instead; lavaan
stopped with “subscript out of bounds”, which the new default would have
made the ordinary result for holzinger_swineford’s
t25_paper_form_board_r and t26_flags by
school. An explicit missing = "fiml" with
se = "robust.sem" or a test of the Satorra-Bentler kind now
stops, as in fa_confirmatory(), where through 1.0.1 lavaan
fit the complete cases with a warning, and so does a ladder lavaan
fitted by another treatment than the FIML asked for (it fits GLS, WLS,
and ULS in two stages).
reliability_omega() forms the default
interval of the model implied omega on the logit scale. The
default ci_method is "mlr_logistic" where it
was "mlr": the same robust standard error, with the limits
formed on the logit scale and transformed back to the scale of the
coefficient (Browne, 1982; Kelley, 2007), so that
reliability_omega(denominator = "model_implied") and the
omega row of fa_confirmatory() give limits of the same
form. It applies under listwise deletion, FIML, and two-stage estimation
alike. The change rests on a Monte Carlo study (in the repository’s
methodology/reliability_limits) in which the logit limits
met every criterion fixed before the run, with normal data under
listwise deletion and FIML and with nonnormal data fitted by MLR. The
study formed both kinds of limits from the same standard error, the
normal theory maximum likelihood one for normal data and the Huber-White
one of MLR for nonnormal data, so it compared the scale on which the
limits are formed: with normal data the logit limits covered from .939
to .956 in every cell, against .910 to .952 for the limits on the scale
of omega, whose misses fell mostly on one side. No cell used the
robust.sem standard error of "mlr_logistic"
with complete data, which estimates the same asymptotic variance as the
Huber-White one. ci_method = "mlr" gives the limits of DMAR
1.0.x; ci_method = NULL is the default, as it was through
1.0.x. Given a covariance matrix with
denominator = "model_implied" and no
ci_method, reliability_omega() now uses
"ml_logistic", the maximum likelihood standard error with
the limits formed the same way, as reliability_H() does,
where through 1.0.x it stopped because its default needed raw data;
reliability(), which chose "ml" on that path,
follows it. Under wald_df = "missing_information", whose
calibration covers the intervals on the scale of omega only, the default
remains "mlr" ("ml" with a covariance matrix).
The estimate and the se row do not change, nor do robust
omega (whose default is no interval) and coefficient alpha’s
defaults.
FIML and two-stage reliability fits count distinct cases
that observe an item. A row observed only on auxiliary
variables informs their distribution, but not the items given them; a
repeated row adds no dimension. The count must exceed the items plus
auxiliary variables, in the data, every bootstrap replication, and each
jackknife deletion. Fitted coefficients under listwise deletion count
distinct complete rows. Listwise alpha is offered as an alternative only
when it would accept those rows, with Feldt’s interval named when
exactly two are complete. A standard error lavaan could not compute is
NA, rather than zero.
FIML and two-stage reliability coefficients refuse a constant composite. When complete rows outnumber the items, their sum is checked directly. A constant sum makes the saturated likelihood unbounded as its variance tends to zero. Otherwise the EM covariance estimate is checked with a tolerance reflecting EM convergence. Bootstrap replications follow the same rule. H’s weighted composite is exempt. For all Holzinger and Swineford pupils on visual perception, cubes, lozenges, and paragraph comprehension, center each pupil’s scores on that pupil’s mean, then delete the first ten visual perception scores. Version 1.0.1’s model implied FIML alpha was 1.49e-13 with limits [0, 9.23e-8]. The data are now refused because the composite is constant in the 291 complete rows.
An indefinite supplied S is named as not a
covariance matrix when its composite variance is not positive.
The message gives its negative eigenvalue, and the composite variance
when clearly negative, rather than calling the matrix singular with a
zero composite variance. For
S <- matrix(-.9, 3, 3); diag(S) <- 1, version 1.0.1
returned alpha 3.38 with limits [4.31, 1] at N = 100 with
ci_method = "feldt". This matrix is now refused; positive
semidefinite ipsative matrices keep the constant-composite
message.
The equal-loading factor analytic plan uses its fitted
coefficient. Let
S <- matrix(.3, 4, 4); diag(S) <- c(1, 1, 1, 4) and
S[4, 1:3] <- S[1:3, 4] <- -.5. At width .20,
ss_aipe_reliability(S = S, model = "True Score", type = "Factor Analytic", width = .20)
plans 244 cases, against 786 in 1.0.1, whose plan used alpha of the
correlation matrix. The Parallel model gave 1669 and is now refused as
far from parallel. Conversely,
S <- diag(c(10, 10, 1, 1)); S[1, 2] <- S[2, 1] <- 4; S[3, 4] <- S[4, 3] <- -.3
has positive alpha but a zero fitted loading. At width .30, 1.0.1
planned 202 for Parallel and 386 for True Score; both now refuse it.
Highly reliable congeneric populations are checked at a feasible
estimation size: four loadings of .99 with error variances
1 - .99^2 at width .20 still plan 2 cases, as in 1.0.1. The
closed form can remain below the minimum estimation size. Every
assurance plan carries no_interval, including zero, and
glance() adds fmi,
necessary_N_complete, and no_interval when
present. The closed form’s tidy() width is NA,
as in 1.0.1.
fa_confirmatory() refuses correlated factors
no case observes together. Their covariance does not enter the
observed-data likelihood; a third factor observed with both does not
identify it. Split the four spatial and three verbal Holzinger and
Swineford tests between forms using
set.seed(3); form <- sample(1:2, 301, replace = TRUE),
with the spatial block missing on form 1 and verbal on form 2. Version
1.0.1 reported the correlation at its starting value, zero, and
omega_spatial .650 with a standard error of zero and limits
[.650, .650]. The fit is now refused by name; fit each factor alone or
link the forms with common items. Orthogonal factors are fitted, since
their correlation is fixed, and their coefficients and standard errors
equal the separate fits. More generally a fit whose information cannot
be inverted reports NA for standard errors, tests, and
intervals in every table output, with a warning.
Zero degrees of freedom with a nonzero statistic gives no
TLI or RMSEA. A binding inequality can leave a positive
statistic at zero degrees of freedom, which lavaan does not count as an
equality. For a factor of three unit-variance variables correlated .8,
.8, and .5, at N = 200 with bounds = "pos.var", the
statistic is 32.8 and CFI .925. Version 1.0.1 reported TLI and NNFI 1
and RMSEA 0 [0, 0]; those rows are now NA, since they
divide by the degrees of freedom. CFI keeps .925. The RMSEA and
following delta_rmsea of such an invariance rung are also
NA. A rung’s CFI decides the version of its indices
independently of its test. An exactly fitting saturated model keeps its
earlier convention.
CFA summaries carry the method and reference of their
results. glance() adds estimator,
missing, test, fit_indices, and
rmsea_ci_level after its earlier columns.
tidy() of CFA, reliability, mlmr(), and
mlmr_mv() results carries wald_df per row, and
reliability’s glance() carries it too. The default is
Inf. Rows on a t reference can therefore be stacked with
rows on the normal reference without losing which interval each has.
measurement_invariance(group_partial = character(0)),
refused in 1.0.1, now frees nothing, as NULL does. Empty
group.partial passed through ... also frees
nothing.
reliability_omega() refuses a single-factor
solution that leaves no reliability coefficient. On three items
whose correlations had a negative product, it returned an omega of 471,
from a loading of 43.6 and an error variance of -1899, with only a
warning about a Heywood case. A single factor implies that the product
of the three correlations is the square of the product of the loadings,
so no real loadings reproduce such correlations and the maximum
likelihood estimate does not exist. The function now stops with a
message that quotes the correlations, under either denominator, for
covariance input, and under FIML. It also stops when the error variances
of an improper solution sum below zero, so that the model implies a
negative error variance for the composite and an omega above one, the
signature of a solution that has run away (McDonald, 2004), and the
model implied reliability_alpha() does the same for the
tau-equivalent model. An improper solution whose coefficient is still a
proportion is returned with a warning, as before, and the warning now
names the item, its error variance, and its observed variance; the
observed denominator, which gave no warning of its own, now gives it. In
a bootstrap, a replication with no coefficient is set aside under either
improper setting (on 80 cases with one loading of .95, one
such replication in 200 had put the bootstrap standard error at 1.02
rather than 0.049), and a replication of
ss_aipe_reliability_sensitivity() whose fit is now refused
is set aside rather than returning an absurd interval. This is a
deliberate, loud change for those samples: a number that described where
the optimizer stopped is replaced by an error. lavaan’s own
per-replication notices of improper solutions no longer reach the
console.
Each confidence limit of a reliability coefficient is
clamped into [0, 1], and a negative estimate is warned about.
Through 1.0.1 the lower limit was raised to 0 and the upper limit
lowered to 1, each on its own. So when a negative coefficient alpha or
KR-20 had an interval lying wholly below zero, the limits came back
inverted, with the estimate outside them. On four binary items, one
nearly the reverse of another
(set.seed(3); z <- rbinom(60, 1, .5); Xb <- cbind(i1 = z, i2 = 1 - z, i3 = rbinom(60, 1, .5), i4 = rbinom(60, 1, .5)); Xb[1:6, 2] <- z[1:6]),
reliability_kr20(Xb) gave -0.821 with Feldt limits [0,
-0.171], and reliability_alpha(as.data.frame(Xb)) gave
Bonett limits [0, -0.196]. Both limits are now clamped into [0, 1], so
they stay in order and that interval is [0, 0]. The estimate is returned
as computed, with a warning (class
dmar_negative_reliability) that it lies outside the range
of a reliability coefficient and that an item keyed in the opposite
direction is the usual cause. The _logistic intervals,
which gave NA limits without a word, now say why. Only results whose
upper limit, before clamping, was below zero change. In
ss_aipe_reliability_sensitivity() such a replication now
has a width of 0 rather than a negative one, and the warning is muffled
inside the loop. At true_reliability = 0 such a replication
also covers the true value, since [0, 0] contains zero, so none can miss
high there; through 1.0.1 its inverted limits counted as a miss high.
With four items, N = 40, and Bonett’s interval
(ss_aipe_reliability_sensitivity(true_reliability = 0, n_items = 4, width = .3, specified_N = 40, ci_method = "bonett", G = 400)
after set.seed(1)), the share of intervals missing zero
goes from .0400, .0325 of it high, to .0075, all of it low. With the
"ml" interval it goes from .0425, .0100 of it high, to
.0350, all of it low; its low misses move from .0325 to .0350 because
that interval’s standard error now comes from the covariance matrix (see
the next entry). The function evaluates the interval a study would
report, and a miss high at zero would place the reliability below zero;
its help says so. Above zero the counts do not change.
The "ml" interval of
reliability_alpha() and reliability_kr20()
takes its standard error from the covariance matrix. The van
Zyl, Neudecker, and Nel (2000) variance was evaluated on the correlation
matrix of the items, which is the variance of alpha of standardized
items, and paired with alpha of the raw items. The paper defines it on
the covariance matrix, as do Terry and Kelley (2012, Eq. 8) and Kelley
and Pornprasertmanit (2016, Eq. 20), and the help already said it was
computed from the covariance matrix. Whenever the item variances differ,
the standard error was wrong in either direction. With equal loadings .7
and error variances .1, .2, 1, and 2 at N = 206 it was .0256 where
alpha-hat varies with a standard deviation of .0339, and the 95 percent
interval covered .863 of 4,000 samples; it now covers .952. With item
standard deviations from .8 to 1.9 it covered .889 (.944 now), and with
the large variances on the high loading items its standard error was
twice the standard deviation and it covered .9998 (.945 now). Items of
equal variance are unaffected. On six binary items of unequal difficulty
KR-20’s standard error moves from .0311 to .0279 against a sampling
standard deviation of .0242. The default Bonett interval and the FIML
and two-stage "ml" intervals, which already used the
covariance matrix, are unchanged, and on complete data the listwise and
FIML standard errors now agree up to the divisor. The code came from
MBESS, whose ci.reliability() (through its internal
.interval.type31_32_type1()) still evaluates the variance
on the correlation matrix, so the two packages’ "ml" alpha
intervals now differ when the item variances do.
Item names lavaan cannot parse no longer stop the
single-factor coefficients. Through 1.0.1,
reliability_omega() (robust omega, the default, and the
model implied omega), the model implied
reliability_alpha(), and reliability() passed
the column names to lavaan. lavaan stopped on a name such as “item 1” or
“1a”, and the error was reported as a model that did not converge; under
FIML the message also gave advice about small samples. The items now
enter the fit as y1, …, yJ, and every result,
message, and warning carries the user’s names. lavaan’s warnings, such
as “some observed variables are perfectly correlated; … variables
involved are”, are raised again with them. So are those of the saturated
model behind FIML, which named the variables v1,
v2, … or y1, y2, … even in 1.0.1,
for analytic alpha under FIML and for the fits with auxiliary variables.
reliability_H() from data, new in 1.1.0, works the same
way. On the five verbal tests of holzinger_swineford
(t5_general_information to t9_word_meaning)
renamed “item 1” to “item 5”, robust omega is 0.900, the same as under
the data’s names, where 1.0.1 stopped with “The single-factor CFA model
did not converge.” A fit that lavaan stops with an error is reported
with lavaan’s own message rather than as a convergence failure, and the
message names the items and auxiliary variables as the data do, in
reliability_omega(), the model implied
reliability_alpha(), and
reliability_omega_categorical(); a fit that ran and did not
converge is still reported as one, with the advice for FIML and
two-stage estimation. reliability_omega_categorical() had
reported every failure of its fit as “The single-factor categorical CFA
model did not converge,” which sent the user looking for a problem with
the sample. Its bootstrap sets a replication without a fit aside and
counts it as before, so no estimate, interval, or count changes.
reliability_omega()’s advice on requesting the robust omega
interval now comes after the estimate, so a call that stops on its fit
gives no advice first.
A name given to two columns is refused by name wherever the columns
are selected by name. Through 1.0.1, under FIML the first of two
same-named columns was analyzed twice, and the call reported a model
that did not converge; with auxiliary variables the second was dropped
without a warning. On t1_visual_perception,
t2_cubes, t4_lozenges, and
t6_paragraph_comprehension, with the t2_cubes
column also named t1_visual_perception and
t9_word_meaning as an auxiliary variable,
reliability_alpha() returned 0.555 with J = 3, the alpha of
the other three tests (0.606 with J = 4 when the columns are named
apart). An auxiliary variable’s name given to an item column as well is
refused too.
reliability() picks the coefficient from the
items alone. Auto-detection read every column of
data, auxiliary variables included. So ordered items with a
continuous auxiliary variable, or an integer one with more than 10
values, went to continuous omega under FIML rather than categorical
omega under multiple imputation, and a factor auxiliary stopped the call
with a base R error. The rule now reads only the columns not named in
aux.
fa_confirmatory() reports the average
variance extracted and coefficient H as NA for a factor
with a negative error variance, and omega as NA when the
factor’s error variances sum below zero. An item whose error
variance is negative has a communality above one. The AVE averages the
communalities, and H, the largest reliability a weighted sum of the
items attains, is at least that communality, since the item weighted
alone attains it, so for such a factor neither is a proportion of
variance. Through 1.0.1 both were reported beside the Heywood warning:
on holzinger_swineford, a factor of visual perception and
lozenges beside one of general information, paragraph comprehension, and
sentence completion had H = 1.321 [0.166, 2.476] and an AVE of .726.
Both rows are now NA, the estimate and its interval alike,
with one warning that names the items and their communalities. Omega
depends on the error variances only through their sum: while the sum is
positive it is a proportion and is reported, as
reliability_omega() reports it (.707 for that factor), and
when the sum is negative the model implies a negative error variance for
the sum score, so omega is NA too (visual perception and
cubes beside three speed tests had omega 1.050, H 1.536, and an AVE of
.788). For ordered items the communality is that of the latent response.
Through 1.0.1 an improper ordered factor stopped the whole fit with the
mvtnorm error “‘corr’ is not a correlation matrix”; its categorical
omega, which is not defined for an improper solution, its AVE, and its H
are now NA, named in the same warning. The classed Heywood
warning is unchanged, and output = "fit" keeps lavaan’s
defined parameters.
average_variance_extracted() reports
NA, with a warning, for a factor whose solution is
improper, as fa_confirmatory() does. An indicator
whose error variance is estimated below zero has a communality above
one, so the AVE of its factor, which averages the communalities, is not
a proportion of variance. fa_confirmatory() now reports
that factor’s ave row as NA, but
average_variance_extracted(fit) averaged the squared
standardized loadings regardless: for visual perception and lozenges
beside a factor of general information, paragraph comprehension, and
sentence completion in holzinger_swineford, where visual
perception’s communality is 1.304, it reported .726. The factor’s
ave, and its interval when one is requested, are now
NA, with a warning that names each such indicator and its
communality in the words of fa_confirmatory()’s. The check
reads the sign of the error variance and takes the communality as one
minus the standardized error variance, so it gives the same answer
whatever the identification of the fit, for the latent responses of
ordered items under either parameterization, and in every group of a
multiple group fit. Proper factors are unchanged (.705 for the verbal
factor there). When every factor of the fit is improper,
ci_method = "percentile" forms no interval: no bootstrap is
run, and no confidence level is recorded or printed.
average_variance_extracted() counts the
improper replications its bootstrap keeps, and its warning gives the
caution for each factor. A replication whose resample solution
is improper for a factor that is proper in the observed fit (an
indicator with a negative error variance) is kept, as the reliability
family’s default improper = "keep" keeps one. Through 1.0.1
it was neither counted nor reported, though its AVE averages a
communality above one and can exceed one.
bootstrap attribute (attempted,
returned, failed, improper_kept,
refit_failed, caution,
improper_kept_by_factor, and the rest), and prints the
family’s bootstrap line under the table, at the console and in a knitted
report. When two or more factors have an interval, a second line gives
the improper replications kept for each.refit_failed, and the line and the warning both say
that they “failed to refit or did not converge” and were set aside,
where 1.0.1’s warning said that they “failed to refit (or did not
converge) and were dropped”.holzinger_swineford
(set.seed(113); d <- holzinger_swineford[sample(301, 80), ]),
fit by lavaan::cfa() with a verbal factor of paragraph
comprehension, sentence completion, and word meaning and a spatial
factor of visual perception, cubes, and lozenges, then
average_variance_extracted(fit, ci_method = "percentile", B = 200, seed = 113).
The spatial factor is proper in the sample (AVE .449), yet 101 of the
198 replications that returned a value were improper for it, and its
upper limit is 1.19. Version 1.0.1’s warning said only that 2 of 200
replications failed to refit (or did not converge) and were dropped. The
warning now says that they were set aside, that 101 of the 198 used came
from an improper (Heywood) solution and were kept, and that for spatial
103 of 200 replications (51.5%) came from a solution improper for it or
were set aside, against the 2.5% beyond each limit of a 95% interval.
The line under the table says “for spatial, 51.5% of the replications
came from a solution improper for the factor or were set aside, at least
the 2.5% beyond each limit”, and the second line “spatial 101, verbal
0”.dmar_ave ahead of
dmar_tbl on every path. No interval changes.average_variance_extracted() reads a loading
fixed or constrained to zero as none. Through 1.0.1 it averaged
the squared standardized loading of every =~ row of a fit,
so a loading fixed at zero (0*item) or constrained to zero
(a*item with a == 0) entered the factor’s
average as a loading of zero. On holzinger_swineford
(lavaan::cfa(model, data = holzinger_swineford, std.lv = TRUE)),
a visual factor of visual perception, cubes, and lozenges written with
0*t7_sentence beside a verbal factor of general
information, paragraph comprehension, and sentence completion is the
same model as the one without that row (chi square 22.876 on 8 degrees
of freedom for both), yet it had an AVE of .279 rather than the .371 of
its three tests. The verbal factor written with
0*t1_visual_perception beside the same visual factor had
.529 (.705 times 3/4) rather than .705. Such a row is now left out of
the average and out of the check for an improper solution described
above, as factor_scores(), bifactor_indices(),
and reliability_H() read it. With a visual factor of visual
perception and lozenges alone, whose visual perception error variance is
negative, 1.0.1 gave the visual factor .726 and the verbal factor
written with 0*t1_visual_perception .529. The visual
factor’s AVE is now NA under that check, and the verbal
factor’s is the .705 of the model without the row, since visual
perception is not one of its indicators. Fits without such a row are
unchanged.
fa_confirmatory(), cfa_1(), and
cfa_2() refuse group passed through
.... fa_confirmatory() labels every
loading, error variance, intercept, and factor correlation, and lavaan
holds a label shared by several groups equal across them. A
group argument therefore fit one pooled model, every
measurement parameter equal across the groups and each factor’s variance
fixed at 1 in each: a model of complete invariance rather than a model
by group. lavaan warned about the single labels, and the table repeated
each labeled row once per group with the same estimate. The call is now
an error that names the route: measurement_invariance(),
whose configural model frees every parameter in each group and whose fit
gives reliability_H(fit = , factor = , group = ) the H of
one group, and reliability_omega() on one group’s
rows.
fa_confirmatory() names the version of its
fit rows after FIML, and its SRMR when the model has a mean
structure. Under missing = "ml" (or
"fiml"), the cfi, tli,
nnfi, rmsea, and RMSEA interval rows have
always been lavaan’s robust indices. For a FIML fit these are the
corrected indices of Zhang and Savalei (2023), version 6 (version 3 with
estimator = "MLR"), and they are not computed from the
chi_square row beside them. The srmr row of a
model with a mean structure is lavaan’s SRMR, which averages in the mean
residuals, so it is not the SRMR sem_fit_measures()
reports: .0229 against .0259 on the verbal and deduction tests of the
help page (hs_factors) with 90 sentence completion scores
deleted at random (set.seed(1), then
t7_sentence[sample(301, 90)] <- NA) under
missing = "ml". Lines under the table now say both, and the
table carries them in a new fit_rows attribute, so a
knitted report shows them too. When the robust indices of a fit are not
available (lavaan computes none, and DMAR computes them only for the
mean and variance adjusted statistic), every index row holds one other
version and a line says which. That is the standard indices for a
maximum likelihood fit of continuous items, and the scaled ones, or else
the standard ones, after FIML or two-stage estimation, for ordered
items, or by least squares. Another line names robust indices DMAR
computed, for the mean and variance adjusted statistic. Whether the fit
used FIML is read from the fit, so a fit lavaan made to the complete
cases (with missing = "ml" and a test it cannot combine
with FIML) is not labeled with the FIML corrected indices. An exactly
fitting model with no degrees of freedom gets no version line under FIML
or two-stage estimation; its SRMR and two-stage method lines stay. With
a nonzero statistic the version line names CFI alone and another line
explains the undefined TLI and RMSEA rows. A help paragraph says which
version each fit row is. No number changes here; the next entry gives
the rows that do.
fa_confirmatory() reports one version of its
fit indices, and the robust indices of a fit with the mean and variance
adjusted statistic.
cfi, tli,
nnfi, rmsea, and RMSEA interval rows took the
first of its robust, scaled, and standard versions that lavaan had
computed, so a row whose robust version was NA could hold
another version than the rows beside it.test = "mean.var.adjusted" passed through
...), for which lavaan computes no robust indices, every
row held the scaled version, whose population value changes with the
kurtosis of the data (Savalei, 2018). The rows now hold the robust
indices, computed by DMAR as Savalei (2018) shows they can be. They
equal lavaan’s under estimator = "MLM", and a line under
the table says that DMAR computed them.fa_confirmatory(data = holzinger_swineford, factors = hs_factors, test = "mean.var.adjusted")),
CFI .961 and RMSEA .046 [0, .089] become .993 and .047 [0, .091], both
intervals at 90 percent, the level of 1.0.x and the default of
rmsea_conf_level. On the three-factor model of nine tests
(see the sem_fit_measures() entry on the mean and variance
adjusted statistic), CFI .734 becomes .932 and RMSEA .089 [.068, .110]
becomes .091 [.070, .113].NA for TLI, NNFI, and RMSEA, as the entry on
zero degrees of freedom explains.mlmr() and mlmr_mv() leave the
covariances among the predictors free when auxiliary variables are
supplied. The saturated correlates of auxiliary
were written as a covariance of each auxiliary with each predictor. That
made the predictors endogenous to lavaan, whose automatic covariances
among exogenous variables then no longer reached them, and nothing else
freed them, so with two or more predictors every covariance among them
was fixed at zero and the model that should be saturated (Graham, 2003)
was not. The covariance of every pair of predictors is now written, and
the model over the outcomes, the predictors, and the auxiliaries is
saturated, with chi square 0 on 0 df. Its coefficients are the
regression implied by the full information maximum likelihood means and
covariances of every variable, auxiliaries included (what
cov_fiml() returns), and on complete data they and R
squared are those of lm(). A fit with one predictor, or
none, is unchanged. When a predictor has missing values, the
coefficients, standard errors, intervals, and p-values change.
On holzinger_swineford, with t6 regressed on t7 and t9, t9
made missing at random given t7 for 110 of the 301 students
(set.seed(113)), and t13 as the auxiliary, the model had
chi square 129.5 on 1 df and an implied covariance of 0 between t7 and
t9, whose correlation is .69. The t7 slope was 0.449 (SE 0.031) and is
now 0.362 (SE 0.044), against 0.362 without the auxiliary and 0.317 on
the complete data, and the t9 slope moves from 0.096 to 0.143 (0.167 on
the complete data). The auxiliary example on the mlmr()
page, where t26_flags is missing for 156 students, had chi square 14.1
on 1 df; its intercept moves from -0.703 to -0.436 and its t7 slope from
0.488 to 0.476. When the predictors are complete, the coefficients and
their standard errors were already right, since their maximum likelihood
estimates do not depend on the predictors’ covariance there, but every
quantity that uses the model implied variance of the outcome was not.
For t6 on t7 and t9 with complete data, R squared was .471 and is now
.602, the value lm() gives; the standardized coefficients
move from .541 and .423 to .469 and .367, and the semi-partial R squared
from .141 and .086 to .106 and .065. The log likelihood moves from
-4123.6 on 13 parameters to -4013.7 on 14, and AIC and BIC with it. The
likelihood ratio test of anova() for adding t9 in the first
example was chi square 8.62 and is now 13.72. In mlmr_mv()
the first example gives t6 a t7 slope of 0.362 rather than 0.450 and
t26_flags a t9 slope of 0.104 rather than -0.050, and with complete data
the R squared of each outcome is now that of lm() (.602 and
.474 for t6 and t8, where they were .471 and .379). The auxiliary
simulation in the mlmr vignette, whose predictors are complete, prints
the same values. With the model saturated, the three estimators of
fraction_missing_information() now agree on such fits as
they do without auxiliaries: in the first example the t7 slope’s
unstructured and Hessian estimates were .016 and -.559, and all three
are now .254. wald_df = "missing_information" therefore
gives that slope 221.5 degrees of freedom rather than 293.1.
mlmr() and mlmr_mv() refuse an
auxiliary variable under listwise or pairwise deletion.
lavaan’s listwise deletion drops every row missing any variable of the
model, the auxiliaries included, so through 1.0.1 an incomplete
auxiliary silently shrank the sample.
mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning, data = holzinger_swineford, missing = "listwise", auxiliary = "t26_flags")
fitted 145 of the 301 rows, with coefficients of 0.971, 0.313, and 0.178
where lm() gives 1.117, 0.317, and 0.167. An auxiliary
carries information only through the full information likelihood, so the
combination now stops and names the remedy (drop the auxiliary or set
missing = "fiml"), as reliability_alpha() and
reliability_H() stop. Under pairwise and available cases
deletion (missing = "pairwise" or
"available.cases") the moments are computed from each pair
of variables on its own. With the model saturated, the coefficients then
depend on the moments of the outcomes and predictors alone, so an
auxiliary can change nothing, while the result recorded it as used. In
mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning + t26_flags, data = holzinger_swineford, missing = "pairwise", auxiliary = "t13_straight_and_curved_capitals"),
1.0.1’s coefficients agreed with those of the fit without the auxiliary
to within 5e-6; its standard errors differed only because the auxiliary
also fixed the covariances among the predictors at zero (see the entry
above), so the t7 slope’s was .0245 against .0355. That combination is
now refused the same way.
Under missing = "fiml" with auxiliary
variables, a case whose items are all missing is used when an auxiliary
variable is observed on it. reliability_alpha()
and reliability_omega() set aside every case with no
observed item, with auxiliary variables or without. The saturated
correlates model estimates the items and the auxiliaries jointly, so
such a case informs the item moments through its auxiliaries, and
lavaan’s own FIML fit of the model keeps it, as cov_fiml()
does. Setting it aside made the analysis conditional on having an
observed item, which depends on the auxiliaries when the items go
missing together at random given them, and the estimates were then
biased. On the nine tests of the three-factor model (visual perception,
cubes, lozenges, paragraph comprehension, sentence completion, word
meaning, addition, counting groups of dots, and straight and curved
capitals), with the whole block set to NA for 77 pupils at
random given general information (set.seed(113), the rows
sample(301, 77, prob = plogis(-1.5 + 1.2 * z)) with z the
standardized t5_general_information, the auxiliary),
N goes from 224 to 301, alpha from .610 to .613, model
implied omega from .350 to .358, and robust omega from .228 to .234.
reliability_H() from data, new in 1.1.0, uses such cases
too. A result changes only when some case has every item missing and an
auxiliary observed, and N now counts such cases. Two-stage
estimation, new in 1.1.0, uses them as well: its first stage reads their
auxiliaries, and the coefficients are computed on every case it reads,
although lavaan’s two-stage estimator leaves such cases out of its N
(see Two-Stage Maximum Likelihood for Incomplete Data). Under multiple
imputation, also new, reliability_omega_categorical()
imputes such a case from its auxiliaries.
The FIML item covariance matrix behind analytic alpha and
robust omega is the EM estimate. Under
missing = "fiml", analytic reliability_alpha()
and the observed denominator of reliability_omega() apply
their formulas to the maximum likelihood covariance matrix of the items
under the saturated model. Through 1.0.1 that matrix came from lavaan
fitting the saturated model from its default starting values, which on
many items stops short of the maximum: on all 26 Holzinger and Swineford
tests, with the Pasteur students missing the last two, it took 2484
iterations and left covariances up to 0.0105 from the maximum likelihood
values, the log likelihood 3e-7 below its maximum. The matrix is now the
EM estimate cov_fiml() returns, the unrestricted moments
lavaan computes for every FIML fit to the same variables (the EM runs on
the standardized items, as in cov_fiml(), so the matrix
does not depend on their units, and agrees with lavaan’s to about seven
digits), and the standard error of the "ml" interval is the
delta method one from the saturated model evaluated at that estimate,
with no optimization run. On the Holzinger and Swineford examples
checked (6, 9, 23, and 26 tests, with the Pasteur students’ missing
tests or with values deleted at random given general information, with
and without an auxiliary variable), alpha changes by at most 1.5e-6,
robust omega by at most 5.8e-6, and the standard error by at most
4.4e-8; on the 26 tests alpha is .8234684 rather than .8234689. The
estimate no longer depends on the interval asked for, the FIML alpha of
complete data equals the listwise one to machine precision (it differed
by 2.3e-9 on nine tests), and the computation is faster: on 23
incomplete tests about three times for alpha with its interval and five
to ten times for robust omega. Two-stage analytic alpha, which uses the
same matrix, changes the same way, as does the fraction of missing
information ss_aipe_reliability_sensitivity() records for
alpha. Model implied alpha and model implied omega are
unchanged.
Under FIML and two-stage estimation, a pair of variables
never observed together is refused by name.
reliability_alpha() and reliability_omega()
now stop, as reliability_H() from data and
cov_fiml(), new in 1.1.0, do, naming each pair of items or
auxiliary variables that no case observes together and pointing to
missing_coverage(). Through 1.0.1 lavaan only warned, and
the covariance it returned for such a pair was arbitrary: on
t1_visual_perception, t2_cubes,
t4_lozenges, and t6_paragraph_comprehension,
with t1_visual_perception deleted in rows 1 to 150 and
t2_cubes in rows 151 to 301,
reliability_alpha(missing = "fiml") returned 0.536 with a
standard error of 0 and the interval [0.536, 0.536]. In a bootstrap, a
resample left with such a pair, or a jackknife deletion behind the BCa
acceleration, is set aside with the message, under two-stage estimation
as under FIML. A pair observed together only where one of them takes a
single value, or by one row only, is refused the same way (see the next
entry).
Under FIML and two-stage estimation, a pair of variables
observed together only where one of them takes a single value is
refused. Rows in which one of two variables is constant say
nothing about how the two vary together, so the estimate of their
covariance rests on the form of the normal model, or is not determined
at all. reliability_alpha() and
reliability_omega() now name such a pair and stop, as they
do for a pair never observed together, and so do
reliability_H() from data and cov_fiml(), new
in 1.1.0. A pair that only one row observes is refused the same way, and
in a bootstrap a resample that leaves such a pair is set aside.
t1_visual_perception,
t2_cubes, t4_lozenges, and
t6_paragraph_comprehension, with
t1_visual_perception deleted in rows 91 to 100 and
t2_cubes set to 25 in rows 1 to 90, the rows that observe
t1_visual_perception. Under missing = "fiml",
version 1.0.1 returned robust omega 1.06, the analytic alpha 0.113 with
no interval, and the model implied omega 0.890 [0.281, 1].t2_cubes among the items. t2_cubes is now
refused as constant in the 90 complete rows (see the entry on an item
that does not vary), and the message says why FIML cannot help.reliability_omega_categorical() refuses
fewer than three items. With two ordered items the
single-factor model is not identified: the one polychoric correlation
determines only the product of the two loadings, while categorical omega
depends on each. The value returned before depended on the start values
of the optimizer and on the order of the columns, and its bootstrap
interval reflected where the optimizer landed. The function now stops
with the message reliability_omega() gives for two
continuous items, and reliability(), whose automatic
routing sends two integer items to categorical omega, lets the error
through rather than substituting another coefficient.
Categorical omega no longer drops an undetermined
polychoric correlation from its total variance. The
unrestricted polychoric correlations behind the observed denominator
came from a second, saturated fit (the approach MBESS also
takes). When the data do not determine a polychoric correlation, as when
two items are skewed in opposite directions and their bivariate table
has empty cells, that fit gives the correlation no weight and returns
its starting value, 0, so the pair was removed from the total variance
without notice and the coefficient was too high, by about .035 for each
such pair. In samples of 500 from the mixed-skew design of Green and
Yang (2009), the help page’s example with N <- 500 and
seeds 1 to 100 in place of 113, 45 of the 99 samples with a proper
solution had at least one such pair. Against the coefficient with the
model implied correlation in the pair, 1.0.1’s was .030 to .045 too high
with one undetermined pair (38 samples) and .065 to .088 with two whose
loadings are identified (3 samples). The correlations now come from the
single-factor fit itself, which agrees with the saturated fit whenever
every correlation is determined, so every other result is unchanged, and
the computation needs one fit instead of two per bootstrap replication;
that fit also no longer computes a test statistic it never used, which
changes no number. Together the two cut the fitting time of a bootstrap
replication by more than half. When a correlation is not determined,
reliability_omega_categorical(denominator = "observed")
stops with an error naming the pair. When the correlations the data do
determine identify the single-factor loadings, it points to
denominator = "model_implied", which does not need the
pair. fa_confirmatory() reports that factor’s categorical
omega as NA with a warning naming the pair. A bootstrap
meets such pairs far more often than the sample does, since a resample
often leaves out the few respondents who determine the correlation
(about half the replications on the help page’s example, whose sample
has no such pair). Those replications are set aside, counted in the new
undetermined element of the bootstrap
attribute, and named in the printed line and the warning. The warning
says that the interval then conditions on the replications that redrew
the respondents who determine the correlation (on the help page’s
example, 80, 95, and 97 of 200 in percentile bootstraps with
seed 1, 2, and 3; with the improper ones, 80, 96, and 98
were set aside, close to the half beyond which no interval is reported,
as the entry on a bootstrap of categorical omega that returns fewer than
half describes), and that denominator = "model_implied"
sets aside only the replications whose determined correlations leave the
loadings unidentified (see Either Variance in the Denominator of Every
Omega). This is a deliberate, loud change for those samples only: a
number that rested on an arbitrary 0 is replaced by an error.
An improper categorical solution is named. When
the single-factor fit returns a standardized loading above one (a
negative residual variance, a Heywood case), categorical omega is not
defined, and the function failed with the error “‘corr’ is not a
correlation matrix” from mvtnorm, under the model implied
denominator only after a message advising that denominator. It now stops
with an error naming each improper item and its loading, before anything
is said about undetermined pairs. A bootstrap replication with an
improper solution is set aside as a non-finite value and counted in the
new improper element of the bootstrap
attribute (it was counted as an error, with the message from
mvtnorm), and the warning from lavaan about
the negative variance no longer repeats once per replication.
reliability_omega_categorical() says which
sum its coefficient describes. Categorical omega scores each
item’s categories 0, 1, …, C - 1 in the order of its codes (Green &
Yang, 2009, Eq. 11; Kelley & Pornprasertmanit, 2016, Eq. 16), as
MBESS and semTools do, and the help page now says so. When an item’s
codes are not equally spaced, or items step by different amounts, the
sum of the codes as given is a different composite, and a message names
the items. In the examples behind the help page its reliability was
0.009 to 0.012 lower for codes -2, -1, 1, 2, and 0.034 lower for items
coded 1, 3, 5 beside items coded 1, 2, 3. The coefficient is
unchanged.
Listwise deletion that leaves too little to analyze says
so. With no complete row, only one, or an item that takes a
single value in the complete rows,
reliability_omega_categorical() printed lavaan’s table of
variables and stopped with “The single-factor categorical CFA model did
not converge.” It now stops before any fit, names the cause, and offers
missing = "pairwise" (when every pair of items is observed
together) or missing = "mi". No estimate changes.
The reliability family treats SPSS user-defined missing
values as missing. A column read by
haven::read_sav(user_na = TRUE) keeps a code declared
missing in SPSS (999 for a test not given, 9 for a refusal) as a value
of class haven_labelled_spss, which SPSS and haven’s
is.na() treat as missing. Through 1.0.1,
reliability_alpha(), reliability_omega(), and
icc() analyzed it as a score or rating.
reliability_kr20() refused it as not 0/1, and
reliability_omega_categorical() counted it as a further
category. fleiss_kappa() summed it as a count.
cohen_kappa() dropped such a pair only when haven was
loaded, and otherwise counted the code as a further category. These
functions and reliability() now set, as
reliability_H() from data, new in 1.1.0, does, the declared
values (na_values and na_range) to missing in
every column before anything is deleted or fitted, as
read_sav() does by default. A message names the columns and
counts, and haven need not be loaded. icc() and
fleiss_kappa(), which do not accept missing values, then
refuse such a column as they refuse NA.
holzinger_swineford,
t1_visual_perception, t2_cubes,
t3_paper_form_board, t4_lozenges, and
t25_paper_form_board_r, with the 156 missing scores of the
last recoded to 999 and declared missing
(x <- holzinger_swineford$t25_paper_form_board_r; haven::labelled_spss(ifelse(is.na(x), 999, x), na_values = 999)).
Version 1.0.1 gave alpha 0.008 and omega 0.015; the functions now give
0.687 and 0.730 under listwise deletion
(missing = "listwise", the default through 1.0.x), the
values for the same data with NA.set.seed(1); t <- rnorm(60), then
round(3 + t + rnorm(60, sd = .5)) for each rater in turn),
with the first six ratings of the third declared missing as 9.
icc() gave ICC(2,1) 0.318, against 0.668 on the complete
rows, and now refuses the data. cohen_kappa() of the first
and third raters gave 0.170 on 60 pairs and seven categories when haven
was not loaded. It now gives 0.196 on 54 pairs and six, as it did with
haven loaded.read_sav()’s default, which already
makes these values NA, are unaffected.reliability_omega_categorical() reads a
factor’s labels as the numbers they name. Through 1.0.1 a
factor item passed only when its labels were 1, 2, …, K in order. A
Likert factor labeled 0 to 4, or one whose lowest category no respondent
chose (levels “2” to “5”), was refused with “column 1 contains
non-integer values”. The labels are now read as numbers, and the item is
analyzed as those numbers under every treatment of missing.
Listwise and pairwise deletion and multiple imputation therefore rank
its categories alike, and the factor gives the result of its integer
codes. For example, the items of the help page’s example recoded as
factors labeled 0 to 4
(as.data.frame(lapply(as.data.frame(items - 1), factor)))
were refused by 1.0.1 and now give 0.720, the value of their codes. Two
kinds of factor are refused, with a message that names the column and
says what to do: one whose labels are words, and an ordered factor whose
levels are not in the order of the numbers they name.
A bootstrap of categorical omega reports no interval when
fewer than half of its replications return a value. Through
1.0.1 reliability_omega_categorical() formed its interval
from any two replications that returned a value. When most were set
aside, the rest were a minority of the resamples, selected by whether
they returned a value, and their spread described those resamples rather
than the sampling distribution of the coefficient.
NA in that
case, under every ci_method and every treatment of
missing. A warning and the printed line say why.set.seed(113); eta <- rnorm(60); y <- 1 * (outer(eta, rep(.7, 6)) + matrix(rnorm(360), 60) * sqrt(1 - .7^2) > 0); y[, 1] <- 0; y[1, 1] <- 1).
A percentile bootstrap (B = 200, seed = 113)
set aside 106 replications in 1.0.1 and gave [0.660, 0.921] from the
other 94. It now reports no interval.Listwise deletion that leaves fewer than two complete rows stops alpha, KR-20, and omega before anything is computed.
reliability_omega() (either denominator) and the model
implied reliability_alpha() passed the data to lavaan,
which printed its table of the variables and stopped on a variance it
could not compute, and the call reported a model that did not converge.
The analytic reliability_alpha() returned NA with a warning
that NaNs were produced, and reliability_kr20() returned NA
for the estimate and every limit with two such warnings. Example: the
first 40 pupils of holzinger_swineford on visual
perception, cubes, lozenges, paragraph comprehension, and sentence
completion, with one score deleted from every row
(d <- holzinger_swineford[1:40, c("t1_visual_perception", "t2_cubes", "t4_lozenges", "t6_paragraph_comprehension", "t7_sentence")]; for (i in 1:40) d[i, (i - 1) %% 5 + 1] <- NA).missing = "fiml", as
reliability_omega_categorical() offers its own treatments
for its items. FIML is offered only when it can use the rows; otherwise
the message says why it cannot, as when a pair of items is never
observed together. An item that no row observes is named instead.
reliability_H() from data, new in 1.1.0, does the
same.reliability_alpha() with
missing = "fiml", which computes the same coefficient
(alpha of 0/1 items is KR-20) from the incomplete rows as well, when
full information maximum likelihood can use them (otherwise the message
says why it cannot). Example: the nine tests
t1_visual_perception to t9_word_meaning, each
scored 1 above its median (as.integer(x > median(x))),
with each of the 301 pupils missing one test in rotation (pupil i misses
test i %% 9 + 1). Version 1.0.1 returned NA;
the call the message offers gives 0.752, 95% CI [0.709, 0.795], against
0.756 for KR-20 of the complete data.An item that is not numeric, or that does not vary, is
refused by name. reliability_alpha(),
reliability_kr20(), and reliability_omega()
now stop before computing anything when an item takes a single value in
the rows analyzed, and name it, as does reliability_H()
from data, new in 1.1.0. The rows analyzed are the complete rows under
listwise deletion, and the rows that observe the item under FIML and
two-stage estimation. A covariance matrix with a zero variance on its
diagonal is refused the same way.
holzinger_swineford with t2_cubes set to 5,
alpha of t1_visual_perception, t2_cubes,
t4_lozenges, and t6_paragraph_comprehension
was 0.487, against 0.548 for the other three. On the first 20 pupils,
with the nine tests t1_visual_perception to
t9_word_meaning each scored 1 above its 90th percentile
among all 301 pupils (as.numeric(x > quantile(x, 0.9))),
no pupil passes four of the tests, and KR-20 was 0.196, against 0.217
for the other five.missing = "fiml"
(for KR-20, reliability_alpha() with
missing = "fiml") only when FIML could use those rows, and
otherwise says why it cannot. A shared column name is a condition to fix
first, rather than an obstacle in the data; fitted coefficients refuse
shared names before checking the rows. FIML is not offered when the
composite is constant in the complete rows and they outnumber the
items.stats::cov()’s “is.numeric(x) || is.logical(x) is not
TRUE”, and FIML with the report about the saturated model. A logical
item is still read as 0 and 1.A bootstrap replication in which an item takes a single value is set aside, with the refusal. The rule above now holds in every bootstrap replication of every continuous coefficient, and in the jackknife behind the BCa acceleration. Categorical omega already set such replications aside, but passed them to lavaan first. It now refuses them by item name before fitting, without printing lavaan’s variable table; their reason is counted as an error with the refusal rather than as an unnamed non-finite value. The same check applies to completed resamples after imputation.
holzinger_swineford,
with the nine tests t1_visual_perception to
t9_word_meaning scored 1 above their medians among all 301
pupils, and t20_deduction scored 1 at the highest score
among the 60 (one pupil). KR-20 is 0.689. Its percentile interval
(B = 1000, seed = 113) was [0.560, 0.779],
with 370 of the 1000 replications computed with
t20_deduction constant. It is now [0.564, 0.775], from the
other 630, with the caution.A composite that does not vary is refused. When
the covariances of the items cancel their variances, the composite has
no variance and no reliability. Ipsative items, whose scores sum to the
same total in every row, are the usual case.
reliability_alpha(), reliability_kr20(), and
reliability_omega() now stop and say so.
t1_visual_perception, t2_cubes,
and t4_lozenges, each centered on the pupil’s mean of the
three (x - rowMeans(x) for all 301 pupils). Alpha was
returned as 1.42e16, with a standard error of -1.42e15 and
NaN limits.The coefficients fitted by the single-factor model need
more complete rows than items. Under listwise deletion, N
complete rows give a covariance matrix of rank at most N - 1. With no
more complete rows than items it is singular, and the maximum likelihood
fit is not defined. That fit lies behind robust and model implied omega
and the model implied alpha, and behind reliability_H()
from data, new in 1.1.0.
t1_visual_perception,
t2_cubes, t4_lozenges,
t6_paragraph_comprehension, and
t7_sentence.missing = "fiml" when FIML could use the incomplete rows,
and otherwise says why it cannot, or that the model needs more
cases.Full information maximum likelihood and two-stage estimation need more cases than variables. Both first estimate the saturated model over the items and any auxiliary variables.
reliability_alpha() and
reliability_omega() now stop before any fit and say so, as
does reliability_H() from data. For the analytic alpha,
listwise deletion is offered only when it would accept the complete
rows, naming Feldt’s interval when exactly two are complete. Bootstrap
and jackknife resamples obey the same rule.t1_visual_perception missing for pupil 4 and
t2_cubes for pupil 5. Version 1.0.1 reported for alpha that
the saturated FIML model did not converge, and for omega that the model
did not converge.At two complete rows, the intervals of alpha and KR-20
that are not defined there are refused. Two rows give a
covariance matrix of rank one, in which every pair of items correlates
+1 or -1, whatever the items. Each interval is refused for its own
reason, as "fisher" already is below N = 4:
"ml" and "adf" standard errors, and
those of their _logistic variants, are zero for any data.
The asymptotic variance of van Zyl, Neudecker, and Nel vanishes at a
matrix of rank one. Each case’s centered cross products are then
proportional to the covariance matrix, which alpha does not change
with.holzinger_swineford (pupil
ids 1 and
54) on t1_visual_perception, t2_cubes,
t4_lozenges, t6_paragraph_comprehension, and
t7_sentence (alpha -0.475). Version 1.0.1 returned:
"ml" and "adf" (a variance of
zero, made negative by rounding error);_logistic variants;bifactor_indices() puts residual covariances
in its model implied denominators and refuses correlated factors and
multiple-group fits. Each model implied denominator is now the
sum of the corresponding block of the item correlations the fitted model
reproduces. For every orthogonal fit without residual covariances that
equals the earlier formula to machine precision, so those results are
unchanged; for a fit that specifies residual covariances the earlier
formula left them out, and a positive one made omega, omega_H, and
omega_HS too high (omega_H .646 against .635 on a test sample with one
cross-group residual covariance). A fit that correlates any two of the
general and group factors, which the help page always excluded, is now
refused with an error naming the pair. A multiple-group fit took its
loadings from the first group and its residual variances from the last,
so its table described neither group: for nine items with general and
group loadings of .5 in two groups of 200 (set.seed(11);
the fixture of the package’s tests), the general factor’s omega was .865
and its omega_H .623, while the model fitted to the first group alone
gives .862 and .621, and to the second group alone .874 and .674. Such a
fit is now refused (see Calls That Are Now Refused); fit the model to
each group’s data to obtain that group’s indices.
bifactor_indices() refuses an item on two
group factors. The help page, the argument, and the error
message all required each item to load on exactly one group factor, but
only an item on none was refused. An item on two took the loading and
the membership of the group factor listed last, so its other loading
dropped out of ECV, omega, omega_HS, H, and PUC, and the result depended
on the order of the model syntax. On the 15-test Holzinger and Swineford
bifactor model with lozenges (t4) also on the verbal factor, the spatial
row described three of its four tests (omega_HS .188, H .254), PUC
counted lozenges as a verbal test, and ECV was .441, where every loading
counted gives .432. Such a fit is now refused, with the items and
factors named. A loading fixed or constrained to zero no longer counts,
so a model written with 0*x3, or with a*x3 and
a == 0, on a second group factor gets the indices of the
same model written without that loading; through 1.0.1 the zero loading
took the item from its real factor. An item listed under one group
factor keeps that loading whatever its value, as before.
bifactor_indices() refuses a fit with one
group factor. DMAR 1.0.0 refused a fit with no group factor
(“No group factors found; a bifactor model needs a general factor plus
two or more group factors.”) and required every item to load on a group
factor (“Every item must load on exactly one group factor.”), so it
described a general factor beside a single group factor when every item
was listed under that factor (some of them with loadings fixed at zero,
and with general named, since both factors then listed
every item), giving a table whose indices divide the common variance
between those two factors alone. Such a fit is not a bifactor model with
subscales, and it is now refused, with a message that names the group
factor and any items on no group factor, as fa_bifactor()
and ci_bifactor_indices(), new in 1.1.0, refuse it. There
is no warning period. The message for an item on two group factors now
says that an item loads on one group factor at most, since an item on
none is accepted when there are two or more group factors.
bifactor_indices() reports H as NA for a
factor with a standardized loading at or beyond one. The index
counts 1 - l^2 as each item’s error, so such a loading leaves an item an
error variance that is not positive and the index is not defined, as
reliability_H() treats an improper solution. Through 1.0.1
the loading was left out, and H described the factor’s other items, with
only the general Heywood warning to show it. On nine simulated items
(set.seed(7); n <- 150; lg <- c(.9, rep(.6, 8)); ls <- c(.42, rep(.5, 8)); g <- rnorm(n); f <- list(rnorm(n), rnorm(n), rnorm(n)),
item i equal to
lg[i] * g + ls[i] * f[[ceiling(i / 3)]] + sqrt(1 - lg[i]^2 - ls[i]^2) * rnorm(n)),
fit as a bifactor model with three orthogonal group factors of three
items each and std.lv = TRUE, the first general loading is
1.06, and the general factor’s H was .841, the H of the other eight
loadings. That value was not a limit of nearby solutions: as that
loading rose to .9999, H rose to .9998, and past one it fell to .841. It
is now NA, and every other cell of the table is unchanged. In
ci_bifactor_indices(), new in 1.1.0, such a replication
yields no coefficient and is set aside under either
improper rule.
bifactor_indices() says which H it reports
and what its group rows mean, and gives the maximal reliability of each
factor. The help page called H “the construct reliability
(maximal reliability) of Hancock and Mueller (2001)”. The column is the
index Rodriguez, Reise, and Haviland (2016) report: Hancock and
Mueller’s closed form on one factor’s standardized loadings, which
treats the variance an item shares with the other factors as
uncorrelated error. In a bifactor model that variance is correlated, so
the index is not the maximal reliability of the factor under the fitted
model, and it can lie on either side of it. In the help page’s example
the general factor’s index is 0.769 and its maximal reliability 0.678,
barely above omega_H (0.673), so no weighting of the items reaches the
index; on the 15-test Holzinger and Swineford model they are .789 and
.681; and with group loadings of mixed sign the maximal reliability is
the larger. On a group factor’s row the maximal reliability is the
larger when the items of the other subscales measure a strong general
factor well, since the optimal weights remove it through them: with nine
items in three subscales, general loadings .7, and group loadings .3, a
group factor’s index is 0.229 and its maximal reliability 0.279. That
maximal reliability belongs to the group factor’s regression score from
all nine items, which weights the other subscales’ items negatively. No
score of the subscale’s own items reaches it, and their best weighting
reaches only the omega_HS of 0.125. The maximal reliability, l’ R^-1 l
with R the item correlations the model reproduces, is the square of the
regression score determinacy that factor_scores() reports.
It is now returned in the new attribute maximal_reliability
(NA for an improper solution), printed under the table, and given an
interval by ci_bifactor_indices(); its sample value, like
the index’s, runs upward, as the help page says. The ECV on a group
factor’s row, which no page defined, is the general factor’s share of
the common variance of that factor’s items. One minus it is the group
factor’s own share, so .85 beside an omega_HS of .08 says the subscale
is mostly general. The omega there is the reliability of the subscale’s
total score. Both are now defined on the help page and in a line under
the printed table, as they are for ci_bifactor_indices().
No value in the table changes.
reliability_H() returns the reliability
family table for every call; the table of 1.0.0 is gone. In
1.0.0 the function took loadings, se_loadings,
and conf_level and returned the row
reliability_H, with ci_lower,
ci_upper, and var_H when
se_loadings was supplied. Such a call now returns the table
every other input returns (rows estimate, se,
ci_lower, ci_upper, conf_level,
N, N_complete, n_items), with no
warning period. With se_loadings it has the same estimate
and the same logit interval, var_H being the square of
se. Without se_loadings it has the estimate,
with se and the limits NA, because
standardized loadings alone carry nothing an interval can be formed
from. Code that read x$value[x$term == "reliability_H"]
reads x$value[x$term == "estimate"]. factor,
group, missing, and aux do not
apply to loadings and are refused. Every interval built from
se_loadings warns, with class
dmar_H_marginal_se, that it ignores the covariances among
the loading estimates. The help page now attributes the logit interval
to Browne (1982) rather than Browne (1968). It replaces the unverified
statement that Hancock and Mueller proved H >= omega >= alpha, as
does the function interactions vignette, which said that H is uniformly
at least omega and omega at least alpha, with the bound that holds by
construction: H is at least the model implied omega of the same
single-factor fit; robust omega need not lie below it. Alpha is at most
omega in the population for congeneric items with uncorrelated
errors.
The bootstrap attribute names two
causes. Every reliability coefficient’s bootstrap
attribute, and its jackknife element, gains
undetermined and improper: the replications
among the non-finite ones that were set aside because the resample left
a polychoric correlation undetermined or gave an improper solution (zero
outside categorical omega). The reasons vector keeps its
1.0.1 shape.
A bootstrap whose returned replicates are all equal
returns that value at both limits. When every item is an exact
linear function of the same variable, every resample gives the same
coefficient. The percentile and BCa intervals of
reliability_alpha() and reliability_kr20()
stopped with “argument is of length zero” (after boot printed “All
values of t are equal”), or with “missing value where TRUE/FALSE needed”
when some replications were set aside. They now return the common value
at both limits, as bootstrap_se already did (and so does
bootstrap_se_logistic, whose limits at a value of one,
identical items, were NA, or one where rounding left the
replicates just below it, since the logit is infinite there), with a
warning that the bootstrap distribution is a single point. A BCa request
falls back to that percentile interval and says why in
ci_method. The engine also warns whenever fewer than two
replications return a value, not only when some were set aside.
B is validated by
reliability_alpha(), reliability_omega(),
reliability_kr20(), and
reliability_omega_categorical(). When a bootstrap
is requested, B must be a single whole number of at least
2, the fewest replications from which a standard deviation or a pair of
quantiles can be formed, as reliability_H() and
ci_bifactor_indices() require. Through 1.0.1
B = 0 or 1 returned NA limits without a
warning (categorical omega did not check B at all, and
B = 1 returned its estimate with NA limits), and a
fractional, negative, or missing B failed inside boot with
an error that did not name it. Each function now stops before anything
is computed, with one message. No estimate or interval changes.
A DMAR table prints to three decimal places, not to three
significant figures. A value in a DMAR table that is not a
whole number printed to digits significant figures, 3 by
default, and format() dropped its trailing zeros and wrote
a small value in scientific notation: 13.018 printed as 13, 0.5596 as
0.56, 233.4107 as 233, 0.041234 as 0.0412, 1.5 as 1.5, and 0.00001234 as
1.23e-05. It now prints to digits decimal places with its
trailing zeros, never in scientific notation: 13.018, 0.560, 233.411,
0.041, 1.500, and 0.000. A value that rounds to zero prints as 0.000
whatever its sign. Whole numbers, such as degrees of freedom and sample
sizes, print without a decimal part as before. A measured quantity that
lands on a whole number, at the bound of its range or where it is fixed,
printed as a whole number too, and where the table shows that it is
measured it now prints to the decimal places of the values beside it
(?dmar_tbl gives the rule):
_lower or _upper,
such as an equivalence margin: a lower limit of exactly 0, which printed
as 0, prints as 0.000 beside an upper limit of 0.083, and a factor
variance fixed at 1 prints as 1.000 beside its interval;pct_ rows of a sensitivity analysis among them), a mean,
median, standard deviation, or other moment, a test statistic, a
standard error, a noncentrality parameter, the delta and the weights of
a model comparison, and a heterogeneity index, wherever they appear: a
CFI of 1 prints as 1.000 beside a TLI of 1.012, a power of 1 as 1.000
beside a power of 0.840, and a standard deviation of the interval widths
of 0 as 0.000 beside their mean of 1.176;dmacs(), where an intercept
given as 2 prints as 2.000 beside one of 2.300.Elsewhere a whole value prints as a count, as before, since nothing
in a long table says which of its other whole numbers are counts: a
planning value given as 15 prints as 15. p-values
(digits_p, four decimal places, with the “< 0.0001”
floor) and the information criteria a table names in its
fixed_terms (digits_fixed, three decimal
places) print as before. print(), format(),
knit_print(), and as_kable() show the same
table, and the displays that lay out cells of their own (those of
fa_exploratory(), factor_count(), and
factor_scores()) and the lines beneath a table print
measured values the same way: a CFI of 1 prints as 1.000 in the display
of factor_count() as it does in the table of
sem_fit_measures(), and a congruence coefficient of 1 as
1.000 in the table of factor_congruence() and in the matrix
beneath it. For code written against 1.0.x:
digits argument of print() and
format() for a DMAR table, and
options(dmar.digits = ), now count decimal places, not
significant figures: print(x, digits = 5) showed five
significant figures and now shows five decimal places, in the table and
in the numbers a display writes beneath it. digits must be
a whole number of 0 or more; a fractional or negative value, which
signif() accepted, stops with an error that says so.format(x) returns the new display strings.x$value, the other columns,
tidy(), and glance() return the numbers at
full precision, as before; read the column, or print with more decimal
places (print(x, digits = 8)), to see more of a number than
the table shows.results_sentence() keeps its own digits,
two decimal places by default, which
options(dmar.digits = ) does not change. It now writes a
value that rounds to zero without a minus sign, as the table does: a
lower limit of -0.004 reads “95% CI [0.00, 0.55]”, where it read
“[-0.00, 0.55]”.A knitted result table carries every line the console
prints. knit_print() and as_kable()
wrote only the sum-of-squares type and the confidence level, so a
knitted omega lost its denominator and residual lines and a knitted
bootstrap interval its replication count (missing since 1.0.1). The
tables of sem_fit_measures(),
equivalence_rmsea(), sem_residual_fit(),
sem_information_criteria(), compare_sem(),
fraction_missing_information(), and
factor_scores() print lines of their own with the table
(which version of a fit measure was reported and why, the sample size
convention, the verdict of a test, what a model comparison checked, how
a fraction of missing information was estimated), and a sample size plan
for incomplete data prints the missing data it assumed. The console
print and the knitr methods now read one footer, which every class
extends through an internal generic; a knitted table sets each footer
line as a paragraph beneath it (two lines no longer run together), and
as_kable() joins them into the caption. A knitted
fraction_missing_information() table leaves out the columns
the console leaves out, and a knitted factor_scores()
result is the summary the console prints (the first rows of each factor,
the reasons values are missing or borrowed, the determinacy of each
missing data pattern, how to read the intervals) rather than every row.
The console’s confidence level is formatted by the same helper as the
knitted one, which prints every conventional level as before, and a
blank line now separates it from the lines after it. The lines above a
fa_confirmatory() table (the measurement structure each
factor’s constraints imply, also for cfa_1() and
cfa_2()) and above a mediation_mbco() table
(the effects and N, the groups, the full model’s deviance, AIC, BIC, and
R squared, how to read each p_value), which print methods
of their own wrote and knit_print() never called, are set
as paragraphs before a knitted table and first in the
as_kable() caption. No stored value changes.
as_kable() and knit_print()
escape the lines they write for LaTeX and HTML. knitr escapes a
table’s cells and column names but writes a caption as it is given, and
as_kable() passed the table’s lines to it without escaping
them. With format = "latex", or the default format in an
.Rnw document, the “%” of “Confidence level: 95%” started a
LaTeX comment that swallowed the caption’s closing brace: a LaTeX
document did not compile, and an R Markdown PDF printed the table’s
LaTeX source as text. An underscore in an item name (the residual
correlation line of an omega) or an ampersand in a citation (the
pairwise deletion line) stopped the compilation as well. The caption is
now escaped for the format knitr renders, and knit_print()
writes its lines in that format’s markup: escaped paragraphs and a
verbatim block for LaTeX, where a “%” had silently cut the rest of its
line, and <p> and <pre> elements
for HTML. Markdown output, the default in R Markdown, is unchanged. A
caption passed through ..., which R refused as
an argument matched twice, is refused with a message that says how to
caption the table otherwise.
Two DMAR tables knitted in one chunk stay apart.
knit_print() wrote its output with no blank line before or
after it, and knitr writes a chunk’s outputs with nothing between them.
In a chunk with echo = FALSE that printed two tables, the
second table’s header row was appended to the first table’s last footer
line (“Confidence level: 95%|term |value |”). Pandoc then read that line
as the second table’s header, in R Markdown’s HTML and PDF output alike:
the footer became a header cell, the second table lost its last column
name, and the first table lost its footer paragraph. The output is now
set off by a blank line on each side, as knitr sets off a kable of its
own.
results_sentence() reads an interval held as
ci_lower and ci_upper, names the estimate the
interval belongs to, and reports the interval’s own confidence
level. In DMAR 1.0.x it read an interval only from rows or
columns then named lower_limit and
upper_limit, and told every table that held one as
ci_lower and ci_upper (a row of
contrast_test(), adjusted_means(),
content_validity_index(), fa_confirmatory(),
or the coefficient table of mlmr() or
mlmr_mv(), and the tidy() results that hold
their limits as ci_lower and ci_upper) that it
“carries no confidence interval”. Every table now names its limits
ci_lower and ci_upper (see Breaking Changes),
so the rows of common_method_adjusted_r(), new in 1.1.0,
are read as well, from the columns of a one-row table, a row beside a
generic estimate column is labeled by its term
(so results_sentence(tidy(ci_smd(...))) writes what
results_sentence(ci_smd(...)) writes), and a table with
neither pair is told what was looked for. Two earlier defects change
sentences it wrote before. The estimate of a wide table was its first
numeric column other than the limits, N, and the p-value,
so the first row of the ?responder_analysis example read “n
= 60.00, 95% CI [0.38, 0.62]” and the second row of
simple_effects_AB(aov(len ~ supp * factor(dose), data = ToothGrowth))
“F_value = 13.33, 95% CI [0.04, 0.35], p = 0.0006”; the estimate is now
the estimate or value column when there is one
and otherwise the first numeric column that is not a limit, standard
error, test statistic, degrees of freedom, p-value, confidence level, or
count (“estimate = 0.50” and “eta_squared_partial = 0.20” there). And
the confidence level was read from the table’s attribute alone, which
the reliability family does not set, so a 90% interval for omega was
written “95% CI”; it is now read from the attribute, the
conf_level row, or the conf_level column. A
tidy() row that the default method builds from a wide table
was labeled “estimate” through 1.0.1 and is now labeled with the
quantity it holds (see the entry on the default tidy()
below). Every other change to a sentence is one of the entries that
follow.
Every table that forms a confidence interval records its
level, and results_sentence() states no level a table does
not record. results_sentence() took 95% when a
table recorded no level, and many recorded none:
ci_eta_squared() and ci_omega_squared()
(and ci_eta_squared_partial(), which this release
removes);vargha_delaney_A() and
pairwise_within();alpha_lower and
alpha_upper (ci_c(), ci_cv(),
ci_mahalanobis(), ci_r(), ci_R(),
ci_rc(), ci_reg_coef(),
ci_rmsea(), ci_sc(), ci_sm(),
ci_smd(), ci_smd_c(),
ci_src());tidy() and glance() results of
every table the default methods handle.A 90% interval was therefore written “95% CI”. With
fit <- aov(len ~ supp * factor(dose), data = ToothGrowth),
results_sentence(ci_eta_squared(fit, conf_level = 0.90)[1, ], digits = 3)
gave “eta_squared = 0.224, 95% CI [0.073, 0.350]”, where the 95%
interval is [0.053, 0.377]. With
set.seed(1); x <- rnorm(40, 0.5); y <- rnorm(40),
results_sentence(vargha_delaney_A(x, y, conf_level = 0.90), digits = 3)
gave “A = 0.674, 95% CI [0.565, 0.767], p = 0.0051”, where the 95%
interval is [0.543, 0.782].
Each of these tables now records its level: conf_level,
or 1 - alpha_lower - alpha_upper when the tail areas are
given, as ci_R2(), ci_pvaf(),
ci_snr(), and ci_srsnr() already did.
vargha_delaney_A(),
pairwise_within(), and the intervals formed from tail areas
print that level beneath the table, which they did not.vargha_delaney_A() and pairwise_within()
return DMAR tables and print as the rest do; their columns are
unchanged. vargha_delaney_A() gains tidy() and
glance() methods whose row is named “A”.pairwise_within(bonferroni_ci = TRUE) records
conf_level, the level its Bonferroni intervals hold
jointly, as contrast_test(adjust = "bonferroni") and the
post hoc intervals record theirs. The line beneath the table of both
functions names the adjustment and the level of each interval:
“Confidence level: 95%, simultaneous across the 3 comparisons
(Bonferroni: each interval at 98.33%)”. With
set.seed(1); Y <- matrix(rnorm(60), 20) + rep(c(0, 0.3, 0.6), each = 20),
the second row of
pairwise_within(Y, bonferroni_ci = TRUE, conf_level = 0.90),
whose interval [0.02, 1.07] is the 96.67% interval of that comparison,
was written “95% CI”; it is written “90% CI”.ci_eta_squared_generalized() recorded the default 0.95
beside the interval that alpha_lower = 0.05 and
alpha_upper = 0.05 define; it records 0.90.tidy() of a wide table with an interval
adds a conf_level column (NA when no interval
was formed). The default tidy() of a long table with an
interval adds a conf_level column that holds the level on
the rows of the interval and NA on every other row. Those
rows are its estimate and its two limits, or an estimate’s
<estimate>_ci_lower and
<estimate>_ci_upper rows at that interval’s own
level. The rows therefore carry their level when tidy()
results are stacked with rbind(): the rows of
tidy(ci_proportion(30, 60, conf_level = 0.99)) stacked
under tidy(ci_proportion(12, 40, conf_level = 0.90)) give
“proportion = 0.50, 99% CI [0.34, 0.66]”.glance() of a long table with an interval
adds estimand and conf_level columns (see the
next entry):
results_sentence(glance(ci_proportion(30, 100, conf_level = 0.90)))
gives “proportion = 0.30, 90% CI [0.23, 0.38]”.results_sentence() reads for the first time
in 1.1.0 carry their level as well. The coefficient tables of
mlmr() and mlmr_mv() record it. The
tidy() results of the reliability family,
fa_confirmatory(), content_validity_index(),
mediation_mbco(),
fraction_missing_information(), mlmr(), and
mlmr_mv() (with conf.int = TRUE) carry a
conf_level column. For
m <- mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning, data = holzinger_swineford, conf_level = 0.90),
results_sentence(m$coef_table[2, ]) reads “t7_sentence =
0.32, 90% CI [0.26, 0.38], p < 0.0001”.results_sentence() now refuses a table that records no
level, such as one assembled by hand, unless its new argument
conf_level gives the level. It also refuses a
conf_level that differs from the level the table records.
No interval changes.
results_sentence() reports the estimate a
long table’s interval belongs to. In a table that holds its
interval in ci_lower and ci_upper rows, the
estimate was the first other row, and two tables list another quantity
first.
correction_for_attenuation(r = .3, reliability_x = .8, reliability_y = .7, N = 200)
gave “correlation_observed = 0.30, 95% CI [0.22, 0.56]”: the observed
correlation beside the interval of the corrected one, .401.proportion_of_superiority(smd = .5, n_1 = 50, n_2 = 50)
gave “smd = 0.50, 95% CI [0.54, 0.82]”, an interval that does not
contain the estimate it names.The two tables now name the row their interval belongs to in a
ci_estimate attribute, which
results_sentence() reads: “correlation_corrected = 0.40,
95% CI [0.22, 0.56]” and “proportion_of_superiority = 0.69, 95% CI
[0.54, 0.82]”. Their rows are unchanged. The tidy() of a
long table names the estimate in its rows, through its
conf_level column, and its glance() names the
column that holds the estimate in an estimand column, so
the sentence of either is the table’s own. Through 1.0.1 the
glance() sentence of the attenuation table also named the
observed correlation.
results_sentence() reports the TOST p-value
of an equivalence test. It appended the first p-value row of a
long table. The tables of equivalence_r(),
equivalence_smd(), and equivalence_c() list
the p-values of the two one-sided tests before the TOST p-value, so a
result that is not equivalent was written with the p-value of one
one-sided test.
equivalence_r(r = .15, N = 300, cor_lower = .2, cor_upper = .2),
whose TOST p-value is .187 (not equivalent), gave “r = 0.15, 90% CI
[0.06, 0.24], p = < 0.0001” (the p-value of the lower test, 5e-10).
It now gives “r = 0.15, 90% CI [0.06, 0.24], p = 0.1870”.equivalence_smd(smd = .3, n_1 = 100, n_2 = 100, smd_lower = .5, smd_upper = .5)
the sentence gives p = 0.0806 where it gave the lower test’s 3e-08.?equivalence_c it gives p = 0.0393
where it gave the lower test’s 3.9e-07.The three tables name p_tost in a
p_sentence attribute, which tidy() and
glance() carry, so their sentences give the same p-value.
The meta-analysis tables, whose first p-value row is the pooled p-value,
are unchanged.
results_sentence() writes “p < 0.0001”,
not “p = < 0.0001”. A p-value below the floor of
format_p() was appended as “, p =” and then “< 0.0001”.
The sentence now writes the bound:
summary_t_test(mean_1 = 2, sd_1 = 1, n_1 = 30, mean_2 = 0, sd_2 = 1, n_2 = 30)
gives “mean_difference = 2.00, 95% CI [1.48, 2.52], p < 0.0001”.
Larger p-values are written as before.
The default tidy() of a wide table with an
interval takes as its estimate the column the interval belongs to, and
its sentence names that quantity. It took the first numeric
column. A row of simple_effects_AB() therefore gave its
F statistic beside the limits for partial eta squared: for
simple_effects_AB(aov(len ~ supp * factor(dose), data = ToothGrowth)),
results_sentence() on its second tidy() row
wrote “estimate = 13.33, 95% CI [0.04, 0.35], p = 0.0006”, where the
estimate is 0.20. A table whose estimate column follows a
count (responder_analysis()) lost the count.
In a table that carries an interval, the estimate is now the table’s
own estimate (or value) column. Otherwise it
is the first numeric column that is not a limit, standard error, test
statistic, degrees of freedom, p-value, confidence level, or
count, the rule results_sentence() follows. The other
numeric columns follow under their own names (the F statistic
as F_value, the count as n).
When the estimate comes from a column with a name of its own, an
estimand column after estimate keeps the name,
and results_sentence() labels the row with it; otherwise it
labels the row with its term. The tidy() row’s
sentence is then the table’s own. content_validity_index()
also gains the estimand column "i_cvi":
content_validity_index(matrix(c(4, 4, 3, 4, 3, 1, 4, 3), nrow = 2), conf_level = .90)
gives “i_cvi = 1.00, 90% CI [0.47, 1.00]” on its first raw or tidy row,
where 1.0.1 refused both as carrying no interval. A tidy FMI row labels
its estimate fmi rather than the parameter’s name. For the
existing examples:
ci_mahalanobis(D2 = .5, n_1 = 40, n_2 = 40, p = 3, conf_level = .9);kappa for fleiss_kappa() and
cohen_kappa(), mean_difference for
pairwise_within(), and ave for
average_variance_extracted();term for icc() (“ICC(2,1)”),
responder_analysis() (“control”), and
adjusted_means() (the group).1.0.1 labeled the others “estimate”; it wrote no sentence for the
tidy() rows of average_variance_extracted()
and adjusted_means(), which hold their limits as
ci_lower and ci_upper, and
pairwise_within() had no tidy() method. The
term column is unchanged. A table without an interval is
tidied as before.
The intervals of ci_eta_squared() for the
effects of a fit with an Error() term are converted with
the size of each effect’s stratum. DMAR 1.0.x (in
ci_eta_squared() and in
ci_eta_squared_partial(), which this release removes)
converted the noncentrality limits of every effect of an
aovlist fit into limits for partial eta squared with the
total number of rows, lambda / (lambda + N). Within
subjects that put the interval on a smaller quantity than the one the
partial eta squared beside it estimates: for the three IQ occasions of
pygmalion analyzed in vignette("DMAR"), 930
rows against F(2, 618), the 95% interval was [0.111, 0.194] and
lay below the estimate of 0.212. Each effect is now converted with the
size of its own stratum. Within subjects that size is the stratum’s
total degrees of freedom, here 2 + 618 = 620, which gives [0.157,
0.265]; the within-subjects example of ?ci_eta_squared goes
from [0.015, 0.322] to [0.023, 0.417]. In the between-subjects stratum
of a mixed design it is the number of subjects: with the expectancy
treatment added to the pygmalion analysis as a
between-subjects factor, the upper limit for treatment, converted with
the 310 children rather than the 930 rows, goes from 0.018 to 0.052.
Fits without an Error() term convert with the total sample
size and the raw-argument interface with the N it is given,
as before, and their results are unchanged; for an effect tested between
subjects the total sample size makes the interval exact for the
population partial eta squared that Steiger (2004), Kelley (2007), and
Smithson (2001, 2003) define. Smithson’s conversion with the effect and
error degrees of freedom plus one, which Fidler and Thompson (2001) also
used, agrees with that definition only in a one-way design. The help of
ci_eta_squared() gives the reasons and the literature,
which does not settle the within-subjects case. The raw-argument
interface now accepts an N as small as
df_effect + df_error, the size of a within-subjects stratum
that tests one effect, which DMAR 1.0.x refused. It does not warn when
N is the number of rows of a within-subjects design, since
it cannot tell that number from the total sample size of a
between-subjects factorial design.
The parametric interval of
ci_eta_squared_generalized() for the effects of a fit with
an Error() term is converted with the size of each effect’s
stratum. DMAR 1.0.x converted the noncentrality limits of every
effect of an aovlist fit with the total number of rows, and
within subjects the interval was for a smaller quantity than the one the
estimate estimates: for the three IQ occasions of
pygmalion, the 95% interval was [0.019, 0.037] against an
estimate of 0.041. Each effect is now converted with the size of its
stratum, as ci_eta_squared() converts it (the entry above),
which gives [0.029, 0.054]. With that size the conversion maps each
limit to the value of the population generalized eta squared of Olejnik
and Algina
(2003) when the ratio of the other sums of squares in its denominator
to the effect’s error sum of squares is at the value it converges to. In
the between-subjects stratum of a mixed design the size is the number of
subjects: with the expectancy treatment added to the
pygmalion analysis between subjects, the upper limit for
treatment goes from 0.015 to 0.044. The within-subjects example of
?ci_eta_squared_generalized goes from [0.005, 0.145] to
[0.008, 0.203]. Fits without an Error() term are converted
with the number of observations, as before, and their intervals are
unchanged. The help now sets out the quantity the conversion is made for
and says that the interval cannot be exact when anything measured, or
another stratum’s residual, enters the denominator: generalized eta
squared then depends on a second parameter besides the noncentrality
parameter, which alone determines the distribution of the effect’s
F, and the interval takes that parameter’s sample value as
known. The F interface now accepts an N as small
as df_effect + df_error, the size of a within-subjects
stratum that tests one effect, which DMAR 1.0.x refused.
eta_squared_generalized() and
ci_eta_squared_generalized() stop when an effect of a fit
with an Error() term is tested in more than one
stratum. When observations are missing within subjects, or a
covariate varies within them, aov() divides the effect’s
sum of squares among strata and tests the effect in each of them. DMAR
1.0.x then returned a row for every stratum each time the effect’s name
came up, each computed from the effect’s share in the first stratum: for
20 subjects in two groups observed three times, with four observations
missing, aov(y ~ group * time + Error(subject / time)) gave
nine rows for its five tests, time and
group:time four times each, and the parametric interval
stopped with an error from inside the noncentral F computation
(“the condition has length > 1”). No one stratum’s share is the
effect’s sum of squares, and Olejnik and Algina (2003) define the
estimate for balanced designs, so both functions now stop and name the
effects and strata involved. A fit in which every subject is observed
equally often in every within-subjects condition, and every covariate is
constant within subjects, tests each effect in one stratum, whatever the
sizes of the between-subjects groups, and its results are
unchanged.
ci_eta_squared_generalized() gives the
parametric interval from sums of squares, and
eta_squared_generalized() accepts the degrees of freedom
that come with them. The interval needs df_effect,
df_error, and N beside the sums of squares.
Without them DMAR 1.0.x stopped and asked for them; with them it stopped
with “‘F_effect’ must be a single non-negative number”, because
eta_squared_generalized(), which computes the estimate,
then read the call as the F interface as well; called directly
with the same arguments, it stopped the same way. The sums of squares
never gave an interval, and with the degrees of freedom present they
gave no estimate either, whatever the method. Both functions now take
df_effect and df_error with the sums of
squares and read only F_effect, F_observed,
and df_observed as the F interface. The estimate
is computed from the sums of squares alone, which need no degrees of
freedom, and the interval from the F they imply,
(SS_effect / df_effect) /
(SS_error / df_error), with N
checked as the F interface checks it. That F is the
effect’s test when SS_error is the sum of squares of the
effect’s error term: the residual in a design without an
Error() term, and otherwise the residual of the stratum in
which the effect is tested, with the residuals of the other strata in
SS_observed, beside the measured sources (Olejnik &
Algina, 2003). The help of both functions now says so, and that each
F of the F interface divides by the effect’s error
mean square. For the one-way analysis of IQ gain by grade in
pygmalion, the sums of squares of the anova()
table, with N = 310, give 0.118 [0.048, 0.176], the row of
the fitted model and of the F interface. Given no fitted model
and both interfaces, ci_eta_squared_generalized() answers,
as DMAR 1.0.x did, when they describe the same test: the F that
the sums of squares imply for the effect, and the ratio of
SS_observed to SS_error, agree with the
F interface’s to a relative tolerance of 1e-6, the tolerance
eta_squared_generalized() applies to the two estimates, and
the interval is formed from the F-values. When they disagree it
now stops, whatever the method, and so it does when only part of the
F interface comes with the sums of squares; a fitted model
still sets every raw argument aside with a warning, and
eta_squared_generalized(), which computes no interval,
still accepts the two interfaces together when they give the same
estimate. DMAR 1.0.x accepted both interfaces whenever they gave the
same estimate and built the interval from the F-values, but the
two can agree on the estimate and still imply different F
statistics for the effect. Sums of squares of 100 for the effect, 70 for
the measured sources, and 200 for the error, on 2 and 50 degrees of
freedom, imply an F of 12.5; an F of 25 for the effect
and of 85 for a measured source on 1 degree of freedom give the same
estimate, 0.270. With N = 53 the sums of squares give the
interval [0.089, 0.413] and the F-values [0.127, 0.381], which
DMAR 1.0.x reported for the two together. Both functions now also refuse
a sum of squares, an F-value, or a number of degrees of freedom
that is missing or infinite, and
ci_eta_squared_generalized() an infinite N.
DMAR 1.0.x stopped on most missing values with R’s “missing value where
TRUE/FALSE needed”, returned an estimate of 0 or NaN for an
infinite value, and gave the F interface the interval [0, 0]
for an infinite N.
The partial omega squared of each effect of a fit with an
Error() term is computed with the size of its stratum, and
the stratum is named. DMAR 1.0.x
(omega_squared_partial(), which this release removes;
omega_squared(fit, estimand = "partial") is its call)
computed df_effect (F - 1) / (df_effect (F - 1) + N) with
the total number of rows: for the three IQ occasions of
pygmalion, 930 rows against F(2, 618), it gave
0.150, below the partial eta squared of 0.212 and its interval [0.157,
0.265]. With the stratum’s 2 + 618 = 620 the estimate is 0.209, an
estimate of the quantity whose interval
ci_eta_squared(estimand = "partial") gives; for treatment
in the mixed design of the entry above, computed with the 310 children,
it goes from 0.004 to 0.012. The value with the number of rows is the
partial omega squared that Olejnik and Algina (2003) compute for a
repeated measures factor, and the raw-argument interface given that
number as N still returns it. The table of an
aovlist fit gains a stratum column after the
estimate, as in eta_squared(), which the description of the
function promised; its N column still reports the number of
observations, and the help says how to find the size of a stratum from
summary() of the fit. Fits without an Error()
term are unchanged. The raw-argument interfaces of
omega_squared() and ci_omega_squared() now
accept an N as small as df_effect + df_error,
which DMAR 1.0.x refused, so that ci_omega_squared() gives
a row’s estimate and its interval from the row’s F, its degrees
of freedom, and the size of its stratum, and
omega_squared() gives the same estimate.
ci_omega_squared() now takes an aovlist fit
itself (see Total and Partial Eta Squared and Omega Squared).
The tidy() of an ANOVA effect size table
returns every effect. tidy() of
ci_eta_squared(), ci_eta_squared_partial(),
ci_eta_squared_generalized(), and
ci_omega_squared() in DMAR 1.0.x returned the first row of
the table alone, so the tidy() of a factorial fit dropped
every other effect, and glance() was the same row. For
fit <- aov(len ~ supp * factor(dose), data = ToothGrowth),
tidy(ci_eta_squared_partial(fit, conf_level = 0.90)) gave
one row, the supp effect (.224 [.073, .350]);
tidy(ci_eta_squared(fit, estimand = "partial", conf_level = 0.90))
gives three, adding factor(dose) (.773 [.659, .814]) and the interaction
(.132 [.011, .246]).
The term names the effect size and, for
ci_eta_squared() and ci_omega_squared(), its
estimand (eta_squared_partial or
eta_squared_total, where 1.0.1 named
eta_squared; see One Function for Each ANOVA Effect Size),
and an effect column after it names the effect; a
stratum column follows it for a fit with an
Error() term. A table of one effect and one estimand, as
from summary statistics, therefore gives the row it gave with the
effect column added. Its glance() is that row
when tidy() gives one row. Otherwise it is one row with the
effect size (term, the estimands separated by a comma when
there are two), the number of effects (n_effects),
N, and conf_level.
manova_split_plot(),
mixed_anova(), and anova_within_two_way()
return DMAR tables. DMAR 1.0.x returned each result as a plain
data.frame, so class(x) was
"data.frame" and the table printed through
print.data.frame(), which formats a column of numbers as a
whole: one small p-value put the whole p_value
column in scientific notation. In the example on the help page of
mixed_anova(), the three p-values printed as
2.148782e-05, 1.817787e-12, and 9.565106e-01. Each function now returns
its table with the class c("dmar_tbl", "data.frame"), as
anova_within() and factorial_anova() do, and
it prints through the dmar_tbl display layer: whole numbers
such as degrees of freedom without a decimal part, other values to 3
decimal places, and p-values to 4 decimal places, with one
below 0.0001 printed as “< 0.0001” (in that example, < 0.0001,
< 0.0001, and 0.9565). Adding the class changes nothing else in the
object, and as.data.frame(x) returns the table as a plain
data frame. For mixed_anova() and
anova_within_two_way() that is the 1.0.x data frame itself
(identical() to it): the columns, rows, and values are
unchanged, as is the sum_of_squares_type attribute of
mixed_anova(). For manova_split_plot() the
columns and rows are those of 1.0.x, but not every value: its default
Type III B rows now test the within-subjects main effect,
where those of 1.0.x tested a simple effect (see the entry above on that
test), and its other rows, with the B rows under
ss_type = 2, agree with those of 1.0.x to rounding error
(within 1e-12, relative, on the designs compared) rather than exactly.
For code written against 1.0.x:
is.data.frame(x) and
inherits(x, "data.frame") are TRUE as before,
and x$p_value and the other columns hold the table’s
numbers at full precision. Code that tests the class itself sees the new
one: identical(class(x), "data.frame") is
FALSE; class(x) == "data.frame" has length 2,
so if () on it stops with “the condition has length > 1”
in R 4.2.0 and later (earlier versions warn and use the first element,
FALSE); and against a saved 1.0.x result of
mixed_anova() or anova_within_two_way(),
identical() is FALSE and
all.equal() reports a difference in the class alone.x[i, ], subset(), and
rbind() with x first keep the class, so their
results print the new way too; merge() still returns a
plain data frame, and summary() and
as.matrix() return what they return for the plain data
frame. print() and format() now dispatch to
the dmar_tbl methods, so format(x) returns the
display strings described above rather than those of
format.data.frame(). In an R Markdown document a table that
a chunk prints is set by knit_print.dmar_tbl() as a
knitr::kable() table of those strings where 1.0.x showed
console text; knitr::kable(x) called directly formats the
numbers itself, as before.tidy() and glance() answer through the
default methods for a wide DMAR table (see ?dmar_tidiers).
tidy() gives one row per row of the table, its
term joining the label columns (effect with
statistic_name, denominator, or
adjustment), its estimate the first numeric
column (statistic_value, ss, or
F_value), and the other numeric columns after it under
their own names. The last row of the tidy() of
manova_split_plot() is the table’s record of the
sum-of-squares type: its term is
"sum_of_squares_type:NA" (the row’s
statistic_name is NA), its
estimate the type, 2 or 3, and its other columns
NA, so code that summarizes the estimates of the tests
leaves that row out, as it would from the table. glance()
gives one row: the number of rows (n_terms), with
conf_level and B_used NA, since
no interval or bootstrap is formed. For the plain data frame of 1.0.x
the generics package has no method, and where broom was loaded its data
frame tidiers, which broom has deprecated, answered: tidy()
stopped with an error on the character columns, and
glance() gave nrow, ncol,
complete.obs, and na.fraction.
results_sentence() refuses these tables as before, since
none holds a confidence interval. as_kable(), which stopped
on the plain data frame of 1.0.x with “no applicable method”, now
returns a knitr::kable() table of the display strings, and
dmar_metadata(), new in 1.1.0 and taking only a DMAR table,
accepts these tables.ancova() already returned a dmar_tbl in
DMAR 1.0.x, and its result is unchanged.
ss_aipe_reliability_sensitivity(estimator = "omega")
returns intervals. Since 1.0.0 it asked
reliability_omega() for the "mlr" interval
under the default observed denominator, for which no Wald interval is
defined; every replication failed silently, and the table showed no
widths and a non-coverage of zero. Omega is now computed with
denominator = "model_implied", the coefficient the
planner’s interval is built on, with the "ml" interval the
plan describes (see the entry on the interval its plan was made for). A
replication that returns no interval is now set aside rather than
counted as covering the true value; the proportions are over the
replications that returned one, one warning states how many were set
aside and why, and a run in which none returned an interval stops with
the reason. The rule changes nothing for alpha with Bonett’s interval,
its default through 1.0.1.
ss_aipe_reliability_sensitivity() evaluates
the assurance plan at the planner’s defaults. With an
assurance, the plan whose N is simulated was searched at 50 and 200
replications (G_initial and G, then
initial_iter and final_iter) against the
planner’s defaults of 500 and 5000, and the noise let a size below the
one the assurance needs pass by chance. For four parallel items of
reliability .80, a width of .15, and an assurance of .80, the evaluated
N ranged from 93 to 101 over seeds 1 to 30 (set.seed(s)
before each call; mean 96.9) against 98 to 101 (mean 99.2) at the
defaults, so a correctly specified plan seemed to fall short of its
assurance. The plan now runs at the planner’s defaults, as the rest of
the sensitivity family does, which adds about a second. With
ci_method = "ml" the replications’ alpha intervals change
as reliability_alpha()’s do, slightly for the unit variance
items simulated here (mean width .1531 to .1527 in 1,000 replications at
N = 74, after set.seed(113)); Bonett’s interval, the
default through 1.0.1, is unchanged.
ss_aipe_reliability_sensitivity() evaluates
the interval its plan was made for. The plan is
ss_aipe_reliability()’s for the parallel model with the
normal theory interval, a Wald interval with the van Zyl, Neudecker, and
Nel (2000) variance. The replications, however, computed Bonett’s (2002)
interval for alpha and the "mlr" interval for omega, so a
correctly specified plan was judged by an interval it was not made for.
The default ci_method is now "ml" for both
estimators, with complete or incomplete data. For alpha that is
reliability_alpha(ci_method = "ml"), whose variance at a
population of parallel items is the compound symmetry form the plan
uses. For omega it is
reliability_omega(denominator = "model_implied", ci_method = "ml"),
whose delta method variance equals that form there to machine precision.
Under missing data these are the Wald intervals with the FIML standard
error that the fmi adjustment plans for.
set.seed(113) before
each call
(true_reliability = .8, supposed_reliability = .8, n_items = 4, width = .15, assurance = .8, G = 2000).
The plan of 99 cases gives intervals no wider than .15 in 81.3 percent
of the 2,000 replications, against 76.75 percent with
ci_method = "bonett" on the same draws, and a mean width of
.1312 against .1346.estimator = "omega", G = 1000), which
returned no intervals in 1.0.x, the default gives 83.3 percent against
81.9 with ci_method = "mlr".G = 20) goes from a
mean width of .1685 and a proportion of .40 to .1630 and .45.ci_method = "bonett" reproduces 1.0.x
without an assurance, apart from the replications the entry on clamped
limits describes. With an assurance the plan is now searched at the
planner’s defaults (see the entry on the assurance plan), which draws
other replications: the call above gave 78.65 percent and a mean width
of .1337 in 1.0.1. Any other interval can still be named.Noncentral chi square intervals stay right for a
statistic in the millions. R’s pchisq() with a
noncentrality stops its series after 10^6 terms, and above a
noncentrality of about 2e6 it returns wrong probabilities (0 where the
value is .19 at 3e6), with a warning that was suppressed.
ci_nc_chisq(), and through it ci_rmsea(),
ecvi(), and the intervals of
sem_fit_measures() and equivalence_rmsea(),
then returned limits collapsed onto one value below the statistic. For a
statistic of 3e6 on 3000 df the 90 percent limits were 2979734 twice and
are now 2991307 and 3002700; the RMSEA interval at N = 1e5, with the
multiplier N, was [0.09966, 0.09966] around an estimate of 0.09995 and
is now [0.09986, 0.10004]. Above a noncentrality of 1e5, the
distribution function is now summed as the Poisson mixture of central
chi square distributions it is, accurate to rounding. At and below 1e5
nothing changes. equivalence_rmsea() and the close fit
tests of sem_fit_measures() use the same function: a bound
of .10 at a statistic of 3e6 had a p-value of 0 and was declared met,
where the p-value is .193. For a fitted model,
sem_fit_measures() computes the limits and tests itself
once the statistic, or the noncentrality of a close fit bound, exceeds
1e5. The robust limits and tests of a FIML, ordered data, or two-stage
fit, which it cannot rebuild, become NA with a warning past
1e6.
ci_rmsea() gains
sample_size_convention = c("N-1", "N"). It sets
the multiplier with which the reported RMSEA was computed; lavaan’s
default likelihood and Mplus use N. The default keeps every earlier
result. The help page now says correctly that the lower limit of the 90
percent interval settles the test of close fit and the upper limit the
test of not close fit (it had described the upper limit as the test of
close fit), and that the function assumes one group.
ci_rmsea() refuses zero degrees of freedom
and a sample size that leaves no multiplier. A saturated model
(lavaan reports an RMSEA of 0 on 0 df) had returned limits of Inf with
no warning, the lower one above the estimate. N = 1 under the default
convention had given an upper limit of Inf, and N between 0 and 1 an
NA. df must now be greater than 0; it need not
be whole, since a mean and variance adjusted statistic has fractional
degrees of freedom. N must make the multiplier positive. An
invalid conf_level is refused in its own name rather than
as alpha_lower and alpha_upper.
ci_rmsea() forms a 90 percent interval by
default. The default conf_level is 0.90, where
DMAR 1.0.x had 0.95, since a 100(1 - 2 alpha) percent interval decides
the one-sided tests of close and not close fit at level alpha
(MacCallum, Browne, & Sugawara, 1996, p. 136), so the limits of the
90 percent interval are those tests at .05 (see Every RMSEA Interval at
90 Percent by Default). For an RMSEA of .035 on 40 df with N = 425, the
interval of the help page’s first example, the limits were .008 and .055
and are .015 and .052. conf_level = 0.95 restores the 1.0.x
interval. An interval given by alpha_lower and
alpha_upper is unchanged.
ss_aipe_rmsea() plans a 90 percent interval
by default. The default conf_level is 0.90, where
DMAR 1.0.x had 0.95, so that the plan is for the interval
ci_rmsea() now forms by default. A lower level needs a
smaller sample for the same width: for an RMSEA of .045 on 5 df and a
width of .060, necessary_N was 1068 and is 802.
conf_level = 0.95 restores the 1.0.x plan.
ss_aipe_rmsea_sensitivity() simulates 90
percent intervals by default and forms each as lavaan does, from the
replication’s statistic with lavaan’s multiplier.
conf_level is 0.90, where DMAR 1.0.x had
0.95, as in ss_aipe_rmsea() and ci_rmsea().
conf_level = 0.95 restores the level.ci_rmsea(). The interval matched neither convention and was
slightly too wide, that of a sample one case smaller. It is now formed
with lavaan’s own multiplier (N under the default likelihood, N - 1
under likelihood = "wishart" or a least squares estimator
passed through ...). This was a defect, and no argument
restores it.ci_rmsea()), so every such
replication was given the interval of that statistic, wider than the
interval of its own. Each interval is now formed from the replication’s
chi square statistic, as lavaan and sem_fit_measures() form
it, and equals the interval lavaan reports for the sample whatever the
statistic; its upper limit is 0 when the statistic is at or below the (1
- conf_level) / 2 quantile of the central chi square, the
convention the two share. A model that fits well, many of whose
replications fall below their degrees of freedom, gets narrower
intervals: for the two-factor population of the help page fit by its own
two-factor model (8 df), with N = 200, G = 40,
conf_level = 0.90, and set.seed(113), 21 of
the 40 statistics fell below 8, the mean width, .0920 in DMAR 1.0.x, is
.0802, and the proportion of intervals narrower than a width of .05, 0
before, is .10. This was a defect too, and no argument restores it. A
replication whose statistic exceeds its degrees of freedom gets the
interval it had, so a markedly misspecified model, such as that of the
help page’s example, is not affected.necessary_N, the size a study would collect. The width
depends on the sample size only through the multiplier, and
ss_aipe_rmsea() plans with N - 1 (Kelley & Lai, 2011),
so under lavaan’s default, whose multiplier is N, the realized intervals
are a little narrower than the planned width, by an amount of order
1/N.set.seed(113), G = 20), DMAR
1.0.x evaluated a sample of 695 and reported a mean width of .05002; the
default now evaluates 493 for the 90 percent interval, with a mean width
of .04990, and conf_level = 0.95 evaluates 695, with a mean
width of .04998.ecvi(fit) divides by the multiplier lavaan
used to form the statistic. For a fit under
likelihood = "wishart" or a least squares estimator (GLS,
WLS, ULS, DWLS) that multiplier is N - G, and the earlier divisor N
disagreed with lavaan::fitMeasures(); the three-factor
Holzinger and Swineford model under the Wishart likelihood gave 0.4220
and now gives lavaan’s 0.4234, and the interval limits move with it.
Results for ML under the default normal likelihood, and for summary
input, are unchanged. The help page now states the identity N ECVI = AIC
+ 2 l_s only for the likelihood ratio statistic.
ecvi() applies the checks of the SEM fit
family and states its boundary cases. A fitted model with an
inadmissible solution is now refused, as sem_fit_measures()
and sem_information_criteria() refuse it; a 60 case
two-factor fit of the Holzinger and Swineford tests with a latent
variance of -72.9 had an ECVI of 1.116 [0.799, 1.598]. A well fitting
model, whose statistic is below the upper alpha/2 quantile, no longer
warns about a prob_greater row that ecvi()
does not return. When the statistic lies at or below the lower alpha/2
quantile, the upper limit is still NA, now with a warning
in ecvi()’s own terms. The help page says why that limit is
not set to the lower one, and that the untruncated estimate lies below
the interval when the statistic is below its degrees of freedom.
n = 0, which returned Inf and NaN, is refused.
An ML fit is read as ML under lavaan 0.7-3.
lavaan 0.7-3 lists Browne’s residual based statistic beside the standard
one for an ML fit by default, and measurement_invariance(),
which asked whether the test option was exactly "standard",
labeled an ML ladder as using the scaled difference test and robust fit
indices. It now sets aside the standard statistic and every Browne
residual statistic (alternative tests, not corrections) when deciding
whether a fit carries a robust test. The profile likelihood interval for
model implied omega and alpha no longer warns at every refit that lavaan
cannot compute Browne’s statistic under its equality constraint. No
number changes.
measurement_invariance() reports one version
of the test and of the fit indices at every rung, and its attributes
name the versions. Outside FIML the ladder read the names of
its measures from its first rung alone. In the examples,
hs_factors is the verbal and deduction model of the help
page.
test = "mean.var.adjusted"), for which lavaan computes no
robust indices, the cfi and rmsea columns held
the standard indices while the fit_indices attribute said
"robust". The columns now hold the robust indices, which
DMAR computes because they do not depend on which corrected statistic
accompanies them (Savalei, 2018); they equal lavaan’s under
estimator = "MLM", and a line under the table says so. On
the help page’s first ladder
(measurement_invariance(holzinger_swineford, hs_factors, group = "school", test = "mean.var.adjusted"))
the configural CFI was .9916 and is .9928, the configural RMSEA was
.0520 and is .0478, and the metric delta_cfi was -.0127 and
is -.0134. A ladder whose configural rung is saturated gets them too: on
the three verbal tests
(model = "verbal =~ t6_paragraph_comprehension + t7_sentence + t9_word_meaning",
configural, metric, and scalar) the metric delta_cfi was
-.0109 and is -.0100.test = "bollen.stine"), the
chi_square column and the difference test were the standard
ones, while fit_indices said "robust" and
test named a scaled difference test. Those numbers stand,
and the attributes now say "standard".NA
at every later rung, and no delta_cfi. lavaan leaves the
robust indices NA at the rungs that constrain the
thresholds. Every rung now reports the scaled indices, and
fit_indices says "scaled". On the help page’s
ordered example (the verbal and deduction tests cut at their quartiles,
configural and thresholds rungs):
NA and are
.996 and .040;delta_cfi was NA and is -.0023.NA on another) or does not compute
it. Such a rung now reports a CFI of 1, which every version implies (see
the entry on a saturated rung).measurement_invariance(missing = "fiml")
reports the FIML corrected fit indices of Zhang and Savalei (2023), and
a line under the table says so. The corrected indices estimate
the values complete data would give, the values a cutoff for a change in
fit (Cheung & Rensvold, 2002) is a statement about, where lavaan’s
standard cfi and rmsea move with the amount
and kind of missing data; fa_confirmatory() reports the
corrected indices for the same kind of fit. Under FIML the
cfi and rmsea columns, and
delta_cfi and delta_rmsea with them, are
lavaan’s robust versions, chosen as fa_confirmatory()
chooses them: Zhang and Savalei’s version 6, or version 3 with
estimator = "MLR".
estimator = "MLR" it reported them with either lavaan;
those numbers stand.set.seed(113), then
t7_sentence[sample(301, 90)] <- NA), the standard and
the corrected indices are:
NA, and that rung reports a CFI of
1, which every version implies. On visual perception, cubes, and
lozenges by school
(model = "visual =~ t1_visual_perception + t2_cubes + t4_lozenges")
with 60 cube scores deleted at random (set.seed(1), then
t2_cubes[sample(301, 60)] <- NA), configural, metric,
and scalar, lavaan’s robust CFI of the saturated configural rung is
NA. Every rung reports the corrected indices (scalar CFI
.852 and RMSEA .172, where the standard indices are .861 and .159), and
the configural CFI is 1.test = "satorra.bentler" or
"mean.var.adjusted"), and such a ladder is reported and
labeled as the listwise ladder it is.fit_indices attribute names the version
("robust", or "scaled" or
"standard" in the fallback). A line under the table, kept
in the new fit_columns attribute, says which indices the
columns hold, at the console and in a knitted report. Complete data and
listwise and pairwise deletion are not affected by this entry.A saturated rung of measurement_invariance()
has a CFI of 1.
NA or 1 by
rounding. The table reported the NA, and the next rung’s
delta_cfi with it. On the verbal tests of
holzinger_swineford under estimator = "MLR"
(items = c("t6_paragraph_comprehension", "t7_sentence", "t9_word_meaning"),
group = "school"; configural, metric, and scalar), the
configural CFI was NA and is 1, and the metric
delta_cfi was NA and is -.0097. The three
visual tests with 60 cube scores deleted under FIML (see the entry
above) behaved the same way.measurement_invariance() reports no fit
index or difference test of a ULS ladder of continuous items that is
formed from the ULS statistic and changes with their units. The
examples are the help page’s first ladder, the verbal and deduction
tests by school.
estimator = "ULSM"
or "ULSMV" and continuous items, the rmsea
column held lavaan’s robust RMSEA, which lavaan forms by putting that
statistic into a correction derived for the ML fit function. It was
1.57, 4.92, and 4.50 at the configural, metric, and scalar rungs of a
ULSM ladder, and 0.0157, 0.0492, and 0.0532 with every score divided by
10. The rmsea and delta_rmsea columns are now
NA.delta_cfi was -.00005 against -.0046: a judgment against
the cutoff of Cheung and Rensvold (2002) that turned on the units. The
cfi and delta_cfi of those rungs are now
NA. The configural and metric CFI (.9991 and .9886) do not
depend on the units and are unchanged. So are the
chi_square column and the scaled difference test, whose
reference distribution does not depend on the units, though their values
do.estimator = "ULS" and
test = "standard", the cfi and
rmsea columns held the standard indices formed from the ULS
statistic: CFI .9956 and .9818 and RMSEA 3.51 and 6.37 at the configural
and metric rungs, and CFI 1 and RMSEA 0 at both with the scores divided
by 10. Both columns are now NA.estimator = "ULS", alone or with
test = "standard", lavaan’s difference test refers the
difference of the ULS statistics to the chi square distribution. The
metric rung’s delta_chi_square was 91712 (p < .0001),
against 9.17 (p = .057) with the scores divided by 10, and the
test attribute named a scaled difference test (a standard
one with test = "standard"). delta_chi_square,
p_value, and test are now
NA.estimator = "ULS" alone, Browne’s residual
statistic remains a valid test. CFI and RMSEA at configural and metric
rungs, which leave the means free, are unchanged. At scalar and strict
rungs, restrictions link means and covariances through the latent means,
so the ULS estimates change with the units. Browne’s statistic at those
estimates and its indices change too: CFI, RMSEA, and their differences
are now NA there. On the help page’s first ladder,
measurement_invariance(holzinger_swineford, hs_factors, group = "school", estimator = "ULS"),
the scalar CFI was .8874 and its RMSEA .0680 in 1.0.1; both are now
NA.NA and why.
estimator = "MLR" is the estimator for nonnormal continuous
items.measurement_invariance() under FIML leaves
an auxiliary variable’s mean and variance free at every rung.
An auxiliary variable passed through ... as lavaan’s
aux enters a FIML fit as a saturated correlate (Graham,
2003). lavaan makes its mean and variance parameters of each group, and
the ladder’s group.equal reached them. The scalar rung
equated the auxiliary’s means across the groups and the strict rung its
variances, so each of those rung tests also tested, with one more degree
of freedom, whether the auxiliary is distributed alike in the groups.
They are now added to group.partial at every rung, beside
any group_partial terms. The ladder then has the degrees of
freedom of the ladder without the auxiliary, and its differences are
those of the same ladder with the auxiliary’s moments free.
Example: a spatial factor of visual perception, cubes, and lozenges
and a verbal factor of paragraph comprehension, sentence completion, and
word meaning by school, with 60 sentence completion scores deleted at
random (set.seed(1), then
t7_sentence[sample(301, 60)] <- NA) and general
information as the auxiliary (means 44.8 and 36.6 in the two
schools).
Under two-stage estimation the auxiliaries enter the first stage only, so its ladders hold no such parameters.
measurement_invariance() merges lavaan’s
group.partial passed through ... into
group_partial, and refuses group.equal
there. Each rung passes lavaan one group.partial,
so through 1.0.1 a call that gave both group_partial and
group.partial stopped with lavaan’s error that a
snake_cased name is duplicated. The terms of both are now freed at every
rung, beside those of an auxiliary variable under FIML. On the six tests
by school above under FIML, without the auxiliary,
group_partial = "t2_cubes ~ 1" with
group.partial = "t4_lozenges ~ 1" gives a scalar test of
1.56 on 2 df, where 1.0.1 stopped.
group.equal in ... constrained the
configural rung, which was still reported as configural, and a ladder
that went past it stopped with the same error. On the six tests by
school above under listwise deletion,
group.equal = "loadings" with
levels = "configural" gave a configural rung of 25.62 on 20
df against the configural model’s 15.27 on 16. It is now an error that
names levels and group_partial.
htmt() returns the same four columns
whatever B. ci_upper existed only
when B > 0, so the shape of the table depended on an
argument. It is now always present, NA when no bootstrap is
run. A bootstrap bound also carries the B_used attribute
(the resamples left after any were dropped), which glance()
reports. No value changes.
htmt() takes the absolute value of every
correlation, the monotrait correlations within each block as well as the
heterotrait correlations between blocks. Through 1.0.1 only the
heterotrait correlations were absolute, a mix that matches neither the
signed ratio nor the absolute one semTools::htmt()
computes. A reverse keyed item that was not recoded then lowered its
block’s average within-construct correlation while the numerator kept
its size, so the ratio was inflated. Take one such item in a block of
four:
set.seed(113); n <- 300
f1 <- rnorm(n); f2 <- 0.5 * f1 + sqrt(0.75) * rnorm(n)
d <- data.frame(a1 = .8 * f1 + rnorm(n, 0, .6),
a2 = .7 * f1 + rnorm(n, 0, .7),
a3 = .8 * f1 + rnorm(n, 0, .6),
a4 = -.6 * f1 + rnorm(n, 0, .8),
b1 = .8 * f2 + rnorm(n, 0, .6),
b2 = .7 * f2 + rnorm(n, 0, .7),
b3 = .6 * f2 + rnorm(n, 0, .8))
blocks <- list(A = c("a1", "a2", "a3", "a4"),
B = c("b1", "b2", "b3"))
htmt(d, blocks)
fa_confirmatory(d, factors = blocks, output = "measurement")The two constructs, which correlate .5 in the population, had an HTMT
of 1.346, a discriminant validity failure that is not there, and
fa_confirmatory(output = "measurement") reported it beside
phi = .473 without a word; a block of three with such an item stopped
the call. Every correlation now enters as an absolute value, so the
ratio does not depend on how an item is keyed (1.346 becomes .463, the
value of the recoded data). When no correlation within a block is
negative, as with consistently keyed items, the point estimates are
unchanged, the example of ?htmt and three factors of
holzinger_swineford included. The bootstrap bound can move,
though: a resample that draws a negative correlation within a block now
enters it with its size, where 1.0.1 averaged it with its sign or
dropped the resample. Take two blocks whose within-block correlations
for B are .04, .11, and .20:
set.seed(4); n <- 80; f <- rnorm(n); g <- rnorm(n)
d <- data.frame(a1 = f + rnorm(n, sd = .5),
a2 = f + rnorm(n, sd = .5),
b1 = .45 * g + rnorm(n), b2 = .45 * g + rnorm(n),
b3 = .45 * g + rnorm(n))
htmt(d, list(A = c("a1", "a2"), B = c("b1", "b2", "b3")), B = 200,
seed = 113)The HTMT is .408, as before, and the bound .728 from 200 resamples,
where it was .869 from 198. The bound of the ?htmt example
is unchanged. On complete data the values agree with
semTools::htmt(htmt2 = FALSE, absolute = TRUE). On
incomplete data they agree only when semTools is given the same
pairwise-complete correlations through sample.cov, since by
default it deletes incomplete rows listwise. The help page no longer
credits the absolute form to a 2021 paper missing from its references,
and a pair whose ratio cannot be computed is reported as the next entry
describes.
htmt() reports a pair whose ratio cannot be
computed as NA, with a warning, and
fa_confirmatory(output = "measurement") keeps its
table. A correlation that cannot be computed leaves a pair’s
ratio undefined. That happens with two items never observed together, as
in a planned missing data design whose forms give two blocks to
different respondents, or with a constant item. Through 1.0.1 such a
pair across two blocks was NA without a word, and a pair
undefined within a block stopped the call with “missing value where
TRUE/FALSE needed”.
The pair is now NA, as is its bootstrap bound, with a
warning that names the pair and the reason, and the other pairs are
reported. The call stops, with a message that says why, only when no
pair is defined. A call with only two blocks whose one pair is
undefined, which returned NA, therefore stops.
fa_confirmatory(output = "measurement") keeps every phi,
omega, AVE, and H row and reports the undefined
htmt row as NA with the warning, as 1.0.1 did
without the warning. Take two three-item factors of 300 cases with a1
and b1 on different forms:
set.seed(113); n <- 300
f1 <- rnorm(n); f2 <- 0.4 * f1 + sqrt(.84) * rnorm(n)
d <- data.frame(a1 = .8 * f1 + rnorm(n, 0, .6),
a2 = .7 * f1 + rnorm(n, 0, .7),
a3 = .7 * f1 + rnorm(n, 0, .7),
b1 = .8 * f2 + rnorm(n, 0, .6),
b2 = .7 * f2 + rnorm(n, 0, .7),
b3 = .7 * f2 + rnorm(n, 0, .7))
d$a1[c(TRUE, FALSE)] <- NA; d$b1[c(FALSE, TRUE)] <- NA
fa_confirmatory(d, factors = list(A = c("a1", "a2", "a3"),
B = c("b1", "b2", "b3")),
missing = "fiml", output = "measurement")This gives phi .380 [.239, .521], omega .793 and .784, AVE .558 and .549, and H .798 and .792.
krippendorff_alpha() returns the same seven
rows whatever boot. The ci_lower,
ci_upper, and B_used rows existed only when
boot = TRUE, so the shape of the table, and of its
tidy() and glance(), depended on an argument,
as htmt()’s did. They are now always present, and
NA when no bootstrap is run, which is the default: the
nominal ratings of Krippendorff (2011, Table 1) give seven rows where
they gave four, with alpha .743 and every other value unchanged. As in
htmt() and average_variance_extracted(), the
confidence level is recorded, and printed, only beside an interval that
was formed. A script that tested for the rows by name
("ci_lower" %in% x$term) should test for a finite value
instead. results_sentence() now refuses a table whose
limits are NA rather than writing “95% CI [NA, NA]”.
average_variance_extracted() has returned its
ci_lower and ci_upper columns under every
ci_method since 1.0.0, and its tests now pin the columns of
its tidy() and glance() as well.
krippendorff_alpha() checks B
and reports the bootstrap resamples it drops. B
must now be a whole number that meets the rule for bootstrap
replications when boot = TRUE (400 at the 95 percent level;
see One Rule for Bootstrap Replications); through 1.0.1 any value was
accepted, and B = 10 gave a percentile interval from ten
resamples. A resample on which alpha cannot be computed (too few
pairable values, or values all alike) is dropped as before, but now with
one warning that gives the count, and the limits are withheld when too
few resamples remain. When the data themselves leave alpha undefined
(every value alike), no bootstrap is run: the interval rows stay
NA and no confidence level is recorded, where the table had
said B_used = 0 beside “Confidence level: 95%”. Intervals
are unchanged.
The bound for close fit in
sem_residual_fit() is credited to the paper that proposed
it. The help page attributed the bound of .05 times the average
R squared to Ximénez, Maydeu-Olivares, Shi, and Revuelta (2022). It is
the bound of Shi, Maydeu-Olivares, and DiStefano (2018). Their criterion
also requires the largest absolute standardized residual covariance to
be at most .10 (with .10 times the average R squared and .15 for
acceptable fit), and they apply both as population values through an
interval or a test. The page and the SEM fit vignette now say so, and
say that the function tests the first part and reports the second as a
sample value. Ximénez et al. (2022) are cited for finding that the bound
works reasonably well for the unbiased SRMR in finite samples, except
when the loadings are small. The sem_fit_measures() page
and the vignette now also cite Steiger (1990, p. 178), which states in
print the result of Steiger (1989) behind gamma hat: gamma hat and
adjusted gamma hat estimate the population values of the GFI and the
adjusted GFI.
The Steiger and Fouladi (1997) chapter is cited by its
title. The help pages of smd(),
ci_smd(), ci_smd_c(), ci_sm(),
cles(), ci_mahalanobis(),
ci_nc_t(), ci_sc_ancova(),
ss_aipe_smd(), ss_aipe_smd_sensitivity(),
ss_aipe_sm(), ss_aipe_sm_sensitivity(),
ss_aipe_sc(), ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova(), and
ss_aipe_sc_ancova_sensitivity() gave its title as
“Noncentrality interval estimation and the evaluation of statistical
methods”; the chapter is “Noncentrality interval estimation and the
evaluation of statistical models” (pp. 221–257). The help page of
sem_fit_measures() and the SEM fit vignette, both new in
1.1.0, cite it by that title for the intervals of gamma hat.
The journal and subtitle of Kelley, Maxwell, and Rausch
(2003) are given as published. The help pages of
ci_smd(), design_consequences(),
ss_aipe_c(), ss_aipe_c_ancova(),
ss_aipe_equivalence_r(),
ss_aipe_equivalence_smd(), ss_aipe_mean(),
ss_aipe_smd(), and ss_power_smd() gave the
journal as “Evaluation and the Health Professions” and the subtitle as
“Delineating methods of sample size planning”; the journal is
Evaluation & the Health Professions and the subtitle is
“Delineating methods of sample-size planning”, as the DOI record of the
article (10.1177/0163278703255242) has them. The volume, issue, and
pages, 26(3), 258–287, already matched the record and are
unchanged.
The edition of Rencher and Christensen (2012) is given as
published. The help page of manova_split_plot()
gave Methods of multivariate analysis (Wiley, 2012) as the
fourth edition; the 2012 book is the third edition, as the catalog
record of its print edition (ISBN 978-0-470-17896-6) and the Crossref
record of its review in International Statistical Review,
81(2), 328–329, have it. The entry gains the book’s DOI,
10.1002/9781118391686.
The edition of Anderson (2003) is given as
published. The help pages of ci_eigenvalue() and
ci_mahalanobis() gave An introduction to multivariate
statistical analysis (Wiley, 2003) as the fourth edition; the 2003
book is the third edition, as the publisher’s page for it (ISBN
978-0-471-36091-9) and catalog records of the print edition have it, and
as the pages of expected_r() and
expected_partial_r() already cite it. The entry on the
ci_mahalanobis() page also sets the title in sentence case,
as the other entries do. Crossref holds no record of the book, so the
entries carry no DOI.
ss_aipe_reliability() returns the AIPE
planner class with an assurance. The closed form plan carried
class dmar_ss_aipe and the assurance plan did not, so
tidy() and glance() answered differently by
path: for an assurance plan tidy() gave five term and
estimate rows rather than the planned N beside the width. Both paths now
carry the class, so an assurance plan’s tidy() is one row
(term = "sample_size", estimate,
width) and its glance() adds
assurance, assurance_attained, and
G as columns. The values of the table are unchanged (its
rows are renamed; see Breaking Changes in 1.1.0). Two arguments are now
checked before they are used: a vector true_residual_var
for the Parallel model stopped with R’s “the condition has length >
1” and now stops with the function’s own message, and a True Score plan
from true_factor_loading without n_items
stopped with “argument is of length zero” and now says that
n_items is needed. A vector
true_factor_loading for the Parallel model was recycled
across the items, so it planned the congeneric pattern it describes
under the Parallel label without saying so (loadings .8, .7, .7, and .8
with true_residual_var = .4 returned N = 96). It is now an
error, as it already was for the True Score model, with a message that
names the Congeneric model, and both models refuse any
true_factor_loading that is not one number.
ss_aipe_reliability() plans for the
coefficient of the raw items. The tau equivalent model on every
route, and the congeneric model given by S, turned the
population covariance matrix into a correlation matrix before the closed
form and the Monte Carlo draws. That plans for alpha or omega of
standardized items rather than for the coefficient
reliability_alpha() and reliability_omega()
estimate, and the same congeneric population entered by loadings or by
S got different searches. Every route now uses the
covariance matrix itself. With equal loadings .7 and error variances .1,
.2, 1, and 2, the normal theory plan for a width of .10 goes from 206 to
367 cases (the factor analytic one to 242; see the entry on that
interval). At 367 the corrected interval averages .1004 wide and covers
.941, while at 206 the 1.0.1 interval was as narrow as planned (.1008)
but covered .869 (1,000 samples of each size, drawn as in the entry on
that interval and analyzed by
reliability_alpha(S = S, N = N, ci_method = "ml")). A
correlation matrix still plans for standardized items, and items of
equal variance keep the closed form they had.
Seeded assurance searches with the normal theory interval of the tau
equivalent model move a little for any population. Each sample’s
interval now takes its standard error from that sample’s covariance
matrix, whose variances differ even when the population’s do not, and a
population whose equal variances are not one is now drawn from its
covariance matrix. With true_item_covariance = .3 and error
variances of .2 on five items, a width of .10, an assurance of .85,
G_initial = 50, G = 200, and seed 113, the
plan goes from 81 cases (empirical assurance .905) to 79 (.885). The
True Score example of the 1.0.1 help page
(true_item_covariance = .3,
true_residual_var = c(.2, .3, .3, .2, .3), width .10) goes
from a closed form of 84 to 86 and, with an assurance of .85
(G_initial = 50, G = 200, seed 113), from 115
to 118. The page now plans for items near unit variance
(true_item_covariance = .5,
true_residual_var = c(.4, .5, .6, .4, .6)): 108 cases, or
140 with that assurance (106 and 138 under 1.0.1). Its plan from a
correlation matrix (pop_mat, width .15) with that assurance
goes from 120 cases (empirical assurance .85) to 121 (.855), because its
Monte Carlo intervals now use the sample covariance matrix. The Parallel
examples give the plans they gave (38 cases, and 57 with an assurance of
.85).
ss_aipe_reliability() plans coefficient
omega from omega’s own variance. Without an assurance the
congeneric model returned Terry and Kelley’s (2012) Eq. 19, the variance
of alpha, which they used only as a starting value for a Monte Carlo
search on omega’s mean width (p. 390). The closed form is now the delta
method variance of the model implied omega from the expected information
of the single-factor model at the population (Raykov, 2002), the
variance behind
reliability_omega(denominator = "model_implied", ci_method = "ml").
When the population is given by S or data, the
model is first fit to it by maximum likelihood. Eq. 19 was too large
when standardized items have unequal loadings and could be far too small
when the item variances differ. At width .15, loadings .9, .8, .3, and
.2 of standardized items go from 297 to 177 (the interval averages .114
wide at 297 and .153 at 177); loadings .3, .5, 1, and 2 with error
variances .1, .3, 1, and 3 go from 75 to 151 (.211 wide at 75, .148 at
151); and the help page’s example goes from 53 to 52. At width .10,
loadings 2, 1, .6, and .4 with unit error variances go from 360 to 229,
and six standardized items with loadings from .4 to .8 from 197 to 183.
The widths come from 1,000 samples of each size, drawn as in the next
entry and analyzed by
reliability_omega(S = S, N = N, denominator = "model_implied", ci_method = "ml");
three of those at 177 return no interval.
The first-order approximation can miss the mean width when samples at the planned size approach improper solutions or empirical underidentification. The median can also exceed the target. The loadings, item count, and sample size act together; no lower bound on individual loadings separates these populations. Three-item models are just identified, and a sample covariance near zero can make their estimated loadings run away. Wider targets can create the same problem with more items. The help describes these cases and points to an assurance above one half, whose search counts the realized widths. A congeneric population with two items is refused.
ss_aipe_reliability() plans the factor
analytic interval of the tau equivalent model for that
interval. Without an assurance the True Score model returned
the normal theory size whatever type was, so
type = "Factor Analytic" planned for the van Zyl,
Neudecker, and Nel (2000) interval of alpha, which through 1.0.1 was
evaluated on the correlation matrix. It did not plan for the interval
the user asked for: the delta method interval of the coefficient implied
by the single-factor model with equal loadings. That is the interval of
reliability_alpha(estimator = "model_implied", ci_method = "ml")
and the one the assurance search draws. That coefficient is the maximum
likelihood estimator under the model, so its variance is no larger than
the normal theory variance of alpha. The closed form now takes its delta
method variance, with the information and gradient of the one loading
summed over the items. A population given by S or
data is first fit by the equal loadings model. Take equal
loadings .7 and error variances .1, .2, 1, and 2 at a width of .10.
1.0.1 returned 206 cases, too few for this interval, which averages .108
wide at 206 cases; about one sample in eight gives an interval no wider
than .10. The normal theory plan for these items is 367, and the factor
analytic plan is now 242, at which that interval averages .10 wide.
These widths come from 1,000 samples of each size drawn after
set.seed(1) as the assurance search draws them, each
S <- var(MASS::mvrnorm(N, rep(0, 4), Sigma)) from the
population matrix Sigma, and analyzed by
reliability_alpha(S = S, N = N, estimator = "model_implied", ci_method = "ml").
With equal error variances the two variances coincide, so the Parallel
model keeps its closed form, and no normal theory plan is
affected.
ss_aipe_reliability() refuses a population
it cannot plan for. Through 1.0.1 each of the following was
planned without a word:
true_residual_var must now be
above zero for the Congeneric model and for the True Score model with
the factor analytic interval, as it already had to be for the Parallel
model. These plans fit the single-factor model, at whose boundary an
error variance of zero lies. At a width of .15, loadings .7, .6, .5, and
.5 with error variances .51, -.1, .75, and .75 gave 81 cases for the
congeneric model, and one loading of .7 on all four items with those
error variances gave 46 for the tau equivalent one.S must. The plan is the one the same matrix gives as
S: 74 cases for that tau equivalent population at a width
of .15, by either route.S or implied on that normal
theory route. Loadings .7, .6, .5, and .5 with error variances .51, -.3,
.75, and .75 imply a matrix whose smallest eigenvalue is -.144; given as
S at a width of .15, it gave 25 cases for the True Score
model and 115 for the Parallel model.true_cov_matrix or
data whose single-factor fit is improper or has no solution
is refused.true_factor_loading = c(.7, .6, 0) with
true_residual_var = c(.51, .64, 1) gave 937 cases at a
width of .15.true_cov_matrix or
data. Alpha at or below zero is equivalent only when the
item variances are equal: at a zero loading, the likelihood weights each
covariance by the inverse of the two item variances, and alpha does not.
A population supplied through true_item_covariance or
true_factor_loading is refused when its loading is zero. A
Parallel plan whose alpha is not positive despite a nonzero fitted
loading is refused as far from parallel, with the True Score model named
as the alternative. The normal theory interval still plans for alpha of
such items.The widths in this entry come from 1,000 samples of each size, drawn
as in the previous entry and analyzed by
reliability_omega(S = S, N = N, denominator = "model_implied", ci_method = "ml").
With the factor analytic interval, the assurance search now starts at no
fewer than one more case than there are items, the fewest a
single-factor model can be fit to. A highly reliable population planned
for a wide interval can have a closed form below that: loadings .95 on
four items need two cases for a width of .2. The search used to stop
there.
ss_aipe_reliability() warns when the
loadings of a congeneric population differ in sign. The warning
names the items that load in the direction opposite to the others. Such
items lower omega of the unweighted sum, as an item keyed in the
opposite direction does when it is not reverse scored, and the plan is
for omega of the sum of the items as given. Standardized items with
loadings .7, -.7, .5, and .5 at a width of .10 now plan 1419 cases with
the warning; 1.0.1, with alpha’s closed form, planned 4884 without
one.
The assurance of ss_aipe_reliability()
counts a simulated study that returns no interval as one whose interval
is too wide. The assurance is the probability of an interval no
wider than desired (Terry & Kelley, 2012), and a study that ends
without an interval has not had one. Through 1.0.1 a Monte Carlo sample
whose single-factor fit did not converge was set aside, and the
proportion was taken over the rest. A sample whose solution had run away
(error variances summing below zero, a model implied coefficient above
one) was kept, and its interval, often inverted (an estimate of 104
between 104 and 1), was counted as narrow or wide.
reliability_omega() now returns no coefficient for a
runaway solution, and the search treats each sample as the estimation
functions treat it. A sample that returns no interval, whether its fit
did not converge, had no solution, or ran away, counts as wide. Every
assurance plan carries their count in no_interval, zero
when all samples returned an interval, and glance() carries
the count as a column. A nonzero count is printed under the table. Both
confidence limits are clamped into [0, 1], as the estimation functions
clamp them, which changes no count. The rule changes nothing in a search
in which every sample returns an interval, as every normal theory search
does. Samples without an interval are rare unless several loadings are
.3 or below. Take standardized items with loadings .8, .3, .2, and .2
and a width of .40 (200 samples of each size, drawn and analyzed as in
the entry on the populations the planner refuses):
G_initial = 50,
G = 200, seed 1), 1.0.1 returned 110 cases at an empirical
assurance of .626, taken over the samples whose fit converged. The plan
is now 118, at .545, and 39 of the 200 samples of the final pass return
no interval.ss_aipe_reliability()’s assurance search can
go below its starting size, says why it stops, and works from
data. The search started at the closed form size and only went
up. When that size already met the assurance, as with an assurance near
one half or below, or with the factor analytic interval of the tau
equivalent model, which is narrower than the normal theory interval
1.0.1’s closed form described, the search returned the closed form size
whatever the assurance needed. For the tau equivalent population of
equal loadings .7 and error variances .1, .2, 1, and 2 with the factor
analytic interval, a width of .10, and an assurance of .80, 1.0.1
planned from the correlation matrix and returned its start, 207 cases,
at an empirical assurance of .99 for standardized items. That is too few
for alpha of the items as they are: at 207 cases .156 of the samples
give an interval no wider than .10 (1,000 samples, drawn as in the entry
on that interval). The plan is now 263 (G_initial = 100,
G = 400, seed = 1), searched up from the
factor analytic closed form of 242. The search now moves down to where
the widths it drew would meet the target and then up one case at a time;
a search whose start falls short, and in which every sample returns an
interval, draws what it drew before from the same matrix (a tau
equivalent population whose variances are not one is now drawn from its
covariance matrix rather than its correlation matrix, and so draws
differently). A size at which no Monte Carlo sample returns an interval
stops with the first error, where the search stopped with “missing value
where TRUE/FALSE needed”. How a sample that returns no interval counts
is in the entry on the assurance of ss_aipe_reliability().
The congeneric model with type = "Normal Theory", which the
help already called an error, is refused at once (the closed form
returned an N without reading type). A population given by
data now sets the number of items, so an assurance plan
from data no longer stops with “invalid ‘times’ argument”.
ss_aipe_reliability() names the common
covariance of the items for what it is. The argument was
cor_est and was called the presumed inter-item correlation,
but the function takes its square root as the common loading beside the
error variances, so it is the common inter-item covariance, and it is
the correlation only when
true_residual_var = 1 - true_item_covariance, Terry and
Kelley’s standardized design. It is now
true_item_covariance, and the help says what it is. The
Parallel examples planned from a covariance of .3 with error variances
of .2, items that correlate .6; they now give the same populations as
standardized items (true_item_covariance = .6,
true_residual_var = .4) and say so, and their plans are the
same. The True Score example now plans another population (see the entry
on the coefficient of the raw items). The page also says that a
correlation matrix plans for standardized items, and how the closed
forms and the assurance search work.
The help says what seed = NULL
does. The pages of the reliability family,
krippendorff_alpha(), randomization_test(),
randomization_test_paired(), mlmr(),
mlmr_mv(), R2_mixed_effects(),
plot_trajectories(), and
plot_trajectories_fitted() said that a NULL
seed leaves the random number state intact. In fact the draws come from,
and advance, the caller’s stream; it is a supplied seed that leaves the
state as it was. The help of reliability_alpha() and
reliability_omega() also now names the delta method behind
bootstrap_se_logistic (Kelley & Pornprasertmanit, 2016,
Eq. 32). No number changes.
Suggested packages. lavaan’s floor rises from
0.7-2 to 0.7-3, the version every two-stage and factor score result was
checked against (factor_scores() refuses ordered fits under
0.7-2 in any case); mice (3.18.0 or newer, which keeps
ordered factors ordered when it augments the data) is added for multiple
imputation in categorical omega; and mvtnorm is no longer
suggested.
A confidence level is displayed with the digits it needs,
never as 100% when it is below 1, and whatever
getOption("digits") is. Through 1.0.1 the labels
of plot_ci(), plot_smd(),
plot_R2(), and plot_mediation_mbco() and the
caption of correlations_test() rounded the level to a whole
percent, so a 97.5% interval was labeled 98% and a 99.9% interval 100%,
a level only an unbounded interval has:
plot_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.975)
read “98% CI [0.043, 0.954]” and now reads “97.5% CI [0.043, 0.954]”,
plot_R2(R2 = 0.25, N = 100, p = 5, conf_level = 0.999) read
“100% CI [0.017, 0.477]” and now reads “99.9% CI [0.017, 0.477]”, and a
correlations_test() result at
conf_level = 0.975 is captioned “Correlations (Pearson,
97.5% CI)”, where it read “98% CI”.
results_sentence() wrote the level with
getOption("digits") significant digits: under
options(digits = 2),
results_sentence(ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.975))
gave “smd = 0.50, 98% CI [0.04, 0.95]”; it gives “smd = 0.50, 97.5% CI
[0.04, 0.95]”.ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.99999995)
printed “Confidence level: 100%” and the sentence “100% CI [-0.61,
1.61]”. Digits are now added until a level below 1 no longer reads 100
(“99.999995%”).getOption("OutDec"),
as the table’s numbers and the sentence already did: under
options(OutDec = ",") it reads “Confidence level: 97,5%”
beneath a table of 0,043 and 0,954, where it read “97.5%”.summary() of an mlmr() fit and
confint() of mlmr() and mlmr_mv()
wrote the level and the interval columns with
getOption("digits"): under
options(digits = 2), a fit with
conf_level = 0.975 gave its intervals at “level = 0.97”
over columns “1.3 %” and “99 %”. It gives them at “level = 0.975” over
“1.25 %” and “98.75 %”, the labels stats::confint() writes.
At the default settings nothing changes.pairwise_within() and
contrast_test() (each of 28 intervals at a family level of
.999 is at 99.996%, which four significant digits would have written as
100%), the interval columns of a printed factor_scores()
table, and the interval that equivalence_rmsea()
names.No interval and no stored level changes; only the display does.
An interval formed from unequal tail areas says so, and a
one-sided interval is written as the confidence bound it is. An
interval with tail area alpha_lower below its lower limit
and alpha_upper above its upper limit has confidence
coefficient 1 - alpha_lower - alpha_upper
whatever the split (Neyman, 1937), and that is the level the table
records, but “95% CI [L, U]” is read as the equal-tailed interval.
Through 1.0.1 nothing recorded the split:
ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, alpha_lower = 0.01, alpha_upper = 0.04)
gave the sentence “smd = 0.50, 95% CI [0.03, 0.85]”, read as the
equal-tailed 95% interval, which is [0.10, 0.90]. Each constructor that
takes alpha_lower and alpha_upper
(ci_c(), ci_cv(),
ci_eta_squared(),
ci_eta_squared_generalized(),
ci_mahalanobis(), ci_omega_squared(),
ci_pvaf(), ci_r(), ci_R(),
ci_R2(), ci_rc(), ci_reg_coef(),
ci_rmsea(), ci_sc(), ci_sm(),
ci_smd(), ci_smd_c(), ci_snr(),
ci_src(), and ci_srsnr()) now records the two
areas when they differ, and:
results_sentence() writes “smd = 0.50, 95% CI [0.03,
0.85] (unequal tail areas: 0.01 below, 0.04 above)”;alpha_lower = 0 and alpha_upper = 0.05 the
sentence of the ci_smd() call above was “smd = 0.50, 95% CI
[-Inf, 0.83]” and is “smd = 0.50, one-sided 95% upper confidence bound
0.83”;
ci_rmsea(rmsea = 0.06, df = 24, N = 300, alpha_lower = 0.05, alpha_upper = 0)
was “rmsea = 0.06, 95% CI [0.04, Inf]” and is “rmsea = 0.06, one-sided
95% lower confidence bound 0.04”; and
ci_R2(R2 = 0.2, N = 100, p = 3, alpha_lower = 0.05, alpha_upper = 0)
was “R2 = 0.20, 95% CI [0.08, 1.00]” and is “R2 = 0.20, one-sided 95%
lower confidence bound 0.08”. The line beneath the table says which
limit is the bound (“one-sided: the upper limit is the 95% upper
confidence bound, with tail area 0.05 above it and none below the lower
limit”). An interval is taken to be one-sided only from a tail area of
0, never from a limit, since an equal-tailed interval can have a lower
limit of 0 because the statistic fell below its critical value;tidy() and glance() add
alpha_lower and alpha_upper columns after
conf_level, holding the two areas on the rows of the
interval (NA on the other rows of a long
tidy()), so that rows stacked from several results keep
their own split and their sentence states it. A stack built with
rbind() from an equal-tailed and an unequal-tailed result
stops on their different columns rather than state either split for the
other’s rows, and one built with dplyr::bind_rows() holds
NA, read as equal tails, on the equal-tailed rows;plot_ci() names the split in a subtitle (see the entry
on plot_ci()).An equal-tailed interval, the usual one, is unchanged: it records no
areas, prints and reads as before, and its tidy() and
glance() gain no column. No limit changes.
ci_c(), ci_sc(),
ci_c_ancova(), ci_sc_ancova(), and
ci_c_ancova_bp() require the standard deviation of the
errors and refuse one that is not a single positive, finite
number. The standard error of a contrast is that standard
deviation times a function of the weights and the sample sizes (and, in
ANCOVA, of the covariate means), and it cannot be recovered from the
means. Through 1.0.1 it was checked only on some paths:
ci_c(means = c(1, 2, 3), c_weights = c(1, -1, 0), n_per_group = c(10, 10, 10)),
with no s_anova, returned a zero-width interval at the
estimate, -1, -1, and -1, labeled “Confidence level: 95%”, with the
sentence “contrast = -1.00, 95% CI [-1.00, -1.00]”;ci_c_ancova(adj_means = c(7.5, 12, 14), c_weights = c(1, -1, 0), n_per_group = c(10, 10, 10), cov_means = c(5, 6, 7), SSwithin_x = 750),
with no s_ancova, returned -4.5, -4.5, and -4.5 labeled
95%, and s_ancova = 0 returned the same;s_anova = -1 in
the ci_c() call gave a lower limit of -0.08 above an upper
limit of -1.92, and s_ancova = -2 in the
ci_c_ancova() call gave -2.66 above -6.34;ci_sc() with means and no s_anova, and
ci_sc_ancova() with no s_ancova, stopped with
“argument is of length zero”; ci_sc_ancova() with
s_ancova = 0 stopped with “lower < upper is not
fulfilled”;ci_c_ancova_bp() stopped on an NA with
“missing value where TRUE/FALSE needed” and returned infinite limits for
Inf.Each of these is now refused with a message that names the argument
and says what it is: s_anova for ci_c() and
ci_sc() on every path, s_ancova for
ci_c_ancova() (either procedure),
ci_sc_ancova(), and ci_c_ancova_bp(), and
s_anova for ci_sc_ancova() when it is the
standardizer. A call with neither the means nor the contrast, which
failed with “arguments imply differing number of rows: 3, 0” in
ci_c(), is refused in words. Every interval formed from a
valid standard deviation is unchanged.
plot_ci() titles a DMAR table with the level
the table records. The title came from the
conf_level argument, used only for the label, so a table
formed at another level was titled with its default:
plot_ci(ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.90))
was titled “95% Confidence Interval” above the limits of a 90% interval,
and a 97.5% table the same. The level the table records (its
conf_level attribute, or a conf_level column
holding one value) is now used when conf_level is not given
(“90% Confidence Interval”), and a conf_level that differs
from it is refused, since the plot draws the limits it is given and
cannot change their level; rows that record several levels are refused
as well. A frame that records no level, and explicit
estimate, lower, and upper
values, keep the argument’s level, .95 by default. An interval with
unequal tail areas is drawn with a subtitle in the words of
results_sentence() (“Unequal tail areas: 0.01 below, 0.04
above”, or “One-sided: the upper limit is the 95% upper confidence
bound”). The limits drawn are unchanged.
equivalence_c() has one more row and one
more verdict. An interval that contains
-contrast_lower and whose upper limit is below
contrast_upper (delta_lower and
delta_upper in 1.0.x) was labeled “Inconclusive” through
version 1.0.1 and is now “Not relevantly superior only”, in the
verdict attribute and in plot_equivalence()’s
colors. The second example of the help page,
equivalence_c(contrast = -5.28, se = 2.49, df_error = 399, contrast_upper = 5),
with 90% limits -9.39 and -1.17, reads so now. “Inconclusive” is left
for an interval that contains both bounds. Code that reads the rows by
term is unaffected; code that reads them by position meets
an 18th row, nonsuperior, after inferior, and
the first 17 keep their places. No estimate, limit, or p-value
changes.
ss_power_R2() stops when
null_R2 is not 0, and the test of an increment in R^2 has
its own function. ss_power_R2() plans the omnibus
test that the population squared multiple correlation coefficient of all
p predictors is zero. A nonzero null_R2, which
DMAR 1.0.x accepted, is now an error of class
dmar_null_R2_unsupported (a dmar_refusal)
whose message says what MBESS calculated with a nonzero
null_R2 and writes the call to
ss_power_R2_change() with the values given. No value is
returned for such a call, in place of the following.
With fixed predictors, 1.0.x and MBESS formed (true_R2 -
null_R2) / (1 - true_R2), which is Cohen’s
(1988) f^2 for the increment when predictors are added to a model
that already explains null_R2, and referred it to an
F distribution with p and N - p - 1
degrees of freedom and noncentrality N f^2. With p read as
the number of predictors added, that is Cohen’s calculation for a total
sample size of N plus the number of predictors in the restricted model:
true_R2 = .30, null_R2 = .25,
p = 4, alpha_level = .01, and
specified_N = 85 gave a power of .2225 with a noncentrality
of 6.071, which is the value Cohen’s formula gives at N = 90 with five
predictors in the restricted model, and the search gave 242 where
Cohen’s form gives 247. With p read as all of the
predictors, as the function defines it, the number is not the power of
the test of the increment, whose numerator degrees of freedom are the
number of predictors added: for .25 against .20 with p = 5
and specified_N = 200 it was .8047, where the test of one
predictor added to four has power .953 and the test of two added to
three .911 (fixed predictors; .945 and .901 with random predictors), and
the search for power .80 gave 199, where those two tests need 120 and
148 (127 and 154 with random predictors). With random predictors, the
default, 1.0.x did not form that effect size: given true_R2
it returned alpha_level when true_R2 was at or
below null_R2 and the power of the omnibus test above it
(.0500 at true_R2 = .20 and .9999 at .200001 with
p = 5, specified_N = 200, and
null_R2 = .20; a search for .25 against .20 returned 47,
the sample size of the omnibus test of .25), and given
true_f2 it took the population value as
(true_f2 + null_R2) / (1 +
true_f2) and returned the power of the omnibus test at it
(.9975 for true_f2 = .1, null_R2 = .2,
p = 5, and specified_N = 100, where the same
call without null_R2 gave .6378). The argument alone cannot
say which calculation is meant, because the test of an increment needs
the number of predictors in the restricted model.
What to call instead. The test of the increment:
ss_power_R2_change(true_R2_full = a, true_R2_restricted = b, p_full = , p_restricted = ).
The number 1.0.x returned with fixed predictors, and MBESS returned:
ss_power_R2(true_f2 = (a - b) / (1 - a), p = p, random_predictors = FALSE),
which gives .8047 and 199 for the second design above and .2225 and 242
for the first. A call that gave true_f2 with a nonzero
null_R2 is refused in the same way. With fixed predictors
1.0.x took that true_f2 as the effect size of the
increment, so the same call without null_R2 returns its
number; with random predictors
ss_power_R2(true_R2 = (true_f2 + null_R2) / (1 + true_f2), p = p)
returns the number 1.0.x gave (.9975 in the case above). When
true_R2 is at or below null_R2 there is no
increment to test: 1.0.x returned alpha_level (MBESS
returns NaN), and the message says so in place of the two
calls, which would stop on a negative effect size. Whether the
population squared multiple correlation coefficient exceeds a stated
value above zero is a third question, for which DMAR has no planner in
this release; that test rejects when the one-sided lower confidence
limit from ci_R2() exceeds the value. null_R2
stays in the signature so that an existing call is answered with the
explanation, and null_R2 = 0 is accepted. An invalid
null_R2 (not a single number of at least 0 and below 1)
keeps its own message.
A search with true_R2 = 0 stops with “With ‘true_R2’ of
0 the null hypothesis is true, so power equals ‘alpha_level’ at every
sample size and no sample size attains ‘desired_power’.” The sample size
search doubles the size and then bisects, so a small effect returns
promptly (true_R2 = .00005 with p = 5,
desired_power = .80, and fixed predictors gives 256,545);
the size it returns is the smallest that attains
desired_power, as before, and the search stops with an
error at ten million. No power is returned from an evaluation at which
R’s noncentral F warned of lost accuracy (“full precision may
not have been achieved in ‘pnbeta’”); with a null value of zero no call
was found that reaches that warning. Results with
null_R2 = 0 are unchanged from DMAR 1.0.0: on 1,500
combinations of true_R2 (from .000001 to .999),
p, specified_N, alpha_level, and
predictor model, and 500 searches, version 1.0.0 and this version
returned the same sample size in every search and powers that differ by
at most 6e-11 (the entry below on a very small true_R2
describes a true_R2 near 1e-8).
ss_power_R2() checks its arguments.
alpha_level must be a single number strictly between 0 and
1 (0 and 1 were accepted and returned NaN, 0, or 1);
true_R2 and null_R2 must be single numbers of
at least 0 and below 1 (true_R2 = 1 returned
NaN with an infinite effect size, and
null_R2 = 1 a power of .05 with an effect size of -1);
p must be a whole number of at least 1
(p = 5.5 was accepted); true_f2 must be
nonnegative; and a call with neither true_R2 nor
true_f2 says so, where it stopped with “object ‘f_Squared’
not found”. random_predictors must be TRUE or
FALSE; 1, 0, and the character strings “TRUE” and “FALSE”
were accepted and are now refused.
The random-predictor power of ss_power_R2()
no longer returns NaN for a very small nonzero
true_R2. The parameters of Lee’s approximation
need the square root of a difference that cancels when rho^2 is small,
and near rho^2 = 1e-8 the difference came out negative:
ss_power_R2(true_R2 = 1e-8, p = 10, specified_N = 12)
returned NaN with the warning “NaNs produced” and now
returns .05. The root has a closed form with no subtraction, which the
function shared by ss_power_R2(), ci_R2(), and
ss_aipe_R2() now uses. The limits of ci_R2()
and the sample sizes of ss_aipe_R2() did not change on the
grids compared (1,500 and 65 calls). The power of
ss_power_R2() changes by less than 1e-13 for a
true_R2 of .001 or more and by up to 9e-11 at 1e-6, where
the earlier square root carried the larger rounding error.
ss_aipe_semipartial_r() without
true_R2_full returns a larger sample size when
true_r_sp squared is above 1/2. DMAR 1.0.x planned
with (1 - r_sp2)2, the variance when the variables partialed
out explain none of the outcome. That is the largest the variance can be
when r_sp^2 is at most 1/2, and there the plan is unchanged. Above 1/2
it is the smallest, and the 1.0.x plan was too small unless the
variables partialed out explained none of the outcome: for r_sp = .8,
one variable partialed out (p = 2), and a width of .10 it
gave 202, where an interval of that width needs 331 when the full-model
R2 is .94 (at N = 202 the mean width in 4,000 replications was .127).
The plan without true_R2_full now uses the largest variance
over every possible full-model R2, r_sp^2 (1 - r_sp^2) when r_sp^2 is
above 1/2: necessary_N for that call was 202 and is 357;
for r_sp = .9, p = 3, and a width of .10 it was 59 and is
240; and for r_sp = .75, p = 2, a width of .10, and
assurance = .90 it was 329 and is 386 (with
seed = 113; the plan with assurance is now
found by simulation, as the entry on the assurance step describes).
true_R2_full = true_r_sp^2 restores the 1.0.x plan made
without assurance (with assurance it does not:
the same call then returns 363, because the assurance step changed
too).
ss_aipe_partial_r_sensitivity() and
ss_aipe_semipartial_r_sensitivity() simulate
p - 1 variables partialed out, the count their planners
plan for, where DMAR 1.0.x simulated one fewer for the same
model. In 1.0.x the count was the argument J
(p = J + 1). It was documented there as the total number of
predictors and the simulated data had J + 1 columns, while
the planner each function calls, and the standard error of the interval
each one formed, took the same J as the number of variables
partialed out. With J = 1 the “partial” correlation
simulated was the simple correlation of two variables, given the
interval of a partial correlation with one variable removed. Both
functions now simulate the model the rest of the family describes:
Y, X, and p - 1 variables partialed out,
so the simulated data have p + 1 columns. The planner call
and total_N are unchanged, and the results at a given seed
change, by little, since the interval already allowed for the variables
partialed out: with set.seed(113),
true_cor = .40, specified_N = 30, three
variables partialed out (p = 4; J = 3 in
1.0.x), a width of .40, and G = 2000, the standard
deviation of the partial correlations was .1666 and is .1717, and with
fisher_z = TRUE, the interval 1.0.x formed, the proportion
of intervals that missed the true value was .048 and is .057 (Monte
Carlo standard error .005). With the default interval of this release,
on the raw scale, that proportion is .091, which also reflects the
change of interval described in the entry on fisher_z. This
was a defect, and no argument restores the earlier results; a model with
nothing partialed out is ss_aipe_r_sensitivity().
ss_aipe_semipartial_r_sensitivity() forms
the interval the planner plans for. Each replication’s interval
was formed on Fisher’s Z scale with standard error 1 / sqrt(N -
p - 1), by analogy with the partial correlation. No function of a
semipartial correlation alone stabilizes its variance, which depends on
the full-model R2, and ss_aipe_semipartial_r() plans the
width of a Wald interval on the correlation’s own scale. Each interval
is now that Wald interval: the estimate plus and minus the normal
critical value times the square root of the first-order variance
(var_semipartial_r() with under_null = FALSE)
at the sample semipartial correlation and the sample full-model R2, with
the limits not truncated at -1 and 1. In the population 1.0.x simulated,
in which the controls explain none of the outcome, the two intervals
nearly coincide: with set.seed(113),
true_r_sp = supposed_r_sp = .30, p = 4
(J = 3 in 1.0.x), a width of .20, and
G = 2000, the mean width was .1988 and is .1986 at the same
total_N of 323. With true_r_sp = .80,
p = 2 (J = 1 in 1.0.x), and a width of .10 the
planner’s change above moves total_N from 202 to 357 and
the mean width from .1007 to .0754. This was a defect, and no argument
restores the earlier interval.
randomization_test() and
randomization_test_paired() count tied reassignments by the
rounding of the arithmetic, and the studentized statistic no longer
drops the reassignments with no within-group spread. A
reassignment whose statistic equals the observed one counts as at least
as extreme. Earlier versions compared with a tolerance relative to the
largest reference statistic, formed each variance as a sum of squares
less n times a squared mean, and set a statistic with a zero
standard error to 0. The comparison now allows the rounding bound of the
statistic compared, which is proportional to the size of the scores;
variances come from residuals about each group’s own mean; and a
reassignment in the reference set whose standard error is zero is
ordered as an infinite statistic with the sign of its difference (0 when
every score is equal). When the observed data themselves have a zero
standard error and a nonzero difference, statistic = "t" is
refused (see the entry on that refusal below). The p-values
that change:
randomization_test(group_1 = c(1, 1, 1, 1, 0), group_2 = c(0, 0, 0, 0, 1), statistic = "t")
gave 0.198413 and gives 52/252 = 0.206349. For
g_1 <- c(4.800000002, 4.8, 4.800000003, 4.800000001, 4.800000002)
and
g_2 <- c(8, 8.000000001, 8.000000003, 8.000000003, 8.000000003)
it gave 0.003968, half the smallest value ten scores can produce, and
gives 2/252 = 0.007937 (the statistic reported for that call was
-339,547,000 and is -3,938,927,000, the value of the Welch formula for
those scores).
randomization_test_paired(rep(0, 6), c(1, 1, 1, 1, 1, -1), statistic = "t")
gave 0.1875 and gives 14/64 = 0.21875 (0.09375 and 7/64 = 0.109375 with
alternative = "greater").statistic = "t".
randomization_test_paired(1:5, 1:5 + 2, statistic = "t")
gave a statistic of 0 and p = 1. The paired t has a
zero standard error there and no value, so the call is now refused
(class dmar_studentized_statistic_undefined) with a message
that says why and gives the p-value of the mean difference,
2/32 = 0.0625, which statistic = "mean" returns.x <- c(1000000.002, 1000000.005, 1000000.000, 1000000.002, 1000000.001, 1000000.001)
and
y <- c(1000000.003, 1000000.007, 1000000.002, 1000000.001, 999999.999, 1000000.001),
randomization_test_paired(x, y) gave 0.75 and gives 0.8125,
the value for the same scores in thousandths without the offset. The
independent test was too small in the same way:
randomization_test(group_1 = 1000000 + c(2, 2, 2, 0) / 1000, group_2 = 1000000 + c(3, 3, 0, 7, 4) / 1000)
gave 0.214286 and gives 28/126 = 0.222222.randomization_test(group_1 = c(3e9, 4, 7, 2), group_2 = c(3, 5, 6, 9, 1))
gave 0.444444 and gives 35/126 = 0.277778;
randomization_test_paired(c(3e9, 4, 7, 2, 5), c(3, 5, 6, 9, 1))
gave 1 and gives 0.6875.randomization_test_paired(rep(0, 6), c(1, 1, 1, 1, 1 + 1e-12, -1))
gave 0.21875 (and 1 with statistic = "t") and gives 12/64 =
0.1875 with either statistic.Scores with no ties and no extreme offsets give the p-values
they gave: 3,240 seeded calls on continuous data (exact and Monte Carlo,
both statistics, three alternatives, and procrustes_phi())
returned the same p-value before and after the change. The
limits of the shift interval move by up to about two millionths of a
standard error, because the earlier tolerance counted near ties at the
edge of the interval. Checked against rational arithmetic in 1,800
comparisons on unbalanced designs, the studentized p-value is
exact for binary, three-valued, Likert, three-decimal (with and without
an offset of a million), and constant scores; with one score of 3e9
among single digits, or scores within a millionth of one of two values,
statistics closer than the rounding bound are counted as tied and the
p-value was above the exact one in 96 of 600 comparisons and
below it in none. ?randomization_test says so under “Ties”.
The repaired statistic costs time, since each reassignment now keeps the
scores of both groups and forms its own residuals: with the default
10,000 reassignments and the shift interval, a call with 150 scores per
group and statistic = "t" takes about 2.8 s where it took
0.16 s, and one with 1,500 per group about 26 s where it took 1.7 s
(statistic = "mean": 0.30 s against 0.11 s, and 3.0 s
against 1.1 s). Most of the time is the inversion for the shift
interval: with shift_ci = FALSE the two studentized calls
take 0.27 s and 1.3 s.
randomization_test() with
statistic = "t" reports the whole line as the shift
interval where no shift can be rejected. With three scores per
group the smallest two-sided p-value is 2/20, so the 95 percent
interval is unbounded, as the help says. Under the studentized statistic
the earlier arithmetic lost the reassignments with no within-group
spread at large shifts and returned limits such as -77,486,400 and
78,526,226 for group_1 = c(80, 84, 79),
group_2 = c(72, 75, 68); the limits are now
-Inf and Inf, as they were for
statistic = "mean". The studentized statistic is now
computed on scores divided by a power of 2 near their largest absolute
value, so scores near 1e-200 or 1e200, whose squared residuals underflow
or overflow, give the statistic and p-value of the same scores
in ordinary units (c(3, 5, 9, 4) * 1e-200 against
c(1, 2, 2, 6, 3) * 1e-200 stopped with “missing value where
TRUE/FALSE needed”). The division is exact, and no other result
changes.
statistic = "t" is refused in
randomization_test() and
randomization_test_paired() when the observed standard
error is zero and the observed difference is not. The
studentized statistic is the mean difference divided by its standard
error. When both observed groups are constant at different values, or
every paired difference is the same nonzero value, that standard error
is 0 and the statistic has no value. DMAR 1.0.0 reported a paired
t of 0 with p = 1 for such differences
(randomization_test_paired(1:5, 1:5 + 2, statistic = "t"))
and, for two constant groups
(randomization_test(group_1 = rep(5, 4), group_2 = rep(3, 4), statistic = "t")),
stopped with an error raised while the standardized mean difference was
computed. Both calls now stop with an error of class
dmar_studentized_statistic_undefined (a
dmar_refusal), made before any arrangement is enumerated or
drawn, so a refused Monte Carlo call leaves the random number stream
where it was. The message says that the denominator is zero, that the
observed data lie at the boundary of the reference set (of the 32 sign
patterns, only the observed one and its mirror image have a zero
standard error; of the 70 reassignments of 4 and 4 scores, only the
observed one and the one that exchanges the groups), that this concerns
the statistic and not the evidence, and what
statistic = "mean" gives: the mean difference 2 with the
exact p-value 2/32 = 0.0625 for the first call and 2/70 =
0.02857 for the second. The condition carries that value as
p_value_mean, with reason
("constant_differences" or "constant_groups"),
alternative, and n or n_1 and
n_2. The value is the smallest the design allows in the
direction of the data: 1/2^n one-sided and 2/2^n
two-sided for n pairs; for a first group of n_1
among N scores, 1/choose(N, n_1)
one-sided, and two-sided 2/choose(N, n_1) with
equal group sizes and 1/choose(N, n_1) otherwise
(1/56 for 3 against 5); it is 1 for the alternative that points away
from the data. When the call would have been run by Monte Carlo sampling
the message adds that such a run estimates the value and cannot report
less than 1/(n_resamples + 1): for 30 against 30 constant
scores the exact value is 2/choose(60, 30), about 1.7e-17, and
statistic = "mean" reports 1/10001 = 0.00009999.
statistic = "mean" is unchanged in every case. What is
kept: a reassignment inside the reference set whose standard error is
zero is still ordered as Inf or -Inf, because
such reassignments occur in ordinary two-valued and coarsely measured
data and the limit puts them where the difference in means does
(group_1 = c(1, 1, 1, 0) against
group_2 = c(0, 0, 0, 1) with statistic = "t"
returns t = 1.4142 and p = 34/70 = 0.4857, with one
-Inf and one Inf among the 70 reference
values, the same p as the difference in means;
randomization_test_paired( rep(0, 5), c(2, 2, 2, -2, 2), statistic = "t")
returns t = 1.5 and p = 12/32 = 0.375); and when every
score is equal the statistic is 0 and p is 1 under either
statistic, as before. The comparison that decides whether the observed
data are constant uses the tolerances already in place, so
c(5, 5, 5, 5 + 1e-12) against rep(3, 4) is not
refused and returns t = 8.0e12.
?randomization_test and
?randomization_test_paired have a paragraph, “When the
studentized statistic is undefined”, and an example of the
refusal.
The shift interval of randomization_test()
rejects a shift whose p-value equals 1 -
conf_level. A shift is in the interval when its
p-value is above alpha, and alpha was computed as 1 -
conf_level, which for 0.90 is 0.09999999999999998 in binary
arithmetic, just below the p-value 1/10 that the test at the
0.10 level rejects. Shifts with exactly that p-value were
therefore kept, and the interval was too wide whenever alpha times the
number of reassignments is a whole number (at 0.90 with 10, 20, 70, or
210 reassignments; at 0.80 likewise). For
group_1 = c(-1.5, 3.6, 0) and
group_2 = c(4.2, 3.0, -1.7) the 90 percent limits were
-Inf and Inf and are -5.7 and 5.3, and the 80
percent limits were -5.7 and 5.3 and are -4.5 and 1.7. The comparison
now allows a few units of rounding in alpha. Intervals at 0.95 and 0.99
are unchanged; the defect was in 1.0.0 as well, and an interval was
never too narrow.
randomization_test() returns its table when
the standardized mean difference is undefined. With no
within-group variance (group_1 = c(1, 1, 1),
group_2 = c(2, 2, 2), or every score equal) the call
stopped inside ci_smd() with “lower < upper is not
fulfilled” or with “missing value where TRUE/FALSE needed”. It now
reports the test, the shift interval, and Cliff’s delta, with the rows
smd, smd_ci_lower, smd_ci_upper,
cles, cles_ci_lower, and
cles_ci_upper as NA and a warning of class
dmar_smd_undefined. For rep(1.1, 4) against
rep(0.2, 4) with statistic = "mean" the
statistic is 0.9, p is 2/70, and both shift limits are 0.9;
statistic = "t" is refused there, since both groups are
constant. A response that is not finite (Inf,
-Inf) is refused in both randomization tests (“must be
finite”), where it gave a base R error or a p-value of
NA.
plot_randomization_test() shades the
reassignments the p-value counted. The figure used an
absolute tolerance of 1e-12 of its own. For the scores of the first help
example multiplied by 1e-13 it shaded every reassignment (share 1.000
beside p = 0.0238), and for
1e5 + c(.1, .4, .7, .2) against
1e5 + c(.3, .5, .6, .9) it shaded 0.2571 beside p
= 0.3143. randomization_test() now stores the logical
vector it counted as the attribute reference_extreme, and
the plot shades those. A reassignment with an infinite studentized
statistic is not drawn, and the subtitle says how many are not, on a
second line. Infinite values occur in the reference distribution only:
the observed statistic is always finite, since
randomization_test() refuses the one case in which it is
not, and its line is always drawn. When every reassignment the
p-value counted is infinite, which can happen only in a Monte
Carlo sample (under enumeration the observed assignment is among those
counted), the subtitle gives p without calling it a shaded area
and says that none is shaded.
procrustes_phi() counts a permutation tied
with the observed arrangement. A permutation that exchanges two
items with the same loading reproduces the observed coefficient, but was
compared with it exactly and could fall one unit in the last place
below. With l1 <- c(0.6, 0.72, 0.65, 0.72, 0.81),
l2 <- c(0.61, 0.76, 0.64, 0.68, 0.85), and
set.seed(113),
procrustes_phi(l1, l2, n_resamples = 60000) gave 0.00772
and gives 0.01637; 2 of the 120 permutations are at least as extreme,
0.01667. The denominator is computed once and the comparison allows
4(n + 2) units of machine precision. A call with no tied
loadings gives the p-value it gave.
n_resamples must be a whole number.
randomization_test_paired(x, y, exact = FALSE, n_resamples = 4.5)
drew four sign patterns and divided by 5.5, and
procrustes_phi(l1, l2, n_resamples = 2.5) drew two
permutations and divided by 3.5; randomization_test()
truncated 4.5 to 4, and procrustes_phi(n_resamples = -3)
returned the coefficient with no message. Each now stops with an error
of class dmar_count_invalid unless the count is a single
whole number of at least 1 (the two randomization tests) or at least 0
(procrustes_phi(), where 0 skips the test). A whole count
gives what it gave. The count is checked whether or not the call draws:
randomization_test_paired(x, y, n_resamples = NULL) with
eight pairs, which enumerates, returned its table and now stops, as
randomization_test() already did. The check of
n_resamples applies under exact enumeration as well, where
the count is not used.
Help corrections for the randomization tests and
procrustes_phi().
?randomization_test_paired said that the paired t
is more robust to pair-to-pair variability than the mean difference.
Over the sign patterns the paired t is an increasing function
of the mean difference, so the two give the same p-value for
every data set and differ only in the statistic reported; the page now
says so, and says that exact enumeration is the default through 20 pairs
and available through 25. ?randomization_test said that the
mean difference and the studentized statistic agree in a balanced design
with equal variances “to within the discreteness of the reference
distribution”; they give the same p-value in every balanced
design, whatever the variances, and differ only when the groups are
unequal in size, and the second example’s comment no longer recommends
the studentized statistic for two groups of five.
?procrustes_phi called the permutation distribution a
sampling distribution and said the function reports the coefficient
“with its uncertainty”; the test takes the two sets of loadings as
given, and the function reports the coefficient and the permutation
p-value, with no standard error or interval.
mediate(): the bootstrap limits of the
indirect effect change in data where a resample makes the predictor or
the mediator a linear combination of the covariates.
lm() returned a finite product a b in such a
resample, by dropping a covariate column in place of the predictor, and
that product entered the interval although the effect is not estimable
there. These resamples are now dropped and counted (see
“mediate() codes its covariates once” above). With a binary
predictor (8 of 40 cases) and a binary covariate that differs from it in
two cases, B = 2000, seed = 113: 248 of the
2000 resamples are of this kind; the percentile limits were [-0.0401,
0.7202] over all 2000 and are [-0.0369, 0.6800] over the 1752 in which
the effect is defined, with the warning of class
dmar_bootstrap_replications_dropped. The estimate (0.1925)
and every other row are the same. The same holds in a small sample with
several covariates, where a resample can hold fewer distinct cases than
the model has coefficients: with 12 cases, three continuous covariates,
and B = 600, 12 resamples are dropped and the limits, which
were [-1.397, 2.023] with 127 warnings from lm() about a
perfect fit, are [-1.177, 1.881]. A call whose resamples all leave
a and b estimable returns the limits it returned
before, to rounding error (relative difference below 1e-10; up to 1e-9
for a date-time predictor, where the earlier fit lost digits to the size
of the origin). No argument restores the earlier limits.
mediate() refuses a logical
outcome. A logical y was read as 0 and 1 and the
linear model fit to it; the call now stops and says that the column must
be numeric. as.numeric() of the column gives the earlier
table. (A logical x or m stopped before, with
subscript out of bounds.)
mediate() returns 18 rows, not 17:
B_used follows B. Code that reads rows by
term is not affected; code that reads the table by position
past row 17 is.
mediate(ci_method = "boot_bca") stops where
it returned an interval from an incomplete jackknife. With one
case carrying all the variation in the predictor
(x = c(3, rep(0, 19)), B = 300,
seed = 113) it returned [-0.1173, 0.4605]; it now stops
with dmar_bca_undefined, naming case 1.
mediate() stops when a column is named more
than once. A covariate named twice returned the answer for the
covariate entered once; y among the covariates returned a
table from a model with the outcome among the predictors of the
mediator. Both now stop.
nnt_from_smd() reports the confidence set
for the number needed to treat correctly when the interval for the
standardized mean difference includes zero, where DMAR 1.0.0 returned
its two ends in reversed order as if they bounded an interval.
The reciprocal of an interval for the success rate difference (SRD) that
includes zero is two intervals that meet at infinity, the NNT of no
effect (Altman, 1998). DMAR 1.0.0 put the two finite ends of that set in
the rows nnt_lower and nnt_upper, the lower
above the upper and the estimate outside them, with nothing to mark the
case:
nnt_from_smd(smd = 0.3, smd_lower = -0.10, smd_upper = 0.70),
the fourth example of the help page (now
nnt_from_smd(smd = 0.3, smd_ci_lower = -0.10, smd_ci_upper = 0.70)),
returned nnt 5.953, nnt_lower 2.636, and
nnt_upper -17.739, and
nnt_from_smd(smd = 0.1, n_1 = 30, n_2 = 30) returned
17.739, 3.015, and -4.417. In that case the NNT limits, now the rows
nnt_ci_lower and nnt_ci_upper, are
NA, as the NNT limits of responder_analysis()
are, the set is in the new nnt_confidence_set attribute
(for the first call, the rows (-Inf, -17.739) and (2.636,
Inf)), the new nnt_set attribute is
"two_intervals", and the table prints the set beneath it:
“NNT <= -17.739 (harm) or NNT >= 2.636 (benefit), joined at
infinity (Altman, 1998)”. Read as an interval, the two rows of 1.0.0
held the true NNT in none of the samples when the population difference
is zero and in 34 percent of them at a population standardized mean
difference of 0.5 with 20 per group (95 percent intervals); the set of
two intervals holds it at the stated level. When the upper limit of the
standardized mean difference is exactly zero, the lower NNT limit is now
-Inf, where 1.0.0 gave Inf
(nnt_from_smd(smd = -0.3, smd_lower = -0.7, smd_upper = 0)
returned nnt_lower Inf and
nnt_upper -2.636, and
nnt_from_smd(smd = -0.3, smd_ci_lower = -0.7, smd_ci_upper = 0)
returns -Inf and -2.636). A result whose interval excludes
zero, or has a lower limit of exactly zero, is unchanged in its first
seven rows, with one exception at the edge of the arithmetic: the SRD is
computed without the subtraction that lost digits for a standardized
mean difference near zero, so results for |d| below about 1e-8 differ
from 1.0.0 in their later digits (at d = 1e-12 the NNT was 1.772373e12
and is 1.772454e12, which is sqrt(pi) / d), and a d so small that the
SRD rounded to zero keeps its sign (nnt_from_smd(-1e-200)
gave an NNT of Inf and gives -Inf). Every
result has the leading class dmar_nnt, which prints the
set, ahead of dmar_tbl and data.frame, and
dmar_metadata()$result is "dmar_nnt"; code
that tests inherits(x, "dmar_tbl") is unaffected.
tidy() and glance() return the rows without
the two attributes; the finite ends of a set of two intervals are 1 /
srd_ci_lower and 1 / srd_ci_upper.
srd_ci_lower and srd_ci_upper,
are appended after nnt_ci_upper: the confidence interval
for the SRD, which is an ordinary interval in every case and is the
interval Kraemer and Kupfer (2006) prefer. The seven rows of 1.0.0 keep
their positions; four are renamed (see Renamed Output Rows and
Columns).smd must be a single finite
number, conf_level a single number in (0, 1), the limits
and the sample sizes each given as a pair or not at all, and
smd_ci_lower not above smd_ci_upper; DMAR
1.0.0 returned the point estimate without an interval and without a
message when one limit or one sample size was given, accepted reversed
limits, and printed “Confidence level: 700%” for
conf_level = 7. An estimate outside its supplied limits is
accepted with a warning. Supplied limits are still used in place of
n_1 and n_2.conf_level is given in the call; DMAR 1.0.0
printed “Confidence level: 95%” for limits whose level it did not
know.The responder_analysis() help page no longer
cites Altman (1998) for withholding the NNT limits. The page
said that an interval on the NNT scale would mislead when the interval
for the difference includes zero. Altman (1998) argues for reporting it:
the confidence set is then two intervals that meet at infinity, with the
reciprocals of the difference limits as its two finite ends. The page
now says that, and that one lower and one upper limit cannot hold such a
set, which is why the two NNT limits are NA. No result
changes.
responder_analysis() carries the confidence
set for the number needed to treat, as nnt_from_smd() does,
and the help pages of both functions are written out in full.
When the interval for the difference of the two responder proportions
includes zero, the NNT limits are NA, as before, and the
set they could not hold is now in the result. With two groups the table
has the attributes nnt_set ("interval" or
"two_intervals", one value per threshold) and
nnt_confidence_set (columns lower and
upper, with threshold first under
sweep), on the signed scale 1 / difference, and
group_labels; for two intervals a line under the table
states the set. For the first example of the help page (30 of 60 control
and 35 of 60 treated participants at a threshold of 10; difference
0.083, interval -0.093 to 0.252; NNT 12 with NA limits),
earlier versions printed the table alone, and the table is now followed
by “NNT <= -10.806 (control responds more often) or NNT >= 3.966
(treatment responds more often), joined at infinity (Altman, 1998)”, the
reciprocals of the two limits of the difference. Every result has the
leading class dmar_responder, which prints that line, ahead
of dmar_tbl and data.frame, and
dmar_metadata()$result is "dmar_responder"
where it was "dmar_tbl"; code that tests
inherits(x, "dmar_tbl") is unaffected. No row of any table
changes (152 calls compared with the build before this change: largest
difference 0). ?responder_analysis now states what is
estimated and how it differs from the NNT of
nnt_from_smd(); the assumptions, among them that the
threshold is fixed before the data are seen and that sweep
describes dependence on the threshold and is not a search; the Wilson
and Newcombe formulas as computed; the construction of the confidence
set; what is returned when the proportions are equal, when no one or
everyone responds in one group or in all (in the last case the chi
square statistic is 0/0 and is reported as NaN), for a
group of one, for outcomes at the threshold, for missing values, and for
more than two groups; what each row is; the example read number by
number; and what dichotomizing costs, from MacCallum, Zhang, Preacher,
and Rucker (2002): a split at the mean multiplies a correlation by 0.798
and its square by 0.637, about the loss of 36 percent of the sample.
?nnt_from_smd now states the estimand in words; which
assumptions serve the point conversion and which the interval; the
conversion with a table of the SRD and the NNT for standardized mean
differences of 0.1 to 1.0, which agrees with Table 1 of Kraemer and
Kupfer (2006); case by case why the reciprocal of an interval that
includes zero is two intervals and why one pair of limits cannot hold
it; the edge cases; the threshold NNT beside this one in numbers (at a
standardized mean difference of 0.5: 3.62 here, and at least 5.07 at any
threshold under the same normal model); and two worked examples. No
result of nnt_from_smd() changes (154 calls compared:
largest difference 0).
vargha_delaney_A() stops on input it used to
order by a rule the user did not choose. It converted whatever
it was given with as.numeric(). An unordered factor was
read through its integer codes, which follow the alphabetical order of
its labels:
vargha_delaney_A(factor(c("10", "9", "8")), c(10, 9, 8))
returned A = 0 with limits [0, 0], and a five-category
agreement item supplied as an unordered factor through
outcome ~ group returned the value of its alphabetical
codes. Two time differences in different units were compared as bare
numbers:
vargha_delaney_A(as.difftime(c(1, 2, 3), units = "hours"), as.difftime(c(30, 90, 150), units = "mins"))
returned A = 0 with limits [0, 0], where the value is 0.667. A
Date beside a POSIXct was compared as days
against seconds. A matrix outcome in a formula,
cbind(s, t) ~ g, was strung out column after column under a
recycled row index, so that 3 rows per group were reported as 6
observations per group. Each of these calls now stops with an error of
class dmar_scores_not_ordered that says what to supply. Two
time differences in the same units, and a time difference outcome in the
formula interface, are on one scale and are ordered as before. Convert
time differences in different units with
as.numeric(x, units = "hours"), and supply an ordinal
response as an ordered factor or as numeric scores.
probability_of_superiority_paired() reports
both limits at the estimate, with a warning, when every pair favors the
same member. It returned NaN for both limits and
no warning:
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7))
gave an estimate of 1 with limits NaN, NaN,
and now gives 1 with limits 1, 1 and the warning of class
dmar_interval_degenerate, which says that the limits are
not a confidence interval with the stated coverage. With a missing value
it returned a table of NA; it now removes the
pair.
vargha_delaney_A() and
cliff_delta() agree with their earlier values to rounding,
not to the last bit. The rank-based computation gives the same
A bit for bit. On 3,000 samples with and without ties (2 to 60
observations per group), the largest differences from the earlier
results were 8e-17 in the standard error of A, 8e-16 in its
limits, 8e-15 in Cliff’s delta, 2e-16 in its variance, and 2e-14 in its
limits; the proportions of pairs won, lost, and tied were identical.
These are the largest differences on the samples compared, not bounds.
Cliff’s delta is now the difference of two exact proportions where it
was the mean of the signs of all pairs (on one sample of 3,000 per group
it was 0.170807999999525 and is 0.170808, the exact ratio). The
z statistic of vargha_delaney_A() moved by at most
1e-13 and its p-value by at most 4e-16.
meta_es(), meta_smd(),
meta_r(), and plot_forest() warn when the
effect sizes are identical under the Hartung-Knapp adjustment, and the
warning states what the conventional analysis gives. When every
effect size is the same, exactly or up to rounding error, the weighted
residuals are zero and so is the adjusted standard error:
meta_es(rep(.5, 4), rep(.04, 4)) returns a standard error
of 0, a t of Inf, and confidence and prediction
limits of 0.5 and 0.5. Those numbers are unchanged (they are what
test = "knha" in metafor returns). Earlier versions
returned them with no message. The functions now give one warning of
class dmar_meta_hk_zero_se. It says that the adjustment
multiplies the conventional standard error by the square root of
q, the weighted mean square of the effect sizes about the
pooled estimate, and that q is 0 when the effect sizes do not
differ; it shows the limits (0.5, 0.5); it says that this is not
evidence that the effect is known exactly, since each effect size has a
sampling variance that agreement among four estimates does not remove;
and it states the conventional standard error and the normal interval at
the level of the call, 0.1 and 0.304 to 0.696 for that call, which
hartung_knapp = FALSE reports. The condition carries
n_studies, estimate,
se_conventional, lower_conventional, and
upper_conventional (4, 0.5, 0.1, 0.3040036, 0.6959964).
When the common effect size is 0 the message says that the t
statistic is undefined (0 divided by 0), and when the standard error is
only numerically zero
(meta_es(c(0.1 + 0.2, 0.3, 0.3), rep(.04, 3)), standard
error 2e-17) it says that the statistic is the estimate divided by a
rounding error. For meta_r() the limits and the normal
interval in the message and in the condition are correlations, as the
reported rows are, and the conventional standard error is on the Fisher
Z scale and is labeled so
(meta_r(r = c(.3, .3, .3), n = c(40, 60, 90)): 0.07433 on
the Fisher Z scale, normal interval 0.1624 to 0.4262). Effect
sizes that differ, however little (0.5, 0.5, and 0.5 + 1e-9), do not
raise the warning, and no warning is given for a q between 0
and 1, which is the method working as defined.
Meta-analysis results record q and the
conventional standard error. Whenever the Hartung-Knapp
adjustment is applied, the tables of meta_es(),
meta_smd(), and meta_r() carry two new
attributes: hartung_knapp_q, the weighted residual mean
square on k - 1 degrees of freedom, and
se_conventional, the standard error without the adjustment
(both on the Fisher Z scale for meta_r()). The
adjusted standard error is the square root of the first times the
second. For effect sizes of .30, .32, .29, .31, and .33 with variances
of .04, .05, .03, .06, and .04 they are 0.006860532 and 0.0912870929,
and the reported standard error is 0.007561158. The tables keep their 20
rows, and no reported number changes (1,104 fits compared with the build
before this change: largest difference 0). ?meta_es has two
new paragraphs. “When q is small” says that the adjusted interval is
narrower than the normal interval exactly when q is below the
squared ratio of the normal quantile to the t quantile on
k - 1 degrees of freedom (0.02, 0.21, 0.50, and 0.75 at
k = 2, 3, 5, and 10 for a 95 percent interval), that a small
q belongs to the reference distribution of the method, and that
both intervals should be reported when the adjusted one is much the
narrower. “Identical effect sizes” describes the case the warning is
for.
The help of meta_es() no longer says that
the Hartung-Knapp adjustment keeps the confidence interval near its
nominal coverage. It now gives the adjusted standard error, the
conventional one times the square root of the weighted residual mean
square q (Röver, Knapp, & Friede, 2015), and says that the
function applies the unmodified method: q is not replaced by
max(1, q), so when q < 1 the interval is narrower
than the normal interval. For effect sizes of .30, .32, .29, .31, and
.33 with variances of .04, .05, .03, .06, and .04, q is 0.0069
and the interval is 0.287 to 0.329, against 0.129 to 0.487 with
hartung_knapp = FALSE. The help says where the coverage of
the adjusted interval is at or near the nominal level (homogeneous or
equally precise studies) and where it falls below it (studies that
differ greatly in precision, with heterogeneous effects, where the
normal interval covers less still), and that the prediction interval
uses the same standard error. No result changes.
The REML estimate of the between-study variance is now
carried to the maximum of the restricted likelihood, in whatever units
the effect sizes are. meta_es(),
meta_smd(), meta_r(), and
plot_forest() with method = "reml" (the
default) searched over tau2 itself with the default
tolerance of optimize(), the fourth root of the machine
epsilon. That search stops within about 0.00003 of the maximum in the
units of tau2, whatever the size of the effects. The search
is now made on tau2 divided by the scale of the data (the
largest sampling variance plus the variance of the effect sizes),
stopping at a distance relative to the estimate, so the estimate for
effect sizes multiplied by a constant c is c squared times the estimate
for the original ones. The change is small for effect sizes of the order
of 1. On 600 random sets of 2 to 15 studies with sampling variances
between .005 and 3, the largest distance of tau2 from the
solution of metafor run to convergence
(control = list(threshold = 1e-12)) fell from 0.00003 to
0.0000004 (598 sets drawn afresh; the remaining distance is the flatness
of the restricted likelihood at its maximum), and no estimate moved
between zero and positive; on those sets tau2 changed by at
most 0.000028, the pooled estimate by at most 0.00006, and its standard
error by at most 0.00003 (0.04% of its size). These are the largest
changes on the sets compared, not bounds. For the 12 simulated studies
of the package’s tests, tau2 was 0.0279437 and is
0.0279387, the pooled estimate was 0.416452 and is 0.416453, and its
Hartung-Knapp standard error was 0.080996 and is 0.080995. The change is
large for effect sizes in small units, where 0.00003 is of the size of
tau2: the earlier estimate could be zero where the maximum
is positive, or several times too large. For ten simulated risk
differences near .02 with 4,000 to 40,000 per group (sampling variances
near 1e-5; set.seed(2718) in the test file),
tau2 was 0 and is 0.0000090755, which is metafor’s
converged value; the pooled estimate was 0.021151 and is 0.021255, I2
was 0 and is 52.2 percent, and the prediction interval was 0.0181 to
0.0242 and is 0.0137 to 0.0289. On 300 random sets with sampling
variances between 1e-6 and 1e-4, the earlier estimate disagreed with
metafor on zero against positive in 76 and was off by more than 1
percent in 184; the estimate now disagrees in none and is within 0.002
percent in all. An analysis of risk differences, proportions, or other
effect sizes with sampling variances below about .001 should be rerun.
The estimate is now accurate relative to itself, including when one
study’s sampling variance is far above the others’ (1.0.0 stopped within
about 0.00003 of the maximum in the units of tau2, about
0.5 percent of a tau2 near .01): for the twelve studies of
the package’s tests with one sampling variance of 1e6
(.outlier_yivi() in the test file), tau2 was
0.012048 and is 0.012058, the root of the score equation; the pooled
estimate, -0.0510, and its Hartung-Knapp standard error, 0.0328, are
unchanged to the digits shown. The results of the other estimators are
unchanged by this entry (the Paule-Mandel estimate and the Q-profile
interval for tau2 are the subject of an entry under Bug
Fixes). metafor at its default threshold of 0.00001 gives 0.0279436 for
the 12 studies above, so a comparison with its default output can differ
in the fifth significant digit of tau2 where it agreed
before.
meta_es(), meta_smd(), and
plot_forest() refuse an infinite effect size or sampling
variance with a message that says so, and meta_r() and
meta_smd() an infinite sample size. Under
method = "reml",
meta_es(c(.1, Inf, .3), rep(.1, 3)) stopped with “invalid
‘xmin’ value” from the search and an infinite variance with “invalid
‘xmax’ value”; under "pm" and "dl" both
stopped with “missing value where TRUE/FALSE needed”. They now stop with
“Supply finite effect sizes” and “Supply a finite sampling variance for
each effect size”. Under method = "fe" these calls returned
a result and now stop as well:
meta_es(c(.1, .2, .3), c(.1, Inf, .1), method = "fe") gave
an estimate of 0.2 (the study with an infinite variance had a weight of
zero), and an infinite effect size gave an infinite estimate. To pool
without a study, leave it out. An infinite n in
meta_r() stopped with “missing value where TRUE/FALSE
needed” or, under method = "fe", returned an estimate of
NaN with a standard error of 0 and no message; an infinite
n_1 or n_2 in meta_smd() stopped
with a message about the effect sizes. Each now stops with its “must
give an integer sample size” message.
moments_nc_t() is right at large degrees of
freedom and at a large noncentrality parameter, where DMAR 1.0.0 lost
digits, changed the sign of the excess kurtosis, or returned a negative
variance. DMAR 1.0.0 formed the variance, skewness, and excess
kurtosis as differences of the raw moments, which grow with the
noncentrality parameter while the differences do not, so the digits left
fell as either argument grew: the excess kurtosis was off by 1e-7 of its
value at 1000 degrees of freedom and a noncentrality parameter of 5 and
by 4e-4 at 2000 and 30, and
moments_nc_t(df = 1e4, ncp = 100) returned -0.0022675 where
the value is 0.0013340. moments_nc_t(df = 1e16, ncp = 2.5)
returned a mean of 1.97e14 and a variance of -3.9e28 (now 2.5 and 1),
moments_nc_t(df = 20, ncp = 1e80) an excess kurtosis of
NaN, and moments_nc_t(df = Inf, ncp = 2)
NaN for the mean, variance, skewness, and excess kurtosis
and NA for the standard deviation. The mean now comes from
the logarithm of the chi mean factor that sd_unbiased()
uses, and the other three from the central moments of the square root of
the degrees of freedom over a chi square variate, so that no raw moments
are subtracted; from 100 degrees of freedom on, the third central moment
and fourth cumulant of that square root are their expansions in the
reciprocal of the degrees of freedom. Against values computed to several
hundred digits, for degrees of freedom from just above each boundary (1,
2, 3, and 4) to 1e300 and a noncentrality parameter up to 1e6 in
magnitude, the largest relative errors found are 3e-16 for the mean,
9e-16 for the variance, 4e-14 for the skewness, and 1.3e-12 for the
excess kurtosis; the same limits hold at a noncentrality parameter of
1e10, 1e100, and 1e300. df = Inf now returns the normal
distribution with mean ncp and variance 1, and an infinite
ncp is refused. For degrees of freedom up to 300 and a
noncentrality parameter up to 5 in magnitude the values differ from
those of 1.0.0 by at most 1.5e-8 of their size (df = 20,
ncp = 2.5 agrees to ten digits).
cv(), ci_cv(),
sd_unbiased(), ci_proportion(), and
power_equivalence_c() refuse inputs for which DMAR 1.0.0
returned a number that meant nothing or an error from inside base
R. Each refusal names the argument. In cv(): a
negative sd (cv(mean = 2, sd = -1) returned
-0.5), a mean of 0 (cv(mean = 0, sd = 1) returned
Inf), cv given with one of mean
and sd (cv(cv = .2, mean = 999) returned 0.2
and ignored the mean), a cv that is not one finite number
(cv(cv = "a") returned a table holding “a”), an
N of 1 or less, or an infinite N, with
unbiased = TRUE
(cv(cv = .2, N = 0, unbiased = TRUE) returned
Inf, and N = Inf returned the uncorrected
0.2), and an unbiased that is not TRUE or
FALSE (unbiased = "yes" returned the
uncorrected value). cv() documented single values and now
requires them: a vector cv, mean,
sd, or N, and unbiased = 1, were
accepted and are refused
(cv(cv = .2, N = c(10, 20), unbiased = TRUE) returned two
rows). A negative mean still gives a negative coefficient. In
ci_cv(): a coefficient of variation that is not positive,
whether given as cv, through sd, or by the
data (ci_cv(cv = -.2, N = 20) and
ci_cv(mean = 10, sd = -2, N = 20) returned the limits
-0.298 and Inf, and
ci_cv(data = c(-1, -2, -3, -4)) returned -6.859 and
Inf), data with a missing or infinite score or fewer than
two scores, data that are not numeric (a logical vector was read as 0
and 1), and an N that is not one finite number above 1;
ci_cv(cv = 0, N = 20),
ci_cv(cv = .2, N = Inf), and
ci_cv(data = c(1, 2, NA, 4)) stopped with “missing value
where TRUE/FALSE needed”. A fractional N above 1 is still
accepted. In sd_unbiased(): a negative or infinite
s (sd_unbiased(s = -1, N = 20) returned
-1.013, and s = Inf returned Inf),
s without N (which stopped with “arguments
imply differing number of rows”), a single score as X
(sd_unbiased(X = 5) returned NA), scores
X that are not numeric (a logical vector was read as 0 and
1), an N of 1 or less, a single s or
N that is missing, and vectors s and
N of different lengths other than 1. One inadmissible
element refuses the whole call:
sd_unbiased(s = 2, N = c(10, 1)) returned 2.056 and
NaN and now stops. A missing element of a vector still
gives a missing row (sd_unbiased(s = c(1, NA), N = 10)
returns 1.028 and NA). A fractional N and
vectors of equal length give what they gave in 1.0.0; an infinite
N gives s (1.0.0 returned NaN).
In ci_proportion(): an infinite count
(ci_proportion(1, Inf) returned the limits 0 and 0). In
power_equivalence_c(): a missing or infinite
true_contrast, n_per_group,
true_sd, bound, or df_error, and a missing
alpha_level (true_psi = NA_real_, an infinite
per-group size, and delta_upper = Inf, in the 1.0.x names,
each returned a power of 0, where n_per_group = 1e12 and
contrast_upper = 1e6 return 1; a missing per-group size,
true_sd, df_error, or alpha_level
given as NA_real_ stopped with “missing value where
TRUE/FALSE needed”). A fractional n_per_group or
df_error gives what it gave.
ss_aipe_cv_sensitivity() leaves a
replication whose sample mean is not positive out of its width and
coverage summaries, where DMAR 1.0.0 entered it with the limits
ci_cv() gave for a negative coefficient, an infinite upper
limit among them. Such a sample has no confidence interval for
the coefficient of variation, which ci_cv() now refuses.
The replication is left out of every statistic row,
mean_cv, median_cv, and sd_cv
included (see the next entry): all ten are taken over the replications
that have an interval; one warning of class
dmar_replications_without_interval states how many were
left out, and the count is attr(x, "simulation")$failed.
The case needs a large coefficient of variation and a small sample: with
true_cv = 1 and specified_N = 10 the sample
mean is not positive in about 8 of 10000 replications. With
set.seed(7) and G = 3000 there, 3 replications
are left out: pct_ci_miss_low was 0.0287 and is 0.0277,
total_type_I_error was 0.0553 and is 0.0544, and
median_ci_width was 3.743 and is 3.735.
mean_ci_width was Inf there in 1.0.0, since
ci_cv() has no finite upper limit for some samples with a
positive mean, and is now taken over the finite widths (see the next
entry). A study in which every sample mean is positive, which is every
study at the coefficients of variation the help page and the vignette
use, differs from DMAR 1.0.0 only where the limits of
ci_cv() changed with the exact noncentral t (see
Exact Noncentral t and Studentized Range
Distributions).
ss_aipe_cv_sensitivity() summarizes the same
replications in every row. A replication whose sample mean is
not positive has no interval, and its sample coefficient is negative.
All ten statistic rows, mean_cv, median_cv,
and sd_cv among them, describe the replications with a
positive mean (DMAR 1.0.0 entered such a replication in every row; see
the entry above). An interval with an infinite upper limit made
mean_ci_width infinite and sd_ci_width
undefined; the two are now over the finite widths,
median_ci_width stays over every width, an infinite width
counts as not meeting the target (as it did), and the number of infinite
widths is attr(, "simulation")$n_infinite_width. DMAR 1.0.0
repeated the warning ci_cv() gave for such an interval in
every replication that had one. ci_cv() no longer gives
that warning (see the entry on ci_cv() and an infinite
upper limit), and the sensitivity function gives one warning of its own,
of class dmar_replications_with_infinite_limit, that states
how many replications had an infinite upper limit and how they enter the
rows (the count is also the field n_replications of the
condition). With set.seed(88), true_cv = 3,
specified_N = 5, width = .5, and
G = 500 (114 replications without an interval, 346 infinite
widths): mean_cv -3.499 in DMAR 1.0.0 and 4.870 now,
median_cv 1.400 and 1.884, sd_cv 118.98 and
22.16, mean_ci_width Inf and 14.85,
sd_ci_width NaN and 37.25. With true_cv = 1,
specified_N = 10 (no replication without an interval, 96
infinite widths): mean_ci_width Inf in DMAR 1.0.0 and 15.55
now, sd_ci_width NaN and 188.3. A simulation in which every
mean is positive and every width finite differs from DMAR 1.0.0 only
through the exact noncentral t limits of ci_cv().
The table also carries the simulation attribute
(replications attempted, with and without an interval, and exact
binomial limits for the share that met the width) and prints the counts
beneath it. In the first example the coverage rows differ from those of
DMAR 1.0.0 as well, for the reason of the entry above.
The number of replications G must be a whole
number, in every function that has the argument. The 31
sensitivity functions with a G argument,
ss_aipe_indirect_effect(),
ss_aipe_composite_sem(), and
ss_power_composite_sem() check it with the one check of a
count (an error of class dmar_count_invalid), before
anything is simulated. The smallest G is what it was: 10
for the three planners, 2 for ss_seq_c_sensitivity() and
ss_aipe_crossed_sensitivity(), and 1 elsewhere. In DMAR
1.0.0 most of them did not check G:
ss_aipe_smd_sensitivity(G = 0) returned a full table from
one replication and G = 2.7 ran two,
ss_aipe_r_sensitivity(G = 0) returned a table of
NA, ss_aipe_cv_sensitivity(G = 0) stopped with
“subscript out of bounds”, and G = -3, Inf,
NA, or a vector stopped with “invalid ‘times’ argument”;
ss_seq_c_sensitivity() accepted G = 2.5. The
messages of the functions that did check G now read “‘G’
must be a single whole number of at least 1.” (or 2, or 10), in place of
“‘G’ must be a positive integer.”, “‘G’ must be a single positive whole
number.”, “‘G’ must be a single number of at least 2.”, and “‘G’ must be
a positive whole number of attempted studies.”.
The six ss_aipe_crd_es_*() planners check
G (nrep in 1.0.x) the same way at entry: a
fractional number, 0, a negative number, NA, a character
string, an infinite value, or more than one value is an error of class
dmar_count_invalid, before any simulation starts (earlier
such a value reached the simulation unchecked). G = NULL
remains the default of the two planners that fix a size and a budget,
where it skips the simulation. Results for a valid G are
unchanged.
The Hofmann (1977) entry of
simple_structure() gives the issue and pages of the
article. The help page gave The Journal of General
Psychology, 96, 58–66; the DOI record of the article
(10.1080/00221309.1977.9920803) has 96(1), 103–110. The entry now gives
that issue and those pages with the DOI, and the Hofmann (1978) entry on
the same page gains its DOI, 10.1207/s15327906mbr1302_9, which
fa_exploratory() already cites. The title, the journal, and
the 1978 volume, issue, and pages, 13(2), 247–250, already matched the
records and are unchanged.
The Owen (1968) entry of moments_nc_t()
gives the DOI of the article. The entry carried the DOI of a
letter to the editor in the preceding issue
(10.1080/00401706.1968.10490590); it is now
10.1080/00401706.1968.10490594.
The Phillips (1990) entry gives the first page of the
article. The help pages of power_equivalence_c(),
power_equivalence_md(),
power_density_equivalence_md(), and
power_equivalence_md_plot() gave the pages as 139–144; they
are 137–144.
The correction factor of Hedges (1981) is right at a very
large sample size, in smd(), smd_c(),
meta_smd(), expected_smd(), and
var_smd(). The factor J(df) = Gamma(df/2) /
{sqrt(df/2) Gamma((df - 1)/2)} was computed as a difference of two
lgamma() values that nearly cancel at a large
df, the form that design_consequences(),
sd_unbiased(), and var_ete() gave up earlier
in this release (the entry on E(S)/sigma). It is now formed from the
logarithm of E(S)/sigma on df - 1 degrees of freedom, log
J(df) = L(df - 1) + log1p(-1/df)/2, two negative terms with nothing to
cancel, through one internal helper that the five functions share.
Against a 60-digit evaluation J is right to a relative 3.4e-16 from 2
df to 1e14; the earlier form was off by a relative 5e-11 at
1e5 df and 8e-9 at 1e7, and at 1e8 it returned a J above 1.
At ordinary sample sizes nothing a user sees moves: over 272 planning
sets with at most 10,000 per group the largest change in any reported
value is 2.6e-10 (in var_smd()), except in
meta_smd() under REML, whose iteration stops at a tolerance
that lets the estimate move by about 1e-9 and I^2 by about 4e-6 of a
percentage point (no printed digit moves), and with 20 per group
smd(unbiased = TRUE) and expected_smd() agree
with 1.0.x to all printed digits. Two results that subtract nearly equal
numbers were wrong much sooner than J itself, and are now taken from the
logarithm with expm1(): the bias row of
expected_smd(), true_smd (1/J - 1), and the
exact variance in var_smd(), which is now computed as
df/(df - 2) + lambda^2 {2/(df - 2) - (1/J^2 - 1)}.
expected_smd(true_smd = 0.5, n_1 = 1e8, n_2 = 1e8) reported
a bias of -1.19e-07 (the wrong sign) and now reports 1.875e-09;
var_smd(true_smd = 3, n_1 = 1e8, n_2 = 2e8) reported an
exact variance of -8.5e-06 and now reports 3.0e-08;
var_smd(true_smd = 0.5, n_1 = 1e8) reported 1.42e-07 and
now reports 2.06e-08; with 100,000 per group the bias of
expected_smd(true_smd = 0.5) moves from 1.875105e-06 to
1.875029e-06. Against the 60-digit values the bias and the exact
variance are now right to a relative 7e-16 over 344 planning sets with
up to 2e8 observations; the earlier relative errors there reached 190
and 285. No argument is added.
moments_nc_F() is right at large degrees of
freedom. The variance, skewness, and excess kurtosis were
formed as differences of raw moments, which agree in their leading
digits when the degrees of freedom are large, so the digits that
remained were rounding error.
moments_nc_F(df_1 = 1e6, df_2 = 1e6) returned an excess
kurtosis of -5.285e-05 where the value is 6.60006e-05, and a skewness of
0.0060000263 where the value is 0.0060000230. The function now writes
the statistic as (df_2/df_1) X Y with X the noncentral chi square and Y
the reciprocal of the central one, and forms the central moments of
F/E(F) from those of X/E(X) - 1 and Y/E(Y) - 1, which are independent
and have mean 0: every term of each sum is nonnegative, so nothing
cancels (the expressions are on the help page; they were derived for
DMAR and checked against exact rational arithmetic). In 14 cases with
degrees of freedom up to 1e12 and noncentrality up to 1e6 the four
moments are right to a relative 5e-16, where the earlier form was off by
a relative 1.8 at (1e6, 1e6, 0) and 0.36 at (3, 1e6, 1e6). At ordinary
degrees of freedom nothing a user sees moves: over 918 combinations with
both degrees of freedom at most 200 and noncentrality at most 50 the
largest relative change is 8.5e-12, and the moments that do not exist
are NA in the same places. The mean is unchanged.
rbind() of DMAR tables no longer carries the
first table’s confidence level over rows computed at another. A
dmar_tbl keeps its confidence level, and the other facts
its printed lines are written from, as attributes, and
rbind() kept the attributes and the class of the first
table:
rbind(ci_smd(..., conf_level = 0.90), ci_smd(..., conf_level = 0.95))
was a table of class dmar_ci_smd that printed “Confidence
level: 90%” beneath six rows, three of them 95 percent limits, and
dmar_metadata() of it gave 0.90. rbind() now
has a method for these tables. When every table stacked is a DMAR table
and all have the same classes and attributes, the stack is a DMAR table
with those attributes, as before. In every other case (different
confidence levels, different functions, or a DMAR table stacked with a
plain data frame, in either order) the result is an ordinary
data.frame with the same rows and columns and none of the
DMAR attributes, and no warning: the numbers are all there, and no
printed line states a level that does not hold for every row. To compare
results across levels, stack tidy() rows, which carry the
level in a column: rbind(tidy(x_90), tidy(x_95)).
dplyr::bind_rows() does not call the method. Code that
relied on the class of a mixed stack
(inherits(rbind(a, b), "dmar_tbl") with tables at two
levels) now gets FALSE. ?dmar_tbl has a
section on stacking.
ss_aipe_R2() with assurance
returns a sample size for a small population R2, where it stopped with
“invalid ‘xmin’ value”. The procedure inverts the expected
value of R2 by matching a value rounded to two decimals against a grid
of expected values. When the one-sided lower limit it uses is 0, which
happens for a small population R2, no grid value has an expected R2 that
low, the match was missing, and optimize() was given a
missing end point. A missing match is now replaced by the end of the
grid when the target lies beyond it and by the grid value with the
nearest expectation otherwise.
ss_aipe_R2(true_R2 = .03, p = 1, width = .2, assurance = .9)
returns 113; of 30 conditions drawn at random with a
true_R2 of .03 to .90, the 8 that stopped now return (107
to 173) and the 22 that returned give the same sample sizes, as do 40
further conditions that returned before and 48 without
assurance (largest difference 0).
The default p-value of
vargha_delaney_A() is larger. The test statistic
is referred to Student’s t with Satterthwaite degrees of
freedom where DMAR 1.0.x referred it to the standard normal distribution
(see “The Order-Based Effect Sizes Share One Contract”). The estimate,
its standard error, the statistic, and the confidence interval are
unchanged, and test = "z" gives the 1.0.x p-value:
on 240 data sets by three levels and two interval methods the eight
numbers of a test = "z" row equal the earlier ones exactly.
For the first example of the help (set.seed(113), 40 per
group) the p-value is 0.0082 on 77.96 degrees of freedom where
it was 0.0067; for
vargha_delaney_A(iq_8 ~ treatment, data = pygmalion) it is
0.0892 on 92.48 degrees of freedom where it was 0.0858; and for
x = c(3.5, 5, 6, 7, 8), y = c(1, 2, 3, 4, 4.2)
it is 0.0033 on 6.63 degrees of freedom where it was 0.0000076 (the
exact permutation p-value is 8/252 = 0.032).
results_sentence() reports the p-value of the test
that was run. The table has two more columns, df and
test, after the nine it had.
Every sensitivity function summarizes its replications
over one denominator and records the counts. The proportion of
intervals that met the width target (pct_ci_less_width),
the proportions that missed the true value, and the summaries of the
widths are taken over the replications that produced a confidence
interval, in every ss_aipe_*_sensitivity() function; a
replication without an interval is in none of those rows. In
ss_aipe_reliability_sensitivity(), where a simulated study
can end without an interval, DMAR 1.0.0 took the width rows over the
studies that returned one, as now, and the three rows for intervals that
missed the true value (pct_ci_miss_low,
pct_ci_miss_high, total_type_I_error) over all
G, so that a study without an interval counted as one whose
interval held the value: with two of four simulated studies returning an
interval and one of the two missing the value, the row is 1/2 where DMAR
1.0.0 gave 1/4, and with no study returning an interval the three rows
are NA where it gave 0.
ss_aipe_cv_sensitivity() and
ss_aipe_cliff_delta_sensitivity() have entries of their
own. The rule changes nothing in a run in which every replication has an
interval. All 31 sensitivity functions, the two
ss_power_*_sensitivity() functions and
ss_seq_c_sensitivity() included, now attach the attribute
simulation, a list with attempted
(G), returned (replications with an interval,
or a test), failed (those without),
denominator (which is returned),
width_target_probability (the
pct_ci_less_width row), and width_target_mc_ci
(exact binomial limits for it over returned, which describe
simulation error); the two width entries are NA for a
function with no width target.
ss_aipe_sem_path_sensitivity() adds n_redrawn,
the data sets whose fit did not converge and were drawn again (such a
data set is replaced, so it is not among the replications without an
interval; the help says how the count is read). The record is the one
place the counts are kept: the rows suc_rep of
ss_aipe_rmsea_sensitivity() and
ss_aipe_sem_path_sensitivity() and
num_probs_with_cis of
ss_aipe_R2_sensitivity(), which held the same counts in
DMAR 1.0.x, are removed (returned and failed
hold them). The line “Replications” beneath each printed table gives
G, and the three counts when a replication was without an
interval. dmar_metadata() returns the record as
simulation.
ss_aipe_cliff_delta_sensitivity() no longer
counts a completely separated replication as an interval of width
0. When the two simulated groups do not overlap, the estimated
variance of Cliff’s delta is 0 and cliff_delta() reports
both limits at the estimate, which are not a confidence interval. DMAR
1.0.x took them as an interval of width 0, which met every width target,
and scored its coverage from the point estimate. Such a replication now
has no interval: its estimate still enters
mean_cliff_delta, median_cliff_delta, and
sd_cliff_delta, and the width, width-target
(pct_ci_less_width), and coverage rows describe the
replications with an interval. One warning of class
dmar_replications_without_interval gives the count (fields
n_without_interval and G), the per-replication
warnings are not passed on, the table carries the
simulation attribute with the counts and exact binomial
limits for the width proportion, and the file named in
filename records such a replication with its estimate and
NA elsewhere. With set.seed(113),
ss_aipe_cliff_delta_sensitivity(true_delta = 0.9, specified_N = 12, width = 0.4, G = 400)
has 162 separated replications of 400. pct_ci_less_width is
0 where it was 0.405; mean_ci_width 0.966 where it was
0.575; median_ci_width 0.909 where it was 0.867;
sd_ci_width 0.112 where it was 0.483;
pct_ci_miss_low 0 where it was 0.405;
pct_ci_miss_high 0.025 where it was 0.015; and
total_type_I_error 0.025 where it was 0.420. The three rows
of the estimate are unchanged (0.904, 0.944, 0.119). A run in which no
replication is separated returns what it returned: 36 conditions under a
common seed are identical. No argument restores the earlier
result.
results_sentence() does not write limits
that are not a confidence interval as one. For a table of
vargha_delaney_A(), cliff_delta(), or
probability_of_superiority_paired() whose estimated
variance is 0, the sentence gives the estimate and the reason.
results_sentence(vargha_delaney_A(c(4, 5, 6), c(1, 2, 3)))
is “A = 1.00 (no confidence interval: the groups are completely
separated, so the estimated variance is 0)” where it was “A = 1.00, 95%
CI [1.00, 1.00]”; for cliff_delta() on those groups
“cliff_delta = 1.00 (no confidence interval: the groups are completely
separated, so the estimated variance is 0)” where it was “cliff_delta =
1.00, 95% CI [1.00, 1.00]”; for
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7))
“probability_of_superiority = 1.00 (no confidence interval: every pair
favors the same member, so the estimated variance is 0)” where DMAR
1.0.0 wrote “probability_of_superiority = 1.00, 95% CI [NaN, NaN]”; and
for vargha_delaney_A(c(2, 2, 2), c(2, 2, 2)) “A = 0.50 (no
confidence interval: every score is equal, so the estimated variance is
0)” where it was “A = 0.50, 95% CI [0.50, 0.50]”. A p-value is
appended only when the table has one (the permutation test). The
sentence of a tidy() or glance() row of such a
table reads “A = 1.00 (no confidence interval: the estimated variance is
0)”. The sentence of every other table is unchanged.
A bootstrap or Monte Carlo interval asked for with fewer
replications than the rule requires is refused (see One Rule
for Bootstrap Replications): fewer than 200, 400, or 2000 at the 90, 95,
or 99 percent level for a percentile or Monte Carlo interval, fewer than
1000, 2000, or 10000 for a BCa interval, and fewer than 100 for a
bootstrap standard error interval or a Bollen-Stine p-value.
Calls that ran in 1.0.x and now stop include
reliability_alpha(ci_method = "percentile", B = 40) (0.6083
[0.5231, 0.6846] before; with B = 400 the function returns
[0.5203, 0.6709], as it did),
reliability_alpha(ci_method = "bca", B = 500) ([0.5204,
0.6692] before; with B = 2000, [0.5348, 0.6768], as it
did),
reliability_alpha(ci_method = "bootstrap_se", B = 50),
htmt(B = 100) to htmt(B = 199),
parallel_analysis(B = 20),
sem_exact_fit_bootstrap(B = 20) (p = 0.05 from 20
samples before),
R2_mixed_effects(ci_method = "boot", B = 20), and
mlmr(ci_method = "boot", B = 20). The error has class
dmar_count_invalid and names the number;
options(dmar.bootstrap_exploratory = TRUE) runs the call
and marks the result.
An interval formed from fewer than half of the
replications requested is no longer reported. The estimate is
returned with NA limits and a warning of class
dmar_bootstrap_insufficient. In mediate(), the
kappa functions, krippendorff_alpha(), htmt(),
the ratio functions, average_variance_extracted(), and
fraction_missing_information() an interval was formed from
any 100 replications that remained (two in the last), and in the
reliability family through 1.0.1 from any two. An interval from at least
half of those requested is reported as before, since the number required
is held to the request and not to the replications that remain, and the
warning now has a class, states the counts, and says when those that
remain are fewer than the number required. With two raters and eight
subjects, six of whom both raters place in one category,
cohen_kappa(ci_method = "percentile", B = 400, seed = 113)
returns 0.6 [0.0, 1.0] from the 361 resamples in which kappa is defined,
as 1.0.x did (and ci_method = "bca", B = 2000 returns [0.0,
1.0] from 1787);
krippendorff_alpha(boot = TRUE, B = 400, seed = 113) on
five units of which four are alike returns [1, 1] from 254 resamples;
htmt(B = 200, seed = 113) with an item that is nonzero in
one of 12 cases returns the bound 1.1723 from 130 resamples; and
ci_ratio_of_means(ci_method = "percentile", B = 400, seed = 113)
with a denominator group that is nonzero in one of 12 cases returns 3.0
[0.3333, 6.0] from 269 replications. Each of these now warns that fewer
than the number required remained and says to request more.
R2_mixed_effects(), mlmr(), and
mlmr_mv() drop a replication for every statistic when it
has no value for one. Each statistic’s limits were the
quantiles of its own finite values, so one missing value moved the
interval of that statistic alone and the intervals of one call rested on
different replications. With 500 replications of which three lack the
marginal and two the conditional R-squared (the fixture of
test-dmar_bootstrap.R), the marginal limits were the
quantiles of 497 values and the conditional limits of 498; both are now
the quantiles of the same 495. When every replication returns every
statistic, which is the usual case, the limits do not change:
R2_mixed_effects(ci_method = "boot", B = 400, seed = 113)
on the sleepstudy random intercept model returns 0.2799
[0.1880, 0.4072] and 0.7043 [0.5838, 0.7964] in both builds.
No interval that is still reported changes for any other
reason. On 75 calls that the rule accepts and in which every
interval is still reported (1311 reported numbers;
mediate(), the kappa functions,
krippendorff_alpha(), htmt(), the two ratio
functions, the reliability family,
average_variance_extracted(),
fraction_missing_information(),
R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(),
mlmr_mv(), parallel_analysis(),
factor_count(), and sem_exact_fit_bootstrap(),
each with a seed), the largest absolute difference between this build
and the earlier one is 0.
ci_nc_t() computes the noncentral t
distribution exactly where R’s pt() is not accurate, so
every interval built on it is exact. DMAR 1.0.x inverted
pt() everywhere and warned only when the observed
ncp passed 37.62, so an interval whose upper limit alone
passed it was approximate without a warning. Limits inside the accurate
range are unchanged. The interval of ci_smd() for two
groups of 20 at d = 10 with 99 percent confidence was [7.003, 12.999]
and is [7.003, 13.099]; ci_nc_t(ncp = 34.5, df = 38) gave
[26.511, 42.273] and gives [26.511, 42.447];
ci_nc_t(ncp = 100, df = 1, conf_level = .99) gave
[-107.157, 257.157] and gives [0.394, 280.717];
ci_cv(cv = .05, N = 15) was [0.0369, 0.0818] and is
[0.0366, 0.0790]; and ci_sm(sm = 10, N = 30) was [7.315,
12.512] and is [7.412, 12.580]. ci_smd_c(),
ci_sc(), ci_sc_ancova(),
ci_reg_coef(), ci_rc(), ci_src(),
smd_trimmed(), and the sensitivity functions of the AIPE
planners inherit the exact limits. The warning “The observed
noncentrality parameter exceeds 37.62 in magnitude” is gone, and the
planners that passed it on (ss_aipe_sc() 425 times for
true_std_contrast = 4, width = .5,
assurance = .9) are silent. With more than 1e7 degrees of
freedom the integral is taken over the standardized chi square: its
density is computed without forming the chi square value, and the
argument of the normal distribution function, q s -
ncp, is formed as (q - ncp) + q(s - 1) with
s - 1 computed without forming s, since a double holds
neither the chi square value nor s finely enough there. Against
a 256-bit oracle, at 13 values of the degrees of freedom from 1e7 to
1e32, with q times the standard deviation of s from
.001 to 10, both tails of the distribution function, P(|T|
<= k), and the density are within 2.0e-14, the probability
the integral leaves out past the 1e-14 and 1 - 1e-14 quantiles of the
chi square. A plan of ss_aipe_sc() or
ss_aipe_sc_ancova() up to 2^53 per group with J
groups has at most J(2^53 - 1) degrees of freedom, fewer than
4.1e31 for any vector of contrast weights R can hold.
The AIPE planners built on ci_nc_t() plan
for the exact interval, and some plans change by one or two.
ss_aipe_smd(), ss_aipe_sm(),
ss_aipe_sc(), and ss_aipe_sc_ancova() plan for
the interval ci_nc_t() now reports, with exact noncentral
t probabilities in the published quantile assurance steps (that
of ss_aipe_sm(), and of ss_aipe_sc_ancova() by
default; the plans of ss_aipe_smd() and
ss_aipe_sc() with assurance are now the exact
ones of the entries below). Of 116 plans scanned with those steps
(standardized effects of 1 to 10, widths of .1 to 2.5, with and without
an assurance of .90), 11 changed, among them
ss_aipe_smd(true_smd = 5, width = 1) from 127 to 128,
ss_aipe_smd(true_smd = 3, width = .5, assurance = .9) from
277 to 278,
ss_aipe_sm(true_sm = 3, width = 1, assurance = .9) from 102
to 104,
ss_aipe_sc(true_std_contrast = 8, c_weights = c(1, -1), width = 1, assurance = .9)
from 300 to 299, and
ss_aipe_sc_ancova(true_std_contrast = 2, c_weights = c(1, -1), width = .25)
from 738 to 739. The two plans with assurance of
ss_aipe_smd() and ss_aipe_sc() quoted here are
also their exact plans.
cv_t() returns exact noncentral critical
values. With alpha_level = .05,
df = 3, and ncp = 50 the critical values were
29.0756 and 429.1783 and are 28.2828 and 186.4939; with
df = 10 and ncp = 45 they were 31.7440 and
83.9369 and are 31.3532 and 79.0839; and with one degree of freedom and
ncp = 38 the upper value was infinite and is 1212.586. The
area_less and area_greater columns are exact
as well.
cv_t() and ci_nc_t() with fewer
than one degree of freedom are exact, and the critical values of
cv_t() there change, by orders of magnitude with
df of 0.3 or less. DMAR 1.0.x took them from
qt(), which inverts pt(), and below one degree
of freedom pt() with a noncentrality is not accurate even
inside 37.62. The critical values moved by factors up to 2e47 with
df = 0.05 and 39 with 0.3, by as much as 15 percent with
0.5, and by less than 1e-4 of themselves with 0.6 and above
(noncentralities up to 37.5 in absolute value, alpha_level
.001 and .05): cv_t(.05, 0.1, ncp = 0.5) gave -33554432 and
305284064 and gives -1.0428e10 and 5.7095e13, and
cv_t(.001, 0.5, ncp = 38) gave -Inf and Inf and gives
8.478261 and 92490299.0. A critical value beyond the largest double is
-Inf or Inf. The limits of ci_nc_t() with
df < 1 change by at most 1.5e-10 of themselves.
The individual intervals of
limits_of_agreement() are exact for large samples.
Their noncentrality, k sqrt(n), passes 37.62 from n = 369 at 95 percent
coverage, and past it each confidence limit was too far from the mean
difference, by as much as 0.0011 standard deviations of the differences.
For
set.seed(1000); a <- rnorm(1000, 100, 15); b <- a + rnorm(1000, 0.5, 3),
the interval on the upper limit of agreement was [6.05197, 6.68620] and
is [6.05119, 6.68513]. The pair intervals (the default) are
unchanged.
Power past a noncentrality of 37.62 is exact.
The power planners whose test is a t test
(ss_power_smd(), ss_power_c(),
ss_power_sc(), ss_power_c_ancova(),
ss_power_contrast(), ss_power_mixed_effects(),
ss_power_reg_coef(), ss_power_rc(),
ss_power_pcm(), and, for the test of a with X
fixed, ss_power_indirect_effect()), the marginal powers of
ss_power_composite_ancova_2group(), the noninferiority
power of power_equivalence_c() and
ss_power_equivalence_c(), and
design_consequences() take the noncentral t
probabilities from the exact routines. No planned sample size changes,
since a search stops far below a noncentrality of 37.62; the power
reported for a specified sample size changes where the noncentrality
passes 37.62 and the test has few degrees of freedom (by as much as .28
with one degree of freedom and .034 with two; with 10 or more it was and
is 1 to within 4e-11).
ss_power_smd(true_smd = 40, n_1 = 2, alpha_level = .001)
reported a power of .7823613 and reports .7981440; the marginal power of
the group test of
ss_power_composite_ancova_2group(true_smd = 40, true_cor = .3, specified_n_per_group = 2, composite_terms = c("group", "covariate"), include_interaction = FALSE, alpha_level = .001)
was .1666 and is .0525; and
design_consequences(true_effect = 40, se = 1, df = 2, alpha_level = .001)
reported a power of .7823613, a Type S error of 1.2e-5, and an
exaggeration ratio of 1.0207, and reports .7981440, a Type S error below
1e-300, and 1.0005. The power of a negative effect at a specified sample
size (ss_power_smd(true_smd = -1, n_1 = 50), and the same
in ss_power_c(), ss_power_sc(),
ss_power_mixed_effects(), and an example of
ss_power_composite_ancova_2group()) passed on the warning
of pt()
full precision may not have been achieved in 'pnt{final}';
these calls are now silent, with the same values.
The Bayesian t tests use the exact noncentral
t density. bayes_one_sample_t(),
bayes_paired_t(), and bayes_independent_t()
integrate the noncentral t likelihood over the prior, and R’s
dt() with a noncentrality rests on the normal approximation
past 37.62 and loses accuracy near t = 0 and with many degrees
of freedom. It is now used only with at most 1,000 degrees of freedom, a
noncentrality of at most 37.62, and |t| of at least .1, where
results are unchanged, and the density is integrated exactly elsewhere,
except at t = 0, where it is exp(-ncp^2 / 2) / {B(df / 2, 1/2)
sqrt(df)}. R’s dt(), which DMAR 1.0.x used there, computes
it at a nonzero noncentrality by differencing two log gammas, and is off
by as much as 3.4e-8 of it short of 1e7 degrees of freedom (at
9,832,229.8): dt(0, 5.6e6, 1.5) is 0.12951758933 where the
density is 0.12951758988, and
bayes_one_sample_t(mean = 0, sd = 1, N = 5600001) gave a
Bayes factor bf_01 of 2097.196624 in DMAR 1.0.0 and gives
2097.196615. bayes_one_sample_t(mean = 5.7, sd = 1, N = 50)
(t = 40.3) gave a posterior median of 5.6038, a 95 percent
interval of [4.4709, 6.7759], and a Bayes factor of 5.3589e35, and gives
5.5967, [4.4709, 6.7901], and 5.3537e35. Such calls take about 0.4
seconds, up from 0.05 to 0.1.
The Tukey and Games-Howell procedures, and the
Bryant-Paulson distribution with no covariates, use the exact
studentized range wherever R’s qtukey() and
ptukey() are not accurate (see Exact Noncentral
t and Studentized Range Distributions). R documents
qtukey() as accurate to the fourth decimal place. It falls
short of that at small error df, by more as the number of means and the
confidence level grow (with 10 means and 2 df its .999 quantile is 41
percent too small, so intervals built on it have a familywise error rate
of about .0029 rather than .001); at large df; far in either tail; below
the .75 point with more than 50 means; and where its search stops short
of the root of ptukey(). The exact computation is used
below 9 error df with up to 6 means, 11 with up to 10, 15 with up to 20,
21 with up to 50, 31 with up to 100, 41 with up to 250, 47 with up to
400, 57 with up to 700, 71 with up to 1000, 81 with up to 1500, 91 with
up to 2000, 101 with up to 3000, and 111 with more (measured to 10000
means and extrapolated beyond); above 10000 and up to 10^8 df; for every
quantile at a probability above .999 or below .001; with more than 50
means, below the .75 point (the .80 point with more than 1000 means);
and wherever qtukey() fails or is more than 2e-6 from the
root of ptukey(). Elsewhere the values are those of
qtukey() and ptukey() (above 10^8 df, their
infinite-df values), which were within 9.6e-6 of the exact quantile and
9.1e-7 of the exact probability at every point checked where they are
kept. The changed values:
| call | DMAR 1.0.x | now |
|---|---|---|
cv_tukey_hsd(.01, df = 2, n_groups = 20) |
31.485 | 26.830 |
cv_tukey_hsd(.01, df = 3, n_groups = 20) |
14.105 | 13.976 |
cv_tukey_hsd(.05, df = 2, n_groups = 20) |
11.867 | 11.857 |
cv_tukey_hsd(.001, df = 2, n_groups = 10) |
41.612 | 71.063 |
cv_tukey_hsd(.001, df = 12, n_groups = 50) |
8.0894 | 8.0897 |
cv_tukey_hsd(.05, df = 30000, n_groups = 10) |
3.16368 | 3.16392 |
cv_tukey_hsd(1e-4, df = 11, n_groups = 10) |
8.491265 | 8.491348 |
cv_tukey_hsd(1e-5, df = 1000, n_groups = 2000) |
9.1427 | 6.9804 |
cv_tukey_hsd(.001, df = 41, n_groups = 1000) |
7.282740 | 7.282842 |
cv_tukey_hsd(.025, df = 12, n_groups = 10) |
4.245304 | 4.245333 |
cv_tukey_hsd(.0025, df = 23, n_groups = 5) |
4.255839 | 4.255754 |
cv_tukey_hsd(.025, df = 15000, n_groups = 100) |
4.479376 | 4.479297 |
qbryant_paulson(.99, n_covariates = 0, n_groups = 20, df = 2) |
44.526 | 37.943 |
qbryant_paulson(.00001, n_covariates = 0, n_groups = 3, df = 150) |
0.000092 | 0.006023 |
ci_tukey_kramer(), 20 groups, 2 error df,
conf_level = .99: lower limit of “2 - 1” |
-4.718 | -3.784 |
| the same, adjusted p of “2 - 1” | .10649 | .10647 |
ci_games_howell(), the three small groups of the entry
under Bug Fixes: lower limit of c - b (2.55 df) |
-6.03754 | -6.03731 |
qbryant_paulson(.99, n_covariates = 1, n_groups = 20, df = 10) |
8.815518 | 8.815520 |
Every example and vignette result DMAR 1.0.x printed is unchanged, as
are the Bryant and Paulson (1976) Table 1 values the tests check. The
probability integral of the range, which the Bryant-Paulson functions
with covariates used below 7 error df in DMAR 1.0.x, is now taken in two
pieces, since in one it could miss the peak of the integrand with many
means (with 7000 means it returned 2.3e-17 where the probability is
9.5e-6), and with covariates the quantiles and probabilities beyond .999
and below .001 integrate the exact studentized range
(qbryant_paulson(.9999, 1, 10, 11) was 12.977072 and is
12.977142). An exact critical value costs a few milliseconds, after
about half a second the first time a number of means is used in a
session: ci_games_howell() with 10 groups of 3 takes about
0.6 s rather than 0.07 s.
Bryant-Paulson values with covariates where DMAR 1.0.x computed them. The new range of the integral (see Bug Fixes) moves the nodes of the quadrature, which moves the values by about 10^-9 at most below 10^5 error df and by less than 10^-6 above, at most in the seventh significant figure (the most for quantiles beyond .999 above 10^7 df). No printed result changes: the examples, the vignettes, and the published Table 1 values the tests check are unchanged. Where DMAR 1.0.x returned about 0, stopped, or returned a wrong critical value, the values are those of the bug fix. Compared with DMAR 1.0.x, the values above 10000 df also carry the change to the exact studentized range in the entry above.
| call | DMAR 1.0.x | now |
|---|---|---|
pbryant_paulson(3.7, n_covariates = 1, n_groups = 4, df = 1e8) |
5.6e-25 | 0.9559514 |
qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 1e8) |
error | 3.6331597 |
qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 4.09e7) |
4.5327092 | 3.6331598 |
qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = Inf) |
error | 3.6331596 |
pbryant_paulson(3.5, n_covariates = 150, n_groups = 3, df = 1e6) |
0 | 0.9644250 |
qbryant_paulson(.95, n_covariates = 200, n_groups = 3, df = 1e6) |
error | 3.3148295 |
qbryant_paulson(.9999, n_covariates = 3, n_groups = 4, df = 1e7) |
6.0828639 | 6.0828670 |
ss_aipe_r() with assurance
returns the smallest sample size that attains it, computed from the
distribution of the sample correlation. The Fisher’s Z
interval is no wider than the target exactly when |r| is at
least a value the function solves for, and the probability of that event
is taken from the exact distribution function of r under
bivariate normality; the new rows assurance and
assurance_attained give the request and that probability.
DMAR 1.0.x multiplied the plan by qchisq(assurance, N - 2) / (N - 2). At
true_cor = .90 and width = .10 it returned 72
for assurance = .80 and 92 for .99, where the
probabilities are .672086 and .887068; the plans are now 83 and 122
(.807291 and .990268). At true_cor = 0 it returned 1585 for
width = .10 and assurance = .80, where the
plan without assurance, 1538, already attains a probability
of 1 and is now returned. ss_aipe_r_sensitivity() with
supposed_cor and assurance changes with
it.
ss_aipe_smd() with assurance
returns the smallest per-group sample size, not below the plan without
assurance, at which the exact probability that the interval
is no wider than width is at least assurance,
for |true_smd|. DMAR 1.0.x and
MBESS::ss.aipe.smd() returned the plan of the procedure of
Kelley and Rausch (2006, pp. 372–374), which does not compute that
probability (see The Assurance Method of the Sample Size Planners).
true_smd = 1, width = .5, and
assurance = .99 it was 150 per group (.994302) and is 149
(.990260). In 45 planning sets (|true_smd| of 0 to 2,
widths of .25 to 1, assurances of .80 to .99) the published plan
attained the request in every set and was one case larger in 7, six of
them at .99.true_smd = 0,
width = .8, and assurance = .95 it was 49
(.949878) and is 50 (.994636).true_smd, so reversing
the groups changed the plan: at width = .5 and
assurance = .90 it returned 125 per group at
true_smd = -.5 and 133 at -1, below the plans
without assurance, where the probability is .146146 and
.064400. They are now 130 and 144 (.915627 and .904165), as for .5 and
1.ss_aipe_smd_sensitivity() changes with it.ss_aipe_equivalence_smd() plans for the
interval equivalence_smd() reports, d -/+ t SE(d)
with the Student t quantile on n_1 + n_2 - 2 degrees of
freedom, where DMAR 1.0.x planned with the narrower standard normal
quantile, so the analysis at the planned size could report an interval
wider than width. The search starts at 2 per group, the
smallest size equivalence_smd() accepts (1.0.x never
returned fewer than 4). For true_smd = 0 and
width = 2.5 the plan was 4 per group, with an
expected_width of 2.3262, where
equivalence_smd() at 4 per group reports 2.7481; it is now
5, with 2.3522. For width = 0.5 the plan was 87 and is 88.
The table records the confidence level of the interval, 1 - 2 alpha, in
its conf_level attribute.
ss_aipe_equivalence_smd() computes the
probability for assurance from the distribution of the
sample standardized mean difference, and reports it in the new
rows assurance and assurance_attained, by
numerical integration (the form of Kelley and Rausch, 2006, Equation
20). DMAR 1.0.x simulated 2,000 widths of the narrower normal-quantile
interval at each size, so its plans fell short of the request and
differed from run to run. With true_smd = 0.5,
width = 0.5, and assurance = 0.80 it returned
91 per group, where the probability is .671; the plan is now 92 (.828).
With true_smd = 0 it returned 87 or 88; the plan is now 88
(.839). The function draws no random numbers and has no
seed argument.
ss_aipe_equivalence_smd_sensitivity()
simulates the interval equivalence_smd() reports,
at conf_level, where DMAR 1.0.x simulated the noncentral
t interval of ci_smd(), which is usually narrower;
the width, miss, and equivalence rows change with it. With
true_smd = 0.3, specified_n_per_group = 61,
conf_level = 0.90, and set.seed(113), the
first simulated sample (d = 0.670) was scored with the limits 0.363 and
0.975 and is now scored with 0.362 and 0.979; with
true_smd = 0, supposed_smd = 0,
width = 0.3, smd_upper = 0.5,
G = 2000, and seed = 113,
pct_ci_less_width was .856 at 342 per group and is .736 at
343. The simulated samples are the ones 1.0.0 drew, so the
mean_smd, median_smd, and sd_smd
rows are unchanged under a common seed and size. No argument restores
the earlier results of ss_aipe_equivalence_smd() or of its
sensitivity function. equivalence_smd() returns what it
returned before, except for sizes given as integers or counted from raw
data whose product passes 2^31 - 1 (46,341 or more in each group, or
40,000 and 60,000), for which it returned NA (see Bug
Fixes).
The plans of ss_aipe_reg_coef(),
ss_aipe_rc(), and ss_aipe_src() without
assurance are the smallest sample size whose interval meets
the width, with the expected squared multiple correlation of the
targeted predictor on the other p - 1 predictors. DMAR 1.0.x,
like MBESS::ss.aipe.reg.coef(), did two things differently.
It took the expected squared multiple correlation of X_j with the other
predictors to be that of a regression on p predictors, which
overstates it and so the planned width. And it searched upward one case
at a time from a start, for the central interval the larger of 2p + 1
and nine below the normal approximation rounded up, and for the
noncentral one the larger of 2p + 2 and five below the central plan, so
it returned the start when a smaller sample size already met the width.
The expectation now has p - 1 predictors (0 with one predictor), and the
search, by doubling and bisection, covers every sample size from p + 2,
one residual degree of freedom. Examples, old to new:
true_R2_Y_X = .8,
true_R2_j_X_without_j = 0, p = 10,
true_reg_coef_j = .3, width = 2: the central
plan was 21 and is
14.true_R2_Y_X = .1,
true_R2_j_X_without_j = .2, p = 2,
true_reg_coef_j = 0, width = .4,
noncentral = TRUE, tails of .0002: the noncentral plan was
358 and is 355.width = .3: the central plan was 157 and is 156, and the
noncentral plan was 165 and is 164.In 1,530 central planning sets (p from 1 to 15, squared multiple correlations from 0 to .9, widths from .05 to 2, tails of .025 and .025, .005 and .005, and .01 and .04) the plan is one case smaller in 1,329, 2 to 13 smaller in 104, and unchanged in 97. In 810 noncentral sets it is one smaller in 596, 2 to 10 smaller in 177, and unchanged in 37. Every plan equals the smallest sample size from p + 2 that a scan of every N finds, and a noncentral plan takes about half the time it did.
The published approximations with assurance start from
these plans and change with them. They are the chi square plan of Kelley
and Maxwell (2003, Equation 3), DMAR 1.0.x’s plan for the central
interval and now the default of
ss_aipe_src(noncentral = FALSE), and the noncentral
approximation, the default of
ss_aipe_reg_coef(noncentral = TRUE) and
ss_aipe_src():
assurance fell to a
few residual degrees of freedom and Equation 3 takes its t
quantile there. At .95 it is smaller in 1,315, unchanged in 80, and 1 to
26 larger in 135. At .80 it attains the request for the interval in the
units of the data in 752 of the sets, where it did in 797.The plans that compute the probability (the exact central plan and
those of assurance_method = "simulation") change only where
their answer was the earlier plan without assurance. In the
1,530 sets, 83 exact plans at .80 and 49 at .95 are 1 to 12 cases
smaller, each still the smallest N at or above the plan without
assurance whose probability meets the request. The exact
and simulated plans in the table of the next entry are unchanged by it,
and in 120 noncentral sets with G = 2000 no simulated plan
changed. The sensitivity functions plan through these planners and
change with them (ss_aipe_src_sensitivity() in an entry
below: 157 to 156). No argument restores the earlier plans.
ss_aipe_reg_coef() and
ss_aipe_rc() with assurance return, for the
central interval, the smallest sample size at which the interval is no
wider than width with at least that probability, computed
exactly. DMAR 1.0.x used the chi square factor of Kelley and
Maxwell (2003, Equation 3; 2008, Equation 37) for the central interval
and, for the noncentral one, a lower confidence limit of the squared
multiple correlation in place of its population value; neither computes
the probability it attains, which can be more or less than the request.
Three intervals are planned for in 1.1.0:
ss_aipe_reg_coef() with noncentral = FALSE
(the default) and ss_aipe_rc() plan the central interval
for the coefficient in the units of the data, even at unit standard
deviations; ss_aipe_reg_coef(noncentral = TRUE) and
ss_aipe_src() the noncentral interval for the standardized
coefficient; and ss_aipe_src(noncentral = FALSE) (new) the
central interval for the standardized coefficient, Kelley and Maxwell’s
(2003) Equation 1. For the central interval in the units of the data the
new argument assurance_method, "analytic" by
default and the only method offered there, gives the smallest sample
size, at or above the plan without assurance, at which that
probability, an exact F probability for random predictors from
a multivariate normal population, is at least assurance
(280 for the example of Kelley and Maxwell, 2008, p. 186, where DMAR
1.0.x and MBESS::ss.aipe.reg.coef() gave 278). For the
standardized coefficient "analytic" gives the
approximations of 1.0.x, from the plans without assurance
of this release (see the entry on those plans), with
assurance_attained NA, and
assurance_method = "simulation" estimates the probability
from G samples (default 20000) of the exact Wishart
sampling distribution, with seed, and gives the smallest
sample size at which the estimate reaches assurance;
assurance_attained is the estimate at the returned size,
its Monte Carlo standard error is the attribute
assurance_mc_se, and the plan can be a case or two from the
one the exact probability gives. Given squared multiple correlations,
which leave the correlation of Y with the predictor open, the
simulated step uses the least favorable value.
Old and new sample sizes, with the probability each attains for
the interval in the second column: “raw central”
(ss_aipe_reg_coef() with noncentral = FALSE,
ss_aipe_rc()), “std noncentral”
(ss_aipe_reg_coef(noncentral = TRUE),
ss_aipe_src()), and “std central”
(ss_aipe_src(noncentral = FALSE), whose 1.0.x plan is the
chi square plan of ss_aipe_reg_coef(noncentral = FALSE)).
Each probability is exact or the proportion of 50,000 or more simulated
samples (Monte Carlo standard error at most .0014). The simulated plans
are those of seed = 1 and the default G;
another seed can give a plan a case away. Variances are 1 unless
stated:
| planning set, width, assurance | interval | 1.0.x N (attains) | 1.1.0 N (attains) | 1.1.0 "simulation" N (attains) |
|---|---|---|---|---|
| two predictors correlated .3, correlations .4 and .3 with Y, .3, .90 | raw central | 182 (.862) | 187 (.902) | not offered |
| the same, .3, .80 | raw central | 174 (.777) | 176 (.801) | not offered |
| the same, .3, .99 | raw central | 203 (.974) | 213 (.990) | not offered |
| the same, .3, .90 | std central | 182 (.989) | 181 (.987) | 171 (.914) |
| the same, .3, .90 | std noncentral | 180 (.957) | 179 (.947) | 176 (.909) |
| the same, .3, .80 | std noncentral | 177 (.923) | 176 (.907) | 172 (.818) |
| five exchangeable predictors (.4 among them, .3 with Y; Kelley & Maxwell, 2003, pp. 312–313), .2, .80 | std central, the interval of the article | 487 (.884) | 486 (.877) | 478 (.807) |
| the same, .2, .80 | raw central | 487 (.773) | 491 (.800) | not offered |
| the same, .2, .99 | std noncentral | 498 (.944) | 497 (.940) | 517 (.989) |
| Kelley and Maxwell’s (2003, p. 313) three predictors, the first, .3, .90 | raw central | 271 (.859) | 278 (.904) | not offered |
| the same, the first, .3, .99 | std noncentral | 314 (.952) | 313 (.949) | 330 (.990) |
| one predictor correlated .4 with Y, .2, .90 | raw central | 362 (.841) | 373 (.904) | not offered |
| the same, .2, .90 | std noncentral | 377 (1.000) | 376 (1.000) | 364 (.911) |
| Kelley and Maxwell’s (2008, p. 186) example (p = 5, R^2 .5 and .2, sigma_Y / sigma_X = 10), 2, .85 | raw central | 278 (.838) | 280 (.853) | not offered |
| its standardized analog (coefficient .3, unit standard deviations), .2, .85 | std noncentral | 296 (.894) | 295 (.886) | 291 (.854) |
The default plans for the standardized coefficient are the 1.0.x
approximations, a case smaller through the plans without
assurance; the probabilities they attain were simulated
from 200,000 or more samples of the exact sampling distribution (Monte
Carlo standard error at most .0011). The five exchangeable predictors
are Kelley and Maxwell’s (2003) example, planned with the central
interval of their Equation 1 (485 in the article, which uses the normal
quantile); at .99 for them the simulated plan’s own estimate is .990 and
the simulation’s .989, within the Monte Carlo error. The plan is the
same for a coefficient and its negative. The three sensitivity functions
plan through these planners, each for the interval it simulates, and
change with them.
No argument restores the central plans of DMAR 1.0.x with
assurance; for unequal tails see the next entry. The
default plans for the standardized coefficient change only through the
plans without assurance. DMAR 1.0.x stopped with an error
on every plan of the three sensitivity functions with one predictor.
ss_aipe_src(noncentral = FALSE) is new and has no earlier
result.
With unequal tails, the central plans of
ss_aipe_reg_coef() and ss_aipe_rc() are for
the interval ci_reg_coef() reports (and so are
those of ss_aipe_src(noncentral = FALSE), new in this
release). That interval runs from b_j + t(alpha_lower) SE to b_j + t(1 -
alpha_upper) SE, so its width is t(1 - alpha_lower) + t(1 - alpha_upper)
times the standard error. DMAR 1.0.x took both quantiles at the average
of the two tails, t(1 - (alpha_lower + alpha_upper)/2), in the plan
without assurance and in its chi square step with
assurance (Kelley and Maxwell, 2003, Equation 3), and so
planned an interval narrower than the one reported. For two predictors
correlated .3, correlated .4 and .3 with Y, and a width of
.3:
| tails (lower, upper) | plan | 1.0.x N | 1.1.0 N |
|---|---|---|---|
| .01, .04 (or .04, .01) | without assurance |
157 (the interval .3108 wide) | 169 (.2992) |
| .001, .049 | without assurance |
157 (.3634) | 228 (.2996) |
| .01, .04 | assurance = .90 |
182 (attains .709) | 201 (attains .902) |
| .001, .049 | assurance = .90 |
182 | 266 (attains .905) |
| .025, .025 | without assurance; assurance = .90 |
157; 182 | 156; 187 |
The widths are those of the reported interval at the expected squared
multiple correlations, and the probabilities are exact. Without
assurance equal tails give the plans they gave, but for the
change in the entry on the plans without assurance; with
assurance the plan is the exact one of the entry above. The
noncentral plans used the two tails as given and are unchanged by this
correction. The chi square step with both quantiles, the default of
ss_aipe_src(noncentral = FALSE) with
assurance, gives 194 (attaining .847 for the interval in
the units of the data) and 258. With unequal tails no argument restores
the 1.0.x central plans of ss_aipe_reg_coef() and
ss_aipe_rc() (or the plan of
MBESS::ss.aipe.reg.coef()). ss_aipe_src() is
not affected: its noncentral plans used both tails in 1.0.x, so this
correction does not change them, and its central plans are new. The
sensitivity functions plan with equal tails, so it does not change them
either.
The limits of the central ci_reg_coef(),
ci_c(), and ci_r() move, by up to 1.25e-7 of
themselves near the floor. With the upper quantile taken in the
upper tail (see Bug Fixes) each limit is the exact one, and the change
grows as the tail shrinks. In 2,160 calls of these three functions, at
levels from .90 to .999999998 and tails from .05 down to 1e-9, against
the development versions of 1.1.0, the limits moved by at most 6.1e-14
of themselves at levels of .90 to .999 and tails of 0 to .05, 2.2e-11 at
a tail of 1e-6, 1.1e-9 at a level of .9999999, and, at a tail of 1e-9 or
a level of .999999998, 1.1e-8 (ci_c() and
ci_reg_coef()) and 1.25e-7 (ci_r(), whose
lower quantile changed as well). Near the floor a limit moves in about
its eighth or ninth significant digit.
ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = .999999998)
(n in 1.0.0) has an upper limit of 3.4711584085612,
symmetric with its lower limit of -1.4711584085612, where DMAR 1.0.0 and
the development versions gave 3.47115840279224, and
ci_r(r = -.9, N = 20, conf_level = .999999998)
(n in 1.0.0) has an upper limit of -0.0175358196414772,
where they gave -0.0175358218277256. The assurance_attained
of the exact central assurance step of ss_aipe_reg_coef()
moves the same way:
ss_aipe_reg_coef(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 4, true_reg_coef_j = .2, width = .3, assurance = .8, conf_level = NULL, alpha_lower = .04, alpha_upper = 1e-9)
reports 0.80195028591575, where the development versions reported
0.801950281617816, for the same plan of 643. The 768 noncentral
intervals of ci_reg_coef() among those calls were
unchanged, as were ci_sc(), ci_sc_ancova(),
ci_sm(), ci_cv(), ci_smd(), and
every sample size in 1,944 further calls, the plans of
ss_aipe_reg_coef(), ss_aipe_sc(),
ss_aipe_sc_ancova(), ss_aipe_c(), and
ss_aipe_r() among them, and the 1,530 central plans of
ss_aipe_reg_coef() over a grid of squared multiple
correlations, predictors, widths, and tails and the 1,080 plans of
ss_aipe_sc_ancova() over a grid of contrasts, correlations,
divisors, weights, widths, and levels. (Other entries change some of
these plans: those on the plans without assurance of
ss_aipe_reg_coef() and of
ss_aipe_sc_ancova().)
ss_aipe_src_sensitivity() plans the sample
size for the standardized coefficient, for the interval it
simulates. The plan came from ss_aipe_reg_coef()
with the supposed standard deviations, so width was taken
in the units of the data while the simulated intervals were
standardized. For true_var_Y = 4,
true_cov_YX = c(.8, .6), unit predictor variances
correlated .3 (correlations of .4 and .3 with Y), supposed
correctly, width = .3, and noncentral = FALSE,
the plan was 610, with a mean width of .150, and is 156 (mean width
.300). With assurance = .90 it was 657 (182 with unit
supposed variances). It is 181 by default, the chi square plan of Kelley
and Maxwell (2003, Equation 3) for the central interval for the
standardized coefficient, which the replications compute (.987 of those
intervals are no wider than .3 at 181), and 171 with
assurance_method = "simulation" and seed = 1,
the smallest sample size at which that interval is no wider than .3 with
probability at least .90 (.914 at 171). No argument restores the 1.0.x
plans. With equal variances of Y and of the targeted predictor,
true and supposed, whatever the variances of the other predictors, and
two or more predictors, the default plans are made as in 1.0.x, a case
smaller (see the entry on the plans without assurance). The
noncentral plan, the default interval, was refused whenever the supposed
variance of Y or of the targeted predictor was not 1, and every
other plan with one predictor stopped with “argument is of length zero”;
such calls now plan (the noncentral plan is 164 here). The numbers for
noncentral = FALSE are also those of DMAR 1.0.x’s
ss_aipe_reg_coef_sensitivity(standardize = TRUE).
The standardized regression coefficient in the
sensitivity functions is judged against the standardized
coefficient. ss_aipe_src_sensitivity()
standardizes the data of every replication, but the true and supposed
values the intervals were judged against were unstandardized; both are
now the standardized coefficient (Kelley & Maxwell, 2003, p. 316).
For the structure above with specified_N = 200,
width = .3, G = 300, and seed 2026, the true
value was .681 beside a mean estimate of .334, and every interval missed
(total_type_I_error 1); it is now
true_std_reg_coef_j .341, with
total_type_I_error .017 (.033 with
noncentral = FALSE). A call with specified_N
changes as well, as does one with one predictor
(true_var_Y = 4, true_cov_YX = .8,
true_cov_XX = matrix(1), and
specified_N = 100: .8 in DMAR 1.0.x, .4 now).
ss_aipe_sc() with assurance
returns the smallest sample size per group, not below the plan without
assurance, at which the exact probability that the
ci_sc() interval is no wider than width is at
least assurance. DMAR 1.0.x returned, in one row,
the plan of the step of Kelley and Rausch (2006) as Lai and Kelley
(2012, pp. 358–359 and 361) apply it to a standardized contrast, which
does not compute that probability (see The Assurance Method of the
Sample Size Planners).
true_std_contrast = 0,
c_weights = c(1, -1), width = .8, and
assurance = .95 it was 49 per group (.949878) and is 50
(.994636).true_std_contrast it
planned the magnitude with the tails as given (see Bug Fixes). At
true_std_contrast = -2.5, width = .4,
alpha_lower = .001, alpha_upper = .049, and
assurance = .90 it was 522 per group (.890) and is 523
(.902); at -3, width .5, and tails .001 and .099 it was 342 (.842) and
is 346 (.902).assurance and
assurance_attained. The plan without assurance
is unchanged.ss_aipe_sc_sensitivity() changes with it.ss_aipe_sc_ancova() without
assurance returns the smallest sample size per group whose
interval at the planning value is no wider than
width. DMAR 1.0.0 started its search at the fixed
point of the central t approximation, n = C r^2 (t_l + t_u)^2 /
w^2 with t on J(n - 1) - 1 degrees of freedom, and searched
only upward. At a small contrast the central quantiles are farther apart
than the noncentral limits, so the start could already meet the width,
and the plan was then the start:
ss_aipe_sc_ancova(true_std_contrast = 0, c_weights = c(1, -1), width = .5)
(psi_standardized in 1.0.0) gave 125 per group, though the
interval is .49985 wide at 123 and .50190 at 122. It now gives 123. The
search now starts at 2 per group, the smallest sample size with positive
error degrees of freedom, and the width falls as the sample size grows
from there (checked on 420 combinations of contrast, ratio of the two
error standard deviations, weights for two to five groups, and level).
Of 1,080 plans over contrasts of 0 to 2, correlations of 0, .5, and .9,
both divisors, weights for two to four groups, widths of .1 to 3.6, and
levels of .95 and .99, 446 are smaller: 304 by 1 per group, 141 by 2,
and 1 by 3. None is larger, and no plan at a contrast of 2 changed. At
widths of 1 or less, 198 of 720 are smaller, by 1 or 2. Each plan meets
the width, and one fewer does not. The published assurance step, the
default (assurance_method = "analytic"), plans from this
plan and changes with it. On the same grid, 302 plans at an assurance of
.80 are 1 or 2 smaller. At .90, 227 are 1 or 2 smaller and 7 are 1
larger: the call above with assurance = .9 gave 125 and
gives 124. The step plans without assurance at a contrast
it adjusts for the assurance, so its plan can move by a case or two per
group, in either direction, even where the plan without
assurance at the planning values did not: at
true_std_contrast = 2, true_cor = .5,
c_weights = c(1, -1), width = .3, and
assurance = .99 it was 545 per group and is 546. The
simulated step (assurance_method = "simulation", new in
1.1.0) also searches upward from this plan. With
conf_level = .99 the call above gave 215 per group without
assurance and gives 213, and with
assurance = .8 it gave 215 and gives 213 by default and 214
with assurance_method = "simulation" and
seed = 1 (assurance_attained .886). The
default plans in the table of the ANCOVA entry under The Assurance
Method of the Sample Size Planners are those of 1.0.x.
ss_aipe_sc_ancova_sensitivity() plans through the planner
and changes with it. Past about 1e11 per group the plans rest on the
precision of doubles, and they move as well (Details of
?ss_aipe_sc_ancova). No argument restores the earlier
plans.
ss_aipe_sc_sensitivity() standardizes each
simulated contrast by the pooled standard deviation, the square
root of the mean square error, which is the s_anova that
ci_sc() takes. DMAR 1.0.x divided by the mean of the group
standard deviations, which is never larger, so its contrasts ran about 2
percent large with three groups of 10. After set.seed(113),
ss_aipe_sc_sensitivity(true_std_contrast = .5, specified_n_per_group = 10, c_weights = c(.5, .5, -1), width = .8, G = 2000)
gave mean_std_contrast .527 and
total_type_I_error .070 and gives .517 and .064. No
argument restores the 1.0.x result.
ss_power_indirect_effect() computes power
from the exact sampling distribution of the two path t
statistics, with X random by default (see Bug Fixes). Every
result changes, and no argument restores the 1.0.x results:
random_predictors = FALSE fixes X as 1.0.x did, and with it
power_a is the 1.0.x power of the test of a, but
the mediator is random in every design, so the 1.0.x
power_b and power_ab are the powers of no
study. power_ab is no longer power_a times
power_b but the probability that both tests reject, which
is below that product. The Sobel power need not rise with N, since the
test refers t statistics on few degrees of freedom to the
normal: with paths of .9 and .05 it falls from .0729 at N = 10 to .0562
at N = 59, and with paths of .8 and .02 and
alpha_level = .5 it is .5072 at N = 12 and .5035 at N = 59,
and does not reach .507 again until N = 216. The search no longer takes
one case at a time; it passes over a range of sample sizes only where an
upper bound of the power shows that none in it reaches
desired_power, so the plan is the smallest N of at least 10
that attains desired_power, whether or not the power rises
with N. It still stops with an error at ten million. The values (1.0.x,
now with X random, now with X fixed):
| call | 1.0.x | now | now, random_predictors = FALSE |
|---|---|---|---|
ss_power_indirect_effect(.39, .39, specified_N = 65):
power_ab, power_a, power_b |
.8017, .9196, .8719 | .7675, .9046, .8503 | .7804, .9196, .8503 |
ss_power_indirect_effect(.39, .39, specified_N = 75):
power_ab |
.8706 | .8436 | .8537 |
ss_power_indirect_effect(.3, -.25, .1, specified_N = 120):
power_ab |
.7056 | .6876 | .6936 |
ss_power_indirect_effect(.6, .5, .2, specified_N = 12):
power_ab |
.2367 | .1631 | .1826 |
the same four with method = "sobel":
power_ab |
.6380, .7001, .5639, .3165 | .6157, .7272, .5067, .1226 | .6305, .7410, .5135, .1367 |
ss_power_indirect_effect(.39, .39, specified_N = 96, method = "sobel") |
.8026 | .8818 | .8907 |
ss_power_indirect_effect(.2, .1, specified_N = 10, alpha_level = .999, method = "sobel") |
.9990 | .9983 | .9983 |
necessary_N,
ss_power_indirect_effect(.39, .39, desired_power = .80) |
65 | 69 | 68 |
the same with method = "sobel" |
96 | 84 | 82 |
necessary_N,
ss_power_indirect_effect(.14, .39, desired_power = .80),
joint significance and Sobel |
395, 438 | 398, 412 | 395, 410 |
necessary_N,
ss_power_indirect_effect(.3, -.25, .1, desired_power = .80),
joint significance and Sobel |
144, 210 | 148, 179 | 147, 178 |
necessary_N,
ss_power_indirect_effect(.1, .59, desired_power = .95),
joint significance and Sobel |
1289, 1312 | 1293, 1296 | 1289, 1292 |
necessary_N,
ss_power_indirect_effect(-.3, .2, desired_power = .50),
joint significance and Sobel |
116, 141 | 119, 157 | 118, 155 |
necessary_N,
ss_power_indirect_effect(.0044, .5, desired_power = .80) |
405411 | 405414 | 405411 |
necessary_N,
ss_power_indirect_effect(.8, .02, desired_power = .507, alpha_level = .5, method = "sobel") |
229 | 12 | 10 |
meta_r() with reliabilities gives each
corrected correlation the sampling variance of a corrected
correlation. Each correlation corrected for attenuation, r /
sqrt(rxx ryy), was given on the Fisher Z scale the variance of
an uncorrected correlation, 1 / (n - 3), which is too small (by a factor
of 1.96 for r = .40 with both reliabilities .80). The variance is now
the delta method variance (1 - r2)2 / (rxx ryy (1 -
r_c2)2 (n - 3)), with r_c the corrected correlation, at each
study’s observed correlation; it sets the weights and every result.
Without reliabilities nothing changes. For five studies with r = .40, n
= 100, and both reliabilities .80 (method = "fe",
hartung_knapp = FALSE), the standard error on the Fisher
Z scale was 0.0454 and is 0.0636, and the interval for the
construct correlation was .430 to .564 and is .401 to .588. For
meta_r(r = c(.21, .35, .42, .18, .30, .47), n = c(48, 120, 75, 300, 60, 150), reliability_x = c(.72, .85, .78, .90, .65, .80), reliability_y = c(.80, .70, .88, .75, .82, .90)),
the pooled correlation was .4095 and is .4051, Q was 20.14 and is 12.05,
and I2 was 69.0 percent and is 54.1 percent. ?meta_r states
the variance and that the reliabilities are treated as known constants.
An analysis that gave reliabilities to meta_r() should be
rerun.
var_alpha() returns each variance in the row
named for its source. DMAR 1.0.x returned 2J(1 -
alpha)^2/((J - 1)(N - 2)) in the row
var_alpha_van_zyl and 2J(1 - alpha)^2/((J
- 1)(N - 1)) in the row var_alpha_bonett. Van Zyl,
Neudecker, and Nel (2000, Equation 13) give the second, and Bonett
(2003, Equations 2 and 4) gives the first, on the scale of the logarithm
of 1 - alpha; the van Zyl variance that
reliability_alpha(ci_method = "ml") and
ss_aipe_reliability() use is, at a compound symmetric
covariance matrix, the N - 1 form. The two values now sit in
the rows their names attribute; the pair is unchanged. No argument
restores the earlier rows.
fa_confirmatory(output = "standardized")
(and cfa_1(), cfa_2(), and
cfa_k()) computes the standardized
loading_sum, error_sum, omega,
ave, and H rows of a factor with
equal_loading or equal_error from each item’s
own standardized loading and error variance, with delta method
standard errors. lavaan’s standardizedSolution(), the
earlier source, evaluates a defined parameter at the first item’s
standardized value of a shared label. For four items with unstandardized
loadings of .7 and error variances of .1, .2, 1, and 2 (a covariance
matrix, N = 500), equal loadings gave a standardized AVE, H, and omega
of .6583, .9094, and .8730 and give .5166, .8899, and .7988; equal error
variances gave .3155, .6642, and .6523 and give .3816, .7710, and .7007.
lavaan’s ambiguity warning no longer reaches the caller. Other factors
and the "measurement" and "verbose" tables are
unchanged.
fa_bifactor(output = "standardized")
computes each index that uses a label several items share in the same
way. On holzinger_swineford with the three groups
of the example, equal loadings on spatial, and equal error
variances on memory, omega_general,
omega_HS_spatial, H_spatial, and
omega_memory were .9113, .1222, .1972, and .8995 and are
.8731, .2195, .4025, and .7233; with equal loadings on the general
factor, ECV_general and H_general were .0879
and .3627 and are .1443 and .5133, the values of the
"measurement" table.
fa_second_order(output = "standardized") no longer passes
on lavaan’s warning about the index rows it leaves out.
The tables of fa_confirmatory() (and of
cfa_1(), cfa_2(), and cfa_k())
withhold every standard error, test, and interval of a fit whose
expected information is singular, with the warning of class
dmar_result_qualified, as they did for a singular observed
information. A two-test factor uncorrelated in the sample with another
factor, fitted by listwise deletion with equal loadings on that factor,
had an H of .853 with a standard error of .028 and the interval [.790,
.900]; the standard errors and intervals are now
NA.
average_variance_extracted() refuses a
multiple-group or multilevel fit, with an error of class
dmar_argument_invalid that names the groups (as
groups) or the levels (as levels). DMAR 1.0.0
averaged the squared standardized loadings of every group, or of both
levels, into one row: a factor whose AVE is .810 in one group and .090
in the other (three items with equal correlations of .81 and .09, groups
of 1000 and 100) had an AVE of .450, which describes neither group. The
AVE of one group or level is
average_variance_extracted(loadings = ) given its
standardized loadings. bifactor_indices() refuses a
multiple-group fit the same way (see Calls That Are Now
Refused).
bifactor_indices() refuses a multilevel
fit with an error of class dmar_argument_invalid.
DMAR 1.0.0 read the loadings of the within level and the residual
variances of the between level: for nine items in three subscales with a
general factor g at both levels (200 clusters of 8,
set.seed(113)), the general factor’s omega was .942 and its
omega_H .740, where the within level’s own solution gives .885 and
.695.
average_variance_extracted(),
bifactor_indices(), and dmacs() refuse a fit
whose optimizer did not converge
(lavaan::lavInspect(fit, "converged") is
FALSE), with an error of class
dmar_not_converged that carries fn and
iterations. DMAR 1.0.0 read the point where the optimizer
stopped as a solution: a one-factor fit to three items whose
correlations are all .81 (N = 200), stopped after one iteration, had an
AVE of .412, where the converged fit has .810. A fit that converged to
an improper solution is treated as before.
The standardized cluster randomized planners
(ss_aipe_crd_es_*()) measure the width of each simulated
interval from its lower limit to its upper limit. The width was
the estimate minus the lower limit, about half the interval, so every
width these planners report is about twice the earlier one, and a plan
made for a width is larger. With G = 2, seed = 113,
ss_aipe_crd_es_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 0, individual_cost = 1, true_es = 0.5, es_type = 1, true_icc = 0.25, pr_treat = 0.5)
reports an expected_width of 0.672 where it reported 0.335;
with G = 12, seed = 113,
ss_aipe_crd_es_n_clusters_fixed_width(width = 0.40, cluster_size = 20, true_es = 0.5, true_icc = 0.10, pr_treat = 0.5)
returns 67 clusters where it returned 14, and
ss_aipe_crd_es_n_individuals_fixed_width(width = 0.60, n_clusters = 60, ...)
a cluster size of 5 where it returned 2.
The standardized cluster randomized planners use only the
intervals of converged fits, count a failed data set as an infinite
width under assurance, and bracket the target before they
bisect. The interval of a data set is used only when the
optimizer status code of the fit and of each limit’s search is 0 or 1,
the diagnostic OpenMx gives each limit is “success”, and the limits are
finite and bracket the estimate; any two limits that were numbers were
used before. The assured width is the assurance quantile of
all G widths, with the data sets without a usable interval
at infinity: with two of four data sets failed and the other two at 0.2,
the assured width at 0.8 was 0.2, and the call is now refused. The
searches upward for a size stepped up one cluster or one individual at a
time and now step up 1, 3, 7, 15, and so on, and then halve the gap.
When the simulated width falls as the size grows the design is the same
(the calls above still return 67 clusters and a cluster size of 5); with
a small G the simulated width rises and falls, and the two
searches can stop at different sizes near the target (with
G = 6, seed = 99,
ss_aipe_crd_es_n_clusters_fixed_width(width = 0.7, cluster_size = 2, true_es = 3, es_type = 1, true_icc = 0.2, pr_treat = 0.5)
returns 233 clusters after simulating 16 designs, where stepping one at
a time stopped at 203 after 107).
ss_aipe_crd_es_n_individuals_fixed_width()
with es_type = 1 plans targets it refused, refuses
unreachable ones before simulating, and starts its search
lower. With a fixed number of clusters the width levels off, at
a value with a closed form, as the clusters grow. The search started
from the cluster size the unstandardized difference needs, which uses
the t quantile and with few clusters needs far more: with 10
clusters and an intraclass correlation of 0.2 it needed more than
100,000 per cluster for every target from 1.19 to about 1.458, and 1,875
for 1.46, which about 10 reach with true_es = 0.5. It now
starts from the smaller of that size and the size at which the
closed-form width, with the within-cluster standard deviation known,
falls below the target; with equal cluster sizes, no covariate, and no
assurance that start is a lower bound on the size the target needs.
width = 1.30, n_clusters = 10, true_es = 0.5, true_icc = 0.2, pr_treat = 0.5, G = 20, seed = 5
returns a cluster size of 18, and width = 1.46
(seed = 11) returns 9. Targets whose search started at 500
or less are now searched upward from the smaller start. When the
simulated width falls as the cluster grows the design is the same; with
a small G it can differ, and far fewer designs are
simulated: with G = 20, seed = 5, width = 1.50
returns 8 after simulating 2 designs, where stepping down one size at a
time from 70, as DMAR 1.0.x searched, stops at 16 after 56. A target at
or below the level (1.187 there) is refused without a simulation (class
dmar_argument_invalid, reason
"width_not_reachable", the level as
width_limit), and so is one whose start is above 500 (from
1.187 to about 1.1914 there; reason
"cluster_size_beyond_reach").
The cluster randomized searches over the number of
clusters keep the floor of two clusters in each condition and return a
number they tested. The floor,
ceiling(2 / min(pr_treat, 1 - pr_treat)) + n_covariates
(MBESS: “at least two groups in each condition”), is where the search
starts when every number from 100 to 1000 meets the target. There the
search returned 3 + n_covariates, which it had not tested:
ss_aipe_crd_n_clusters_fixed_width(width = 100, cluster_size = 20, pr_treat = .1, true_var_between = .2, true_var_within = .8)
returned 3 clusters, none treated, with an expected_width
of Inf, in DMAR 1.0.0 and the development versions of
1.1.0, and returns 20 (1.534). Where the floor is above 100
(pr_treat below .02) the search stopped with “wrong sign in
‘by’ argument” (pr_treat = .01, now 200 clusters, 1.373) or
returned a number below the floor (pr_treat = .005: 101
clusters, one treated, now 400, 1.365). With unequal cluster sizes the
same call with diff_size = c(-2, 0, 2) returned 3 clusters
with a width of NaN in both, and returns 20. Every number
returned meets the target and is at least the floor. In a grid of 6,912
calls (pr_treat from .005 to .9, 18 widths from .12 to 8
that are not the width of a design, n_covariates 0 to 2,
assurance NULL and .8, equal sizes and two
diff_size patterns) the 3,989 results of the development
versions that met the target from the floor are unchanged; the 2,923
others change: 919 that stopped with an error, 1,693 returns of
3 + n_covariates whose width did not meet the target, and
311 that met it below the floor. With equal cluster sizes and no
assurance the width falls as clusters are added, and the
number returned is the first from the floor that meets the target. With
diff_size, or with an assurance below .5, the
width need not fall at every step, and the number returned can be larger
than the first that meets the target, as before:
ss_aipe_crd_n_clusters_fixed_width(width = 0.363312492117768, cluster_size = 50, pr_treat = .9, true_var_between = 1, true_var_within = 4, conf_level = .9, diff_size = c(.5, 1, 1.5))
returns 1002, although 999 meets the target (1000 and 1001 do not). In
3,000 calls at targets set from the widths of designs, 152 results were
larger than the first such number, 150 of them with
diff_size and 2 at assurance = .3. The two
both planners consider the same floor:
ss_aipe_crd_both_fixed_width(width = 3, cluster_cost = 1000, individual_cost = 1, pr_treat = .005, ...)
returned 101 clusters of 3 (budget 101,303, one cluster treated) and
returns 400 clusters of 2 (budget 400,800). The standardized planners
start their search over the number of clusters from this plan, so a
standardized plan that started from the untested
3 + n_covariates changes: with
G = 10, seed = 113,
ss_aipe_crd_es_n_clusters_fixed_width(width = 6, cluster_size = 10, true_es = .5, true_icc = .2, pr_treat = .5)
returned 3 clusters (one of them a control cluster) and returns 4; at
pr_treat = .3 it returned 3 (one treated) and returns 5.
DMAR 1.0.0 also returned 3 in both. Eleven standardized calls whose
start was not the untested one, among them the calls the entries on the
standardized planners cite (67 and 233 clusters, and a cluster size of
18), are unchanged.
A target equal to the width of a cluster randomized
design is met by that design. A width meets the target when it
is no larger than the target, inside a bracket of the search as at its
ends (see “Bug Fixes”). DMAR 1.0.0 and the development versions of 1.1.0
asked for a width below the target inside a bracket, so the width a plan
reported, given back as width, gave the next size:
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8)
plans 18 clusters with an expected width of 0.979142316129524, and that
width as the target gave 19 clusters and gives 18;
ss_aipe_crd_n_individuals_fixed_width(width = 1, n_clusters = 30, ...)
plans a cluster size of 4 (0.946118731780349), which gave 5 and gives 4;
and the first call with assurance = .8 plans 22 clusters
(0.975082804893471), which gave 23 and gives 22. Only a target equal to
a width the search computes is affected: in 3,000 calls of
ss_aipe_crd_n_clusters_fixed_width() at targets set to the
width of a design and a part in 1e9 above or below it, 741 of the 2,536
results of the development versions that met the target from the floor
change, every one at a target equal to a design’s width (in 672 the
number they returned was one more), and none at the others; in 500 calls
of ss_aipe_crd_n_individuals_fixed_width() at targets set
from the widths of designs 100 plans change (28 of them with equal
cluster sizes), all at a target equal to a design’s width, and 24 that
stopped with “‘from’ must be a finite number” return a size. A
standardized plan starts from these searches, so its start can move by
one where the target equals the unstandardized width of a design. The
budget searches already counted a cost equal to the budget as within it
inside a bracket.
A between-cluster proportion of variance explained no
longer shrinks a cluster randomized plan without a covariate.
With n_covariates = 0 the six unstandardized planners took
true_R2_between out of the between-cluster variance. The
six standardized planners started their search from that smaller plan,
while the data they simulated left it out. DMAR 1.0.0
(R2_between with num_predictors = 0) and the
development versions of 1.1.0 gave these plans with
true_R2_between = 0 and .9:
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8)
18 clusters (expected width 0.979) and 7 (0.962);
ss_aipe_crd_n_individuals_fixed_width(width = 1, n_clusters = 20, ...)
a cluster size of 10 (0.994) and 4 (0.881); and
ss_aipe_crd_both_fixed_width(width = 1, cluster_cost = 10, individual_cost = 1, ...)
22 clusters of 7 (budget 374) and 8 clusters of 13 (budget 184). In the
development versions
ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, G = 20, seed = 113)
planned 18 clusters (0.982) with either value. Its search started at 22
or at 8 and simulated 6 or 8 designs. The calls with .9 are
now refused (see Calls That Are Now Refused). In 216 calls of the six
unstandardized planners (true_R2_between 0, .3, and .9,
true_R2_within 0 and .5, n_covariates 0 to 2,
pr_treat .3 and .5), the 48 with a nonzero
true_R2_between and n_covariates = 0 are
refused and the other 168 are unchanged.
mediation_mbco() statistics change where the
fits were wrong, and some calls of it and of
ss_power_reg_coef() and ss_power_rc() that
returned a result are refused (see Bug Fixes for the calls and
values).
ss_power_reg_coef(true_cor_XX = diag(2), true_cor_YX = c(.3, .2), which_predictor = 1.5)
returned necessary_N = 89, the plan for the first
predictor, and is refused.
tidy() and results_sentence()
read the intervals of ancova(),
factorial_anova(), variance_components_mls(),
mediate(), R2_mixed_effects(),
nnt_from_smd(), cles(),
meta_es(), meta_r(), and
meta_smd(), whose limits are now named
<estimate>_ci_lower and
<estimate>_ci_upper (see Renamed Output Rows
and Columns). The values are unchanged. tidy() of
ancova(), variance_components_mls(),
mediate(), nnt_from_smd(), and
cles() has a third column, conf_level, on the
rows of each interval (DMAR 1.0.x returned term and
estimate alone), and in tidy() of the
meta-analysis functions that column also marks tau2 and
I2 and their limits. results_sentence() on
these tables stopped (“this table has none of these”); it now writes the
sentence of the estimate term names or, by default, of the
first estimate with limits (for ancova() of the Pygmalion
example, “eta_squared_partial = 0.02, 95% CI [0.00, 0.06]”), and on one
row of factorial_anova() with
term = "eta_squared_partial" it writes that row’s sentence.
A table that records no level (nnt_from_smd() with supplied
limits and no conf_level) is refused with the message that
asks for conf_level.
cles(), nnt_from_smd(), and
proportion_of_superiority() still report their own
interval. Now that the limits on d are named for
d, each table holds two intervals named for their estimates,
and a new attribute, ci_source, names d as the
estimate whose interval the function’s own is computed from.
results_sentence() reports the common language effect size,
the SRD, and the proportion of superiority, and tidy()
gives NA as the confidence level on the rows of d,
as in DMAR 1.0.x; results_sentence(x, term = "smd") now
gives the interval for d, which DMAR 1.0.x refused.
The renames themselves change no value. A call that gives the new names returns what the DMAR 1.0.x call returned, except where another entry of this release changes the result; renamed rows and columns hold the values they held under their 1.0.x names.
mr_cv() estimates the variance of the sample
coefficient of variation by Equation 32 of Kelley, Darku, and
Chattopadhyay (2018), so its results differ from those of Chattopadhyay
and Kelley (2016) and of MBESS for nearly any data. DMAR 1.0.x
used Equation 22 of the 2016 article, as MBESS::mr.cv()
does. Written with the skewness gamma_1 and the excess kurtosis gamma_2
of the scores, that equation estimates kappa^2 / 2 + kappa^4 - kappa^3
gamma_1 + kappa^4 gamma_2 / 4, the first-order term of Bao (2009,
Equation 3) as printed, and the kurtosis term should be kappa^2 gamma_2
/ 4 (Equation 30 of the 2018 article, and the erratum Bao posted for his
Equation 3, which ?mr_cv describes). The two population
values are equal only when the excess kurtosis is 0 or the coefficient
of variation is 1; for gamma scores with a coefficient of variation of
.8 the value of Equation 22 is 0.304 and the corrected value is 0.525.
In a sample the two estimates differ by (k^2 / 4)(1 - k^2)(mu4_hat / s^4
- 3), so the stopping point changes with normal scores too: in 4,000
simulated runs with A = 5,000,000, c = 10, and a coefficient of
variation of .3 it changed in 95 percent of the runs, by 8 observations
on average, while the mean final sample size changed little (172 with
Equation 22, 170 now). For scores with positive excess kurtosis and a
coefficient of variation below 1 the population value of Equation 22 is
the smaller of the two, and with strongly skewed scores DMAR 1.0.x could
stop well short of the large-sample value sqrt(A / c) xi. In 20,000
simulated runs with the same A and c, the mean final sample size for
gamma scores with a coefficient of variation of .8 was 375 with Equation
22 and is 480 now, where the large-sample value is 512; for lognormal
scores with a coefficient of variation of .6 it was 220 and is 440,
where that value is 545. For the article’s example (35 observations, A =
400,000, and a sampling cost of 75) DMAR 1.0.x reported a risk of
4891.284 and a stopping rule that was met, as the article does.
mr_cv() now reports a risk of 4977.259 and a criterion of
1261.332 against N^2 = 1225: the rule is not met, and another
observation is collected. ?mr_cv has a part on Equation
22.
mr_cv() does not stop sampling on a zero or
negative variance estimate. The estimate can be zero or
negative in a small sample, and DMAR 1.0.x put it into the stopping rule
as it was, which lowered the criterion.
mr_cv(data = c(2.3, 0.61, 1.6, 0.54, 0.51, 0.41, 0.67, 0.37), A = 485, sampling_cost = 1),
a pilot sample of the size the function plans for those costs, gave an
estimate of -0.058, a risk of 4.468 (less than the 8 already spent on
sampling), and a rule that was met; 18 equal scores gave an estimate of
0 and a rule that was met. Such an estimate now means that the rule is
not met: is_satisfied is FALSE, the risk and
the criterion are NA, the new attribute
variance_unusable is TRUE, and a line under
the table says to collect another observation. Scores that differ only
by rounding error (a range of at most 1e-10 times the largest absolute
score), such as equal gains computed as posttest minus pretest, and an
estimate within 1e-10 of the size of its four terms are taken as an
estimate of 0. A positive estimate enters the rule and the risk as the
larger of itself and n^-3 (Equation 31 of the 2018 article). The
estimate is usually below n^-3 in a small sample of scores with a small
coefficient of variation: for normal scores, while n is below about (2 /
kappa2)(1/3), which is 27 for a coefficient of variation of
.01 and 9 for .05. From the pilot sample size on, n^-3 adds less than 1
to the criterion, so it changes the decision only at the pilot sample
size and only when N^2 there is within 1 of the criterion, and the stop
then usually comes one observation later. With
verbose = TRUE the row v2_n holds the estimate
itself in every case. The third and fourth central moments are computed
from centered scores. The row mu_4_n, which DMAR 1.0.x
computed from raw power sums, lost its digits to rounding for scores far
from zero: for z + 100000 after
set.seed(113); z <- rnorm(30) it was 3060.977 on the
computer used here, where 2.531 is right.
mr_cv() and mr_smd() print
their decision. Whether the stopping rule was met was in the
attribute is_satisfied only; the printed table showed the
risk, the sample size, and the estimate. A line under the table now
gives the decision, the two numbers compared, and what to do next:
“Stopping rule: not met (N^2 = 1225 is below the criterion, 1261.332);
collect another observation.” from mr_cv(), for example,
and “Stopping rule: met (the sample size per group, 75, is at least the
criterion, 71.054); sampling can stop.” from
mr_smd(d = .5, n_per_group = 75, A = 10000, sampling_cost = 2.4).
The tables of both functions have the class dmar_mr in
front of dmar_tbl, and no row or attribute of DMAR 1.0.x is
removed.
var_cv() returns one row,
var_cv_normal. It holds the first-order variance
of the sample coefficient of variation under normality, kappa^2 (1/2 +
kappa^2) / (N - 1), which DMAR 1.0.x returned in the row
var_cv_mckay; that number is unchanged. The second row of
1.0.x, var_cv_vangel, is removed. It was kappa^2 (1/2 +
kappa^2 (N + 1)/(N - 1)) / (N - 1), which Vangel (1996) does not state
and for which no source was found, and it is the less accurate of the
two for a population coefficient of .40 or less with 10 to 500 cases.
Against the variance of the sample coefficient under normality, computed
with the sample mean kept away from zero, it is 4.9 percent too large at
a coefficient of .30 with 10 cases, where the kept value is 1.4 percent
too large; at .50 with 10 cases it is 1.3 percent too large, where the
kept value is 5.7 percent too small. Code that reads a row by its 1.0.x
name gets numeric(0), and x$value[2] is
NA.
ci_pvaf() reports the stated coverage when
its lower limit is 0. The actual_coverage row is
1 - alpha_lower - alpha_upper for every interval the
function reports. In DMAR 1.0.0, when the observed F was below
the alpha_lower critical value (an F test not
significant at alpha_lower), so that the lower limit was 0,
the row was 1 minus the p-value of the F test minus
alpha_upper:
ci_pvaf(F_value = 1.5, df_1 = 4, df_2 = 50, N = 55)
returned 0.759 there for a 95 percent interval. The interval from 0 to
the upper limit has the stated coverage, because a lower limit of 0
cannot lie above the population value. In 20,000 simulated analyses of 5
groups of 11 at each of five population values from 0 to .20, the 95
percent interval contained the population value in 95.0 to 95.1 percent
of them, and its lower limit was 0 in 27 to 98 percent of them. An
analysis whose F was so small that no upper limit exists has no
interval and was counted as not containing the population value (2.5
percent of the analyses at a population value of 0). The limits, the
point estimate, and the tail columns are unchanged; the help says what
the tail columns hold when the lower limit is 0.
A half-width plan of ss_aipe_R2() holds at
every larger sample size. With
which_width = "Lower" or "Upper" the function
plans for the distance from the expected sample squared multiple
correlation to one confidence limit, and it returns the smallest
N such that the distance is no larger than width
at that N and at every larger one. DMAR 1.0.0 searched upward
from N = 2p + 3 and returned the first sample size at
which the distance was no larger than width. With
"Upper", larger sample sizes can miss that target: with
several predictors and a small N the expected sample value is
far above the population value, the upper limit is close to that
expected value or below it, and the distance is small or negative; it
rises to a maximum as N grows and then falls.
ss_aipe_R2(true_R2 = .70, width = .05, p = 10, which_width = "Upper")
(population_R2 in 1.0.x) returned 23, where the distance is
.043, and the distance is larger than .05 at every N from 27 to
201; the call now returns 202. These calls with
which_width = "Upper" and a 95 percent interval change:
true_R2 |
p |
width |
random_predictors |
DMAR 1.0.0 | now |
|---|---|---|---|---|---|
| .50 | 5 | .15 | TRUE |
13 | 33 |
| .70 | 5 | .10 | TRUE |
13 | 36 |
| .70 | 10 | .05 | TRUE |
23 | 202 |
| .10 | 10 | .05 | FALSE |
23 | 216 |
| .30 | 5 | .10 | FALSE |
13 | 107 |
| .30 | 10 | .05 | FALSE |
23 | 449 |
| .50 | 5 | .10 | FALSE |
13 | 44 |
| .50 | 10 | .05 | FALSE |
23 | 279 |
| .70 | 5 | .05 | FALSE |
13 | 123 |
They are the 9 that change among 72 "Upper" plans
(true_R2 of .10, .30, .50, and .70; 3, 5, and 10
predictors; a width of .05, .10, and .15; random and fixed
predictors), and none of the 72 "Lower" plans for the same
conditions changes. Among 3,456 plans (true_R2 from .05 to
.90; 1 to 20 predictors; a width from .03 to .30; 90, 95,
and 99 percent) 133 change. For 127 "Upper" plans DMAR
1.0.0 returned 2p + 3 and the plan is now larger by 2 to 2,889;
105 of them have 8 or more predictors. One "Upper" plan
with random predictors is 4 larger (true_R2 = .30,
p = 15, width = .03,
conf_level = .99: 3,691 where DMAR 1.0.0 returned 3,687),
because the limits for random predictors are computed to a tolerance and
the distance at 3,690 is above .03 by less than .000001. Five
"Lower" plans with random predictors are 1 smaller: DMAR
1.0.0 began its search for random predictors at the plan for fixed
predictors, and in these the distance with random predictors is within
width one sample size earlier
(ss_aipe_R2(true_R2 = .10, width = .10, p = 3, conf_level = .90, which_width = "Lower")
returned 103 and now returns 102). The other 3,323 plans are unchanged,
and so is every plan for the full width. With a half-width the search
first bounds the sample size, for which it needs a sample size, of those
it tries up to 10,000,000, at which the full width of the interval at
the expected sample value and the distances from that value to its two
limits are all within width. A call for which it finds none
is refused (class dmar_argument_invalid,
arg = "width"). The refusal does not mean that no sample
size meets width: the bound is about four times the sample
size a half-width needs, and there the noncentrality parameter is in the
millions, beyond the range in which R’s noncentral F and beta
distribution functions are accurate (?pf says its code for
a noncentrality parameter is meant for moderate values). With fixed
predictors, true_R2 = .98, 5 predictors, and 95 percent,
the full width of the interval at the expected sample value is .00068 at
a sample size of 26,624 and .00117 at 53,248, where it should be
smaller, and pf() gives no warning. The message says that
the width is narrower than can be planned with R’s
distribution functions. It was found only with a very small
width: of 600 calls with a width of .002 to
.005 (random and fixed predictors; true_R2 from .10 to .90;
1 to 20 predictors; 90, 95, and 99 percent), 17 of the 120 with .002 and
4 of the 120 with .0025 are refused, and none with .003 or more. DMAR
1.0.0, which stepped through the sample sizes one at a time, returned a
plan for two such calls after a wait:
ss_aipe_R2(true_R2 = .30, width = .0015, p = 5, which_width = "Upper", random_predictors = FALSE)
returned 848,292 after about ten minutes, and with
true_R2 = .98 and width = .0005 it returned
11,351 after about ten seconds. Both calls are now refused.
ci_R2() and ci_R() with random
predictors report an upper limit that does not exist as NA,
with a warning. When the observed R2 is at or below the
alpha_upper quantile of its distribution at a population
value of 0, no population value is consistent with the sample at that
level. DMAR 1.0.0 reported an upper limit of 0 there, with the requested
tail areas and no warning: ci_R2(R2 = .003, N = 50, p = 3)
gave an interval from 0 to 0 around an observed .003, which is below the
.025 quantile, .0046. The ci_upper row is now
NA in its value and in both tail columns, with a warning of
class dmar_result_qualified, as it already was with fixed
predictors. ss_aipe_R2_sensitivity() with random predictors
counts such a replication among those without an interval, as it does
with fixed predictors; in DMAR 1.0.0 it entered the width rows as an
interval of width 0.
mr_cv() and mr_smd() give a
pilot sample size that is a whole number in exact arithmetic as that
number. A sample size is a whole number, so the formula of the
pilot sample size is rounded up, and R can compute a whole number a
little above itself:
mr_cv(pilot = TRUE, A = 1e5, sampling_cost = 1, gamma = .25)
and
mr_smd(pilot = TRUE, A = 1e6, sampling_cost = 5, gamma = .25)
returned 101 in DMAR 1.0.0, where the formula gives exactly 100
(100,000^0.4, which R computes as 100.00000000000003). Both now
return
100. Pilot sample sizes whose formula is not a whole number are unchanged.
The two options of expected_r() are named in
lower case, dmar.expected_r.tol and
dmar.expected_r.max_iter, like
dmar.digits and dmar.bootstrap_exploratory.
DMAR 1.0.0 read DMAR.expected_r.tol and
DMAR.expected_r.max_iter; those names are no longer read,
so a value set under them is ignored and the default is used.
expected_partial_r() reads the same two options.
Never submitted to CRAN: these changes were released in DMAR 1.1.0. The section is kept as the record of the patch as it was prepared.
A patch release for the bootstrap intervals of the reliability family, prompted by a report from an analysis of the Big Five Inventory constructs (2026-10-14), in which every BCa interval for categorical omega failed with “estimated adjustment ‘a’ is NA” and no interval said how many replications it rested on.
Every bootstrap interval in the reliability family now
says how many replications it conditions on.
reliability_alpha(), reliability_kr20(),
reliability_omega(),
reliability_omega_categorical(), and
reliability() through them record, in a
bootstrap attribute, how many replications were attempted,
how many returned a finite value, how many were set aside and why (an
error from the refit, or no finite value, as from a fit that did not
converge), and the distinct error messages with their counts. The
standard error is the standard deviation of the returned replicates and
the percentile and BCa limits are their quantiles, which the help pages
now state; the printed table shows the counts in one line, and a single
warning states them whenever a replication was set aside. Through 1.0.0
a failed replication was dropped silently. No existing row or attribute
changes.
The BCa interval no longer fails when a jackknife refit
fails. With B below the number of cases, boot
estimates the BCa acceleration from jackknife influence values, and one
leave-one-out refit that returned no value made the whole vector
NA and stopped the call. The influence values are now
computed in the package under boot’s own rule (regression on the
resampling frequencies when at least as many replications as cases
returned a value, the jackknife otherwise), the leave-one-out refits
that fail are counted and set aside, and the acceleration is estimated
from the rest; the jackknife counts and the acceleration join the
bootstrap attribute. When the acceleration or the bias
correction cannot be estimated at all, the percentile interval is
returned in its place with a warning, and the ci_method
attribute reads
"percentile (BCa acceleration could not be estimated: k jackknife replicates failed)",
so the substitution is on the record. On a run where nothing fails the
interval and the acceleration equal boot’s to machine precision, in both
regimes, so no interval that 1.0.0 computed changes.
reliability_omega() states which intervals
apply to which denominator. The closed forms, the Wald
intervals, and the profile likelihood are derived under the model
implied ratio and apply to denominator = "model_implied"
only; the bootstrap methods and "none" apply to both. The
help page now says so in one place, and a request for a combination that
does not apply, such as ci_method = "mlr" with the default
observed denominator, stops with a message naming the denominator that
does.
First public release of DMAR (pronounced “Dee-Mar,” for “Design, Measurement, and Analysis in R”), a greatly expanded reimagining of the MBESS package, published on CRAN on 2026-09-21.
Every line of code in every help page example now runs. Earlier
drafts carried a slow call (a bootstrap interval, a Monte Carlo planner,
a refit) as a comment so that a reader could see the syntax without the
check paying for it; the CRAN review of 2026-09-04 asked that no example
line be commented out, and none is. Where a call was slow, it runs with
a small replication count and a comment naming the count a reported
analysis deserves (the defaults are unchanged); where a page
demonstrated the same thing twice, it now demonstrates it once. The
package still ships no \donttest{} and no
\dontrun{}, and a parse-based detector in the test suite
fails on any comment line that would parse as code.
The sensitivity family no longer writes a file unless asked to,
and never to a default location. The save argument is gone
from <filename,
which defaults to NULL. Supplying a path writes the
per-replication results there exactly as save = TRUE used
to; a throwaway run points it at
tempfile(fileext = ".csv").
A supplied seed is now set through
withr::local_seed() (withr, which has no dependencies of
its own, joins Imports), by way of one internal helper that every seeded
function calls. The behavior is what it was, the user’s generator state
is restored when the function returns, but the package no longer touches
.Random.seed or the global environment to do it, and no
function uses <<-: the handlers that count clamped or
non-converged replications keep their counts in a local
environment.
The six Monte Carlo functions that silenced every warning for
their duration with options(warn = -1) now muffle only the
warnings their loops are known to raise (lavaan convergence and variance
warnings on borderline replicates, the noncentral clamp); any other
warning reaches the user.
The Bryant-Paulson distribution function integrates over the
covariate shrinkage factor after the change of variables delta = 1 -
u^2, under which the Beta weight of Bryant and Paulson (1976, Equation
12) is smooth on the unit interval for every number of covariates
instead of carrying an integrable singularity at the endpoint where the
distribution puts most of its mass. The two forms agree to about 1e-10
(checked on a grid of 288 quantile, covariate, group, and degrees of
freedom combinations), the Table 1 reproduction in the tests is
unchanged, and a critical value now costs about a twentieth of what it
did, which is what lets every qbryant_paulson(),
cv_bryant_paulson(), and ci_c_ancova_bp()
example run live.
Two smaller changes from the same pass. mlmr(),
mlmr_mv(), and average_variance_extracted()
resample through lavaan::lavBootstrap(), the entry point
that replaced bootstrapLavaan() in lavaan 0.7-2 (the
package already requires that version); the results are identical. The
five sensitivity functions whose ... was never forwarded
anywhere (ss_aipe_sc_sensitivity(),
ss_aipe_smd_sensitivity(),
ss_aipe_sm_sensitivity(),
ss_aipe_R2_sensitivity(), and
ss_power_R2_sensitivity()) no longer accept it, so a
misspelled or retired argument such as save = TRUE now
stops with an error instead of being swallowed.
ss_aipe_cv() with assurance supplied reports
its noncentral accuracy note once, with the combined count, where it
used to report it twice; and ss_aipe_cv_sensitivity() and
ss_aipe_sc_ancova_sensitivity() report that note once per
run, with the number of replications it arose in, rather than once per
replication.
A package-wide audit (numerical verification against complex-step
derivatives and Monte Carlo, documentation contracts run rather than
read, and mechanical convention gates) closed out with these fixes.
analysis_of_change() now fits an intercept-only polynomial
(order = 0) under both methods; it previously stopped
inside stats::poly(), and its no-fit message for the
polynomial now counts occasions instead of suggesting
start, which the polynomial rejects.
The equivalence sensitivity siblings speak their parents’
language: ss_aipe_equivalence_smd_sensitivity() takes
delta_lower / delta_upper and
ss_aipe_equivalence_r_sensitivity() takes
rho_lower / rho_upper, positive magnitudes
with the upper bound required and the lower defaulting to symmetric,
exactly as in equivalence_smd() and
equivalence_r(). The signed equivalence_lower
/ equivalence_upper spellings, and their silent default of
plus or minus 0.20, are gone.
Every AIPE planner that reports an expected interval width names
the row ci_width_expected and reports the expected full
width: ss_aipe_equivalence_smd() and
ss_aipe_mixed_effects() previously reported the half-width
as ci_half_width_expected while
ss_aipe_equivalence_r() reported the full width, a trap for
anyone extracting the row programmatically.
The equivalence trio attaches the conf_level
attribute the ci_* family always carried, and the
equivalence_c() verdict label is spelled
"Noninferior only", matching the solid
noninferior the rest of its page uses.
equivalence_r() now validates raw data (matching lengths,
at least 4 complete pairs) the way its summary-statistics path always
did. convert_r_Z() and convert_Z_r() insist on
a single value, as documented; vector input previously recycled the term
column into duplicated rows. A follow-up ruling extended the same
single-value guard to the whole scalar conversion family
(convert_R2_f() and its three siblings,
convert_delta_lambda() and its inverse, and
convert_z_normal()), with domain checks where a map’s
algebra ends (an R2 at or past 1, a nonpositive group
size), so no conversion can recycle vector input into duplicated
rows.
Documentation corrections from the same audit: the four nonlinear
simulators state that the first-order delta method behind
reliability and reliability_by_occasion can
drift from the realized variance ratio when random variances are large,
and document the schedule attribute; the Richards example
comment places the delta = 3 inflection at about 63% of total change,
per its own formula; the mixed-effects examples on
?analysis_of_change now run live.
The four multiple-comparison interval functions moved into the
ci_* family: ci_dunnett,
ci_scheffe, ci_games_howell, and
ci_tukey_kramer replace dunnett_ci,
scheffe_ci, games_howell_ci, and
tukey_kramer_ci. The suffix forms were the only four
exports naming a confidence interval outside the family prefix, and each
function now pairs with its critical-value sibling
(cv_dunnett, cv_scheffe,
cv_tukey_hsd). The package is unreleased, so the old names
are gone rather than aliased.
The noncentral-distribution toolkit is named by one rule
(2026-08-21): nc_ and then the distribution, so
ci_nc_t(), ci_nc_F(), and
ci_nc_chisq() replace conf_limits_nct(),
conf_limits_ncf(), and conf_limits_nc_chisq()
(the snake-cased descendants of MBESS’s conf.limits.nct()
and kin), and moments_nc_t() and
moments_nc_F() replace moments_nct() and
moments_ncf() to match moments_nc_chisq(). The
intervals join the ci_* family because a confidence
interval, on the noncentrality parameter that the effect size intervals
are built from, is what they compute, and “confidence limits” was
vocabulary that existed only in MBESS. Splitting nc from
the distribution token retires the one inherited exception
(nct, ncf against nc_chisq) and
restores the capital the package writes everywhere else
(F_value, convert_F_chisq). The noncentral
marker stays because the noncentrality parameter is the estimand.
plot_ci() keeps its name: it is a plot first. The old names
are gone rather than aliased; each page names its earlier self in the
description.
The correlation intervals now carry the names of their estimands,
matching the rest of the correlation family (ss_aipe_r,
ss_power_r, var_r, expected_r,
equivalence_r): ci_r() is the confidence
interval for the simple Pearson correlation (renamed from the
MBESS-heritage ci_cc(), with the estimate row renamed from
est_cor to r), and ci_R() is the
interval for the population multiple correlation coefficient (its former
lowercase alias spelling is gone). Cross-references across the package
were audited against this distinction and every one now points at the
interval it meant. Because the two natural file names differ only by
case, which case-insensitive filesystems refuse, both functions live in
R/ci_correlation.R and share the
ci_correlation help page.
Kish’s design effect function is design_effect();
the abbreviation deft() is gone as a function name, while
the returned table keeps its design_effect and
deft rows, deft being the standard term for the standard
error inflation factor, the square root of the design effect.
The remaining exported alias pairs were resolved to single names.
Fisher’s Z is written with a capital Z throughout the package, because
it is the variance-stabilizing transform of a correlation and not a
z-score, so the converts are convert_r_Z() and
convert_Z_r() only (the lowercase spellings are gone) and
the prose and math were swept to match. The limits of agreement function
is limits_of_agreement(), with loa() kept as
its short alias and bland_altman_loa() gone (the method is
named for what it computes; Bland and Altman are credited in the
references). reliability_omega_categorical() and
covmat_from_cfa() are the only names for those functions;
the reliability_omega_c() and
covmat_from_cfm() aliases are gone. The dataset alias
bindings HS_Data and Prime_Time are gone as
well: every dataset carries exactly one name, the documented snake_case
one, and unlike the alias bindings the canonical names work with
data().
The vocabulary of the longitudinal simulators names what the rows
are: the identifier column identifies units (persons, animals, trees,
classrooms), n counts units, and the former
group column is population, one level per data
generating parameter vector.
Measurement schedules can now be unit-specific, in every
simulator: in place of a shared target_times grid,
time_range = c(lower, upper) draws each unit’s own
measurement times uniformly between the bounds, with
occasions fixing the number of times per unit or
c(min, max) letting it vary, so designs such as age at
testing rather than grade at testing are simulated directly. The
polynomial simulator additionally requires every unit’s count of
occasions to reach P + 1, so each simulated trajectory
identifies the polynomial it came from.
analysis_of_change() closes the loop the simulators
open: it fits any of the four nonlinear change models, or a polynomial
of any order, to longitudinal data. The default two-stage method fits
one curve per unit, from that unit’s data alone, with data-driven
starting values, and summarizes the unit-level parameters by their mean,
their standard deviation and variance across units (the individual
differences), and the standard error of the mean;
method = "mixed" instead estimates the proper
random-coefficients model simultaneously, lme4::lmer() for
the polynomial and nlme::nlme() for the nonlinear curves
started at the two-stage estimates, so sd_units becomes a
variance component purged of estimation noise. The help page positions
the function candidly against nlme::lmList(),
nlme::nlsList(), and the self-starting SS*
curves: what it adds is the landmark parameterizations, the simulator
match, and one tidy interface across linear and nonlinear change. With a
single trajectory (or id = NULL) it reduces to one
nonlinear least squares fit reported with asymptotic standard errors, so
N = 1 is one unit’s change. Non-converging units are dropped with a
single counted warning and the effective count travels as the
n_used attribute; the full matrix of unit-level estimates
rides along for plotting or as starting values for a simultaneous
nlme::nlme() fit, and the help page states plainly that the
between-unit spread of estimates includes estimation noise, which a
variance-component model separates.
simulate_longitudinal_polynomial(), sharing its interface
(several populations of units, between-unit parameter variances and
correlations, level-one error by variance or by target reliability,
error correlation structures, assessment-time jitter) and its
long-format return that feeds plot_trajectories() and
nonlinear mixed-model fitters:
simulate_longitudinal_negative_exponential() (asymptotic
regression), simulate_longitudinal_logistic(),
simulate_longitudinal_gompertz(), and
simulate_longitudinal_richards(), whose shape parameter
delta subsumes the logistic (delta = 1) and the Gompertz (delta -> 0)
as special cases, relations the tests pin exactly. The parameterizations
are those of Kelley (2005, dissertation; 2008, Methodology), in which
every parameter is a landmark of the change process (floor, ceiling,
moment of fastest change, curvature, inflection height) and the
intercept-shifting zeta frees the lower asymptote from zero; the
Richards application follows Guo, Cheng, and Kelley (2016). A new
vignette, “Nonlinear Growth Curves and the Meaning of Their Parameters”,
illustrates all four curves, the Richards unification, and the
polynomial comparison: a ninth-order polynomial needs ten
uninterpretable coefficients to track a four parameter Gompertz inside
the data and still collapses the moment it extrapolates.equivalence_smd(), equivalence_r(), and
equivalence_c() replace tost_smd(),
tost_r(), and tost_c(), and the AIPE planners
follow (ss_aipe_equivalence_smd(),
ss_aipe_equivalence_r(), with their sensitivity siblings).
The whole family now speaks the same word as the planning and plotting
surfaces that always did (power_equivalence_md(),
ss_power_equivalence_c(), plot_equivalence()).
The procedure is unchanged, Schuirmann’s two one-sided tests, and each
help page says so by that name; the reported TOST -value keeps its
standard p_tost term. The old names are gone rather than
aliased.ss_aipe_equivalence_r() plans the sample size for an
equivalence question about a Pearson correlation: the smallest whose
100(1 - 2 alpha)% Fisher’s interval (the interval
equivalence_r() and ci_r() invert) has
expected width at or below the target, with an optional Monte Carlo
assurance correction. At the conservative default planning value of 0
the answer has a closed form, and the tests anchor the search to it. Its
Monte Carlo sibling ss_aipe_equivalence_r_sensitivity()
follows the family API and reports, alongside the family’s width and
coverage summaries, the realized proportion of equivalence verdicts
inside the chosen bounds. This fills the gap noted when the equivalence
family was reviewed: the SMD had an AIPE planner for its equivalence
interval and the correlation did not.depression_bdi with its wait list control, the studentized
range procedures on test_market’s six panels, the
unequal-variance and rank methods on drinks_trial’s skewed
outcome, the factorial and generalized eta squared examples on
pygmalion’s manipulated treatment crossed with measured
grade, and the correlation, regression, reliability, and multivariate
examples on the holzinger_swineford battery, whose
second-form tests carry real missingness that now powers the FIML
demonstrations in mlmr() and mlmr_mv() in
place of artificially punched holes. Examples that fit lme4 models keep
lme4::sleepstudy, since those pages require lme4
regardless. The ecvi() example and tests likewise moved
from lavaan’s copy of the 1939 data to the package’s own
holzinger_swineford. Numeric claims in example comments
were recomputed against the new output throughout.Language: en-US and inst/WORDLIST carries the
package’s technical vocabulary, so devtools::spell_check()
runs clean.plot_equivalence() no longer passes the
fatten argument of ggplot2::geom_pointrange(),
which ggplot2 4.0.0 deprecates. The point size is set through the
size aesthetic at the value that draws the same figure on
every supported ggplot2 (>= 3.4.0), so building the plot under
ggplot2 4.0 no longer raises a deprecation warning.reliability_omega() and
reliability_omega_categorical() now validate their inputs
before the courtesy message that a default call reports no confidence
interval, so a call that is about to fail with an informative error no
longer receives advice first.Config/testthat/parallel: true, with the slowest files
scheduled first through Config/testthat/start-first; the
testthat floor moves to 3.2.0). The heaviest Monte Carlo checks were
recalibrated to smaller replication counts whose assertions still hold
with room to spare, duplicated planner calls across neighboring tests
were consolidated into single calls, and tests no longer leak messages,
notes, or printed tables into the run’s output. A full
devtools::test() on the maintainer’s machine dropped from
about 20 minutes to about 5, with no warnings and nothing skipped.ss_aipe_*_sensitivity() member now returns the
family’s documented schema: mean_ci_width /
median_ci_width / sd_ci_width for the realized
interval widths, pct_ci_less_w for the proportion of
intervals at or below the planning width, pct_ci_miss_low /
pct_ci_miss_high / total_type_I_error for the
empirical non-coverage (proportions on the 0 to 1 scale, so the total is
the sum of its tails), mean_X / median_X /
sd_X for the member’s own estimand, and input echoes named
for their unit (total_N or n_per_group for the
evaluated size, true_X, estimated_X,
width, conf_level, and, when one was supplied,
assurance). The core term vector lives once, as the
internal registry constant .SS_AIPE_SENS_CORE_TERMS in
R/dmar_tidiers.R, and a family-wide contract test asserts
every member against it. Every member’s @return now lists
exactly the rows the function returns.ss_aipe_R2_sensitivity() drops the lowercase
_r2 suffixes and pct_less_w
(mean_r2 is mean_R2,
mean_ci_width_r2 is mean_ci_width, the
realized-limit rows are mean_lower_limit and siblings, the
one-sided widths are mean_ci_width_lower /
mean_ci_width_upper); the four term-matching reads inside
ss_aipe_R2()’s assurance search were updated with it.
ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity(), and
ss_aipe_sm_sensitivity() retire
mean_full_width and pct_Width_obs_narrower /
pct_width_obs_narrower;
ss_aipe_smd_sensitivity() retires
pct_less_desired;
ss_aipe_c_ancova_sensitivity() retires
width_narrower and mean_width_obs (its
standard error comparison is now mean_se_ratio);
ss_aipe_rmsea_sensitivity() retires rmsea_pop,
desired_width, mean_width, and, most
importantly, an output row named assurance that actually
reported the realized width-attainment proportion, now
pct_ci_less_w so the name no longer collides with the
planning input; ss_aipe_sem_path_sensitivity() retires
width_less_than_desired and the type_I_err*
trio.ss_aipe_sm_sensitivity(),
ss_aipe_smd_sensitivity(),
ss_aipe_sc_ancova_sensitivity() (both divisor branches),
and ss_aipe_c_ancova_sensitivity() (which also scaled its
width-attainment row). All are now proportions on the 0 to 1 scale,
completing the sweep that earlier fixed
ss_aipe_sc_sensitivity(),
ss_aipe_cv_sensitivity(), and
ss_aipe_reg_coef_sensitivity().divisor = "s_anova" branch of
ss_aipe_sc_ancova_sensitivity() read the confidence limits
out of ci_sc_ancova() by row position ([2, 2]
and [4, 2]) when that function returns three rows, so the
branch treated the point estimate as the lower limit and NA as the
upper: its realized widths were NA and its “Type I error” rows were
nonsense. The limits are now read by term name, as the
s_ancova branch always did, and the branch’s summaries are
meaningful for the first time.mean_psi and siblings in the three contrast
members, mean_sm, mean_smd,
mean_rmsea, and mean_path, computed from the
same replications the widths come from. The echo rows the schema calls
for were added where missing (ss_aipe_R2_sensitivity(),
ss_aipe_cv_sensitivity(),
ss_aipe_reg_coef_sensitivity() and its rc /
src wrappers, the contrast members, and
ss_seq_c_sensitivity(), which now echoes
half_width, true_psi, true_sd,
alpha_level, and m0).ss_power_R2_sensitivity() and
ss_power_reg_coef_sensitivity() echo their planning inputs
(p, true_R2 / true_b_j,
estimated_R2 / estimated_b_j,
desired_power, NA when a size was specified directly, and
alpha_level), and their realized-R^2 rows
follow the meaningful-capital rule (mean_R2, and in the
omnibus member mean_F / F_crit).
tidy() and glance() for the
dmar_ss_power_sensitivity class are unchanged; the echoes
ride along in glance().ss_aipe_icc() accepted and documented the six
Shrout-Fleiss type labels but planned every one of them on
the single-rater scale, so average-of-k plans were materially
undersized. The planning value and target width are now interpreted on
the scale of the requested form: an average-of-k value is mapped to the
single-rater scale through the inverse Spearman-Brown relation and each
candidate confidence limit is mapped back (the convention
var_icc() uses), and type is now validated, so
an unrecognized label fails instead of silently planning
ICC(1,1). Average-of-k recommendations move: at
rho = .70, k = 3, and
width = .20, planning for ICC(1,k) now
recommends n = 110 (realized mean width .20 across 2,000 Monte Carlo
replications of the F-based interval) where the undersized plan
recommended n = 69 (realized width .26); at width = .10 the
recommendation is n = 421, and at rho = .90,
width = .10 it is n = 52. Single-rater plans are unchanged.
The planned form travels as the icc_type attribute on the
returned table.ss_aipe_icc_sensitivity() had the matching defect on
the generator side: it treated true_rho as the single-rater
ICC no matter the type. For the average-of-k forms it now
simulates data whose population ICC at the average-of-k level equals
true_rho, so the realized estimates, widths, and coverage
refer to the form being planned; type is validated the same
way.?ss_aipe_icc that the assurance correction
over-recommends sample size by 25 to 40 subjects was backwards. At the
page’s own condition (rho = .70, k = 3,
width = .20, assurance = .80) the recommended
n = 79 delivers an empirical assurance of about .77, and n = 81 is the
smallest sample size that reaches .80 (10,000 Monte Carlo replications).
The Details section now says so and points to
ss_aipe_icc_sensitivity() for checking a strict assurance
target.alpha_lower
critical value of the central F-distribution, the lower
noncentrality limit is 0 by construction; that is a normal consequence
of a small observed effect, not a failure. ci_nc_F() still
warns once in that case, but the message now states the consequence for
the interval (the lower confidence limit is 0) instead of describing
achieved tail probabilities, and every function that builds its interval
by inverting the noncentral F through it
(ci_snr(), ci_srsnr(), ci_pvaf(),
ci_R2() with fixed predictors,
ci_eta_squared(), ci_eta_squared_partial(),
ci_omega_squared(),
ci_eta_squared_generalized() with the parametric method,
and ci_mahalanobis()) restates it for its own effect size,
for example “the lower confidence limit on the signal-to-noise ratio is
0”, at most once per call. Previously ci_snr() and
ci_srsnr() surfaced the inner wording, which pointed users
to a prob_greater column those functions do not return. The
warning carries the condition class dmar_nc_F_clamp, which
the iterative callers (ss_aipe_R2(),
ss_aipe_omega_squared(), factorial_anova(),
simple_effects_AB()) now match by class when
deduplicating.uniroot() message: the error now names the function that
was called, reports the F-statistic, degrees of freedom, and
tail probabilities involved, and says what to try.df, the significance lens of
design_consequences() (power,
type_s_error, exaggeration_ratio) is now
computed from the noncentral t distribution of the test
statistic, the exact distribution when the standard error is estimated
from the data and the same sampling model the precision lens already
used; the exaggeration ratio integrates the truncated normal moments
over the chi distribution of the estimated standard error. The previous
code evaluated a location-shifted central t, the known-se
approximation behind Gelman and Carlin’s retrodesign(), so
the help page promised the noncentral t while the code
delivered the approximation. Small-df results move: at n = 5
per group with a true effect of d = 1, power is now 0.2863 (was
0.2469; a two-million replication Monte Carlo of the design gives 0.2860
with simulation standard error 0.0003, and base R’s
power.t.test(strict = TRUE) agrees exactly), the Type S
error is 0.00129 (was 0.00937), and the exaggeration ratio is 1.658 (was
1.915). The differences fade as df grows (negligible by
about 60 per group), and the df = Inf normal case is
unchanged.smd_trimmed() applied the reciprocal of the
Algina-Keselman-Penfield scaling constant at every trimming proportion
other than the 0.20 default: the internal constant computed the
Winsorized variance of a standard normal correctly and then returned
1 / sqrt(win_var) where the definition calls for
sqrt(win_var), and a hard-coded 0.642 at
trim = 0.20 masked the error at the default while making
the estimate discontinuous there (trim = 0.1999 returned
2.43 times the value trim = 0.20 returned). The constant is
now sqrt(win_var) at every trim and the hard-coded branch
is gone. At the default the constant moves from the rounded 0.642 to its
exact value 0.6419398, so estimates at trim = 0.20 change
by less than one part in ten thousand; at any other trim the correction
is substantial (previous estimates were 1.47 times too large at
trim = 0.10, 4.99 times too large at
trim = 0.30, and 17.99 times too large at
trim = 0.40). The help page identity is corrected to match:
0.642 is SD(X_W) / SD(X), that is sqrt(Var(X_W) / Var(X)), for a
standard normal Winsorized at 0.20, not Var(X) / Var(X_W).smd_trimmed()’s help page promised a confidence
interval on the Yuen-Welch degrees of freedom, but the code inverted the
noncentral t at h_1 + h_2 - 2 degrees of freedom
with noncentrality d_R * sqrt(h_1 * h_2 / (h_1 + h_2)); the
Yuen-Welch value was computed, returned in the df_yuen row,
and never used. The interval now follows the construction in Keselman,
Algina, Lix, Wilcox, and Deering (2008): Yuen’s t-statistic on
the trimmed-mean difference (their Equation 8) is referred to a
noncentral t distribution with the Yuen-Welch approximate
degrees of freedom (their Equation 9), and the noncentrality limits are
rescaled to the d_R metric. Feeding the summary statistics
of the paper’s worked example (their Tables 1 and 3) through this
construction reproduces the printed robust effect size intervals [0.31,
3.37] and [0.10, 1.11] (p. 119) to the precision the rounded published
inputs support, while h_1 + h_2 - 2 degrees of freedom give
[0.40, 3.31] for the first: the paper’s intervals use the Yuen-Welch
value. The former noncentrality also understated the observed statistic
(by about 16% at the default trim under equal Winsorized variances), so
intervals move even where the two degrees of freedom nearly agree: on
the help page example the interval is now [-0.88, 0.08] where the old
construction gave [-0.97, 0.17].lin_ccc() now builds its confidence interval on Lin’s
(1989) z-transformed standard error, with the correction noted
in Lin (2000), and the former default
method = "king_chinchilli" is removed. The removed variance
was not a consistent estimator of the sampling variance of the
z-transformed CCC: when the two means and variances are equal
it inflates the correct asymptotic variance by exactly the square of (1
+ rho^2) / (1 - rho^2), already about 2.8-fold at a CCC of .5 and
several hundred-fold at .95 (and by a comparable factor otherwise), so
its intervals were nearly vacuous. The help page’s own first example
returned CCC = .928 with CI [-0.977, 1.000]; the same example now
returns [0.871, 0.960], matching the independent
DescTools::CCC() z-transform interval to ten
decimals, and the Lin interval’s simulated coverage is .952 at nominal
.95 (bivariate normal, rho = .5, n = 50, where the removed
default covered .982 with intervals 61% wider). The removed formula was
also not King and Chinchilli’s (2001) estimator, so keeping it under
that name would have credited a wrong formula to real authors; a correct
King-Chinchilli variance may return in a later release. Calls that
request method = "king_chinchilli" now fail loudly.fleiss_kappa() computed its z statistic and
p-value from a null variance that treats the category marginals
as known constants, not from the estimated-marginals null variance of
Fleiss, Nee, and Landis (1979), the correction of the standard errors in
Fleiss (1971), even though the help page credited that paper. The two
expressions coincide at uniform marginals, which is how the error hid,
and diverge as the marginals skew: at N = 1000, m = 5,
and marginals (.85, .10, .05), the old null standard deviation was 3.8
times the empirical one (.0303 versus .0080). Because the
known-marginals expression is never smaller than the corrected one (they
are equal only at uniform marginals), every affected z was
understated and every affected p-value overstated; the test was
conservative, never anticonservative. On the help page’s own Fleiss
(1971) Table 1 example the z statistic moves from 15.64 to
17.65, now matching irr::kappam.fleiss to ten decimals. The
point estimate and the confidence interval do not move: kappa is
unchanged, and the se, lower_limit, and
upper_limit columns come from the Gwet (2008)
linearization, which was correct all along.ci_dunnett(alternative = "less") formed its
simultaneous upper bounds with the signed critical value that
cv_dunnett() reports for that alternative, a negative
lower-tail quantile, which placed every bound at
diff - |d| * se: below the point estimate it was supposed
to bound, and in contradiction with the adjusted p-values
printed beside it (on the PlantGrowth example both “less” rows reported
an upper limit below zero, a rejection, beside adjusted
p-values of 0.162 and 0.989, no rejection). Simultaneous
coverage of the true differences measured 0.00115 against the nominal
.95 in a balanced null design with four groups and n = 10 per
group. The critical value’s magnitude is now applied on the side the
alternative dictates, so the “less” bound is
diff + |d| * se; coverage in the same design measures
0.94685, and the PlantGrowth “less” upper limits move from -0.928 and
-0.063 to 0.186 and 1.051, agreeing with multcomp::glht()’s
one-sided limits to about four decimal places (the remaining daylight is
multcomp’s simulated-quantile error) and with the adjusted
p-values on every row. The alternative = "greater"
interval already used the positive critical value and is unchanged, as
are the two-sided intervals and all adjusted p-values.anova_within_two_way() previously
reported these artifacts as estimates. The Greenhouse-Geisser and
Huynh-Feldt rows for such an effect now carry NA in epsilon, the
adjusted degrees of freedom, and the p-value, with one warning
naming the condition; car::Anova likewise declines to
report the corrections when the effect’s error matrix is singular. The
unadjusted and lower-bound rows are unchanged: the lower bound 1/df is
Geisser and Greenhouse’s a priori bound, not an estimate, and remains
valid however few the subjects (Maxwell, Delaney, & Kelley, 2027,
Chapters 11 and 13).ci_r() accepted any sample size and went quietly wrong
below n = 4: at n = 3 it returned the vacuous interval [-1, 1] (the
Fisher’s Z variance 1/(n - 3) is infinite there), and at n = 2 it
returned NaN limits. The function now stops with an informative error
for n < 4, and the n = 3 message explains why no usable interval
exists at that boundary.ci_method = "fisher" in reliability_alpha(),
reliability_omega(), and reliability_kr20();
the Fisher interval there now stops for N < 4 with the same
explanation. The package’s other Fisher’s Z consumers
(var_r(), equivalence_r(),
meta_r(), ss_power_r(),
correlations_test(),
correction_for_attenuation(), expected_r())
already validated their sample sizes.ss_aipe_*_sensitivity family
scaled some or all of their coverage and width-attainment rows by 100
while the rest of the family reported proportions.
ss_aipe_sc_sensitivity() reported
type_I_error_upper and type_I_error_lower as
percentages; ss_aipe_cv_sensitivity() reported
pct_ci_less_w, pct_ci_miss_low, and
pct_ci_miss_high as percentages beside a
total_type_I_error that was already a proportion, so its
own rows did not add up; and ss_aipe_reg_coef_sensitivity()
(inherited by ss_aipe_rc_sensitivity() and
ss_aipe_src_sensitivity()) scaled all four. These rows are
now proportions on the 0 to 1 scale everywhere, so
total_type_I_error equals the sum of
pct_ci_miss_low and pct_ci_miss_high, and a
realized Type I error compares directly to 1 - conf_level
with no per-function rescaling. Term names are unchanged; only the scale
moved.mediation_mbco()’s null hypothesis for an indirect
effect is a union of branches (the product is zero when any factor is),
and the constrained search could converge to a worse-fitting branch on
some platforms and OpenMx builds, inflating the likelihood ratio
statistic. When every constrained algebra is a pure product of free
parameters, the branches are now also fit directly as ordinary
unconstrained models with one factor fixed to zero, and the reported
statistic is defined by the best-fitting branch on every platform.
Non-product constraints keep the multi-start constrained search.p_value, se,
ci_lower, ci_upper, p_adjusted,
conf_level, std_estimate, R2,
adj_R2, df_residual, and so on, replacing
broom’s dotted p.value / std.error /
conf.low across every tidier in the package. One naming
system now covers every DMAR surface; a pipeline that feeds a tool
expecting broom’s dotted schema renames the columns at that
boundary.content_validity_index(), dmacs(),
measurement_invariance(), and
measurement_alignment() gained bespoke
tidy()/glance() pairs (items, ladder rungs,
and groups as terms; the scale and model level summaries as the one-row
glance), and a generic wide branch in the default methods serves
htmt(), average_variance_extracted(), and any
future wide table (term from the label columns, estimate from the first
numeric column, remaining columns passed through). The output vignette’s
promise that the verbs answer everywhere is now literally true.B is the number of bootstrap replications
everywhere. mlmr() and mlmr_mv()
rename boot_R, R2_mixed_effects() and
ci_eta_squared_generalized() rename R, and
krippendorff_alpha() renames n_boot; the
effective-count row is now B_used. Old argument names fail
loudly.
cohen_kappa() and
fleiss_kappa() gain bootstrap intervals
(ci_method = "percentile" or "bca",
B = 10000, seed), redeeming the kappa page’s
own advice that the Wald interval can have poor small-sample coverage
(Blackman & Koval, 2000; Zapf, Castell, Morawietz, & Karch,
2016). Subjects are resampled, table input is expanded to
the equivalent pairs, and the se, z_value, and
p_value columns keep their asymptotic definitions; only the
interval changes.
average_variance_extracted() gains a
percentile bootstrap (ci_method = "percentile",
B = 1000, refitting the model per replication) and now
always returns ci_lower / ci_upper columns
(NA under ci_method = "none"). No interval is
possible from loadings alone, and the function says
so.
krippendorff_alpha() no longer bootstraps by
default (boot = FALSE), matching the package-wide
rule that no analysis runs a bootstrap unless asked; its verbal
reliability benchmarks were removed in favor of reporting the
coefficient with its interval.
Failed bootstrap replications are dropped, never
fatal: mediate() no longer errors when a
degenerate resample returns no indirect effect, and htmt()
no longer aborts when a resample leaves a block’s average
within-construct correlation nonpositive. Both drop the replication,
warn once with the count, and stop only when fewer than 100 replications
survive. (Correction, 1.1.0: this was not true of every failure. In
1.0.0 a resample in which a factor level was absent stopped
mediate() with an error; see the 1.1.0 entry on dropped
replications.)
No help page wraps its examples in
\donttest{} or \dontrun{} any more.
Under R CMD check --as-cran, one \donttest{}
block anywhere makes R run the whole example corpus twice, so the
wrapper cost time rather than saving it. The examples that were slow are
now fast. Replication counts in the cheap demonstrations were lowered,
with a comment naming what a reported analysis deserves. No
example runs a bootstrap confidence interval: those calls are
carried as commented code, so the syntax is still on the page for a
reader who wants it, and the surrounding prose explains the interval and
when to ask for it. The randomization tests keep their resampling, since
permuting the data is what those functions do and their examples cost a
tenth of a second. Anything else expensive is commented out the same
way, each passage introduced by a sentence saying what it does and why
it is not run. The data behind every example are unchanged. The slowest
help page now takes 0.38 seconds, and all 302 together take 8
seconds.
cfa_1() is now the one factor special case of
cfa_k(), a convenience wrapper rather than an
independent implementation, and the new
cfa_2() is the two factor sibling.
cfa_1() only requires the data (and, when the data hold
more than the items, a vector of item names); cfa_2() takes
the items of each factor as factor_1 and
factor_2. Both forward everything to cfa_k(),
so there is now a single fitting implementation, one output schema (with
confidence interval columns), and one set of fit index choices; under a
robust estimator every CFA surface now reports the robust index
versions. The former cfa_1() extras moved or dissolved:
composite reliability with the observed denominator is
reliability_omega(denominator = "observed"), the
omega output mode is the omega_f1 row of the
standard table, and the term names follow the cfa_k()
convention (lambda_f1_<item>,
psi_f1_<item>, omega_f1).data and S are now separate
arguments across the CFA family. Raw data are passed as
data and a covariance matrix as S (with
N); an argument never means both. cfa_k()
refuses a square symmetric matrix passed as data, naming
the fix. cfa_1() keeps its conveniences: unnamed input is
auto-named y1, y2, …, and items
defaults to every column.measurement_invariance() now fits any
measurement model, not only the one-factor case. It accepts
model as lavaan syntax or as a named list mapping factors
to their items (the items argument remains as the
one-factor convenience), along with ordered,
missing for full information maximum likelihood,
group_partial for the partial invariance case, and
parameterization. With ordered indicators the ladder itself
changes: thresholds carry the location information, so a
thresholds rung is fitted between configural and
metric, following Wu and Estabrook (2016) and Millsap and Yun-Tein
(2004). Constraining loadings before thresholds, as the continuous
ladder does, tests the wrong hypothesis for ordered items. The chi
square difference test is the scaled one whenever the estimator is
robust or the data are ordered, and the fitted lavaan objects are
returned on a "fits" attribute so a caller can run score
tests and partial invariance refits without paying for the ladder
twice.
dmacs() reports the dMACS effect
size of measurement noninvariance (Nye & Drasgow, 2011): the
expected difference between two groups’ measurement equations for an
item, integrated over the focal group’s latent distribution and
standardized by the pooled item standard deviation. A score test says a
loading or intercept differs detectably; dMACS says whether the
difference is large enough to matter. The defining integral has a closed
form under a normal latent variable, so no numerical integration is
used, and the two agree to machine precision in the tests. Accepts a
fitted multiple group lavaan model or the parameters a paper reports.
The help page states plainly that the index is meaningless from a
configural fit, where the groups share no metric.
measurement_alignment() implements
the alignment method of Asparouhov and Muthen (2014). Exact invariance
essentially never holds across many groups, which leaves the ladder
stalled at configural and group comparison blocked. Alignment estimates
the group factor means and variances that make the measurement
parameters as nearly invariant as possible, minimizing a simplicity
function whose fourth-root component loss tolerates a few large
differences and punishes many small ones. Both the fixed and free
identifications are available, the optimizer runs from several starts
because the surface has local minima, and the number of distinct optima
found is reported so a solution is never presented as unique when it is
not.
irt_grm() fits Samejima’s (1969)
graded response model. It does so the way the rest of the package works,
by fitting the categorical factor analysis model with lavaan and
converting the solution: discrimination
a_i = lambda_i / sqrt(1 - lambda_i^2) and location
b_ik = tau_ik / lambda_i. The two models are the same model
in different parameterizations (Takane & de Leeuw, 1987), so this
adds item response theory without a second estimation engine and without
leaving the factor analytic tradition. Both the normal ogive and the
logistic metric are available.
irt_information() and
plot_irt_information() give the item and
test information functions and the standard error of the latent trait,
SE(theta) = 1 / sqrt(I(theta)). This is precision as a
function of where the respondent sits on the trait, which is what a
single reliability coefficient cannot express: a scale can have
excellent omega and still measure poorly over the range a study cares
about. Verified against the closed form for the dichotomous case, which
the graded model must reproduce exactly.
content_validity_index() computes the
item level content validity index from a panel of expert relevance
ratings, with the modified kappa that corrects it for chance agreement
(Polit, Beck, & Owen, 2007), Lawshe’s (1975) content validity ratio,
and the scale level S-CVI/Ave and S-CVI/UA. Each I-CVI carries an exact
binomial confidence interval, because an index computed from five or six
experts is a proportion with real uncertainty and reporting it as a bare
point estimate overstates what a small panel establishes. This is
evidence about the items, gathered before any data are collected.mediation_mbco() implements the
model-based constrained optimization (MBCO) procedure of Tofighi and
Kelley (2020, Psychological Methods): a likelihood ratio test
of any smooth function of path coefficients (an indirect effect, a total
effect, a contrast of two indirect effects) formed by refitting the
mediation model subject to the nonlinear constraint that the function
equals zero. The model is specified in lavaan syntax (observed or latent
variables; parallel or sequential mediators) and fit in OpenMx, whose
optimizers support the nonlinear equality constraints the null model
requires. The function enumerates the total, direct, total indirect, and
every specific indirect pathway from x to y,
reports each with a delta method standard error and a profile
likelihood, Monte Carlo, or Wald confidence interval, tests each with
the MBCO likelihood ratio statistic and its p-value, and
reports AIC and BIC differences and the change in each endogenous
R-squared under every null model. Because the null set of a product
constraint is a union of surfaces, each null model is refit from several
starting configurations and the best feasible solution is kept, so the
reported statistic reflects the globally best-fitting null model rather
than a local branch. Accepts raw data or the summary statistics a paper
reports (covariance matrix, means, and sample size), which reproduce the
raw-data analysis exactly; the tests replicate the published empirical
example from its Table 1 moments. A group argument turns
the model into a multiple-group SEM in which every effect is estimated
per group and its between-group difference is tested, which is moderated
mediation with a categorical moderator. A moderator
argument probes continuous moderation stated in the syntax with
: interaction terms (or precomputed product columns): each
moderated pathway effect becomes a symbolically derived polynomial in
the moderator, reported as conditional effects at probe values (the mean
and one standard deviation either side by default), the index of
moderated mediation (Hayes, 2015) tested by likelihood ratio rather than
bootstrap, and, when a pathway is moderated in several places, a joint
constancy test whose null model imposes several nonlinear constraints at
once. hypotheses accepts a named list whose
multi-expression elements are likewise tested jointly. Constrained null
models are started on every branch of the constraint’s null set
(including exact conditional-coefficient branch starts for probed
effects) and each solution is polished by a warm restart, so the
reported statistic reflects the best-fitting feasible null model.
Structural guardrails refuse pathway enumeration for nonrecursive
(feedback) structures and warn on binary endogenous variables, on
interaction terms no declared moderator accounts for, and on
interactions missing their main effect (the principle of
marginality).plot_mediation_mbco() draws the
conditional effects of a moderated mediation_mbco()
analysis as curves over the moderator’s observed range, one per
moderated pathway effect, with a pointwise Monte Carlo confidence band,
the probed values marked, a dashed zero line whose band crossings
estimate the Johnson-Neyman boundaries, and a rug of the observed
moderator values so extrapolation is visible. The band’s polynomial is
the same stored quantity the table probes, evaluated over the whole
range; the help page is explicit that the band is pointwise and that
boundary locations read from it are estimates, with the table’s
constancy test as the formal companion. Returns a plain ggplot object in
the Okabe-Ito palette.var_ete() computes the variance of
the estimated treatment effect at selected covariate values in a
two-group ANCOVA with heterogeneity of regression and a random covariate
(Li, McLouth, & Delaney, 2020), the reimplementation of
MBESS::var.ete(); tested against the MBESS reference on
every branch and dogfooded in the Pygmalion vignette.
tidy() and glance() answer on
every DMAR result. Two default methods on the
dmar_tbl class are the floor under the package: any result
table answers tidy() with the broom-shaped long view
(term, estimate) and glance()
with the one-row wide view, repeated terms disambiguated rather than
dropped. The family methods sit ahead of the defaults and keep their
richer views, and two families join them: the ss_aipe_*
planners carry a new dmar_ss_aipe class whose
tidy() reports the planned size beside the desired width it
was planned against (term, estimate,
width), through the same size registry the power planners
use and a width registry beside it, and whose glance()
widens the echoed inputs. ss_power_pcm(), the one power
planner missing its family class, joins dmar_ss_power, so
its tidy()/glance() no longer error, and
ci_eta_squared_generalized() joins its siblings in
dmar_ci_anova.
One sample size vocabulary across every planner.
A planner’s answer row is now named for what it counts:
necessary_N (total), necessary_n_per_group,
necessary_n_per_cell, or necessary_n_clusters;
a user-fixed size echoes as specified_*; and
total_N appears only as the implied-total companion beside
a per-unit answer, never as an answer’s name. Before this sweep the same
word meant different things in different planners:
sample_size was the total in ss_aipe_R2() and
its seven siblings but the per-group size in ss_aipe_c()
and its three, ss_aipe_smd() said
sample_size_per_group, ss_power_pcm() carried
the MBESS-era ss_c and ss_t pair (two rows for
one number in a balanced design, now one branch-named row),
ss_power_factorial_ancova() used one bare
n_per_cell for both its planned and its user-supplied
branch, and the cluster planners’ answers had no prefix at all.
Sensitivity outputs echo the evaluated size under its bare unit name
(total_N, n_per_group), and the six
sensitivity functions whose specified_N argument actually
meant a per-group size now call it n_per_group
(ss_aipe_c/sc/sc_ancova/smd/equivalence_smd/pcm_sensitivity).
Two term names in the cluster family that contained literal spaces are
underscored. The tidy()/glance() size recognizer no longer accepts the
legacy names, so a stray old producer fails a test instead of slipping
through, and a 216-table characterization grid recorded before the sweep
reproduces identically after it, so only names moved, never values. The
package is unreleased; old names fail loudly.
The number of predictors is p
everywhere. ci_rc(), ci_src(),
ss_aipe_rc(), ss_aipe_src(),
ss_power_rc(), and plot_R2() renamed
J to p, and ci_R() renamed
K to p, matching ci_R2(),
ci_reg_coef(), and the rest of the regression family.
J remains only in the partial and semipartial correlation
family, where it counts the variables partialed out (a different
quantity; J = 0 is the simple correlation). The package is
unreleased, so the old names fail loudly rather than being
aliased.
ci_rc() and ci_src() are
documented as the thin wrappers they are around
ci_reg_coef(), the general engine, with titles that
distinguish the unstandardized, standardized, and general
cases.
Every function help-page title is now AP title case with no trailing period, matching base R convention (previously the package mixed sentence case and title case).
A numerical audit compared every displayed formula and quoted magnitude on these pages against the quantity the function computes. In each case the computation was correct and the page was not, so no returned value changes; the documentation now matches the code, and new tests recompute each corrected expression independently and assert agreement with the function output.
expected_partial_r() displayed the
expectation with every index one lower than the Olkin-Pratt formula
under the n to n - J substitution the code makes; as printed, the
formula can exceed 1, an impossible value for the expectation of a
correlation. The page now prints E[r] = rho 2F1(1/2, 1/2; (n - J + 1)/2;
rho^2) Gamma((n - J)/2)^2 / (Gamma((n - J - 1)/2) Gamma((n - J + 1)/2)),
which matches the function to machine precision. The bias prose also
described E[r] - rho while the returned bias column is rho
- E[r]; the sign convention now matches the column, and the quoted
magnitudes are the true +0.00624 (rho = 0.4, n = 30, J = 2) and +0.00888
(J = 10).expected_r() quoted bias magnitudes
with the wrong sign and size (-0.024 and -0.006); the true values are
+0.021 (rho = 0.5, n = 10) and +0.0065 (n = 30), as the page’s own
example already reported.icc_lmer() printed the variance of the
Bonett (2002) L-transform as 1 / (2 (n - 2)), omitting the k / (k - 1)
factor the code applies; the page now prints k / (2 (k - 1) (n - 2)). A
reader hand-building the interval from the old page at k = 2 would have
an SE too small by a factor of sqrt(2) and roughly .84 coverage instead
of .95.anova_within_two_way() printed the
lower-bound epsilon as 1 / (df - 1); the code computes 1 / df, the
attainable infimum, with df the effect’s numerator degrees of
freedom.loa() printed a symmetric central t
interval for the CIs on the limits of agreement; the code computes the
Carkeet (2015) exact noncentral t interval, which is asymmetric about
the sample LoA. The page now shows the noncentral form: quantiles of the
noncentral t with noncentrality parameter k sqrt(n) for the upper LoA
and its negative for the lower.variance_components_mls() printed a
symmetric interval with unsquared constants and a -MS_b MS_w / n cross
term that appears nowhere in the method; the code computes the genuine
Burdick-Graybill (1992, equations 2.4.1–2.4.5) form, and the page now
prints V_L and V_U with the squared constants and the G_12 / H_12 cross
terms.ss_aipe_mixed_effects() printed
Var(beta-hat) with a trailing design-effect factor the code does not
apply; for the cluster-mean-centered level-1 slope the variance is
sigma2_y (1 - rho_I) / (N sigma2_x), and the page now explains why the
design effect does not enter.ci_smd() pointed paired designs to
ci_smd_c(), which is the interval for Glass’s estimator
(two independent groups, control group SD) and takes no correlation
between paired measurements, so following the pointer reproduced
essentially the independent groups interval. The page now states plainly
that a paired-design SMD interval is not currently provided.power_fisher_exact() attributed the
alternative distribution to Wallenius; the code computes Fisher’s
noncentral hypergeometric, the conditional distribution of one binomial
count given the total of two independent binomials, which is what
conditioning on the margins of the 2 x 2 table produces. The prose and
references now cite Fisher
(1935) and Fog’s (2008) sampling-methods paper.ci_scheffe() said the default returns
a - 1 pairwise contrasts; it returns all a (a - 1) / 2 of them, as the
sibling pages already stated.A continuation of the audit above, covering another two dozen help pages. As before, the computations were correct and the pages were not; no returned value changes, and where a corrected number is load-bearing a test now recomputes it independently.
ci_rmsea()’s 90 percent example described its upper limit
as landing below 0.05; the limit is 0.052, just above the Browne and
Cudeck close fit threshold, and the page now draws the conclusion that
follows (close fit is not established, even though the point estimate
sits below the threshold). obrien_test()’s Hunter example
quoted p = .2595, which matches neither the quoted F = 1.29 nor the
exact statistic; the page now quotes p = .260, as computed.
combine_p()’s Edgington sum for the Raudenbush example
prints 7, not “near 6.9”. ci_c()’s hypertension example
described 24 subjects across group sizes of 4, 6, 5, and 5; it says
20.mediation_mbco() separates the published
memory-example likelihood ratios from what its example
computes. Run from the rounded Table 1 summary statistics, the
two null branches give LRT = 71.31 (the best-fitting branch, the
statistic reported) and 179.02; the 72.54 and 175.77 the page previously
quoted are the full-precision-moments values, and 175.77 is now
attributed to Tofighi and Kelley (2020) as the published value, from the
worse-fitting branch on which their optimizer stopped. The page notes
that the difference comes from running from the published (rounded)
summary statistics.ss_power_indirect_effect()’s example no longer
equates the approximation with the simulation benchmark. The
joint significance approximation returns necessary_N = 65
for a = b = .39 at power .80; raw-data simulation puts the power at 65
nearer .77 and reaches .80 near N = 70, in line with the somewhat larger
requirement in Fritz and MacKinnon’s (2007) simulation-based table, and
the example comment now says exactly that. The raw-data validation
test’s relative tolerance is widened from 0.03 to 0.06 with the reason
recorded in place: the approximation sits about 3 percent above the
simulation at those settings on any seed, so the old tolerance failed on
some seeds for the approximation gap alone.prime_time_achievement’s corporation summaries
are recomputed from the derived corp_id key. The
bare corp column merges the two corporations that share
code 2400, so the page overstated the largest corporation: corporations
have 15 to 756 students (median 117), not 15 to 808 (median 124), and
the three-level null model decomposition is 16.29 / 22.72 / 240.43 with
a corporation ICC of 0.058 (previously 16.50 / 22.70 / 240.43 and
0.059). The data set’s tests now anchor the corrected values.bessel_errors’ expected counts imply a normal
sigma near 0.22, not 0.2. A least squares fit of the
half-normal bin probabilities to Bessel’s expected frequencies gives
0.216; at 0.2 the first bin alone is off by about eight
observations.which_width contract of
ss_aipe_omega_squared(), ss_aipe_icc(), and
ss_aipe_partial_r() now says what the code does.
The pages described "Lower" / "Upper" as a
one-sided half-width; both settings interpret width as half
the full width and return the same sample size. The pages now say so,
note that an asymmetric interval’s realized half-widths differ from each
other and from half the full width, and state that a genuinely one-sided
width target is not currently offered.ci_mahalanobis() documents the tail arguments
its backend accepts. The page promised that supplying
alpha_lower and alpha_upper recomputes
conf_level; that call errors, because the tails pass
straight through to ci_nc_F(), which refuses a non-NULL
conf_level beside them. The page now directs users to set
conf_level = NULL and supply both alphas, and a test pins
the contract.ci_nc_F() and ci_nc_chisq() state
the monotonicity each search actually exploits. The pages
claimed each tail probability is strictly decreasing in the
noncentrality parameter; the lower-limit condition works on the upper
tail, which is strictly increasing. Each page now gives the direction
per tail and notes that both roots are located on the decreasing
lower-tail scale.ss_aipe_equivalence_smd() described an expected half-width
target of omega where the code targets omega / 2 (full width omega), and
a 1-row return where the table has 5 rows; both now match, and the
return’s stale term names are corrected. The stale
sample_size return term on the ss_aipe_c(),
ss_aipe_c_ancova(), and ss_aipe_sm() pages now
reads necessary_n_per_group,
necessary_n_per_group, and necessary_N.
ci_eta_squared()’s N is documented as the
total number of observations (for aovlist fits, one more
than the sum of all effect and residual degrees of freedom), matching
its Details. mlmr()’s adj_R2 is documented as
using the complete cases (N_complete), which the code
deliberately uses, rather than the lavaan reported N.
ci_c_ancova_bp()’s contrast_type entry dropped
inline width expressions that omitted the 1/sqrt(2) and sqrt(1/n)
factors and now points to the correct display in Details.
ci_sc()’s note no longer contains a sentence directing
users to pass the error variance where the argument (and the rest of the
page) wants the error standard deviation. The multiplicative
diff_size illustration on the ss_aipe_crd_*
pages was a verbatim copy of the additive one; it now works the
multiplicative form’s arithmetic (cluster size 25 with
diff_size = c(0.8, 1, 1.2) gives sizes 20, 25, 30, recycled
across clusters).equivalence_c() performs the two
one-sided tests procedure and the companion noninferiority test for a
linear contrast of group means against bounds stated in raw units of the
response, with one pooled error term, a summary-statistic and a direct
(estimate, SE, df) interface, a benchmark argument for
comparisons against a known constant, and a five-way verdict
(equivalent, superior, inferior, non-inferior only, inconclusive).
Reproduces
emmeans::test(..., side = "equivalence" | "noninferiority")
to machine precision.power_equivalence_c() computes the
exact TOST power for a contrast by integration over the chi distribution
of the estimated error standard deviation, generalizing
power_equivalence_md() to arbitrary weights and unequal
group sizes, and the noninferiority power in closed noncentral
t form.ss_power_equivalence_c() finds the
smallest per-group sample size whose exact TOST (or noninferiority)
power reaches a target, the declaration-probability counterpart of the
width-targeting ss_aipe_c().ss_seq_c() and
ss_seq_c_sensitivity() implement the
purely sequential fixed-width confidence interval procedure for a
contrast of Chattopadhyay, Bandyopadhyay, Kelley, and Padalunkal (2025),
with cost-optimal allocation across groups; the sensitivity sibling
verifies first-order efficiency and near-nominal coverage by Monte
Carlo.plot_equivalence() draws contrast
estimates and their intervals against the equivalence region, colored by
the equivalence_c() verdict.R2_mixed_effects() reports the
Nakagawa and Schielzeth (2013) marginal and conditional R-squared for a
fitted lme4 or nlme mixed model. It agrees to
machine precision with r2mlm and with the direct Johnson
quadratic form across the model classes the tests pin: random-intercept,
correlated random-slope, split (||) random-slope,
multi-slope, and nested two-level models. On split random-slope models
performance/insight omits the random-slope
variance from its marginal and conditional R-squared, so DMAR matches
r2mlm and the Johnson form there and, by design, not
performance::r2_nakagawa; the two agree on random-intercept
and correlated-slope models.R2_mixed_effects_decomposition()
implements the Rights and Sterba
(2019) integrative framework of mixed-effects model R-squared measures:
the full family of total, within-cluster, and between-cluster measures
from a complete five-source decomposition of the outcome variance,
matching the authors’ r2mlm reference implementation to
machine precision across the same tested model classes.knit_print.dmar_tbl() renders DMAR results as formatted
tables in knitted documents, as_kable() produces a
knitr::kable view, and results_sentence()
writes an APA-style “estimate, CI” sentence from any interval-carrying
result.broom support (tidy() /
glance()) added for the elementary tests
(welch_t(), summary_t_test(),
contrast_test()) and the simultaneous-comparison intervals
(ci_dunnett(), ci_tukey_kramer(),
ci_scheffe()).tidy() and glance()
summarize every closed-form ss_power_* planner that reports
one size and one power, from the effect size planners to the ANOVA,
ANCOVA, contrast, cluster, and mediation designs, reporting the design’s
planning unit (per group, per cell, per subject, or per cluster, or the
total for the one-way ANOVA) beside its power. The lookup that resolves
the size row learned the design-specific names (n_per_cell,
n_subjects, J_per_arm,
ss_per_group), so the meaningful names are kept rather than
homogenized. The Monte Carlo sensitivity siblings
ss_power_R2_sensitivity() and
ss_power_reg_coef_sensitivity() carry a
dmar_ss_power_sensitivity class whose tidy()
places the empirical and analytic power side by side.ss_power_composite_ancova_2group()
plans the per-group sample size for composite power in a two-group
ANCOVA: the probability that the group effect, the covariate effect, and
the group by covariate interaction are all significant in the same
study, the quantity a design must be planned against when its conclusion
needs more than one result to hold at once. Composite power is not the
product of the marginal powers, because every test divides by the same
error estimate and the tests are positively dependent even when the
effects are orthogonal. A one dimensional integral over the chi square
distribution of that estimate evaluates the composite deterministically,
with no simulation, and the plot() method draws the
population effects the plan rests on. This opens the new composite power
family.ss_power_composite_factorial_ancova() and
ss_power_composite_factorial_anova() carry
composite power to any balanced factorial design: name any set of main
effects and interactions in effects, each with its Cohen’s
f or partial eta squared, and the planner returns the per-cell
sample size at which all of them are jointly significant. The effects
are noncentral F tests sharing one error estimate, so the same
shared-error integral applies; a single effect reproduces
ss_power_factorial_anova() (or
ss_power_factorial_ancova() with a covariate) exactly, and
the two-effect composite matches a direct simulation. The ANOVA function
is the no-covariate case named directly and admits no covariate. With no
covariate the composite is exact to quadrature precision. The effects
can be stated as sizes (Cohen’s f or partial eta squared) or as
a full array of population cell means with a common within-cell standard
deviation, from which each effect’s f is read off the analysis
of variance decomposition of the means. The plot() method
draws the purported population values: the cell-mean pattern (with error
bars of one within-cell SD) when means were given, or the effect sizes
annotated with their marginal power otherwise, with the composite power
in the subtitle either way.ss_power_composite_factorial_ancova_het()
carries composite power to a factorial ANCOVA whose covariate slope
differs across the cells. The average slope (the covariate main effect)
and the factor by covariate slope heterogeneity are then testable
effects that can join the composite alongside the factorial mean
effects. The full model fits the means, the covariate, and every factor
by covariate slope, so the residual has N minus twice the cells degrees
of freedom; a “mean”, “covariate”, or “slope” effect is named in
effects and sized by a Cohen’s f or read from
population values (cell means and a covariate outcome correlation per
cell, with a common within-cell SD). The one-factor two-level case
reproduces ss_power_composite_ancova_2group() to machine
precision, and a three-effect composite matches a direct simulation of
the heterogeneous-slope model. Kept separate from the common-slope
ss_power_composite_factorial_ancova() for ease of use. The
plot() method draws the population regression line in each
cell, so heterogeneous slopes read as lines of different angle.ss_power_composite_ancova() and
ss_power_composite_anova() are the general entry
points to the composite power family, covering a one-way design with any
number of groups as well as any factorial arrangement. The ANCOVA
planner takes a slopes argument, "homogeneous"
for one common covariate slope or "heterogeneous" to let
the slope differ across cells and make the covariate and
slope-heterogeneity effects testable; its heterogeneous one-way case is
the a-group generalization of the two-group
ss_power_composite_ancova_2group(), and its two-level case
reproduces it exactly. The ANOVA planner is the no-covariate design
named directly, with a covariate-free interface that points to the
ANCOVA version when a covariate is present. Both accept effect sizes or
population values (cell means with a common within-cell SD) and forward
to the factorial planners, so the same broom
tidy()/glance() summaries and
plot() figures apply.ss_power_composite_sem() and
ss_aipe_composite_sem() carry composite sample
size planning to structural equation models, for both design goals. The
researcher states the population as a fully fixed lavaan model (or its
cov_sem() covariance matrix), labels the parameters of
interest in the free analysis model (structural paths, loadings,
covariances, or :=-defined quantities such as an indirect
effect), and the planner finds the smallest N at which every
labeled parameter is statistically significant in the same study with
the desired probability (ss_power_composite_sem()), or at
which every confidence interval is sufficiently narrow, in expectation
or with a stated assurance for the joint event
(ss_aipe_composite_sem()). No closed form covers a set of
dependent SEM estimates, so both planners run an a priori Monte Carlo
simulation (Muthén & Muthén, 2002; Maxwell, Kelley, & Rausch,
2008): data are drawn from the population covariance matrix, the
analysis model is fit G times per candidate N, and
a search seeded by the analytic Wald approximation brackets and bisects
to the smallest integer meeting the goals; each reported power or
proportion travels with its simulation standard error, and a
seed argument makes a plan reproducible. The power planner
joins the dmar_ss_power broom family; marginal
power_<label> and
width_within_desired_<label> rows show which
parameter binds the design. Both planners handle a population mean
structure, so latent growth curve targets such as the slope factor’s
mean are planned the same way: cov_sem() now also returns
mu_theta, the model implied means of the observed
variables, a mu argument supplies means beside a hand-built
Sigma, and the simulated data carry those means whenever
the analysis model has a mean structure. The “Composite Sample Size
Planning for SEM” vignette
(vignette("composite_sem_planning")) walks through the full
workflow for a mediation model with observed variables and for a linear
latent growth curve, planning both composite power and joint
accuracy.cohen_h() returns Cohen’s h,
the effect size for the difference between two proportions on the
arcsine (variance-stabilizing) scale, h = 2 asin(sqrt(p1))
minus 2 asin(sqrt(p2)). It is signed and is the proportion analog of the
standardized mean difference (smd()), so a given h
carries the same detectability wherever the proportions sit, which a raw
difference does not.contrast_adjusted() tests an arbitrary
contrast among covariate-adjusted cell means in a factorial ANCOVA,
matching emmeans::contrast().dmar_tbl
display layer, and literature-synonym @concept tags
(Cohen’s d, Cronbach’s alpha, Cohen’s U3, …) make the marquee estimators
searchable by their eponyms.ss_power_smd() and ss_aipe_smd()
echo their user-supplied planning inputs as rows of the returned
table, so the assumptions a design was planned under travel
with the result as tidy data rather than a printed footer. The supposed
effect is labeled supposed_smd to make clear it is a value
the researcher posits (a minimally important effect or a value believed
to be true in the population), not a sample estimate; power planning
also echoes desired_power, alpha_level, and a
numeric tails (2 or 1), and AIPE echoes width
(and assurance when supplied). The value
column stays numeric.reliability_omega_h() now places the observed
total-variance denominator on the same maximum-likelihood (N-divisor)
metric as the fitted loadings, matching
MBESS::ci.reliability(type = "hierarchical") and
semTools::compRelSEM(obs.var = TRUE).DMAR is the modern, more general reimplementation of MBESS that reflects how the methods are now used across quantitative psychology, sociology, education, management, marketing, and information systems. MBESS remains stable on CRAN; DMAR is the recommended path forward for new users.
reliability_alpha() and
reliability_omega() handle missing data by full information
maximum likelihood, with auxiliary variables. The measurement
family thereby catches up with mlmr(), whose vignette makes
the case against listwise deletion that these functions previously
ignored. Two new arguments, also passed through by
reliability():
missing = c("listwise", "fiml"), defaulting to listwise
deletion so no existing result changes, and aux, a
character vector naming auxiliary columns of data that
enter as saturated correlates (Graham, 2003): correlated freely with
each other and with every item’s residual, never loading on the factor.
Supplying aux implies missing = "fiml". The
analytic alpha applies the classical formula to the FIML estimate of the
item covariance matrix; the model based estimators fit with lavaan’s
missing = "ml"; robust omega’s observed total variance
comes from the FIML covariance matrix. MBESS documents an
aux argument on ci.reliability() that no
longer runs (upstream API drift in semTools), so DMAR implements the
saturated correlates model directly in lavaan syntax rather than
depending on it. The returned table now always carries an
N_complete row beside N, making the cost of
listwise deletion visible at a glance, and the treatment is recorded in
missing and aux attributes. Interval methods
that cannot be made correct under FIML (the complete-data closed forms,
ADF, the profile likelihood) are errors, never silent fallbacks; the
bootstrap resamples partially observed rows and refits by FIML. Under
MAR missingness driven by an auxiliary, the test suite shows
FIML-with-auxiliary reducing the bias of listwise deletion roughly
38-fold across 200 replications, and a hand-specified saturated
correlates model in lavaan reproduces the estimates exactly.
reliability_omega() gains a
denominator argument selecting how the total variance in
the denominator of coefficient omega is estimated:
"observed" (the default: robust omega, the variance of the
composite estimated directly from the data, the coefficient Kelley and
Pornprasertmanit, 2016, call hierarchical omega and
MBESS::ci.reliability() calls type
"hierarchical") or "model_implied" (the
textbook form). The two definitions coincide in the population when the
single-factor model is correctly specified; only robust omega retains
its interpretation as the proportion of the variance of the composite
actually computed when the model is misspecified, which is why it is the
default. The help page states the properties of each choice and the
distinction from the bifactor omega-hierarchical of Zinbarg, Revelle,
Yovel, and Li (2005). The returned object records the choice in a
denominator attribute.
cfa_k() accepts ordered-categorical
items (ordered = TRUE or a vector of item names;
raw data required, each factor all ordered or all continuous). The model
is fit by WLSMV to polychoric correlations with thresholds in the theta
parameterization, which keeps every defined measurement quantity
available, with robust standard errors. Because a sum score of ordered
items lives on the metric of the observed categories rather than the
latent response metric, each ordered factor’s omega is the Green and
Yang (2009) categorical sum score omega computed from the same fit; the
substitution is announced in a message, recorded per factor in an
omega_metric attribute, and the delta method interval
columns are NA for those rows (use
reliability_omega_categorical() for a bootstrap interval).
A single-factor ordered cfa_k() reproduces
reliability_omega_categorical() to 1e-6 in the regression
tests despite the different parameterizations, which pins the
parameterization invariance of the computation.
reliability_alpha() gains an
estimator argument carrying the two routes to
coefficient alpha, which were briefly two functions.
estimator = "analytic" (the default) is the classical
closed-form equation applied to the observed covariance matrix, the
number a hand calculation produces;
estimator = "model_implied" is the reliability implied by
the tau-equivalent (equal loadings) single-factor model fit by maximum
likelihood, which brings a delta method standard error, a robust
(Satorra-Bentler) variant, the profile likelihood interval, and a
testable fit of the equal-loadings claim. Users of MBESS will know them
as ci.reliability(type = "alpha") and
type = "alpha-cfa"); the point estimates match those to
1e-6 in the regression tests. The value "model_implied" is
deliberately the word reliability_omega() already uses for
the analogous choice, so one vocabulary covers both coefficients, and
the reliability() wrapper gains a matching
estimator argument beside its denominator.
This replaces the short-lived
reliability_alpha_analytic(), whose name was inherited
MBESS jargon that described neither the estimand nor the contrast.
Merging the two also removes a defect their separation created:
reliability_alpha(ci_method = "likelihood") used to report
the classical point estimate beside an interval profiling the model
implied coefficient, two different quantities, so on a misspecified
model the interval could exclude the estimate printed above it (on the
psych bfi A scale, estimate 0.431 against an
interval of [0.591, 0.636]). Each interval method now belongs to the
estimator that can supply it, and asking for one the chosen estimator
cannot give is an error naming the estimator to use instead.
Profile likelihood confidence intervals join the
closed forms: ci_method = "likelihood" in
reliability_omega() (model implied denominator) and
reliability_alpha() (profiling the tau-equivalent,
CFA-based alpha). The interval is the set of population values not
rejected by the likelihood ratio test, computed by refitting the
single-factor model under a nonlinear constraint on the model implied
reliability. It respects [0, 1], is not forced to be symmetric, works
from raw data or a covariance matrix, and matches
MBESS::ci.reliability(interval.type = "ll") to the fourth
decimal in the regression tests.
No bootstrap runs unless the user requests one, anywhere
in the reliability family. Robust omega and categorical omega,
whose confidence intervals are bootstrap based, report the point
estimate by default with a message naming the exact call that produces
the recommended interval (percentile or BCa for robust omega; BCa for
categorical omega). The model implied omega keeps its closed-form robust
ML interval as the default, and alpha and KR-20 keep their closed forms.
When a bootstrap is requested, B = 10000 replications is
the default.
reliability_omega_h() was removed.
Its coefficient is exactly
reliability_omega(denominator = "observed"), and the “h”
(for “hierarchical”) described no hierarchy in the single-factor model
the function fits while inviting confusion with the bifactor
omega-hierarchical. The reliability() wrapper drops
type = "omega_h" and instead forwards a
denominator argument to reliability_omega().
The documentation refers to the coefficient as robust omega, records the
original motivation (model misfit as minor common factors, with the
coefficient isolating the general factor’s variance against the observed
composite variance) and the authors’ retrospective preference for a name
stating the behavior, states the qualifications the word robust requires
(robust to misspecification of the total variance only; distinct from
outlier-robust estimation and from robust standard errors), and notes
the design principle shared with categorical omega: in both, the total
variance in the denominator is not taken from the fitted factor
model.
reliability_omega_categorical() is the
categorical omega function’s full name, and its only name (an
earlier reliability_omega_c() alias was removed). The
reliability() wrapper’s canonical type is
"omega_categorical" with "omega_c" accepted as
a shorthand. The coefficient attribute is now
"omega_categorical".
The reliability vignette was rewritten around the framing of Kelley and Pornprasertmanit (2016): choosing the coefficient (a claim about the measurement model and the composite being scored) and choosing the interval (an empirical performance question their Monte Carlo studies answered, which is what the family’s defaults encode). One running example walks alpha versus omega, the two omega denominators under a minor-factor contamination, categorical omega under same versus differing threshold patterns, and coefficient H as a different composite rather than a different assumption.
cohen_kappa() now accepts a
published k x k frequency table (table =) in place of raw
rater vectors, and custom weight matrices in either scaling: agreement
weights (diagonal 1) or Cohen’s (1968) ratio-scaled disagreement weights
(weight_scaling = "disagreement", zero diagonal, invariant
to positive rescaling, converted internally via w = 1 - v / max(v)).
Asymmetric weight matrices are supported for validity designs where the
two directions of a confusion carry different costs. The help page
replicates Cohen’s (1968) Table 1 analyses in full: unweighted kappa
.492, weighted kappa .348 under his disagreement weights, .574 with the
6 and 1 weights interchanged, his asymmetric computer-diagnosis validity
example (.353), and his Formula 10 and 13 standard errors (.0901, .0916,
z = 3.80), which the examples reproduce for the historical record while
the function reports the Fleiss, Cohen, and Everitt (1969) standard
error that superseded them. When both raters are supplied as factors
with the same level set, their level order is now respected instead of
alphabetical sorting, which previously could silently misalign ordinal
categories with linear, quadratic, or custom weights. Every result also
carries a cells attribute holding the per-cell detail in
the form of Cohen’s Table 1: observed proportion, chance-expected
proportion, and the weights (both scalings when disagreement weights
were supplied), one row per cell of the confusion matrix.
diagnosis_agreement ships Cohen’s
(1968) Table 1 as a data set in its original layout (Judge B in rows,
Judge A in columns): one row per cell with the frequency, Cohen’s
ratio-scaled disagreement weight, the observed proportion, and the
chance-expected proportion. The reconstruction is verified against every
quantity computed from the table in the paper.
A weighted kappa vignette works Cohen’s (1968)
illustration in full on the diagnosis_agreement data:
unweighted and weighted kappa, both weight scalings, his Formula 10 and
13 standard errors beside the Fleiss-Cohen-Everitt interval, linear and
quadratic weighting with the weighted-kappa-equals-r identity under
equal marginals, and the asymmetric validity example, including a
neutral working of how the printed weight display’s orientation relates
to the published values.
cfa_k() fits a confirmatory factor
analysis model with one or more factors, each specified by naming its
indicators. The measurement structure is specified by describing what is
constrained (equal_loading, equal_intercept,
equal_error, each a single value or per-factor), and the
function names the classical structure the description implies
(congeneric, essentially tau-equivalent, tau-equivalent, essentially
parallel, or parallel; Graham, 2006) in the printed header and the
"model" attribute. The table reports every estimate with a
confidence interval, and per factor coefficient omega, the average
variance extracted, and coefficient H as lavaan defined
parameters with delta method standard errors and intervals (no semTools
involvement). output = "measurement" gathers the
measurement properties, the latent correlations, and htmt()
per factor pair; output = "fit" hands back the
lavaan object so two descriptor fits feed
lavaan::lavTestLRT() directly. The RMSEA interval level is
reported explicitly as the rmsea_ci_level row.
plot_cfa_k() displays the
item-level loadings, error variances, or intercepts of a
cfa_k() fit with confidence intervals, one panel per
factor, with a dashed reference line that shows the equated value (or,
for free estimates, the informal “one common value” anchor), so the
equality questions behind the classical structures can be seen before
they are tested.
bifactor_indices() computes the
bifactor dimensionality and reliability indices (ECV, omega, omega
hierarchical and hierarchical subscale, PUC, and coefficient H) from a
fitted bifactor lavaan model, with a guard that flags
improper (Heywood) solutions (Rodriguez, Reise, & Haviland,
2016).
simple_structure() quantifies
Thurstonian simple structure in a loading matrix: Hofmann item
complexity, the hyperplane proportion, and pure/complex item
counts.
ecvi() gives the Browne and Cudeck
(1989) expected cross-validation index for a covariance-structure model,
with a confidence interval derived from the noncentral chi square
(ci_nc_chisq()); accepts a lavaan fit or a
published fit table.
common_method_single_factor() and
common_method_marker() implement the
single-common-factor (Harman) screen and the Lindell and Whitney (2001)
marker-variable adjustment for common method variance. The Harman screen
is implemented factor analytically: a one-factor model is fit by maximum
likelihood (stats::factanal()) and the reported proportion
of variance is the common factor’s, not a principal
component’s.
cv_bryant_paulson() gives the
simultaneous critical value (the ANCOVA member of the cv_*
family, reducing to sqrt(2) * cv_tukey_hsd() when there are
no covariates), and ci_c_ancova_bp() places simultaneous
(familywise) confidence intervals on contrasts of adjusted means, the
familywise counterpart of the per-comparison ci_c_ancova().
Both are computed exactly from the Bryant–Paulson generalized
studentized range distribution (qbryant_paulson() /
pbryant_paulson() / dbryant_paulson()),
evaluated as a Beta mixture of ptukey() rather than read
from a table. Three vignettes cover the critical values, an end-to-end
ANCOVA workflow, and a simulation confirming exact familywise error
control. Implements Bryant and Paulson (1976) and Bryant and Bruvold
(1980).regions_of_significance(): the values
of the covariate at which two groups differ significantly when the
within-group regression slopes are not equal (heterogeneity of
regression), where the group difference is a function of the covariate
rather than a single number. The boundaries solve the quadratic that
sets the squared group difference against its sampling variance at the
critical value, the Johnson and Neyman (1936) procedure; with more than
two groups the calculation is carried out for every pair.
plot_regions_of_significance() draws the estimated
difference across the covariate with the confidence band the region is
read from, so the plot is the decision rule.cv_f(), cv_chisq(), and
cv_bonferroni_f() complete the cv_*
family’s coverage of the Maxwell, Delaney, and Kelley (2027) Appendix.
The family had cv_t() and cv_z() but not the
F or chi square counterparts, and the Bonferroni F
table had no function at all. cv_f() covers Appendix Table
A.2, cv_bonferroni_f() Table A.3, and
cv_chisq() Table A.9; the tests assert each against the
printed values.cv_f() and cv_chisq() default to
alternative = "greater" rather than the
"not_equal" that cv_t() and
cv_z() use. Neither distribution is symmetric, and both are
used one-sided in the upper tail for the tests they serve: a restricted
model fits worse than a full one, so evidence against a restriction is a
large F, never a small one. Both tails remain available,
through alternative = "not_equal" or through
alpha_lower and alpha_upper, for an interval
on a variance or on a ratio of variances. Both accept a noncentral
parameter, as cv_t() does.cv_bonferroni_f() reports the per-comparison rate alpha
/ C back in its area_greater column, which is the whole of
what the adjustment does. Its help page separates it from the rank-sum
procedure of dunn_test(): Dunn (1961) is the Bonferroni
procedure, Dunn (1964) is the nonparametric one.ci_games_howell(): simultaneous
confidence intervals for all pairwise comparisons when homogeneity of
variance is not assumed. Every other all-pairs procedure in the package
pools the within-group variances into MS_W, so none of them is robust
when that assumption fails. Games-Howell uses a separate error term and
a Welch-Satterthwaite degrees of freedom for each pair, then takes its
critical value from the studentized range, following Maxwell, Delaney,
and Kelley (2027, Chapter 5, Equations 5.13 and 5.14). It is the
heterogeneity-robust counterpart of ci_tukey_kramer() and
handles unequal n as a matter of course. With two groups it is exactly
Welch’s t test, which the tests assert against
t.test(var.equal = FALSE). Implements Games and Howell
(1976).dunn_test(): Dunn’s rank-sum test of
all pairwise differences, the follow-up to a significant Kruskal-Wallis
test. It ranks all observations together, as the omnibus test does, and
uses the variance of the ranks implied by the Kruskal-Wallis null (with
the tie correction), so it stays coherent with the omnibus result in a
way that running a Mann-Whitney test on each pair does not. The
method argument passes the pairwise p-values to
p.adjust(). Implements Dunn (1964). Note this is the
nonparametric Dunn procedure, not the Bonferroni procedure of
Dunn (1961) that Maxwell, Delaney, and Kelley
(2027) call Dunn’s procedure; the help page disambiguates the two.randomization_test() and
randomization_test_paired(): randomization
(permutation) tests for two independent groups and for paired
observations. The two-group test refers the observed statistic to its
distribution over reassignments of the observed scores to the groups, so
the p-value needs no assumption about the population’s shape,
and the result travels with the effect sizes the test only screens for:
the mean difference with a randomization-based interval, the
standardized mean difference with a noncentral t interval, the
common language effect size, and Cliff’s delta. The paired test treats
the within-pair sign of each difference as the randomization mechanism,
enumerating all 2^n sign patterns exactly for small samples and sampling
them otherwise. Implements the logic of Fisher
(1935) as developed by Edgington and Onghena (2007).
plot_randomization_test() displays the randomization
distribution the p-value is read from, with the observed
statistic marked.alpha_level
everywhere. Twenty-two functions took a bare alpha
while twenty-eight already took alpha_level, and the bare
name was carrying three incompatible meanings inside one package: the
Type I error rate in the cv_*, equivalence, TOST,
sequential, and Fisher-exact families; coefficient alpha in
var_alpha(); and ggplot2 transparency in
plot_trajectories(). A reader who learned one meaning would
misread the others. The Type I error uses are now
alpha_level throughout, so fifty functions share one
unambiguous name, and the argument that echoes it in a result table is
named to match. The other two keep alpha, where it is
unmistakable: var_alpha(alpha = ) is the coefficient the
function is named for and parallels var_omega(omega = ),
and transparency is the universal ggplot2 convention in a plotting call
where no error rate appears. Since the package is unreleased, the old
name fails loudly rather than being aliased. alpha_lower
and alpha_upper, which name the two tails of an asymmetric
critical value, are unchanged.cv_smm() and cv_dunnett() are now
deterministic. Both formerly obtained their
multivariate-t quantile from mvtnorm::qmvt(), a
Monte Carlo integrator whose sampling error reached a few hundredths at
alpha_level = .01, enough to move the second decimal of a
tabled critical value. They now evaluate the quantile by exact numerical
quadrature and root finding: the studentized maximum modulus factorizes
into a single one-dimensional integral over the shared scale (the
m statistics are independent given it), and the balanced-design
Dunnett statistic, with its constant correlation 1/2,
factorizes through a one-factor representation into two nested
one-dimensional integrals. The returned values are reproducible to the
solver tolerance, reproduce the published Dunnett and studentized
maximum modulus tables more closely than the Monte Carlo path did, and
no longer depend on mvtnorm. Both functions now accept
df = Inf (the known-variance normal limit).seed argument of cv_smm() and
cv_dunnett() is removed: the computation is no longer
random, so there is nothing to seed. Call sites that passed
seed should drop it.ci_dunnett() adjusted p-values are now
exact. They were computed from mvtnorm::pmvt(), a
Monte Carlo integrator (and, when was absent, from a conservative
Sidak-Bonferroni fallback). They now use the same deterministic
one-factor integral as the critical value, so the reported
p_adjusted is reproducible and no longer depends on . The
shared numeric engine lives in R/dunnett_internals.R.cv_* family reproducing the Appendix
critical-value tables and validates them by simulation.qbryant_paulson() /
pbryant_paulson() / cv_bryant_paulson() are
now accurate at small error df. They obtained the
studentized-range part of the distribution from
stats::ptukey(), whose algorithm loses accuracy at small df
(at nu = 3, k = 20 by ~3e-4 in probability, enough to move the critical
value by ~0.2, and more at nu = 2). For nu < 7 the studentized-range
distribution is now evaluated directly, without ptukey, by
integrating the probability integral of the range against the chi
squared error density. The range CDF is splined and cached per group
count; the knots are placed at a fixed spacing of about 0.003 in the
range argument, fine enough that the monotone (Fritsch and Carlson)
interpolation error stays below 1e-9 and the returned critical value
matches the direct integral to about seven figures, while the small-df
path still costs a fraction of a second per group count. The functions
reproduce Bryant and Paulson’s (1976) Table 1 exactly, to the two
decimal places tabled, over the whole of its range: both tail areas, all
three covariate counts, every tabled group count, and every tabled error
degrees of freedom from nu = 2 to nu = 120, which is 1188 critical
values in all. Two entries, q_.01;2,8,3 = 23.165013 and q_.01;2,20,4 =
19.745008, sit about 1e-5 above the point where the second decimal turns
over, so they round to 23.17 and 19.75 while the 1976 table rounds them
down; both were confirmed to fourteen significant figures by two
independent high-order quadrature engines that share no code with the
package. A large-scale simulation of the statistic confirms the computed
values independently. Values for nu >= 7 are unchanged.adjusted_means() extracts the full adjusted
(least-squares) means table. From an
lm/aov factorial or ANCOVA fit it returns
every cell’s covariate-adjusted mean with SE and CI, or marginal means
over chosen factors via by, with
weights = "equal" (the population marginal means of Searle,
Speed, and Milliken, 1980) or "proportional" (observed
frequencies). The reference grid follows the model’s own terms, so
transformed covariates and interactions are handled exactly, and
nonestimable cells in rank-deficient designs error plainly by name.
Validated against emmeans 2.0.3 at 1e-10 across fourteen fits
(unbalanced factorials, multiple and transformed covariates, character
and ordered predictors, weighted fits, an empty-cell design) for both
weightings, with the comparisons pinned in tests and re-run live in
tools/oracle_checks.R; also pinned to ancova()’s adjusted
means and to contrast_adjusted() at 1e-10.
loa() reports the Carkeet (2015) pair
intervals by default. The exact CIs for the limits of agreement
considered as a pair, the construction Carkeet recommends for most uses,
replace the per-limit intervals as the default;
method = "individual" keeps the one-sided tolerance-factor
form. Both constructions are anchored in tests to the paper’s printed
coefficients and worked example, and a misattributed comment (the Bland
and Altman 1999 approximate SE, previously credited to Carkeet) is
corrected.
Sixteen packages leave Suggests. DMAR’s test oracles (MBESS, emmeans, semTools, metafor, mirt, sirt, multcomp, performance, r2mlm, irr, irrCAC, psych, BayesFactor, gsl) are no longer dependencies: every live comparison was replaced by its pinned value, with provenance comments naming package, version, and date, and the live comparisons themselves moved to tools/oracle_checks.R, which re-runs all of them against the installed oracles at release time. BiasedUrn is replaced by a self-contained Fisher noncentral hypergeometric density (agreement 1.2e-13), kableExtra by knitr-only HTML and LaTeX table construction, and AMCP by shipping the depression_bdi data directly. Suggests drops from thirty packages to fourteen, none of them needing compiled system libraries beyond what the remaining features genuinely use.
The Bayes factor functions take informed priors, stated
either way. The prior on the standardized effect can now be an
informed Cauchy, moving prior_location off zero (Gronau,
Ly, & Wagenmakers, 2020), or a normal with prior_mean
and prior_sd for beliefs stated as moments; a Cauchy has no
mean and no variance, so moment beliefs could not previously be
expressed at all. The two families are exclusive, and the help pages
explain what the Cauchy scale fixes (the quartiles: half the prior mass
within one scale of the location) and the exact identity linking the
families: a Cauchy is a normal prior whose variance is itself uncertain,
and the test suite uses that identity to cross-validate the two code
paths against each other with no external oracle. Every result now
carries the full posterior of the effect in a "posterior"
attribute (a data frame of delta and density),
so any posterior probability can be computed, not only the reported
ones; the prior is echoed in prior_location and
prior_scale rows with the family recorded as an attribute.
Default calls reproduce the previous JZS results exactly.
The Bayes factor functions accept summary
statistics. The JZS Bayes factor depends on the data only
through the t statistic and the sample sizes, so
bayes_one_sample_t(), bayes_paired_t(), and
bayes_independent_t() now take the summary statistics a
paper reports (mean, sd, and n;
mean_diff, sd_diff, and n; or
mean_1, sd_1, n_1,
mean_2, sd_2, n_2) as readily as
raw data, following the same exactly-one-path rule as the rest of the
package. The summary form is exact, not an approximation, and the help
pages show how a standardized effect size enters (a d of 0.5 is
mean_1 = 0.5, mean_2 = 0 with unit standard deviations).
Tests pin the raw and summary forms to each other at machine
precision.
The directional test family speaks snake_case.
The canonical alternative values of
ci_dunnett(), power_fisher_exact(),
randomization_test(),
randomization_test_paired(), summary_t_test(),
and welch_t() are now "two_sided",
"less", and "greater", with the base-R
spelling "two.sided" accepted as an alias so existing calls
keep working; anything stored on a returned object carries the
underscore form. The cv_* critical value family keeps its
deliberately wider synonym vocabulary with "not_equal"
canonical, unchanged.
var_omega_squared() returned a variance that
did not shrink with the sample size. The worker multiplied the
wrong combination of terms, and the consequence was not a small bias:
the returned variance converged to a positive constant instead of to
zero, so its error grew with N. Against a 200,000-replication Monte
Carlo of the sampling distribution it was too large by a factor of 1.4
at the smallest cell tested and by 115 at the largest, and N times the
variance climbed from 0.8 at N = 30 to 51 at N = 5100 where the true
value settles near 0.31. Any standard error or Wald interval built on it
was badly inflated, and worse in larger samples. The variance is now the
delta-method transfer of Fleishman’s (1980, Eq. 22) exact variance of
the unbiased estimator of the signal-to-noise ratio, carried to the
omega squared scale by his Eq. 8 with Jacobian (1 - omega2)2,
which tracks the Monte Carlo to within a few percent across every cell
tested and to 1.00 by N = 300. Two regression tests now guard it:
agreement with a fixed-seed Monte Carlo, and the consistency check that
N times the variance stays bounded, which the old formula would have
failed. The help page also no longer attributes an omega squared
variance to Fleishman, who gives one only for the signal-to-noise ratio
and says explicitly that the correlation ratio has an interval and a
median but not a variance.
Generalized eta squared now classifies interactions the
way its sources do. Olejnik and Algina (2003, Eq. 5) and
Bakeman (2005) define an effect as a measured source of variance when
any factor in its term is measured, so a
measured-by-manipulated interaction belongs in the denominator
automatically. eta_squared_generalized() and
ci_eta_squared_generalized() had instead included only the
effects explicitly listed in observed, a rule the help page
misattributed to Bakeman (2005), who recommends the opposite. Listing a
factor in observed now also places every interaction
containing it in the denominator, so observed = "c"
reproduces the papers’ worked examples as written; an explicit
interaction label is still honored as given. Values change only for
model-interface calls on designs with measured-by-manipulated
interactions, where the corrected values are smaller. Tests now anchor
the Keppel and Kirk worked values printed in Olejnik and Algina
(2003).
signal_to_noise_R2()’s nonlinear estimator now
follows Muirhead’s
(1985) Equation 10 exactly: the c/Y correction is added to the untruncated linear estimate before truncation at zero, rather than after the linear estimate was itself truncated. The two orders agree whenever the sample R-squared is not tiny; for sample R-squared below p/(N-1) the reported nonlinear value is now smaller, as the paper defines it.
In ss_aipe_pcm()’s assurance search, the critical
t now tracks each candidate sample size’s degrees of freedom
instead of reusing the value frozen from the expected-width search. The
frozen t was very slightly conservative; the correction moves
nothing by as much as one subject, and every documented example returns
the same sample size as before.
The ci_c_ancova() example carried the wrong sum of
squares for its covariate. The Maxwell, Delaney, and Kelley Chapter 9
depression example needs the within-groups sum of squares of the
pretest, which the chapter data give as 752.5 (and which the sibling
ci_sc_ancova() example already used); the example instead
passed 313.37, the within-groups sum of squares the covariate explains
in the posttest, a different row of the same ANCOVA decomposition. The
interval moves only in the third decimal, but the two pages now agree
with each other and with the chapter data, and a test recomputes 752.5
from the data (via the AMCP package) so the constant cannot drift
again.
ci_sc() now works when the noncentrality parameter
is supplied directly. The ncp branch set the noncentrality
but never the standardized contrast, so the documented call errored on
the closing table construction (three terms against two values). The
branch now inverts the same relationship the means and
psi branches use (the noncentrality is the standardized
contrast divided by sqrt(sum(c_weights^2 / n))), so the
three parameterizations of one effect return identical tables; that
equivalence and the ncp path’s coverage are now tested.
ci_c() and ci_sc() now recycle a scalar
n by the number of contrast weights. The scalar was
recycled by length(means), which is zero on the
psi-only path (and, for ci_sc(), the
ncp-only path), so those documented calls stopped on the
n / c_weights length check. No result changes
on the means path, where the two lengths agree; a
means vector whose length differs from
c_weights is now rejected with a clear error instead of
being silently recycled.
ss_aipe_cv() now accepts the documented
mu / sigma parameterization, deriving
c_of_v = sigma / mu; previously neither formal was read and
the call stopped on “argument is of length zero”. Supplying
c_of_v together with mu or sigma
is rejected as conflicting. The mu / sigma
path returns exactly the sample size of the equivalent
c_of_v call (for mu = 10,
sigma = 1, width = .1,
conf_level = .99, both give N = 20). The
example prose on the help page said the population coefficient of
variation was .25 while the calls passed .1 (which plans
N = 20, where .25 plans N = 100); the prose
now matches the calls, and a mu / sigma
example was added.
cfa_1() with missing = "ml" now
estimates the mean structure with free item intercepts. The bare
lavaan::lavaan() interface it calls turns the mean
structure on for FIML but leaves int.ov.free at
FALSE, so every item intercept was fixed to zero and the
loadings absorbed the item means; on any data not centered at zero the
FIML estimates were wrong (the function’s own example, simulated with
mean zero, could not show it). The fit now matches
lavaan::cfa(..., missing = "ml") exactly. Listwise fits are
unaffected.
ss_power_split_plot_anova() now uses the correct
between-subjects noncentrality. The between-subjects test used the
per-group sample size n where its noncentrality should use
the total sample size N = n a, so the noncentrality was too
small by the factor a (the number of between-subjects
groups), halved for two groups. This understated between-subjects power
and, planning in reverse, overstated the necessary per-group sample
size; the within-subjects and interaction tests were unaffected. The
corrected between-subjects test now agrees with a split-plot
aov() simulation and, for two groups, reproduces the
two-level treatment test of ss_power_mixed_effects()
exactly (the between-subjects F(1, .) is the two-level
treatment t squared), the identity that ties the two planners
together. Both a value anchor and the cross-planner identity are now
tested.
cv_smm() and cv_dunnett() now work at a
large error df. Both integrated the chi squared error variate over
(0, Inf), and once df is large that density is a narrow
bump far from the origin, so the adaptive rule sampled its way past the
mass and returned zero: at df = 200 the integral evaluated to exactly 0
for every candidate quantile, which left the root finder with no bracket
and the functions stopped with “f() values at end points not of opposite
sign.” The limits are now the extreme quantiles of that variate, which
puts the quadrature on the mass at any df. Both functions gained a
large-df test; the values they already returned are unchanged.
cv_f() no longer returns NaN when the
numerator df is infinite. Supplying ncp = 0 explicitly
sends qf() and pf() down their noncentral
algorithm, which does not admit an infinite numerator df, so the
infinite-numerator column of Appendix Table A.2 came back
NaN. Both functions now pass ncp only when it
is nonzero, so the central algorithm serves the central case.
cv_chisq() takes the same guard.
ancova() now reports the covariate-adjusted omnibus
F for the treatment effect. It previously entered the treatment
before the covariate and read the sequential (Type I) sum of squares,
which is the unadjusted treatment F; the covariate is
now entered first so the treatment’s sequential sum of squares is its
adjusted sum of squares.
ss_aipe_sm() corrects the noncentrality used in the
assurance branch (it was the reciprocal sm / sqrt(n)
instead of sm * sqrt(n)), which previously left the
assurance target unreachable and pinned the search to a bracket
endpoint.
ci_c() now honors the documented
df_error argument; supplying it previously left the error
degrees of freedom undefined and errored.
tidy() and glance() on
ss_power_r() and ss_power_smd() now return the
user-supplied sample size on the realized-power path (previously
NA).
ss_aipe_reliability() now computes confidence
intervals when the default interval = TRUE is used (a
string-only equality check had skipped the interval for the logical
default).
ss_aipe_reliability(type = "Factor Analytic") works
again. The path read cfa_1()’s legacy list layout
($factor_loadings, $parameter_cov), which the
reworked cfa_1() no longer returns, so it errored;
coefficient omega and its delta method interval now route through the
maintained reliability internals
(.omega_fit_cfa(),
.ci_omega_delta()).
mlmr() and mlmr_mv() are
the new lm-like front end to full information maximum likelihood (FIML)
regression. The univariate mlmr() mirrors the
lm() API (formula interface, coef /
vcov / confint / summary /
anova / predict / update S3
methods, profile / Wald / bootstrap CIs); the multivariate sibling
mlmr_mv() takes cbind(y1, y2) ~ ... and models
the joint distribution of correlated outcomes, with the residual
covariance among outcomes estimated as part of the fit. See
vignette("mlmr", package = "DMAR") for when the FIML route
actually buys you something over lm() + listwise
deletion.mlmr() and mlmr_mv() gained an
auxiliary argument that brings variables related to the
missingness or to the incomplete outcome into the model as saturated
correlates (Graham, 2003): each auxiliary is correlated with the outcome
residual, every predictor, and each other auxiliary, but never enters as
a predictor, so the focal regression coefficients keep their meaning.
This is the inclusive analysis strategy (Collins, Schafer, & Kam,
2001); it leaves complete-data estimates unchanged and, under MAR,
recovers information that listwise deletion discards. The differential
missingness scenario in vignette("mlmr", package = "DMAR")
works through when it helps and when randomization already protects the
estimand.tidy() and
glance() integration via the generics
package: a uniform interface to the broom ecosystem for
mlmr, mlmr_mv, cfa_1, the
reliability family, the long- format CI family, the ANOVA effect size CI
family, and the power planner family.
purrr::map_dfr(fits, generics::tidy) now works across DMAR
outputs.convert_R2_* family cut iterative AIPE / sensitivity
planning calls by roughly 3x; representative ss_aipe_R2()
calls go from ~2.7s to ~0.9s.New data set test_market: a small balanced ANCOVA
example (sales by promotion type, adjusted for a baseline covariate)
used to illustrate the Bryant–Paulson simultaneous intervals in
vignette("bryant_paulson_ancova", package = "DMAR") and
ci_c_ancova_bp().
New data set drinks_trial: nine-month follow-up
drinks-per-week outcomes for the N = 88 homeless
alcohol-dependent participants in Smith, Meyers, and Delaney’s
(1998) randomized trial of the Community Reinforcement Approach at the Salvation Army Adult Rehabilitation Center in Albuquerque, New Mexico. Two consecutive cohorts: Cohort 1 compared Standard, CRA, and CRA + Disulfiram with cell sizes 17, 15, 19; Cohort 2 dropped the disulfiram cell after Cohort 1 results and compared Standard against CRA with cell sizes 20,
17. The outcome ships in raw form (heavily right-skewed, range 0 to 624.6 drinks per week) and on the log10 scale used in the published analyses to recover approximate normality. Reproduced in Maxwell, Delaney, and Kelley (2027, Designing Experiments and Analyzing Data: A Model Comparison Perspective, 4th ed., Routledge), Chapter 3, Section 3.10.4.
New data set bessel_errors: Friedrich Wilhelm
Bessel’s (1818) 9-bin grouped frequency distribution of the absolute
errors of 300 stellar position observations made by British Astronomer
Royal James Bradley at the Greenwich Observatory between 1750 and 1762.
Both the observed and expected (normal-model) frequency columns sum to
300, matching Maxwell, Delaney, and Kelley (2027, 4th ed.), Table 1.4.
Documented as a worked example for approximating moments from grouped
frequency data (frequency-weighted means and variances using bin
midpoints) and for plotting empirical-versus-theoretical frequency
comparisons. Ships in the original grouped form Bessel reported;
individual error values are not extant.
New data set prime_time_achievement (also accessible
via the short alias Prime_Time): the full Indiana Prime
Time third grade achievement evaluation file (Lapsley, Daytner, Kelley,
and Maxwell, 2002, ERIC ED466679), built from the original Indiana
Department of Education SPSS system file. 10,927 students nested in 586
classrooms in 163 schools in 61 school corporations (district x region
combinations) in 9 educational service regions on 113 variables.
Includes ISTEP+ NCE composites (the criterion in the published HLM
analyses), Gates-MacGinitie and AANCE test scores, NPA cognitive ability
scores, classroom enrollment, pupil to teacher ratio, Prime Time aide
indicator and status, school and corporation demographics and finance,
and three derived unique cluster identifiers (corp_id,
school_id, class_id) that respect the nesting
irregularities in the source file (one corporation ID spans two
regions). Original Indiana DOE variable names and the original
six-category race coding are preserved verbatim; the SPSS variable
labels are retained as a label attribute on every column.
The SPSS Select Cases artifact FILTER_$ and the 888 “not
applicable” codes have been dropped or recoded to NA.
Includes documented examples that show level-1, level-2, and level-3
lmer fits mapped onto the multilevel framework used in
Lapsley et al. (2002) and in Finch, Bolin, and Kelley (2019,
Multilevel Modeling Using R, 2nd ed., CRC Press, chapters 3, 4,
6, 9, 10).
New data set holzinger_swineford (also accessible
via the short alias HS_Data): the complete Holzinger and
Swineford (1939) factor analysis data, 301 pupils on 26 ability tests
from the Pasteur (n = 156) and Grant-White (n = 145) elementary schools
in Chicago. Variable names follow the MBESS convention (e.g.,
t1_visual_perception through t26_flags) so
scripts written against MBESS::HS port over with a single
rename of the object. The values are the corrected version of the data,
identical to MBESS::HS as of MBESS 4.9.3 and to
psychTools::holzinger.raw. The documentation discusses the
bi- factor study design, the five ability blocks (spatial, verbal,
mental speed, memory, reasoning), the Joreskog (1969) 9 test subset, and
the silent post-4.6.0 MBESS correction that is the source of values
still found in sem::HS.data and
OpenMx::HS.ability.data.
New data set pygmalion: the teacher-expectancy data
from Rosenthal and Jacobson’s (1968) Pygmalion in the
Classroom, 310 elementary school pupils in grades 1 to 6, of whom
64 were randomly designated to their teachers as likely intellectual
“bloomers” and 246 served as controls, with pretest and follow-up IQ.
This is the classic benchmark for analysis of covariance with
heterogeneity of regression, and the running example for that
topic in Maxwell, Delaney, and Kelley (Designing Experiments and
Analyzing Data, Chapter 9). The within-group slopes (0.778 for
controls, 0.969 for bloomers), the pooled residual variance (175.3251),
and the covariate variance (348.91) reproduce the worked example for the
variance of the estimated treatment effect at selected covariate values
(cf. MBESS::var.ete). The same numbers ship with the book’s
data companion AMCP as
chapter_9_exercise_15; here the experimental condition is a
labeled factor (Control, Bloomer) and the
columns use DMAR’s descriptive names, with no measured value altered.
Pairs with ancova() and is documented in
vignette("pygmalion", package = "DMAR").
The largest visible change is a uniform snake_case
interface for both function names and argument names.
Argument names that were dot.case in MBESS are
snake_case in DMAR. The most common renames a user will hit
when porting a script:
| MBESS | DMAR |
|---|---|
conf.level |
conf_level |
Random.Predictors |
random_predictors |
Specified.N |
specified_N |
alpha.lower |
alpha_lower |
alpha.upper |
alpha_upper |
degrees.of.freedom |
Use df or the explicit df_1 /
df_2 |
Group.1, Mean.1 |
group_1, mean_1 |
The rule of thumb: change every . between words to
_, and lowercase the non-statistical prefix words. Capitals
are preserved when the capital is statistically meaningful:
R2 (squared multiple correlation), N (sample
size), S (a covariance matrix), Lambda (a
factor-loadings matrix), F_value (an
F-statistic).
aipe_smd() is renamed to ss_aipe_smd()
to align it with the rest of the ss_aipe_* family.
Every estimation, inference, and planning function returns a
data.frame with term and value
columns. (Plotting functions return a ggplot object, and a
few utilities return their natural type.) Return objects in MBESS were
sometimes named lists, sometimes data.frames, and sometimes vectors; the
unified tidy return makes the package compose cleanly with the rest of
the modern R ecosystem.
Confidence intervals and sample size annotations travel with
every effect size plot. plot_smd(), plot_ci(),
and plot_R2() default to show_ci = TRUE and
show_n = TRUE.
There are no deprecation shims. Old MBESS argument names will throw “unused argument” errors at the call site, which is intentional; the fix is mechanical, and a silent-forwarding shim would hide it.
DMAR adds 40+ functions across families that were absent or under-developed in MBESS. New families include:
Mediation, both halves: mediate() analyzes the
simple mediation model (optional covariates) with percentile bootstrap,
BCa, Monte Carlo, and Sobel intervals for the indirect effect, and
ss_power_indirect_effect() plans the study by joint
significance with exact noncentral t component powers (Sobel
power for comparison), validated against raw-data simulation.
mediation_mbco() extends the inference side to arbitrary
mediation structures through the likelihood ratio model comparison
framework (see its own section above).
Measurement invariance: measurement_invariance()
fits the configural, metric, scalar, and strict multi-group ladder for
any measurement model, with a thresholds rung for ordered indicators,
and reports the full comparison table (fit, the likelihood ratio test
per step, and delta CFI / delta RMSEA), pinned to direct lavaan fits in
the tests. See the entry at the top of this file for the arguments and
the returned "fits" attribute.
Construct validity: htmt() (the Henseler
heterotrait-monotrait ratio with an optional bootstrap upper bound) and
average_variance_extracted() (Fornell-Larcker AVE from a
lavaan fit or standardized loadings). Composite reliability needs no new
function; it is coefficient omega
(reliability_omega()).
Clinical and behavioral endpoints:
responder_analysis() (per-group responder proportions with
Wilson intervals, the Newcombe risk difference interval, the number
needed to treat, an omnibus chi square, and a threshold sweep) and
ci_proportion(), the package’s Wilson score interval for a
single proportion.
A meta-analysis family. meta_es() pools any effect
sizes given sampling variances (REML between-study variance by default,
with Paule-Mandel, DerSimonian-Laird, and fixed effect options, the
Hartung-Knapp adjustment on by default, Q-profile confidence intervals
for tau-squared mapped to I-squared, and a prediction interval always
reported); meta_smd() (exact Hedges correction by default)
and meta_r() (Fisher’s Z pooling, with optional per-study
attenuation corrections) are the metric front ends.
combine_p() provides the four classical combined
significance tests and meta_contrast() the Rosenthal and
Rubin contrast among effect sizes; plot_forest() draws the
studies, the pool, and the prediction interval. Validated against
metafor, and against Raudenbush (1984) line by line in the teacher
expectancy vignette; the 19 effect sizes of that synthesis ship as
teacher_expectancy.
Factorial ANCOVA power: ss_power_factorial_ancova()
extends the factorial ANOVA planner to baseline covariates (error
variance scaled by 1 - R2, one error df per covariate), with the
complete 2 x 4 x 3 worked example, planning, simulation, Type III
analysis, interaction plots, and focused complex comparisons, in the
ancova_2x4x3_power vignette.
A new vignette, “Power and Precision for the One-Way ANOVA: A
Model Comparison Perspective,” works the one-way design from the
comparison of a full model and a restricted model: the omnibus power and
sample size through ss_power_one_way_anova(), a planned
contrast through ss_power_contrast(), effect size
confidence intervals through ci_omega_squared(),
ci_pvaf(), ci_snr(), and
ci_srsnr(), the model comparison made literal with
mlmr(), the Type S and Type M consequences of the design
through design_consequences(), and accuracy in parameter
estimation for a contrast through ss_aipe_c(), following
Maxwell, Delaney, and Kelley (2027).
A Bayesian t family with the probability statement front and
center: bayes_one_sample_t(),
bayes_paired_t(), and bayes_independent_t()
report the JZS posterior of the standardized effect (median, mean,
credible interval, and P(delta > 0 | data)) with the default Bayes
factor as a secondary row, computed by exact quadrature and validated
against the BayesFactor package.
correction_for_attenuation() (renamed from its
working name correct_attenuation()) documents and
demonstrates the latent variable route it approximates: when item-level
data exist, prefer the two-factor model’s latent correlation to the
plug-in formula.
Moments of the noncentral distributions:
moments_nc_t(), moments_nc_F(), and
moments_nc_chisq() return the mean, variance, standard
deviation, skewness, and excess kurtosis of the noncentral t,
F, and chi square distributions, with NA for
moments whose degrees of freedom conditions fail. The noncentral
t mean is the quantity behind the upward bias of the
standardized mean difference.
A random-coefficients polynomial growth simulator:
simulate_longitudinal_polynomial() generates longitudinal
data of any polynomial order (order 0 is a flat line) for one or several
groups, ties the level-one error to a target measurement reliability
(reported per occasion), allows assessment-time jitter around the
nominal schedule, and supports structured level-one error covariance
(AR(1), compound symmetry, Toeplitz, heteroscedastic, or a full matrix).
It is the Monte Carlo companion to ss_power_pcm().
ss_power_pcm() now plans power for any polynomial
change coefficient (intercept, linear, quadratic, cubic, …) through a
trend argument, carrying the general Raudenbush and Liu
(2001) sampling variance; the linear default reproduces the National
Youth Survey benchmark.
orthogonal_polynomial() returns orthogonal
polynomial trend contrast weights stored levels-by-trends (ready for
contrasts() and lm()) and printed in the
Maxwell, Delaney, and Kelley Table A.10 layout with a trailing
sum-of-squared-weights column.
unbiased_R2() gives the Olkin and Pratt (1958)
exactly unbiased estimator of the population squared multiple
correlation alongside the Ezekiel (1930) adjusted estimator (the
adj.r.squared of summary.lm).
Design consequences: design_consequences() reports
what a chosen design delivers under both of the package’s lenses, the
significance lens (power, the Type S wrong-sign error rate, and the Type
M exaggeration ratio, after Gelman and Carlin, 2014, computed exactly
rather than by their simulation) and the precision lens (the expected,
median, and standard deviation of the realized confidence interval
width, and the probability the realized interval beats a target width,
the closed-form versions of the ss_aipe_*_sensitivity()
Monte Carlo terms). Accepts a standard error directly or derives it from
sd and per-group n. Every ss_power_* and
ss_aipe_* help page points to it.
The Spearman correction for attenuation:
correction_for_attenuation() disattenuates a correlation
for measurement error in either or both variables, with a confidence
interval when the sample size is supplied, using reliabilities from the
reliability_* family.
New parameterization conversions: convert_d_r() /
convert_r_d() (standardized mean difference and
point-biserial correlation, with an unequal-group factor) and
convert_d_or() / convert_or_d() (the
Hasselblad and Hedges logistic link to the odds ratio).
convert_F_chisq() and
convert_chisq_F() move a test statistic between
the F and chi square metrics, returning the converted statistic
itself (a value, not a p-value). Two conversions are offered
through one argument, df_denominator. The default,
df_denominator = Inf, treats the F’s error
variance as known and returns the standard scaling,
df_numerator * F_value (and back,
chi_square / df); an F is a chi square whose error
variance is estimated rather than known, and infinite denominator
degrees of freedom is the case where it is known. A finite
df_denominator instead returns the chi square value with
the same upper-tail probability (the same p-value) as the
F, which is exact at any denominator degrees of freedom; the
two conversions agree as df_denominator grows, so the
scaling default is the large-sample limit of the same family. The finite
case is computed from ordinary upper-tail p-values (no
logarithms): the upper tail is used because the lower-tail probability
rounds to 1 in double precision by about F = 500 at small
denominator degrees of freedom, which would send the result to infinity,
whereas the upper-tail computation stays accurate past F =
1e20. Each help page states the exact computation. The map preserves the
p-value but does not transport a noncentrality parameter, so
noncentral work belongs in ci_nc_F() and
ci_nc_chisq().
Display helpers extending the package’s p-value
convention to objects DMAR does not produce: format_p(),
print_anova(), and print_summary().
DMAR no longer requires the GSL system library: the Gauss hypergeometric function behind the exact squared multiple correlation moments is computed in base R (and remains accurate as the squared multiple correlation approaches 1, where the previous route lost precision). Installation now has no system prerequisites.
Plot functions in the plot_* family:
plot_smd(), plot_ci(), plot_R2(),
plot_trajectories(),
plot_trajectories_fitted(). All built on
ggplot2 and colored by default with a neutral,
colorblind-safe palette; each accepts a palette argument
("okabe_ito" or "tableau") and, where
applicable, a colors override for full manual
control.
Plot colors come from base R. The plot_* family
colors itself with base R’s Okabe-Ito colorblind-safe palette by
default, with base R’s Tableau 10 available through each plot’s
palette argument. DMAR defines no palette of its own and
adds no color dependency; a user who wants other colors adds an ordinary
ggplot2 scale to the plot.
Ordinal and non-parametric effect sizes:
cliff_delta(), vargha_delaney_A(),
probability_of_superiority_paired(), cles(),
proportion_of_superiority() (sometimes called Cohen’s
U3).
Agreement and reliability extensions:
bland_altman_loa(), lin_ccc(),
cohen_kappa(), fleiss_kappa(),
gwet_ac(), krippendorff_alpha(),
icc(), icc_lmer(),
reliability_omega_categorical(),
reliability_omega_h(), reliability_H(),
reliability_kr20(), plus the var_* family of
asymptotic variance utilities.
Within-subjects, mixed, and multivariate ANOVA:
anova_within(), anova_within_two_way(),
mixed_anova(), manova_split_plot(),
mauchly_test(), epsilon_corrections(),
pairwise_within(),
simple_effects_AB().
Sample size planning has been broadened, particularly the
ss_power_* family, which now covers between-subjects,
within-subjects, mixed, and multi-level designs across the chapters of
Maxwell, Delaney, and Kelley (2027).
Parameterization conversions in the convert_* family
(convert_R2_f, convert_f_R2,
convert_lambda_R2, convert_R2_lambda,
convert_r_Z, convert_Z_r,
convert_delta_lambda, convert_lambda_delta,
convert_cor_cov). Every conversion is exact-invertible and
the inverse direction is shipped as a sibling function where it makes
sense.
A worked simulation study of the AIPE family reports expected and realized CI widths, realized coverage, and assurance-achievement rates across 10,000 Monte Carlo replications per cell. It establishes how the methods perform rather than how they are used, so it is maintained alongside the package rather than shipped in it.
ss_aipe_reliability() no longer embeds a vendored
copy of MBESS 3.2.0’s ci.reliability() for the Monte Carlo
assurance search. The interval at each candidate sample size now comes
from DMAR’s own reliability internals (the single-factor delta method
interval for the factor analytic type; the van Zyl, Neudecker, and Nel
closed form for the normal theory type). A before-and-after grid
covering every model and interval type combination confirmed the
computed interval widths, and therefore every planned sample size, are
unchanged.
icc_lmer() now reads its grouping factor from lme4’s
grouping list, so interaction groupings such as
(1 | school:teacher) work; and mlmr_mv gained
the tidy() / glance() methods its
documentation promised (one row per coefficient per outcome, with a
response column).
The test suite runs with warnPartialMatchArgs,
warnPartialMatchDollar, and
warnPartialMatchAttr enabled, so any internal reliance on
partial matching fails loudly, and the display layer (the
dmar_tbl print methods, format_p(),
print_anova(), print_summary()) is covered by
testthat snapshots.
The seed argument defaults to NULL
everywhere it is exposed, meaning the function uses the caller’s current
RNG state and does not seed; a user who wants reproducibility passes an
explicit integer, and a supplied seed restores the caller’s RNG state on
exit. The value 113 is used only in @examples
and tests, never baked into a function default.
The cluster-randomized design helpers under
ss_aipe_crd* were consolidated into a single set of shared
internals (R/ss_aipe_crd_internals.R); both the difference
and effect size families now call the same .find_*_crd_*()
back end.
ss_aipe_pcm() had a typo in the cubic change
coefficient constant (K_3 was coded as
1/1000800, about one tenth of the correct
1/100800). Because the constant sits in the denominator of
the slope variance, this inflated the cubic variance, and hence the
resolved sample size, roughly tenfold for trend = "cubic".
The constant now matches the closed form
K_p = (p!)^2 / [(2p)! (2p+1)!] of Raudenbush and Liu (2001,
p. 392), and a regression test recovers it directly from an OLS fit. The
linear (1/12) and quadratic (1/720) constants
were already correct.
ss_aipe_pcm() now honors its documented contract
that a user may supply either error_variance or the
converted variance variance_true_minus_estimated_trend.
Previously error_variance was effectively required, and the
consistency check between the two used round(..., 3), which
mishandled the small variances typical of slope-change designs. The
cross-check now uses all.equal(), and supplying neither
argument raises an informative error.
ss_aipe_pcm() now scales the within-subject
contribution to the slope variance by frequency^(2p),
matching ss_power_pcm() and Raudenbush and Liu (2001,
p. 392). The change coefficient is a per-unit-time rate, so its sampling
variance is
error_variance * frequency^(2p) / (sum of the squared polynomial weights);
the factor was previously omitted. Every Kelley and Rausch (2011)
benchmark uses frequency = 1, where the factor is 1, so all
tabled sample sizes are unchanged; the correction only affects designs
with frequency != 1, where it now agrees with the direct
OLS slope variance on the actual time grid.
ss_aipe_pcm_sensitivity() now simulates the same
estimand its planner targets: the between-group difference in mean
slopes (the group-by-time change parameter), using two independent
groups, a pooled standard error, and 2n - 2 degrees of
freedom. The previous simulator built a one-group mean-slope interval,
whose width was systematically 1/sqrt(2) of the planned
target at every frequency. The realized mean CI width now tracks the
planned width. The schema terms mean_slope /
median_slope / sd_slope are renamed to
mean_slope_diff / median_slope_diff /
sd_slope_diff to reflect the difference estimand.
Ken Kelley (Department of Information Technology, Analytics, and Operations; Mendoza College of Business; University of Notre Dame) is the package author and maintainer. Bug reports and feature requests are welcomed by email to kkelley@nd.edu; please put “DMAR” in the subject line.