When analysing partially-observed data, any statistical method makes either explicit or implicit assumptions about the missing values which can never be verified from the data at hand. Typically, most analyses rely on a missing at random (MAR) assumption, that is they assume that the observed data are able to fully explain missingness. However, there is always the chance that this assumption is not correct and missingness may depend on some values which are not observed, leading to a missing not at random (MNAR) assumption. Thus, it is extremely important that the robustness of the results to a range of alternative missingness assumptions is assessed in sensitivity analysis, including MNAR.
Each of the three types of missingness models in
missingHE
, namely selection,
pattern mixture, and hurdle models,
can be fitted under MNAR for either or both the effectiveness and cost
outcomes. In missingHE
, MNAR assumptions are codified in
terms of some suitably-defined departures from MAR for the mean
effectiveness and cost parameters, which are the main quantities of
interest in the economic evaluation. This tutorial shows how MNAR
assumptions can be specified for each type of model in
missingHE
. Throughout, we will use the built-in dataset
called MenSS
as a toy example, which is directly available
when installing and loading missingHE
in your
R
workspace. See the vignette called Introduction to
missingHE for an introductory tutorial of each function in
missingHE
and a general presentation of the data from the
MenSS
dataset. See also the vignette called Model
customisation in missingHE for few examples on how to customise the
functions in missingHE
to handle different types of issues
in the analysis.
If you would like to have more information on the package, or would like to point out potential issues in the current version, feel free to contact the maintainer at a.gabrio@maastrichtuniversity.nl. Suggestions on how to improve the package are also very welcome.
Selection models specify MNAR assumptions by directly modelling the
missingness mechanisms, that is the models for the missing data
indicators, as a function of some partially-observed variables. In
missingHE
, MNAR is defined by including the outcome
variables (\(e\) and \(c\)) inside the logistic regression models
for the corresponding missing indicators (model.me
and
model.mc
). Since some outcome values are not observed, the
parameters capturing the dependence between missingness and the outcomes
(called delta.e
and delta.c
) cannot be fully
estimated from the observed data but are, at least partially, informed
from some external sources of information, therefore defining a MNAR
assumption. Two sources of external information are used to identify
these parameters: informative prior distributions and
distributional assumptions.
Informative priors on delta.e
and delta.c
can be provided in the form of a list object containing the hyperprior
values to be passed to the selection
function using the
optional argument prior
. By default, missingHE
specifies standard normal distributions on these parameters, which are
likely to have little impact compared to the results under MAR
(i.e. when the parameters are set to zero). Different values for the
priors on delta.e
and delta.c
can be passed to
the function to overwrite the default values to assess the robustness of
the results to different choices. The type of distributions for each
outcome can be specified among a pre-defined set of choices using the
arguments dist_e
and dist_c
. Type
help(selection)
to access the full list of available
distributions for \(e\) and \(c\).
For example, we can fit a selection model under MNAR assumptions for
the effectiveness variables (QALYs) in the MenSS
dataset
using the selection
function. We must set the argument
type = "MNAR"
and then add the terms e
inside
the model for the corresponding missingness indicator, namely
model.me = me ~ e
.
> NN.sel1=selection(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ model.me = me ~ e, model.mc = mc ~ 1, type = "MNAR",
+ n.iter = 1000, dist_e = "norm", dist_c = "norm")
The model assumes normal distributions for both outcomes and includes
the baseline utilities as covariates in the model of \(e\). Since we did not provide any prior,
default prior values are used for the MNAR parameter
delta.e
. missingHE
allows a flexible
specification in terms of the variables assumed to be MNAR (either \(e\), \(c\)
or both). In addition, other fully-observed variables can be included
into the models model.me
and model.mc
, either
under MAR or MNAR, to improve the estimation of the missingness
probabilities. We can retrieve the estimates for the mean effectiveness
and cost outcomes from the model using the print
function.
> print(NN.sel1)
mean sd 2.5% 97.5% Rhat n.eff
mu_c[1] 209.956 52.249 111.556 312.403 1.012 580
mu_c[2] 189.938 39.945 113.229 264.546 1.013 170
mu_e[1] 0.872 0.016 0.839 0.904 1.009 210
mu_e[2] 0.922 0.022 0.883 0.964 1.008 200
We now consider an alternative MNAR specification where we provide
some informative prior distributions on delta.e
. In
general, it is difficult to attach any specific interpretation to the
values for this parameter because its effect may vary depending on the
type of distributional assumptions made. We first define our prior
values by creating a list object called my.prior
. Within
this list, we create a vector of length two called
"delta.prior.e"
which contains the prior values to be
passed to selection
.
> my.prior <- list(
+ "delta.prior.e" = c(10, 1)
+ )
As a simple exercise, we increase the prior mean of
delta.e
to \(10\) to
assess the impact on posterior estimates of a more informative prior
about this parameter (relatively high positive value on the logit
scale). We then fit the second MNAR model using the new prior values by
setting the argument prior = my.prior
> NN.sel2=selection(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ model.me = me ~ e, model.mc = mc ~ 1, type = "MNAR",
+ n.iter = 1000, dist_e = "norm", dist_c = "norm", prior = my.prior)
We can now check the results and compare them to those obtained under the first MNAR model.
> print(NN.sel2)
mean sd 2.5% 97.5% Rhat n.eff
mu_c[1] 208.152 52.202 108.603 307.084 1.001 1000
mu_c[2] 190.555 38.189 115.044 267.196 1.000 1000
mu_e[1] 0.974 0.036 0.913 1.055 1.000 1000
mu_e[2] 0.979 0.034 0.930 1.053 1.012 1000
We see that, with respect to the results from NN.sel1
,
the mean effectiveness estimates are on average higher in both treatment
groups. To have a better idea of the impact in terms of
cost-effectiveness conclusions for the different MNAR assumptions, we
can use the function ceac.plot
inside the package
BCEA
to display the cost-effectiveness acceptability curves
based on the results from each model.
> par(mfrow=c(1,2))
> BCEA::ceac.plot(NN.sel1$cea)
> BCEA::ceac.plot(NN.sel2$cea)
The comparison between the two graphs shows that CEA conclusions are substantially affected by the specific assumptions made about the missing effects, therefore suggesting that the results of the model are not robust to the missingness assumptions considered. It is very important that the specific MNAR scenarios explored are informed based on some external information (e.g. expert opinion) so to provide a range of plausible assumptions to assess.
Pattern mixture models specify MNAR assumptions through the combinations of two elements: identifying restrictions and sensitivity parameters. Since these models are defined within each missingness pattern, parameters that cannot be idenfitied from the patterns are typically identified by imposing some modelling restrictions, that is they are set equal to the corresponding parameters from other patterns which can be identified from the observed data.
For example, the complete case restriction identifies all unidentified parameters in each pattern by setting them equal to those estimated from the complete cases. Under MAR, these restrictions are the only element used to achieve the identification of the model. However, when MNAR assumptions are specified, identifying restrictions are combined with sensitivity parameters, that is parameters that are entirely identified based on evidence external to the data, to achieve the identification of the model. Sensitivity parameters are identified based on informative prior distributions but, unlike the priors for the MNAR parameters in selection models, they have more natural interpretations in terms of the impact on the posterior results.
missingHE
allows the specification of MNAR assumptions
for either or both outcome variables using the function
pattern
via the arguments restriction
,
Delta_e
and Delta_c
. The first is the type of
identifying restrictions imposed: available choices are complete case
("CC"
) or available case ("AC"
) restrictions.
The second and third are the prior values for the sensitivity parameters
associated with the mean effectiveness and costs. Under MAR, these are
set to \(0\) (default values). Under
MNAR, prior values for these parameters must be provided by the user in
the form of a \(2\times2\) matrix. For
example, assuming a MNAR mechanism for the effectiveness, a possible
choice for the prior values on Delta_e
is
> Delta_e <- matrix(NA, 2, 2)
> Delta_e[1, ] <- c(- 0.3, - 0.2)
> Delta_e[2, ] <- c(-0.1, 0)
The rows represent the treatment group while the columns represent
the lower and upper bounds for the uniform prior distributions that
missingHE
assumes for these parameters under MNAR. The
values specified aboved correspond to assuming that, on average, we
expect the mean effectiveness in the control group is between \(0.3\) and \(0.2\) lower and that the mean effectiveness
in the intervention group is between \(0.1\) and \(0\) lower than the corresponding values
under MAR. We proceed to fit the MNAR pattern mixture model using
pattern
by setting the argument type = "MNAR"
and by passing our prior values contained in the object
Delta_e
to the function.
> NN.pat2=pattern(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ type = "MNAR", restriction = "CC", n.iter = 1000, Delta_e = Delta_e, Delta_c = 0,
+ dist_e = "norm", dist_c = "norm")
The function includes the baseline utilities in the model for \(e\) and achieves identification under MNAR
using complete case restrictions (restriction = "CC"
) and
informative priors on the sensitivity parameters for the mean \(e\) (Delta_e = Delta_e
).
Economic results in terms of posterior summaries about the mean
parameters from the model can be seen using the print
function
> print(NN.pat2)
mean sd 2.5% 97.5% Rhat n.eff
mu_c[1] 208.837 53.806 103.712 319.379 1.007 1000
mu_c[2] 189.257 39.142 109.332 267.530 1.001 1000
mu_e[1] 0.717 0.029 0.664 0.772 1.018 100
mu_e[2] 0.878 0.031 0.819 0.940 1.000 1000
For comparison, we also fit the same model under MAR, by typing
> NN.pat1=pattern(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ type = "MAR", restriction = "CC", n.iter = 1000, Delta_e = 0, Delta_c = 0,
+ dist_e = "norm", dist_c = "norm")
We assess the impact on the cost-effectiveness results between the
MNAR and MAR models by looking at the acceptability curves associated
with each model using again the function ceac.plot
inside
the package BCEA
.
> par(mfrow=c(1,2))
> BCEA::ceac.plot(NN.pat1$cea)
> BCEA::ceac.plot(NN.pat2$cea)
Results under MNAR (NN.pat2
) clearly suggest a higher
chance for the new intervention to be cost-effective compared with those
from the MAR model (NN.pat1
). This is in accordance with
our MNAR assumptions under which we expect, on average, lower QALYs in
the control with respect to the intervention group compared with the
results under MAR (when Delta_e = 0
). The range of values
for the sensitivity parameters under MNAR should be informed based on
some external source of information (e.g. expert opinion) which can be
used to guide the choice of the values and the number of scenarios to
explore.
Even though, technically speaking, hurdle models cannot be qualified as missingness models, they can still be specified so to assess the impact of some MNAR assumptions on the posterior results. This can be achieved by making arbitrary assumptions about the number of individuals with missing outcomes who are assigned a structural value in the model.
Consider first a standard hurdle model specification under MAR. We
specify the model using hurdle
to handle both structural
ones and zeros in \(e\) and \(c\) from our economic data in
MenSS
(setting the arguments se = 1
and
sc = 0
).
> NN.hur1=hurdle(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ model.se = se ~ 1, model.sc = sc ~ 1, type = "SCAR", se = 1, sc = 0,
+ n.iter = 1000, dist_e = "norm", dist_c = "norm")
The model assumes that the mechanisms of the structural values in both outomes do not depend on any observed covariate, i.e. it is structural completely at random (SCAR). The function automatically assignes all individuals with an observed one and zero to the structural components of the effectiveness and cost mixture distributions, while all the remaining individuals are modelled using normal distributions. In general, we do not know to which component of the mixture individuals with a missing outcome value should be assigned, as this information cannot be obtained from the data. However, based on some external information that we may have, we can impose this assignment, which effectively corresponds to a MNAR mechanism.
We can perform this type of analysis in missingHE
by
first creating an indicator variable (called d_e
), telling
for each individual whether a structural value is observed
(d_e = 1
) not observed (d_e = 0
) or missing
(d_e = NA
). For example, focussing on the effectiveness
variables, we can obtain this indicator by typing
> d_e <- ifelse(MenSS$e == 1, 1, 0)
>
> #number of ones
> sum(d_e == 1, na.rm = T)
[1] 17
Next, for all or some of the individuals with a missing effect value,
we set the value of d_e = 1
to assign them to the
structural component of the hurdle model. For example, we may believe
that it is likely for all individuals aged \(< 22\) to be associated with a perfect
health status (i.e. e = 1
). We can obtain this by
typing
> myd_e <- ifelse(is.na(d_e) & MenSS$age < 22, 1, d_e)
>
> #number of ones
> sum(myd_e == 1, na.rm = T)
[1] 41
The number of individuals associated with \(e = 1\) has considerbly increased with
respect to that based on the observed data alone. We can now proceed to
fit our model using this new indicator variable for the structural
values of \(e\) by setting the optional
argument d_e = myd_e
.
> NN.hur2=hurdle(data = MenSS, model.eff = e ~ u.0, model.cost = c ~ 1,
+ model.se = se ~ 1, model.sc = sc ~ 1, type = "SCAR", se = 1, sc = 0,
+ n.iter = 1000, dist_e = "norm", dist_c = "norm", d_e = myd_e)
We can inspect the posterior results by typing,
> print(NN.hur2)
mean sd 2.5% 97.5% Rhat n.eff
mu_c[1] 203.030 50.336 117.465 312.938 1.000 1000
mu_c[2] 182.136 38.041 110.688 259.550 1.005 450
mu_e[1] 0.930 0.017 0.896 0.960 1.001 1000
mu_e[2] 0.950 0.018 0.911 0.980 1.008 240
and we can look at how imputations in each treatment group are
carried out based on our model using the generic plot
function.
> plot(NN.hur2, outcome = "effects")
As it is possible to see, for some individuals, imputed values are
essentially equal to one with very small credible intervals. These
imputations are due to the fact that the outcome values for these people
are assumed to be one with almost no uncertainty. Finally, we compare
the economic results from the two alternative hurdle models using the
ceac.plot
function from the BCEA
package.
> par(mfrow=c(1,2))
> BCEA::ceac.plot(NN.hur1$cea)
> BCEA::ceac.plot(NN.hur2$cea)
The probabiility of cost-effectiveness for the “standard” hurdle
model (NN.hur1
) remains stable around \(0.6\) for most willingness to pay values.
However, for the MNAR model (NN.hur2
), results indicate an
higher chance of cost-effectiveness up to about \(0.8\) for most threshold values. This is
due to the fact that, under our MNAR assumptions, the difference between
the treatment groups in terms of the number of individuals assigned to a
structural one is more in favour of the new intervention compared with
that under MAR.