Skip to main content

Advertisement

Table 1 Comparison of methods for total causal effects for simulated data with moderate variability ( σ=0.1 )

From: Joint estimation of causal effects from observational and intervention gene expression data

Setting Criterion MCMC-Mallows Pinna IDA (opt) IDA (pes)
  AUROC 0.749 (0.043) 0.76 (0.062) 0.643 (0.079)
  AUPRC 0.638 (0.053) 0.628 (0.078) 0.527 (0.088)
Observation only Spearman 0.48 (0.091) 0.491 (0.128) 0.254 (0.177)
  MSE 0.056 (0.007) 0.182 (0.054) 0.126 (0.034)
  AUROC 0.948 (0.03) 0.825 (0.048) 0.733 (0.068) 0.67 (0.073)
  AUPRC 0.868 (0.042) 0.737 (0.059) 0.569 (0.087) 0.53 (0.091)
Mixed Spearman 0.696 (0.053) 0.553 (0.097) 0.42 (0.14) 0.318 (0.186)
  MSE 0.026 (0.012) 0.104 (0.011) 0.334 (0.137) 0.196 (0.067)
  AUROC 0.845 (0.059) 0.795 (0.017) 0.736 (0.056) 0.646 (0.085)
  AUPRC 0.734 (0.078) 0.725 (0.038) 0.588 (0.075) 0.514 (0.092)
Partial KO Spearman 0.587 (0.104) 0.636 (0.034) 0.449 (0.099) 0.285 (0.187)
  MSE 0.035 (0.015) 0.081 (0.008) 0.215 (0.066) 0.146 (0.049)
  AUROC 0.959 (0.016) 0.83 (0.035) 0.733 (0.068) 0.67 (0.073)
  AUPRC 0.886 (0.028) 0.725 (0.039) 0.569 (0.087) 0.53 (0.091)
Multiple KO Spearman 0.712 (0.028) 0.625 (0.058) 0.42 (0.14) 0.318 (0.186)
  MSE 0.015 (0.006) 0.107 (0.008) 0.334 (0.137) 0.196 (0.067)
  AUROC 0.932 (0.046) 0.574 (0.165) 0.58 (0.145) 0.562 (0.121)
Multiple KO AUPRC 0.539 (0.078) 0.36 (0.105) 0.353 (0.086) 0.35 (0.08)
(3 hidden genes) Spearman 0.67 (0.109) 0.037 (0.372) 0.076 (0.316) 0.076 (0.31)
  MSE 0.044 (0.034) 0.15 (0.041) 0.45 (0.225) 0.294 (0.124)
  1. Several intervention designs were simulated: 1) 20 observational (wild-type) replicates with no interventions, 2) mixed setting with 10 wild-types and one knock-out per gene, 3) partial knock-out design with 15 wild-types and one knock-out for five genes {N1, N4, N6, N7, N9}, 4) multiple knock-out design with 10 wild types, one knock-out per gene and five double knock-outs: {N1, N5}, {N1, N6}, {N4, N7}, {N6, N9}, and {N7, N10}, and 5) a multiple knock-out design as in the previous setting, with three hidden variables. Results were averaged over 100 simulations (standard deviations in parentheses): area under the ROC curve (AUROC), area under the precision-recall curve (AUPRC), Spearman correlation between true and estimated total causal effects, and mean squared error (MSE) of estimated total causal effects.