Skip to main content

Table 4 Performance statistics of mutation effect prediction algorithms using only functionally tested single nucleotide variants not present in the COSMIC database (n = 297)

From: Benchmarking mutation effect prediction algorithms using functionally validated cancer-related missense mutations

Prediction algorithm

Accuracy (95% CI)

Sensitivity (95% CI)

Specificity (95% CI)

PPV (95% CI)

NPV (95% CI)

Composite (95% CI)

CHASM (breast)

80.13% (75.42-84.18%)

72.34% (65.70-78.34%)

93.58% (88.89-97.96%)

95.10% (91.27-98.47%)

66.23% (58.62-73.42%)

3.2726 (3.1090-3.4151)

CHASM (lung)

82.49% (78.11-86.53%)

79.26% (73.26-84.95%)

88.07% (81.98-94.18%)

91.98% (87.58-96.20%)

71.11% (63.57-78.74%)

3.3042 (3.1370-3.4665)

CHASM (melanoma)

81.82% (77.44-86.20%)

86.17% (80.93-90.96%)

74.31% (65.71-82.46%)

85.26% (80.21-90.00%)

75.70% (67.29-83.91%)

3.2145 (3.0259-3.3971)

FATHMM (cancer)

80.13% (75.42-84.51%)

92.02% (87.98-95.83%)

59.63% (50.43-69.00%)

79.72% (74.21-84.82%)

81.25% (72.06-89.22%)

3.1263 (2.9242-3.3078)

FATHMM (missense)

75.42% (70.71-79.80%)

86.70% (81.82-91.37%)

55.96% (46.29-64.82%)

77.25% (71.67-82.33%)

70.93% (60.92-80.23%)

2.9085 (2.7014-3.1033)

Mutation Assessor

68.69% (63.64-73.40%)

71.28% (64.77-77.25%)

64.22% (54.46-72.81%)

77.46% (71.26-82.93%)

56.45% (47.82-64.87%)

2.6941 (2.4757-2.8892)

MutationTaster

84.51% (80.13-88.55%)

84.04% (78.53-89.25%)

85.32% (78.18-91.75%)

90.80% (86.05-95.00%)

75.61% (67.42-82.88%)

3.3578 (3.1879-3.5244)

PolyPhen-2

69.36% (64.31-74.41%)

78.72% (73.16-84.53%)

53.21% (43.59-62.50%)

74.37% (68.08-80.21%)

59.18% (49.47-69.01%)

2.6549 (2.4296-2.8672)

PROVEAN

58.59% (52.86-63.64%)

51.06% (43.78-57.73%)

71.56% (62.83-80.00%)

75.59% (68.61-82.81%)

45.88% (38.61-53.76%)

2.4410 (2.2263-2.6521)

SIFT

69.70% (64.65-74.75%)

81.91% (76.09-87.57%)

48.62% (38.89-58.62%)

73.33% (67.31-78.92%)

60.92% (51.14-71.64%)

2.6479 (2.4175-2.8681)

VEST

54.88% (49.49-60.27%)

71.81% (65.80-77.89%)

25.69% (18.01-34.19%)

62.50% (56.34-68.78%)

34.57% (24.99-45.21%)

1.9456 (1.7302-2.1701)

CanDrA (breast)

61.69% (55.89-66.78%)

97.85% (95.50-99.48%)

0% (0-0%)

62.54% (57.04-67.82%)

0% (0-0%)

1.6039 (1.5437-1.6593)

CanDrA (lung)

84.70% (80.50-88.65%)

79.12% (73.21-84.57%)

94.95% (89.81-98.94%)

96.64% (93.29-99.31%)

71.21% (63.41-78.30%)

3.4193 (3.2675-3.5628)

CanDrA (melanoma)

82.61% (77.94-86.64%)

78.09% (71.43-83.82%)

90.82% (85.06-96.00%)

93.92% (89.86-97.33%)

69.53% (61.10-77.05%)

3.3236 (3.1491-3.4780)

Condel

76.77% (72.05-81.48%)

93.09% (89.36-96.32%)

48.62% (39.62-58.33%)

75.76% (70.38-81.03%)

80.30% (70.58-89.55%)

2.9777 (2.7814-3.1877)

  1. Based on the prediction results of non-neutral and neutral single nucleotide variants not present in the COSMIC database, the accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and composite score for each predictor were computed. The 95% confidence intervals (CI) generated by bootstrapping are shown in parentheses.