DOI: 10.1021/ci200211n. Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient

Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient

10.1021/ci200211n

Crossref journal-article

American Chemical Society (ACS)

Journal of Chemical Information and Modeling (316)

Bibliography

Chirico, N., & Gramatica, P. (2011). Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient. Journal of Chemical Information and Modeling, 51(9), 2320â2335.

Authors 2

Nicola Chirico (first)
Paola Gramatica (additional)

References 42 Referenced 644

10.1016/S1093-3263(01)00123-1 / J. Mol. Graphics Modell. by Golbraikh A. (2002)
10.1002/1521-3838(200210)21:4<348::AID-QSAR348>3.0.CO;2-D / Quant. Struct-Act. Relat. by Kubinyi H. (2002)
10.1016/S0165-9936(03)00607-1 / Trends Anal. Chem. by Baumann K (2003)
10.1002/qsar.200390007 / QSAR Comb. Sci. by Tropsha A. (2003)
10.1007/s10822-004-4071-5 / J. Comput.-Aided Mol. Des. by Baumann K. (2004)
10.1002/qsar.200610151 / QSAR Comb. Sci. by Gramatica P. (2007)
10.1517/17460441.2.12.1567 / Expert Opin. Drug Discovery by Roy K. (2007)
10.1590/S0103-50532009000400021 / J. Braz. Chem. Soc. by Kiralj R. (2009)
10.1002/minf.201000061 / Mol. Inf. by Tropsha A. (2010)
10.1021/ci0342472 / J. Chem. Inf. Comput. Sci. by Hawkins D. M. (2004)
10.1080/10629360802349058 / SAR QSAR Environ. Res. by Hawkins D. M. (2008)
10.1080/10629360412331297407 / SAR QSAR Environ. Res. by Helma C. (2004)
10.1080/10629360802550473 / SAR QSAR Environ. Res. by Jensen G. E. (2008)
10.1023/A:1025386326946 / J. Comput.-Aided Mol. Des. by Golbraikh A. (2003)
10.1021/ci049923u / J. Chem. Inf. Comput. Sci. by Gramatica P. (2004)
10.1002/qsar.200510161 / QSAR Comb. Sci. by Leonard J. T. (2006)
10.1021/tx900252h / Chem. Res. Toxicol. by Bhhatarai B. (2010)
10.1021/ci000066d / J. Chem. Inf. Comput. Sci. by Shi L. M. (2001)
10.1021/ci800253u / J. Chem. Inf. Model. by Schüürmann G. (2008)
10.1021/ci900115y / J. Chem. Inf. Model. by Consonni V. (2009)
10.1002/cem.1290 / J. Chemom. by Consonni V. (2010)
10.1002/qsar.200710043 / QSAR Comb. Sci. by Roy P. P. (2008)
10.3390/molecules14051660 / Molecules by Roy P. P. (2009)
10.1002/cem.1268 / J. Chemom. by Mitra I. (2010)
10.1016/j.chemolab.2011.03.011 / Chemom. Intell. Lab. Syst. by Ojha P. K. (2011)
{'key': 'ref26/cit26', 'volume-title': 'MOBY DIGS', 'author': 'Todeschini R.', 'year': '2002'} / MOBY DIGS by Todeschini R. (2002)
Organization for Economic Co-operation and Development (OECD). Guidance document on the validation of (quantitative) structure-activity relationship [(Q)SAR] models, 2007. OECD Web Site.http://www.oecd.org/officialdocuments/displaydocumentpdf/?cote=env/jm/mono%282007%292&doclanguage=en(accessed July 18, 2011).
10.1002/qsar.200430909 / QSAR Comb. Sci. by Aptula O. A. (2005)
10.1002/minf.201000011 / Mol. Inf. by Bhhatarai B. (2010)
10.1007/s11030-009-9212-2 / Mol. Diversity by Li J. (2010)
10.1007/s11030-010-9268-z / Mol. Diversity by Bhhatarai B. (2011)
10.1016/j.watres.2010.11.006 / Water Res. by Bhhatarai B. (2011)
10.1002/minf.201000133 / Mol. Inf. by Bhhatarai B. (2011)
10.2307/2532051 / Biometrics by Lin L. I. (1989)
10.2307/2532314 / Biometrics by Lin L. I. (1992)
Benigni, R.QSARs for Mutagenicity and Carcinogenicity. In The report from the expert group on (quantitative) structure-activity relationships [(Q)SARs] on the principles for the validation of (Q)SARs, 2004, pp 84–100. OECD Web Site.http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=ENV/JM/MONO(2004)24&docLanguage=En(accessed July 18, 2011).
10.1080/10629360601054388 / SAR QSAR Environ. Res. by Gramatica P. (2007)
ChemIDplus Advanced.http://chem.sis.nlm.nih.gov/chemidplus(accessed July 18, 2011).
10.1021/ie50517a034 / Ind. Eng. Chem. by Hendricks J. O. (1953)
10.1016/j.chemolab.2007.07.004 / Chemom. Intell. Lab. Syst. by Roy P. P. (2008)
10.1016/S0169-7439(99)00027-1 / Chemom. Intell. Lab. Syst. by Faber N. M. (1999)
{'key': 'ref42/cit42', 'volume-title': 'QSARINS, software for QSAR model calculation and validation, under development', 'author': 'Chirico N.', 'year': '2011'} / QSARINS, software for QSAR model calculation and validation, under development by Chirico N. (2011)

Dates

Type	When
Created	14 years ago (July 29, 2011, 9:57 a.m.)
Deposited	2 years, 4 months ago (April 15, 2023, 5:50 a.m.)
Indexed	16 minutes ago (Aug. 24, 2025, 2:49 p.m.)
Issued	14 years ago (Aug. 12, 2011)
Published	14 years ago (Aug. 12, 2011)
Published Online	14 years ago (Aug. 12, 2011)
Published Print	13 years, 10 months ago (Sept. 26, 2011)

Funders 0

None

BibTeX

@article{Chirico_2011, title={Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient}, volume={51}, ISSN={1549-960X}, url={http://dx.doi.org/10.1021/ci200211n}, DOI={10.1021/ci200211n}, number={9}, journal={Journal of Chemical Information and Modeling}, publisher={American Chemical Society (ACS)}, author={Chirico, Nicola and Gramatica, Paola}, year={2011}, month=aug, pages={2320–2335} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        24
      ]
    ],
    "date-time": "2025-08-24T18:49:14Z",
    "timestamp": 1756061354904
  },
  "reference-count": 42,
  "publisher": "American Chemical Society (ACS)",
  "issue": "9",
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2011,
        9,
        26
      ]
    ]
  },
  "DOI": "10.1021/ci200211n",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2011,
        7,
        29
      ]
    ],
    "date-time": "2011-07-29T13:57:22Z",
    "timestamp": 1311947842000
  },
  "page": "2320-2335",
  "source": "Crossref",
  "is-referenced-by-count": 644,
  "title": "Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient",
  "prefix": "10.1021",
  "volume": "51",
  "author": [
    {
      "given": "Nicola",
      "family": "Chirico",
      "sequence": "first",
      "affiliation": [
        {
          "name": "QSAR Research Group in Environmental Chemistry and Ecotoxicology, Department of Structural and Functional Biology, University of Insubria, Via Dunant 3, 21100, Varese, Italy"
        }
      ]
    },
    {
      "given": "Paola",
      "family": "Gramatica",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "QSAR Research Group in Environmental Chemistry and Ecotoxicology, Department of Structural and Functional Biology, University of Insubria, Via Dunant 3, 21100, Varese, Italy"
        }
      ]
    }
  ],
  "member": "316",
  "published-online": {
    "date-parts": [
      [
        2011,
        8,
        12
      ]
    ]
  },
  "reference": [
    {
      "key": "ref1/cit1",
      "doi-asserted-by": "crossref",
      "first-page": "269",
      "DOI": "10.1016/S1093-3263(01)00123-1",
      "volume": "20",
      "author": "Golbraikh A.",
      "year": "2002",
      "journal-title": "J. Mol. Graphics Modell."
    },
    {
      "key": "ref2/cit2",
      "doi-asserted-by": "crossref",
      "first-page": "348",
      "DOI": "10.1002/1521-3838(200210)21:4<348::AID-QSAR348>3.0.CO;2-D",
      "volume": "21",
      "author": "Kubinyi H.",
      "year": "2002",
      "journal-title": "Quant. Struct-Act. Relat."
    },
    {
      "key": "ref3/cit3",
      "doi-asserted-by": "crossref",
      "first-page": "395",
      "DOI": "10.1016/S0165-9936(03)00607-1",
      "volume": "22",
      "author": "Baumann K",
      "year": "2003",
      "journal-title": "Trends Anal. Chem."
    },
    {
      "key": "ref4/cit4",
      "doi-asserted-by": "crossref",
      "first-page": "69",
      "DOI": "10.1002/qsar.200390007",
      "volume": "22",
      "author": "Tropsha A.",
      "year": "2003",
      "journal-title": "QSAR Comb. Sci."
    },
    {
      "key": "ref5/cit5",
      "doi-asserted-by": "crossref",
      "first-page": "549",
      "DOI": "10.1007/s10822-004-4071-5",
      "volume": "18",
      "author": "Baumann K.",
      "year": "2004",
      "journal-title": "J. Comput.-Aided Mol. Des."
    },
    {
      "key": "ref6/cit6",
      "doi-asserted-by": "crossref",
      "first-page": "694",
      "DOI": "10.1002/qsar.200610151",
      "volume": "5",
      "author": "Gramatica P.",
      "year": "2007",
      "journal-title": "QSAR Comb. Sci."
    },
    {
      "key": "ref7/cit7",
      "doi-asserted-by": "crossref",
      "first-page": "1567",
      "DOI": "10.1517/17460441.2.12.1567",
      "volume": "2",
      "author": "Roy K.",
      "year": "2007",
      "journal-title": "Expert Opin. Drug Discovery"
    },
    {
      "key": "ref8/cit8",
      "doi-asserted-by": "crossref",
      "first-page": "770",
      "DOI": "10.1590/S0103-50532009000400021",
      "volume": "20",
      "author": "Kiralj R.",
      "year": "2009",
      "journal-title": "J. Braz. Chem. Soc."
    },
    {
      "key": "ref9/cit9",
      "doi-asserted-by": "crossref",
      "first-page": "476",
      "DOI": "10.1002/minf.201000061",
      "volume": "29",
      "author": "Tropsha A.",
      "year": "2010",
      "journal-title": "Mol. Inf."
    },
    {
      "key": "ref10/cit10",
      "doi-asserted-by": "crossref",
      "first-page": "1",
      "DOI": "10.1021/ci0342472",
      "volume": "44",
      "author": "Hawkins D. M.",
      "year": "2004",
      "journal-title": "J. Chem. Inf. Comput. Sci."
    },
    {
      "key": "ref11/cit11",
      "doi-asserted-by": "crossref",
      "first-page": "525",
      "DOI": "10.1080/10629360802349058",
      "volume": "16",
      "author": "Hawkins D. M.",
      "year": "2008",
      "journal-title": "SAR QSAR Environ. Res."
    },
    {
      "key": "ref12/cit12",
      "doi-asserted-by": "crossref",
      "first-page": "367",
      "DOI": "10.1080/10629360412331297407",
      "volume": "15",
      "author": "Helma C.",
      "year": "2004",
      "journal-title": "SAR QSAR Environ. Res."
    },
    {
      "key": "ref13/cit13",
      "doi-asserted-by": "crossref",
      "first-page": "631",
      "DOI": "10.1080/10629360802550473",
      "volume": "19",
      "author": "Jensen G. E.",
      "year": "2008",
      "journal-title": "SAR QSAR Environ. Res."
    },
    {
      "key": "ref14/cit14",
      "doi-asserted-by": "crossref",
      "first-page": "241",
      "DOI": "10.1023/A:1025386326946",
      "volume": "17",
      "author": "Golbraikh A.",
      "year": "2003",
      "journal-title": "J. Comput.-Aided Mol. Des."
    },
    {
      "key": "ref15/cit15",
      "doi-asserted-by": "crossref",
      "first-page": "1794",
      "DOI": "10.1021/ci049923u",
      "volume": "44",
      "author": "Gramatica P.",
      "year": "2004",
      "journal-title": "J. Chem. Inf. Comput. Sci."
    },
    {
      "key": "ref16/cit16",
      "doi-asserted-by": "crossref",
      "first-page": "235",
      "DOI": "10.1002/qsar.200510161",
      "volume": "3",
      "author": "Leonard J. T.",
      "year": "2006",
      "journal-title": "QSAR Comb. Sci."
    },
    {
      "key": "ref17/cit17",
      "doi-asserted-by": "crossref",
      "first-page": "528",
      "DOI": "10.1021/tx900252h",
      "volume": "23",
      "author": "Bhhatarai B.",
      "year": "2010",
      "journal-title": "Chem. Res. Toxicol."
    },
    {
      "key": "ref18/cit18",
      "doi-asserted-by": "crossref",
      "first-page": "186",
      "DOI": "10.1021/ci000066d",
      "volume": "41",
      "author": "Shi L. M.",
      "year": "2001",
      "journal-title": "J. Chem. Inf. Comput. Sci."
    },
    {
      "key": "ref19/cit19",
      "doi-asserted-by": "crossref",
      "first-page": "2140",
      "DOI": "10.1021/ci800253u",
      "volume": "48",
      "author": "Sch\u00fc\u00fcrmann G.",
      "year": "2008",
      "journal-title": "J. Chem. Inf. Model."
    },
    {
      "key": "ref20/cit20",
      "doi-asserted-by": "crossref",
      "first-page": "1669",
      "DOI": "10.1021/ci900115y",
      "volume": "49",
      "author": "Consonni V.",
      "year": "2009",
      "journal-title": "J. Chem. Inf. Model."
    },
    {
      "key": "ref21/cit21",
      "doi-asserted-by": "crossref",
      "first-page": "194",
      "DOI": "10.1002/cem.1290",
      "volume": "24",
      "author": "Consonni V.",
      "year": "2010",
      "journal-title": "J. Chemom."
    },
    {
      "key": "ref22/cit22",
      "doi-asserted-by": "crossref",
      "first-page": "302",
      "DOI": "10.1002/qsar.200710043",
      "volume": "27",
      "author": "Roy P. P.",
      "year": "2008",
      "journal-title": "QSAR Comb. Sci."
    },
    {
      "key": "ref23/cit23",
      "doi-asserted-by": "crossref",
      "first-page": "1660",
      "DOI": "10.3390/molecules14051660",
      "volume": "14",
      "author": "Roy P. P.",
      "year": "2009",
      "journal-title": "Molecules"
    },
    {
      "key": "ref24/cit24",
      "doi-asserted-by": "crossref",
      "first-page": "22",
      "DOI": "10.1002/cem.1268",
      "volume": "24",
      "author": "Mitra I.",
      "year": "2010",
      "journal-title": "J. Chemom."
    },
    {
      "key": "ref25/cit25",
      "doi-asserted-by": "crossref",
      "first-page": "194",
      "DOI": "10.1016/j.chemolab.2011.03.011",
      "volume": "107",
      "author": "Ojha P. K.",
      "year": "2011",
      "journal-title": "Chemom. Intell. Lab. Syst."
    },
    {
      "key": "ref26/cit26",
      "volume-title": "MOBY DIGS",
      "author": "Todeschini R.",
      "year": "2002"
    },
    {
      "key": "ref27/cit27",
      "unstructured": "Organization for Economic Co-operation and Development (OECD). Guidance document on the validation of (quantitative) structure-activity relationship [(Q)SAR] models, 2007. OECD Web Site.http://www.oecd.org/officialdocuments/displaydocumentpdf/?cote=env/jm/mono%282007%292&doclanguage=en(accessed July 18, 2011)."
    },
    {
      "key": "ref28/cit28",
      "doi-asserted-by": "crossref",
      "first-page": "385",
      "DOI": "10.1002/qsar.200430909",
      "volume": "24",
      "author": "Aptula O. A.",
      "year": "2005",
      "journal-title": "QSAR Comb. Sci."
    },
    {
      "key": "ref29/cit29",
      "doi-asserted-by": "crossref",
      "first-page": "511",
      "DOI": "10.1002/minf.201000011",
      "volume": "29",
      "author": "Bhhatarai B.",
      "year": "2010",
      "journal-title": "Mol. Inf."
    },
    {
      "key": "ref30/cit30",
      "doi-asserted-by": "crossref",
      "first-page": "687",
      "DOI": "10.1007/s11030-009-9212-2",
      "volume": "14",
      "author": "Li J.",
      "year": "2010",
      "journal-title": "Mol. Diversity"
    },
    {
      "key": "ref31/cit31",
      "doi-asserted-by": "crossref",
      "first-page": "467",
      "DOI": "10.1007/s11030-010-9268-z",
      "volume": "15",
      "author": "Bhhatarai B.",
      "year": "2011",
      "journal-title": "Mol. Diversity"
    },
    {
      "key": "ref32/cit32",
      "doi-asserted-by": "crossref",
      "first-page": "1463",
      "DOI": "10.1016/j.watres.2010.11.006",
      "volume": "45",
      "author": "Bhhatarai B.",
      "year": "2011",
      "journal-title": "Water Res."
    },
    {
      "key": "ref33/cit33",
      "doi-asserted-by": "crossref",
      "first-page": "189",
      "DOI": "10.1002/minf.201000133",
      "volume": "30",
      "author": "Bhhatarai B.",
      "year": "2011",
      "journal-title": "Mol. Inf."
    },
    {
      "key": "ref34/cit34",
      "doi-asserted-by": "crossref",
      "first-page": "255",
      "DOI": "10.2307/2532051",
      "volume": "45",
      "author": "Lin L. I.",
      "year": "1989",
      "journal-title": "Biometrics"
    },
    {
      "key": "ref35/cit35",
      "doi-asserted-by": "crossref",
      "first-page": "599",
      "DOI": "10.2307/2532314",
      "volume": "48",
      "author": "Lin L. I.",
      "year": "1992",
      "journal-title": "Biometrics"
    },
    {
      "key": "ref36/cit36",
      "unstructured": "Benigni, R.QSARs for Mutagenicity and Carcinogenicity. In The report from the expert group on (quantitative) structure-activity relationships [(Q)SARs] on the principles for the validation of (Q)SARs, 2004, pp 84\u2013100. OECD Web Site.http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=ENV/JM/MONO(2004)24&docLanguage=En(accessed July 18, 2011)."
    },
    {
      "key": "ref37/cit37",
      "doi-asserted-by": "crossref",
      "first-page": "169",
      "DOI": "10.1080/10629360601054388",
      "volume": "18",
      "author": "Gramatica P.",
      "year": "2007",
      "journal-title": "SAR QSAR Environ. Res."
    },
    {
      "key": "ref38/cit38",
      "unstructured": "ChemIDplus Advanced.http://chem.sis.nlm.nih.gov/chemidplus(accessed July 18, 2011)."
    },
    {
      "key": "ref39/cit39",
      "doi-asserted-by": "crossref",
      "first-page": "99",
      "DOI": "10.1021/ie50517a034",
      "volume": "45",
      "author": "Hendricks J. O.",
      "year": "1953",
      "journal-title": "Ind. Eng. Chem."
    },
    {
      "key": "ref40/cit40",
      "doi-asserted-by": "crossref",
      "first-page": "31",
      "DOI": "10.1016/j.chemolab.2007.07.004",
      "volume": "90",
      "author": "Roy P. P.",
      "year": "2008",
      "journal-title": "Chemom. Intell. Lab. Syst."
    },
    {
      "key": "ref41/cit41",
      "doi-asserted-by": "crossref",
      "first-page": "79",
      "DOI": "10.1016/S0169-7439(99)00027-1",
      "volume": "49",
      "author": "Faber N. M.",
      "year": "1999",
      "journal-title": "Chemom. Intell. Lab. Syst."
    },
    {
      "key": "ref42/cit42",
      "volume-title": "QSARINS, software for QSAR model calculation and validation, under development",
      "author": "Chirico N.",
      "year": "2011"
    }
  ],
  "container-title": "Journal of Chemical Information and Modeling",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://pubs.acs.org/doi/pdf/10.1021/ci200211n",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2023,
        4,
        15
      ]
    ],
    "date-time": "2023-04-15T09:50:09Z",
    "timestamp": 1681552209000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://pubs.acs.org/doi/10.1021/ci200211n"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2011,
        8,
        12
      ]
    ]
  },
  "references-count": 42,
  "journal-issue": {
    "issue": "9",
    "published-print": {
      "date-parts": [
        [
          2011,
          9,
          26
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1021/ci200211n"
  ],
  "URL": "http://dx.doi.org/10.1021/ci200211n",
  "relation": {},
  "ISSN": [
    "1549-9596",
    "1549-960X"
  ],
  "subject": [],
  "container-title-short": "J. Chem. Inf. Model.",
  "published": {
    "date-parts": [
      [
        2011,
        8,
        12
      ]
    ]
  }
}