DOI: 10.1002/minf.201000159. Development and Comparison of hERG Blocker Classifiers: Assessment on Different Datasets Yields Markedly Different Results

Development and Comparison of hERG Blocker Classifiers: Assessment on Different Datasets Yields Markedly Different Results

10.1002/minf.201000159

Crossref journal-article

Wiley

Molecular Informatics (311)

Abstract

AbstractIn recent years, considerable effort has been invested in the development of classification models for prospective hERG inhibitors, due to the implications of hERG blockade for cardiotoxicity and the low throughput of functional hERG assays. We present novel approaches for binary classification which seek to separate strong inhibitors (IC50<1 µM) from ‘non‐blockers′ exhibiting moderate (1–10 µM) or weak (IC50≥10 µM) inhibition, as required by the pharmaceutical industry. Our approaches are based on (discretized) 2D descriptors, selected using Winnow, with additional models generated using Random Forest (RF) and Support Vector Machines (SVMs). We compare our models to those previously developed by Thai and Ecker and by Dubus et al. The purpose of this paper is twofold: 1. To propose that our approaches (with Matthews Correlation Coefficients from 0.40 to 0.87 on truly external test sets, when extrapolation beyond the applicability domain was not evident and sufficient quantities of data were available for training) are competitive with those currently proposed in the literature. 2. To highlight key issues associated with building and assessing truly predictive models, in particular the considerable variation in model performance when training and testing on different datasets.

Authors 3

Richard L. Marchese Robinson (first)
Robert C. Glen (additional)
John B. O. Mitchell (additional)

References 57 Referenced 35

10.1038/nature04710
10.1093/eurheartj/ehi312
ICH Anon. ICH S7B Guideline (Step 4 Version): “The Nonclinical Evaluation of the Potential for Delayed Ventricular Repolarization (QT Interval Prolongation) By Human Pharmaceuticals” to be found under http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Safety/S7B/Step4/S7B_Guideline.pdf 2005.
10.1007/s10822-006-9095-6
10.1002/jat.1395
10.1016/j.vascn.2006.02.003
10.1002/cmdc.200700264
10.2174/157340908784533256
10.1007/s11030-009-9117-0
H. H. Tie MD thesis University of New South Wales (Australia) 2002.
10.1038/bjp.2008.267
10.1016/S0008-6363(02)00846-5
10.1021/tx800063r
10.1002/1439-7633(20020503)3:5<455::AID-CBIC455>3.0.CO;2-L
10.1002/cmdc.200500099
10.1016/j.bmc.2008.01.017
10.1002/cmdc.201000024
10.1021/mp700124e
10.1016/j.bmcl.2005.03.080
10.1016/j.bmc.2007.06.028
10.1016/j.vascn.2004.06.003
10.1111/j.1365-2125.2008.03327.x
10.1016/j.vascn.2005.04.002
10.1021/jm060500o
J. K. Martin D. S. Hirschberg “Small Sample Statistics for Classification Error Rates I: Error Rate Measurements” to be found under http://www.ics.uci.edu/∼dan/pub.html 1996.
{'key': 'e_1_2_9_26_2', 'first-page': '479', 'volume-title': 'Information Theory, Inference and Learning Algorithms', 'author': 'MacKay D. J. C.', 'year': '2003'} / Information Theory, Inference and Learning Algorithms by MacKay D. J. C. (2003)
{'key': 'e_1_2_9_27_2', 'first-page': '1661', 'volume-title': 'Proc. IJCNN‐2006, Vancouver, BC, Canada', 'author': 'Cawley G. C.', 'year': '2006'} / Proc. IJCNN‐2006, Vancouver, BC, Canada by Cawley G. C. (2006)
10.1016/j.jspi.2007.06.003
10.1093/bioinformatics/16.5.412
10.1016/j.compbiolchem.2004.09.006
10.1021/ci700350n
10.1021/ci800079x
C.‐W. Hsu C.‐C. Chang C.‐J. Lin “A Practical Guide to Support Vector Classification” to be found under http://www.csie.ntu.edu.tw/∼cjlin/papers/guide/guide.pdf 2010.
K. Morik P. Brockhausen T. Joachims inProc. ICML‐99 Bled Slovenia 1999 pp. 268–277 to be found under http://www.cs.cornell.edu/People/tj/publications/morik_etal_99a.pdf.
{'key': 'e_1_2_9_35_2', 'first-page': '169', 'volume-title': 'Advances in Kernel Methods – Support Vector Learning', 'author': 'Joachims T.', 'year': '1999'} / Advances in Kernel Methods – Support Vector Learning by Joachims T. (1999)
10.1023/A:1010933404324
The R Project for Statistical Computing http://www.r‐project.org/.
10.1021/ci034160g
10.1021/ci100050t
Python Programming Language(v.2.5.2) http://www.python.org/.
U. M. Fayyad K. B. Irani inProc. IJCAI‐93 Chambery France 1993 pp. 1022–1027 available from http://sci2s.ugr.es/keel/pdf/algorithm/congreso/fayyad1993.pdf.
J. Demsar B. Zupan “Orange: From Experimental Machine Learning to Interactive Data Mining” White Paper to be found under http://www.ailab.si/orange/wp/orange.pdf 2004. (10.1007/978-3-540-30116-5_58)
10.1016/S1093-3263(00)00068-1
Chemaxon Budapest Hungary http://www.chemaxon.com.
Pipeline Pilot Student Edition(v. 6.1.5) Accelrys San Diego CA USA http://accelrys.com/products/pipeline‐pilot/.
10.1186/1752-153X-2-5
MOE (Molecular Operating Environment) v. 2008.10 Chemical Computing Group Montreal Canada http://www.chemcomp.com.
10.1021/ci00062a008
M. Kubat S. Matwin inProc. ICML‐97 1997 pp. 179–186 available from http://sci2s.ugr.es/keel/pdf/algorithm/congreso/kubat97adressing.pdf.
10.1186/1471-2105-10-213
10.1021/ci050308f
10.1021/jm060379l
10.1021/jm801236n
10.2174/1568016033477432
10.1038/sj.bjp.0705336
10.1002/cmdc.200900374
10.1021/jm900002x

Dates

Type	When
Created	14 years, 3 months ago (May 6, 2011, 7:35 a.m.)
Deposited	1 year, 10 months ago (Oct. 9, 2023, 3:18 p.m.)
Indexed	2 days, 17 hours ago (Aug. 31, 2025, 6:25 a.m.)
Issued	14 years, 3 months ago (May 6, 2011)
Published	14 years, 3 months ago (May 6, 2011)
Published Online	14 years, 3 months ago (May 6, 2011)
Published Print	14 years, 3 months ago (May 16, 2011)

Funders 0

None

BibTeX

@article{Marchese_Robinson_2011, title={Development and Comparison of hERG Blocker Classifiers: Assessment on Different Datasets Yields Markedly Different Results}, volume={30}, ISSN={1868-1751}, url={http://dx.doi.org/10.1002/minf.201000159}, DOI={10.1002/minf.201000159}, number={5}, journal={Molecular Informatics}, publisher={Wiley}, author={Marchese Robinson, Richard L. and Glen, Robert C. and Mitchell, John B. O.}, year={2011}, month=may, pages={443–458} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        31
      ]
    ],
    "date-time": "2025-08-31T10:25:55Z",
    "timestamp": 1756635955046
  },
  "reference-count": 57,
  "publisher": "Wiley",
  "issue": "5",
  "license": [
    {
      "start": {
        "date-parts": [
          [
            2011,
            5,
            6
          ]
        ],
        "date-time": "2011-05-06T00:00:00Z",
        "timestamp": 1304640000000
      },
      "content-version": "vor",
      "delay-in-days": 0,
      "URL": "http://onlinelibrary.wiley.com/termsAndConditions#vor"
    }
  ],
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2011,
        5,
        16
      ]
    ]
  },
  "abstract": "<jats:title>Abstract</jats:title><jats:p>In recent years, considerable effort has been invested in the development of classification models for prospective hERG inhibitors, due to the implications of hERG blockade for cardiotoxicity and the low throughput of functional hERG assays. We present novel approaches for binary classification which seek to separate strong inhibitors (<jats:italic>IC</jats:italic><jats:sub>50</jats:sub>&lt;1\u2005\u00b5M) from \u2018non\u2010blockers\u2032 exhibiting moderate (1\u201310\u2005\u00b5M) or weak (<jats:italic>IC</jats:italic><jats:sub>50</jats:sub>\u226510\u2005\u00b5M) inhibition, as required by the pharmaceutical industry. Our approaches are based on (discretized) 2D descriptors, selected using Winnow, with additional models generated using Random Forest (RF) and Support Vector Machines (SVMs). We compare our models to those previously developed by Thai and Ecker and by Dubus et\u2005al. The purpose of this paper is twofold: 1. To propose that our approaches (with Matthews Correlation Coefficients from 0.40 to 0.87 on truly external test sets, when extrapolation beyond the applicability domain was not evident and sufficient quantities of data were available for training) are competitive with those currently proposed in the literature. 2. To highlight key issues associated with building and assessing truly predictive models, in particular the considerable variation in model performance when training and testing on different datasets.</jats:p>",
  "DOI": "10.1002/minf.201000159",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2011,
        5,
        6
      ]
    ],
    "date-time": "2011-05-06T11:35:32Z",
    "timestamp": 1304681732000
  },
  "page": "443-458",
  "source": "Crossref",
  "is-referenced-by-count": 35,
  "title": "Development and Comparison of hERG Blocker Classifiers: Assessment on Different Datasets Yields Markedly Different Results",
  "prefix": "10.1002",
  "volume": "30",
  "author": [
    {
      "given": "Richard L.",
      "family": "Marchese\u2005Robinson",
      "sequence": "first",
      "affiliation": []
    },
    {
      "given": "Robert C.",
      "family": "Glen",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "John B. O.",
      "family": "Mitchell",
      "sequence": "additional",
      "affiliation": []
    }
  ],
  "member": "311",
  "published-online": {
    "date-parts": [
      [
        2011,
        5,
        6
      ]
    ]
  },
  "reference": [
    {
      "key": "e_1_2_9_1_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/nature04710"
    },
    {
      "key": "e_1_2_9_2_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1093/eurheartj/ehi312"
    },
    {
      "key": "e_1_2_9_3_2",
      "unstructured": "ICH Anon. ICH S7B Guideline (Step 4 Version): \u201cThe Nonclinical Evaluation of the Potential for Delayed Ventricular Repolarization (QT Interval Prolongation) By Human Pharmaceuticals\u201d to be found under http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Safety/S7B/Step4/S7B_Guideline.pdf 2005."
    },
    {
      "key": "e_1_2_9_4_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/s10822-006-9095-6"
    },
    {
      "key": "e_1_2_9_5_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/jat.1395"
    },
    {
      "key": "e_1_2_9_6_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.vascn.2006.02.003"
    },
    {
      "key": "e_1_2_9_7_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/cmdc.200700264"
    },
    {
      "key": "e_1_2_9_8_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.2174/157340908784533256"
    },
    {
      "key": "e_1_2_9_9_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/s11030-009-9117-0"
    },
    {
      "key": "e_1_2_9_10_2",
      "unstructured": "H.\u2005H. Tie MD thesis University of New South Wales (Australia) 2002."
    },
    {
      "key": "e_1_2_9_11_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/bjp.2008.267"
    },
    {
      "key": "e_1_2_9_12_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0008-6363(02)00846-5"
    },
    {
      "key": "e_1_2_9_13_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/tx800063r"
    },
    {
      "key": "e_1_2_9_14_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/1439-7633(20020503)3:5<455::AID-CBIC455>3.0.CO;2-L"
    },
    {
      "key": "e_1_2_9_15_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/cmdc.200500099"
    },
    {
      "key": "e_1_2_9_16_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.bmc.2008.01.017"
    },
    {
      "key": "e_1_2_9_17_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/cmdc.201000024"
    },
    {
      "key": "e_1_2_9_18_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/mp700124e"
    },
    {
      "key": "e_1_2_9_19_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.bmcl.2005.03.080"
    },
    {
      "key": "e_1_2_9_20_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.bmc.2007.06.028"
    },
    {
      "key": "e_1_2_9_21_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.vascn.2004.06.003"
    },
    {
      "key": "e_1_2_9_22_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1111/j.1365-2125.2008.03327.x"
    },
    {
      "key": "e_1_2_9_23_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.vascn.2005.04.002"
    },
    {
      "key": "e_1_2_9_24_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/jm060500o"
    },
    {
      "key": "e_1_2_9_25_2",
      "unstructured": "J.\u2005K. Martin D.\u2005S. Hirschberg \u201cSmall Sample Statistics for Classification Error Rates I: Error Rate Measurements\u201d to be found under http://www.ics.uci.edu/\u223cdan/pub.html 1996."
    },
    {
      "key": "e_1_2_9_26_2",
      "first-page": "479",
      "volume-title": "Information Theory, Inference and Learning Algorithms",
      "author": "MacKay D. J. C.",
      "year": "2003"
    },
    {
      "key": "e_1_2_9_27_2",
      "first-page": "1661",
      "volume-title": "Proc. IJCNN\u20102006, Vancouver, BC, Canada",
      "author": "Cawley G. C.",
      "year": "2006"
    },
    {
      "key": "e_1_2_9_28_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.jspi.2007.06.003"
    },
    {
      "key": "e_1_2_9_29_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1093/bioinformatics/16.5.412"
    },
    {
      "key": "e_1_2_9_30_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.compbiolchem.2004.09.006"
    },
    {
      "key": "e_1_2_9_31_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci700350n"
    },
    {
      "key": "e_1_2_9_32_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci800079x"
    },
    {
      "key": "e_1_2_9_33_2",
      "unstructured": "C.\u2010W. Hsu C.\u2010C. Chang C.\u2010J. Lin \u201cA Practical Guide to Support Vector Classification\u201d to be found under http://www.csie.ntu.edu.tw/\u223ccjlin/papers/guide/guide.pdf 2010."
    },
    {
      "key": "e_1_2_9_34_2",
      "unstructured": "K. Morik P. Brockhausen T. Joachims inProc. ICML\u201099 Bled Slovenia 1999 pp.\u2005268\u2013277 to be found under http://www.cs.cornell.edu/People/tj/publications/morik_etal_99a.pdf."
    },
    {
      "key": "e_1_2_9_35_2",
      "first-page": "169",
      "volume-title": "Advances in Kernel Methods \u2013 Support Vector Learning",
      "author": "Joachims T.",
      "year": "1999"
    },
    {
      "key": "e_1_2_9_36_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/A:1010933404324"
    },
    {
      "key": "e_1_2_9_37_2",
      "unstructured": "The R Project for Statistical Computing http://www.r\u2010project.org/."
    },
    {
      "key": "e_1_2_9_38_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci034160g"
    },
    {
      "key": "e_1_2_9_39_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci100050t"
    },
    {
      "key": "e_1_2_9_40_2",
      "unstructured": "Python Programming Language(v.2.5.2) http://www.python.org/."
    },
    {
      "key": "e_1_2_9_41_2",
      "unstructured": "U.\u2005M. Fayyad K.\u2005B. Irani inProc. IJCAI\u201093 Chambery France 1993 pp.\u20051022\u20131027 available from http://sci2s.ugr.es/keel/pdf/algorithm/congreso/fayyad1993.pdf."
    },
    {
      "key": "e_1_2_9_42_2",
      "doi-asserted-by": "crossref",
      "unstructured": "J. Demsar B. Zupan \u201cOrange: From Experimental Machine Learning to Interactive Data Mining\u201d White Paper to be found under http://www.ailab.si/orange/wp/orange.pdf 2004.",
      "DOI": "10.1007/978-3-540-30116-5_58"
    },
    {
      "key": "e_1_2_9_43_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S1093-3263(00)00068-1"
    },
    {
      "key": "e_1_2_9_44_2",
      "unstructured": "Chemaxon Budapest Hungary http://www.chemaxon.com."
    },
    {
      "key": "e_1_2_9_45_2",
      "unstructured": "Pipeline Pilot Student Edition(v. 6.1.5) Accelrys San Diego CA USA http://accelrys.com/products/pipeline\u2010pilot/."
    },
    {
      "key": "e_1_2_9_46_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1186/1752-153X-2-5"
    },
    {
      "key": "e_1_2_9_47_2",
      "unstructured": "MOE (Molecular Operating Environment) v. 2008.10 Chemical Computing Group Montreal Canada http://www.chemcomp.com."
    },
    {
      "key": "e_1_2_9_48_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci00062a008"
    },
    {
      "key": "e_1_2_9_49_2",
      "unstructured": "M. Kubat S. Matwin inProc. ICML\u201097 1997 pp.\u2005179\u2013186 available from http://sci2s.ugr.es/keel/pdf/algorithm/congreso/kubat97adressing.pdf."
    },
    {
      "key": "e_1_2_9_50_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1186/1471-2105-10-213"
    },
    {
      "key": "e_1_2_9_51_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/ci050308f"
    },
    {
      "key": "e_1_2_9_52_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/jm060379l"
    },
    {
      "key": "e_1_2_9_53_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/jm801236n"
    },
    {
      "key": "e_1_2_9_54_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.2174/1568016033477432"
    },
    {
      "key": "e_1_2_9_55_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/sj.bjp.0705336"
    },
    {
      "key": "e_1_2_9_56_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/cmdc.200900374"
    },
    {
      "key": "e_1_2_9_57_2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1021/jm900002x"
    }
  ],
  "container-title": "Molecular Informatics",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://api.wiley.com/onlinelibrary/tdm/v1/articles/10.1002%2Fminf.201000159",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "text-mining"
    },
    {
      "URL": "https://onlinelibrary.wiley.com/doi/pdf/10.1002/minf.201000159",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2023,
        10,
        9
      ]
    ],
    "date-time": "2023-10-09T19:18:44Z",
    "timestamp": 1696879124000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://onlinelibrary.wiley.com/doi/10.1002/minf.201000159"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2011,
        5,
        6
      ]
    ]
  },
  "references-count": 57,
  "journal-issue": {
    "issue": "5",
    "published-print": {
      "date-parts": [
        [
          2011,
          5,
          16
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1002/minf.201000159"
  ],
  "URL": "http://dx.doi.org/10.1002/minf.201000159",
  "relation": {},
  "ISSN": [
    "1868-1743",
    "1868-1751"
  ],
  "subject": [],
  "container-title-short": "Molecular Informatics",
  "published": {
    "date-parts": [
      [
        2011,
        5,
        6
      ]
    ]
  }
}