DOI: 10.1287/ijoc.1060.0182. A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions

A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions

10.1287/ijoc.1060.0182

Crossref journal-article

Institute for Operations Research and the Management Sciences (INFORMS)

INFORMS Journal on Computing (109)

Abstract

We introduce a new framework for the global optimization of computationally expensive multimodal functions when derivatives are unavailable. The proposed Stochastic Response Surface (SRS) Method iteratively utilizes a response surface model to approximate the expensive function and identifies a promising point for function evaluation from a set of randomly generated points, called candidate points. Assuming some mild technical conditions, SRS converges to the global minimum in a probabilistic sense. We also propose Metric SRS (MSRS), which is a special case of SRS where the function evaluation point in each iteration is chosen to be the best candidate point according to two criteria: the estimated function value obtained from the response surface model, and the minimum distance from previously evaluated points. We develop a global optimization version and a multistart local optimization version of MSRS. In the numerical experiments, we used a radial basis function (RBF) model for MSRS and the resulting algorithms, Global MSRBF and Multistart Local MSRBF, were compared to 6 alternative global optimization methods, including a multistart derivative-based local optimization method. Multiple trials of all algorithms were compared on 17 multimodal test problems and on a 12-dimensional groundwater bioremediation application involving partial differential equations. The results indicate that Multistart Local MSRBF is the best on most of the higher dimensional problems, including the groundwater problem. It is also at least as good as the other algorithms on most of the lower dimensional problems. Global MSRBF is competitive with the other alternatives on most of the lower dimensional test problems and also on the groundwater problem. These results suggest that MSRBF is a promising approach for the global optimization of expensive functions.

Bibliography

Regis, R. G., & Shoemaker, C. A. (2007). A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions. INFORMS Journal on Computing, 19(4), 497â509.

Authors 2

Rommel G. Regis (first)
Christine A. Shoemaker (additional)

References 44 Referenced 386

10.1023/A:1011584207202
10.1007/BF01197708
{'key': 'B3', 'volume-title': 'Empirical Model-Building and Response Surfaces', 'author': 'Box G. E. P.', 'year': '1987'} / Empirical Model-Building and Response Surfaces by Box G. E. P. (1987)
10.1017/CBO9780511543241
10.1007/BF02614326
10.1002/9781119115151
10.1137/0801027
{'key': 'B8', 'first-page': '1', 'volume-title': 'Towards Global Optimization 2', 'author': 'Dixon L. C. W.', 'year': '1978'} / Towards Global Optimization 2 by Dixon L. C. W. (1978)
10.1214/aos/1176347963
10.1137/0805015
10.1007/BFb0026589
10.1007/BF00933356
10.1023/A:1011255519438
{'key': 'B14', 'volume-title': 'Introduction to Global Optimization', 'author': 'Horst R.', 'year': '1995'} / Introduction to Global Optimization by Horst R. (1995)
10.1109/20.767363
10.1023/A:1012771025575
10.1023/A:1008306431147
10.1137/1.9781611970920
10.1016/S0169-7161(96)13011-X
{'key': 'B20', 'first-page': '193', 'volume-title': 'Optimization Software Class Libraries', 'author': 'Laguna M.', 'year': '2002'} / Optimization Software Class Libraries by Laguna M. (2002)
10.1007/978-1-4615-0337-8
{'key': 'B22', 'first-page': '1066', 'volume': '8', 'author': 'Moore A.', 'year': '1996', 'journal-title': 'Neural Inform. Processing Systems'} / Neural Inform. Processing Systems by Moore A. (1996)
{'key': 'B23', 'first-page': '386', 'volume-title': 'Proc. Fifteenth Internat. Conf. Machine Learn.', 'author': 'Moore A.', 'year': '1998'} / Proc. Fifteenth Internat. Conf. Machine Learn. by Moore A. (1998)
10.1016/0378-3758(94)00035-T
{'key': 'B25', 'volume-title': 'Response Surface Methodology: Process and Product Optimization Using Designed Experiments', 'author': 'Myers R. H.', 'year': '1995'} / Response Surface Methodology: Process and Product Optimization Using Designed Experiments by Myers R. H. (1995)
10.1093/comjnl/7.4.308
10.1007/b98874
Powell M. J. D., Light W.Advances in Numerical Analysis, Volume 2: Wavelets, Subdivision Algorithms and Radial Basis Functions(1992) (Oxford University Press, Oxford, UK) 105–210 (10.1093/oso/9780198534396.003.0003)
10.1007/978-3-0348-8696-3_14
10.1007/s101070100290
10.1007/s10107-003-0430-6
10.1007/BF02592071
10.1214/ss/1177012413
10.1007/BF01096734
10.2514/6.1998-4755
10.1002/0471722138
10.1137/S1052623493250780
{'key': 'B40', 'volume-title': 'Global Optimization, Lecture Notes in Computer Science', 'volume': '350', 'author': 'Torn A.', 'year': '1989'} / Global Optimization, Lecture Notes in Computer Science by Torn A. (1989)
{'key': 'B41', 'author': 'Ugray Z.', 'year': '2006', 'journal-title': 'INFORMS J. Comput.'} / INFORMS J. Comput. by Ugray Z. (2006)
10.1016/j.cam.2004.11.029
{'key': 'B43', 'volume-title': 'Nonlinear Optimization: Complexity Issues', 'author': 'Vavasis S. A.', 'year': '1991'} / Nonlinear Optimization: Complexity Issues by Vavasis S. A. (1991)
10.1109/4235.585893
10.1016/S0378-3758(00)00105-1
10.1061/(ASCE)0733-9496(1999)125:1(54)

Dates

Type	When
Created	17 years, 9 months ago (Nov. 5, 2007, 4 p.m.)
Deposited	1 year, 6 months ago (Feb. 18, 2024, 4:31 p.m.)
Indexed	5 days, 13 hours ago (Aug. 21, 2025, 12:36 p.m.)
Issued	17 years, 9 months ago (Nov. 1, 2007)
Published	17 years, 9 months ago (Nov. 1, 2007)
Published Print	17 years, 9 months ago (Nov. 1, 2007)

Funders 0

None

BibTeX

@article{Regis_2007, title={A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions}, volume={19}, ISSN={1526-5528}, url={http://dx.doi.org/10.1287/ijoc.1060.0182}, DOI={10.1287/ijoc.1060.0182}, number={4}, journal={INFORMS Journal on Computing}, publisher={Institute for Operations Research and the Management Sciences (INFORMS)}, author={Regis, Rommel G. and Shoemaker, Christine A.}, year={2007}, month=nov, pages={497–509} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        21
      ]
    ],
    "date-time": "2025-08-21T16:36:18Z",
    "timestamp": 1755794178158
  },
  "reference-count": 44,
  "publisher": "Institute for Operations Research and the Management Sciences (INFORMS)",
  "issue": "4",
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2007,
        11
      ]
    ]
  },
  "abstract": "<jats:p>We introduce a new framework for the global optimization of computationally expensive multimodal functions when derivatives are unavailable. The proposed Stochastic Response Surface (SRS) Method iteratively utilizes a response surface model to approximate the expensive function and identifies a promising point for function evaluation from a set of randomly generated points, called candidate points. Assuming some mild technical conditions, SRS converges to the global minimum in a probabilistic sense. We also propose Metric SRS (MSRS), which is a special case of SRS where the function evaluation point in each iteration is chosen to be the best candidate point according to two criteria: the estimated function value obtained from the response surface model, and the minimum distance from previously evaluated points. We develop a global optimization version and a multistart local optimization version of MSRS. In the numerical experiments, we used a radial basis function (RBF) model for MSRS and the resulting algorithms, Global MSRBF and Multistart Local MSRBF, were compared to 6 alternative global optimization methods, including a multistart derivative-based local optimization method. Multiple trials of all algorithms were compared on 17 multimodal test problems and on a 12-dimensional groundwater bioremediation application involving partial differential equations. The results indicate that Multistart Local MSRBF is the best on most of the higher dimensional problems, including the groundwater problem. It is also at least as good as the other algorithms on most of the lower dimensional problems. Global MSRBF is competitive with the other alternatives on most of the lower dimensional test problems and also on the groundwater problem. These results suggest that MSRBF is a promising approach for the global optimization of expensive functions.</jats:p>",
  "DOI": "10.1287/ijoc.1060.0182",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2007,
        11,
        5
      ]
    ],
    "date-time": "2007-11-05T21:00:53Z",
    "timestamp": 1194296453000
  },
  "page": "497-509",
  "source": "Crossref",
  "is-referenced-by-count": 386,
  "title": "A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions",
  "prefix": "10.1287",
  "volume": "19",
  "author": [
    {
      "given": "Rommel G.",
      "family": "Regis",
      "sequence": "first",
      "affiliation": [
        {
          "name": "Cornell Theory Center and School of Operations Research and Industrial Engineering, Cornell University, Ithaca, New York 14853"
        }
      ]
    },
    {
      "given": "Christine A.",
      "family": "Shoemaker",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "School of Civil and Environmental Engineering and School of Operations Research and Industrial Engineering, Cornell University, Ithaca, New York 14853"
        }
      ]
    }
  ],
  "member": "109",
  "reference": [
    {
      "key": "B1",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/A:1011584207202"
    },
    {
      "key": "B2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF01197708"
    },
    {
      "key": "B3",
      "volume-title": "Empirical Model-Building and Response Surfaces",
      "author": "Box G. E. P.",
      "year": "1987"
    },
    {
      "key": "B4",
      "doi-asserted-by": "publisher",
      "DOI": "10.1017/CBO9780511543241"
    },
    {
      "key": "B5",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF02614326"
    },
    {
      "key": "B6",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/9781119115151"
    },
    {
      "key": "B7",
      "doi-asserted-by": "publisher",
      "DOI": "10.1137/0801027"
    },
    {
      "key": "B8",
      "first-page": "1",
      "volume-title": "Towards Global Optimization 2",
      "author": "Dixon L. C. W.",
      "year": "1978"
    },
    {
      "key": "B9",
      "doi-asserted-by": "publisher",
      "DOI": "10.1214/aos/1176347963"
    },
    {
      "key": "B10",
      "doi-asserted-by": "publisher",
      "DOI": "10.1137/0805015"
    },
    {
      "key": "B11",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BFb0026589"
    },
    {
      "key": "B12",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF00933356"
    },
    {
      "key": "B13",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/A:1011255519438"
    },
    {
      "key": "B14",
      "volume-title": "Introduction to Global Optimization",
      "author": "Horst R.",
      "year": "1995"
    },
    {
      "key": "B15",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/20.767363"
    },
    {
      "key": "B16",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/A:1012771025575"
    },
    {
      "key": "B17",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/A:1008306431147"
    },
    {
      "key": "B18",
      "doi-asserted-by": "publisher",
      "DOI": "10.1137/1.9781611970920"
    },
    {
      "key": "B19",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0169-7161(96)13011-X"
    },
    {
      "key": "B20",
      "first-page": "193",
      "volume-title": "Optimization Software Class Libraries",
      "author": "Laguna M.",
      "year": "2002"
    },
    {
      "key": "B21",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/978-1-4615-0337-8"
    },
    {
      "key": "B22",
      "first-page": "1066",
      "volume": "8",
      "author": "Moore A.",
      "year": "1996",
      "journal-title": "Neural Inform. Processing Systems"
    },
    {
      "key": "B23",
      "first-page": "386",
      "volume-title": "Proc. Fifteenth Internat. Conf. Machine Learn.",
      "author": "Moore A.",
      "year": "1998"
    },
    {
      "key": "B24",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/0378-3758(94)00035-T"
    },
    {
      "key": "B25",
      "volume-title": "Response Surface Methodology: Process and Product Optimization Using Designed Experiments",
      "author": "Myers R. H.",
      "year": "1995"
    },
    {
      "key": "B26",
      "doi-asserted-by": "publisher",
      "DOI": "10.1093/comjnl/7.4.308"
    },
    {
      "key": "B27",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/b98874"
    },
    {
      "key": "B28",
      "doi-asserted-by": "crossref",
      "unstructured": "Powell M. J. D., Light W.Advances in Numerical Analysis, Volume 2: Wavelets, Subdivision Algorithms and Radial Basis Functions(1992) (Oxford University Press, Oxford, UK) 105\u2013210",
      "DOI": "10.1093/oso/9780198534396.003.0003"
    },
    {
      "key": "B29",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/978-3-0348-8696-3_14"
    },
    {
      "key": "B30",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/s101070100290"
    },
    {
      "key": "B31",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/s10107-003-0430-6"
    },
    {
      "key": "B32",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF02592071"
    },
    {
      "key": "B33",
      "doi-asserted-by": "publisher",
      "DOI": "10.1214/ss/1177012413"
    },
    {
      "key": "B34",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF01096734"
    },
    {
      "key": "B35",
      "doi-asserted-by": "publisher",
      "DOI": "10.2514/6.1998-4755"
    },
    {
      "key": "B36",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/0471722138"
    },
    {
      "key": "B39",
      "doi-asserted-by": "publisher",
      "DOI": "10.1137/S1052623493250780"
    },
    {
      "key": "B40",
      "volume-title": "Global Optimization, Lecture Notes in Computer Science",
      "volume": "350",
      "author": "Torn A.",
      "year": "1989"
    },
    {
      "key": "B41",
      "author": "Ugray Z.",
      "year": "2006",
      "journal-title": "INFORMS J. Comput."
    },
    {
      "key": "B42",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.cam.2004.11.029"
    },
    {
      "key": "B43",
      "volume-title": "Nonlinear Optimization: Complexity Issues",
      "author": "Vavasis S. A.",
      "year": "1991"
    },
    {
      "key": "B44",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/4235.585893"
    },
    {
      "key": "B45",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0378-3758(00)00105-1"
    },
    {
      "key": "B46",
      "doi-asserted-by": "publisher",
      "DOI": "10.1061/(ASCE)0733-9496(1999)125:1(54)"
    }
  ],
  "container-title": "INFORMS Journal on Computing",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://pubsonline.informs.org/doi/pdf/10.1287/ijoc.1060.0182",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2024,
        2,
        18
      ]
    ],
    "date-time": "2024-02-18T21:31:58Z",
    "timestamp": 1708291918000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://pubsonline.informs.org/doi/10.1287/ijoc.1060.0182"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2007,
        11
      ]
    ]
  },
  "references-count": 44,
  "journal-issue": {
    "issue": "4",
    "published-print": {
      "date-parts": [
        [
          2007,
          11
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1287/ijoc.1060.0182"
  ],
  "URL": "http://dx.doi.org/10.1287/ijoc.1060.0182",
  "relation": {},
  "ISSN": [
    "1091-9856",
    "1526-5528"
  ],
  "subject": [],
  "container-title-short": "INFORMS Journal on Computing",
  "published": {
    "date-parts": [
      [
        2007,
        11
      ]
    ]
  }
}