DOI: 10.1111/j.1467-9868.2005.00532.x. Model Selection and Estimation in Regression with Grouped Variables

Model Selection and Estimation in Regression with Grouped Variables

10.1111/j.1467-9868.2005.00532.x

Crossref journal-article

Oxford University Press (OUP)

Journal of the Royal Statistical Society Series B: Statistical Methodology (286)

Abstract

SummaryWe consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor analysis-of-variance problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection. The lasso, the LARS algorithm and the non-negative garrotte are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We study the similarities and the differences between these methods. Simulations and real examples are used to illustrate the methods.

Bibliography

Yuan, M., & Lin, Y. (2005). Model Selection and Estimation in Regression with Grouped Variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1), 49â67.

Authors 2

Ming Yuan (first)
Yi Lin (additional)

References 14 Referenced 4,906

{'volume-title': 'Adaptive regression and model selection in data mining problems', 'year': '1999', 'author': 'Bakin', 'key': '2023040800100213700_'} / Adaptive regression and model selection in data mining problems by Bakin (1999)
10.1080/00401706.1995.10484371 / Technometrics / Better subset regression using the nonnegative garrote by Breiman (1995)
10.1214/009053604000000067 / Ann. Statist. / Least angle regression by Efron (2004)
10.1198/016214501753382273 / J. Am. Statist. Ass. / Variable selection via nonconcave penalized likelihood and its oracle properties by Fan (2001)
10.1214/aos/1176325766 / Ann. Statist. / The risk inflation criterion for multiple regression by Foster (1994)
10.1080/10618600.1998.10474784 / J. Comput. Graph. Statist. / Penalized regressions: the bridge versus the lasso by Fu (1999)
10.1093/biomet/87.4.731 / Biometrika / Calibration and empirical Bayes variable selection by George (2000)
10.1080/01621459.1993.10476353 / J. Am. Statist. Ass. / Variable selection via Gibbs sampling by George (1993)
{'volume-title': 'Applied Logistic Regression', 'year': '1989', 'author': 'Hosmer', 'key': '2023040800100213700_'} / Applied Logistic Regression by Hosmer (1989)
{'volume-title': 'Technical Report 1072', 'year': '2003', 'author': 'Lin', 'key': '2023040800100213700_'} / Technical Report 1072 by Lin (2003)
{'volume-title': 'Technical Report', 'year': '2004', 'author': 'Rosset', 'key': '2023040800100213700_'} / Technical Report by Rosset (2004)
10.1198/016214502753479356 / J. Am. Statist. Ass. / Adaptive model selection by Shen (2002)
10.1111/j.2517-6161.1996.tb02080.x / J. R. Statist. Soc. B / Regression shrinkage and selection via the lasso by Tibshirani (1996)
{'volume-title': 'Statistics Discussion Paper 2005-25', 'year': '2005', 'author': 'Yuan', 'key': '2023040800100213700_'} / Statistics Discussion Paper 2005-25 by Yuan (2005)

Dates

Type	When
Created	19 years, 8 months ago (Dec. 21, 2005, 6:07 a.m.)
Deposited	7 months, 2 weeks ago (Jan. 6, 2025, 12:40 p.m.)
Indexed	7 hours, 27 minutes ago (Aug. 21, 2025, 2:28 p.m.)
Issued	19 years, 8 months ago (Dec. 21, 2005)
Published	19 years, 8 months ago (Dec. 21, 2005)
Published Online	19 years, 8 months ago (Dec. 21, 2005)
Published Print	19 years, 6 months ago (Feb. 1, 2006)

Funders 1

National Science Foundation 10.13039/100000001
Region: Americas
gov (National government)
Labels4
1. U.S. National Science Foundation
2. NSF
3. US NSF
4. USA NSF
Awards1
1. DMS-0134987

BibTeX

@article{Yuan_2005, title={Model Selection and Estimation in Regression with Grouped Variables}, volume={68}, ISSN={1467-9868}, url={http://dx.doi.org/10.1111/j.1467-9868.2005.00532.x}, DOI={10.1111/j.1467-9868.2005.00532.x}, number={1}, journal={Journal of the Royal Statistical Society Series B: Statistical Methodology}, publisher={Oxford University Press (OUP)}, author={Yuan, Ming and Lin, Yi}, year={2005}, month=dec, pages={49–67} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        21
      ]
    ],
    "date-time": "2025-08-21T18:28:27Z",
    "timestamp": 1755800907932,
    "version": "3.37.3"
  },
  "reference-count": 14,
  "publisher": "Oxford University Press (OUP)",
  "issue": "1",
  "license": [
    {
      "start": {
        "date-parts": [
          [
            2023,
            4,
            7
          ]
        ],
        "date-time": "2023-04-07T00:00:00Z",
        "timestamp": 1680825600000
      },
      "content-version": "vor",
      "delay-in-days": 6316,
      "URL": "https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model"
    }
  ],
  "funder": [
    {
      "DOI": "10.13039/100000001",
      "name": "National Science Foundation",
      "doi-asserted-by": "publisher",
      "award": [
        "DMS-0134987"
      ],
      "id": [
        {
          "id": "10.13039/100000001",
          "id-type": "DOI",
          "asserted-by": "publisher"
        }
      ]
    }
  ],
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2006,
        2,
        1
      ]
    ]
  },
  "abstract": "<jats:title>Summary</jats:title><jats:p>We consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor analysis-of-variance problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection. The lasso, the LARS algorithm and the non-negative garrotte are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We study the similarities and the differences between these methods. Simulations and real examples are used to illustrate the methods.</jats:p>",
  "DOI": "10.1111/j.1467-9868.2005.00532.x",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2005,
        12,
        21
      ]
    ],
    "date-time": "2005-12-21T11:07:16Z",
    "timestamp": 1135163236000
  },
  "page": "49-67",
  "source": "Crossref",
  "is-referenced-by-count": 4906,
  "title": "Model Selection and Estimation in Regression with Grouped Variables",
  "prefix": "10.1093",
  "volume": "68",
  "author": [
    {
      "given": "Ming",
      "family": "Yuan",
      "sequence": "first",
      "affiliation": [
        {
          "name": "Georgia Institute of Technology , Atlanta , USA"
        }
      ]
    },
    {
      "given": "Yi",
      "family": "Lin",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "University of Wisconsin\u2014Madison , USA"
        }
      ]
    }
  ],
  "member": "286",
  "published-online": {
    "date-parts": [
      [
        2005,
        12,
        21
      ]
    ]
  },
  "reference": [
    {
      "volume-title": "Adaptive regression and model selection in data mining problems",
      "year": "1999",
      "author": "Bakin",
      "key": "2023040800100213700_"
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "373",
      "DOI": "10.1080/00401706.1995.10484371",
      "article-title": "Better subset regression using the nonnegative garrote",
      "volume": "37",
      "author": "Breiman",
      "year": "1995",
      "journal-title": "Technometrics"
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "407",
      "DOI": "10.1214/009053604000000067",
      "article-title": "Least angle regression",
      "volume": "32",
      "author": "Efron",
      "year": "2004",
      "journal-title": "Ann. Statist."
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "1348",
      "DOI": "10.1198/016214501753382273",
      "article-title": "Variable selection via nonconcave penalized likelihood and its oracle properties",
      "volume": "96",
      "author": "Fan",
      "year": "2001",
      "journal-title": "J. Am. Statist. Ass."
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "1947",
      "DOI": "10.1214/aos/1176325766",
      "article-title": "The risk inflation criterion for multiple regression",
      "volume": "22",
      "author": "Foster",
      "year": "1994",
      "journal-title": "Ann. Statist."
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "397",
      "DOI": "10.1080/10618600.1998.10474784",
      "article-title": "Penalized regressions: the bridge versus the lasso",
      "volume": "7",
      "author": "Fu",
      "year": "1999",
      "journal-title": "J. Comput. Graph. Statist."
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "731",
      "DOI": "10.1093/biomet/87.4.731",
      "article-title": "Calibration and empirical Bayes variable selection",
      "volume": "87",
      "author": "George",
      "year": "2000",
      "journal-title": "Biometrika"
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "881",
      "DOI": "10.1080/01621459.1993.10476353",
      "article-title": "Variable selection via Gibbs sampling",
      "volume": "88",
      "author": "George",
      "year": "1993",
      "journal-title": "J. Am. Statist. Ass."
    },
    {
      "volume-title": "Applied Logistic Regression",
      "year": "1989",
      "author": "Hosmer",
      "key": "2023040800100213700_"
    },
    {
      "volume-title": "Technical Report 1072",
      "year": "2003",
      "author": "Lin",
      "key": "2023040800100213700_"
    },
    {
      "volume-title": "Technical Report",
      "year": "2004",
      "author": "Rosset",
      "key": "2023040800100213700_"
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "210",
      "DOI": "10.1198/016214502753479356",
      "article-title": "Adaptive model selection",
      "volume": "97",
      "author": "Shen",
      "year": "2002",
      "journal-title": "J. Am. Statist. Ass."
    },
    {
      "key": "2023040800100213700_",
      "doi-asserted-by": "crossref",
      "first-page": "267",
      "DOI": "10.1111/j.2517-6161.1996.tb02080.x",
      "article-title": "Regression shrinkage and selection via the lasso",
      "volume": "58",
      "author": "Tibshirani",
      "year": "1996",
      "journal-title": "J. R. Statist. Soc. B"
    },
    {
      "volume-title": "Statistics Discussion Paper 2005-25",
      "year": "2005",
      "author": "Yuan",
      "key": "2023040800100213700_"
    }
  ],
  "container-title": "Journal of the Royal Statistical Society Series B: Statistical Methodology",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://api.wiley.com/onlinelibrary/tdm/v1/articles/10.1111%2Fj.1467-9868.2005.00532.x",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "text-mining"
    },
    {
      "URL": "https://academic.oup.com/jrsssb/article-pdf/68/1/49/49794691/jrsssb_68_1_49.pdf",
      "content-type": "application/pdf",
      "content-version": "vor",
      "intended-application": "syndication"
    },
    {
      "URL": "https://academic.oup.com/jrsssb/article-pdf/68/1/49/49794691/jrsssb_68_1_49.pdf",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2025,
        1,
        6
      ]
    ],
    "date-time": "2025-01-06T17:40:37Z",
    "timestamp": 1736185237000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://academic.oup.com/jrsssb/article/68/1/49/7110631"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2005,
        12,
        21
      ]
    ]
  },
  "references-count": 14,
  "journal-issue": {
    "issue": "1",
    "published-online": {
      "date-parts": [
        [
          2005,
          12,
          21
        ]
      ]
    },
    "published-print": {
      "date-parts": [
        [
          2006,
          2,
          1
        ]
      ]
    }
  },
  "URL": "http://dx.doi.org/10.1111/j.1467-9868.2005.00532.x",
  "relation": {},
  "ISSN": [
    "1369-7412",
    "1467-9868"
  ],
  "subject": [],
  "published-other": {
    "date-parts": [
      [
        2006,
        2
      ]
    ]
  },
  "published": {
    "date-parts": [
      [
        2005,
        12,
        21
      ]
    ]
  }
}