DOI: 10.1038/nature16961. Mastering the game of Go with deep neural networks and tree search

Mastering the game of Go with deep neural networks and tree search

10.1038/nature16961

Crossref journal-article

Springer Science and Business Media LLC

Nature (297)

Bibliography

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484â489.

Authors 20

David Silver (first)
Aja Huang (additional)
Chris J. Maddison (additional)
Arthur Guez (additional)
Laurent Sifre (additional)
George van den Driessche (additional)
Julian Schrittwieser (additional)
Ioannis Antonoglou (additional)
Veda Panneershelvam (additional)
Marc Lanctot (additional)
Sander Dieleman (additional)
Dominik Grewe (additional)
John Nham (additional)
Nal Kalchbrenner (additional)
Ilya Sutskever (additional)
Timothy Lillicrap (additional)
Madeleine Leach (additional)
Koray Kavukcuoglu (additional)
Thore Graepel (additional)
Demis Hassabis (additional)

References 62 Referenced 10,279

Allis, L. V. Searching for Solutions in Games and Artificial Intelligence. PhD thesis, Univ. Limburg, Maastricht, The Netherlands (1994)
van den Herik, H., Uiterwijk, J. W. & van Rijswijck, J. Games solved: now and in the future. Artif. Intell. 134, 277–311 (2002) (10.1016/S0004-3702(01)00152-7) / Artif. Intell. by H van den Herik (2002)
Schaeffer, J. The games computers (and people) play. Advances in Computers 52, 189–266 (2000) (10.1016/S0065-2458(00)80019-4) / Advances in Computers by J Schaeffer (2000)
Campbell, M., Hoane, A. & Hsu, F. Deep Blue. Artif. Intell. 134, 57–83 (2002) (10.1016/S0004-3702(01)00129-1) / Artif. Intell. by M Campbell (2002)
Schaeffer, J. et al. A world championship caliber checkers program. Artif. Intell. 53, 273–289 (1992) (10.1016/0004-3702(92)90074-8) / Artif. Intell. by J Schaeffer (1992)
Buro, M. From simple features to sophisticated evaluation functions. In 1st International Conference on Computers and Games, 126–145 (1999) (10.1007/3-540-48957-6_8)
Müller, M. Computer Go. Artif. Intell. 134, 145–179 (2002) (10.1016/S0004-3702(01)00121-7) / Artif. Intell. by M Müller (2002)
Tesauro, G. & Galperin, G. On-line policy improvement using Monte-Carlo search. In Advances in Neural Information Processing, 1068–1074 (1996)
Sheppard, B. World-championship-caliber Scrabble. Artif. Intell. 134, 241–275 (2002) (10.1016/S0004-3702(01)00166-7) / Artif. Intell. by B Sheppard (2002)
Bouzy, B. & Helmstetter, B. Monte-Carlo Go developments. In 10th International Conference on Advances in Computer Games, 159–174 (2003) (10.1007/978-0-387-35706-5_11)
Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In 5th International Conference on Computers and Games, 72–83 (2006) (10.1007/978-3-540-75538-8_7)
Kocsis, L. & Szepesvári, C. Bandit based Monte-Carlo planning. In 15th European Conference on Machine Learning, 282–293 (2006) (10.1007/11871842_29)
Coulom, R. Computing Elo ratings of move patterns in the game of Go. ICGA J. 30, 198–208 (2007) (10.3233/ICG-2007-30403) / ICGA J. by R Coulom (2007)
Baudiš, P. & Gailly, J.-L. Pachi: State of the art open source Go program. In Advances in Computer Games, 24–38 (Springer, 2012) (10.1007/978-3-642-31866-5_3)
Müller, M., Enzenberger, M., Arneson, B. & Segal, R. Fuego – an open-source framework for board games and Go engine based on Monte-Carlo tree search. IEEE Trans. Comput. Intell. AI in Games 2, 259–270 (2010) (10.1109/TCIAIG.2010.2083662) / IEEE Trans. Comput. Intell. AI in Games by M Müller (2010)
Gelly, S. & Silver, D. Combining online and offline learning in UCT. In 17th International Conference on Machine Learning, 273–280 (2007) (10.1145/1273496.1273531)
Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 1097–1105 (2012)
Lawrence, S., Giles, C. L., Tsoi, A. C. & Back, A. D. Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8, 98–113 (1997) (10.1109/72.554195) / IEEE Trans. Neural Netw. by S Lawrence (1997)
Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015) (10.1038/nature14236) / Nature by V Mnih (2015)
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015) (10.1038/nature14539) / Nature by Y LeCun (2015)
Stern, D., Herbrich, R. & Graepel, T. Bayesian pattern ranking for move prediction in the game of Go. In International Conference of Machine Learning, 873–880 (2006) (10.1145/1143844.1143954)
Sutskever, I. & Nair, V. Mimicking Go experts with convolutional neural networks. In International Conference on Artificial Neural Networks, 101–110 (2008) (10.1007/978-3-540-87559-8_11)
Maddison, C. J., Huang, A., Sutskever, I. & Silver, D. Move evaluation in Go using deep convolutional neural networks. 3rd International Conference on Learning Representations (2015)
Clark, C. & Storkey, A. J. Training deep convolutional neural networks to play go. In 32nd International Conference on Machine Learning, 1766–1774 (2015)
Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992) / Mach. Learn. by RJ Williams (1992)
Sutton, R., McAllester, D., Singh, S. & Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems, 1057–1063 (2000)
Sutton, R. & Barto, A. Reinforcement Learning: an Introduction (MIT Press, 1998) (10.1109/TNN.1998.712192)
Schraudolph, N. N., Dayan, P. & Sejnowski, T. J. Temporal difference learning of position evaluation in the game of Go. Adv. Neural Inf. Process. Syst. 6, 817–824 (1994) / Adv. Neural Inf. Process. Syst. by NN Schraudolph (1994)
Enzenberger, M. Evaluation in Go by a neural network using soft segmentation. In 10th Advances in Computer Games Conference, 97–108 (2003). 267 (10.1007/978-0-387-35706-5_7)
Silver, D., Sutton, R. & Müller, M. Temporal-difference search in computer Go. Mach. Learn. 87, 183–219 (2012) (10.1007/s10994-012-5280-0) / Mach. Learn. by D Silver (2012)
Levinovitz, A. The mystery of Go, the ancient game that computers still can’t win. Wired Magazine (2014)
Mechner, D. All Systems Go. The Sciences 38, 32–37 (1998) (10.1002/j.2326-1951.1998.tb03356.x) / The Sciences by D Mechner (1998)
Mandziuk, J. Computational intelligence in mind games. In Challenges for Computational Intelligence, 407–442 (2007) (10.1007/978-3-540-71984-7_15)
Berliner, H. A chronology of computer chess and its literature. Artif. Intell. 10, 201–214 (1978) (10.1016/S0004-3702(78)80012-5) / Artif. Intell. by H Berliner (1978)
Browne, C. et al. A survey of Monte-Carlo tree search methods. IEEE Trans. Comput. Intell. AI in Games 4, 1–43 (2012) (10.1109/TCIAIG.2012.2186810) / IEEE Trans. Comput. Intell. AI in Games by C Browne (2012)
Gelly, S. et al. The grand challenge of computer Go: Monte Carlo tree search and extensions. Commun. ACM 55, 106–113 (2012) (10.1145/2093548.2093574) / Commun. ACM by S Gelly (2012)
Coulom, R. Whole-history rating: A Bayesian rating system for players of time-varying strength. In International Conference on Computers and Games, 113–124 (2008) (10.1007/978-3-540-87608-3_11)
KGS. Rating system math. http://www.gokgs.com/help/rmath.html
Littman, M. L. Markov games as a framework for multi-agent reinforcement learning. In 11th International Conference on Machine Learning, 157–163 (1994) (10.1016/B978-1-55860-335-6.50027-1)
Knuth, D. E. & Moore, R. W. An analysis of alpha-beta pruning. Artif. Intell. 6, 293–326 (1975) (10.1016/0004-3702(75)90019-3) / Artif. Intell. by DE Knuth (1975)
Sutton, R. Learning to predict by the method of temporal differences. Mach. Learn. 3, 9–44 (1988) / Mach. Learn. by R Sutton (1988)
Baxter, J., Tridgell, A. & Weaver, L. Learning to play chess using temporal differences. Mach. Learn. 40, 243–263 (2000) (10.1023/A:1007634325138) / Mach. Learn. by J Baxter (2000)
Veness, J., Silver, D., Blair, A. & Uther, W. Bootstrapping from game tree search. In Advances in Neural Information Processing Systems (2009)
Samuel, A. L. Some studies in machine learning using the game of checkers II - recent progress. IBM J. Res. Develop. 11, 601–617 (1967) (10.1147/rd.116.0601) / IBM J. Res. Develop. by AL Samuel (1967)
Schaeffer, J., Hlynka, M. & Jussila, V. Temporal difference learning applied to a high-performance game-playing program. In 17th International Joint Conference on Artificial Intelligence, 529–534 (2001)
Tesauro, G. TD-gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput. 6, 215–219 (1994) (10.1162/neco.1994.6.2.215) / Neural Comput. by G Tesauro (1994)
Dahl, F. Honte, a Go-playing program using neural nets. In Machines that learn to play games, 205–223 (Nova Science, 1999)
Rosin, C. D. Multi-armed bandits with episode context. Ann. Math. Artif. Intell. 61, 203–230 (2011) (10.1007/s10472-011-9258-6) / Ann. Math. Artif. Intell. by CD Rosin (2011)
Lanctot, M., Winands, M. H. M., Pepels, T. & Sturtevant, N. R. Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In IEEE Conference on Computational Intelligence and Games, 1–8 (2014) (10.1109/CIG.2014.6932903)
Gelly, S., Wang, Y., Munos, R. & Teytaud, O. Modification of UCT with patterns in Monte-Carlo Go. Tech. Rep. 6062, INRIA (2006) / Tech. Rep. by S Gelly (2006)
Silver, D. & Tesauro, G. Monte-Carlo simulation balancing. In 26th International Conference on Machine Learning, 119 (2009) (10.1145/1553374.1553495)
Huang, S.-C., Coulom, R. & Lin, S.-S. Monte-Carlo simulation balancing in practice. In 7th International Conference on Computers and Games, 81–92 (Springer-Verlag, 2011) (10.1007/978-3-642-17928-0_8)
Baier, H. & Drake, P. D. The power of forgetting: improving the last-good-reply policy in Monte Carlo Go. IEEE Trans. Comput. Intell. AI in Games 2, 303–309 (2010) (10.1109/TCIAIG.2010.2100396) / IEEE Trans. Comput. Intell. AI in Games by H Baier (2010)
Huang, S. & Müller, M. Investigating the limits of Monte-Carlo tree search methods in computer Go. In 8th International Conference on Computers and Games, 39–48 (2013) (10.1007/978-3-319-09165-5_4)
Segal, R. B. On the scalability of parallel UCT. Computers and Games 6515, 36–47 (2011) (10.1007/978-3-642-17928-0_4) / Computers and Games by RB Segal (2011)
Enzenberger, M. & Müller, M. A lock-free multithreaded Monte-Carlo tree search algorithm. In 12th Advances in Computer Games Conference, 14–20 (2009) (10.1007/978-3-642-12993-3_2)
Huang, S.-C., Coulom, R. & Lin, S.-S. Time management for Monte-Carlo tree search applied to the game of Go. In International Conference on Technologies and Applications of Artificial Intelligence, 462–466 (2010) (10.1109/TAAI.2010.78)
Gelly, S. & Silver, D. Monte-Carlo tree search and rapid action value estimation in computer Go. Artif. Intell. 175, 1856–1875 (2011) (10.1016/j.artint.2011.03.007) / Artif. Intell. by S Gelly (2011)
Baudiš, P. Balancing MCTS by dynamically adjusting the komi value. ICGA J. 34, 131 (2011) (10.3233/ICG-2011-34302) / ICGA J. by P Baudiš (2011)
Baier, H. & Winands, M. H. Active opening book application for Monte-Carlo tree search in 19×19 Go. In Benelux Conference on Artificial Intelligence, 3–10 (2011)
Dean, J. et al. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, 1223–1231 (2012)
Go ratings. http://www.goratings.org

Dates

Type	When
Created	9 years, 6 months ago (Jan. 26, 2016, 12:44 p.m.)
Deposited	2 years, 3 months ago (May 18, 2023, 1:49 p.m.)
Indexed	7 hours, 55 minutes ago (Aug. 22, 2025, 12:54 a.m.)
Issued	9 years, 6 months ago (Jan. 27, 2016)
Published	9 years, 6 months ago (Jan. 27, 2016)
Published Online	9 years, 6 months ago (Jan. 27, 2016)
Published Print	9 years, 6 months ago (Jan. 28, 2016)

Funders 0

None

BibTeX

@article{Silver_2016, title={Mastering the game of Go with deep neural networks and tree search}, volume={529}, ISSN={1476-4687}, url={http://dx.doi.org/10.1038/nature16961}, DOI={10.1038/nature16961}, number={7587}, journal={Nature}, publisher={Springer Science and Business Media LLC}, author={Silver, David and Huang, Aja and Maddison, Chris J. and Guez, Arthur and Sifre, Laurent and van den Driessche, George and Schrittwieser, Julian and Antonoglou, Ioannis and Panneershelvam, Veda and Lanctot, Marc and Dieleman, Sander and Grewe, Dominik and Nham, John and Kalchbrenner, Nal and Sutskever, Ilya and Lillicrap, Timothy and Leach, Madeleine and Kavukcuoglu, Koray and Graepel, Thore and Hassabis, Demis}, year={2016}, month=jan, pages={484–489} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        22
      ]
    ],
    "date-time": "2025-08-22T04:54:45Z",
    "timestamp": 1755838485616
  },
  "reference-count": 62,
  "publisher": "Springer Science and Business Media LLC",
  "issue": "7587",
  "license": [
    {
      "start": {
        "date-parts": [
          [
            2016,
            1,
            27
          ]
        ],
        "date-time": "2016-01-27T00:00:00Z",
        "timestamp": 1453852800000
      },
      "content-version": "tdm",
      "delay-in-days": 0,
      "URL": "https://www.springer.com/tdm"
    },
    {
      "start": {
        "date-parts": [
          [
            2016,
            1,
            27
          ]
        ],
        "date-time": "2016-01-27T00:00:00Z",
        "timestamp": 1453852800000
      },
      "content-version": "vor",
      "delay-in-days": 0,
      "URL": "https://www.springer.com/tdm"
    }
  ],
  "content-domain": {
    "domain": [
      "link.springer.com"
    ],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2016,
        1,
        28
      ]
    ]
  },
  "DOI": "10.1038/nature16961",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2016,
        1,
        26
      ]
    ],
    "date-time": "2016-01-26T17:44:19Z",
    "timestamp": 1453830259000
  },
  "page": "484-489",
  "update-policy": "http://dx.doi.org/10.1007/springer_crossmark_policy",
  "source": "Crossref",
  "is-referenced-by-count": 10279,
  "title": "Mastering the game of Go with deep neural networks and tree search",
  "prefix": "10.1038",
  "volume": "529",
  "author": [
    {
      "given": "David",
      "family": "Silver",
      "sequence": "first",
      "affiliation": []
    },
    {
      "given": "Aja",
      "family": "Huang",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Chris J.",
      "family": "Maddison",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Arthur",
      "family": "Guez",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Laurent",
      "family": "Sifre",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "George",
      "family": "van den Driessche",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Julian",
      "family": "Schrittwieser",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Ioannis",
      "family": "Antonoglou",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Veda",
      "family": "Panneershelvam",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Marc",
      "family": "Lanctot",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Sander",
      "family": "Dieleman",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Dominik",
      "family": "Grewe",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "John",
      "family": "Nham",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Nal",
      "family": "Kalchbrenner",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Ilya",
      "family": "Sutskever",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Timothy",
      "family": "Lillicrap",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Madeleine",
      "family": "Leach",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Koray",
      "family": "Kavukcuoglu",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Thore",
      "family": "Graepel",
      "sequence": "additional",
      "affiliation": []
    },
    {
      "given": "Demis",
      "family": "Hassabis",
      "sequence": "additional",
      "affiliation": []
    }
  ],
  "member": "297",
  "published-online": {
    "date-parts": [
      [
        2016,
        1,
        27
      ]
    ]
  },
  "reference": [
    {
      "key": "BFnature16961_CR1",
      "unstructured": "Allis, L. V. Searching for Solutions in Games and Artificial Intelligence. PhD thesis, Univ. Limburg, Maastricht, The Netherlands (1994)"
    },
    {
      "key": "BFnature16961_CR2",
      "doi-asserted-by": "publisher",
      "first-page": "277",
      "DOI": "10.1016/S0004-3702(01)00152-7",
      "volume": "134",
      "author": "H van den Herik",
      "year": "2002",
      "unstructured": "van den Herik, H., Uiterwijk, J. W. & van Rijswijck, J. Games solved: now and in the future. Artif. Intell. 134, 277\u2013311 (2002)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR3",
      "doi-asserted-by": "publisher",
      "first-page": "189",
      "DOI": "10.1016/S0065-2458(00)80019-4",
      "volume": "52",
      "author": "J Schaeffer",
      "year": "2000",
      "unstructured": "Schaeffer, J. The games computers (and people) play. Advances in Computers 52, 189\u2013266 (2000)",
      "journal-title": "Advances in Computers"
    },
    {
      "key": "BFnature16961_CR4",
      "doi-asserted-by": "publisher",
      "first-page": "57",
      "DOI": "10.1016/S0004-3702(01)00129-1",
      "volume": "134",
      "author": "M Campbell",
      "year": "2002",
      "unstructured": "Campbell, M., Hoane, A. & Hsu, F. Deep Blue. Artif. Intell. 134, 57\u201383 (2002)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR5",
      "doi-asserted-by": "publisher",
      "first-page": "273",
      "DOI": "10.1016/0004-3702(92)90074-8",
      "volume": "53",
      "author": "J Schaeffer",
      "year": "1992",
      "unstructured": "Schaeffer, J. et al. A world championship caliber checkers program. Artif. Intell. 53, 273\u2013289 (1992)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR6",
      "doi-asserted-by": "crossref",
      "unstructured": "Buro, M. From simple features to sophisticated evaluation functions. In 1st International Conference on Computers and Games, 126\u2013145 (1999)",
      "DOI": "10.1007/3-540-48957-6_8"
    },
    {
      "key": "BFnature16961_CR7",
      "doi-asserted-by": "publisher",
      "first-page": "145",
      "DOI": "10.1016/S0004-3702(01)00121-7",
      "volume": "134",
      "author": "M M\u00fcller",
      "year": "2002",
      "unstructured": "M\u00fcller, M. Computer Go. Artif. Intell. 134, 145\u2013179 (2002)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR8",
      "unstructured": "Tesauro, G. & Galperin, G. On-line policy improvement using Monte-Carlo search. In Advances in Neural Information Processing, 1068\u20131074 (1996)"
    },
    {
      "key": "BFnature16961_CR9",
      "doi-asserted-by": "publisher",
      "first-page": "241",
      "DOI": "10.1016/S0004-3702(01)00166-7",
      "volume": "134",
      "author": "B Sheppard",
      "year": "2002",
      "unstructured": "Sheppard, B. World-championship-caliber Scrabble. Artif. Intell. 134, 241\u2013275 (2002)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR10",
      "doi-asserted-by": "crossref",
      "unstructured": "Bouzy, B. & Helmstetter, B. Monte-Carlo Go developments. In 10th International Conference on Advances in Computer Games, 159\u2013174 (2003)",
      "DOI": "10.1007/978-0-387-35706-5_11"
    },
    {
      "key": "BFnature16961_CR11",
      "doi-asserted-by": "crossref",
      "unstructured": "Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In 5th International Conference on Computers and Games, 72\u201383 (2006)",
      "DOI": "10.1007/978-3-540-75538-8_7"
    },
    {
      "key": "BFnature16961_CR12",
      "doi-asserted-by": "crossref",
      "unstructured": "Kocsis, L. & Szepesv\u00e1ri, C. Bandit based Monte-Carlo planning. In 15th European Conference on Machine Learning, 282\u2013293 (2006)",
      "DOI": "10.1007/11871842_29"
    },
    {
      "key": "BFnature16961_CR13",
      "doi-asserted-by": "publisher",
      "first-page": "198",
      "DOI": "10.3233/ICG-2007-30403",
      "volume": "30",
      "author": "R Coulom",
      "year": "2007",
      "unstructured": "Coulom, R. Computing Elo ratings of move patterns in the game of Go. ICGA J. 30, 198\u2013208 (2007)",
      "journal-title": "ICGA J."
    },
    {
      "key": "BFnature16961_CR14",
      "doi-asserted-by": "crossref",
      "unstructured": "Baudi\u0161, P. & Gailly, J.-L. Pachi: State of the art open source Go program. In Advances in Computer Games, 24\u201338 (Springer, 2012)",
      "DOI": "10.1007/978-3-642-31866-5_3"
    },
    {
      "key": "BFnature16961_CR15",
      "doi-asserted-by": "publisher",
      "first-page": "259",
      "DOI": "10.1109/TCIAIG.2010.2083662",
      "volume": "2",
      "author": "M M\u00fcller",
      "year": "2010",
      "unstructured": "M\u00fcller, M., Enzenberger, M., Arneson, B. & Segal, R. Fuego \u2013 an open-source framework for board games and Go engine based on Monte-Carlo tree search. IEEE Trans. Comput. Intell. AI in Games 2, 259\u2013270 (2010)",
      "journal-title": "IEEE Trans. Comput. Intell. AI in Games"
    },
    {
      "key": "BFnature16961_CR16",
      "doi-asserted-by": "crossref",
      "unstructured": "Gelly, S. & Silver, D. Combining online and offline learning in UCT. In 17th International Conference on Machine Learning, 273\u2013280 (2007)",
      "DOI": "10.1145/1273496.1273531"
    },
    {
      "key": "BFnature16961_CR17",
      "unstructured": "Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 1097\u20131105 (2012)"
    },
    {
      "key": "BFnature16961_CR18",
      "doi-asserted-by": "publisher",
      "first-page": "98",
      "DOI": "10.1109/72.554195",
      "volume": "8",
      "author": "S Lawrence",
      "year": "1997",
      "unstructured": "Lawrence, S., Giles, C. L., Tsoi, A. C. & Back, A. D. Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8, 98\u2013113 (1997)",
      "journal-title": "IEEE Trans. Neural Netw."
    },
    {
      "key": "BFnature16961_CR19",
      "doi-asserted-by": "publisher",
      "first-page": "529",
      "DOI": "10.1038/nature14236",
      "volume": "518",
      "author": "V Mnih",
      "year": "2015",
      "unstructured": "Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529\u2013533 (2015)",
      "journal-title": "Nature"
    },
    {
      "key": "BFnature16961_CR20",
      "doi-asserted-by": "publisher",
      "first-page": "436",
      "DOI": "10.1038/nature14539",
      "volume": "521",
      "author": "Y LeCun",
      "year": "2015",
      "unstructured": "LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436\u2013444 (2015)",
      "journal-title": "Nature"
    },
    {
      "key": "BFnature16961_CR21",
      "doi-asserted-by": "crossref",
      "unstructured": "Stern, D., Herbrich, R. & Graepel, T. Bayesian pattern ranking for move prediction in the game of Go. In International Conference of Machine Learning, 873\u2013880 (2006)",
      "DOI": "10.1145/1143844.1143954"
    },
    {
      "key": "BFnature16961_CR22",
      "doi-asserted-by": "crossref",
      "unstructured": "Sutskever, I. & Nair, V. Mimicking Go experts with convolutional neural networks. In International Conference on Artificial Neural Networks, 101\u2013110 (2008)",
      "DOI": "10.1007/978-3-540-87559-8_11"
    },
    {
      "key": "BFnature16961_CR23",
      "unstructured": "Maddison, C. J., Huang, A., Sutskever, I. & Silver, D. Move evaluation in Go using deep convolutional neural networks. 3rd International Conference on Learning Representations (2015)"
    },
    {
      "key": "BFnature16961_CR24",
      "unstructured": "Clark, C. & Storkey, A. J. Training deep convolutional neural networks to play go. In 32nd International Conference on Machine Learning, 1766\u20131774 (2015)"
    },
    {
      "key": "BFnature16961_CR25",
      "first-page": "229",
      "volume": "8",
      "author": "RJ Williams",
      "year": "1992",
      "unstructured": "Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229\u2013256 (1992)",
      "journal-title": "Mach. Learn."
    },
    {
      "key": "BFnature16961_CR26",
      "unstructured": "Sutton, R., McAllester, D., Singh, S. & Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems, 1057\u20131063 (2000)"
    },
    {
      "key": "BFnature16961_CR27",
      "doi-asserted-by": "crossref",
      "unstructured": "Sutton, R. & Barto, A. Reinforcement Learning: an Introduction (MIT Press, 1998)",
      "DOI": "10.1109/TNN.1998.712192"
    },
    {
      "key": "BFnature16961_CR28",
      "first-page": "817",
      "volume": "6",
      "author": "NN Schraudolph",
      "year": "1994",
      "unstructured": "Schraudolph, N. N., Dayan, P. & Sejnowski, T. J. Temporal difference learning of position evaluation in the game of Go. Adv. Neural Inf. Process. Syst. 6, 817\u2013824 (1994)",
      "journal-title": "Adv. Neural Inf. Process. Syst."
    },
    {
      "key": "BFnature16961_CR29",
      "doi-asserted-by": "crossref",
      "unstructured": "Enzenberger, M. Evaluation in Go by a neural network using soft segmentation. In 10th Advances in Computer Games Conference, 97\u2013108 (2003). 267",
      "DOI": "10.1007/978-0-387-35706-5_7"
    },
    {
      "key": "BFnature16961_CR30",
      "doi-asserted-by": "publisher",
      "first-page": "183",
      "DOI": "10.1007/s10994-012-5280-0",
      "volume": "87",
      "author": "D Silver",
      "year": "2012",
      "unstructured": "Silver, D., Sutton, R. & M\u00fcller, M. Temporal-difference search in computer Go. Mach. Learn. 87, 183\u2013219 (2012)",
      "journal-title": "Mach. Learn."
    },
    {
      "key": "BFnature16961_CR31",
      "unstructured": "Levinovitz, A. The mystery of Go, the ancient game that computers still can\u2019t win. Wired Magazine (2014)"
    },
    {
      "key": "BFnature16961_CR32",
      "doi-asserted-by": "publisher",
      "first-page": "32",
      "DOI": "10.1002/j.2326-1951.1998.tb03356.x",
      "volume": "38",
      "author": "D Mechner",
      "year": "1998",
      "unstructured": "Mechner, D. All Systems Go. The Sciences 38, 32\u201337 (1998)",
      "journal-title": "The Sciences"
    },
    {
      "key": "BFnature16961_CR33",
      "doi-asserted-by": "crossref",
      "unstructured": "Mandziuk, J. Computational intelligence in mind games. In Challenges for Computational Intelligence, 407\u2013442 (2007)",
      "DOI": "10.1007/978-3-540-71984-7_15"
    },
    {
      "key": "BFnature16961_CR34",
      "doi-asserted-by": "publisher",
      "first-page": "201",
      "DOI": "10.1016/S0004-3702(78)80012-5",
      "volume": "10",
      "author": "H Berliner",
      "year": "1978",
      "unstructured": "Berliner, H. A chronology of computer chess and its literature. Artif. Intell. 10, 201\u2013214 (1978)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR35",
      "doi-asserted-by": "publisher",
      "first-page": "1",
      "DOI": "10.1109/TCIAIG.2012.2186810",
      "volume": "4",
      "author": "C Browne",
      "year": "2012",
      "unstructured": "Browne, C. et al. A survey of Monte-Carlo tree search methods. IEEE Trans. Comput. Intell. AI in Games 4, 1\u201343 (2012)",
      "journal-title": "IEEE Trans. Comput. Intell. AI in Games"
    },
    {
      "key": "BFnature16961_CR36",
      "doi-asserted-by": "publisher",
      "first-page": "106",
      "DOI": "10.1145/2093548.2093574",
      "volume": "55",
      "author": "S Gelly",
      "year": "2012",
      "unstructured": "Gelly, S. et al. The grand challenge of computer Go: Monte Carlo tree search and extensions. Commun. ACM 55, 106\u2013113 (2012)",
      "journal-title": "Commun. ACM"
    },
    {
      "key": "BFnature16961_CR37",
      "doi-asserted-by": "crossref",
      "unstructured": "Coulom, R. Whole-history rating: A Bayesian rating system for players of time-varying strength. In International Conference on Computers and Games, 113\u2013124 (2008)",
      "DOI": "10.1007/978-3-540-87608-3_11"
    },
    {
      "key": "BFnature16961_CR38",
      "unstructured": "KGS. Rating system math. http://www.gokgs.com/help/rmath.html"
    },
    {
      "key": "BFnature16961_CR39",
      "doi-asserted-by": "crossref",
      "unstructured": "Littman, M. L. Markov games as a framework for multi-agent reinforcement learning. In 11th International Conference on Machine Learning, 157\u2013163 (1994)",
      "DOI": "10.1016/B978-1-55860-335-6.50027-1"
    },
    {
      "key": "BFnature16961_CR40",
      "doi-asserted-by": "publisher",
      "first-page": "293",
      "DOI": "10.1016/0004-3702(75)90019-3",
      "volume": "6",
      "author": "DE Knuth",
      "year": "1975",
      "unstructured": "Knuth, D. E. & Moore, R. W. An analysis of alpha-beta pruning. Artif. Intell. 6, 293\u2013326 (1975)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR41",
      "first-page": "9",
      "volume": "3",
      "author": "R Sutton",
      "year": "1988",
      "unstructured": "Sutton, R. Learning to predict by the method of temporal differences. Mach. Learn. 3, 9\u201344 (1988)",
      "journal-title": "Mach. Learn."
    },
    {
      "key": "BFnature16961_CR42",
      "doi-asserted-by": "publisher",
      "first-page": "243",
      "DOI": "10.1023/A:1007634325138",
      "volume": "40",
      "author": "J Baxter",
      "year": "2000",
      "unstructured": "Baxter, J., Tridgell, A. & Weaver, L. Learning to play chess using temporal differences. Mach. Learn. 40, 243\u2013263 (2000)",
      "journal-title": "Mach. Learn."
    },
    {
      "key": "BFnature16961_CR43",
      "unstructured": "Veness, J., Silver, D., Blair, A. & Uther, W. Bootstrapping from game tree search. In Advances in Neural Information Processing Systems (2009)"
    },
    {
      "key": "BFnature16961_CR44",
      "doi-asserted-by": "publisher",
      "first-page": "601",
      "DOI": "10.1147/rd.116.0601",
      "volume": "11",
      "author": "AL Samuel",
      "year": "1967",
      "unstructured": "Samuel, A. L. Some studies in machine learning using the game of checkers II - recent progress. IBM J. Res. Develop. 11, 601\u2013617 (1967)",
      "journal-title": "IBM J. Res. Develop."
    },
    {
      "key": "BFnature16961_CR45",
      "unstructured": "Schaeffer, J., Hlynka, M. & Jussila, V. Temporal difference learning applied to a high-performance game-playing program. In 17th International Joint Conference on Artificial Intelligence, 529\u2013534 (2001)"
    },
    {
      "key": "BFnature16961_CR46",
      "doi-asserted-by": "publisher",
      "first-page": "215",
      "DOI": "10.1162/neco.1994.6.2.215",
      "volume": "6",
      "author": "G Tesauro",
      "year": "1994",
      "unstructured": "Tesauro, G. TD-gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput. 6, 215\u2013219 (1994)",
      "journal-title": "Neural Comput."
    },
    {
      "key": "BFnature16961_CR47",
      "unstructured": "Dahl, F. Honte, a Go-playing program using neural nets. In Machines that learn to play games, 205\u2013223 (Nova Science, 1999)"
    },
    {
      "key": "BFnature16961_CR48",
      "doi-asserted-by": "publisher",
      "first-page": "203",
      "DOI": "10.1007/s10472-011-9258-6",
      "volume": "61",
      "author": "CD Rosin",
      "year": "2011",
      "unstructured": "Rosin, C. D. Multi-armed bandits with episode context. Ann. Math. Artif. Intell. 61, 203\u2013230 (2011)",
      "journal-title": "Ann. Math. Artif. Intell."
    },
    {
      "key": "BFnature16961_CR49",
      "doi-asserted-by": "crossref",
      "unstructured": "Lanctot, M., Winands, M. H. M., Pepels, T. & Sturtevant, N. R. Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In IEEE Conference on Computational Intelligence and Games, 1\u20138 (2014)",
      "DOI": "10.1109/CIG.2014.6932903"
    },
    {
      "key": "BFnature16961_CR50",
      "first-page": "INRIA",
      "volume": "6062",
      "author": "S Gelly",
      "year": "2006",
      "unstructured": "Gelly, S., Wang, Y., Munos, R. & Teytaud, O. Modification of UCT with patterns in Monte-Carlo Go. Tech. Rep. 6062, INRIA (2006)",
      "journal-title": "Tech. Rep."
    },
    {
      "key": "BFnature16961_CR51",
      "doi-asserted-by": "crossref",
      "unstructured": "Silver, D. & Tesauro, G. Monte-Carlo simulation balancing. In 26th International Conference on Machine Learning, 119 (2009)",
      "DOI": "10.1145/1553374.1553495"
    },
    {
      "key": "BFnature16961_CR52",
      "doi-asserted-by": "crossref",
      "unstructured": "Huang, S.-C., Coulom, R. & Lin, S.-S. Monte-Carlo simulation balancing in practice. In 7th International Conference on Computers and Games, 81\u201392 (Springer-Verlag, 2011)",
      "DOI": "10.1007/978-3-642-17928-0_8"
    },
    {
      "key": "BFnature16961_CR53",
      "doi-asserted-by": "publisher",
      "first-page": "303",
      "DOI": "10.1109/TCIAIG.2010.2100396",
      "volume": "2",
      "author": "H Baier",
      "year": "2010",
      "unstructured": "Baier, H. & Drake, P. D. The power of forgetting: improving the last-good-reply policy in Monte Carlo Go. IEEE Trans. Comput. Intell. AI in Games 2, 303\u2013309 (2010)",
      "journal-title": "IEEE Trans. Comput. Intell. AI in Games"
    },
    {
      "key": "BFnature16961_CR54",
      "doi-asserted-by": "crossref",
      "unstructured": "Huang, S. & M\u00fcller, M. Investigating the limits of Monte-Carlo tree search methods in computer Go. In 8th International Conference on Computers and Games, 39\u201348 (2013)",
      "DOI": "10.1007/978-3-319-09165-5_4"
    },
    {
      "key": "BFnature16961_CR55",
      "doi-asserted-by": "publisher",
      "first-page": "36",
      "DOI": "10.1007/978-3-642-17928-0_4",
      "volume": "6515",
      "author": "RB Segal",
      "year": "2011",
      "unstructured": "Segal, R. B. On the scalability of parallel UCT. Computers and Games 6515, 36\u201347 (2011)",
      "journal-title": "Computers and Games"
    },
    {
      "key": "BFnature16961_CR56",
      "doi-asserted-by": "crossref",
      "unstructured": "Enzenberger, M. & M\u00fcller, M. A lock-free multithreaded Monte-Carlo tree search algorithm. In 12th Advances in Computer Games Conference, 14\u201320 (2009)",
      "DOI": "10.1007/978-3-642-12993-3_2"
    },
    {
      "key": "BFnature16961_CR57",
      "doi-asserted-by": "crossref",
      "unstructured": "Huang, S.-C., Coulom, R. & Lin, S.-S. Time management for Monte-Carlo tree search applied to the game of Go. In International Conference on Technologies and Applications of Artificial Intelligence, 462\u2013466 (2010)",
      "DOI": "10.1109/TAAI.2010.78"
    },
    {
      "key": "BFnature16961_CR58",
      "doi-asserted-by": "publisher",
      "first-page": "1856",
      "DOI": "10.1016/j.artint.2011.03.007",
      "volume": "175",
      "author": "S Gelly",
      "year": "2011",
      "unstructured": "Gelly, S. & Silver, D. Monte-Carlo tree search and rapid action value estimation in computer Go. Artif. Intell. 175, 1856\u20131875 (2011)",
      "journal-title": "Artif. Intell."
    },
    {
      "key": "BFnature16961_CR59",
      "doi-asserted-by": "publisher",
      "first-page": "131",
      "DOI": "10.3233/ICG-2011-34302",
      "volume": "34",
      "author": "P Baudi\u0161",
      "year": "2011",
      "unstructured": "Baudi\u0161, P. Balancing MCTS by dynamically adjusting the komi value. ICGA J. 34, 131 (2011)",
      "journal-title": "ICGA J."
    },
    {
      "key": "BFnature16961_CR60",
      "unstructured": "Baier, H. & Winands, M. H. Active opening book application for Monte-Carlo tree search in 19\u00d719 Go. In Benelux Conference on Artificial Intelligence, 3\u201310 (2011)"
    },
    {
      "key": "BFnature16961_CR61",
      "unstructured": "Dean, J. et al. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, 1223\u20131231 (2012)"
    },
    {
      "key": "BFnature16961_CR62",
      "unstructured": "Go ratings. http://www.goratings.org"
    }
  ],
  "container-title": "Nature",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "http://www.nature.com/articles/nature16961.pdf",
      "content-type": "application/pdf",
      "content-version": "vor",
      "intended-application": "text-mining"
    },
    {
      "URL": "http://www.nature.com/articles/nature16961",
      "content-type": "text/html",
      "content-version": "vor",
      "intended-application": "text-mining"
    },
    {
      "URL": "http://www.nature.com/articles/nature16961.pdf",
      "content-type": "application/pdf",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2023,
        5,
        18
      ]
    ],
    "date-time": "2023-05-18T17:49:53Z",
    "timestamp": 1684432193000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://www.nature.com/articles/nature16961"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2016,
        1,
        27
      ]
    ]
  },
  "references-count": 62,
  "journal-issue": {
    "issue": "7587",
    "published-print": {
      "date-parts": [
        [
          2016,
          1,
          28
        ]
      ]
    }
  },
  "alternative-id": [
    "BFnature16961"
  ],
  "URL": "http://dx.doi.org/10.1038/nature16961",
  "relation": {
    "has-review": [
      {
        "id-type": "doi",
        "id": "10.3410/f.726105619.793525345",
        "asserted-by": "object"
      }
    ]
  },
  "ISSN": [
    "0028-0836",
    "1476-4687"
  ],
  "subject": [],
  "container-title-short": "Nature",
  "published": {
    "date-parts": [
      [
        2016,
        1,
        27
      ]
    ]
  },
  "assertion": [
    {
      "value": "11 November 2015",
      "order": 1,
      "name": "received",
      "label": "Received",
      "group": {
        "name": "ArticleHistory",
        "label": "Article History"
      }
    },
    {
      "value": "5 January 2016",
      "order": 2,
      "name": "accepted",
      "label": "Accepted",
      "group": {
        "name": "ArticleHistory",
        "label": "Article History"
      }
    },
    {
      "value": "27 January 2016",
      "order": 3,
      "name": "first_online",
      "label": "First Online",
      "group": {
        "name": "ArticleHistory",
        "label": "Article History"
      }
    },
    {
      "value": "The authors declare no competing financial interests.",
      "order": 1,
      "name": "Ethics",
      "group": {
        "name": "EthicsHeading",
        "label": "Competing interests"
      }
    }
  ]
}