DOI: 10.1162/neco.2008.03-08-727. Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity

Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity

10.1162/neco.2008.03-08-727

Crossref journal-article

MIT Press - Journals

Neural Computation (281)

Abstract

Recently multineuronal recording has allowed us to observe patterned firings, synchronization, oscillation, and global state transitions in the recurrent networks of central nervous systems. We propose a learning algorithm based on the process of information maximization in a recurrent network, which we call recurrent infomax (RI). RI maximizes information retention and thereby minimizes information loss through time in a network. We find that feeding in external inputs consisting of information obtained from photographs of natural scenes into an RI-based model of a recurrent network results in the appearance of Gabor-like selectivity quite similar to that existing in simple cells of the primary visual cortex. We find that without external input, this network exhibits cell assembly–like and synfire chain–like spontaneous activity as well as a critical neuronal avalanche. In addition, we find that RI embeds externally input temporal firing patterns to the network so that it spontaneously reproduces these patterns after learning. RI provides a simple framework to explain a wide range of phenomena observed in in vivo and in vitro neuronal networks, and it will provide a novel understanding of experimental results for multineuronal activity and plasticity from an information-theoretic point of view.

Bibliography

Tanaka, T., Kaneko, T., & Aoyagi, T. (2009). Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity. Neural Computation, 21(4), 1038â1067.

Authors 3

Takuma Tanaka (first)
Takeshi Kaneko (additional)
Toshio Aoyagi (additional)

References 35 Referenced 47

10.1016/S0079-6123(06)65002-4
10.1017/CBO9780511574566
10.1088/0954-898X/3/2/009
10.1162/089976602760805368
10.1103/PhysRevLett.59.381
10.1523/JNEUROSCI.23-35-11167.2003
10.1162/neco.1995.7.6.1129
10.1016/S0042-6989(97)00121-1
10.1152/jn.01250.2004
10.1038/nature01614
{'key': 'B11', 'volume-title': 'Elements of information theory', 'author': 'Cover T. M.', 'year': '2006'} / Elements of information theory by Cover T. M. (2006)
10.1038/990101
10.1016/j.neuron.2006.02.023
10.1093/cercor/bhj010
{'key': 'B15', 'volume-title': 'The theory of branching processes', 'author': 'Harris T. E.', 'year': '1989'} / The theory of branching processes by Harris T. E. (1989)
{'key': 'B16', 'volume-title': 'The organization of behavior: A neuropsychological theory', 'author': 'Hebb D. O.', 'year': '1949'} / The organization of behavior: A neuropsychological theory by Hebb D. O. (1949)
Hinton, G. E. & Sejnowski, T. J. (1983). Optimal perceptual inference. Washington, DC: IEEE Computer Society Press. (pp. 448–453).
10.1113/jphysiol.1959.sp006308
10.1126/science.1093173
10.1038/nn831
10.1109/2.36
10.1016/S0896-6273(01)00186-6
10.1162/089976602760407955
10.1523/JNEUROSCI.19-21-09497.1999
10.1038/381607a0
10.1016/S0042-6989(97)00169-7
10.1162/neco.2006.18.6.1318
10.1523/JNEUROSCI.2423-06.2006
10.1002/j.1538-7305.1948.tb01338.x
10.1126/science.271.5257.1870
10.1007/s10827-006-0014-6
10.1073/pnas.0500495102
10.1007/BF00326706
10.1523/JNEUROSCI.3508-05.2005
10.1038/nn1895

Dates

Type	When
Created	16 years, 10 months ago (Oct. 17, 2008, 11:27 a.m.)
Deposited	4 years, 5 months ago (March 12, 2021, 4:36 p.m.)
Indexed	4 weeks ago (Aug. 2, 2025, 12:23 a.m.)
Issued	16 years, 4 months ago (April 1, 2009)
Published	16 years, 4 months ago (April 1, 2009)
Published Print	16 years, 4 months ago (April 1, 2009)

Funders 0

None

BibTeX

@article{Tanaka_2009, title={Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity}, volume={21}, ISSN={1530-888X}, url={http://dx.doi.org/10.1162/neco.2008.03-08-727}, DOI={10.1162/neco.2008.03-08-727}, number={4}, journal={Neural Computation}, publisher={MIT Press - Journals}, author={Tanaka, Takuma and Kaneko, Takeshi and Aoyagi, Toshio}, year={2009}, month=apr, pages={1038–1067} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        2
      ]
    ],
    "date-time": "2025-08-02T04:23:44Z",
    "timestamp": 1754108624821
  },
  "reference-count": 35,
  "publisher": "MIT Press - Journals",
  "issue": "4",
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2009,
        4
      ]
    ]
  },
  "abstract": "<jats:p> Recently multineuronal recording has allowed us to observe patterned firings, synchronization, oscillation, and global state transitions in the recurrent networks of central nervous systems. We propose a learning algorithm based on the process of information maximization in a recurrent network, which we call recurrent infomax (RI). RI maximizes information retention and thereby minimizes information loss through time in a network. We find that feeding in external inputs consisting of information obtained from photographs of natural scenes into an RI-based model of a recurrent network results in the appearance of Gabor-like selectivity quite similar to that existing in simple cells of the primary visual cortex. We find that without external input, this network exhibits cell assembly\u2013like and synfire chain\u2013like spontaneous activity as well as a critical neuronal avalanche. In addition, we find that RI embeds externally input temporal firing patterns to the network so that it spontaneously reproduces these patterns after learning. RI provides a simple framework to explain a wide range of phenomena observed in in vivo and in vitro neuronal networks, and it will provide a novel understanding of experimental results for multineuronal activity and plasticity from an information-theoretic point of view. </jats:p>",
  "DOI": "10.1162/neco.2008.03-08-727",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2008,
        10,
        17
      ]
    ],
    "date-time": "2008-10-17T15:27:25Z",
    "timestamp": 1224257245000
  },
  "page": "1038-1067",
  "source": "Crossref",
  "is-referenced-by-count": 47,
  "title": "Recurrent Infomax Generates Cell Assemblies, Neuronal Avalanches, and Simple Cell-Like Selectivity",
  "prefix": "10.1162",
  "volume": "21",
  "author": [
    {
      "given": "Takuma",
      "family": "Tanaka",
      "sequence": "first",
      "affiliation": [
        {
          "name": "Department of Morphological Brain Science, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan"
        }
      ]
    },
    {
      "given": "Takeshi",
      "family": "Kaneko",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Department of Morphological Brain Science, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan, and CREST, Japan Science and Technology Agency, Kawaguchi, Saitama 332-0012, Japan"
        }
      ]
    },
    {
      "given": "Toshio",
      "family": "Aoyagi",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Department of Applied Analysis and Complex Dynamical Systems, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan, and CREST, Japan Science and Technology Agency, Kawaguchi, Saitama 332-0012, Japan"
        }
      ]
    }
  ],
  "member": "281",
  "reference": [
    {
      "key": "B1",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0079-6123(06)65002-4"
    },
    {
      "key": "B2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1017/CBO9780511574566"
    },
    {
      "key": "B3",
      "doi-asserted-by": "publisher",
      "DOI": "10.1088/0954-898X/3/2/009"
    },
    {
      "key": "B4",
      "doi-asserted-by": "publisher",
      "DOI": "10.1162/089976602760805368"
    },
    {
      "key": "B5",
      "doi-asserted-by": "publisher",
      "DOI": "10.1103/PhysRevLett.59.381"
    },
    {
      "key": "B6",
      "doi-asserted-by": "publisher",
      "DOI": "10.1523/JNEUROSCI.23-35-11167.2003"
    },
    {
      "key": "B7",
      "doi-asserted-by": "publisher",
      "DOI": "10.1162/neco.1995.7.6.1129"
    },
    {
      "key": "B8",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0042-6989(97)00121-1"
    },
    {
      "key": "B9",
      "doi-asserted-by": "publisher",
      "DOI": "10.1152/jn.01250.2004"
    },
    {
      "key": "B10",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/nature01614"
    },
    {
      "key": "B11",
      "volume-title": "Elements of information theory",
      "author": "Cover T. M.",
      "year": "2006"
    },
    {
      "key": "B12",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/990101"
    },
    {
      "key": "B13",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.neuron.2006.02.023"
    },
    {
      "key": "B14",
      "doi-asserted-by": "publisher",
      "DOI": "10.1093/cercor/bhj010"
    },
    {
      "key": "B15",
      "volume-title": "The theory of branching processes",
      "author": "Harris T. E.",
      "year": "1989"
    },
    {
      "key": "B16",
      "volume-title": "The organization of behavior: A neuropsychological theory",
      "author": "Hebb D. O.",
      "year": "1949"
    },
    {
      "key": "B17",
      "unstructured": "Hinton, G. E. & Sejnowski, T. J. (1983). Optimal perceptual inference. Washington, DC: IEEE Computer Society Press.  (pp. 448\u2013453)."
    },
    {
      "key": "B18",
      "doi-asserted-by": "publisher",
      "DOI": "10.1113/jphysiol.1959.sp006308"
    },
    {
      "key": "B19",
      "doi-asserted-by": "publisher",
      "DOI": "10.1126/science.1093173"
    },
    {
      "key": "B20",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/nn831"
    },
    {
      "key": "B21",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/2.36"
    },
    {
      "key": "B22",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0896-6273(01)00186-6"
    },
    {
      "key": "B23",
      "doi-asserted-by": "publisher",
      "DOI": "10.1162/089976602760407955"
    },
    {
      "key": "B24",
      "doi-asserted-by": "publisher",
      "DOI": "10.1523/JNEUROSCI.19-21-09497.1999"
    },
    {
      "key": "B25",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/381607a0"
    },
    {
      "key": "B26",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0042-6989(97)00169-7"
    },
    {
      "key": "B27",
      "doi-asserted-by": "publisher",
      "DOI": "10.1162/neco.2006.18.6.1318"
    },
    {
      "key": "B28",
      "doi-asserted-by": "publisher",
      "DOI": "10.1523/JNEUROSCI.2423-06.2006"
    },
    {
      "key": "B29",
      "doi-asserted-by": "publisher",
      "DOI": "10.1002/j.1538-7305.1948.tb01338.x"
    },
    {
      "key": "B30",
      "doi-asserted-by": "publisher",
      "DOI": "10.1126/science.271.5257.1870"
    },
    {
      "key": "B31",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/s10827-006-0014-6"
    },
    {
      "key": "B32",
      "doi-asserted-by": "publisher",
      "DOI": "10.1073/pnas.0500495102"
    },
    {
      "key": "B33",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/BF00326706"
    },
    {
      "key": "B34",
      "doi-asserted-by": "publisher",
      "DOI": "10.1523/JNEUROSCI.3508-05.2005"
    },
    {
      "key": "B35",
      "doi-asserted-by": "publisher",
      "DOI": "10.1038/nn1895"
    }
  ],
  "container-title": "Neural Computation",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://www.mitpressjournals.org/doi/pdf/10.1162/neco.2008.03-08-727",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2021,
        3,
        12
      ]
    ],
    "date-time": "2021-03-12T21:36:33Z",
    "timestamp": 1615584993000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://direct.mit.edu/neco/article/21/4/1038-1067/7409"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2009,
        4
      ]
    ]
  },
  "references-count": 35,
  "journal-issue": {
    "issue": "4",
    "published-print": {
      "date-parts": [
        [
          2009,
          4
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1162/neco.2008.03-08-727"
  ],
  "URL": "http://dx.doi.org/10.1162/neco.2008.03-08-727",
  "relation": {},
  "ISSN": [
    "0899-7667",
    "1530-888X"
  ],
  "subject": [],
  "container-title-short": "Neural Computation",
  "published": {
    "date-parts": [
      [
        2009,
        4
      ]
    ]
  }
}