DOI: 10.1162/neco.2009.10-08-881. Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

10.1162/neco.2009.10-08-881

Crossref journal-article

MIT Press - Journals

Neural Computation (281)

Abstract

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

Bibliography

Turaga, S. C., Murray, J. F., Jain, V., Roth, F., Helmstaedter, M., Briggman, K., Denk, W., & Seung, H. S. (2010). Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation. Neural Computation, 22(2), 511â538.

Authors 8

Srinivas C. Turaga (first)
Joseph F. Murray (additional)
Viren Jain (additional)
Fabian Roth (additional)
Moritz Helmstaedter (additional)
Kevin Briggman (additional)
Winfried Denk (additional)
H. Sebastian Seung (additional)

References 32 Referenced 290

10.1109/TITB.2002.1006304
10.1007/978-3-540-69321-5_15
10.1109/34.969114
10.1016/j.conb.2006.08.010
10.1109/42.293928
{'key': 'B6', 'volume-title': 'Introduction to algorithms', 'author': 'Cormen T. H.', 'year': '2000'} / Introduction to algorithms by Cormen T. H. (2000)
10.1109/CVPR.2005.332
10.1371/journal.pbio.0020329
10.1145/1081870.1081948
10.1016/S0031-3203(01)00178-9
10.1023/B:VISI.0000022288.19776.77
10.1111/j.1365-2818.2005.01466.x
10.1109/CVPR.2003.1211452
10.1109/CVPR.1999.784979
{'key': 'B15', 'first-page': '142', 'author': 'Helmstaedter M. N.', 'year': '2007', 'journal-title': 'Society for Neuroscience Abstracts'} / Society for Neuroscience Abstracts by Helmstaedter M. N. (2007)
10.1109/ICCV.2007.4408909
10.1016/j.media.2008.05.002
10.1162/neco.1989.1.4.541
10.1109/5.726791
10.1007/3-540-49430-8_2
10.1016/j.media.2005.09.002
10.1109/TPAMI.2004.1261075
10.1016/j.jneumeth.2008.09.006
10.1109/72.363449
10.1109/TIP.2005.852470
10.1109/TMI.2007.898551
10.1109/34.868688
10.1136/bjo.83.8.902
10.1016/j.conb.2007.11.004
{'key': 'B30', 'volume-title': 'Information retrieval', 'author': 'Van Rijsbergen C.', 'year': '1979'} / Information retrieval by Van Rijsbergen C. (1979)
10.1109/TPAMI.2002.1114849
10.1098/rstb.1986.0056

Dates

Type	When
Created	15 years, 9 months ago (Nov. 18, 2009, 7:59 p.m.)
Deposited	4 years, 5 months ago (March 12, 2021, 4:37 p.m.)
Indexed	1 month, 3 weeks ago (June 27, 2025, 5:51 a.m.)
Issued	15 years, 6 months ago (Feb. 1, 2010)
Published	15 years, 6 months ago (Feb. 1, 2010)
Published Print	15 years, 6 months ago (Feb. 1, 2010)

Funders 0

None

BibTeX

@article{Turaga_2010, title={Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation}, volume={22}, ISSN={1530-888X}, url={http://dx.doi.org/10.1162/neco.2009.10-08-881}, DOI={10.1162/neco.2009.10-08-881}, number={2}, journal={Neural Computation}, publisher={MIT Press - Journals}, author={Turaga, Srinivas C. and Murray, Joseph F. and Jain, Viren and Roth, Fabian and Helmstaedter, Moritz and Briggman, Kevin and Denk, Winfried and Seung, H. Sebastian}, year={2010}, month=feb, pages={511–538} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        6,
        27
      ]
    ],
    "date-time": "2025-06-27T09:51:27Z",
    "timestamp": 1751017887509
  },
  "reference-count": 32,
  "publisher": "MIT Press - Journals",
  "issue": "2",
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        2010,
        2
      ]
    ]
  },
  "abstract": "<jats:p> Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. </jats:p><jats:p> We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. </jats:p><jats:p> In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types. </jats:p>",
  "DOI": "10.1162/neco.2009.10-08-881",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2009,
        11,
        19
      ]
    ],
    "date-time": "2009-11-19T00:59:10Z",
    "timestamp": 1258592350000
  },
  "page": "511-538",
  "source": "Crossref",
  "is-referenced-by-count": 290,
  "title": "Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation",
  "prefix": "10.1162",
  "volume": "22",
  "author": [
    {
      "given": "Srinivas C.",
      "family": "Turaga",
      "sequence": "first",
      "affiliation": [
        {
          "name": "Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A."
        }
      ]
    },
    {
      "given": "Joseph F.",
      "family": "Murray",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Howard Hughes Medical Institute, Cambridge, MA 02139, U.S.A."
        }
      ]
    },
    {
      "given": "Viren",
      "family": "Jain",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A."
        }
      ]
    },
    {
      "given": "Fabian",
      "family": "Roth",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Howard Hughes Medical Institute, Cambridge, MA 02139, U.S.A."
        }
      ]
    },
    {
      "given": "Moritz",
      "family": "Helmstaedter",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Max-Planck Institute for Medical Research, D-69120, Heidelberg, Germany"
        }
      ]
    },
    {
      "given": "Kevin",
      "family": "Briggman",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Max-Planck Institute for Medical Research, D-69120, Heidelberg, Germany"
        }
      ]
    },
    {
      "given": "Winfried",
      "family": "Denk",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Max-Planck Institute for Medical Research, D-69120, Heidelberg, Germany"
        }
      ]
    },
    {
      "given": "H. Sebastian",
      "family": "Seung",
      "sequence": "additional",
      "affiliation": [
        {
          "name": "Departments of Brain and Cognitive Sciences and Physics, Massachusetts Institute of Technology, Howard Hughes Medical Institute, Cambridge, MA 02139, U.S.A."
        }
      ]
    }
  ],
  "member": "281",
  "reference": [
    {
      "key": "B1",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/TITB.2002.1006304"
    },
    {
      "key": "B2",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/978-3-540-69321-5_15"
    },
    {
      "key": "B3",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/34.969114"
    },
    {
      "key": "B4",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.conb.2006.08.010"
    },
    {
      "key": "B5",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/42.293928"
    },
    {
      "key": "B6",
      "volume-title": "Introduction to algorithms",
      "author": "Cormen T. H.",
      "year": "2000"
    },
    {
      "key": "B7",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/CVPR.2005.332"
    },
    {
      "key": "B8",
      "doi-asserted-by": "publisher",
      "DOI": "10.1371/journal.pbio.0020329"
    },
    {
      "key": "B9",
      "doi-asserted-by": "publisher",
      "DOI": "10.1145/1081870.1081948"
    },
    {
      "key": "B10",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/S0031-3203(01)00178-9"
    },
    {
      "key": "B11",
      "doi-asserted-by": "publisher",
      "DOI": "10.1023/B:VISI.0000022288.19776.77"
    },
    {
      "key": "B12",
      "doi-asserted-by": "publisher",
      "DOI": "10.1111/j.1365-2818.2005.01466.x"
    },
    {
      "key": "B13",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/CVPR.2003.1211452"
    },
    {
      "key": "B14",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/CVPR.1999.784979"
    },
    {
      "key": "B15",
      "first-page": "142",
      "author": "Helmstaedter M. N.",
      "year": "2007",
      "journal-title": "Society for Neuroscience Abstracts"
    },
    {
      "key": "B16",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/ICCV.2007.4408909"
    },
    {
      "key": "B17",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.media.2008.05.002"
    },
    {
      "key": "B18",
      "doi-asserted-by": "publisher",
      "DOI": "10.1162/neco.1989.1.4.541"
    },
    {
      "key": "B19",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/5.726791"
    },
    {
      "key": "B20",
      "doi-asserted-by": "publisher",
      "DOI": "10.1007/3-540-49430-8_2"
    },
    {
      "key": "B21",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.media.2005.09.002"
    },
    {
      "key": "B22",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/TPAMI.2004.1261075"
    },
    {
      "key": "B23",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.jneumeth.2008.09.006"
    },
    {
      "key": "B24",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/72.363449"
    },
    {
      "key": "B25",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/TIP.2005.852470"
    },
    {
      "key": "B26",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/TMI.2007.898551"
    },
    {
      "key": "B27",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/34.868688"
    },
    {
      "key": "B28",
      "doi-asserted-by": "publisher",
      "DOI": "10.1136/bjo.83.8.902"
    },
    {
      "key": "B29",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/j.conb.2007.11.004"
    },
    {
      "key": "B30",
      "volume-title": "Information retrieval",
      "author": "Van Rijsbergen C.",
      "year": "1979"
    },
    {
      "key": "B31",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/TPAMI.2002.1114849"
    },
    {
      "key": "B32",
      "doi-asserted-by": "publisher",
      "DOI": "10.1098/rstb.1986.0056"
    }
  ],
  "container-title": "Neural Computation",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://www.mitpressjournals.org/doi/pdf/10.1162/neco.2009.10-08-881",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2021,
        3,
        12
      ]
    ],
    "date-time": "2021-03-12T21:37:43Z",
    "timestamp": 1615585063000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://direct.mit.edu/neco/article/22/2/511-538/7530"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        2010,
        2
      ]
    ]
  },
  "references-count": 32,
  "journal-issue": {
    "issue": "2",
    "published-print": {
      "date-parts": [
        [
          2010,
          2
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1162/neco.2009.10-08-881"
  ],
  "URL": "http://dx.doi.org/10.1162/neco.2009.10-08-881",
  "relation": {},
  "ISSN": [
    "0899-7667",
    "1530-888X"
  ],
  "subject": [],
  "container-title-short": "Neural Computation",
  "published": {
    "date-parts": [
      [
        2010,
        2
      ]
    ]
  }
}