DOI: 10.1098/rstb.1976.0090. Early processing of visual information

10.1098/rstb.1976.0090

Crossref journal-article

The Royal Society

Philosophical Transactions of the Royal Society of London. B, Biological Sciences (175)

Abstract

An introduction is given to a theory of early visual information processing. The theory has been implemented, and examples are given of images at various stages of analysis. It is argued that the first step of consequence is to compute a primitive but rich description of the grey-level changes present in an image. The description is expressed in a vocabulary of kinds of intensity change (EDGE, SHADING-EDGE, EXTENDED-EDGE, LINE, BLOB etc.). Modifying parameters are bound to the elements in the description, specifying their POSITION, ORIENTATION, TERMINATION points, CONTRAST, SIZE and FUZZINESS. This description is obtained from the intensity array by fixed techniques, and it is called the primal sketch . For most images, the primal sketch is large and unwieldy. The second important step in visual information processing is to group its contents in a way that is appropriate for later recognition. From our ability to interpret drawings with little semantic content, one may infer the presence in our perceptual equipment of symbolic processes that can define ‘ placetokens’ in an image in various ways, and can group them according to certain rules. Homomorphic techniques fail to account for many of these grouping phenomena, whose explanations require mechanisms of construction rather than mechanisms of detection. The necessary grouping of elements in the primal sketch may be achieved by a mechanism that has available the processes inferred from above, together with the ability to select items by first order discriminations acting on the elements’ parameters. Only occasionally do these mechanisms use downward-flowing information about the contents of the particular image being processed. It is argued that ‘non-attentive’ vision is in practice implemented by these grouping operations and first order discriminations acting on the primal sketch. The class of computations so obtained differs slightly from the class of second order operations on the intensity array. The extraction of a form from the primal sketch using these techniques amounts to the separation of figure from ground. It is concluded that most of the separation can be carried out by using techniques that do not depend upon the particular image in question. Therefore, figure-ground separation can normally precede the description of the shape of the extracted form. Up to this point, higher-level knowledge and purpose are brought to bear on only a few of the decisions taken during the processing. This relegates the widespread use of downward-flowing information to a later stage than is found in current machine-vision programs, and implies that such knowledge should influence the control of, rather than interfering with, the actual data-processing that is taking place lower down.

Bibliography

Early processing of visual information. (1976). Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 275(942), 483â519.

Authors 0

None

References 35 Referenced 637

Bajcsy R . 1972 C o m p u ter identification of textured visual scenes. Stanford A.I. Lab. Memo. 180. (10.21236/AD0759712)
{'key': 'p_2', 'first-page': '56', 'article-title': "S um m ation a n d inhibition in the frog's retin a. J . Physiol", 'volume': '119', 'author': 'Barlow H .', 'year': '1953', 'journal-title': 'Lond.'} / Lond. / S um m ation a n d inhibition in the frog's retin a. J . Physiol by Barlow H . (1953)
B rodatz P. 1966 Textures: a photographic albumfor artists and designers. New Y ork: D over Publications.
F re u d er E. C. 1974 A co m p u ter vision system for visual recognition using active know ledge M .I. T.A.I. Lab. Technical Report 345.
10.1126/science.180.4091.1194
H erskovits A. & Binford T . O . 1970 O n b o u n d ary detection. M .I.T .A .I. Lab. Memo 183.
H o rn B. K . P. 1973 T h e B in fo rd -H o rn linefjnder. M .I.T .A .I. Lab. Memo. 285.
{'key': 'p_8', 'first-page': '106', 'article-title': "R eceptive fields, b in o cu lar in teractio n an d functional arch itectu re in th e c a t's visual cortex. J . Physiol", 'volume': '160', 'year': '1962', 'journal-title': 'Lond.'} / Lond. / R eceptive fields, b in o cu lar in teractio n an d functional arch itectu re in th e c a t's visual cortex. J . Physiol (1962)
10.1145/321623.321635
{'key': 'p_10', 'first-page': '634', 'article-title': 'A n o p era to r w hich recognizes edges an d lines. J', 'volume': '20', 'year': '1973', 'journal-title': 'Ass. Comput. Mach.'} / Ass. Comput. Mach. / A n o p era to r w hich recognizes edges an d lines. J (1973)
Ju lesz B. 1962 V isual p a tte rn discrim ination. IR E Transactions o f Information Theory I T - 8 8T-92. (10.1109/TIT.1962.1057694)
Ju lesz B. 1971 Foundations o f cyclopean perception.C hicago: T h e U niversity of Chicago Press.
{'key': 'p_13', 'first-page': '34', 'article-title': 'E xperim ents in the visual perceptio n of texture. Sci', 'volume': '232', 'author': 'Julesz B.', 'year': '1975', 'journal-title': 'Am.'} / Am. / E xperim ents in the visual perceptio n of texture. Sci by Julesz B. (1975)
10.1068/p020391
K anizsa G . 1955 M argini quasi-percettivi in cam pi con stim ulazioni om ogenea. Rivista di Psicologia 49 7-30.
{'key': 'p_16', 'first-page': '1940', 'article-title': "W h a t th e frog's eye tells th e frog's b rain", 'volume': '47', 'author': 'Pitts W .', 'year': '1959', 'journal-title': 'Proc. Inst. Radio Engrs'} / Proc. Inst. Radio Engrs / W h a t th e frog's eye tells th e frog's b rain by Pitts W . (1959)
M c C arth y J . etal 1963 LISP 1.5 Programmer's Manual. C am bridge M ass.: T h e M .I.T . Press.
M acleod I. D. G. 1970 O n finding stru ctu re in pictures. In Picture language machines (ed. S. K an eff) p. 231. N ew Y ork: A cadem ic Press.
10.1016/0042-6989(73)90201-0
M arcus M . P. 1974 W ait-and-see strategies for parsing n a tu ra l language. M .I.T .A .I. Lab. Working Paper 75.
10.1098/rstb.1971.0078
M a rr D. 1974 A note on the com putation of b in o cu lar disparity in a symbolic low-level visual processor M .I. T.A .I. Lab. Memo 327.
M a rr D. 1976a T echnical problem s in the early processing of visual inform ation. (In p rep aratio n .)
10.1101/SQB.1976.040.01.060
M insky M . & P apert S. 1969 Perceptrons. C am bridge M ass.: M .I.T . Press.
{'key': 'p_26', 'first-page': '33', 'article-title': 'H u m a n perception of hom ogeneous d o t p atterns', 'volume': '3', 'year': '1974', 'journal-title': 'Perception'} / Perception / H u m a n perception of hom ogeneous d o t p atterns (1974)
O 'C allaghan J . F. 19746 C om puting the p ercep tu al boundaries o f d o t p atterns. Computer graphics and image processing 3 141-162. (10.1016/S0146-664X(74)80004-3)
Poggio T . & R eich ard t W . 1976 V isual control o f o rientation behaviour in the fly. P art I I : tow ards the underlying neural interactions. Quart. Revs Biophys. (In the press.) (10.1017/S0033583500002535)
R atliff F. 1965 Mach bands: quantitative studies on neural networks in the retina. San Francisco: H olden-D ay.
R oberts L. 1963 M achine perception o f three-dim ensional solids. Technical 315 L incoln L ab o rato ry M .I.T .
10.1109/T-C.1971.223290
10.1109/T-C.1972.223573
S hirai Y. 1973 A context-sensitive line finder for recognition o f polyhedra. Artificial intelligence 4 95- 120. (10.1016/0004-3702(73)90002-7)
W altz D. 1975 U nderstan d in g line draw ings of scenes w ith shadows. In : The psychology of computer vision (Ed. P. H . W inston) pp. 19-91. N ew Y ork: M cG raw -H ill.
{'key': 'p_35', 'first-page': '301', 'article-title': 'U ntersuchungen zur L ehre von d er G estalt', 'volume': '4', 'year': '1923', 'journal-title': 'II. Psychol. Forsch.'} / II. Psychol. Forsch. / U ntersuchungen zur L ehre von d er G estalt (1923)

Dates

Type	When
Created	18 years, 8 months ago (Dec. 18, 2006, 5:23 p.m.)
Deposited	4 years, 6 months ago (Feb. 20, 2021, 9:14 a.m.)
Indexed	3 weeks, 1 day ago (July 31, 2025, 11:47 p.m.)
Issued	48 years, 10 months ago (Oct. 19, 1976)
Published	48 years, 10 months ago (Oct. 19, 1976)
Published Online	28 years, 7 months ago (Jan. 1, 1997)
Published Print	48 years, 10 months ago (Oct. 19, 1976)

Funders 0

None

BibTeX

@article{1976, volume={275}, ISSN={2054-0280}, url={http://dx.doi.org/10.1098/rstb.1976.0090}, DOI={10.1098/rstb.1976.0090}, number={942}, journal={Philosophical Transactions of the Royal Society of London. B, Biological Sciences}, publisher={The Royal Society}, year={1976}, month=oct, pages={483–519} }

JSON

{
  "indexed": {
    "date-parts": [
      [
        2025,
        8,
        1
      ]
    ],
    "date-time": "2025-08-01T03:47:20Z",
    "timestamp": 1754020040162
  },
  "reference-count": 35,
  "publisher": "The Royal Society",
  "issue": "942",
  "license": [
    {
      "start": {
        "date-parts": [
          [
            1976,
            10,
            19
          ]
        ],
        "date-time": "1976-10-19T00:00:00Z",
        "timestamp": 214531200000
      },
      "content-version": "tdm",
      "delay-in-days": 0,
      "URL": "https://royalsociety.org/journals/ethics-policies/data-sharing-mining/"
    }
  ],
  "content-domain": {
    "domain": [],
    "crossmark-restriction": false
  },
  "published-print": {
    "date-parts": [
      [
        1976,
        10,
        19
      ]
    ]
  },
  "abstract": "<jats:p>\n            An introduction is given to a theory of early visual information processing. The theory has been implemented, and examples are given of images at various stages of analysis. It is argued that the first step of consequence is to compute a primitive but rich description of the grey-level changes present in an image. The description is expressed in a vocabulary of kinds of intensity change (EDGE, SHADING-EDGE, EXTENDED-EDGE, LINE, BLOB etc.). Modifying parameters are bound to the elements in the description, specifying their POSITION, ORIENTATION, TERMINATION points, CONTRAST, SIZE and FUZZINESS. This description is obtained from the intensity array by fixed techniques, and it is called the\n            <jats:italic>primal sketch</jats:italic>\n            . For most images, the primal sketch is large and unwieldy. The second important step in visual information processing is to group its contents in a way that is appropriate for later recognition. From our ability to interpret drawings with little semantic content, one may infer the presence in our perceptual equipment of symbolic processes that can define \u2018 placetokens\u2019 in an image in various ways, and can group them according to certain rules. Homomorphic techniques fail to account for many of these grouping phenomena, whose explanations require mechanisms of construction rather than mechanisms of detection. The necessary grouping of elements in the primal sketch may be achieved by a mechanism that has available the processes inferred from above, together with the ability to select items by first order discriminations acting on the elements\u2019 parameters. Only occasionally do these mechanisms use downward-flowing information about the contents of the particular image being processed. It is argued that \u2018non-attentive\u2019 vision is in practice implemented by these grouping operations and first order discriminations acting on the primal sketch. The class of computations so obtained differs slightly from the class of second order operations on the intensity array. The extraction of a form from the primal sketch using these techniques amounts to the separation of figure from ground. It is concluded that most of the separation can be carried out by using techniques that do not depend upon the particular image in question. Therefore, figure-ground separation can normally\n            <jats:italic>precede</jats:italic>\n            the description of the shape of the extracted form. Up to this point, higher-level knowledge and purpose are brought to bear on only a few of the decisions taken during the processing. This relegates the widespread use of downward-flowing information to a later stage than is found in current machine-vision programs, and implies that such knowledge should influence the control of, rather than interfering with, the actual data-processing that is taking place lower down.\n          </jats:p>",
  "DOI": "10.1098/rstb.1976.0090",
  "type": "journal-article",
  "created": {
    "date-parts": [
      [
        2006,
        12,
        18
      ]
    ],
    "date-time": "2006-12-18T22:23:53Z",
    "timestamp": 1166480633000
  },
  "page": "483-519",
  "source": "Crossref",
  "is-referenced-by-count": 637,
  "title": "Early processing of visual information",
  "prefix": "10.1098",
  "volume": "275",
  "member": "175",
  "published-online": {
    "date-parts": [
      [
        1997,
        1
      ]
    ]
  },
  "reference": [
    {
      "key": "p_1",
      "doi-asserted-by": "crossref",
      "unstructured": "Bajcsy R . 1972 C o m p u ter identification of textured visual scenes. Stanford A.I. Lab. Memo. 180.",
      "DOI": "10.21236/AD0759712"
    },
    {
      "key": "p_2",
      "first-page": "56",
      "article-title": "S um m ation a n d inhibition in the frog's retin a. J . Physiol",
      "volume": "119",
      "author": "Barlow H .",
      "year": "1953",
      "journal-title": "Lond."
    },
    {
      "key": "p_3",
      "unstructured": "B rodatz P. 1966 Textures: a photographic albumfor artists and designers. New Y ork: D over Publications."
    },
    {
      "key": "p_4",
      "unstructured": "F re u d er E. C. 1974 A co m p u ter vision system for visual recognition using active know ledge M .I. T.A.I. Lab. Technical Report 345."
    },
    {
      "key": "p_5",
      "doi-asserted-by": "publisher",
      "DOI": "10.1126/science.180.4091.1194"
    },
    {
      "key": "p_6",
      "unstructured": "H erskovits A. & Binford T . O . 1970 O n b o u n d ary detection. M .I.T .A .I. Lab. Memo 183."
    },
    {
      "key": "p_7",
      "unstructured": "H o rn B. K . P. 1973 T h e B in fo rd -H o rn linefjnder. M .I.T .A .I. Lab. Memo. 285."
    },
    {
      "key": "p_8",
      "first-page": "106",
      "article-title": "R eceptive fields, b in o cu lar in teractio n an d functional arch itectu re in th e c a t's visual cortex. J . Physiol",
      "volume": "160",
      "year": "1962",
      "journal-title": "Lond."
    },
    {
      "key": "p_9",
      "doi-asserted-by": "publisher",
      "DOI": "10.1145/321623.321635"
    },
    {
      "key": "p_10",
      "first-page": "634",
      "article-title": "A n o p era to r w hich recognizes edges an d lines. J",
      "volume": "20",
      "year": "1973",
      "journal-title": "Ass. Comput. Mach."
    },
    {
      "key": "p_11",
      "doi-asserted-by": "crossref",
      "unstructured": "Ju lesz B. 1962 V isual p a tte rn discrim ination. IR E Transactions o f Information Theory I T - 8 8T-92.",
      "DOI": "10.1109/TIT.1962.1057694"
    },
    {
      "key": "p_12",
      "unstructured": "Ju lesz B. 1971 Foundations o f cyclopean perception.C hicago: T h e U niversity of Chicago Press."
    },
    {
      "key": "p_13",
      "first-page": "34",
      "article-title": "E xperim ents in the visual perceptio n of texture. Sci",
      "volume": "232",
      "author": "Julesz B.",
      "year": "1975",
      "journal-title": "Am."
    },
    {
      "key": "p_14",
      "doi-asserted-by": "publisher",
      "DOI": "10.1068/p020391"
    },
    {
      "key": "p_15",
      "unstructured": "K anizsa G . 1955 M argini quasi-percettivi in cam pi con stim ulazioni om ogenea. Rivista di Psicologia 49 7-30."
    },
    {
      "key": "p_16",
      "first-page": "1940",
      "article-title": "W h a t th e frog's eye tells th e frog's b rain",
      "volume": "47",
      "author": "Pitts W .",
      "year": "1959",
      "journal-title": "Proc. Inst. Radio Engrs"
    },
    {
      "key": "p_17",
      "unstructured": "M c C arth y J . etal 1963 LISP 1.5 Programmer's Manual. C am bridge M ass.: T h e M .I.T . Press."
    },
    {
      "key": "p_18",
      "unstructured": "M acleod I. D. G. 1970 O n finding stru ctu re in pictures. In Picture language machines (ed. S. K an eff) p. 231. N ew Y ork: A cadem ic Press."
    },
    {
      "key": "p_19",
      "doi-asserted-by": "publisher",
      "DOI": "10.1016/0042-6989(73)90201-0"
    },
    {
      "key": "p_20",
      "unstructured": "M arcus M . P. 1974 W ait-and-see strategies for parsing n a tu ra l language. M .I.T .A .I. Lab. Working Paper 75."
    },
    {
      "key": "p_21",
      "doi-asserted-by": "publisher",
      "DOI": "10.1098/rstb.1971.0078"
    },
    {
      "key": "p_22",
      "unstructured": "M a rr D. 1974 A note on the com putation of b in o cu lar disparity in a symbolic low-level visual processor M .I. T.A .I. Lab. Memo 327."
    },
    {
      "key": "p_23",
      "unstructured": "M a rr D. 1976a T echnical problem s in the early processing of visual inform ation. (In p rep aratio n .)"
    },
    {
      "key": "p_24",
      "doi-asserted-by": "publisher",
      "DOI": "10.1101/SQB.1976.040.01.060"
    },
    {
      "key": "p_25",
      "unstructured": "M insky M . & P apert S. 1969 Perceptrons. C am bridge M ass.: M .I.T . Press."
    },
    {
      "key": "p_26",
      "first-page": "33",
      "article-title": "H u m a n perception of hom ogeneous d o t p atterns",
      "volume": "3",
      "year": "1974",
      "journal-title": "Perception"
    },
    {
      "key": "p_27",
      "doi-asserted-by": "crossref",
      "unstructured": "O 'C allaghan J . F. 19746 C om puting the p ercep tu al boundaries o f d o t p atterns. Computer graphics and image processing 3 141-162.",
      "DOI": "10.1016/S0146-664X(74)80004-3"
    },
    {
      "key": "p_28",
      "doi-asserted-by": "crossref",
      "unstructured": "Poggio T . & R eich ard t W . 1976 V isual control o f o rientation behaviour in the fly. P art I I : tow ards the underlying neural interactions. Quart. Revs Biophys. (In the press.)",
      "DOI": "10.1017/S0033583500002535"
    },
    {
      "key": "p_29",
      "unstructured": "R atliff F. 1965 Mach bands: quantitative studies on neural networks in the retina. San Francisco: H olden-D ay."
    },
    {
      "key": "p_30",
      "unstructured": "R oberts L. 1963 M achine perception o f three-dim ensional solids. Technical 315 L incoln L ab o rato ry M .I.T ."
    },
    {
      "key": "p_31",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/T-C.1971.223290"
    },
    {
      "key": "p_32",
      "doi-asserted-by": "publisher",
      "DOI": "10.1109/T-C.1972.223573"
    },
    {
      "key": "p_33",
      "doi-asserted-by": "crossref",
      "unstructured": "S hirai Y. 1973 A context-sensitive line finder for recognition o f polyhedra. Artificial intelligence 4 95- 120.",
      "DOI": "10.1016/0004-3702(73)90002-7"
    },
    {
      "key": "p_34",
      "unstructured": "W altz D. 1975 U nderstan d in g line draw ings of scenes w ith shadows. In : The psychology of computer vision (Ed. P. H . W inston) pp. 19-91. N ew Y ork: M cG raw -H ill."
    },
    {
      "key": "p_35",
      "first-page": "301",
      "article-title": "U ntersuchungen zur L ehre von d er G estalt",
      "volume": "4",
      "year": "1923",
      "journal-title": "II. Psychol. Forsch."
    }
  ],
  "container-title": "Philosophical Transactions of the Royal Society of London. B, Biological Sciences",
  "original-title": [],
  "language": "en",
  "link": [
    {
      "URL": "https://royalsocietypublishing.org/doi/pdf/10.1098/rstb.1976.0090",
      "content-type": "application/pdf",
      "content-version": "vor",
      "intended-application": "text-mining"
    },
    {
      "URL": "https://royalsocietypublishing.org/doi/pdf/10.1098/rstb.1976.0090",
      "content-type": "unspecified",
      "content-version": "vor",
      "intended-application": "similarity-checking"
    }
  ],
  "deposited": {
    "date-parts": [
      [
        2021,
        2,
        20
      ]
    ],
    "date-time": "2021-02-20T14:14:15Z",
    "timestamp": 1613830455000
  },
  "score": 1,
  "resource": {
    "primary": {
      "URL": "https://royalsocietypublishing.org/doi/10.1098/rstb.1976.0090"
    }
  },
  "subtitle": [],
  "short-title": [],
  "issued": {
    "date-parts": [
      [
        1976,
        10,
        19
      ]
    ]
  },
  "references-count": 35,
  "journal-issue": {
    "issue": "942",
    "published-print": {
      "date-parts": [
        [
          1976,
          10,
          19
        ]
      ]
    }
  },
  "alternative-id": [
    "10.1098/rstb.1976.0090"
  ],
  "URL": "http://dx.doi.org/10.1098/rstb.1976.0090",
  "relation": {},
  "ISSN": [
    "0080-4622",
    "2054-0280"
  ],
  "subject": [],
  "container-title-short": "Phil. Trans. R. Soc. Lond. B",
  "published": {
    "date-parts": [
      [
        1976,
        10,
        19
      ]
    ]
  }
}