A dynamically configurable coprocessor for convolutional neural networks
10.1145/1816038.1815993
Crossref journal-article
Association for Computing Machinery (ACM)
ACM SIGARCH Computer Architecture News (320)
Abstract

Convolutional neural networks (CNN) applications range from recognition and reasoning (such as handwriting recognition, facial expression recognition and video surveillance) to intelligent text applications such as semantic text analysis and natural language processing applications. Two key observations drive the design of a new architecture for CNN. First, CNN workloads exhibit a widely varying mix of three types of parallelism : parallelism within a convolution operation, intra-output parallelism where multiple input sources (features) are combined to create a single output, and inter-output parallelism where multiple, independent outputs (features) are computed simultaneously. Workloads differ significantly across different CNN applications, and across different layers of a CNN. Second, the number of processing elements in an architecture continues to scale (as per Moore's law) much faster than off-chip memory bandwidth (or pin-count) of chips. Based on these two observations, we show that for a given number of processing elements and off-chip memory bandwidth, a new CNN hardware architecture that dynamically configures the hardware on-the-fly to match the specific mix of parallelism in a given workload gives the best throughput performance. Our CNN compiler automatically translates high abstraction network specification into a parallel microprogram (a sequence of low-level VLIW instructions) that is mapped, scheduled and executed by the coprocessor. Compared to a 2.3 GHz quad-core, dual socket Intel Xeon, 1.35 GHz C870 GPU, and a 200 MHz FPGA implementation, our 120 MHz dynamically configurable architecture is 4x to 8x faster. This is the first CNN architecture to achieve real-time video stream processing (25 to 30 frames per second) on a wide range of object detection and recognition tasks.

Bibliography

Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010). A dynamically configurable coprocessor for convolutional neural networks. ACM SIGARCH Computer Architecture News, 38(3), 247–257.

Authors 4
  1. Srimat Chakradhar (first)
  2. Murugan Sankaradas (additional)
  3. Venkata Jakkula (additional)
  4. Srihari Cadambi (additional)
References 30 Referenced 132
  1. 10.1109/5.726791
  2. 10.1145/1390156.1390177
  3. Benkrid , K. ; Belkacemi , S. , " Design and implementation of a 2D convolution core for video applications on FPGAs," Digital and Computational Video, 2002 . DCV 2002. Proceedings. Third International Workshop on , pp. 85 -- 92 , 14--15 Nov. 2002 . Benkrid, K.; Belkacemi, S., "Design and implementation of a 2D convolution core for video applications on FPGAs," Digital and Computational Video, 2002. DCV 2002. Proceedings. Third International Workshop on, pp. 85--92, 14--15 Nov. 2002. / DCV 2002. Proceedings. Third International Workshop on by Benkrid K. (2002)
  4. 10.1109/TCSII.2005.857091
  5. 10.1109/TCSII.2006.886898
  6. 10.1109/TNN.2006.883002
  7. 10.1007/s11265-005-4961-3
  8. 10.1145/1390156.1390170
  9. 10.1109/FPL.2009.5272559
  10. Dixon , J. D. ( 1981 ). Asymptotically fast factorization of integers. Math. Comput., 36, 255--260 . Dixon, J. D. (1981). Asymptotically fast factorization of integers. Math. Comput., 36, 255--260. / Asymptotically fast factorization of integers. Math. Comput., 36, 255--260 by Dixon J. D. (1981)
  11. 10.5555/1507435.1507438
  12. Haykin , S. ( 2008 ). Neural networks and learning machines . Prentice Hall . Haykin, S. (2008). Neural networks and learning machines. Prentice Hall. / Neural networks and learning machines by Haykin S. (2008)
  13. 10.1109/VLSIC.2005.1469371
  14. Lisboa , P. , Ifeachor , E. , & Szczepaniak , P. ( 2009 ). Artificial neural networks in Biomedicine . Springer Lisboa, P., Ifeachor, E., & Szczepaniak, P. (2009). Artificial neural networks in Biomedicine. Springer / Artificial neural networks in Biomedicine by Lisboa P. (2009)
  15. McNelis , P. D. ( 2005 ). Neural Networks in Finance: Gaining Predictive Edge in the Market . Academic Press . McNelis, P. D. (2005). Neural Networks in Finance: Gaining Predictive Edge in the Market. Academic Press. / Neural Networks in Finance: Gaining Predictive Edge in the Market by McNelis P. D. (2005)
  16. 10.1109/MLSP.2008.4685487
  17. 10.1109/CVPR.2006.200
  18. 10.1109/ISSCC.2006.1696216
  19. Nichols , K. , Moussa , M. , & Areibi , S. ( 2002 ). Feasibility of floating-point arithmetic in FPGA based artificial neural networks . Proceedings of the 15th International Conference on Computer Applications in Industry and Engineering . San Diego, California Nichols, K., Moussa, M., & Areibi, S. (2002). Feasibility of floating-point arithmetic in FPGA based artificial neural networks. Proceedings of the 15th International Conference on Computer Applications in Industry and Engineering. San Diego, California / Proceedings of the 15th International Conference on Computer Applications in Industry and Engineering by Nichols K. (2002)
  20. Nomura , O. , & Morie , T. ( 2007 ). Projection-Field-Type VLSI Convolutional Neural Networks Using Merged/Mixed Analog-Digital approach . International Conference on Neural Information Processing (pp. 1081--1090) . Springer-Verlag. Nomura, O., & Morie, T. (2007). Projection-Field-Type VLSI Convolutional Neural Networks Using Merged/Mixed Analog-Digital approach. International Conference on Neural Information Processing (pp. 1081--1090). Springer-Verlag. / International Conference on Neural Information Processing (pp. 1081--1090) by Nomura O. (2007)
  21. Omondi , A. , & Rajapakse , J. ( 2006 ). FPGA Implementations of Neural Networks . Springer . Omondi, A., & Rajapakse, J. (2006). FPGA Implementations of Neural Networks. Springer. (10.1007/0-387-28487-7) / FPGA Implementations of Neural Networks by Omondi A. (2006)
  22. Prasad , B. , & Prasanna , S. ( 2008 ). Speech , Audio, Image and Biomedical Signal Processing using Neural Networks . Springer . Prasad, B., & Prasanna, S. (2008). Speech, Audio, Image and Biomedical Signal Processing using Neural Networks. Springer. (10.1007/978-3-540-75398-8) / Audio, Image and Biomedical Signal Processing using Neural Networks by Prasad B. (2008)
  23. 10.5555/1484785.1484787
  24. Wolf , D. F. , Romero , R. A. , & Marques , E. ( 2001 ). Using embedded processors in hardware models of artificial neural networks . Proceedings of SBAI - Simposio Brasileiro de Automao Inteligente, (pp. 78--83) . Wolf, D. F., Romero, R. A., & Marques, E. (2001). Using embedded processors in hardware models of artificial neural networks. Proceedings of SBAI - Simposio Brasileiro de Automao Inteligente, (pp. 78--83). / Proceedings of SBAI - Simposio Brasileiro de Automao Inteligente, (pp. 78--83) by Wolf D. F. (2001)
  25. 10.1109/72.554195
  26. 10.1007/978-3-642-03767-2_10
  27. 10.1109/CVPR.2005.254
  28. 10.1109/CVPR.2005.177
  29. 10.1145/1553374.1553486
  30. 10.1145/1553374.1553453
Dates
Type When
Created 12 years, 10 months ago (Oct. 11, 2012, 10:55 a.m.)
Deposited 2 months, 1 week ago (June 18, 2025, 7:22 a.m.)
Indexed 1 month, 3 weeks ago (July 5, 2025, 12:25 a.m.)
Issued 15 years, 2 months ago (June 19, 2010)
Published 15 years, 2 months ago (June 19, 2010)
Published Online 15 years, 2 months ago (June 19, 2010)
Published Print 15 years, 2 months ago (June 19, 2010)
Funders 0

None

@article{Chakradhar_2010, title={A dynamically configurable coprocessor for convolutional neural networks}, volume={38}, ISSN={0163-5964}, url={http://dx.doi.org/10.1145/1816038.1815993}, DOI={10.1145/1816038.1815993}, number={3}, journal={ACM SIGARCH Computer Architecture News}, publisher={Association for Computing Machinery (ACM)}, author={Chakradhar, Srimat and Sankaradas, Murugan and Jakkula, Venkata and Cadambi, Srihari}, year={2010}, month=jun, pages={247–257} }