Crossref journal-article
Springer Science and Business Media LLC
Nature Communications (297)
Abstract

AbstractNeuromorphic computing has emerged as a promising avenue towards building the next generation of intelligent computing systems. It has been proposed that memristive devices, which exhibit history-dependent conductivity modulation, could efficiently represent the synaptic weights in artificial neural networks. However, precise modulation of the device conductance over a wide dynamic range, necessary to maintain high network accuracy, is proving to be challenging. To address this, we present a multi-memristive synaptic architecture with an efficient global counter-based arbitration scheme. We focus on phase change memory devices, develop a comprehensive model and demonstrate via simulations the effectiveness of the concept for both spiking and non-spiking neural networks. Moreover, we present experimental results involving over a million phase change memory devices for unsupervised learning of temporal correlations using a spiking neural network. The work presents a significant step towards the realization of large-scale and energy-efficient neuromorphic computing systems.

Bibliography

Boybat, I., Le Gallo, M., Nandakumar, S. R., Moraitis, T., Parnell, T., Tuma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2018). Neuromorphic computing with multi-memristive synapses. Nature Communications, 9(1).

Authors 10
  1. Irem Boybat (first)
  2. Manuel Le Gallo (additional)
  3. S. R. Nandakumar (additional)
  4. Timoleon Moraitis (additional)
  5. Thomas Parnell (additional)
  6. Tomas Tuma (additional)
  7. Bipin Rajendran (additional)
  8. Yusuf Leblebici (additional)
  9. Abu Sebastian (additional)
  10. Evangelos Eleftheriou (additional)
References 64 Referenced 691
  1. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). (10.1038/nature14539) / Nature by Y LeCun (2015)
  2. Schemmel, J. et al. A wafer-scale neuromorphic hardware system for large-scale neural modeling. In Proc. IEEE International Symposium on Circuits and Systems (ISCAS), 1947–1950 (IEEE, Paris, France, 2010). (10.1109/ISCAS.2010.5536970)
  3. Painkras, E. et al. SpiNNaker: a 1-W 18-core system-on-chip for massively-parallel neural network simulation. IEEE J. Solid-State Circuits 48, 1943–1953 (2013). (10.1109/JSSC.2013.2259038) / IEEE J. Solid-State Circuits by E Painkras (2013)
  4. Merolla, P. A. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668–673 (2014). (10.1126/science.1254642) / Science by PA Merolla (2014)
  5. Qiao, N. et al. A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses. Front. Neurosci. 9, 141 (2015). (10.3389/fnins.2015.00141) / Front. Neurosci. by N Qiao (2015)
  6. Beck, A., Bednorz, J., Gerber, C., Rossel, C. & Widmer, D. Reproducible switching effect in thin oxide films for memory applications. Appl. Phys. Lett. 77, 139–141 (2000). (10.1063/1.126902) / Appl. Phys. Lett. by A Beck (2000)
  7. Strukov, D. B., Snider, G. S., Stewart, D. R. & Williams, R. S. The missing memristor found. Nature 453, 80–83 (2008). (10.1038/nature06932) / Nature by DB Strukov (2008)
  8. Chua, L. Resistance switching memories are memristors. Appl. Phys. A 102, 765–783 (2011). (10.1007/s00339-011-6264-9) / Appl. Phys. A by L Chua (2011)
  9. Wong, H. S. P. & Salahuddin, S. Memory leads the way to better computing. Nat. Nanotech. 10, 191–194 (2015). (10.1038/nnano.2015.29) / Nat. Nanotech. by HSP Wong (2015)
  10. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations, (eds Rumelhart, D. E. & McClelland, J. L.) 318–362 (MIT Press, Cambridge, MA, 1986).
  11. Markram, H., Lübke, J., Frotscher, M. & Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215 (1997). (10.1126/science.275.5297.213) / Science by H Markram (1997)
  12. Anwani, N. & Rajendran, B. NormAD - normalized approximate descent based supervised learning rule for spiking neurons. In Proc. International Joint Conference on Neural Networks (IJCNN), 1–8 (IEEE, Killarney, Ireland, 2015). (10.1109/IJCNN.2015.7280618)
  13. Saighi, S. et al. Plasticity in memristive devices for spiking neural networks. Front. Neurosci. 9, 51 (2015). (10.3389/fnins.2015.00051) / Front. Neurosci. by S Saighi (2015)
  14. Burr, G. W. et al. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Adv. Phys. X 2, 89–124 (2016). / Adv. Phys. X by GW Burr (2016)
  15. Rajendran, B. & Alibart, F. Neuromorphic computing based on emerging memory technologies. IEEE J. Emerg. Sel. Top. Circuits Syst. 6, 198–211 (2016). (10.1109/JETCAS.2016.2533298) / IEEE J. Emerg. Sel. Top. Circuits Syst. by B Rajendran (2016)
  16. Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595 (2011). (10.1038/nmat3054) / Nat. Mater. by T Ohno (2011)
  17. Yu, S., Wu, Y., Jeyasingh, R., Kuzum, D. & Wong, H. S. P. An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE Trans. Electron Dev. 58, 2729–2737 (2011). (10.1109/TED.2011.2147791) / IEEE Trans. Electron Dev. by S Yu (2011)
  18. Ambrogio, S. et al. Neuromorphic learning and recognition with one-transistor-one-resistor synapses and bistable metal oxide RRAM. IEEE Trans. Electron Dev. 63, 1508–1515 (2016). (10.1109/TED.2016.2526647) / IEEE Trans. Electron Dev. by S Ambrogio (2016)
  19. Covi, E. et al. Analog memristive synapse in spiking networks implementing unsupervised learning. Front. Neurosci. 10, 482 (2016). (10.3389/fnins.2016.00482) / Front. Neurosci. by E Covi (2016)
  20. van de Burgt, Y. et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat. Mater. 16, 414–418 (2017). (10.1038/nmat4856) / Nat. Mater. by Y van de Burgt (2017)
  21. Wang, Z. et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat. Mater. 16, 101–108 (2017). (10.1038/nmat4756) / Nat. Mater. by Z Wang (2017)
  22. Boyn, S. et al. Learning through ferroelectric domain dynamics in solid-state synapses. Nat. Commun. 8, 14736 (2017). (10.1038/ncomms14736) / Nat. Commun. by S Boyn (2017)
  23. Wu, W., Zhu, X., Kang, S., Yuen, K. & Gilmore, R. Probabilistically programmed STT-MRAM. IEEE J. Emerg. Sel. Top. Circuits Syst. 2, 42–51 (2012). (10.1109/JETCAS.2012.2187401) / IEEE J. Emerg. Sel. Top. Circuits Syst. by W Wu (2012)
  24. Vincent, A. F. et al. Spin-transfer torque magnetic memory as a stochastic memristive synapse for neuromorphic systems. IEEE Trans. Biomed. Circ. Syst. 9, 166–174 (2015). (10.1109/TBCAS.2015.2414423) / IEEE Trans. Biomed. Circ. Syst. by AF Vincent (2015)
  25. Kuzum, D., Jeyasingh, R. G. D., Lee, B. & Wong, H. S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179–2186 (2012). (10.1021/nl201040y) / Nano Lett. by D Kuzum (2012)
  26. Ambrogio, S. et al. Unsupervised learning by spike timing dependent plasticity in phase change memory (PCM) synapses. Front. Neurosci. 10, 56 (2016). (10.3389/fnins.2016.00056) / Front. Neurosci. by S Ambrogio (2016)
  27. Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013). (10.1038/ncomms3072) / Nat. Commun. by F Alibart (2013)
  28. Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G. & Prodromakis, T. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24, 384010 (2013). (10.1088/0957-4484/24/38/384010) / Nanotechnology by G Indiveri (2013)
  29. Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015). (10.1038/nature14441) / Nature by M Prezioso (2015)
  30. Kim, S. et al. NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning. In Proc. IEEE International Electron Devices Meeting (IEDM), 17-1 (IEEE, Washington, DC, USA, 2015). (10.1109/IEDM.2015.7409716)
  31. Mostafa, H. et al. Implementation of a spike-based perceptron learning rule using TiO2−x memristors. Front. Neurosci. 9, 357 (2015). (10.3389/fnins.2015.00357) / Front. Neurosci. by H Mostafa (2015)
  32. Tuma, T., Le Gallo, M., Sebastian, A. & Eleftheriou, E. Detecting correlations using phase-change neurons and synapses. IEEE Electron Dev. Lett. 37, 1238–1241 (2016). (10.1109/LED.2016.2591181) / IEEE Electron Dev. Lett. by T Tuma (2016)
  33. Wozniak, S., Tuma, T., Pantazi, A. & Eleftheriou, E. Learning spatio-temporal patterns in the presence of input noise using phase-change memristors. In Proc. IEEE International Symposium on Circuits and Systems (ISCAS), 365–368 (IEEE, Montreal, QC, Canada, 2016). (10.1109/ISCAS.2016.7527246)
  34. Burr, G. W. et al. Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element. IEEE Trans. Electron Dev. 62, 3498–3507 (2015). (10.1109/TED.2015.2439635) / IEEE Trans. Electron Dev. by GW Burr (2015)
  35. Koelmans, W. W. et al. Projected phase-change memory devices. Nat. Commun. 6, 8181 (2015). (10.1038/ncomms9181) / Nat. Commun. by WW Koelmans (2015)
  36. Fuller, E. J. et al. Li-ion synaptic transistor for low power analog computing. Adv. Mater. 29, 1604310 (2017). (10.1002/adma.201604310) / Adv. Mater by EJ Fuller (2017)
  37. Suri, M. et al. Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction. In Proc. IEEE International Electron Devices Meeting (IEDM), 4.4.1–4.4.4 (IEEE, Washington, DC, USA, 2011). (10.1109/IEDM.2011.6131488)
  38. Bill, J. & Legenstein, R. A compound memristive synapse model for statistical learning through STDP in spiking neural networks. Front. Neurosci. 8, 412 (2014). / Front. Neurosci. by J Bill (2014)
  39. Garbin, D. et al. HfO2-based OxRAM devices as synapses for convolutional neural networks. IEEE Trans. Electron Dev. 62, 2494–2501 (2015). (10.1109/TED.2015.2440102) / IEEE Trans. Electron Dev. by D Garbin (2015)
  40. Burr, G. W. et al. Recent progress in phase-change memory technology. IEEE J. Emerg. Sel. Top. Circuits Syst. 6, 146–162 (2016). (10.1109/JETCAS.2016.2547718) / IEEE J. Emerg. Sel. Top. Circuits Syst. by GW Burr (2016)
  41. Sebastian, A., Le Gallo, M. & Krebs, D. Crystal growth within a phase change memory cell. Nat. Commun. 5, 4314 (2014). (10.1038/ncomms5314) / Nat. Commun. by A Sebastian (2014)
  42. Close, G. F. et al. Device, circuit and system-level analysis of noise in multi-bit phase-change memory. In Proc. IEEE Int. Electron Devices Meeting (IEDM), 29.5.1–29.5.4 (IEEE, San Francisco, CA, USA, 2010). (10.1109/IEDM.2010.5703445)
  43. Boybat, I. et al. Stochastic weight updates in phase-change memory-based synapses and their influence on artificial neural networks. In Proc. Ph.D. Research in Microelectronics and Electronics (PRIME), 13–16 (IEEE, Giardini Naxos, Italy, 2017). (10.1109/PRIME.2017.7974095)
  44. Gokmen, T. & Vlasov, Y. Acceleration of deep neural network training with resistive cross-point devices: design considerations. Front. Neurosci. 10, 333 (2016). (10.3389/fnins.2016.00333) / Front. Neurosci. by T Gokmen (2016)
  45. Nandakumar, S. R. et al. Supervised learning in spiking neural networks with MLC PCM synapses. In Proc. Device Research Conference (DRC), 1–2 (IEEE, South Bend, IN, USA, 2017). (10.1109/DRC.2017.7999481)
  46. Le Gallo, M., Tuma, T., Zipoli, F., Sebastian, A. & Eleftheriou, E. Inherent stochasticity in phase-change memory devices. In Proc. European Solid-State Device Research Conference (ESSDERC), 373–376 (IEEE, Lausanne, Switzerland, 2016). (10.1109/ESSDERC.2016.7599664)
  47. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998). (10.1109/5.726791) / Proc. IEEE by Y LeCun (1998)
  48. Querlioz, D., Bichler, O. & Gamrat, C. Simulation of a memristor-based spiking neural network immune to device variations. In Proc. International Joint Conference on Neural Networks (IJCNN), 1775–1781 (IEEE, San Jose, CA, USA, 2011). (10.1109/IJCNN.2011.6033439)
  49. Gütig, R., Aharonov, R., Rotter, S. & Sompolinsky, H. Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J. Neurosci. 23, 3697–3714 (2003). (10.1523/JNEUROSCI.23-09-03697.2003) / J. Neurosci. by R Gütig (2003)
  50. Tuma, T., Pantazi, A., Le Gallo, M., Sebastian, A. & Eleftheriou, E. Stochastic phase-change neurons. Nat. Nanotech. 11, 693–699 (2016). (10.1038/nnano.2016.70) / Nat. Nanotech. by T Tuma (2016)
  51. Sebastian, A. et al. Temporal correlation detection using computational phase-change memory. Nat. Commun. 8, 1115 (2017). (10.1038/s41467-017-01481-9) / Nat. Commun. by A Sebastian (2017)
  52. Edwards, F. LTP is a long term problem. Nature 350, 271–272 (1991). (10.1038/350271a0) / Nature by F Edwards (1991)
  53. Bolshakov, V. Y., Golan, H., Kandel, E. R. & Siegelbaum, S. A. Recruitment of new sites of synaptic transmission during the cAMP-dependent late phase of LTP at CA3-CA1 synapses in the hippocampus. Neuron 19, 635–651 (1997). (10.1016/S0896-6273(00)80377-3) / Neuron by VY Bolshakov (1997)
  54. Malenka, R. C. & Nicoll, R. A. Long-term potentiation-a decade of progress? Science 285, 1870–1874 (1999). (10.1126/science.285.5435.1870) / Science by RC Malenka (1999)
  55. Malenka, R. C. & Bear, M. F. LTP and LTD: an embarrassment of riches. Neuron 44, 5–21 (2004). (10.1016/j.neuron.2004.09.012) / Neuron by RC Malenka (2004)
  56. Benke, T. A., Luthi, A., Isaac, J. T. R. & Collingridge, G. L. Modulation of AMPA receptor unitary conductance by synaptic activity. Nature 393, 793–797 (1998). (10.1038/31709) / Nature by TA Benke (1998)
  57. Le Gallo, M., Sebastian, A., Cherubini, G., Giefers, H. & Eleftheriou, E. Compressed sensing recovery using computational memory. In Proc. IEEE International Electron Devices Meeting (IEDM), 28.3.1–28.3.4 (IEEE, San Francisco, CA, USA, 2017). (10.1109/IEDM.2017.8268469)
  58. Li, C. et al. Analogue signal and image processing with large memristor crossbars. Nat. Electron. 1, 52–59 (2018). (10.1038/s41928-017-0002-z) / Nat. Electron. by C Li (2018)
  59. Le Gallo, M. et al. Mixed-precision in-memory computing. Nat. Electron. 1, 246–253 (2018). (10.1038/s41928-018-0054-8) / Nat. Electron. by M Le Gallo (2018)
  60. Eryilmaz, S. B. et al. Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array. Front. Neurosci. 8, 205 (2014). (10.3389/fnins.2014.00205) / Front. Neurosci. by SB Eryilmaz (2014)
  61. Sidler, S. et al. Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Impact of conductance response. In Proc. European Solid-State Device Research Conference (ESSDERC), 440–443 (IEEE, Lausanne, Switzerland, 2016). (10.1109/ESSDERC.2016.7599680)
  62. Fumarola, A. et al. Accelerating machine learning with non-volatile memory: exploring device and circuit tradeoffs. In Proc. IEEE International Conference on Rebooting Computing (ICRC), 1–8 (IEEE, San Diego, CA, USA, 2016). (10.1109/ICRC.2016.7738684)
  63. Sebastian, A., Krebs, D., Le Gallo, M., Pozidis, H. & Eleftheriou, E. A collective relaxation model for resistance drift in phase change memory cells. In Proc. IEEE International Reliability Physics Symposium (IRPS), MY-5 (IEEE, Monterey, CA, USA, 2015). (10.1109/IRPS.2015.7112808)
  64. Song, S., Miller, K. D. & F., A. L. Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919–926 (2000). (10.1038/78829) / Nat. Neurosci. by S Song (2000)
Dates
Type When
Created 7 years, 2 months ago (June 22, 2018, 6:15 a.m.)
Deposited 1 month, 3 weeks ago (July 5, 2025, 3:52 a.m.)
Indexed 2 hours, 52 minutes ago (Aug. 28, 2025, 8:08 a.m.)
Issued 7 years, 2 months ago (June 28, 2018)
Published 7 years, 2 months ago (June 28, 2018)
Published Online 7 years, 2 months ago (June 28, 2018)
Funders 0

None

@article{Boybat_2018, title={Neuromorphic computing with multi-memristive synapses}, volume={9}, ISSN={2041-1723}, url={http://dx.doi.org/10.1038/s41467-018-04933-y}, DOI={10.1038/s41467-018-04933-y}, number={1}, journal={Nature Communications}, publisher={Springer Science and Business Media LLC}, author={Boybat, Irem and Le Gallo, Manuel and Nandakumar, S. R. and Moraitis, Timoleon and Parnell, Thomas and Tuma, Tomas and Rajendran, Bipin and Leblebici, Yusuf and Sebastian, Abu and Eleftheriou, Evangelos}, year={2018}, month=jun }