Alain Cappy - Neuro-inspired Information Processing

1 Cover

2 Acknowledgments

3 Introduction

4 1 Information Processing 1.1. Background 1.2. Information processing machines 1.3. Information and energy 1.4. Technologies of the future 1.5. Microprocessors and the brain 1.6. Conclusion

5 2 Information Processing in the Living 2.1. The brain at a glance 2.2. Cortex 2.3. An emblematic example: the visual cortex 2.4. Conclusion

6 3 Neurons and Synapses 3.1. Background 3.2. Cell membrane 3.3. Membrane at equilibrium 3.4. The membrane in dynamic state 3.5. Synapses 3.6. Conclusion

7 4 Artificial Neural Networks 4.1. Software neural networks 4.2. Hardware neural networks 4.3. Conclusion

8 References

9 Index

10 End User License Agreement

1 Chapter 1 Table 1.1. Scaling rules for a MOSFET transistor, and consequence of different c... Table 1.2. The different approaches for improving the energy efficiency of infor... Table 1.3. Comparison of the physical characteristics of the human brain with th...Table 1.4. Comparison of human brain/microprocessor in information processing

2 Chapter 3Table 3.1. Extra- and intracellular ionic concentrationsTable 3.2. Numerical parameters of the Hodgkin–Huxley modelTable 3.3. Parameters of the Morris–Lecar model

3 Chapter 4Table 4.1. Comparison of several properties of biological and artificial systemsTable 4.2. Width of transistors and value of capacitances of the circuit in Figu...

1 Chapter 1 Figure 1.1. Block diagram of information processing systems. For a color version... Figure 1.2. Different information encoding types Figure 1.3. The Turing machine and its practical realization (http://aturingmach... Figure 1.4. von Neumann architecture Figure 1.5. Physical and electrical diagrams of NMOS and PMOS transistors. Curre... Figure 1.6. Inverter composed of NMOS and PMOS transistors and forming a CMOS pa... Figure 1.7. Principle of downscaling. All dimensions and voltages are divided by... Figure 1.8. Evolution in the number of transistors, clock frequency and total po... Figure 1.9. Switching of a CMOS inverter. Charge and discharge currents of the c... Figure 1.10. Drain–source current of an NMOS as a function of the control voltag... Figure 1.11. Inverter loaded by two logic gates. C gis the input capacitance of ... Figure 1.12. Charge of an RC circuit by a voltage variable over time Figure 1.13. Drain–source current as a function of control voltage in logarithmi... Figure 1.14. Bloch sphere Figure 1.15. (a) Schematic diagram of the CNOT gate; (b) CNOT gate conversion ta...Figure 1.16. (a) Diagram of interconnections within an integrated circuit; (b) a...

2 Chapter 2Figure 2.1. The lobes of the left hemisphere and cross-sectional view of the bra...Figure 2.2. Organization of the cortex in six layers and cortical columns 5Figure 2.3. Detail of cortical layers, L1–L6. Columns and mini-columns. For a co...Figure 2.4. Hierarchical organization of the cortex. Each box in the diagram rep...Figure 2.5. Internal and external connections of a cortical column. The black ar...Figure 2.6. Organization of the human visual system. The eye diameter has been a...Figure 2.7. Organization of visual area V1. The color-sensitive cells are locate...

3 Chapter 3Figure 3.1. Structure of a neuron. The diameter of the soma is of the order of a...Figure 3.2. Network of synapses between pre-neurons (dotted lines) and a post-ne...Figure 3.3. Details of a synapse. The intracellular media of the two neurons are...Figure 3.4. Mechanism of ion channel opening by neurotransmittersFigure 3.5. Structure of the plasma membrane surrounding the neuron. For a color...Figure 3.6. The ATP sodium–potassium pump, or ‘NaK’ pump. For a color version of...Figure 3.7. Modeling of ionic currents: simplifying hypotheses and boundary cond...Figure 3.8. Simplified equivalent circuit of the membrane in equilibrium. For a ...Figure 3.9. Representation of resting potential V ras a function of G K/G Na. The ...Figure 3.10. Equivalent circuit of the membrane in dynamic state. For a color ve...Figure 3.11. Simplified equivalent circuit of the membrane in dynamic stateFigure 3.12. Other equivalent circuit of the membrane in dynamic state. The arro...Figure 3.13. Rule for the variation of the activation coefficient, n, leading to...Figure 3.14. Coefficients n ss, m ssand h ss, extracted from expressions [3.24]. T...Figure 3.15. Time constants τ nτ mand τ h, extracted from expressions [3.24]. The...Figure 3.16. Membrane potential, V m(t), in response to a current impulse of 3.5 ...Figure 3.17. Activation coefficients, n(t), m(t) and h(t), in response to a curr...Figure 3.18. Sodium, G Na(t), and potassium, G K(t), conductances in response to a...Figure 3.19. Sodium, potassium and leakage currents in response to a current imp...Figure 3.20. Membrane potential in response to a current pulse of 6 μA/cm 2appli...Figure 3.21. Coefficient h as a function of n in the event of a spike. The red d...Figure 3.22. Nullclines of n (blue line) and of V mIex=0 (red dotted line) and I...Figure 3.23. Nullclines of n and V m(Iex=6μA/cm 2) and Vm(t), n(t) cycles resulti...Figure 3.24. Isoclines n and V mfor different values of excitation current, Iex....Figure 3.25. Action potentials of the ML model for two excitation current values...Figure 3.26. Equivalent circuit of a non-myelinated axonFigure 3.27. Axon represented by an RC line. For a color version of the figure, ...Figure 3.28. Axon represented by an active line. Conductances G Naand G Kare gov...Figure 3.29. Generation of a spike at the soma-axon junction (red disc). Propaga...Figure 3.30. Equivalent circuit of a myelinated axon. For a color version of the...Figure 3.31. Shape of Excitation current, I ex(t), in red, and associated respons...3.32. Excitatory postsynaptic potentials in red, inhibitory in green. For a colo...Figure 3.33. Mathematical models of EPSPs given by expressions [3.50a, b, c]. Fo...Figure 3.34. EPSP of an excitatory synapse for different synaptic weight values....Figure 3.35. Principle of STDP, according to Bi and Poo. w is the synaptic weigh...

4 Chapter 4Figure 4.1. Signal-flow graph of the perceptron. Coefficients w iare the synapti...Figure 4.2. Signal-flow graph of the multilayer perceptron or feedforward networ...Figure 4.3. Principle of a convolutional networkFigure 4.4. Signal-flow graph of a hidden-layer recurrent neural network. The re...Figure 4.5. Graph of an “Elman”-type recurrent network. For a color version of t...Figure 4.6. Graph of a “Hopfield”-type recurrent network, for which the intercon...Figure 4.7. Principle of reservoir computing. The black arrows represent fixed-w...Figure 4.8. Modeling of the Pavlov experiment with three neurons (N 1“food”, N 2...Figure 4.9. Current control by ion channel in a biological membrane and by a tra...Figure 4.10. Two “current-mirror” circuits enabling the relationship between I 1...Figure 4.11. Diagram of a DPI composed of a differential pair and two current mi...Figure 4.12. Diagram of a DPI neuron. The yellow block models the leakage conduc...Figure 4.13. Circuit diagram, transfer function V out(V in) and voltage gain Gv of...Figure 4.14. Transfer function, V out(V in), of a subthreshold inverter (V dd=200 m...Figure 4.15. Reduction of the effective subthreshold slope of a transistor throu...Figure 4.16. Electrical circuit of an axon-hillock artificial neuronFigure 4.17. Signal-flow graph of the amplifier (a), ideal transfer function (b).