Citation: Diego Fasoli, Stefano Panzeri. Mathematical studies of the dynamics of finite-size binary neural networks: A review of recent progress[J]. Mathematical Biosciences and Engineering, 2019, 16(6): 8025-8059. doi: 10.3934/mbe.2019404
[1] | W. W. Lytton, From computer to brain:Foundations of computational neuroscience, SpringerVerlag New York, 2002, URL https://doi.org/10.1007/b98859. |
[2] | B. Ermentrout, Neural networks as spatio-temporal pattern-forming systems, Rep. Prog. Phys., 61 (1998), 353-430, URL http://doi.org/10.1088/0034-4885/61/4/002. |
[3] | E. M. Izhikevich, Dynamical systems in neuroscience, MIT Press, 2007, URL https://mitpress.mit.edu/books/dynamical-systems-neuroscience. |
[4] | P. Ashwin, S. Coombes and R. Nicks, Mathematical frameworks for oscillatory network dynamics in neuroscience, J. Math. Neurosci., 6 (2016), 2, URL https://doi.org/10.1186/s13408-015-0033-6. |
[5] | S. I. Amari, Learning patterns and pattern sequences by self-organizing nets of threshold elements, IEEE Trans. Comput., C-21 (1972), 1197-1206, URL https://doi.org/10.1109/T-C.1972.223477. |
[6] | W. A. Little, The existence of persistent states in the brain, Math. Biosci., 19 (1974), 101-120, URL https://doi.org/10.1016/0025-5564(74)90031-5. |
[7] | J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. U.S.A., 79 (1982), 2554-2558, URL http://dx.doi.org/10.1073/pnas.79.8.2554. |
[8] | A. C. C. Coolen, Chapter 14, Statistical mechanics of recurrent neural networks I:Statics, vol. 4 of Handbook of Biological Physics, North-Holland, 2001, 553-618, URL https://doi.org/10.1016/S1383-8121(01)80017-8. |
[9] | A. C. C. Coolen, Chapter 15, Statistical mechanics of recurrent neural networks Ⅱ:Dynamics, vol. 4 of Handbook of Biological Physics, North-Holland, 2001, 619-684, URL https://doi.org/10.1016/S1383-8121(01)80018-X. |
[10] | D. Kernell, The adaptation and the relation between discharge frequency and current strength of cat lumbosacral motoneurones stimulated by long-lasting injected currents, Acta Physiol. Scand., 65 (1965), 65-73, URL https://doi.org/10.1111/j.1748-1716.1965.tb04250.x. |
[11] | D. Kernell, High-frequency repetitive firing of cat lumbosacral motoneurones stimulated by longlasting injected currents, Acta Physiol. Scand., 65 (1965), 74-86, URL https://doi.org/10.1111/j.1748-1716.1965.tb04251.x. |
[12] | P. Peretto, Collective properties of neural networks:A statistical physics approach, Biol. Cybern., 50 (1984), 51-62, URL https://doi.org/10.1007/BF00317939. |
[13] | I. Ginzburg and H. Sompolinsky, Theory of correlations in stochastic neural networks, Phys. Rev. E, 50 (1994), 3171-3191, URL https://doi.org/10.1103/PhysRevE.50.3171. |
[14] | D. Fasoli, A. Cattani and S. Panzeri, Pattern storage, bifurcations and groupwise correlation structure of an exactly solvable asymmetric neural network model, Neural Comput., 30 (2018), 1258-1295, URL https://doi.org/10.1162/NECO_a_01069. |
[15] | D. Fasoli and S. Panzeri, Optimized brute-force algorithms for the bifurcation analysis of a binary neural network model, Phys. Rev. E, 99 (2019), 012316, URL https://doi.org/10.1103/PhysRevE.99.012316. |
[16] | D. Fasoli and S. Panzeri, Stationary-state statistics of a binary neural network model with quenched disorder, Entropy, 21 (2019), 630, URL https://doi.org/10.3390/e21070630. |
[17] | D. J. Amit, Modeling brain function:The world of attractor neural networks, Cambridge University Press, 1989, URL https://doi.org/10.1017/CBO9780511623257. |
[18] | S. I. Amari, Dynamics of pattern formation in lateral-inhibition type neural fields, Biol. Cybern., 27 (1977), 77-87, URL https://doi.org/10.1007/BF00337259. |
[19] | C. van Vreeswijk and H. Sompolinsky, Chaos in neuronal networks with balanced excitatory and inhibitory activity, Science, 274 (1996), 1724-1726, URL https://doi.org/10.1126/science.274.5293.1724. |
[20] | S. I. Amari, Homogeneous nets of neuron-like elements, Biol. Cybern., 17 (1975), 211-220, URL https://doi.org/10.1007/BF00339367. |
[21] | D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett., 35 (1976), 1792-1796, URL https://doi.org/10.1103/PhysRevLett.35.1792. |
[22] | S. Kirkpatrick and D. Sherrington, Infinite-ranged models of spin-glasses, Phys. Rev. B, 17 (1978), 4384-4403, URL https://doi.org/10.1103/PhysRevB.17.4384. |
[23] | M. Mézard, N. Sourlas, G. Toulouse, et al., Replica symmetry breaking and the nature of the spin glass phase, J. Phys., 45 (1984), 843-854, URL https://doi.org/10.1051/jphys:01984004505084300. doi: 10.1051/jphys:01984004505084300 |
[24] | M. Mézard, G. Parisi and M. Virasoro, Spin glass theory and beyond:An introduction to the replica method and its applications, World Scientific Singapore, 1986, URL http://www.worldscientific.com/worldscibooks/10.1142/0271. |
[25] | R. J. Glauber, Time dependent statistics of the Ising model, J. Math. Phys., 4 (1963), 294-307, URL https://doi.org/10.1063/1.1703954. |
[26] | A. C. C. Coolen and D. Sherrington, Dynamics of fully connected attractor neural networks near saturation, Phys. Rev. Lett., 71 (1993), 3886-3889, URL https://doi.org/10.1103/PhysRevLett.71.3886. |
[27] | R. W. Williams and K. Herrup, The control of neuron number, Ann. Rev. Neurosci., 11 (1988), 423-453, URL https://doi.org/10.1146/annurev.ne.11.030188.002231. |
[28] | V. B. Mountcastle, The columnar organization of the neocortex, Brain, 120 (1997), 701-722, URL https://doi.org/10.1093/brain/120.4.701. |
[29] | M. Helmstaedter, C. P. de Kock, D. Feldmeyer, et al., Reconstruction of an average cortical column in silico, Brain Res. Rev., 55 (2007), 193-203, URL https://doi.org/10.1016/j.brainresrev.2007.07.011. |
[30] | H. S. Meyer, V. C. Wimmer, M. Oberlaender, et al., Number and laminar distribution of neurons in a thalamocortical projection column of rat vibrissal cortex, Cereb. Cortex, 20 (2010), 2277-2286, URL https://doi.org/10.1093/cercor/bhq067. |
[31] | M. De Luca, C. F. Beckmann, N. De Stefano, et al., fMRI resting state networks define distinct modes of long-distance interactions in the human brain, NeuroImage, 29 (2006), 1359-1367, URL https://doi.org/10.1016/j.neuroimage.2005.08.035. |
[32] | D. Mantini, M. G. Perrucci, C. Del Gratta, et al., Electrophysiological signatures of resting state networks in the human brain, Proc. Natl. Acad. Sci. USA, 104 (2007), 13170-13175, URL https://doi.org/10.1073/pnas.0700668104. |
[33] | R. D. Beer, On the dynamics of small continuous-time recurrent neural networks, Adapt. Behav., 3 (1995), 469-509, URL https://doi.org/10.1177/105971239500300405. |
[34] | F. Pasemann, Complex dynamics and the structure of small neural networks, Network-Comp. Neural, 13 (2002), 195-216, URL http://doi.org/10.1080/net.13.2.195.216. |
[35] | D. Fasoli, A. Cattani and S. Panzeri, The complexity of dynamics in small neural circuits, PLoS Comput. Biol., 12 (2016), e1004992, URL https://doi.org/10.1371/journal.pcbi.1004992. |
[36] | C. van Vreeswijk and H. Sompolinsky, Chaotic balanced state in a model of cortical circuits, Neural Comput., 10 (1998), 1321-1371, URL https://doi.org/10.1162/089976698300017214. |
[37] | B. Cessac, Increase in complexity in random neural networks, J. Phys. I France, 5 (1995), 409-432, URL http://doi.org/10.1051/jp1:1995135. |
[38] | A. Renart, J. De La Rocha, P. Bartho, et al., The asynchronous state in cortical circuits, Science, 327 (2010), 587-590, URL https://doi.org/10.1126/science.1179850. |
[39] | V. Pernice, B. Staude, S. Cardanobile, et al., Recurrent interactions in spiking networks with arbitrary topology, Phys. Rev. E, 85 (2012), 031916, URL https://doi.org/10.1103/PhysRevE.85.031916. |
[40] | J. Trousdale, Y. Hu, E. Shea-Brown, et al., Impact of network structure and cellular response on spike time correlations, PLoS Comput. Biol., 8 (2012), e1002408, URL https://doi.org/10.1371/journal.pcbi.1002408. |
[41] | T. Tetzlaff, M. Helias, G. T. Einevoll, et al., Decorrelation of neural-network activity by inhibitory feedback, PLoS Comput. Biol., 8 (2012), e1002596, URL https://doi.org/10.1371/journal.pcbi.1002596. |
[42] | M. Helias, T. Tetzlaff and M. Diesmann, Echoes in correlated neural systems, New J. Phys., 15 (2013), 023002, URL https://doi.org/10.1088/1367-2630/15/2/023002. |
[43] | T. Schwalger, M. Deger and W. Gerstner, Towards a theory of cortical columns:From spiking neurons to interacting neural populations of finite size, PLoS Comput. Biol., 13 (2017), e1005507, URL https://doi.org/10.1371/journal.pcbi.1005507. |
[44] | D. Dahmen, H. Bos and M. Helias, Correlated fluctuations in strongly coupled binary networks beyond equilibrium, Phys. Rev. X, 6 (2016), 031024, URL https://doi.org/10.1103/PhysRevX.6.031024. |
[45] | Y. A. Kuznetsov, Elements of applied bifurcation theory, vol. 112, Springer-Verlag New York, 1998, URL https://doi.org/10.1007/978-1-4757-3978-7. |
[46] | J. R. L. De Almeida and D. J. Thouless, Stability of the Sherrington-Kirkpatrick solution of a spin glass model, J. Phys. A:Math. Gen., 11 (1978), 983-990, URL https://doi.org/10.1088/0305-4470/11/5/028. |
[47] | R. I. Leine, D. H. Van Campen and B. L. Van De Vrande, Bifurcations in nonlinear discontinuous systems, Nonlinear Dyn., 23 (2000), 105-164, URL https://doi.org/10.1023/A:1008384928636. |
[48] | J. Awrejcewicz and C. H. Lamarque, Bifurcation and chaos in nonsmooth mechanical systems, World Scientific, 2003, URL http://doi.org/10.1142/5342. |
[49] | R. I. Leine and D. H. Van Campen, Bifurcation phenomena in non-smooth dynamical systems, Eur. J. Mech. A-Solid, 25 (2006), 595-616, URL https://doi.org/10.1016/j.euromechsol.2006.04.004. |
[50] | O. Makarenkov and J. S. W. Lamb, Dynamics and bifurcations of nonsmooth systems:A survey, Physica D, 241 (2012), 1826-1844, URL https://doi.org/10.1016/j.physd.2012.08.002. |
[51] | J. Harris and B. Ermentrout, Bifurcations in the Wilson-Cowan equations with nonsmooth firing rate, SIAM J. Appl. Dyn. Syst., 14 (2015), 43-72, URL https://doi.org/10.1137/140977953. |
[52] | S. Parui and S. Banerjee, Border collision bifurcations at the change of state-space dimension, Chaos, 12 (2002), 1054-1069, URL https://doi.org/10.1063/1.1521390. |
[53] | B. Cessac and T. Viéville, On dynamics of integrate-and-fire neural networks with conductance based synapses, Front. Comput. Neurosci., 2 (2008), 2, URL https://doi.org/10.3389/neuro.10.002.2008. |
[54] | V. Avrutin, M. Schanz and S. Banerjee, Multi-parametric bifurcations in a piecewise-linear discontinuous map, Nonlinearity, 19 (2006), 1875-1906, URL https://doi.org/10.1088/0951-7715/19/8/007. |
[55] | R. Kötter, Neuroscience databases:A practical guide, Springer US, 2003, URL http://doi. org/10.1007/978-1-4615-1079-6. |
[56] | T. C. Potjans and M. Diesmann, The cell-type specific cortical microcircuit:Relating structure and activity in a full-scale spiking network model, Cereb. Cortex, 24 (2014), 785-806, URL https://doi.org/10.1093/cercor/bhs358. |
[57] | C. Boucsein, M. Nawrot, P. Schnepel, et al., Beyond the cortical column:Abundance and physiology of horizontal connections imply a strong role for inputs from the surround, Front. Neurosci., 5 (2011), 32, URL https://doi.org/10.3389/fnins.2011.00032. |
[58] | S. E. Palmer, O. Marre, M. J. Berry, et al., Predictive information in a sensory population, Proc. Natl. Acad. Sci. USA, 112 (2015), 6908-6913, URL https://doi.org/10.1073/pnas.1506855112. |
[59] | V. Rostami, P. Porta Mana, S. Grün, et al., Bistability, non-ergodicity, and inhibition in pairwise maximum-entropy models, PLoS Comput. Biol., 13 (2017), e1005762, URL https://doi.org/10.1371/journal.pcbi.1005762. |
[60] | S. I. Amari, Characteristics of random nets of analog neuron-like elements, IEEE Trans. Syst. Man Cybern., SMC-2 (1972), 643-657, URL https://doi.org/10.1109/TSMC.1972.4309193. |
[61] | G. Hermann and J. Touboul, Heterogeneous connections induce oscillations in large-scale networks, Phys. Rev. Lett., 109 (2012), 018702, URL https://doi.org/10.1103/PhysRevLett.109.018702. |
[62] | R. J. Vaughan and W. N. Venables, Permanent expressions for order statistic densities, J. R. Stat. Soc. Ser. B, 34 (1972), 308-310, URL http://www.jstor.org/stable/2985190. |
[63] | R. B. Bapat and M. I. Beg, Order statistics for nonidentically distributed variables and permanents, Sankhyā Ser. A, 51 (1989), 79-93, URL http://www.jstor.org/stable/25050725. |
[64] | R. B. Bapat, Permanents in probability and statistics, Linear Algebra Appl., 127 (1990), 3-25, URL https://doi.org/10.1016/0024-3795(90)90332-7. |
[65] | S. Hande, A note on order statistics for nondentically distributed variables, Sankhyā Ser. A, 56 (1994), 365-368, URL http://www.jstor.org/stable/25050995. |
[66] | C. H. Papadimitriou and K. Steiglitz, Combinatorial optimization:Algorithms and complexity, Prentice Hall, 1982. |
[67] | O. Faugeras, J. Touboul and B. Cessac, A constructive mean-field analysis of multi-population neural networks with random synaptic weights and stochastic inputs, Front. Comput. Neurosci., 3 (2009), 1, URL https://doi.org/10.3389/neuro.10.001.2009. |
[68] | T. Cabana and J. Touboul, Large deviations, dynamics and phase transitions in large stochastic and disordered neural networks, J. Stat. Phys., 153 (2013), 211-269, URL https://doi.org/10.1007/s10955-013-0818-5. |
[69] | K. Balasubramanian, Combinatorics and diagonals of matrices, PhD thesis, Indian Statistical Institute, 1980, URL http://library.isical.ac.in:8080/jspui/handle/10263/3603. |
[70] | E. Bax and J. Franklin, A finite-difference sieve to compute the permanent, Technical Report CalTech-CS-TR-96-04, 1996. |
[71] | E. Bax, Finite-difference algorithms for counting problems, PhD thesis, California Institute of Technology, 1998, URL https://thesis.library.caltech.edu/223/. |
[72] | D. G. Glynn, The permanent of a square matrix, Eur. J. Combin., 31 (2010), 1887-1891, URL https://doi.org/10.1016/j.ejc.2010.01.010. |
[73] | H. Sompolinsky and I. Kanter, Temporal association in asymmetric neural networks, Phys. Rev. Lett., 57 (1986), 2861-2864, URL http://dx.doi.org/10.1103/PhysRevLett.57.2861. |
[74] | D. Kleinfeld, Sequential state generation by model neural networks, Proc. Natl. Acad. Sci. U.S.A., 83 (1986), 9469-9473, URL http://dx.doi.org/10.1073/pnas.83.24.9469. |
[75] | S. Dehaene, J. P. Changeux and J. P. Nadal, Neural networks that learn temporal sequences by selection, Proc. Natl. Acad. Sci. U.S.A., 84 (1987), 2727-2731, URL https://doi.org/10.1073/pnas.84.9.2727. |
[76] | J. Buhmann and K. Schulten, Noise-driven temporal association in neural networks, Europhys Lett., 4 (1987), 1205, URL http://dx.doi.org/10.1209/0295-5075/4/10/021. |
[77] | J. DeFelipe, P. Marco, I. Busturia, et al., Estimation of the number of synapses in the cerebral cortex:Methodological considerations, Cereb. Cortex, 9 (1999), 722, URL http://dx.doi.org/10.1093/cercor/9.7.722. |
[78] | L. Personnaz, I. Guyon and G. Dreyfus, Collective computational properties of neural networks:New learning mechanisms, Phys. Rev. A, 34 (1986), 4217-4228, URL http://dx.doi.org/10.1103/PhysRevA.34.4217. |
[79] | M. Helias, T. Tetzlaff and M. Diesmann, The correlation structure of local neuronal networks intrinsically results from recurrent dynamics, PLoS Comput. Biol., 10 (2014), e1003428, URL https://doi.org/10.1371/journal.pcbi.1003428. |
[80] | G. Dumont, A. Payeur and A. Longtin, A stochastic-field description of finite-size spiking neural networks, PLoS Comput. Biol., 13 (2017), e1005691, URL https://doi.org/10.1371/journal.pcbi.1005691. |
[81] | M. A. Buice and C. C. Chow, Generalized activity equations for spiking neural network dynamics, Front. Comput. Neurosci., 7 (2013), 162, URL https://doi.org/10.3389/fncom.2013.00162. |
[82] | V. Klinshov and I. Franović, Mean-field dynamics of a random neural network with noise, Phys. Rev. E, 92 (2015), 062813, URL https://doi.org/10.1103/PhysRevE.92.062813. |
[83] | S. Hwang, V. Folli, E. Lanza, et al., On the number of limit cycles in asymmetric neural networks, J. Stat. Mech.:Theory Exp., 2019 (2019), 053402, URL https://doi.org/10.1088/1742-5468/ab11e3. |
[84] | O. Faugeras and J. MacLaurin, Asymptotic description of neural networks with correlated synaptic weights, Entropy, 17 (2015), 4701-4743, URL https://doi.org/10.3390/e17074701. |
[85] | D. Martí, N. Brunel and S. Ostojic, Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks, Phys. Rev. E, 97 (2018), 062314, URL https://doi.org/10.1103/PhysRevE.97.062314. |
[86] | S. Panzeri, S. R. Schultz, A. Treves, et al., Correlations and the encoding of information in the nervous system, Proc. Biol. Sci., 266 (1999), 1001-1012, URL https://doi.org/10.1098/rspb.1999.0736. doi: 10.1098/rspb.1999.0736 |
[87] | S. Panzeri, J. H. Macke, J. Gross, et al., Neural population coding:Combining insights from microscopic and mass signals, Trends Cogn. Sci., 19 (2015), 162-172, URL https://doi.org/10.1016/j.tics.2015.01.002. |
[88] | Y. Zuo, H. Safaai, G. Notaro, et al., Complementary contributions of spike timing and spike rate to perceptual decisions in rat S1 and S2 cortex, Curr. Biol., 25 (2015), 357-363, URL https://doi.org/10.1016/j.cub.2014.11.065. |
[89] | C. A. Runyan, E. Piasini, S. Panzeri, et al., Distinct timescales of population coding across cortex, Nature, 548 (2017), 92-96, URL https://doi.org/10.1038/nature23020. |
[90] | E. Zohary, M. N. Shadlen and W. T. Newsome, Correlated neuronal discharge rate and its implications for psychophysical performance, Nature, 370 (1994), 140-143, URL https://doi.org/10.1038/370140a0. |
[91] | W. Singer, Neuronal synchrony:A versatile code for the definition of relations?, Neuron, 24 (1999), 49-65, URL https://doi.org/10.1016/S0896-6273(00)80821-1. |
[92] | A. S. Ecker, P. Berens, G. A. Keliris, et al., Decorrelated neuronal firing in cortical microcircuits, Science, 327 (2010), 584-587, URL https://doi.org/10.1126/science.1179867. |
[93] | M. R. Cohen and A. Kohn, Measuring and interpreting neuronal correlations, Nat. Neurosci., 14 (2011), 811-819, URL https://doi.org/10.1038/nn.2842. |
[94] | R. Brette, Computing with neural synchrony, PLoS Comput. Biol., 8 (2012), e1002561, URL https://doi.org/10.1371/journal.pcbi.1002561. |
[95] | R. Moreno-Bote, J. Beck, I. Kanitscheider, et al., Information-limiting correlations, Nat. Neurosci., 17 (2014), 1410-1417, URL https://doi.org/10.1038/nn.3807. |
[96] | W.-C. A. Lee, V. Bonin, M. Reed, et al., Anatomy and function of an excitatory network in the visual cortex, Nature, 532 (2016), 370-374, URL https://doi.org/10.1038/nature17192. |
[97] | J. L. R. Rubenstein and M. M. Merzenich, Model of autism:Increased ratio of excitation/inhibition in key neural systems, Genes Brain Behav., 2 (2003), 255-267, URL https://doi.org/10.1034/j.1601-183X.2003.00037.x. |
[98] | H. Y. Zoghbi, Postnatal neurodevelopmental disorders:Meeting at the synapse?, Science, 302 (2003), 826-830, URL https://doi.org/10.1126/science.1089071. |
[99] | M. Sahin and M. Sur, Genes, circuits, and precision therapies for autism and related neurodevelopmental disorders, Science, 350 (2015), aab3897, URL https://doi.org/10.1126/science.aab3897. |
[100] | J. P. K. Ip, N. Mellios and M. Sur, Rett syndrome:Insights into genetic, molecular and circuit mechanisms, Nat. Rev. Neurosci., 19 (2018), 368-382, URL https://doi.org/10.1038/s41583-018-0006-3. |