We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.
Citation: Joanna Tyrcha, John Hertz. Network inference with hidden units[J]. Mathematical Biosciences and Engineering, 2014, 11(1): 149-165. doi: 10.3934/mbe.2014.11.149
Related Papers:
| [1] |
Bedr'Eddine Ainseba, Mostafa Bendahmane, Yuan He .
Stability of conductivities in an inverse problem in the reaction-diffusion system in electrocardiology. Networks and Heterogeneous Media, 2015, 10(2): 369-385.
doi: 10.3934/nhm.2015.10.369
|
| [2] |
Feiyang Peng, Yanbin Tang .
Inverse problem of determining diffusion matrix between different structures for time fractional diffusion equation. Networks and Heterogeneous Media, 2024, 19(1): 291-304.
doi: 10.3934/nhm.2024013
|
| [3] |
Aslam Khan, Abdul Ghafoor, Emel Khan, Kamal Shah, Thabet Abdeljawad .
Solving scalar reaction diffusion equations with cubic non-linearity having time-dependent coefficients by the wavelet method of lines. Networks and Heterogeneous Media, 2024, 19(2): 634-654.
doi: 10.3934/nhm.2024028
|
| [4] |
Avner Friedman .
PDE problems arising in mathematical biology. Networks and Heterogeneous Media, 2012, 7(4): 691-703.
doi: 10.3934/nhm.2012.7.691
|
| [5] |
L.L. Sun, M.L. Chang .
Galerkin spectral method for a multi-term time-fractional diffusion equation and an application to inverse source problem. Networks and Heterogeneous Media, 2023, 18(1): 212-243.
doi: 10.3934/nhm.2023008
|
| [6] |
Danielle Hilhorst, Hideki Murakawa .
Singular limit analysis of a reaction-diffusion system with precipitation and dissolution in a porous medium. Networks and Heterogeneous Media, 2014, 9(4): 669-682.
doi: 10.3934/nhm.2014.9.669
|
| [7] |
Toshiyuki Ogawa, Takashi Okuda .
Oscillatory dynamics in a reaction-diffusion system in the presence of 0:1:2 resonance. Networks and Heterogeneous Media, 2012, 7(4): 893-926.
doi: 10.3934/nhm.2012.7.893
|
| [8] |
Mostafa Bendahmane, Kenneth H. Karlsen .
Analysis of a class of degenerate reaction-diffusion systems and the bidomain model of cardiac tissue. Networks and Heterogeneous Media, 2006, 1(1): 185-218.
doi: 10.3934/nhm.2006.1.185
|
| [9] |
Alexander Mielke, Sina Reichelt, Marita Thomas .
Two-scale homogenization of nonlinear
reaction-diffusion systems with slow diffusion. Networks and Heterogeneous Media, 2014, 9(2): 353-382.
doi: 10.3934/nhm.2014.9.353
|
| [10] |
Verónica Anaya, Mostafa Bendahmane, David Mora, Ricardo Ruiz Baier .
On a vorticity-based formulation for reaction-diffusion-Brinkman systems. Networks and Heterogeneous Media, 2018, 13(1): 69-94.
doi: 10.3934/nhm.2018004
|
Abstract
We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.
References
|
[1]
|
Cogn. Sci., 9 (1985), 147-169.
|
|
[2]
|
IEE Transactions on Automatic Control, AC-19 (1974), 716-723.
|
|
[3]
|
chapter 11, Cambridge Univ. Press, 2012.
|
|
[4]
|
J. Roy. Stat. Soc. B, 39 (1977), 1-38.
|
|
[5]
|
Phys. Rev. E, 87 (2013), 022127.
|
|
[6]
|
J. Math. Phys., 4 (1963), 294-307.
|
|
[7]
|
in "Principles of Neural Coding" (eds. S. Panzeri and R. R. Quiroga), CRC Press, (2013), 527-546.
|
|
[8]
|
chapter 2, World Scientific Lecture Notes in Physics, 9, World Scientific Publishing Co., Inc., Teaneck, NJ, 1987.
|
|
[9]
|
Neural Computation, 1 (1989), 263-269.
|
|
[10]
|
Biol. Cybern., 50 (1984), 51-62.
|
|
[11]
|
Phys. Rev. Lett., 59 (1987), 2229-2232.
|
|
[12]
|
Phys. Rev. Lett., 106 (2011), 048702.
|
|
[13]
|
Phys. Rev. E, 79 (2009), 051915.
|
|
[14]
|
in "Parallel Distributed Processing" (eds. D. E. Rumelhart and J. L. McClelland), Vol. 1, Chapter 8, MIT Press, 1986.
|
|
[15]
|
J. Art. Intel. Res., 4 (1996), 61-76.
|
|
[16]
|
Nature, 440 (2006), 1007-1012.
|
|
[17]
|
Annals of Statistics, 6 (1978), 461-464.
|
|
[18]
|
Scand. J. Statistics, 1 (1974), 49-58.
|
|
[19]
|
Phys. Rev. Lett., 35 (1975), 1792-1796.
|
|
[20]
|
Philos. Mag., 92 (1974), 272-279.
|
|
[21]
|
Neural Comp., 1 (1989), 270-280.
|
DownLoad: