Network inference with hidden units

  • Received: 01 December 2012 Accepted: 29 June 2018 Published: 01 September 2013
  • MSC : Primary: 62M45, 82C20; Secondary: 62J02.

  • We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.

    Citation: Joanna Tyrcha, John Hertz. Network inference with hidden units[J]. Mathematical Biosciences and Engineering, 2014, 11(1): 149-165. doi: 10.3934/mbe.2014.11.149

    Related Papers:

  • We derive learning rules for finding the connections between units in stochastic dynamical networks from the recorded history of a ``visible'' subset of the units. We consider two models. In both of them, the visible units are binary and stochastic. In one model the ``hidden'' units are continuous-valued, with sigmoidal activation functions, and in the other they are binary and stochastic like the visible ones. We derive exact learning rules for both cases. For the stochastic case, performing the exact calculation requires, in general, repeated summations over an number of configurations that grows exponentially with the size of the system and the data length, which is not feasible for large systems. We derive a mean field theory, based on a factorized ansatz for the distribution of hidden-unit states, which offers an attractive alternative for large systems. We present the results of some numerical calculations that illustrate key features of the two models and, for the stochastic case, the exact and approximate calculations.


    加载中
    [1] Cogn. Sci., 9 (1985), 147-169.
    [2] IEE Transactions on Automatic Control, AC-19 (1974), 716-723.
    [3] chapter 11, Cambridge Univ. Press, 2012.
    [4] J. Roy. Stat. Soc. B, 39 (1977), 1-38.
    [5] Phys. Rev. E, 87 (2013), 022127.
    [6] J. Math. Phys., 4 (1963), 294-307.
    [7] in "Principles of Neural Coding" (eds. S. Panzeri and R. R. Quiroga), CRC Press, (2013), 527-546.
    [8] chapter 2, World Scientific Lecture Notes in Physics, 9, World Scientific Publishing Co., Inc., Teaneck, NJ, 1987.
    [9] Neural Computation, 1 (1989), 263-269.
    [10] Biol. Cybern., 50 (1984), 51-62.
    [11] Phys. Rev. Lett., 59 (1987), 2229-2232.
    [12] Phys. Rev. Lett., 106 (2011), 048702.
    [13] Phys. Rev. E, 79 (2009), 051915.
    [14] in "Parallel Distributed Processing" (eds. D. E. Rumelhart and J. L. McClelland), Vol. 1, Chapter 8, MIT Press, 1986.
    [15] J. Art. Intel. Res., 4 (1996), 61-76.
    [16] Nature, 440 (2006), 1007-1012.
    [17] Annals of Statistics, 6 (1978), 461-464.
    [18] Scand. J. Statistics, 1 (1974), 49-58.
    [19] Phys. Rev. Lett., 35 (1975), 1792-1796.
    [20] Philos. Mag., 92 (1974), 272-279.
    [21] Neural Comp., 1 (1989), 270-280.
  • Reader Comments
  • © 2014 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(1707) PDF downloads(486) Cited by(19)

Article outline

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog