R. V. Florian Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity ================================================================================================== "The learning rules that we propose below can be applied to any type of spiking neural model; their applicability is not restricted to probabilistic models nor to the SRM model that we previously used in the analytical derivation." "If we simulate the network in discrete time with a timestep dt, the dynamics of the synapses is defined, ... for MSTDP, by (41)-(44)." "Thus, the dynamics of the neuronsfmembrane potential was given by (45) "More biologically-plausible simulations of reward-modulated STDP were presented elsewhere (Florian, 2005); here, we aim to present the effects of the proposed learning rules using minimalist setups." Florian, R. V. (2005), A reinforcement learning algorithm for spiking neural networks, in D. Zaharie, D. Petcu, V. Negru, T. Jebelean, G. Ciobanu, A. CicortaCS, A. Abraham and M. Paprzycki, eds, Proceedings of the Seventh International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2005), IEEE Computer Society, Los Alamitos, CA, pp. 299.306. http://www.coneural.org/florian/papers/05_RL_for_spiking_NNs.php -------------------------------------------------------------------------------------------------- "In the following simulations we used networks composed of integrate-and-fire neurons with resting potential ur = -70 mV, firing threshold ƒÊ = -54 mV, reset potential equal to the resting potential, and decay time constant \tau = 20 ms (parameters from (Gutig et al., 2003) and similar to those in (Song et al., 2000)). Gutig, R., Aharonov, R., Rotter, S. and Sompolinsky, H. (2003), eLearning input correlations through nonlinear temporally asymmetric Hebbian plasticityf, Journal of Neuroscience 23(9), 3697.3714. http://www.jneurosci.org/cgi/content/full/23/9/3697 => done Song, S., Miller, K. D. and Abbott, L. F. (2000), 'Competitive hebbian learning through spike-timing-dependent synaptic plasticity, Nature Neuroscience 3, 919.926. => done