m8ta
You are not authenticated, login. |
|
{1540} | ||||||
Two Routes to Scalable Credit Assignment without Weight Symmetry This paper looks at five different learning rules, three purely local, and two non-local, to see if they can work as well as backprop in training a deep convolutional net on ImageNet. The local learning networks all feature forward weights W and backward weights B; the forward weights (+ nonlinearities) pass the information to lead to a classification; the backward weights pass the error, which is used to locally adjust the forward weights. Hence, each fake neuron has locally the forward activation, the backward error (or loss gradient), the forward weight, backward weight, and Hebbian terms thereof (e.g the outer product of the in-out vectors for both forward and backward passes). From these available variables, they construct the local learning rules:
Each of these serves as a "regularizer term" on the feedback weights, which governs their learning dynamics. In the case of backprop, the backward weights B are just the instantaneous transpose of the forward weights W. A good local learning rule approximates this transpose progressively. They show that, with proper hyperparameter setting, this does indeed work nearly as well as backprop when training a ResNet-18 network. But, hyperparameter settings don't translate to other network topologies. To allow this, they add in non-local learning rules:
In "Symmetric Alignment", the Self and Decay rules are employed. This is similar to backprop (the backward weights will track the forward ones) with L2 regularization, which is not new. It performs very similarly to backprop. In "Activation Alignment", Amp and Sparse rules are employed. I assume this is supposed to be more biologically plausible -- the Hebbian term can track the forward weights, while the Sparse rule regularizes and stabilizes the learning, such that overall dynamics allow the gradient to flow even if W and B aren't transposes of each other. Surprisingly, they find that Symmetric Alignment to be more robust to the injection of Gaussian noise during training than backprop. Both SA and AA achieve similar accuracies on the ResNet benchmark. The authors then go on to explain the plausibility of non-local but approximate learning rules with Regression discontinuity design ala Spiking allows neurons to estimate their causal effect. This is a decent paper,reasonably well written. They thought trough what variables are available to affect learning, and parameterized five combinations that work. Could they have done the full matrix of combinations, optimizing just they same as the metaparameters? Perhaps, but that would be even more work ... Regarding the desire to reconcile backprop and biology, this paper does not bring us much (if at all) closer. Biological neural networks have specific and local uses for error; even invoking 'error' has limited explanatory power on activity. Learning and firing dynamics, of course of course. Is the brain then just an overbearing mess of details and overlapping rules? Yes probably but that doesn't mean that we human's can't find something simpler that works. The algorithms in this paper, for example, are well described by a bit of linear algebra, and yet they are performant. | ||||||
{1455} | ||||||
Conducting credit assignment by aligning local distributed representations
Lit review.
| ||||||
{1441} | ||||||
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
| ||||||
{1432} | ||||||
Direct Feedback alignment provides learning in deep neural nets
| ||||||
{1423} | ||||||
PMID-27824044 Random synaptic feedback weights support error backpropagation for deep learning.
Our proof says that weights W0 and W evolve to equilibrium manifolds, but simulations (Fig. 4) and analytic results (Supple- mentary Proof 2) hint at something more specific: that when the weights begin near 0, feedback alignment encourages W to act like a local pseudoinverse of B around the error manifold. This fact is important because if B were exactly W + (the Moore- Penrose pseudoinverse of W ), then the network would be performing Gauss-Newton optimization (Supplementary Proof 3). We call this update rule for the hidden units pseudobackprop and denote it by ∆hPBP = W + e. Experiments with the linear net- work show that the angle, ∆hFA ]∆hPBP quickly becomes smaller than ∆hFA ]∆hBP (Fig. 4b, c; see Methods). In other words feedback alignment, despite its simplicity, displays elements of second-order learning. | ||||||
{1159} | ||||||
I highly agree with this philosophy / this deconstruction of the flow of information in human structures: http://www.lostgarden.com/2012/04/loops-and-arcs.html On criticism as a meta-arc game: "In the past I've discussed criticism as a game that attempts to revisit an arc repeatedly and embellish it with additional meaning. The game is to generate essays superficially based on some piece of existing art. In turn, other players generate additional essays based off the first essays. This acts as both a referee mechanism and judge. Score is accumulated via reference counts and by rising through an organization hierarchy. It is a deliciously political game of wit that is both impenetrable to outsiders and nearly independent of the actual source arcs. Here creating an arc becomes a move in the larger game. " | ||||||
{1081} | ||||||
PMID-20855421[0] Mapping Go-No-Go performance within the subthalamic nucleus region.
____References____
| ||||||
{1082} | ||||||
Just fer kicks, I tested what happens to low-order butterworth filters when you maladjust one of the feedback coefficients. [B, A] = butter(2, 0.1); [h, w] = freqz(B,A); A(2) = A(2) * 0.9; [h2, ~] = freqz(B,A); hold off subplot(1,2,1) plot(w,abs(h)) hold on; plot(w,abs(h2), 'r') title('10% change in one FB filter coef 2nd order butterworth') xlabel('freq, rads / sec'); ylabel('filter response'); % do the same for a higher order filter. [B, A] = butter(3, 0.1); [h, w] = freqz(B,A); A(2) = A(2) * 0.9; [h2, ~] = freqz(B,A); subplot(1,2,2) hold on plot(w,abs(h), 'b') plot(w,abs(h2), 'r') title('10% change in one FB filter coef 3rd order butterworth') xlabel('freq, rads / sec'); ylabel('filter response'); The filters show a resonant peak, even though feedback was reduced. Not surprising, really; a lot of systems will show reduced phase margin and will begin to oscillate when poles are moved. Does this mean that a given coefficient (anatomical area) is responsible for resonance? By itself, of course not; one can not extrapolate one effect from one manipulation in a feedback system, especially a higher-order feedback system. This, of course hold in the mapping of digital (or analog) filters to pathology or anatomy. Pathology is likely reflective of how the loop is structured, not how one element functions (well, maybe). For a paper, see {1083} | ||||||
{1066} | ||||||
PMID-6869036[0] The Piper rhythm--a phenomenon related to muscle resonance characteristics?
____References____
| ||||||
{1045} | ||||||
PMID-95711[0] Spike separation in multiunit records: A multivariate analysis of spike descriptive parameters
____References____
| ||||||
{907} | ||||||
PMID-7418770[0] Operant control of precentral neurons: the role of audio and visual feedback.
____References____
| ||||||
{349} |
ref: thesis-0
tags: clementine 042007 operant conditioning biofeedback tlh24
date: 01-06-2012 03:08 gmt
revision:4
[3] [2] [1] [0] [head]
|
|||||
channel 29 controlled the X direction: channel 81, the Y direction (this one was very highly modulated, and the monkey could get to a high rate ~60Hz. note that both units are sorted as one -- I ought to do the same on the other channels from now on, as this was rather predictive (this is duplicating Debbie Won's results): However, when I ran a wiener filter on the binned spike rates (this is not the rates as estimated through the polynomial filter), ch 81 was most predictive for target X position; ch 29, Y target position (?). This is in agreement with population-wide predictions of target position: target X was predicted with low fidelity (1.12; cc = 0.35 or so); target Y was, apparently, unpredicted. I don't understand why this is, as I trained the monkey for 1/2 hour on just the opposite. Actually this is because the targets were not in a random sequence - they were in a CCW sequence, hence the neuronal activity was correlated to the last target, hence ch 81 to target X! for reference, here is the ouput of bmi_sql: order of columns: unit,channel, lag, snr, variable ans = 1.0000 80.0000 5.0000 1.0909 7.0000 1.0000 80.0000 4.0000 1.0705 7.0000 1.0000 80.0000 3.0000 1.0575 7.0000 1.0000 80.0000 2.0000 1.0485 7.0000 1.0000 80.0000 1.0000 1.0402 7.0000 1.0000 28.0000 4.0000 1.0318 8.0000 1.0000 76.0000 2.0000 1.0238 11.0000 1.0000 76.0000 5.0000 1.0225 11.0000 1.0000 17.0000 0 1.0209 11.0000 1.0000 63.0000 3.0000 1.0202 8.0000 movies of the performance are here: | ||||||
{949} | ||||||
PMID-18509337[0] Cortical control of a prosthetic arm for self-feeding
____References____
| ||||||
{924} | ||||||
PMID-5941514[0] Feeding induced in cats by electrical stimulation of the brain stem.
____References____
| ||||||
{912} | ||||||
PMID-6457106 Processing visual feedback information for movement control.
| ||||||
{896} |
ref: Friston-2002.1
tags: neuroscience philosophy feedback top-down sensory integration inference
date: 10-25-2011 23:24 gmt
revision:0
[head]
|
|||||
PMID-12450490 Functional integration and inference in the brain
| ||||||
{715} |
ref: Legenstein-2008.1
tags: Maass STDP reinforcement learning biofeedback Fetz synapse
date: 04-09-2009 17:13 gmt
revision:5
[4] [3] [2] [1] [0] [head]
|
|||||
PMID-18846203[0] A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback
____References____
| ||||||
{329} | ||||||
PMID-17234689[0] Volitional control of neural activity: implications for brain-computer interfaces (part of a symposium)
humm.. this paper came out a month ago, and despite the fact that he is much older and more experienced than i, we have arrived at the same conclusions by looking at the same set of data/papers. so: that's good, i guess. ____References____ | ||||||
{577} | ||||||
I found this on my computer tucked away into a dusty corner. Such fascinating information should not be left hidden - | ||||||
{531} | ||||||
http://www.uspto.gov/go/pac/doc/general/
| ||||||
{479} | ||||||
http://pespmc1.vub.ac.be/books/IntroCyb.pdf -- dated, but still interesting, useful, a book in and of itself!
| ||||||
{106} | ||||||
PMID-15208695[0] PDF HTML summary Optimal feedback control and the neural basis of volitional motor control by Stephen S. Scott ____References____ | ||||||
{141} |
ref: learning-0
tags: motor control primitives nonlinear feedback systems optimization
date: 0-0-2007 0:0
revision:0
[head]
|
|||||
http://hardm.ath.cx:88/pdf/Schaal2003_LearningMotor.pdf not in pubmed. |