m8ta
You are not authenticated, login. |
|
{1552} | ||
Modularizing Deep Learning via Pairwise Learning With Kernels
I think in general this is an important result, even if its not wholly unique / somewhat anticipated (it's a year old at the time of writing). Modular training of neural networks is great for efficiency, parallelization, and biological implementations! Transport of weights between layers is hence non-essential. Classes still are, but I wonder if temporal continuity can solve some of these problems? (There is plenty of other effort in this area -- see also {1544}) | ||
{1547} | ||
Meta-Learning Update Rules for Unsupervised Representation Learning
This is a clearly-written, easy to understand paper. The results are not highly compelling, but as a first set of experiments, it's successful enough. I wonder what more constraints (fewer parameters, per the genome), more options for architecture modifications (e.g. different feedback schemes, per neurobiology), and a black-box optimization algorithm (evolution) would do? | ||
{1528} | ||
Discovering hidden factors of variation in deep networks
| ||
{1455} | ||
Conducting credit assignment by aligning local distributed representations
Lit review.
| ||
{1426} | ||
Training neural networks with local error signals
| ||
{1432} | ||
Direct Feedback alignment provides learning in deep neural nets
| ||
{1423} | ||
PMID-27824044 Random synaptic feedback weights support error backpropagation for deep learning.
Our proof says that weights W0 and W evolve to equilibrium manifolds, but simulations (Fig. 4) and analytic results (Supple- mentary Proof 2) hint at something more specific: that when the weights begin near 0, feedback alignment encourages W to act like a local pseudoinverse of B around the error manifold. This fact is important because if B were exactly W + (the Moore- Penrose pseudoinverse of W ), then the network would be performing Gauss-Newton optimization (Supplementary Proof 3). We call this update rule for the hidden units pseudobackprop and denote it by ∆hPBP = W + e. Experiments with the linear net- work show that the angle, ∆hFA ]∆hPBP quickly becomes smaller than ∆hFA ]∆hBP (Fig. 4b, c; see Methods). In other words feedback alignment, despite its simplicity, displays elements of second-order learning. |