m8ta
You are not authenticated, login. |
|
{1571} | ||||
One model for the learning of language
A more interesting result is Deep symbolic regression for recurrent sequences, where the authors (facebook/meta) use a Transformer -- in this case, directly taken from Vaswini 2017 (8-head, 8-layer QKV w/ a latent dimension of 512) to do both symbolic (estimate the algebraic recurrence relation) and numeric (estimate the rest of the sequence) training / evaluation. Symbolic regression generalizes better, unsurprisingly. But both can be made to work even in the presence of (log-scaled) noise! While the language learning paper shows that small generative programs can be inferred from a few samples, the Meta symbolic regression shows that Transformers can evince either amortized memory (less likely) or algorithms for perception -- both new and interesting. It suggests that 'even' abstract symbolic learning tasks are sufficiently decomposable that the sorts of algorithms available to an 8-layer transformer can give a useful search heuristic. (N.B. That the transformer doesn't spit out perfect symbolic or numerical results directly -- it also needs post-processing search. Also, the transformer algorithm has search (in the form of softmax) baked in to it's architecture.) This is not a light architecture: they trained the transformer for 250 epochs, where each epoch was 5M equations in batches of 512. Each epoch took 1 hour on 16 Volta GPUs w 32GB of memory. So, 4k GPU-hours x ~10 TFlops = 1.4e20 Flops. Compare this with grammar learning above; 7 days on 32 cores operating at ~ 3Gops/sec is 1.8e15 ops. Much, much smaller compute. All of this is to suggest a central theme of computer science: a continuum between search and memorization.
Most interesting for a visual neuroscientist (not that I'm one per se, but bear with me) is where on these axes (search, heuristic, memory) visual perception is. Clearly there is a high degree of recurrence, and a high degree of plasticity / learning. But is there search or local optimization? Is this coupled to the recurrence via some form of energy-minimizing system? Is recurrence approximating E-M? | ||||
{1506} | ||||
Improved asymmetric locality sensitive hashing for maximum inner product search
Use case: Capsule: a camera based positioning system using learning
| ||||
{1387} |
ref: -1977
tags: polyethylene surface treatment plasma electron irradiation mechanical testing saline seawater accelerated lifetime
date: 04-15-2017 06:06 gmt
revision:0
[head]
|
|||
Enhancement of resistance of polyethylene to seawater-promoted degradation by surface modification
| ||||
{1192} |
ref: -2002
tags: sea slugs flexible electrodes polymide Washington
date: 01-04-2013 18:46 gmt
revision:0
[head]
|
|||
IEEE-1002325 (pdf) Silicon micro-needles with flexible interconnections
| ||||
{166} | ||||
PMID-19744484 What can man do without basal ganglia motor output? The effect of combined unilateral subthalamotomy and pallidotomy in a patient with Parkinson's disease.
| ||||
{1092} | ||||
PMID-18701754[0] Neuronal activity of the human subthalamic nucleus in the parkinsonian and nonparkinsonian state
____References____
| ||||
{828} |
ref: RodriguezOroz-2001.09
tags: STN SNr parkinsons disease single unit recording spain 2001 tremor oscillations DBS somatotopy organization
date: 02-22-2012 18:24 gmt
revision:12
[11] [10] [9] [8] [7] [6] [head]
|
|||
PMID-11522580[0] The subthalamic nucleus in Parkinson's disease: somatotopic organization and physiological characteristics
Old notes:
____References____
| ||||
{1129} | ||||
PMID-15272269 Stem cell therapy for human neurodegenerative disorders-how to make it work.
Stroke:
ALS:
Synthesis:
| ||||
{1096} | ||||
There seems to be an interesting connection between excessive grip force, isometric muscle contraction causing coherence between motor cortex and EMG, lack of inhibition on delayed response and go-no-go task, and experiments with STN lesioned rats, and the high/low oscillation hypothesis. Rather tenuous, I suppose, but let me spell it out. ( My personal impression, post-hoc, is that this is an epiphenomena of something else; evidence is contradictory.)
footnote: how much is our search for oscillations informed by our available analytical techniques? Hypothesis: Impulsivity may be the cognitive equivalent of excess grip force; maintenance of consistent 'force' or delayed decision making benefits from Piper-band rhythms, something which PD abolishes (gradually, through brain adaptation). DBS disrupts the beta (resting, all synchronized) rhythm, and thereby permits movement. However it also effectively 'lesions' the STN, which leads to cognitive deficits and poor force control. (Wait .. DBS plus levodopa improves 40-60Hz energy -- this would argue against the hypothesis. Also, stroke in the STN in normal individuals causes hemiballismus, which resolves gradually; this is not consistent with oscillations, but rather connectivity and activity.) Testing this hypothesis: well, first of all, is there beta-band oscillations in our data? what about piper band? We did not ask the patients to delay response, so any tests thereof will be implicit. Can look at relative energy 10Hz-30Hz and 30Hz-60Hz in the spike traces & see if this is modulated by hand position. (PETH as usual). So. I made PETHs for beta / gamma power ratio of the spiking rate, controlled by shuffling the PETH triggers. Beta power was between 12 and 30 Hz; gamma between 30 and 75 Hz, as set by a noncausal IIR bandpass filter. The following is a non-normalized heatmap of all significant PETHs over all sessions triggered when the hand crossed the midpoint between targets. (A z-scored heatmap was made as well; it looked worse). X is session number, Y time, 0 = -1 sec. sampling rate = 200 Hz. In one file (the band) there seems to be selective gamma inhibition about 0.5 sec before peak movement. Likely it is an outlier. 65 neurons of 973 (single and multiunits together) were significantly 'tuned' = 6.6%; marginally significant by binomial test (p=0.02). Below is an example PETH, with the shuffled distribution represented by mean +- 1 STD in blue. The following heatmap is created from the significant PETHs triggered on target appearance. 80 of the 204 significant PETHs are from PLEX092606005_a. The total number of significant responses (204/1674, single units and multiunits) is significant by the binomial test p < 0.001, with and without Sept. 26 removed. Below is an example plot (092606005). Looks pretty damn good, actually. Let's see how stable this relationship is by doing a leave-half out cross-validation, 10 plies, in red below (all triggers plotted in black) Looks excellent! Problem is we are working with a ratio, which is prone to spikes. Fix: work in log space. Aggregate response remains about the same. 192 / 1674 significant (11.5%) In the above figure, positive indicates increased power relative to power. The square shape is likely relative to (negative lags) hold time and (positive lags) reaction time, though the squareness is somewhat concerning. Recording is from VIM. | ||||
{5} |
ref: bookmark-0
tags: machine_learning research_blog parallel_computing bayes active_learning information_theory reinforcement_learning
date: 12-31-2011 19:30 gmt
revision:3
[2] [1] [0] [head]
|
|||
hunch.net interesting posts:
| ||||
{827} |
ref: OSuilleabhain-1998.11
tags: analysis tremor parkinsons disease
date: 07-19-2010 19:22 gmt
revision:2
[1] [0] [head]
|
|||
PMID-9827772[0] Time-frequency analysis of tremor
____References____
| ||||
{660} | ||||
In the process of installing compiz - which I decided I didn't like - I removed Xfce4's window manager, xfwm4, and was stuck with metacity. Metacity probably allows focus-follows-mouse, but this cannot be configured with Xfce's control panel, hence I had to figure out how to change it back. For this, I wrote a command to look for all files, opening each, and seeing if there are any lines that match "metacity". It's a brute force approach, but one that does not require much thinking or googling. find . -print | grep -v mnt | \ perl -e 'while($k = <STDIN>){open(FH,"< $k");while($j=<FH>){if($j=~/metacity/){print "found $k";}}close FH;}'This led me to discover ~/.cache/sessions/xfce4-session-loco:0 (the name of the computer is loco). I changed all references of 'metacity' to 'xfwm4', and got the proper window manager back. | ||||
{409} |
ref: bookmark-0
tags: optimization function search matlab linear nonlinear programming
date: 08-09-2007 02:21 gmt
revision:0
[head]
|
|||
http://www.mat.univie.ac.at/~neum/ very nice collection of links!! | ||||
{18} | ||||
SELECT * FROM `base` WHERE MATCH(`From`, `To`) AGAINST('hanson') ORDER BY `Date` DESC Limit 0, 100
| ||||
{31} |
ref: bookmark-0
tags: job_search professional employment wisdom
date: 0-0-2006 0:0
revision:0
[head]
|
|||