m8ta
You are not authenticated, login. |
|
{1571} | ||
One model for the learning of language
A more interesting result is Deep symbolic regression for recurrent sequences, where the authors (facebook/meta) use a Transformer -- in this case, directly taken from Vaswini 2017 (8-head, 8-layer QKV w/ a latent dimension of 512) to do both symbolic (estimate the algebraic recurrence relation) and numeric (estimate the rest of the sequence) training / evaluation. Symbolic regression generalizes better, unsurprisingly. But both can be made to work even in the presence of (log-scaled) noise! While the language learning paper shows that small generative programs can be inferred from a few samples, the Meta symbolic regression shows that Transformers can evince either amortized memory (less likely) or algorithms for perception -- both new and interesting. It suggests that 'even' abstract symbolic learning tasks are sufficiently decomposable that the sorts of algorithms available to an 8-layer transformer can give a useful search heuristic. (N.B. That the transformer doesn't spit out perfect symbolic or numerical results directly -- it also needs post-processing search. Also, the transformer algorithm has search (in the form of softmax) baked in to it's architecture.) This is not a light architecture: they trained the transformer for 250 epochs, where each epoch was 5M equations in batches of 512. Each epoch took 1 hour on 16 Volta GPUs w 32GB of memory. So, 4k GPU-hours x ~10 TFlops = 1.4e20 Flops. Compare this with grammar learning above; 7 days on 32 cores operating at ~ 3Gops/sec is 1.8e15 ops. Much, much smaller compute. All of this is to suggest a central theme of computer science: a continuum between search and memorization.
Most interesting for a visual neuroscientist (not that I'm one per se, but bear with me) is where on these axes (search, heuristic, memory) visual perception is. Clearly there is a high degree of recurrence, and a high degree of plasticity / learning. But is there search or local optimization? Is this coupled to the recurrence via some form of energy-minimizing system? Is recurrence approximating E-M? | ||
{1547} | ||
Meta-Learning Update Rules for Unsupervised Representation Learning
This is a clearly-written, easy to understand paper. The results are not highly compelling, but as a first set of experiments, it's successful enough. I wonder what more constraints (fewer parameters, per the genome), more options for architecture modifications (e.g. different feedback schemes, per neurobiology), and a black-box optimization algorithm (evolution) would do? | ||
{1482} | ||
Rapid learning or feature reuse? Towards understanding the effectiveness of MAML
| ||
{1391} | ||
Evolutionary Plasticity and Innovations in Complex Metabolic Reaction Networks
Summary thoughts: This is a highly interesting study, insofar that the authors show substantial support for their hypotheses that phenotypes can be explored through random-walk non-lethal mutations of the genotype, and this is somewhat invariant to the source of carbon for known biochemical reactions. What gives me pause is the use of linear programming / optimization when setting the relative concentrations of biomolecules, and the permissive criteria for accepting these networks; real life (I would imagine) is far more constrained. Relative and absolute concentrations matter. Still, the study does reflect some robustness. I suggest that a good control would be to ‘fuzz’ the list of available reactions based on statistical criteria, and see if the results still hold. Then, go back and make the reactions un-biological or less networked, and see if this destroys the measured degrees of robustness. | ||
{1357} | ||
Physical Metallurgy of Refactory Metals and Alloys Properties of tungsten-rhenium alloys
| ||
{1330} | ||
META II: Digital Vellum in the Digital Scriptorium: Revisiting Schorre's 1962 compiler-compiler
| ||
{1315} | ||
| ||
{1272} | ||
| ||
{1252} | ||
PMID-20119944 Characterization of parylene C as an encapsulation material for implanted neural prostheses.
| ||
{1242} | ||
We're making parylene electrodes for neural recording, and one critical step is connecting them to recording electronics. Currently Berkeley uses ACF (anisotropic conductive film) for connection, which is widely used for connecting flex tape to LCD panels, or for connecting driver chips to LCD glass. According to the internet, pitches can be as low as 20um, with pad areas as low as 800um^2. source However, this does not seem to be a very reliable nor compact process with platinum films on parylene, possibly because ACF bonding relies on raised areas between mated conductors (current design has the Pt recessed into the parylene), and on rigid substrates. ACF consists of springy polymer balls coated in Ni and Au and embedded in a thermoset epoxy resin. The ACF film is put under moderate temperature (180C) and pressure (3mpa, 430psi), which causes the epoxy to cure in a state that leaves the gold/nickel/polymer balls to be compressed between the two conductors. Hence, even if the conductors move slightly due to thermal cycling, the small balls maintain good mechanical and electrical contact. The balls are dispersed sufficiently in the epoxy matrix that there is little to no chance of conduction between adjacent pads. (Or so I have learned from the internet.) Now, as mentioned, this is an imperfect method for joining Pt on parylene films, possibly because the parylene is so flexible, and the platinum foil is very thin (200-300 nm). Indeed, platinum does not bond very strongly to parylene, hence care must be taken to allow sufficient overlap to prevent water ingress. My proposed solution -- to be tested shortly -- is to use a low-melting temperature metal with strong wetting ability -- such as Field's metal (bismuth, tin, indium, melting point 149F, see http://www.gizmology.net/fusiblemetals.htm) to low-temperature solder the platinum to a carrier board (initially) or to a custom amplifier ASIC (later!). Parylene is stable to 200C (392F), so this should be safe. One worry is that the indium/bismuth will wet the parylene or polyimide, too; however I consider this unlikely due to the difficulty in attaching parylene to any metal. That said, there must be good reason why ACF is so popular, so perhaps a better ultimate solution is to stiffen the parylene (or ultimately polyimide) substrate so that it can support both the temperature/pressure of ACF bonding and the stress of a continued electrical/mechanical bond to polyimide fan-out board or ASIC. It may also be possible to gold or nickel electroplate the connector pads to be slightly raised instead of recessed. Update: ACF bond to rigid 1/2 oz copper, 4mil trace / space connector (3mil trace/space board): Note that the copper traces are raised, and the parylene is stretched over the uneven surface (this is much easier to see with the stereo microscope). To the left of the image, the ACF paste has been sqeezed out from between the FR4 and parylene. Also note that the platinum can make potential contact with vias in the PCB. Update 7/2: Fields metal (mentioned above) does stick to platinum reasonably well, but it also sticks to parylene (somewhat), and glass (exceptionally well!). In fact, I had a difficult time removing traces of field's metal from the Pyrex beakers that I was melting the metal with. These beakers were filled with boiling water, which may have been the problem. When I added flux (Kester flux-pen 951 No-clean MSDS), the metal became noticeably more shiny, and the contact angle increased on the borosilicate glass (e.g. looked more like mercury); this leads me to believe that it is not the metal itself that attaches to glass, but rather oxides of indium and bismuth. Kester 951 flux consists of:
After coating the parylene/platinum sample with flux, I raised the field's metal to the flux activation point, which released some smoke and left brown organic residues on the bottom of the glass dish. Then I dipped the parylene probe into the molten metal, causing the flux again to be activated, and partially wetting the platinum contacts. The figure below shows the result: Note the incomplete wetting, all the white solids left from the process, and how the field's metal caused the platinum to delaminate from the parylene when the cable was (accidentally) flexed. Tests with platinum foil revealed that the metal bond was not actually that strong, significantly weaker than that made with a flux-core SnPb solder. also, I'm not sure of the activation temperature of this flux, and think I may have overheated the parylene. Update 7/10: Am considering electrodeless Ni / Pt / Au deposition, which occurs in aqueous solution, hence at much lower temperatures than e-beam evaporation Electrodeless Ni ref. On polyimide substrates, there is extensive literature describing how to activate the surface for plating: Polyimides and Other High Temperature Polymers: Synthesis ..., Volume 4. Parylene would likely need a different possibly more aggressive treatment, as it does not have imide bonds to open. Furthermore, if the parylene / polyimide surface is *not* activated, the electrodeless plating could be specific to the exposed electrode and contact sites, which could help to solve the connector issue by strengthening & thickening the contact areas. The second fairly obvious solution is to planarize the contact site on the PCB, too, as seen above. ACF bonds can be quite reliable; last night I took apart (and successfully re-assembled) my 32" Samsung LCD monitor, and none of the flex-on-glass or chip-on-flex binds failed (despite my clumsy hands!). | ||
{999} | ||
IEEE-4065599 (pdf) Comments on Microelectrodes
____References____ ' ''' () | ||
{718} | ||
Timetable / Plan:
Contingency Plan:
| ||
{838} |
ref: -0
tags: meta learning Artificial intelligence competent evolutionary programming Moshe Looks MOSES
date: 08-07-2010 16:30 gmt
revision:6
[5] [4] [3] [2] [1] [0] [head]
|
|
| ||
{815} | ||
Jacques Pitrat seems to have many of the same ideas that I've had (only better, and he's implemented them!)-- A Step toward and Artificial Scientist
Artificial beings - his book. | ||
{768} | ||
http://mlpost.lri.fr/ -- allows drawing Latex or postscript figures programmatically. Interesting. Included in Debian. source | ||
{215} | ||
for a barco data 3200LC, you need a HMI575/SE (single ended) lamp. unfortunately, this only lasts 750 hours :( and costs $150 http://www.bulbman.com/index.php?main_page=product_bulb_info&cPath=5399&products_id=10858 | ||
{35} | ||
Overview: a projector light should have good luminous efficiency, have a long life, and most importantly have plenty of energy in the red region of the spectrum. most metal halides have yellow/green lines and blue lines, few have good red lines. http://www.osram.no/brosjyrer/english/K01KAP5_en.pdf in 1000 watt, the Osram Powerstar HQI-TS 1000/d/s looks the best: CRI > 90, 5900K color temperature. Unfortunately, I cannot seem to find any american places to buy this bulb, nor can i determine its average life. It can be bought, at a price, from http://www.svetila.com/eProdaja/product_info.php/products_id/442 { n.b. the osram HMI bulbs are no good-the lifetime is too short} In 400 watt, the Eye Clean Arc MT400D/BUD looks quite good, with a CRI of 90, 6500K color temp. http://www.eyelighting.com/cleanarc.html. EYE also has a ceraarc line, but the 400w bulb is not yet in production (and it has a lower color temperature, 4000K). Can be bought from http://www.businesslights.com/ (N.B. they have spectral charts for many of the lights!)
and fYI, the electrodelass bulbs are made by Osram and are called "ICETRON". They are rather expensive, but last 1e5 hours (!). Typical output is 80 lumens/watt more things of interest:
| ||
{8} |
ref: bookmark-0
tags: machine_learning algorithm meta_algorithm
date: 0-0-2006 0:0
revision:0
[head]
|
|
Boost learning or AdaBoost - the idea is to update the discrete distribution used in training any algorithm to emphasize those points that are misclassified in the previous fit of a classifier. sensitive to outliers, but not overfitting. | ||
{50} |
ref: bookmark-0
tags: teflon PTFE bonding metal polytetrafluoroethylene tetraflouroethylene
date: 0-0-2006 0:0
revision:0
[head]
|
|
http://pslc.ws/mactest/ptfeidea.htm block copolymer: http://en.wikipedia.org/wiki/Copolymer |