m8ta
use https for features.
text: sort by
tags: modified
type: chronology
{1482}
hide / / print
ref: -2019 tags: meta learning feature reuse deepmind date: 10-06-2019 04:14 gmt revision:1 [0] [head]

Rapid learning or feature reuse? Towards understanding the effectiveness of MAML

  • It's feature re-use!
  • Show this by freezing the weights of a 5-layer convolutional network when training on Mini-imagenet, either 5shot 1 way, or 5shot 5 way.
  • From this derive ANIL, where only the last network layer is updated in task-specific training.
  • Show that ANIL works for basic RL learning tasks.
  • This means that roughly the network does not benefit much from join encoding -- encoding both the task at hand and the feature set. Features can be learned independently from the task (at least these tasks), with little loss.

{1434}
hide / / print
ref: -0 tags: convolutional neural networks audio feature extraction vocals keras tensor flow fourier date: 02-18-2019 21:40 gmt revision:3 [2] [1] [0] [head]

Audio AI: isolating vocals from stereo music using Convolutional Neural Networks

  • Ale Koretzky
  • Fairly standard CNN, but use a binary STFT mask to isolate vocals from instruments.
    • Get Fourier-type time-domain artifacts as a results; but it sounds reasonable.
    • Didn't realize it until this paper / blog post: stacked conv layers combine channels.
    • E.g. Input size 513*25*16 513 * 25 * 16 (512 freq channels + DC, 25 time slices, 16 filter channels) into a 3x3 Conv2D -> 3*3*16+16=1603 * 3 * 16 + 16 = 160 total parameters (filter weights and bias).
    • If this is followed by a second Conv2D layer of the same parameters, the layer acts as a 'normal' fully connected network in the channel dimension.
    • This means there are (3*3*16)*16+16=2320(3 * 3 * 16) * 16 + 16 = 2320 parameters.
      • Each input channel from the previous conv layer has independent weights -- they are not shared -- whereas the spatial weights are shared.
      • Hence, same number of input channels and output channels (in this case; doesn't have to be).
      • This, naturally, falls out of spatial weight sharing, which might be obvious in retrospect; of course it doesn't make sense to share non-spatial weights.
      • See also: https://datascience.stackexchange.com/questions/17064/number-of-parameters-for-convolution-layers
  • Synthesized a large training set via acapella youtube videos plus instrument tabs .. that looked like a lot of work!
    • Need a karaoke database here.
  • Authors wrapped this into a realtime extraction toolkit.