An interesting field in ML is nonlinear dimensionality reduction - data may appear to be in a high-dimensional space, but mostly lies along a nonlinear lower-dimensional subspace or manifold. (Linear subspaces are easily discovered with PCA or SVD(*)). Dimensionality reduction projects high-dimensional data into a low-dimensional space with minimum information loss -> maximal reconstruction accuracy; nonlinear dim reduction does this (surprise!) using nonlinear mappings. These techniques set out to find the manifold(s):
- Spectral Clustering
- Locally Linear Embedding
- related: The manifold ways of perception
- Would be interesting to run nonlinear dimensionality reduction algorithms on our data! What sort of space does the motor system inhabit? Would it help with prediction? Am quite sure people have looked at Kohonen maps for this purpose.
- Random irrelevant thought: I haven't been watching TV lately, but when I do, I find it difficult to recognize otherwise recognizable actors. In real life, I find no difficulty recognizing people, even some whom I don't know personally - is this a data thing (little training data), or mapping thing (not enough time training my TV-not-eyes facial recognition).
- A Global Geometric Framework for Nonlinear Dimensionality Reduction method:
- map the points into a graph by connecting each point with a certain number of its neighbors or all neighbors within a certain radius.
- estimate geodesic distances between all points in the graph by finding the shortest graph connection distance
- use MDS (multidimensional scaling) to embed the original data into a smaller-dimensional euclidean space while preserving as much of the original geometry.
- Doesn't look like a terribly fast algorithm!
(*) SVD maps into 'concept space', an interesting interpretation as per Leskovec's lecture presentation. |