Deep Learning with Coherent Nanophotonic Circuits
-
- Used a series of Mach-Zehnder interferometers with thermoelectric phase-shift elements to realize the unitary component of individual layer weight-matrix computation.
- Weight matrix was decomposed via SVD into UV*, which formed the unitary matrix (4x4, Special unitary 4 group, SU(4)), as well as diagonal matrix via amplitude modulators. See figure above / original paper.
- Note that interfereometric matrix multiplication can (theoretically) be zero energy with an optical system (modulo loss).
- In practice, you need to run the phase-moduator heaters.
- Nonlinearity was implemented electronically after the photodetector (e.g. they had only one photonic circuit; to get multiple layers, fed activations repeatedly through it. This was a demonstration!)
- Fed network FFT'd / banded recordings of consonants through the network to get near-simulated vowel recognition.
- Claim that noise was from imperfect phase setting in the MZI + lower resolution photodiode read-out.
- They note that the network can more easily (??) be trained via the finite difference algorithm (e.g. test out an incremental change per weight / parameter) since running the network forward is so (relatively) low-energy and fast.
- Well, that's not totally true -- you need to update multiple weights at once in a large / deep network to descend any high-dimensional valleys.
|