We present a perspective on neural networks stemming from quiver representation theory. This point of view emphasizes the symmetries inherent in neural networks, interacts nicely with gradient descent, and has the potential to improve training algorithms. As an application, we prove an analogue of the QR decomposition for radial neural networks, which leads to a dimensional reduction result. We assume a basic machine learning background, while explaining all necessary representation theory concepts from first principles. \\ \\ The talk is based on joint work-in-progress with Robin Walters.