The chips that are best at running neural nets are the same ones that are best at training. If you want to run state-of-the-art deep neural nets in your phone, e.g. for translation, image recognition, always-on voice recognition, etc, then you're going to need dedicated hardware to fit in a phone's power constraints. State-of-the-art deep neural nets are already too large to run well on a phone CPU; today's applications either drastically cut down the neural net (compromising on performance) or send the data to a server for processing.