There’s a common misconception that neural networks’ recent success on a slew of problems is due to the increasing speed and decreasing cost of GPUs. In reality, that’s not the case. Modern processing power plays a critical role, but only when combined with a series of innovations in architecture and training. You can’t process million-image datasets like ImageNet without a GPU, but without Resnet you won’t be able to achieve good results.