Training on GPU vs CPU

In this experiment, we will dig into the details of training a model on GPU vs CPU, what is causing the difference in performance between the two?