CPU only processes data resides in RAM, so whatever be the data need to keep in RAM in order to get processed by the processors.
So the time is taken is as follows
(cache memory->(if data is missing in cache)RAM->(if data is missing in RAM) others(e.g. HDD or EXTD)