Are we talking professional context here?
Depending on your company you'll be creating technical debt without the time to "pay" it . There was a time I'd stop and try to refactor but then I realised time was going by and I still had stuff to do so making things work was the smart answer at that time.
If you're talking personal projects, depends right?
Are you building a product and on your way to an MVP? You really should not get into the refactoring loop until you have things up and running because you will only delay your project.
Are you submitting to your github? Then yes, you should get into the refactoring loop so you show what you're made of.
Time for refactoring is when you have time and depending on the context when you're allowed to, and also when you're obligated to do it because there's no other way out and you'd end up spending more time creating more logic than it would be to refactor the one at hand.
My opinion only though!