The need for an additional refactor phase in TDD

I've used TDD for a long time and in a few different settings. I usually can't develop whole projects driven by tests, but for many problems, it is my preferred problem-solving approach. Practice makes it better, but I still don't feel too confident about the short red-green-refactor cycles.

I am constantly uncertain about the refactoring phase.

Which technically is refactoring, but I think it would be more honest to call it cleanup rather than just plain refactoring.

It provides an opportunity to polish the changes made in the cycle, remove fakes, and clean accumulated mess from previous cycles. It doesn't have exact goals rather than "make it better". For me it's mostly based on looking for code smells and try to go for clean code rules.

While it's a necessary thing to do, this phase doesn't address my need to modify existing code to provide better support for new requirements.

This is usually a pain point for me. Driven by tests, I come up with the next requirement when I write a failing test. When I switch to the production code I often find myself in a situation where I would make small rearrangements in the code before I could comfortably start working on the next feature.

In this case, I do the following:

1. switch back to the tests
2. comment out the recently added failing case
3. do the necessary refactoring
4. run the test suite again to see I didn't break anything
5. continue where I left off

This seems clunky, but I find it more efficient than fighting my instincts and try to make the initial test pass. While this breaks the red-green-refactor flow, these extra steps can be done pretty quickly.

However, I was wondering if this is experienced by others as well, and what to do about it.

Maybe a possible solution could be to aim for a more general code by creating smaller methods, but it feels like a bit overdoing. In the refactor phase I try to make the production code appealing to the (my) eyes roughly based on clean code guidelines, rather than taking them as a strict rulebook.

Do you face this when doing TDD?