Update: Fairly new to ComfyUI and made some mistakes when wiring this up. When using AI gens for the face reference it worked fine, however if I used a photo the results didn't really look like them.

So went back to the drawing board, simplified some parts and it seems to be resulting in a resemblance to the face reference photo now. I've also added in an IPAdapter for style transfer. This is optional and there's a Group Bypasser, so you can switch that off if you don't want it. One thing to note is that the IPA is more aggressive with the Face Detailer group enabled. You can use the Fast Group Bypass to have both on, off or one on and the other off, always good to have extra options to play :)

If you have any feedback please pop it in the comments so I can improve this :)

Based off my previous workflow, came across seven947's excellent 1minute 8K Upscale -  https://openart.ai/workflows/seven947/1minute-8k-upscale/1IPTks1gL7v0EPmvsMcx

This allowed me to generate very high resolution images without losing detail or colour, with no errors on my 2080Ti. So gutted the upscaler out of my workflow and replaced it with theirs. Also did a bit of testing and decided I prefer the results from InstantID, so took FaceID out and put that in.

The Upscaler section has it's own Checkpoint loader. I prefer doing the generation using a standard SDXL model. I did try hooking up the upscaler to the initial Efficient Loader, but get very noisy images. As a result that's set to JuggernautXL Lightning instead which doesn't have the noise issue.