WEBVTT

00:00.000 --> 00:03.333
Welcome to AI for visual effects
I'm Doug Hogan.

00:03.333 --> 00:06.625
In this video, I want to walk you
through a powerful workflow

00:06.625 --> 00:10.458
for doing automated face swaps
inside of ComfyUI.

00:10.958 --> 00:15.041
This setup uses a combination of Florence
2 for object detection,

00:15.083 --> 00:20.000
Sam 2 for segmentation,
and Wan Video for temporal generation.

00:20.083 --> 00:25.041
The goal here is to take an input video,
isolate the head, and replace it

00:25.041 --> 00:30.375
with a new identity while preserving
motion, lighting, and performance.

00:30.541 --> 00:33.125
We start by bringing in our source
footage.

00:33.125 --> 00:37.041
This gets decoded into individual frames
along with useful metadata

00:37.083 --> 00:40.958
like resolution,
frame count, and frame rate.

00:41.000 --> 00:42.375
At the same time, we load

00:42.375 --> 00:46.833
the core models needed for the workflow
that includes the one video model,

00:46.875 --> 00:52.875
the VAE clip vision, and the detection tools
used for masking and motion analysis.

00:53.083 --> 00:55.666
Next, we generate a mask of the head.

00:55.666 --> 00:59.541
We use Florence 2 to identify the region
we want to swap.

00:59.541 --> 01:02.166
In this case, we're targeting the head.

01:02.166 --> 01:05.875
That bounding box
then gets passed into Sam 2,

01:05.875 --> 01:11.083
which creates a more accurate segmentation
mask for the area we want to replace.

01:11.208 --> 01:14.041
After that, we refine the mask.

01:14.041 --> 01:19.416
One of the most important controls
here is the grow mask expand value.

01:19.458 --> 01:20.750
This needs to be large enough

01:20.750 --> 01:24.958
that the mask is not hugging
the silhouette of the head too tightly.

01:24.958 --> 01:27.541
If the mask is too close
to the existing outline,

01:27.541 --> 01:33.583
the model has less freedom to reinterpret
shape and integrate to the new face naturally.

01:33.583 --> 01:36.083
It sometimes needs a little room to be creative,

01:36.083 --> 01:38.000
so keep an eye on this.

01:38.000 --> 01:41.000
Depending on the shot, this is one
of the first settings I would adjust.

01:41.041 --> 01:43.458
From there we prep the driving data.

01:43.458 --> 01:47.458
We run pose and face detection
on the original footage,

01:47.458 --> 01:52.333
so the generated result can follow the existing motion and performance of the source footage.

01:52.375 --> 01:55.458
At the same time,
we bring in a reference image

01:55.458 --> 01:58.916
which defines the new identity
we want to apply.

01:59.083 --> 02:02.083
We also generate an auto prompt using QWEN VL.

02:02.125 --> 02:04.958
This analyzes
the reference image and creates

02:04.958 --> 02:09.125
a detailed text description that helps
guide the model during generation.

02:09.166 --> 02:13.208
That means the workflow is not just
copying pixels, it's also building

02:13.208 --> 02:18.000
a better semantic understanding of what
the replacement person should look like.

02:18.000 --> 02:22.208
So just review the prompt
and then copy and paste it directly

02:22.208 --> 02:26.291
into Wan Video text encode cached node.

02:26.333 --> 02:29.458
Now everything comes together inside Wan.

02:29.458 --> 02:31.708
We combine the reference identity,

02:31.708 --> 02:35.333
the pose information,
the original background, and the mask.

02:35.375 --> 02:39.666
The sampler then generates a new sequence
where the face has been replaced,

02:39.666 --> 02:43.625
while still following the timing, motion,
and overall structure

02:43.625 --> 02:45.291
of the source footage.

02:45.291 --> 02:49.541
Once sampling is complete,
we decode the latent result back

02:49.541 --> 02:54.041
into image frames
and reassemble those into a final video.

02:54.291 --> 02:57.458
We can also bring the original audio
back in at this stage,

02:57.458 --> 03:00.625
which makes it easy to preserve
sync from the source shot.

03:00.666 --> 03:04.458
The result is a temporarily
consistent face swap.

03:04.500 --> 03:07.541
Now a couple tips, because a few things
may need to be adjusted

03:07.541 --> 03:08.916
depending on your footage.

03:08.916 --> 03:12.958
The grow mass value is a big one
like we talked about earlier, but you may

03:12.958 --> 03:17.666
also want to tweak the reference image
and the prompt to get the best results.

03:17.750 --> 03:22.166
Shots with strong motion, motion
blur, extreme angles,

03:22.166 --> 03:26.291
or difficult lighting
may need a little more iteration.

03:26.333 --> 03:29.958
One nice thing about this workflow
is that it also includes an app mode,

03:29.958 --> 03:34.166
so if you're not comfortable
working inside a dense node graph,

03:34.208 --> 03:38.083
you can still use the setup through a much
simpler step-by-step process.

03:38.083 --> 03:42.916
You simply upload your footage,
upload your reference image for the object swap,

03:42.916 --> 03:48.583
identify what you want to replace,
in this case the head, and then click go.

03:48.583 --> 03:51.625
That makes this much more approachable
for artists who want the result

03:51.625 --> 03:54.708
without needing to manage every node
manually.

03:55.041 --> 03:59.583
So that's the workflow, a flexible face
swap setup that combines automatic

03:59.583 --> 04:03.333
masking, motion analysis, reference
driven prompting,

04:03.333 --> 04:06.833
and temporal video
generation all in one place.