WEBVTT 00:00.083 --> 00:01.875 Welcome to AI for Visual Effects, 00:01.875 --> 00:03.166 I'm Doug Hogan. 00:03.166 --> 00:06.541 In this video, I want to show you a really powerful workflow for doing 00:06.541 --> 00:10.833 sky replacement using ComfyUI and Wan 2.2. 00:11.083 --> 00:14.666 What makes this setup interesting is that we're not just replacing a sky 00:14.666 --> 00:17.791 in a single frame, we're generating a fully animated sky 00:17.833 --> 00:21.333 that integrates into the shot and stays consistent over time. 00:21.375 --> 00:23.458 We start by bringing in our source footage. 00:23.458 --> 00:27.791 This gets decoded into frames along with metadata like frame count 00:27.791 --> 00:32.208 and frame rate, which we'll use later when rebuilding the final shot. 00:32.291 --> 00:35.250 From there, we generate a new skyplate. 00:35.250 --> 00:36.000 In this case, 00:36.000 --> 00:39.833 we're using an image generation model to reinterpret the original frame. 00:39.833 --> 00:44.000 For example, taking the daytime sky and converting it into a sunset. 00:44.041 --> 00:47.000 This gives us a stylized version of the original image 00:47.000 --> 00:49.416 that we can use as our replacement base. 00:49.416 --> 00:52.708 But instead of stopping there, we push this further. 00:52.750 --> 00:55.750 We pass that generated image into QWEN VL 00:55.791 --> 00:59.291 to automatically create a detailed video prompt. 00:59.333 --> 01:03.416 This step is really important because instead of writing prompts manually, 01:03.458 --> 01:07.541 we're letting the system analyze the image and describe how it should behave. 01:07.625 --> 01:09.708 Now we feed everything into the Wan pipeline. 01:09.708 --> 01:12.916 We take the generated sky image, the encoded visual features 01:12.916 --> 01:16.291 from clip, and the original shot context. 01:16.333 --> 01:21.666 Then we use image to video encoding to turn that single frame into a sequence. 01:21.666 --> 01:24.708 This is where Wan 2.2 really shines. 01:24.708 --> 01:28.750 It takes that static sky and generates motion like clouds, 01:28.750 --> 01:33.500 drifting, lighting changes, and subtle atmospheric movement. 01:33.500 --> 01:37.875 We're also using Unit 3 C control net here. 01:37.875 --> 01:40.875 This helps anchor the generation to the original structure 01:40.875 --> 01:44.541 of the shot so we don't lose composition or perspective. 01:44.541 --> 01:48.375 The result is a dynamic sky replacement that feels integrated 01:48.375 --> 01:51.750 into the shot rather than composited on top. 01:52.041 --> 01:54.833 A few things you may want to adjust depending on your footage. 01:54.833 --> 01:59.500 The initial Sky generation prompt will have a big impact on the final look. 01:59.583 --> 02:04.458 can push this towards sunset, storm, overcast, or even stylized environments. 02:04.541 --> 02:08.500 may also want to tweak resolution and frame count 02:08.541 --> 02:11.041 depending on your shot length and performance needs, 02:11.041 --> 02:14.708 and if the motion feels too subtle or to aggressive. 02:14.791 --> 02:18.375 Adjusting the prompt or sampling settings can help dial that in. 02:18.375 --> 02:22.416 This workflow also includes an app mode, so if you're not comfortable 02:22.416 --> 02:26.500 working inside a full node graph, you can still use this setup 02:26.500 --> 02:28.208 in a much simpler way. 02:28.208 --> 02:33.708 You just upload your footage, generate your sky reference, and click generate. 02:33.791 --> 02:36.875 This makes it much more approachable while still giving you 02:36.875 --> 02:39.916 access to the full workflow when you need it. 02:39.958 --> 02:43.000 This is a great example of combining traditional 02:43.000 --> 02:46.083 visual effects thinking with modern AI tools. 02:46.083 --> 02:49.833 Instead of just replacing pixels, we're generating a whole new atmosphere 02:49.833 --> 02:53.791 and mood directly inside a workflow, and that's where this starts to become 02:53.791 --> 02:54.875 really powerful.