Yusuf's Thoughts

Viral: Real-Time AI Repaint Reel

3 min read

Live footage continuously repainted into an oil painting, a Ghibli watercolour, or molten glass, in real time, following your every move. This is the headline AI-art trend of 2026 and a guaranteed save on Reels. This tutorial focuses on getting a clean, postable repaint loop, building on the engine setup from the main diffusion recipe.

Watch the trend

Engine setup first: Live AI Painting with TouchDiffusion. Pose-locked version: ControlNet Pose-to-Art.

On a Mac?

Local StreamDiffusion needs NVIDIA. On Apple Silicon use Hosted StreamDiffusion (Daydream) or DotSimulate’s StreamDiffusionTD in cloud mode. The reel workflow below is identical; only the engine runs remotely.

1. Pick the Two Routes

  • TouchDiffusion / StreamDiffusionTD: fastest path, one node, frame-by-frame repaint. Best for live performance.
  • ComfyUI bridge: you run a ComfyUI graph and pipe frames between it and TouchDesigner (Spout/NDI or a TCP bridge). Best when you want multiple models, LoRAs, or ControlNets stacked.

For a first viral clip, use the single-node route. Reach for ComfyUI when you want a very specific look.

2. The Repaint Loop

  1. Feed your webcam (or a clip of you two) into the diffusion node at 512x512.
  2. Prompt the style: Studio Ghibli watercolour, soft light, hand-painted or thick oil painting, golden hour.
  3. Set denoise/strength to 0.4–0.55 so your motion stays readable while the style fully takes over.
  4. Lock the seed so the look is consistent frame to frame.

3. Kill the Flicker (the pro step)

Raw real-time diffusion shimmers. Tame it:

  1. Add a slight Lag or Blur on the input so the model sees smoother frames.
  2. Add a light Feedback TOP blend (mix ~15% of the previous output) to stabilise across frames.
  3. Keep movement slow and deliberate while recording.

4. Audio-React It (optional, very viral)

  1. Bind denoise strength to a bass envelope so the painting “melts” harder on the drop, then resolves.
  2. Or swap the prompt on a beat via a DAT so the style morphs through the song.

5. Post It

  1. Output 1080x1920 vertical.
  2. A before/after reveal (real face, then painted) performs extremely well; show a second of raw webcam, then let the repaint take over.
  3. Record over trending audio and post.

Troubleshooting

  • “It flickers like crazy.” - Lower strength, add input blur and the feedback blend, slow your movement.
  • “4 FPS.” - Drop resolution and denoise steps, close other GPU apps, ensure the optimized (TensorRT) engine is active. On Mac, use the cloud path.
  • “My face turns into a monster.” - Lower strength so structure is preserved, or add a ControlNet (see the pose recipe) to lock composition.

Parameter Tuning & Behavior

ParameterBehavior
StrengthHigher = fully AI, ignores you; Lower = your footage shows through.
SeedFixed = stable style; Random = shifting, less coherent.
Feedback blendHigher = very stable but smeary; Lower = crisp but flickery.
StepsHigher = detailed but slow; Lower = fast and painterly.

Network Architecture

[ SOURCE ]    [ Video Device In TOP ] ──▶ [ Resolution 512 ] ──▶ [ slight Blur/Lag ]
                                                                       │
                                                                       ▼
[ ENGINE ]    [ TouchDiffusion / StreamDiffusionTD / ComfyUI bridge ]
              (Prompt + Strength + Seed)   ◀── optional bass env on Strength
                                                                       │
                                                                       ▼
[ STABILISE ] [ Feedback TOP ] (blend ~15% previous output)
                                                                       │
                                                                       ▼
[ OUT ]       [ Bloom / grade ] ──▶ [ OUT 1080x1920 ] ──▶ [ Movie File Out ]

(y) Return to Trending 2026 | (y) Return to Recipes & Projects | (y) Return to TouchDesigner | (y) Return to Home

Graph View

Backlinks