Super interesting to study as an FX artist. There’s a lot in there to just digest. Definitely a technique I’m going to play with in the future.
2 Likes
From what I understand, it’s pretty much the same as Andrew Benson’s old optical flow technique (pixel difference gradient from previous to current frame) - which is pretty standard for TouchDesigner etc:
- the newer optical flow TOP uses Nvidia ML stuff, so will give you better motion as pixel diff only does silhouettes: Optical Flow TOP | Derivative
Here’s a GLSL impl: GitHub - WaltzBinaire/ofxMIOFlowGLSL: Optical Flow GPU Version - based upon Andrew Benson's solution
Unity impl: GitHub - mattatz/unity-optical-flow: A simple optical flow implementation by fragment shader for Unity.
One tip I found useful from this tutorial is that to get the best from it, you need to do it at multiple scales (e.g. do it at multiple downsample/lower mips and sum results) - otherwise you only get fine line motion across silhouette edges at the resolution you process at: https://www.youtube.com/watch?v=XVh_eHL_7XU&ab_channel=DavidBraun
1 Like