Motion Extraction Video

Super interesting to study as an FX artist. There’s a lot in there to just digest. Definitely a technique I’m going to play with in the future.


From what I understand, it’s pretty much the same as Andrew Benson’s old optical flow technique (pixel difference gradient from previous to current frame) - which is pretty standard for TouchDesigner etc:

  • the newer optical flow TOP uses Nvidia ML stuff, so will give you better motion as pixel diff only does silhouettes: Optical Flow TOP | Derivative

Here’s a GLSL impl: GitHub - WaltzBinaire/ofxMIOFlowGLSL: Optical Flow GPU Version - based upon Andrew Benson's solution
Unity impl: GitHub - mattatz/unity-optical-flow: A simple optical flow implementation by fragment shader for Unity.

One tip I found useful from this tutorial is that to get the best from it, you need to do it at multiple scales (e.g. do it at multiple downsample/lower mips and sum results) - otherwise you only get fine line motion across silhouette edges at the resolution you process at:

1 Like