This right here.
That was it.
Add Lifetime to Custom Vertex Streams and add it to the panner node.
This will sadly slow down the panner. You will have to manually create a panner and add adjustable scrolling speed to balance out the slow down. Don’t know if thats the ideal way but it works.