Camera/Scene Capture Projection Mask in material

Technical & Engine Help Unreal Engine Help
,
#1

I’m trying to create an effect where if you point a handheld camera (SceneCapture2D) at an object it reveals certain properties in the material if its in frame.

Some constraints of my current project

  • Forward Renderer
  • No Post Processing
  • Unlit material
  • Don’t need account for distance and back faces. As long as its in frame its fine.

Can anyone help me understand the math or reference any material function nodes in Unreal to achieve this effect and how I could match it to the frame of my scene capture component?

#2

First a warning : SceneCapture2D is rather expensive.

I think the easiest setup would be something like this ;

  • SceneCapture2D has an array of render only actors, you then set it to only render that array.
  • Turn off render every frame and render on movement, set tick to process after scene processing, use the render scene node to render only when the camera is in use. (otherwise your capture might be delayed by 1 tick due to potential race conditions)
  • Switch the materials on the actors out before the render scene node (and put back after) to the material you want it to render in the camera. (all of this after movement has been processed)
#3

Hey Wyvery, thanks for the quick reply! Great optimization tricks, that’ll actually really come in handy for using the camera to see hidden secrets in my environment, I didn’t realize you could choose to only render specific actors.

As for materials, I think I’m trying to do something on a pixel level instead of an actor level. I’m prototyping a system where you have a limited number of photos to document an environment. After you take all the photos I want to build a little diorama of the same environment where you can only see the portions that were in frame of a photo.

I’m trying to figure out if I save an array of transform data for each photo from my SceneCapture if Its possible to build out an opacity mask for my environment.

For now I’m only concerned about very simple contained circular/square indoor rooms. I won’t worry about object occlusion for larger sprawling rooms until later.

1 Like