Amplify vertex offset with odd black outline issue

Hey there!

I’m pretty new to shader creation and I’m having an issue with a vertex offset shader. Right now I’m getting this strange black outline at certain points when my sphere is deforming.

20210630_122006_%pn_ef32d511

Any idea on how I can correct this?

4 Likes

Oh and here’s the block of nodes for the vertex offset part of the shader:

Do you use clamp on alpha? I think that could be the issue. Simply add a 0-1 clamp at the end

I’ve seen some similar issues but on fresnel and refraction. see if removing fresnel or refraction from your material fixes it if you’re using any of those.

Could it be the smoothed vertex normals of the sphere that don’t get updated from the vertex offset? Offsetting vertices that are on the backside of the sphere to the point where they are visible would leave them black if the normals are not updated with the vertex positions.

1 Like

Correct, they are not. The vertex normals are the normals that the mesh had prior to deformation. Deforming the normals with the vertex position is a non-trivial thing to do in a vertex shader as you don’t necessarily know where the other vertices are to know what the new normal should be. Some water shaders handle this by assuming the original mesh is a flat regular grid, but you can’t do that with arbitrary geometry. For a sphere like above you could calculate offset positions along the tangent and bitangent to get a decent approximation.

So yes, the bug is sometimes you can see the “back side” of the sphere such that the vertex normals being used are facing away from you. To fix this you usually need to clamp the dot product of the view direction and surface normal to avoid negative values.

1 Like

I tried this where I think you meant to try it and it didn’t change much (if anything)

I don’t use refraction, but I do use fresnel. I removed it and unfortunately it didn’t seem to help.

1 Like

It reminds me of toon outlines, so yes it very well could be this.

The short version is it’s definitely not something you can fix by modifying your vertex distortion code (apart from removing it).

The only way to easily track it down is to go node by node within the rest of the shader and see where it first shows up.

You could try multiplying the result with vertex normal

When you offset the position of the vertices some of the vertices from the back of the sphere will move in a way that lets you see them. These vertices probably have a normal vector which is going to face away from the camera, or in other words, they surface is looking away from the viewer instead of towards the viewer.

Some operations like the fresnel that you seem to be using in this material use the “dot multiplication” between the surface normal and the view vector to calculate the result. If the normal vector of the surface is looking somewhat towards the camera, which is commonly the case, the result of the dot multiplication will be a positive value. If the surface is instead looking away from the camera, which would be the case in the displaced vertices that would normally not be visible, the result will be a negative value. You are probably using this result to multiply or lerp between the green color of the fresnel effect in some way. When this result is negative, you can end with a negative RGB value that will be shown as black.

In your last screenshot you seem to be clamping the vertex offset result. To avoid the problem you should clamp the result of any fresnel or dot multiplications that you are using to calculate the color of the material, not the vertex displacement. You could post a screenshot of the nodes you are using to calculate the color to see where the clamp could do the trick.

I could be wrong but that’s where I’d look next.