Smoke Lighting and texture re-usability in Skull & Bones

particles
shaders

#1

Hi guys! Mederic here, VFX Tech Director on Skull&Bones.

A few months ago we’ve discussed clouds lighting and normal maps and a bunch of techniques like DLUT and 6D Lightmaps. Since I’ve then I’ve gotten a bunch of email regarding that so I’ll try to answer a few questions and give an overview of the technique we’ve been using on Skull & Bones. Major props and thanks to @sethhall for the original ideas!

As you all know, better looking explosions is a forever quest in our field and a popular method of making it possible in games is through usage of animated textures, aka flipbooks. There is a few problems with that…

Problem #1: Memory, Resolution and Variety
Using an animated texture mean we’re seeing 1/64th of the resolution at any point (for an 8x8 frames texture) . This is a sacrifice we’re used to make by now and it leaves us with a 256x256 pixel reso for what is a 64 frames, 2k x 2k texture. Traditionally, you need a RGBA texture for colors and alpha and a 2nd one for normals and emissive mask. So we’re two 2k textures deep already, for 1 effect. Adding more pair of texture like this to prevent always getting the same explosions over and over again will quickly run your memory budget down. So usually, we’d slash down on quality (down size texture) or quantity (less variety in explosions).

Problem #2: Lighting
With normal maps, lighting never really looks good in non-fully baked out scenarios where the Time of day is not constant, or in fully gameplay situation where you don’t control the camera view angle vs the sun direction etc. With that in mind….

Our current solution: Making textures more flexible and using lightmaps instead of Normals.
In order to get variety without exploding the texture counts. What we’ve started doing was splitting form and function. So instead of using texture for colors and normal, we’ve started to pack in our texture entirely different information. Here’s our current setup of 2 texture masks. Mask1 is uncompressed (Haven’t found a suitable compression since the data on each channels differ too greatly), mask2 is DXT1.

Part #1: Making the animated texture flexible.

Mask1 RGB: Tangent Lightmaps Top, Left, Right. (1k, 8x8 @128x128 per frame)
Mask1_A: Tangent Lightmap Bottom
12

Mask2_R: Color Key (1k, 8x8 @128x128 per frame) for us, in explosion it’s the render of the heat / emissive pseudo mix we “artistically” render in Houdini while doing
Mask2_G: Alpha
Mask2_B: Free (unused currently)
R:
3
G:
4

From there, we use 2 ramp textures. Very small ones. The first ramp is an albedo ramp and the 2nd one an emissive ramp. For an explosion it would look like this:
Albedo Ramp: DXT1, 16x16 px
5

The reason this is not a simple vector color picker in material is so we can easily edit the tone of the smoke in our explosions across the game by opening this small texture in photoshop and changing the level of gray. This way, we don’t have to edit each effect individually and we maintain uniformity of colors when using more than 1 animated texture in a given fx, so they blend better together. A very small cost to pay to keep an easy pipeline.

Emissive ramp: BC7 128x16 px
RGB: Emissive Color
6
A: Emissive Pre Multiply
7

Basically, we directly assign the albedo ramp into the albedo term, but for the emissive ramp, we sample it using: UV = float2( 1.0 + (-animatedTextureMask2.r), 0.5); So the Mask2’s animated R channel animates the balance of smoke and fire.

Result:
89

By changing the albedo or the emissive ramps we can achieve different looks re-using the expensive part (the animated textures)

Part #2: 4Direction Lightmaps.
While the best would be 6Direction, we want to save memory / performance so we only encode 4, and math out an approxiamation for front/back.

As seen above, here’s the Mask1 (Lightmaps) setup.

Mask1 RGB: Tangent Lightmaps Top, Left, Right. (1k, 8x8 @128x128 per frame)
Mask1_A: Tangent Lightmap Bottom
12

To render this we simply render lights and shadows from a Light that’s tangently positioned to the camera. You can do it all in 1 pass if you are a Houdini guru but to put it simply you can do it in 2 passes: Red Light Top + Green Light Bottom, render. Red light Right + Green Light Left, render. Combine them in photoshop in a single RGBA texture. From there, all you need is to unpack it and mix it in the shader. Here’s the shader code I am currently using:

First I pack the lighting information I am going to need in a struct:

struct LightingInfo
{
            float4 rawLightMap;
            float4 lightDir;
            float frontMap;
            float backMap;
};

Here’s how the struct is filled up:

 LightingInfo L1;             // L# Because L1 is sunlight and Ln for n number of point light you wanna support. 
 L1.rawLightMap = lightMap;
 L1.lightDir = lightDirPS;    // we have Tangent Light Direction in lightDirPS, this needs to be tangent space. 
 
 // If you find better approximations for front and back map using the 4channel we have… I am all ears.
 L1.frontMap = 0.25f * (lightMap.x+lightMap.y+lightMap.z+lightMap.w);
 L1.frontMap = pow(L1.frontMap, 0.625);
 L1.backMap =  1.0f - L1.frontMap;
 L1.backMap = saturate(0.25*(1.0-normals.x) + 0.5*(L1.backMap*L1.backMap*L1.backMap*L1.backMap));

Then we just call the following function using the struct as param.

float ComputeLightMap(LightingInfo L)
{
    float hMap = (L.lightDir.x > 0.0f) ? (L.rawLightMap.x) : (L.rawLightMap.y);   // Picks the correct horizontal side.
    float vMap = (L.lightDir.y > 0.0f) ? (L.rawLightMap.w) : (L.rawLightMap.z);   // Picks the correct Vertical side.
    float dMap = (L.lightDir.z > 0.0f) ? (L.frontMap) : (L.backMap);              // Picks the correct Front/back Pseudo Map
    float lightMap = hMap*L.lightDir.x*L.lightDir.x + vMap*L.lightDir.y*L.lightDir.y + dMap*L.lightDir.z*L.lightDir.z; // Pythagoras!
            return lightMap;
}

Currently, we decided to support the sunlight + 1 point light.

Result using the same animated texture from the explosion in the example above:

In action:
Using 1k per 1k animated texture (128px squared per frame)

Here is a few snaps of different Times of Day and different orientations of the particle:
11121413

All in all, it makes for a flexible system where we can leverage the same animated texture for a variety of applications. Best part, the lighting is still done in vertex shader. All we need is not to collapse the light color, ambient color in VS into 1 and instead pass both colors to PS individually so we can multiply the light map on the light color before adding it to the ambient color. So the lighting equation looks something like:

Lighting = lightColor*lightmap + ambientColor;

The total texture memory used for everything in this post comes to a total of 8.06MB.

I hope this helps you guys out!

Cheers,


6-way lightmap WIP
Nate Lane: VFX Work
Frost Grenade
#2

These are the most breathtaking clouds/smoke in real-time I’ve ever seen…


#3

This is SUPER helpful, thank you so much


#4

Really impressed with this process!


#5

Have you thought about using a tetrahedral light map? Something like the old Half Life 2 three direction basis light map idea plus one from behind?


#6

Thanks for sharing in so much detail. The smoke plume looks gorgeous!


#7

Have you thought about using a tetrahedral light map? Something like the old Half Life 2 three direction basis light map idea plus one from behind?

I did yes, but by then we’d already produced half the assets and I’m trying to keep changes to the pipeline as least destructive of previous work as possible unless we have very good reasons for it! :sweat_smile:


#8

Dude! Outstanding results. Thank you for sharing your process and data. This is really amazing stuff.


#9

What seth said, great work dude, crazy seeing you composite the “normal map” at run time :wink:


#10

Maderic, thank you so much for sharing the process, outstanding and efficient way to do it using a custom shader. Cheers man


#11

sir Mederic i have a few shilly question.
could you more explain how to render Mask2_R informaton.

Mask2_R: Color Key (1k, 8x8 @128x128 per frame) for us, in explosion it’s the render of the heat / emissive pseudo mix we “artistically” render in Houdini while doing

any information is really helpful to me.

sorry about my english and thank your share this technic.


#12

Could you pack the lightmaps into fewer channels?

i.e. have the Tangent Lightmaps top and bottom channels in one channel, by have the top values go from 255 to 128, and the bottom values go from 128 to 0, that way each channel would contain a dimension rather than a direction. you then store the front/back in the 3rd channel…?


#13

i.e. have the Tangent Lightmaps top and bottom channels in one channel,

I tried at first but the problems is that these are not normals, they’re shadows. And the self shadows are projected based on the density profile from a given light direction. When you flip that light direction , the profile that cast a shadow is very different than the mirrored shadow caster from the original light direction. So something can be in shadow in both opposing light direction, or can be lit from both light direction… In short, the shadow from one side doesn’t guarantee it will be lit from the flip side.

In theory, it works on a sphere, but it fails on an irregular animated puff of smoke simulation.


#14

could you more explain how to render Mask2_R informaton.

It really depends on what and how you want to color whatever it is that you are using this tech to light. For smoke and fire, we opted to assume the smoke color is constant (some gray) and we only cared for the fire part, so in houdini we rendered the explosion with instead of a orange to red to gray smoke ramp of color, to something that was black and white, and based on how much emissive the explosion would be. We tried rendering out the temperature channel too, we tried mixing it up in photoshop… But at the end of the day, just using the ramp in the houdini pyro rendering option and making a custom black and white ramp profile was the simplest and best way to do it that we found. So we went with that.


#15

i’m having a go at setting up a shader and getting awesome results, but different colors on the particles pop when their sorting order changes due to a view angle change. as the lighting make the particle difference more pronouced than normal the effect seems more noticable.

Its a common problem with particle in general,

Did you find any ways around this issue? using a sorting order based on age resolves problem, but can make particle look incorrect due to seeing distant particle draw on top of closer particles.


#16

I usually use a form of old to young, or young to old sorting order depending on the FX on hand. Also, sometimes, it can be worth it to split the emitter in 2-3 smaller ones if your particles are clusters but with some distance between them.

There is no real magic solution here. One thing though, I very rarely leave sorting up to Z sorting, it always fuck up somewhere when you don’t want it to. I find that in most cases, you either don’t wanna sort them (then its random what’s in front of what but its at least stable) or sort them in a predictable way (I’m a fan of having older particle draw on top of younger ones as they dissipate and reveal the next, makes things more natural imho for anything looping or like a column of smoke.

You can check the video in the first post. The “one texture many use cases” video is not sorted and cycles around 3 particles per cluster. The “point ligth support” smoke column one is sorted drawing older on top of younger…


#17

Thanks for the extra info…


#18

thank you i will check this infomation. again thank you