Omni-Extractor: Decoupling Signal from Structure to Save 80% VRAM and Fix Spectral Artifacts

[ninpo_3d]

Hi everyone! I’ve been part of this community for a long time as a silent observer, but today I’m excited to share a project I’ve been developing to solve some fundamental issues with texture data. It started as a personal struggle with energy loss in VFX masks, but evolved into a universal engine-agnostic protocol for data decoupling and reconstruction.

The Genesis: Fighting the Rec.709 Legacy and the “Blue Vein”

It started with a frustrating technical artifact in a fire VFX. Every time I converted a high-quality flame asset into a standard grayscale mask, the “energy” vanished. Standard color-to-grayscale conversion relies on the legacy Rec.709 formula (0.21R + 0.72G + 0.07B), designed for 1950s television luminosity perception, not modern VFX.

This formula treats bright blue and red channels as dark areas, creating a “dirty” look and a “blue vein” where the peak energetic part of the plasma should be. I realized that we are using CRT-era math to drive modern PBR and AI pipelines.

Source Asset (Blue Flame)	Standard Grayscale Loss	State Atlas (Ramp)

The Core Concept: Data Disentanglement (Topology vs. State Atlas)

Instead of merging all data into one lossy channel, Omni-Extractor decouples an image into two independent, high-precision layers:

• Structural Topology: A 16-bit map (I;16) that stores spatial logic as a “presence index”. With 65,536 steps of precision, it acts as a high-fidelity coordinate map for signal reconstruction.
• State Atlas: A 1D-palette (Ramp) extracted via K-Means clustering. It holds the optimized “states” of the asset—whether it’s color, PBR properties, or vectors.

Balancing between extreme compression and near-lossless visual fidelity.

Universal Applicability: Engine and Platform Agnostic

The technology itself is entirely engine-agnostic. The reconstruction logic is incredibly simple and can be implemented in any shader language:

// High-precision reconstruction logic
float latentCoord = TopologyMap.Sample(MapSampler, UV).r;
float3 finalSignal = StateAtlas.Sample(AtlasSampler, float2(latentCoord, 0.5)).rgb;
return pow(finalSignal, 2.2); // Gamma correction

• Hardware-Native Reconstruction: The process leverages the GPU’s Texture Management Unit (TMU). By using the 16-bit topology as a UV coordinate, the hardware performs native bilinear interpolation between atlas centroids, ensuring infinite smoothness.
• Deployment Anywhere: Ready for PC, Consoles, Mobile, and VR/AR hardware.

Beyond Visuals: Normal Maps and PBR Encoding

We can encode Normal maps and other physical parameters into this protocol. This allows us to “re-skin” an object’s surface properties as easily as swapping a State Atlas.

Original Normal Map	State Atlas (Ramp)	16-bit Topology Map

The Efficiency Imperative: VRAM Liberation

• VRAM Optimization: Transitioning from 32-bit RGBA textures to decoupled 16-bit structural maps can reduce texture memory footprints by 70-80%.
• Hybrid Export: The pipeline enforces 16-bit precision for masks while keeping atlases in 8-bit RGB, optimizing both bandwidth and storage.

Technical Constraints & Transparency

To maintain absolute 16-bit integrity of the topology, this protocol does not store Alpha data within the primary map. Transparency should be handled as a separate stream or derived mathematically in the shader.

Open Source

The project is now open-source. You can grab the Example_Workflow.json and start experimenting with this new paradigm of information preservation.

GitHub Link: https://github.com/ninpo3d/ComfyUI_omni_extractor

[Niels]

Guys, do not install comfyui unless you have express permission from IT
(or if it’s your own pc, at your own risk), it’s an extremely vulnerable program

[ninpo_3d]

I completely agree. ComfyUI is a powerful framework that essentially executes arbitrary Python code, so installing it in a corporate environment without explicit IT approval is a major red flag.
For those concerned about security: use an isolated environment. Running it inside a virtual environment (venv/conda), a Docker container, or a dedicated sandbox machine is the gold standard for mitigating these risks. In this case, caution isn’t just panic — it’s basic engineering hygiene

[Niels]

buddy, we make stuff go boom, do you think we know basic engineering hygiene?

[ninpo_3d]

Haha, fair point! I guess I got a bit too lost in my ‘latent spaces’ and forgot that at the end of the day, we just want things to look cool and go boom.
Thanks for the reality check! I’ve completely overhauled the post to be more ‘VFX-friendly’—swapped the academic jargon for a State Atlas approach and added a simple HLSL snippet. Engineering hygiene might be boring, but this 16-bit precision actually makes those ‘booms’ look much cleaner without the banding artifacts. Hope it makes more sense now!

[Luos_83]

Hey @Niels, I kinda feel like this is another one of those “user hasn’t been online for a long time, suddenly starts posting AI-generated (semi-nonsense) texts” situations.
I think by now, we need to forward this happening to @Keith , because it’s worrisome.
I honestly think they are scraping RTVFX.com for AI reasons. (and possibly then posting here to see if what its gibberishing makes sense)

Besides it being atrocious to read AI-only text, I honestly have my doubts about what has been posted. In theory, it works, but --besides it feeling like 90% ai AI-generated suggestions taken as gospel-- in reality it’s pretty pointless.

Not to mention that dumb AI catchphrases like “CRT era math” make no sense.

IIRC Simon already researched this matter.
To utilize this, you’d have to disable mipmaps, texture compression, and filtering, which causes a plethora of issues. (especially for normal maps)

Since compression ratios are either 8:1 or 4:1, depending on the presence of an alpha channel. Meaning if you use only 1-3 channels, you are using 8x as much memory. If you actually use all 4 channels, you only reach parity with the compressed image before factoring in the extra texture sample and memory for the gradient.

Then there is that 80% vram/disk reduction claim.
Even if you magically manage to make every RGB texture a grayscale, I’m pretty sure that, at least in Unreal, there is no compressed grayscale format. So that would only be true if you go from uncompressed RGBA to uncompressed grayscale.
But who even uses uncompressed (32-bit) textures systematically anyway?
I doubt you’re actually getting even close to that 80% reduction in disk space usage.

I’m also wondering if this is related to a trick called “Basic Vectors” where the ML kernel is decoding i.e. 5(x) channels back into 12(y) colors.

Ben Clowards & Deathrey’s research paper “Palettes, grayscales, and HLSL assembly” also hints that what has been posted should be taken with a grain of salt.

Since ninpo doesn’t post any evidence of the claims, not even a proper DXT1/3/5 BC1-5 comparison, or how to set up shaders/textures properly, I have absolute doubt this is anything but semi-dreamt-up VFX-related-scraped partial-mumbo-jumbo.

[ninpo_3d]

Subject: Clarification on Omni-Extractor (Technical Specs & Evidence)

Hi everyone, thanks for the feedback. To clarify a few points:

1. AI Usage: Yes, I use LLMs to help structure my posts and translate technical thoughts as English is not my native language. However, the logic, HLSL, and the extraction workflow are my own R&D.
2. VRAM/Disk Claims: Let’s look at the numbers for a 1024x1024 texture.

• Original (BC7): ~1.33 MB.
• Omni-Extractor (BC1 Mask + 1D Atlas): ~684 KB.
• This is a ~50% reduction using “standard” settings. Moving from uncompressed 16-bit or high-precision formats to this protocol is where the 80% figure comes from.

3. Filtering & Mipmaps: You do not need to disable them. By using a 16-bit Topology Map (R16/G16 or even BC4), we maintain sub-pixel precision. In fact, mip-mapping on a coordinate map often produces cleaner results than color-averaging in BC7, which tends to “muddy” high-energy VFX colors.
4. Coherent Signal (V2): I am currently finalizing an update that implements Greedy Sorting for the Atlas. This ensures that neighboring indices in the Topology Map point to neighboring colors in the Atlas, making the signal temporally stable and resistant to compression artifacts.
5. Format Support: Unreal Engine handles BC4 and G16/R16 perfectly. The “CRT-era math” comment refers to fixed-palette indexing logic, which I believe is still highly relevant for VRAM-constrained modern pipelines.

I will post a side-by-side DXT1/BC1 vs BC7 comparison and a video showing temporal stability shortly.

Here on the left is a reconstructed image, on the right is the original

[raytheonly]

This still doesn’t really make much sense why someone would use that in production (for everything).
Also I don’t understand why the claim of helping with an VFX specific issue but not showing comparisons in that field. Maybe it would help to know todays date and know what you would prefer. Can you provide that please?

I think Luos is on to something.

[ninpo_3d]

This is a very good optimization for textures, but here’s what I didn’t take into account right now: it’s really not suitable for effects at close range, although at medium and distant distances details are even more visible. It also looks interesting for stylized effects.