Quick KHR_materials_dispersion implementation by MiiBond · Pull Request #14501 · BabylonJS/Babylon.js

MiiBond · 2023-11-08T06:07:15Z

Here's a go at implementing dispersion in the renderer as well as support for the glTF KHR_materials_dispersion extension.

I'm not completely satisfied with the implementation in the shader. It's a bit ugly but creating another function was the only way I could see to avoid duplicating lots of code.
To mimic the dispersion effect, I compute the refraction coords 3 times with slightly different IOR's and then sample the environment refraction with each of these. The final refraction is a combination of R from the first, G from the second and B from the last.

bjsplat · 2023-11-08T06:08:47Z

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

bjsplat · 2023-11-08T06:24:10Z

Snapshot stored with reference name:
refs/pull/14501/merge

Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/14501/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/14501/merge
https://gui.babylonjs.com/?snapshot=refs/pull/14501/merge
https://nme.babylonjs.com/?snapshot=refs/pull/14501/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/14501/merge#BCU1XR#0

bjsplat · 2023-11-08T06:41:21Z

Visualization tests for webgl1 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl1/index.html

If tests were successful afterwards, this report might not be available anymore.

bjsplat · 2023-11-08T06:46:42Z

Visualization tests for webgl2 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl2/index.html

If tests were successful afterwards, this report might not be available anymore.

RaananW · 2023-11-08T11:28:24Z

Wanted to comment about the failed vis tests - it seems like this PR breaks the node material PBR playground. You can see it here (using the build snapshot of this PR)

https://playground.babylonjs.com/?snapshot=refs/pull/14501/merge#D8AK3Z#17

bjsplat · 2023-11-08T17:51:17Z

Visualization tests for webgl1 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl1/index.html

If tests were successful afterwards, this report might not be available anymore.

bjsplat · 2023-11-08T17:55:30Z

Visualization tests for webgl2 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl2/index.html

If tests were successful afterwards, this report might not be available anymore.

RaananW · 2023-11-09T10:10:48Z

Seems to be now related to a shader compilation issue. Some object is incorrectly deserialized:

This can be seen in the same example I pasted in the last message.

Popov72 · 2023-11-09T16:09:19Z

Waiting for the CI to run without errors before reviewing!

bghgary

I only reviewed the glTF parts. I'll leave the shader parts to @Popov72.

…persion.ts Co-authored-by: Gary Hsu <bghgary@users.noreply.github.com>

…_dispersion.ts Co-authored-by: Gary Hsu <bghgary@users.noreply.github.com>

bjsplat · 2023-11-09T19:57:42Z

Visualization tests for webgl1 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl1/index.html

If tests were successful afterwards, this report might not be available anymore.

bjsplat · 2023-11-09T20:02:04Z

Visualization tests for webgl2 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl2/index.html

If tests were successful afterwards, this report might not be available anymore.

bjsplat · 2023-11-09T23:16:35Z

Visualization tests for webgl1 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl1/index.html

If tests were successful afterwards, this report might not be available anymore.

bjsplat · 2023-11-09T23:21:42Z

Visualization tests for webgl2 have failed. If some tests failed because the snapshots do not match, the report can be found at

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge/testResults/webgl2/index.html

If tests were successful afterwards, this report might not be available anymore.

emackey · 2023-11-10T19:40:36Z

I have a draft sample model in the works: KhronosGroup/glTF#2340 (comment)

sebavan · 2023-11-10T21:56:24Z

Can not wait this is way too cool to see all of you here :-)

MiiBond · 2023-11-11T01:13:42Z

Sorry, I'm just getting back to this. Is there a way that I can run the failing tests locally? Or see some better output?
I don't see "Visualization" test listed as a debug target in VSCode.

RaananW · 2023-11-13T11:00:50Z

Sorry, I'm just getting back to this. Is there a way that I can run the failing tests locally? Or see some better output? I don't see "Visualization" test listed as a debug target in VSCode.

yes, you can run visualization tests locally or against the snapshot of this PR.

If you want to run locally, first run npm run start to start the dev server. Afterwards, run npm run test:visualization -- -i "webgl2" -t "name of test you want to run" . This will run the visualization test locally and will display any console error in your console. You can also try running it with the new playwright ui (still experimental) using npm run test:playwright -w @tools/tests -- --project=webgl2 --ui

If you want to test against the pull request (no need for the local server) you can change the environment variable CDN_BASE_URL to https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14501/merge. But you have have the dev enrionment running it is simpler to use the dev server, because then you can make changes to the code and run the test again.

Popov72

Looking good!

deltakosh · 2023-12-05T17:51:21Z

Can we merge that one?

deltakosh · 2023-12-05T17:51:38Z

@bghgary ? Or else should we move it to draft

sebavan

one only tiny comment but great job on the PR @MiiBond !!!!

bghgary

LGTM for the glTF parts

into mbond/dispersion

echadwick-artist · 2024-01-25T00:10:09Z

Sample asset of the Khronie Award, suggested by @emackey. I couldn't get it to drag-n-drop into the Babylon.js demo above, guess I need to use a Playground?

Anyhow here's the asset so far, screenshot from the main Sandbox.

Also in Dassault Systemes' PBR Sample Renderer:

  "materials": [
    {
      "name": "KhronieAward_Clear_Glass",
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1.0
        },
        "KHR_materials_volume": {
          "attenuationColor": [
            1.0,
            1.0,
            1.0
          ],
          "thicknessFactor": 0.01
        },
        "KHR_materials_dispersion": {
          "dispersion": 0.33
        }
      },
      "pbrMetallicRoughness": {
        "metallicFactor": 0.0,
        "roughnessFactor": 0.0
      }
    },
    {
      "name": "KhronieAward_Etched_Text",
      "alphaMode": "BLEND",
      "pbrMetallicRoughness": {
        "baseColorTexture": {
          "index": 0
        },
        "baseColorFactor": [
          1,
          1,
          1,
          0.4
        ],
        "metallicFactor": 0.0,
        "roughnessFactor": 0.7
      },
      "normalTexture": {
        "index": 1,
        "scale": 1
      }
    },
    {
      "name": "KhronieAward_Rubber_Feet",
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1.0
        },
        "KHR_materials_volume": {
          "attenuationColor": [
            1.0,
            1.0,
            1.0
          ],
          "thicknessFactor": 0.01
        }
      },
      "pbrMetallicRoughness": {
        "metallicFactor": 0.0,
        "roughnessFactor": 0.4
      }
    },
    {
      "name": "KhronieAward_Smoked_Glass",
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1.0
        },
        "KHR_materials_volume": {
          "attenuationColor": [
            1.0,
            1.0,
            1.0
          ],
          "thicknessFactor": 0.01
        },
        "KHR_materials_dispersion": {
          "dispersion": 0.33
        }
      },
      "pbrMetallicRoughness": {
        "baseColorFactor": [
          0.3,
          0.3,
          0.3019610047340393,
          1.0
        ],
        "metallicFactor": 0.0,
        "roughnessFactor": 0.0
      }
    }
  ],

KhronieAward.zip

echadwick-artist · 2024-01-25T19:22:38Z

Improved the asset a little.

Front-facing text is now alphaMode:Mask without Transmission, so rasterizers can render it behind the transmissive glass. Babylon.js can render the Blend material behind a Tranmission material, but Three.js and Filament will only do this if Alpha is set to Mask.
Rear-facing text is now alphaMode:Blend with Transmission, to prevent rasterizers from rendering a duplicate within the transmission of the glass.

Also here's a V-Ray render with the corresponding Abbe value:

KhronieAward_dispersion.zip

echadwick-artist · 2024-01-28T17:58:27Z

I've adjusted the materials for the text a bit more, so the rear-facing text is a better match to the look of the front-facing text.
KhronieAward-glTF.zip

I'm also able to see dispersion rendering in the Sandbox. However it only seems to appear when I disable "Use Thickness as Depth". This asset is modeled to scale, and the glass is 1cm thick, so shouldn't we be seeing more dispersion here with it enabled?

materials": [
    {
      "name": "KhronieAward_Clear_Glass",
      "extensions": {
        "KHR_materials_transmission": {
          "transmissionFactor": 1.0
        },
        "KHR_materials_volume": {
          "thicknessFactor": 0.01
        },
        "KHR_materials_dispersion": {
          "dispersion": 0.33
        }
      },
      "pbrMetallicRoughness": {
        "metallicFactor": 0.0,
        "roughnessFactor": 0.0
      }
    },

Use Thickness as Depth = Enabled

Use Thickness as Depth = Disabled

Photo of the real object.

echadwick-artist · 2024-01-29T21:40:49Z

FYI, submitted the KhronieAward asset to the glTF Sample Assets repo: KhronosGroup/glTF-Sample-Assets#99

Popov72 · 2024-01-31T15:52:49Z

When the "use thickness as depth" option is ticked, the effect is not visible because the thickness used is too low: this is the "Max thickness value", which is 0.01 by default. If you increase this value/increase the intensity of dispersion, you should start to see something.

emackey · 2024-01-31T16:27:59Z

@Popov72 Thanks. Actually we had some interesting discussion of this effect on this week's PBR call, and I've investigated a little bit further since then. My current thoughts are:

The thickness of 0.01 (meters) represents a real-world thickness of 1 cm, which is about as thick as the glass really is. The wording is etched into the back side of the glass, so when viewing it from the front, the words sit at exactly the thickness depth, no more and no less. Turning on "use thickness as depth" makes the words (but not the background) behave almost exactly as the real thing, because the assumption of depth is correct for the words alone. The background, however, is much further away than 1cm, and makes the glass look too thin and incorporeal.
Turning off "use thickness as depth" improves the background a lot! The glass distorts the background substantially, and looks like the real object. But the words "swim around" and get clipped and so on. They're too deep now.

But now the bad news: There's no fix for only 1 opaque pass. Consider the following screenshot with "use thickness as depth" turned off:

The background isn't too exciting, but I want to call out a particular tan spot on the ground, behind the swirly letter O in the middle of the Khronos logo. This same tan spot on the ground can be seen in 3 different places: You can see it through the main piece of glass, you can see it again looking through the top bevel that angles your line of sight down to the spot behind the glass, and you can see the same tan spot a third time by looking through the left-edge bevel in the glass which angles your line of sight to the right and finds the same spot behind the glass.

So there are 3 lines of sight to the same background spot. But the single opaque-pass render has plastered the swirly letter O on top of the one spot, so now that logo is visible in all three locations.

Long story short, I don't think there's any easy fix for this using only a single opaque pass. But it's interesting to study where the limitations are for this type of rendering (and someday this may become moot when realtime raytracing is common).

MiiBond added 2 commits November 2, 2023 16:51

WIP dispersion support

a258713

Finish initial test of dispersion in the shader

716efe0

Make dispersion input optional

e94c5b7

sebavan requested review from Popov72 and bghgary November 9, 2023 15:34

bghgary reviewed Nov 9, 2023

View reviewed changes

MiiBond and others added 2 commits November 9, 2023 11:21

Update packages/dev/loaders/src/glTF/2.0/Extensions/KHR_materials_dis…

077d4ff

…persion.ts Co-authored-by: Gary Hsu <bghgary@users.noreply.github.com>

Update packages/dev/serializers/src/glTF/2.0/Extensions/KHR_materials…

adcbbf9

…_dispersion.ts Co-authored-by: Gary Hsu <bghgary@users.noreply.github.com>

Remove unneeded instance checks from KHR_materials_dispersion

e1b764e

Popov72 requested changes Nov 13, 2023

View reviewed changes

Comment thread packages/dev/core/src/Materials/Node/Blocks/PBR/subSurfaceBlock.ts

Comment thread packages/dev/core/src/Shaders/ShadersInclude/pbrBlockSubSurface.fx

Check for undefined dispersion prop

82b6ffc

Popov72 reviewed Nov 14, 2023

View reviewed changes

Comment thread packages/dev/core/src/Shaders/ShadersInclude/pbrBlockSubSurface.fx Outdated

Correction for dispersion ior computation

1589230

Popov72 approved these changes Dec 4, 2023

View reviewed changes

sebavan requested a review from bghgary December 5, 2023 05:56

sebavan reviewed Dec 6, 2023

View reviewed changes

Comment thread packages/dev/core/src/Materials/Node/Blocks/PBR/subSurfaceBlock.ts Outdated

bghgary approved these changes Dec 6, 2023

View reviewed changes

MiiBond added 2 commits December 5, 2023 21:58

Minor fix for dispersion define

89b1e38

Merge branch 'mbond/dispersion' of https://github.com/MiiBond/Babylon.js

bdc53e4

into mbond/dispersion

RaananW merged commit 3f1ccc1 into BabylonJS:master Dec 6, 2023

MiiBond mentioned this pull request Dec 8, 2023

KHR_materials_dispersion KhronosGroup/glTF#2340

Merged

Conversation

MiiBond commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

RaananW commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

bjsplat commented Nov 8, 2023

Uh oh!

RaananW commented Nov 9, 2023

Uh oh!

Popov72 commented Nov 9, 2023

Uh oh!

bghgary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bjsplat commented Nov 9, 2023

Uh oh!

bjsplat commented Nov 9, 2023

Uh oh!

bjsplat commented Nov 9, 2023

Uh oh!

bjsplat commented Nov 9, 2023

Uh oh!

emackey commented Nov 10, 2023

Uh oh!

sebavan commented Nov 10, 2023

Uh oh!

MiiBond commented Nov 11, 2023

Uh oh!

RaananW commented Nov 13, 2023

Uh oh!

Popov72 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

deltakosh commented Dec 5, 2023

Uh oh!

deltakosh commented Dec 5, 2023

Uh oh!

sebavan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bghgary left a comment

Choose a reason for hiding this comment

Uh oh!

echadwick-artist commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echadwick-artist commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echadwick-artist commented Jan 28, 2024

Uh oh!

echadwick-artist commented Jan 29, 2024

Uh oh!

Popov72 commented Jan 31, 2024

Uh oh!

emackey commented Jan 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

echadwick-artist commented Jan 25, 2024 •

edited

Loading

echadwick-artist commented Jan 25, 2024 •

edited

Loading