Welcome to WSL!

Make yourself at home, but before posting, please may I ask you to read the following topics.


Posting 101
Server space, screenshots, and you

Thank you!

PS. please pretty please:


Image

[DEV] image based relight

Where the future is being made, today.

Welcome to the WSL development corner!

In this forum, please post your development projects. You get kudos and feedback here.
Topics ideally have preset prefixes, and this is what they (might) mean:

  • [DEV] - very much work in progress, don't build a business on this, could go anywhere
  • [BETA] - should kinda do what it's supposed to do, please test, give feedback
  • [RC] - this may end up in Reactor soon, polishing up, now's the time for last minute thoughts
  • [ABD] - died a premature death, sadness, will not see the light of day ever (unless someone picks up the scraps)

Once a development project has been released (hurray), topics can be marked as - you guessed it - [RELEASED] :cheer:

Development topics only, please. For generic questions, how-to's, questions and inquiries about existing tools etc, please go to the appropriate other forums.
User avatar
Millolab
Fusionista
Posts: 535
Joined: Wed Oct 24, 2018 6:26 am
Answers: 2
Been thanked: 72 times
Contact:

[DEV] image based relight

#1

Post by Millolab » Sun Jun 21, 2020 6:29 am

Here something I'm working on.
It needs some testing. Who wants to join in?
You need a NormalPass to make it work.

Basically what I'm doing is converting Normals into UVs (and somehow normalizing them) and using a Texture node.



Have a look and let me know your thoughts
ml_IBRelight.setting
You do not have the required permissions to view the files attached to this post.

User avatar
SteveWatson
Fusioneer
Posts: 121
Joined: Sat Oct 12, 2019 6:03 am
Answers: 3
Been thanked: 14 times

Re: [DEV] image based relight

#2

Post by SteveWatson » Mon Jun 22, 2020 4:24 am

Hi,

this looks interesting but I tried it in resolve studio 16.1.2 and it doesn't seem to work. Tried with a normal map from c4d split from exr multiplass and an hdri. I played around with the map scale and i got some colours in the viewer with values around 1 but nothing like in your video. The normal map is 1920x1080 and the hdri is 3072x1536 so I don't know if that makes a difference.

Image

EDIT: Just tried it on my laptop which is running resolve studio 16.2.1 and it doesn't work there either.

User avatar
Midgardsormr
Fusionator
Posts: 1817
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 15
Location: Los Angeles, CA, USA
Been thanked: 109 times
Contact:

Re: [DEV] image based relight

#3

Post by Midgardsormr » Mon Jun 22, 2020 6:58 am

Are your normals actually in the Normals channels? If they're only in RGB, then the macro won't work correctly.

That said, Millolab, perhaps you should add an option for the user to choose whether they have real normals or normals in RGB.

It looks pretty cool, and it works like I'd expect for the tests I threw at it. I recommend setting the Blur mode to either Bartlett or Multi-Box, and add a slider for the amount. Right now I think it's set to Fast Gaussian, which causes the tool to fail in Fusion 9. Also, Gaussian creates ringing when applied to HDR images:


Image


Gaussian on the left, Multi-Box on the right. Blur size turned up to emphasize the ringing effect.

User avatar
SteveWatson
Fusioneer
Posts: 121
Joined: Sat Oct 12, 2019 6:03 am
Answers: 3
Been thanked: 14 times

Re: [DEV] image based relight

#4

Post by SteveWatson » Mon Jun 22, 2020 8:04 am

Midgardsormr wrote:
Mon Jun 22, 2020 6:58 am
Are your normals actually in the Normals channels? If they're only in RGB, then the macro won't work correctly.

Hi,

you were right I was just using the colour RGB, I used a channel boolean to switch it to normals -assuming R-X normal, G-Y Normal, B- Z normal.



Though watching this through there are limitations to this method - ie reflections are stuck to the surface so moving camera doesn't show them moving across - unless the lighting texture is animated as well, but there are possibilities with this tool.

User avatar
Midgardsormr
Fusionator
Posts: 1817
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 15
Location: Los Angeles, CA, USA
Been thanked: 109 times
Contact:

Re: [DEV] image based relight

#5

Post by Midgardsormr » Mon Jun 22, 2020 9:24 am

It's going to depend on what the normals reference is. World normals should work correctly, but object normals will 'stick,' and camera normals will cause the lighting set up to be apparently parented under the camera, so a moving camera will cause the reflections/lights to slide over the geo incorrectly.

User avatar
Millolab
Fusionista
Posts: 535
Joined: Wed Oct 24, 2018 6:26 am
Answers: 2
Been thanked: 72 times
Contact:

Re: [DEV] image based relight

#6

Post by Millolab » Tue Jun 23, 2020 10:50 am

Midgardsormr wrote:
Mon Jun 22, 2020 6:58 am
Are your normals actually in the Normals channels? If they're only in RGB, then the macro won't work correctly.

That said, Millolab, perhaps you should add an option for the user to choose whether they have real normals or normals in RGB.

It looks pretty cool, and it works like I'd expect for the tests I threw at it. I recommend setting the Blur mode to either Bartlett or Multi-Box, and add a slider for the amount. Right now I think it's set to Fast Gaussian, which causes the tool to fail in Fusion 9. Also, Gaussian creates ringing when applied to HDR images:


Image


Gaussian on the left, Multi-Box on the right. Blur size turned up to emphasize the ringing effect.
Thanks! Will add this!!
Do you know if there's any way to convert object normals to world normals?

User avatar
Midgardsormr
Fusionator
Posts: 1817
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 15
Location: Los Angeles, CA, USA
Been thanked: 109 times
Contact:

Re: [DEV] image based relight

#7

Post by Midgardsormr » Tue Jun 23, 2020 11:20 am

Theoretically, yes, I'm sure there's a way. My math-fu is not good enough to tell you what it is, though. I think it would require knowing the object's transformations, though, which is not generally something that you get from a render.

User avatar
Millolab
Fusionista
Posts: 535
Joined: Wed Oct 24, 2018 6:26 am
Answers: 2
Been thanked: 72 times
Contact:

Re: [DEV] image based relight

#8

Post by Millolab » Tue Jun 23, 2020 1:48 pm

Midgardsormr wrote:
Tue Jun 23, 2020 11:20 am
Theoretically, yes, I'm sure there's a way. My math-fu is not good enough to tell you what it is, though. I think it would require knowing the object's transformations, though, which is not generally something that you get from a render.
My math is not good enough either. :)
Thanks Bryan!

This guy made a more complex version of the tool I'm working on, but I find it to be a bit too complicated.
Very interesting though
http://hiroshisaito.net/blog/2020/03/hs ... tools.html

User avatar
James 2020
Fusioneer
Posts: 51
Joined: Sat Feb 08, 2020 11:01 pm
Been thanked: 3 times

Re: [DEV] image based relight

#9

Post by James 2020 » Sun Jun 28, 2020 1:51 am

This tool relight is very helpful for CG composition

User avatar
Kristof
Fusionista
Posts: 778
Joined: Thu Aug 07, 2014 1:30 pm
Answers: 2
Been thanked: 23 times

Re: [DEV] image based relight

#10

Post by Kristof » Sun Jun 28, 2020 2:16 am

For the math, use nodes that apply matrix math. Vonk toolset I have linked to in the past for instance. You need object-to-world for each object in view though. Then world-to-camera at least. And projection matrix, while you're at it. This will open up even more possibilities.

I don't agree with the name of your tool;you're doing a texture lookup (going by your stills shared in here) which is not the same as a relighting process. Or am I missing something?

You can build the matrices from an alembic import for instance, if you wanted to explore this even more but are unable to export that data from your rendering application.

User avatar
Millolab
Fusionista
Posts: 535
Joined: Wed Oct 24, 2018 6:26 am
Answers: 2
Been thanked: 72 times
Contact:

Re: [DEV] image based relight

#11

Post by Millolab » Mon Jun 29, 2020 7:18 am

Kristof wrote:
Sun Jun 28, 2020 2:16 am
I don't agree with the name of your tool;you're doing a texture lookup (going by your stills shared in here) which is not the same as a relighting process. Or am I missing something?
You're totally right! :)
It just sounded fancy! All I'm doing is converting Normals into UVs to simulate this kind of environment reflection/relight.

About matrices I don't think I'd be able to use them. I really have zero knowledge about them.
But I'd be glad to have a look at Vonk toolset. Where could I find it?

User avatar
Kristof
Fusionista
Posts: 778
Joined: Thu Aug 07, 2014 1:30 pm
Answers: 2
Been thanked: 23 times

Re: [DEV] image based relight

#12

Post by Kristof » Tue Jun 30, 2020 12:13 am

Hehe, I see. There is of course the default Shader node that does this too, but regardless, it's always nice and inspiring to build things yourself so I don't want to spoil the fun.

This is the link: https://github.com/Spicy-Acorn