Welcome to WSL!

New to the forum? Please read this and this.

[DEV] DCTL Convolve Bokeh Fuse

Where the future is being made, today.

Welcome to the WSL development corner!

In this forum, please post your development projects. You get kudos and feedback here.
Topics ideally have preset prefixes, and this is what they (might) mean:

  • [DEV] - very much work in progress, don't build a business on this, could go anywhere
  • [BETA] - should kinda do what it's supposed to do, please test, give feedback
  • [RC] - this may end up in Reactor soon, polishing up, now's the time for last minute thoughts
  • [ABD] - died a premature death, sadness, will not see the light of day ever (unless someone picks up the scraps)

Once a development project has been released (hurray), topics can be marked as - you guessed it - [RELEASED] :cheer:

Development topics only, please. For generic questions, how-to's, questions and inquiries about existing tools etc, please go to the appropriate other forums.
User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

[DEV] DCTL Convolve Bokeh Fuse

#1

Post by tida »

Just follow up a discussion that was initiated by @Millolab in another thread viewtopic.php?p=36271#p36271.
Please find below a simple convolutional filter, where you can achieve a bokeh from a self-drawn image.

Input Channel0: Connection for Video
Input Channel1: Connection for Bokeh Image
Input Channel2: (Optional) connection for B&W depth map

I did limit the Bokeh Image to 100x100 pixels. Image is multiplied and added to the Video in a typical convolutional way. But I did set 50% grey to zero influence. Hence below 50% it will darken, above 50% it will brighten.

Up to now, you are able to influence the size and the overall brightness by sliders.
td_Bokeh.fuse
25x40_ellipse.png
40x40_pentagon.png
Bokeh.JPG
You do not have the required permissions to view the files attached to this post.

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#2

Post by Millolab »

@tida will test it asap!
Will it support shapes with alpha?

Cheers!

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#3

Post by Millolab »

@tida First of, great work! Then I have a couple of things I'd do differently. You don't want the bokeh to drive the brightness of your image, but rather the opposite. Take a look at the defocus node!I'0d get rid of the darkening/brightening due the The Bokeh image... which should look like an alpha channel image. Black and white. This 50% gray is very confusing! I'd rather add a bloom/threshold control as the Defocus node!
I'm not sure I made myself 100% clear :) Let me know if you need a better explanation!

cheers!!

Added in 9 minutes 45 seconds:
I played a little more. And I have to say that, if you implement this changes this could be a wonderful add to Fusion! :)
The depth map works really nicely!

User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

Re: [DEV] DCTL Convolve Bokeh Fuse

#4

Post by tida »

@Millolab thanks for the reply and all the fruitful suggestions.
I think for the darkening/brightening I have to evaluate the all-over luma of the bokeh image - at the moment I only had the quick manual answer with the brightening slider. I thougth the 0.5 limit could give some fun as, depending on the bokeh image you could also implement sharpening/edge detection etc. as this gives access to negative values https://en.wikipedia.org/wiki/Kernel_(image_processing) - but I would make this adjustable to zero.

In terms of anamorphic mimic you can already play with a resize node between bokeh image and this node but it might be handy to drive this by another slider.
Next upload will contain functionality for chromatic aberration as well.

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#5

Post by Millolab »

@tida generally you don't want a blur node to change the overall brightness of your image.
That's why I'm telling you to remove that feature. I'd keep this simple and on point for the task. For Anamorphic mimic a simple X/Y size it's enough.
Chromatic aberration could be a good touch!

User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

Re: [DEV] DCTL Convolve Bokeh Fuse

#6

Post by tida »

@Millolab yes, should be at first sight as practical as possible. The brightening effect comes from image pixels itself - has nothing to do, whether I set neutral at 0.0, 0.5 or 1.0. That means, if you draw another bokeh picture, you will change the brightness of your outcome as the sum-up luminance changes.

This is why I first need to normalize the bokeh image in terms of luminance when it comes in. That means I need to preprocess it with another Kernel that the calculations for that will not slow down the convolutional calculation. Due to the need of pre-processing the bokeh image I can add something like CA or Anamorphic-DeSqueeze, Rotation etc.

In terms of CA I would be happy to get some feedback. At the moment I did integrate a displacement that goes in one direction whereas I can variate the angle. On the other hand, it might be also an idea to spread the CA circular - would that make sense to look at.

Furthermore, I'm thinking about pre-processing and post-processing the video within the fuse by two additional kernels as well. @Millolab did you make some experience when you developed your FilmicBlur Macro that it might give some nice effects if you increase the contrast before and reduce contrast to same amount afterwards...
(I have seen examples with LinearToSRGB in pre, Convolving, and SRGBToLinear in post.... - increase contrast with the power of 2.2 and decrease with the power of 1/2.2 curve)

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#7

Post by Millolab »

@tida i'm not sure i follow. I'm sorry...
All i know is that i'd very much rather not have the brightness connected to the Bokeh shape in anyway.
And i would love to be able to feed B/W images i would normally do as bokeh.
Like this, for example:
Image

You might want to check this article, though.
https://docs.unrealengine.com/udk/Three ... Field.html

About filmicblur: it uses 2 different defocus nodes: one for the whole image, one only for the highlights. FilmicBlur is meant to be used in a Linear colourspace.

User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

Re: [DEV] DCTL Convolve Bokeh Fuse

#8

Post by tida »

The next trial...
It's nice to use the new shape tool to customize the bokeh. The Fuse is now set on default to use normal B&W bokeh images as requested by @Millolab.

Amplify: does pronounce Bokeh at contrasty zones.
Luma Balance: might be used if Luma does change too much compared to the original.
Neutral Grey: raise it, if you like to build up a sharpen filter
Anam Desqueeze: will change the bokeh aspect ratio. Multiply by 4/3 to calculate the aspect ratio of an anamorphic image. (1.8 * 4/3 = 2.40)
CA: Chromatic Abberation (you can change strength and angle)

You are still able to add a depth map to Channel2 (purple input).

Would be happy to get any feedback :)
td_Bokeh_V02.JPG
td_Bokeh.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#9

Post by Millolab »

@tida I tested this on a 4K footage and it was EXTREMELY slow. It was filling my GPU completely.
I had to scale down the footage to 960p to be able to see something happening. The previous version seemed to be less process intensive.
Luma Balance and Neutral Grey are like a mystery to me... Not sure I understand why they are there and what they do.
The Cromatic aberration are not really convincing...

Also I get this strange artifact on the blur:

Image

I'd honestly keep this as simple as possible.
The convolve itself is more than enough.

:)

User avatar
phoenixart
Fusioneer
Posts: 61
Joined: Fri Sep 25, 2020 3:49 pm
Answers: 1
Been thanked: 1 time
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#10

Post by phoenixart »

@tida

Thanks for this fuse, it's working great here!

The only part I'm not understanding as Millolab pointed out is the Neutral Grey slider. Other than that, results are looking good!
Image

I'll do more tests this weekend with other 3d renders and proper beauty passes.

User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

Re: [DEV] DCTL Convolve Bokeh Fuse

#11

Post by tida »

@Millolab ,@phonixart Thanks for the reply. I see the artifacts coming from the kind of blur-size integration. The good thing is, that it's easy to integrate the depth map functionality - the bad is the build-up of patterns which becomes more obvious if you use Amplify slider.
If you add a blur with a small radius you can get rid of that (Bartlett or multi-box works best without losing much detail of the bokeh). This leads to the general idea to combine it internally to accelerate the whole process and keep depth map functionality. Maybe something like @Shem Namo fast 2-pass filter, which he did post at viewtopic.php?p=36256#p36256 will be convenient.

Second, the power functions for the Amplify functionality decelerates the process currently, hence I could deactivate that as soon as you don't need to amplify. I see on my computer, that it doubles the time.

Third, I agree that a final fuse should be as simple as possible. At least chromatic aberration and other stuff should be skipped. Please note, that my intention is at the moment to learn more about DCTL processing and I did play with it, as I wanted to check how an additional kernel works out.

User avatar
phoenixart
Fusioneer
Posts: 61
Joined: Fri Sep 25, 2020 3:49 pm
Answers: 1
Been thanked: 1 time
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#12

Post by phoenixart »

tida wrote: Thu Feb 18, 2021 2:57 am [...]
Third, I agree that a final fuse should be as simple as possible. At least chromatic aberration and other stuff should be skipped. Please note, that my intention is at the moment to learn more about DCTL processing and I did play with it, as I wanted to check how an additional kernel works out.
I think I missed that part: what's the problem with chromatic aberration? I think it's nice to have it.

User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#13

Post by Millolab »

phoenixart wrote: Thu Feb 18, 2021 10:40 am I think I missed that part: what's the problem with chromatic aberration? I think it's nice to have it.
Yeah, it could be nice (you can add them after the blur though), but i honestly prefer having a simple convolution filter that runs smoothly instead! :)

User avatar
phoenixart
Fusioneer
Posts: 61
Joined: Fri Sep 25, 2020 3:49 pm
Answers: 1
Been thanked: 1 time
Contact:

Re: [DEV] DCTL Convolve Bokeh Fuse

#14

Post by phoenixart »

Millolab wrote: Thu Feb 18, 2021 11:28 am [..]
Yeah, it could be nice (you can add them after the blur though), but i honestly prefer having a simple convolution filter that runs smoothly instead! :)
I see. Now I re-read your previous post: so, the fuse is running slow on your end?
Strange, the first tests I did everything was running smooth, but I tested it only on still images (HD).
I'll post my results after I'll test it on 4k footage.

User avatar
tida
Fusionista
Posts: 257
Joined: Mon Jan 27, 2020 4:37 am
Answers: 1
Real name: none
Been thanked: 41 times

Re: [DEV] DCTL Convolve Bokeh Fuse

#15

Post by tida »

The following update might be most exciting in case you are OK with the quality. Would be great to get some feedback about it.

I reduced the fuse to only two sliders - but that's not exciting.....

Please try this: Make your bokeh image really small like 10x10 pixels. Increase Bokeh size by 10 or whatever. Even if you increase the Amplify slider you should not see any pattern formation.
The speed is not the best but for convolving it should be OK by now. It strongly depends on Bokeh input size but you can use now much smaller sizes to get similar results. In addition, if you set Amplify to zero you should see an increase in speed as mentioned power function is disabled.
td_Bokeh.fuse
I have integrated the 2-Pass Gaussian blur filter with automatic radius adjustment. This opens up a much better situation for Image computing.
  1. Example - convolve for  100x100 pixel bokeh:
  2. Before: 100x100 (convolve)                                 = 10.000 times to calculate full image in cycle ---> very expensive
  3. Now:     10x 10 (convolve) + 10x3x2 (Gaussian with 3sigma) =    160 times to calculate full image in cycle ---> much cheaper
Due to the integration, you are still able to feed the depth map to Channel2 as the Gaussian filter gets locally adjusted as well.
You do not have the required permissions to view the files attached to this post.