Welcome to WSL!

Make yourself at home, but before posting, please may I ask you to read the following topics.


Posting 101
Server space, screenshots, and you

Thank you!

PS. please pretty please:


Image

What algorithm does the Chroma Keyer Node use?

User avatar
Shem Namo
Fusionista
Posts: 592
Joined: Sun Oct 06, 2019 9:15 pm
Location: North Israel
Real name: David Kohen
Been thanked: 35 times

What algorithm does the Chroma Keyer Node use?

#1

Post by Shem Namo » Sun Aug 30, 2020 3:36 am

Hi everyone, hope you're all doing great!!

I'm working on a little project with @Millolab,
and am trying to get the same results of the chroma keyer node in a fuse.

I can't seem to find much information about anything like this online.
Does anyone know how the color ranges algorithm work in the chroma keyer?

Thanks in advanced,
David

User avatar
danell
Fusioneer
Posts: 161
Joined: Mon Dec 12, 2016 6:32 am
Been thanked: 32 times

Re: What algorithm does the Chroma Keyer Node use?

#2

Post by danell » Sun Aug 30, 2020 12:02 pm

Convert from RGB to HLS, then Luma key the Hue channel with your wanted range to get your matte

User avatar
Millolab
Fusionista
Posts: 615
Joined: Wed Oct 24, 2018 6:26 am
Answers: 3
Been thanked: 93 times
Contact:

Re: What algorithm does the Chroma Keyer Node use?

#3

Post by Millolab » Sun Aug 30, 2020 12:44 pm

I was trying to do that in a CustomTool. Luma key is easy. I could not find an easy solution for the “soft range”. That’s why I went for the chroma key.

User avatar
Shem Namo
Fusionista
Posts: 592
Joined: Sun Oct 06, 2019 9:15 pm
Location: North Israel
Real name: David Kohen
Been thanked: 35 times

Re: What algorithm does the Chroma Keyer Node use?

#4

Post by Shem Namo » Mon Aug 31, 2020 9:33 am

danell wrote:
Sun Aug 30, 2020 12:02 pm
Convert from RGB to HLS, then Luma key the Hue channel with your wanted range to get your matte
Thanks Danell, but I'm trying to figure out how the color ranges work more than the actual keying process.
Up until recently I thought it was
  1. if channel > low and channel < high then channel = 0.0 end
But after testing that against the chroma keyer, I can see that is not the case.

Does anyone happen to know how the ranges work in the chroma keyer?
Millolab wrote:
Sun Aug 30, 2020 12:44 pm
I was trying to do that in a CustomTool. Luma key is easy. I could not find an easy solution for the “soft range”. That’s why I went for the chroma key.
Could you please share what you made in the CustomTool?
Maybe there is a way to smooth out the ranges.

Thanks again,
David

User avatar
Chad
Fusionator
Posts: 1502
Joined: Fri Aug 08, 2014 1:11 pm
Been thanked: 12 times

Re: What algorithm does the Chroma Keyer Node use?

#5

Post by Chad » Mon Aug 31, 2020 10:47 am

Your intuition is correct. You can verify this with a 3D Histogram and LUT cube image. The Soft Range is a linear falloff.

One tricky thing is the ranges are named "Red" "Green" and "Blue" even though the default space is Chroma, not Color.

User avatar
Midgardsormr
Fusionator
Posts: 1913
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 18
Location: Los Angeles, CA, USA
Been thanked: 144 times
Contact:

Re: What algorithm does the Chroma Keyer Node use?

#6

Post by Midgardsormr » Mon Aug 31, 2020 10:58 am

I think it's just a linear interpolation between A = 0 at the threshold level and A = 1 at threshold +/- soft range.

For instance, here I have a simple horizontal ramp from 0 - 1 in the Red channel. I've set the Red High to 0.5 and Soft Range to 0.1. Every pixel to the left of R = 0.5 has an Alpha of 0. At R = 6.0, alpha is 1.0, and at R = 0.55, alpha = 0.5.

Code: Select all

{
	Tools = ordered() {
		Background2 = Background {
			Inputs = {
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				Type = Input { Value = FuID { "Horizontal" }, },
				TopRightRed = Input { Value = 1, },
			},
			ViewInfo = OperatorInfo { Pos = { 173, 139 } },
		},
		ChromaKeyer1 = ChromaKeyer {
			CtrlWZoom = false,
			Inputs = {
				KeyType = Input { Value = 1, },
				ColorRange = Input { Value = 1, },
				RedHigh = Input { Value = 0.5, },
				SoftRange = Input { Value = 0.1, },
				FringeSize = Input { Value = 0, },
				Input = Input {
					SourceOp = "Background2",
					Source = "Output",
				},
			},
			ViewInfo = OperatorInfo { Pos = { 412, 139.5 } },
		}
	}
}

I did this test in the Color mode because I'm not sure exactly how the relationship between the RGB channels and chroma works.

edit: Chad got here first, but I'm leaving it.

User avatar
Shem Namo
Fusionista
Posts: 592
Joined: Sun Oct 06, 2019 9:15 pm
Location: North Israel
Real name: David Kohen
Been thanked: 35 times

Re: What algorithm does the Chroma Keyer Node use?

#7

Post by Shem Namo » Mon Aug 31, 2020 11:05 am

Thanks for confirming @Chad!!
I didn't even think of the default space.
Chad wrote:
Mon Aug 31, 2020 10:47 am
The Soft Range is a linear falloff.
A linear falloff between the clipped image and a desaturated image?

Also, does the chroma keyer compute the alpha matte as max(r,g,b) after clipping the ranges?
Or maybe min or average?

Thanks again,
I really appreciate your help!!
David

User avatar
Millolab
Fusionista
Posts: 615
Joined: Wed Oct 24, 2018 6:26 am
Answers: 3
Been thanked: 93 times
Contact:

Re: What algorithm does the Chroma Keyer Node use?

#8

Post by Millolab » Mon Aug 31, 2020 11:06 am

Mmmm I don't think it's an average.

User avatar
Midgardsormr
Fusionator
Posts: 1913
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 18
Location: Los Angeles, CA, USA
Been thanked: 144 times
Contact:

Re: What algorithm does the Chroma Keyer Node use?

#9

Post by Midgardsormr » Mon Aug 31, 2020 12:37 pm

The keyer merely tests for the color values, and applies its logic to the Alpha. If all of the elements of the color triad fall inside of the slider range of their corresponding control, then the alpha at that location is 0. Then the channels are multiplied by the alpha to create the final color. Here's another sample:

Code: Select all

{
	Tools = ordered() {
		ChromaKeyer1 = ChromaKeyer {
			CtrlWZoom = false,
			Inputs = {
				KeyType = Input { Value = 1, },
				ColorRange = Input { Value = 1, },
				RedLow = Input { Value = 0.25, },
				RedHigh = Input { Value = 0.75, },
				GreenLow = Input { Value = 0.25, },
				GreenHigh = Input { Value = 0.75, },
				FringeSize = Input { Value = 0, },
				Input = Input {
					SourceOp = "Merge1",
					Source = "Output",
				},
			},
			ViewInfo = OperatorInfo { Pos = { 277, 67 } },
		},
		Merge1 = Merge {
			Inputs = {
				Background = Input {
					SourceOp = "Background2",
					Source = "Output",
				},
				Foreground = Input {
					SourceOp = "Background1",
					Source = "Output",
				},
				ApplyMode = Input { Value = FuID { "Lighten" }, },
				PerformDepthMerge = Input { Value = 0, },
			},
			ViewInfo = OperatorInfo { Pos = { 167, 67 } },
		},
		Background2 = Background {
			Inputs = {
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				Type = Input { Value = FuID { "Vertical" }, },
				TopLeftGreen = Input { Value = 1, },
			},
			ViewInfo = OperatorInfo { Pos = { 57, 68 } },
		},
		Background1 = Background {
			Inputs = {
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				Type = Input { Value = FuID { "Horizontal" }, },
				TopRightRed = Input { Value = 1, },
			},
			ViewInfo = OperatorInfo { Pos = { 55, 16.5 } },
		}
	}
}

We have simple linear ramps for Red and Green in different directions. I've set the Color key thresholds for both Red and Green to 0.25 and 0.75. Any pixel that satisfies both of those conditions becomes 0. If you change the Blue Low control at all, all pixels become solid because Blue is 0 at every pixel, and therefore no pixel satisfies the test condition.

If you turn up Soft Range, the alpha becomes a linear ramp between the threshold values and the values +/- the Range value. So if you turn up Soft Range to 0.1, you'll get a linear ramp in alpha on all sides of the rectangle. If you then start to raise the Blue Low control, the center will fill in until Blue Low = 0.1.


Image


I'm not sure what the most efficient way of expressing it in shader language would be. I'd have to give it some thought that I don't have the cycles for right now. Generally, though, you'd want to avoid conditionals. There's surely a relatively simple and fast mathematical formula to get it done.

And again, this is only in the Color mode because I don't ken Chroma. The math will be the same, but I am not sure how it applies through the color space conversion.

edit2: Sorted it. Never mind, if you saw this edit comment. :D

User avatar
Millolab
Fusionista
Posts: 615
Joined: Wed Oct 24, 2018 6:26 am
Answers: 3
Been thanked: 93 times
Contact:

Re: What algorithm does the Chroma Keyer Node use?

#10

Post by Millolab » Tue Sep 01, 2020 1:31 am

@Shem Namo here what I was doing with the CustomTool. Just RGB.
It's just a proof of concept. So it's all done with conditionals. No soft range. It's now cheated with a blur node which is NOT the same thing.

Maybe it helps. Idk. :D

Code: Select all

{
	Tools = ordered() {
		Background1 = Background {
			CtrlWZoom = false,
			Inputs = {
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				Type = Input { Value = FuID { "Corner" }, },
				TopLeftRed = Input { Value = 1, },
				TopRightGreen = Input { Value = 0.304, },
				TopRightBlue = Input { Value = 1, },
				BottomLeftRed = Input { Value = 1, },
				BottomLeftGreen = Input { Value = 0.816000000000001, },
				BottomRightRed = Input { Value = 0.0800000000000001, },
				BottomRightGreen = Input { Value = 1, },
			},
			ViewInfo = OperatorInfo { Pos = { 791.333, 28.5757 } },
		},
		CustomTool1 = Custom {
			Inputs = {
				LUTIn1 = Input {
					SourceOp = "CustomTool1LUTIn11",
					Source = "Value",
				},
				LUTIn2 = Input {
					SourceOp = "CustomTool1LUTIn21",
					Source = "Value",
				},
				LUTIn3 = Input {
					SourceOp = "CustomTool1LUTIn31",
					Source = "Value",
				},
				LUTIn4 = Input {
					SourceOp = "CustomTool1LUTIn41",
					Source = "Value",
				},
				Intermediate1 = Input { Value = "if(r1>n1,0,1)", },
				Intermediate2 = Input { Value = "i1+if(r1<n2,0,1)", },
				Intermediate3 = Input { Value = "if(g1>n3,0,1)", },
				Intermediate4 = Input { Value = "i3+if(g1<n4,0,1)", },
				Intermediate5 = Input { Value = "if(b1>n5,0,1)", },
				Intermediate6 = Input { Value = "i5+if(b1<n6,0,1)", },
				Intermediate7 = Input { Value = "min((i2+i4+i6),1)", },
				Intermediate8 = Input { Value = "if((n1+n2+n3+n4+n5+n6)==3,a1,i7)", },
				AlphaExpression = Input { Value = "i8", },
				NumberControls = Input { Value = 1, },
				NameforNumber1 = Input { Value = "Red Low", },
				NameforNumber2 = Input { Value = "Red High", },
				NameforNumber3 = Input { Value = "Green Low", },
				NameforNumber4 = Input { Value = "Green High", },
				NameforNumber5 = Input { Value = "Blue Low", },
				NameforNumber6 = Input { Value = "Blue High", },
				ShowNumber7 = Input { Value = 0, },
				ShowNumber8 = Input { Value = 0, },
				ShowPoint1 = Input { Value = 0, },
				ShowPoint2 = Input { Value = 0, },
				ShowPoint3 = Input { Value = 0, },
				ShowPoint4 = Input { Value = 0, },
				ShowLUT1 = Input { Value = 0, },
				ShowLUT2 = Input { Value = 0, },
				ShowLUT3 = Input { Value = 0, },
				ShowLUT4 = Input { Value = 0, },
				Image1 = Input {
					SourceOp = "Background1",
					Source = "Output",
				},
				NumberIn1 = Input { Value = 0.702, },
				NumberIn2 = Input { Value = 0.944, },
				NumberIn3 = Input { Value = 0.135, },
				NumberIn4 = Input { Value = 0.326, },
				NumberIn5 = Input { Value = 0.065, },
				NumberIn6 = Input { Value = 0.2, },
			},
			ViewInfo = OperatorInfo { Pos = { 791.333, 104.606 } },
			UserControls = ordered() {
				NumberIn1 = {
					LINKS_Name = "Red Low",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 2,
					IC_ControlID = 0,
					ICS_ControlPage = "Controls"
				},
				NumberIn2 = {
					LINKS_Name = "Red High",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 2,
					IC_ControlID = 1,
					ICS_ControlPage = "Controls"
				},
				NumberIn3 = {
					LINKS_Name = "Green Low",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 3,
					IC_ControlID = 0,
					ICS_ControlPage = "Controls"
				},
				NumberIn4 = {
					LINKS_Name = "Green High",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 3,
					IC_ControlID = 1,
					ICS_ControlPage = "Controls"
				},
				NumberIn5 = {
					LINKS_Name = "Blue Low",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 4,
					IC_ControlID = 0,
					ICS_ControlPage = "Controls"
				},
				NumberIn6 = {
					LINKS_Name = "Blue High",
					LINKID_DataType = "Number",
					INPID_InputControl = "RangeControl",
					INP_Integer = false,
					INP_MinScale = 0,
					INP_MaxScale = 1,
					INP_MinAllowed = -1000000,
					INP_MaxAllowed = 1000000,
					IC_ControlGroup = 4,
					IC_ControlID = 1,
					ICS_ControlPage = "Controls"
				}
			}
		},
		CustomTool1LUTIn11 = LUTBezier {
			KeyColorSplines = {
				[0] = {
					[0] = { 0, RH = { 0.333333333333333, 0.333333333333333 }, Flags = { Linear = true } },
					[1] = { 1, LH = { 0.666666666666667, 0.666666666666667 }, Flags = { Linear = true } }
				}
			},
			SplineColor = { Red = 204, Green = 0, Blue = 0 },
			NameSet = true,
		},
		CustomTool1LUTIn21 = LUTBezier {
			KeyColorSplines = {
				[0] = {
					[0] = { 0, RH = { 0.333333333333333, 0.333333333333333 }, Flags = { Linear = true } },
					[1] = { 1, LH = { 0.666666666666667, 0.666666666666667 }, Flags = { Linear = true } }
				}
			},
			SplineColor = { Red = 0, Green = 204, Blue = 0 },
			NameSet = true,
		},
		CustomTool1LUTIn31 = LUTBezier {
			KeyColorSplines = {
				[0] = {
					[0] = { 0, RH = { 0.333333333333333, 0.333333333333333 }, Flags = { Linear = true } },
					[1] = { 1, LH = { 0.666666666666667, 0.666666666666667 }, Flags = { Linear = true } }
				}
			},
			SplineColor = { Red = 0, Green = 0, Blue = 204 },
			NameSet = true,
		},
		CustomTool1LUTIn41 = LUTBezier {
			KeyColorSplines = {
				[0] = {
					[0] = { 0, RH = { 0.333333333333333, 0.333333333333333 }, Flags = { Linear = true } },
					[1] = { 1, LH = { 0.666666666666667, 0.666666666666667 }, Flags = { Linear = true } }
				}
			},
			SplineColor = { Red = 204, Green = 204, Blue = 204 },
			NameSet = true,
		},
		Blur1 = Blur {
			Inputs = {
				Filter = Input { Value = FuID { "Bartlett" }, },
				Red = Input { Value = 0, },
				Green = Input { Value = 0, },
				Blue = Input { Value = 0, },
				XBlurSize = Input { Value = 33.3, },
				Input = Input {
					SourceOp = "CustomTool1",
					Source = "Output",
				},
			},
			ViewInfo = OperatorInfo { Pos = { 791.333, 181.303 } },
		},
		AlphaMultiply1 = AlphaMultiply {
			Inputs = {
				Input = Input {
					SourceOp = "Blur1",
					Source = "Output",
				},
			},
			ViewInfo = OperatorInfo { Pos = { 791.333, 232.152 } },
		}
	}
}

User avatar
Shem Namo
Fusionista
Posts: 592
Joined: Sun Oct 06, 2019 9:15 pm
Location: North Israel
Real name: David Kohen
Been thanked: 35 times

Re: What algorithm does the Chroma Keyer Node use?

#11

Post by Shem Namo » Tue Sep 01, 2020 4:56 am

Thank you so much for you reply @Midgardsormr!!
I think I understand.

Thanks Millolab, I'll try to take apart your CustomTool to see if I can get information out of it. :mrgreen:

I've been testing for a while now, and with all of your help have come to this result:
I matched the color channels with the Chroma keyer exactly.

If you view each color channel individually, you will see that the results are identical.
Even the soft range control, which is why I now can't figure out how to make the alpha work.

So this is applied to every color channel:
  1.     float Low = color.x * 1.0f - smoothstep(rangeLow - softRange, rangeLow, color.x);
  2.     float High = color.x * smoothstep(rangeHigh, rangelumHigh + softRange, color.x);
  3.     color.x = _fmaxf(Low, High);
DCTL's smoothstep() function really nails the soft range.

The only problem not is: Since it isn't conditional anymore, I can't figure out how to put these 3 values together to get an alpha value.
After that, I'll probably just multiply the color channels by the alpha to get the final result, right?

Thanks again guys,
I really appreciate your help!!
David