Possible color space problem when importing Canon's MP4 clips

Moderator: SecondMan

User avatar
splinefx
Posts: 4
Joined: Fri Mar 01, 2019 10:10 pm

Possible color space problem when importing Canon's MP4 clips

#1

Post by splinefx » Fri Jul 05, 2019 10:06 am

Hi. I have some footage shot on Canon 550D, it's in mp4 with h.264 codec. When I import these clips into Fusion, they are a bit contrasty. In Resolve and AE they're fine, I see a lot more details in shadows. But here, in Fusion, blacks are just cut. I tried to play with Import settings in a Loader node, experimented with different gamma space parameters, no luck. Tried to use Gamut node with lots of different combinations of Source and Output Space, still no luck. Blacks are always cut, more or less. I simply don't know what else to do. In AE these clips are fine if I choose Working Space to be None. Any ideas?

User avatar
Midgardsormr
Fusionator
Posts: 1211
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 3
Location: Los Angeles, CA, USA
Been thanked: 92 times
Contact:

Re: Possible color space problem when importing Canon's MP4 clips

#2

Post by Midgardsormr » Fri Jul 05, 2019 11:56 am

First, have you examined the waveform Sub-View? That will tell you if you actually have clamped blacks, or if it's just a display problem.

Do you have a Viewer LUT enabled? If you have only a Loader and haven't changed anything in the Format or Import tabs, then it should appear the same as it would in something like VLC, as long as the LUT is turned off.

Can you supply a short sample?

User avatar
splinefx
Posts: 4
Joined: Fri Mar 01, 2019 10:10 pm

Re: Possible color space problem when importing Canon's MP4 clips

#3

Post by splinefx » Fri Jul 05, 2019 11:16 pm

Midgardsormr wrote:
Fri Jul 05, 2019 11:56 am
First, have you examined the waveform Sub-View? That will tell you if you actually have clamped blacks, or if it's just a display problem.

Do you have a Viewer LUT enabled? If you have only a Loader and haven't changed anything in the Format or Import tabs, then it should appear the same as it would in something like VLC, as long as the LUT is turned off.

Can you supply a short sample?
Honestly, I don't know much about display settings, LUTs and stuff. I'll try to gather as much data as I can and provide it here.

Added in 27 minutes 41 seconds:
Clip in Fusion:


Clip in Resolve:


Clip in AE:


Same clip, same frame. As far as I can say by looking at these scopes, blacks are indeed clipped in Fusion. And you can just look at the jacket, completely no details, everything is dark. LUT in viewport is not enabled. Settings in the Loader node are untouched. VLC plays this clip similar to Fusion, it's also dark for some reason. If I use Gamut node, I can get this result:


But it's still clippy. If I use BC node, I get this:


Thank you for your time.
P.S.: Sorry, I'm not sure how to insert pictures correctly.

User avatar
Midgardsormr
Fusionator
Posts: 1211
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 3
Location: Los Angeles, CA, USA
Been thanked: 92 times
Contact:

Re: Possible color space problem when importing Canon's MP4 clips

#4

Post by Midgardsormr » Sat Jul 06, 2019 7:56 am

Well, that is definitely not right. I'm not sure what else we can do to help test, though, without an actual sample of the video.

I grabbed an h264 mp4 test clip from here:
https://www.h264info.com/clips.html

It behaved exactly as expected (I used the I Am Legend trailer).

Image



BTW, to use the forum's image-handling, click the "Full Editor & Preview" button. In the toolbar across the top of the text-entry field, there's an Imgur button. Clicking that will let you select an image from your computer, and it automatically uploads it to Imgur and links it here in one step:
viewtopic.php?f=2&t=2057

User avatar
splinefx
Posts: 4
Joined: Fri Mar 01, 2019 10:10 pm

Re: Possible color space problem when importing Canon's MP4 clips

#5

Post by splinefx » Sat Jul 06, 2019 9:30 am

This trailer looks correct.
Image

Well, here's the sample.
Actually, I was wrong about the file format: it's not MP4 but MOV. But I'm not sure if it makes some difference anyway since it's the same h.264 codec.

User avatar
Midgardsormr
Fusionator
Posts: 1211
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 3
Location: Los Angeles, CA, USA
Been thanked: 92 times
Contact:

Re: Possible color space problem when importing Canon's MP4 clips

#6

Post by Midgardsormr » Sat Jul 06, 2019 4:30 pm

Yep, I see the same thing you do. It looks as though it has illegal values that Fusion is failing to detect. The good news is that you can still get the full range back by changing the Depth in the Loader to float16 or float32. That's the top set of controls in the Import tab. That will unclamp it, making values below 0 and above 100 IRE available. Then a Lift operation in the BrightnessContrast can boost the sub-zero code values into the positives. A Lift of about 0.06 makes it look most natural, and that appears to create a reasonable match to the scopes in Resolve.


Image


I don't have an answer for why Resolve can interpret this clip correctly but Fusion does not. Maybe someone with a stronger understanding of digital video encoding will come along to help out—I confess it's a weak area for me. @cinewrangler, perhaps?

Anyway, from here, you'd probably want to use a Gamut to go from Rec601 to "No Change" to linearize for your VFX work. Activate the LUT button in the Viewer, use the pop-up to choose the GamutViewLUT, and then use the pop-up again to choose "Edit" and set it up like this:

Image

That way you'll be working in a linearized space and monitoring in sRGB, which is probably your monitor's color space. At the end of the process, use another Gamut to return the image to 601 to match the non-VFX shots in the sequence.

The only issue here is that your end result will not have the same illegal values as the input clip, so it probably won't match the rest of the show—you'll probably need to do an operation in color correction to fix it. Fortunately, once you've figured out the operation you need (should be a -0.06 Lift/Pedestal, but I wouldn't stake my reputation on that), it will be exactly the same for every VFX shot.
Last edited by Midgardsormr on Sat Jul 06, 2019 10:58 pm, edited 1 time in total.

User avatar
splinefx
Posts: 4
Joined: Fri Mar 01, 2019 10:10 pm

Re: Possible color space problem when importing Canon's MP4 clips

#7

Post by splinefx » Sat Jul 06, 2019 10:33 pm

Thank you so much for such detailed answer. I'll give it a try. I hope this won't affect my keying too much: the thing is I'm working on a short film shot almost 90% on green screen and I've already completed several tricky shots at the moment I noticed this black clipping.

User avatar
cinewrangler
Posts: 30
Joined: Wed Nov 15, 2017 6:47 am
Location: Europe
Been thanked: 9 times

Re: Possible color space problem when importing Canon's MP4 clips

#8

Post by cinewrangler » Sat Jul 06, 2019 11:29 pm

Midgardsormr wrote:
Sat Jul 06, 2019 4:30 pm
@cinewrangler, perhaps?
Well, for quite a while the Canon DSLRs shooting H264 used Rec601 and full-range for the videos, presumably since that is what they have been using for JPEG all the years (yes, JPEG is some sort of full-range Y'CbCr under the hood). QuickTime MOV files don't have any flags to indicate full-range vs. legal-range, so the application trying to open these files cannot know that it is full-range.

There is an (optional) flag in the H264 data stream that can carry that information, but I'm not sure whether the 550D files have this flag included or not. If Resolve gets this correct, then I'd guess BM put in some extra code which either reads that flag or checks whether the file is from a Canon DSLR and then reacts differently from what it normally would have done.

User avatar
Midgardsormr
Fusionator
Posts: 1211
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 3
Location: Los Angeles, CA, USA
Been thanked: 92 times
Contact:

Re: Possible color space problem when importing Canon's MP4 clips

#9

Post by Midgardsormr » Sun Jul 07, 2019 8:19 am

Thanks for that insight. I've put in a bug report at the official forums.

User avatar
theotheo
Fusionista
Posts: 312
Joined: Thu Aug 07, 2014 8:35 am
Answers: 2
Been thanked: 20 times

Re: Possible color space problem when importing Canon's MP4 clips

#10

Post by theotheo » Mon Jul 08, 2019 1:05 am

The QT format does support a full-range via the atoms tags in the header. However, its up to the implementation/software of reading and writing the QTs to support these additional tags. You don't have to support them.....

edit:removed wrongfully quoted source.

I once spent two weeks trying to figure out why some vendors were sending back shots in a format we just couldn't match. Turns out we were the only vendor who accurately implemented the usage of atoms in our encoder. All the other vendors were technically wrong; but it was in the last few months of a show so we just had to break our encoder to match the rest. "As long as we're all equally wrong etc etc...."

Just wait until you start realising QT actually has ties into the OS (windows more so) for color management, that's when things gets really messed up.

QT is a very flakey container for professional use.
Last edited by theotheo on Mon Jul 08, 2019 3:28 am, edited 1 time in total.

User avatar
cinewrangler
Posts: 30
Joined: Wed Nov 15, 2017 6:47 am
Location: Europe
Been thanked: 9 times

Re: Possible color space problem when importing Canon's MP4 clips

#11

Post by cinewrangler » Mon Jul 08, 2019 2:33 am

theotheo wrote:
Mon Jul 08, 2019 1:05 am
The QT format does support a full-range via the atoms tags in the header. However, its up to the implementation/software of reading and writing the QTs to support these additional tags. You don't have to support them.....
And what would that full-range atom be? (I'm asking, not saying it dons't exist since I'd like to be proven wrong.) - Your excerpt from the QTFF specs is about 'free' and 'skip' atoms which is something entirely different. For H264 you might be able to extract the full-range flag from the AvcC extradata which is stored in a separate atom in the MOV file. But this is basically what I was referring to as the H264 data stream carrying a flag.

User avatar
theotheo
Fusionista
Posts: 312
Joined: Thu Aug 07, 2014 8:35 am
Answers: 2
Been thanked: 20 times

Re: Possible color space problem when importing Canon's MP4 clips

#12

Post by theotheo » Mon Jul 08, 2019 3:27 am

Sorry I don't recall the exact atom, but besides the usual "gama" and "colr" there were a few additional atoms that were skippable. Kevin Wheatly (OCIO/Academy/FS) were looking into this when we were doing Age of Ultron (which is now a few years back) and were flabbergasted when we found out Nuke actually didn't read or set the atoms either. I believe we were having issues with MJPEG (for Vfx review) codecs specifically; if I recall correctly we had no issues with DNXHD (for Vfx editorial).

I've removed the quote from the QTFF since I quoted a wrong source. But the concept of choosing to use atoms still stands as far as I know. Assuming QT haven't changed a lot since then.

User avatar
cinewrangler
Posts: 30
Joined: Wed Nov 15, 2017 6:47 am
Location: Europe
Been thanked: 9 times

Re: Possible color space problem when importing Canon's MP4 clips

#13

Post by cinewrangler » Mon Jul 08, 2019 4:01 am

The funky thing is that the QTFF document (version from 2014) describes how the Y'CbCr conversion should be carried and then says:
These normalized values can be mapped onto the stored integer values of a particular compression type's Y´,
Cb, and Cr components using two different schemes, which we will call Scheme A and Scheme B.
Scheme A is what equates to full-range:
This maps normalized values to stored values so that, for example, 8-bit unsigned values for Y´ go from 0-255
as the normalized value goes from 0 to 1, and 8-bit signed valued for Cb and Cr go from -127 to +127 as the
normalized values go from -0.5 to +0.5.
Scheme B is video-range aka. legal-range:
Scheme B uses "Video-Range" mapping with unsigned Y´ and offset binary Cb and Cr values.
Note: Scheme B, shown in Figure 4-5 (page 164), comes from digital video industry specifications
such as Rec. ITU-R BT. 601-4. All standard digital video tape formats (e.g., SMPTE D-1, SMPTE D-5)
and all standard digital video links (e.g., SMPTE 259M-1997 serial digital video) use this scheme.
Professional video storage and processing equipment from vendors such as Abekas, Accom, and SGI
also use this scheme. MPEG-2, DVC and many other codecs specify source Y´CbCr pixels using this
scheme.
All this is mentioned in the description of the 'colr' atom... but there is no (documented) way of specifying in the 'colr' atom whether scheme A or B was actually used.

In a QuickTime MOV the 'colr' atom has a parameter type of 'nclc'. For the MP4 format a parameter type of 'nclx' was specified that actually includes a flag to indicate which scheme was used.