Extract camera from EXR metadata

[Chad]

I might have to give this a try, as our old fuse and script (and others) depends on Matrix4 , which is broken in Fusion7.

Thanks for posting

Was it just a Number modifier fuse with outputs for the 3D transform tab, or was it something else?

[NNois]

Hi,
Seems like i hit like you guys the same wall...

Do someone has a working script to get Exr Metadatas on Camera now ?

Cheers

[Chad]

You can't just use the metadata directly in the expression for the camera transform expressions?

[Protean]

I've done this but I need to have the image attached to the image input of the camera to access the metadata, for example

Code: Select all

ImageInput.Metadata.camPos[8]

I thought this was supposed to work

Code: Select all

Loader1.Output.Metadata.camPos[8]

But it doesn't

In the same way

Code: Select all

Loader1.Output.Metadata.Filename

doesn't work as at states in the manual

Unless I have missed something simple.

EDIT: I have discovered that this indeed does not work in FU7 but only in FU8

J

[miaz3]

hi,
Fusion_8_Scripting_Guide.pdf ( page 171)

Code: Select all

metadata = comp.Loader1.Output.GetValue().Metadata
print(“Image was loaded from ” + metadata[“Filename”])

cf:

Code: Select all

metadata = comp.Loader1.Output.GetValue().Metadata
for field in metadata:
    print field
print metadata['worldToNDC']

[JCB91]

These RSCameraTransform expressions just wont work for me.
I dont understand the problem. Is there something wrong with the code?

Code: Select all

ImageInput.Metadata.RSCameraTransform[15]

[AndrewHazelden]

These RSCameraTransform expressions just wont work for me.
I dont understand the problem. Is there something wrong with the code?

Hi JCB91.

Would you be willing to render out a tiny sized EXR image that has the metadata tags you need present in it from Red Shift. It can be a very reduced 3D scene with simple primitive shapes. Take that EXR image and zip it. Then attach the less then 1MB in size zip file to this WSL thread.

This would make it easier for someone like Chad, Kristof, Me, or SecondMan to be able to see what's going on with Fu + the Red Shift image metadata entries and try to assist.

Andrew

[JCB91]

Hi Andrew,

I have attached an exr, its just a single pass, but hass all the metadata.
Maybe you or others will get this working.
Thanks in advance

[miaz3]

Code: Select all

meta = Loader1.Output:GetValue().Metadata
meta['RSCameraTransform']

Do not forget multiply by 25.4 your RSCameraAperture.

edit : crazy bracket on console tab...

[AndrewHazelden]

I've just taken a moment to look at your sample EXR image coming from Red Shift.

Probing the Metadata Values

In its simplest form this lets you access the raw "RSCameraTransform" metadata tag using Lua:

Code: Select all

xform = Loader1.Output:GetValue().Metadata.RSCameraTransform
-- Result = 0.962218,0.000000,0.272280,0.000000,-0.018337,0.997730,0.064803,0.000000,0.271662,0.067347,-0.960033,0.000000,-1077.789185,253.851807,4898.295410,1.000000

You could split these comma separated values into a Lua table format using:

Code: Select all

xform = Loader1.Output:GetValue().Metadata.RSCameraTransform
xformTable = split(xform, ",")
dump(xformTable)

--  Result: table: 0x77a965b0
--  1 = 0.962218
--  2 = 0.000000
--  3 = 0.272280
--  4 = 0.000000
--  5 = -0.018337
--  6 = 0.997730
--  7 = 0.064803
--  8 = 0.000000
--  9 = 0.271662
--  10 = 0.067347
--  11 = -0.960033
--  12 = 0.000000
--  13 = -1077.789185
--  14 = 253.851807
--  15 = 4898.295410
--  16 = 1.000000

You can print a list to the console tab of all the entries in an image's metadata records with:

Code: Select all

dump(Loader1.Output:GetValue().Metadata)

--  table: 0x77ac3650
--    RSCameraBlackCrushThreshold = 0.250000
--    RSCameraDOFRadius = 1.000000
--    CreationTime = 2017:10:17:20:01:26
--    RSVersion = 2.5.35
--    RSCameraAspect = 1.777000
--    RSCamera2DTo3DVector = -0.359999,0.000375,0.202588,-0.000375
--    screenWindowWidth = 1
--    RSCameraWhitepoint = 1.000000,1.000000,1.000000
--    Filename = /Users/andrew/Desktop/RedShift.exr
--    type = scanlineimage
--    RSCameraISO = 100.000000
--    RSHostname = Maverick
--    RSCameraVignetting = 5.000000
--    RSCameraFarPlane = 100000.000000
--    RSCameraShutterRatio = 100.000000
--    RSCameraTransform = 0.962218,0.000000,0.272280,0.000000,-0.018337,0.997730,0.064803,0.000000,0.271662,0.067347,-0.960033,0.000000,-1077.789185,253.851807,4898.295410,1.000000
--    RSCameraBlackCrushAmount = 0.000000
--    dwaCompressionLevel = 45
--    RSCameraShift = 0.000000,0.000000
--    RSCameraDOFAspect = 1.000000
--    RSCameraAperture = 1.417320,0.797591
--    RSCameraAllowedOverexposure = 0.200000
--    screenWindowCenter = table: 0x77ac3698
--      1 = 0
--      2 = 0
--    RSCameraFOV = 39.597660
--    RSCameraDOFFocusDistance = 5100.000000
--    RSCameraCDM2 = 10000.000000
--    RSCameraFStop = 8.000000
--    RSCameraSaturation = 1.000000
--    RSCameraNearPlane = 1000.000000

Sample Fusion Comp

red-shift-metadata-node-flow.png

This is the starting point of an example Fusion composite that sets up a Camera3D node automatically using the metadata coming from a Loader node.

(Chad or Kristof might be able to show you how to process the comma separated RSCameraTransform 4x4 matrix data into the required XYZ rotation and transform based position values. I don't have any insight on that process.)

RedShift Camera3D Metadata.comp

The loader node is wirelessly linked to the Camera3D node with the help of a UserControls "ImageControl" attribute called MetadataSource that was added manually using a text editor to the bottom of Camera3D node.

red-shift-metadata-tags.png

At the moment the following Red Shift attributes are processed:
RSCameraFOV is used to drive the Angle Of View
RSCameraDOFFocusDistance is used to drive the Plane of Focus (for depth of field)
RSCameraNearPlane is used to drive the Near Clip
RSCameraFarPlane is used to drive the Far Clip
RSCameraTransform is added to the Comments text field section

red-shift-transform-in-comments.png

Todo:
Process the RSCameraTransform camera matrix to strip the commas and then convert it from a 16 cell 4x4 matrix into a XYZ rotation and transform based position
Process RSCameraAperture into the Aperture Width and Height values (Fusion might make it hard to expression link these controls via the GUI.)
Process RSCameraShift into Lens Shift X/Y

[AndrewHazelden]

I wasn't sure how to decode the camera transform matrix using an expression based approach on the Camera3D node so I implemented an intool script based approach that does the same process.

Download the Comp

RedShift Camera3D Metadata v2.comp

About the Intool Script

To create the new intool script in this example comp, I used the camera transform code snippet that SecondMan posted on this tread at post #2 as the basis for my intool script's rotate and transform handling. ''

I'm not 100% sure if I got the transform matrix coordinates decoded correctly for the exact coordinate system layout that Red Shift uses internally so you will have to compare the results from this example .comp file to against the original the camera position back in your host 3D package to verify the math.

Here is an inline view of the Intool script code I wrote if you'd like to take a look at it before downloading and opening the comp file.

Code: Select all

-- Paste this into the Camera3D node's Frame Render Intool Script Textfield

-- Transform
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraTransform ~= nil then
  m1,m2,m3,m4,m5,m6,m7,m8,m9,m10,m11,m12,m13,m14,m15,m16 = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraTransform, "([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+)")

  -- print("[RSCameraTransform Matrix]")
  -- print(m1,m2,m3,m4)
  -- print(m5,m6,m7,m8)
  -- print(m9,m10,m11,m12)
  -- print(m13,m14,m15,m16)

  -- print("[Adding the XYZ Offset]")
  Transform3DOp.Translate.X = m13
  Transform3DOp.Translate.Y = m14
  Transform3DOp.Translate.Z = m15

  -- print("[Adding the XYZ Rotation]")
  -- Rotate X
  if cos(-asin(m3)) ~= 0 then
    Transform3DOp.Rotate.X = deg(atan2(m7,m11))
  else
    Transform3DOp.Rotate.X = deg(atan2(m5,m6)) - 90
  end

  -- Rotate Y
  Transform3DOp.Rotate.Y = deg(-asin(m3))

  -- Rotate Z
  if cos(-asin(m3)) ~=0 then 
    Transform3DOp.Rotate.Z = deg(atan2(m2,m1))
  else
    Transform3DOp.Rotate.Z = 0
  end
end

-- Aperture
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraAperture ~= nil then
  a1,a2 = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraAperture, "([%w%-%.]+),([%w%-%.]+)")
  ApertureW = a1
  ApertureH = a2
end

-- Lens Shift X/Y
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraShift ~= nil then
  lx,ly = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraShift, "([%w%-%.]+),([%w%-%.]+)")
  LensShiftX = lx
  LensShiftY = ly
end

-- print("-")

Screenshots

red-shift-camera3d-node-controls.png

red-shift-camera3d-node-transform.png

[JCB91]

Hi Andrew,
thanks for your help. The camera seems to be rotated wrong, 180° on the Z and Y axis. Also the Intool Script doesnt work for me in 7.7.1.
I have a working script for Vray exrs that creates keyframes, I think it just needs some minor adjustments to get i working with RS metadata. What do do you say?

Code: [Select all] [Expand/Collapse] [Download] (Untitled.lua)

1. tool_pos = comp["CurrentFrame"]["FlowView"].GetPosTable(tool)
2.     tool_in = int(tool.GetAttrs()["TOOLNT_Clip_Start"][1])
3.     tool_out = int(tool.GetAttrs()["TOOLNT_Clip_End"][1])
4.     comp_in = int(comp.GetAttrs()["COMPN_GlobalStart"])
5.     comp_out = int(comp.GetAttrs()["COMPN_GlobalEnd"])
6.  
7.     if tool_in < comp_in:
8.         cam_in = comp_in
9.     else:
10.         cam_in = tool_in
11.     if tool_out > comp_out:
12.         cam_out = comp_out
13.     else:
14.         cam_out = tool_out
15.  
16.  
17.     print(str(cam_in) + ", " + str(cam_out))
18.    
19.     cam_tool = comp.AddTool("Camera3D", tool_pos[1], tool_pos[2] + 1)
20.    
21.     cam_tool.AddModifier("Transform3DOp.Rotate.X", "BezierSpline")
22.     cam_tool.AddModifier("Transform3DOp.Rotate.Y", "BezierSpline")
23.     cam_tool.AddModifier("Transform3DOp.Rotate.Z", "BezierSpline")
24.     cam_tool.AddModifier("Transform3DOp.Translate.X", "BezierSpline")
25.     cam_tool.AddModifier("Transform3DOp.Translate.Y", "BezierSpline")
26.     cam_tool.AddModifier("Transform3DOp.Translate.Z", "BezierSpline")
27.     cam_tool.AddModifier("AoV", "BezierSpline")
28.     cam_tool.AddModifier("ApertureW", "BezierSpline")
29.     cam_tool.AddModifier("PlaneOfFocus", "BezierSpline")
30.    
31.    
32.    
33.     for i in range(cam_in, cam_out + 1):
34.         cam_aperture = tool["Output"][i]["Metadata"]["Aperture"]
35.         cam_fov = tool["Output"][i]["Metadata"]["cameraFov"]
36.         cam_target_distance = tool["Output"][i]["Metadata"]["cameraTargetDistance"]
37.         cam_xf = tool["Output"][i]["Metadata"]["cameraTransform"]
38.        
39.         cam_tool.SetInput("Transform3DOp.Translate.X", cam_xf[13], i)
40.         cam_tool.SetInput("Transform3DOp.Translate.Y", cam_xf[14], i)
41.         cam_tool.SetInput("Transform3DOp.Translate.Z", cam_xf[15], i)
42.         if math.cos(-math.asin(cam_xf[3])) != 0:
43.             rotate_x = math.degrees(math.atan2(cam_xf[7], cam_xf[11])) + 90
44.         else:
45.             rotate_x = math.degrees(math.atan2(cam_xf[5], cam_xf[6])) + 90
46.         rotate_y = 180 - math.degrees(-math.asin(cam_xf[3]))   
47.         if math.cos(-math.asin(cam_xf[3])) != 0:
48.             rotate_z = math.degrees(math.atan2(cam_xf[2], cam_xf[1])) - 180
49.         else:
50.             rotate_z = -180
51.            
52.         cam_tool.SetInput("Transform3DOp.Rotate.X", rotate_x, i)
53.         cam_tool.SetInput("Transform3DOp.Rotate.Y", rotate_y, i)
54.         cam_tool.SetInput("Transform3DOp.Rotate.Z", rotate_z, i)
55.         cam_tool.SetInput("AoV", cam_fov, i)
56.         cam_tool.SetInput("PlaneOfFocus", cam_target_distance, i)
57.         cam_tool.SetInput("ApertureW", cam_aperture / 10 / 2.54, i)
58.    
59.     cam_tool.SetInput("AovType", 1)
60.     cam_tool.SetInput("ResolutionGateFit", "Width")
61.     cam_tool.SetAttrs({"TOOLS_Name": "CAM3D_from_" + tool["Name"], "TOOLB_NameSet": True})

[AndrewHazelden]

Hi Andrew,
thanks for your help. The camera seems to be rotated wrong, 180° on the Z and Y axis. Also the Intool Script doesnt work for me in 7.7.1.

The intool script I created was designed to work with the example composite’s Camera3D node that had a UserControls "ImageControl" attribute called MetadataSource added. This meant the Loader node name was not hardcoded in the script and the connection could be flexibly dragged and dropped onto the Camera3D node’s MetadataSource attribute section using a 100% wireless connection approach.

The coordinate system differences would be from how Houdini and RedShift interpret a 4x4 transform matrix.

[miaz3]

The coordinate system differences would be from how Houdini and RedShift interpret a 4x4 transform matrix.

i thinking, for 3dsmax users careful > Z-up / Y-up

via colorMatrix (if wpp for example ) 3x3 :

Code: Select all

1 0 0 
0 0 1 
0 -1 0

or rotate x-axis (if camera for exemple) 4x4 :

Code: Select all

1 0 0 0
0 0 1 0
0 -1 0 0
0 0 0 1

@JCB91 :
if i remember well, on RS (maya) :
- rotateY(math.radians(-180)) # backwards
- transform to Vector3(0,0,0)
- rotations must be > ZXY

[Protean]

I’ve had to look at some Redshift renders recently and while I had a bit of an issue with the ‘frame render script’ method as it doesn’t update on cached frames.

It’s a pity that the transform matrix comes as CSVs instead of a table but to get around that you csn use a fuse to process the CSV string to a table and then you have direct access it directly in a ‘simple expression’ and avoid the frame render script limitation.

John