Welcome to WSL!

New to the forum? Please read this and this.

Extract camera from EXR metadata

User avatar
Chad
Fusionator
Posts: 1565
Joined: Fri Aug 08, 2014 1:11 pm
Been thanked: 1 time

Re: Extract camera from EXR metadata

#16

Post by Chad »

svenneve wrote:I might have to give this a try, as our old fuse and script (and others) depends on Matrix4 , which is broken in Fusion7.

Thanks for posting

Was it just a Number modifier fuse with outputs for the 3D transform tab, or was it something else?

User avatar
NNois
Posts: 6
Joined: Mon Sep 26, 2016 5:20 am

Re: Extract camera from EXR metadata

#17

Post by NNois »

Hi,
Seems like i hit like you guys the same wall...

Do someone has a working script to get Exr Metadatas on Camera now ?

Cheers

User avatar
Chad
Fusionator
Posts: 1565
Joined: Fri Aug 08, 2014 1:11 pm
Been thanked: 1 time

Re: Extract camera from EXR metadata

#18

Post by Chad »

You can't just use the metadata directly in the expression for the camera transform expressions?

User avatar
Protean
Posts: 42
Joined: Mon Nov 03, 2014 12:37 pm
Location: www.axisanimation.com

Re: Extract camera from EXR metadata

#19

Post by Protean »

I've done this but I need to have the image attached to the image input of the camera to access the metadata, for example

Code: Select all

ImageInput.Metadata.camPos[8]
I thought this was supposed to work

Code: Select all

Loader1.Output.Metadata.camPos[8]
But it doesn't

In the same way

Code: Select all

Loader1.Output.Metadata.Filename
doesn't work as at states in the manual

Unless I have missed something simple.

EDIT: I have discovered that this indeed does not work in FU7 but only in FU8

J
Last edited by Protean on Mon Nov 28, 2016 3:05 pm, edited 1 time in total.

User avatar
miaz3
Fusioneer
Posts: 228
Joined: Sat Jan 03, 2015 1:43 am
Location: Angoulême / France
Contact:

Re: Extract camera from EXR metadata

#20

Post by miaz3 »

hi,
Fusion_8_Scripting_Guide.pdf ( page 171)

Code: Select all

metadata = comp.Loader1.Output.GetValue().Metadata
print(“Image was loaded from ” + metadata[“Filename”])
cf:

Code: Select all

metadata = comp.Loader1.Output.GetValue().Metadata
for field in metadata:
    print field
print metadata['worldToNDC']

User avatar
JCB91
Fusioneer
Posts: 96
Joined: Thu Mar 10, 2016 1:46 am

Re: Extract camera from EXR metadata

#21

Post by JCB91 »

These RSCameraTransform expressions just wont work for me.
I dont understand the problem. Is there something wrong with the code?

Code: Select all

ImageInput.Metadata.RSCameraTransform[15]

User avatar
AndrewHazelden
Fusionator
Posts: 1679
Joined: Fri Apr 03, 2015 3:20 pm
Answers: 9
Location: West Dover, Nova Scotia, Canada
Been thanked: 1 time
Contact:

Re: Extract camera from EXR metadata

#22

Post by AndrewHazelden »

JCB91 wrote:These RSCameraTransform expressions just wont work for me.
I dont understand the problem. Is there something wrong with the code?
Hi JCB91.

Would you be willing to render out a tiny sized EXR image that has the metadata tags you need present in it from Red Shift. It can be a very reduced 3D scene with simple primitive shapes. Take that EXR image and zip it. Then attach the less then 1MB in size zip file to this WSL thread.

This would make it easier for someone like Chad, Kristof, Me, or SecondMan to be able to see what's going on with Fu + the Red Shift image metadata entries and try to assist.

Andrew

User avatar
JCB91
Fusioneer
Posts: 96
Joined: Thu Mar 10, 2016 1:46 am

Re: Extract camera from EXR metadata

#23

Post by JCB91 »

Hi Andrew,

I have attached an exr, its just a single pass, but hass all the metadata.
Maybe you or others will get this working.
Thanks in advance
You do not have the required permissions to view the files attached to this post.

User avatar
miaz3
Fusioneer
Posts: 228
Joined: Sat Jan 03, 2015 1:43 am
Location: Angoulême / France
Contact:

Re: Extract camera from EXR metadata

#24

Post by miaz3 »

Code: Select all

meta = Loader1.Output:GetValue().Metadata
meta['RSCameraTransform']
Do not forget multiply by 25.4 your RSCameraAperture.

edit : crazy bracket on console tab...

User avatar
AndrewHazelden
Fusionator
Posts: 1679
Joined: Fri Apr 03, 2015 3:20 pm
Answers: 9
Location: West Dover, Nova Scotia, Canada
Been thanked: 1 time
Contact:

Re: Extract camera from EXR metadata

#25

Post by AndrewHazelden »

I've just taken a moment to look at your sample EXR image coming from Red Shift. :)

Probing the Metadata Values

In its simplest form this lets you access the raw "RSCameraTransform" metadata tag using Lua:

Code: Select all

xform = Loader1.Output:GetValue().Metadata.RSCameraTransform
-- Result = 0.962218,0.000000,0.272280,0.000000,-0.018337,0.997730,0.064803,0.000000,0.271662,0.067347,-0.960033,0.000000,-1077.789185,253.851807,4898.295410,1.000000
You could split these comma separated values into a Lua table format using:

Code: Select all

xform = Loader1.Output:GetValue().Metadata.RSCameraTransform
xformTable = split(xform, ",")
dump(xformTable)

--  Result: table: 0x77a965b0
--  1 = 0.962218
--  2 = 0.000000
--  3 = 0.272280
--  4 = 0.000000
--  5 = -0.018337
--  6 = 0.997730
--  7 = 0.064803
--  8 = 0.000000
--  9 = 0.271662
--  10 = 0.067347
--  11 = -0.960033
--  12 = 0.000000
--  13 = -1077.789185
--  14 = 253.851807
--  15 = 4898.295410
--  16 = 1.000000

You can print a list to the console tab of all the entries in an image's metadata records with:

Code: Select all

dump(Loader1.Output:GetValue().Metadata)

--  table: 0x77ac3650
--    RSCameraBlackCrushThreshold = 0.250000
--    RSCameraDOFRadius = 1.000000
--    CreationTime = 2017:10:17:20:01:26
--    RSVersion = 2.5.35
--    RSCameraAspect = 1.777000
--    RSCamera2DTo3DVector = -0.359999,0.000375,0.202588,-0.000375
--    screenWindowWidth = 1
--    RSCameraWhitepoint = 1.000000,1.000000,1.000000
--    Filename = /Users/andrew/Desktop/RedShift.exr
--    type = scanlineimage
--    RSCameraISO = 100.000000
--    RSHostname = Maverick
--    RSCameraVignetting = 5.000000
--    RSCameraFarPlane = 100000.000000
--    RSCameraShutterRatio = 100.000000
--    RSCameraTransform = 0.962218,0.000000,0.272280,0.000000,-0.018337,0.997730,0.064803,0.000000,0.271662,0.067347,-0.960033,0.000000,-1077.789185,253.851807,4898.295410,1.000000
--    RSCameraBlackCrushAmount = 0.000000
--    dwaCompressionLevel = 45
--    RSCameraShift = 0.000000,0.000000
--    RSCameraDOFAspect = 1.000000
--    RSCameraAperture = 1.417320,0.797591
--    RSCameraAllowedOverexposure = 0.200000
--    screenWindowCenter = table: 0x77ac3698
--      1 = 0
--      2 = 0
--    RSCameraFOV = 39.597660
--    RSCameraDOFFocusDistance = 5100.000000
--    RSCameraCDM2 = 10000.000000
--    RSCameraFStop = 8.000000
--    RSCameraSaturation = 1.000000
--    RSCameraNearPlane = 1000.000000
Sample Fusion Comp
red-shift-metadata-node-flow.png
This is the starting point of an example Fusion composite that sets up a Camera3D node automatically using the metadata coming from a Loader node.

(Chad or Kristof might be able to show you how to process the comma separated RSCameraTransform 4x4 matrix data into the required XYZ rotation and transform based position values. I don't have any insight on that process.)
RedShift Camera3D Metadata.comp
The loader node is wirelessly linked to the Camera3D node with the help of a UserControls "ImageControl" attribute called MetadataSource that was added manually using a text editor to the bottom of Camera3D node.
red-shift-metadata-tags.png
At the moment the following Red Shift attributes are processed:
RSCameraFOV is used to drive the Angle Of View
RSCameraDOFFocusDistance is used to drive the Plane of Focus (for depth of field)
RSCameraNearPlane is used to drive the Near Clip
RSCameraFarPlane is used to drive the Far Clip
RSCameraTransform is added to the Comments text field section
red-shift-transform-in-comments.png
Todo:
Process the RSCameraTransform camera matrix to strip the commas and then convert it from a 16 cell 4x4 matrix into a XYZ rotation and transform based position
Process RSCameraAperture into the Aperture Width and Height values (Fusion might make it hard to expression link these controls via the GUI.)
Process RSCameraShift into Lens Shift X/Y
You do not have the required permissions to view the files attached to this post.

User avatar
AndrewHazelden
Fusionator
Posts: 1679
Joined: Fri Apr 03, 2015 3:20 pm
Answers: 9
Location: West Dover, Nova Scotia, Canada
Been thanked: 1 time
Contact:

Re: Extract camera from EXR metadata

#26

Post by AndrewHazelden »

I wasn't sure how to decode the camera transform matrix using an expression based approach on the Camera3D node so I implemented an intool script based approach that does the same process.

Download the Comp
RedShift Camera3D Metadata v2.comp
About the Intool Script

To create the new intool script in this example comp, I used the camera transform code snippet that SecondMan posted on this tread at post #2 as the basis for my intool script's rotate and transform handling. ''

I'm not 100% sure if I got the transform matrix coordinates decoded correctly for the exact coordinate system layout that Red Shift uses internally so you will have to compare the results from this example .comp file to against the original the camera position back in your host 3D package to verify the math.

Here is an inline view of the Intool script code I wrote if you'd like to take a look at it before downloading and opening the comp file.

Code: Select all

-- Paste this into the Camera3D node's Frame Render Intool Script Textfield

-- Transform
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraTransform ~= nil then
  m1,m2,m3,m4,m5,m6,m7,m8,m9,m10,m11,m12,m13,m14,m15,m16 = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraTransform, "([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+),([%w%-%.]+)")

  -- print("[RSCameraTransform Matrix]")
  -- print(m1,m2,m3,m4)
  -- print(m5,m6,m7,m8)
  -- print(m9,m10,m11,m12)
  -- print(m13,m14,m15,m16)

  -- print("[Adding the XYZ Offset]")
  Transform3DOp.Translate.X = m13
  Transform3DOp.Translate.Y = m14
  Transform3DOp.Translate.Z = m15

  -- print("[Adding the XYZ Rotation]")
  -- Rotate X
  if cos(-asin(m3)) ~= 0 then
    Transform3DOp.Rotate.X = deg(atan2(m7,m11))
  else
    Transform3DOp.Rotate.X = deg(atan2(m5,m6)) - 90
  end

  -- Rotate Y
  Transform3DOp.Rotate.Y = deg(-asin(m3))

  -- Rotate Z
  if cos(-asin(m3)) ~=0 then 
    Transform3DOp.Rotate.Z = deg(atan2(m2,m1))
  else
    Transform3DOp.Rotate.Z = 0
  end
end

-- Aperture
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraAperture ~= nil then
  a1,a2 = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraAperture, "([%w%-%.]+),([%w%-%.]+)")
  ApertureW = a1
  ApertureH = a2
end

-- Lens Shift X/Y
if self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraShift ~= nil then
  lx,ly = string.match(self:GetSourceTool("MetadataSource").Output.Metadata.RSCameraShift, "([%w%-%.]+),([%w%-%.]+)")
  LensShiftX = lx
  LensShiftY = ly
end

-- print("-")
Screenshots
red-shift-camera3d-node-controls.png
red-shift-camera3d-node-transform.png
You do not have the required permissions to view the files attached to this post.

User avatar
JCB91
Fusioneer
Posts: 96
Joined: Thu Mar 10, 2016 1:46 am

Re: Extract camera from EXR metadata

#27

Post by JCB91 »

Hi Andrew,
thanks for your help. The camera seems to be rotated wrong, 180° on the Z and Y axis. Also the Intool Script doesnt work for me in 7.7.1.
I have a working script for Vray exrs that creates keyframes, I think it just needs some minor adjustments to get i working with RS metadata. What do do you say?
  1. tool_pos = comp["CurrentFrame"]["FlowView"].GetPosTable(tool)
  2.     tool_in = int(tool.GetAttrs()["TOOLNT_Clip_Start"][1])
  3.     tool_out = int(tool.GetAttrs()["TOOLNT_Clip_End"][1])
  4.     comp_in = int(comp.GetAttrs()["COMPN_GlobalStart"])
  5.     comp_out = int(comp.GetAttrs()["COMPN_GlobalEnd"])
  6.  
  7.     if tool_in < comp_in:
  8.         cam_in = comp_in
  9.     else:
  10.         cam_in = tool_in
  11.     if tool_out > comp_out:
  12.         cam_out = comp_out
  13.     else:
  14.         cam_out = tool_out
  15.  
  16.  
  17.     print(str(cam_in) + ", " + str(cam_out))
  18.    
  19.     cam_tool = comp.AddTool("Camera3D", tool_pos[1], tool_pos[2] + 1)
  20.    
  21.     cam_tool.AddModifier("Transform3DOp.Rotate.X", "BezierSpline")
  22.     cam_tool.AddModifier("Transform3DOp.Rotate.Y", "BezierSpline")
  23.     cam_tool.AddModifier("Transform3DOp.Rotate.Z", "BezierSpline")
  24.     cam_tool.AddModifier("Transform3DOp.Translate.X", "BezierSpline")
  25.     cam_tool.AddModifier("Transform3DOp.Translate.Y", "BezierSpline")
  26.     cam_tool.AddModifier("Transform3DOp.Translate.Z", "BezierSpline")
  27.     cam_tool.AddModifier("AoV", "BezierSpline")
  28.     cam_tool.AddModifier("ApertureW", "BezierSpline")
  29.     cam_tool.AddModifier("PlaneOfFocus", "BezierSpline")
  30.    
  31.    
  32.    
  33.     for i in range(cam_in, cam_out + 1):
  34.         cam_aperture = tool["Output"][i]["Metadata"]["Aperture"]
  35.         cam_fov = tool["Output"][i]["Metadata"]["cameraFov"]
  36.         cam_target_distance = tool["Output"][i]["Metadata"]["cameraTargetDistance"]
  37.         cam_xf = tool["Output"][i]["Metadata"]["cameraTransform"]
  38.        
  39.         cam_tool.SetInput("Transform3DOp.Translate.X", cam_xf[13], i)
  40.         cam_tool.SetInput("Transform3DOp.Translate.Y", cam_xf[14], i)
  41.         cam_tool.SetInput("Transform3DOp.Translate.Z", cam_xf[15], i)
  42.         if math.cos(-math.asin(cam_xf[3])) != 0:
  43.             rotate_x = math.degrees(math.atan2(cam_xf[7], cam_xf[11])) + 90
  44.         else:
  45.             rotate_x = math.degrees(math.atan2(cam_xf[5], cam_xf[6])) + 90
  46.         rotate_y = 180 - math.degrees(-math.asin(cam_xf[3]))   
  47.         if math.cos(-math.asin(cam_xf[3])) != 0:
  48.             rotate_z = math.degrees(math.atan2(cam_xf[2], cam_xf[1])) - 180
  49.         else:
  50.             rotate_z = -180
  51.            
  52.         cam_tool.SetInput("Transform3DOp.Rotate.X", rotate_x, i)
  53.         cam_tool.SetInput("Transform3DOp.Rotate.Y", rotate_y, i)
  54.         cam_tool.SetInput("Transform3DOp.Rotate.Z", rotate_z, i)
  55.         cam_tool.SetInput("AoV", cam_fov, i)
  56.         cam_tool.SetInput("PlaneOfFocus", cam_target_distance, i)
  57.         cam_tool.SetInput("ApertureW", cam_aperture / 10 / 2.54, i)
  58.    
  59.     cam_tool.SetInput("AovType", 1)
  60.     cam_tool.SetInput("ResolutionGateFit", "Width")
  61.     cam_tool.SetAttrs({"TOOLS_Name": "CAM3D_from_" + tool["Name"], "TOOLB_NameSet": True})

User avatar
AndrewHazelden
Fusionator
Posts: 1679
Joined: Fri Apr 03, 2015 3:20 pm
Answers: 9
Location: West Dover, Nova Scotia, Canada
Been thanked: 1 time
Contact:

Re: Extract camera from EXR metadata

#28

Post by AndrewHazelden »

JCB91 wrote:Hi Andrew,
thanks for your help. The camera seems to be rotated wrong, 180° on the Z and Y axis. Also the Intool Script doesnt work for me in 7.7.1.
The intool script I created was designed to work with the example composite’s Camera3D node that had a UserControls "ImageControl" attribute called MetadataSource added. This meant the Loader node name was not hardcoded in the script and the connection could be flexibly dragged and dropped onto the Camera3D node’s MetadataSource attribute section using a 100% wireless connection approach.

The coordinate system differences would be from how Houdini and RedShift interpret a 4x4 transform matrix.

User avatar
miaz3
Fusioneer
Posts: 228
Joined: Sat Jan 03, 2015 1:43 am
Location: Angoulême / France
Contact:

Re: Extract camera from EXR metadata

#29

Post by miaz3 »

The coordinate system differences would be from how Houdini and RedShift interpret a 4x4 transform matrix.
i thinking, for 3dsmax users careful > Z-up / Y-up

via colorMatrix (if wpp for example ) 3x3 :

Code: Select all

1 0 0 
0 0 1 
0 -1 0
or rotate x-axis (if camera for exemple) 4x4 :

Code: Select all

1 0 0 0
0 0 1 0
0 -1 0 0
0 0 0 1

@JCB91 :
if i remember well, on RS (maya) :
- rotateY(math.radians(-180)) # backwards
- transform to Vector3(0,0,0)
- rotations must be > ZXY

User avatar
Protean
Posts: 42
Joined: Mon Nov 03, 2014 12:37 pm
Location: www.axisanimation.com

Re: Extract camera from EXR metadata

#30

Post by Protean »

I’ve had to look at some Redshift renders recently and while I had a bit of an issue with the ‘frame render script’ method as it doesn’t update on cached frames.

It’s a pity that the transform matrix comes as CSVs instead of a table but to get around that you csn use a fuse to process the CSV string to a table and then you have direct access it directly in a ‘simple expression’ and avoid the frame render script limitation.

John