Extract camera from EXR metadata

Mikhey
Posts: 1
Joined: Sat Sep 27, 2014 6:51 pm
Location: Kiev? Ukraine
Contact:

Extract camera from EXR metadata

#1

Post by Mikhey » Sun Dec 07, 2014 6:06 am

Hello all.
Any idea how to get the camera from exr metadata rendered with Houdini Mantra?
Thanks!

User avatar
SecondMan
Site Admin
Posts: 3239
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 1
Location: Vancouver, Canada
Been thanked: 51 times
Contact:

Re: Extract camera from EXR metadata

#2

Post by SecondMan » Sun Dec 07, 2014 7:23 am

You can access that metadata with expressions in Fusion.

Here's an example Camera3D node made by GringoFX, which you can adjust for your needs. Where it says LoadersName in the expressions you need to input the name of the Loader containing your EXR files. Or if you want quick and dirty I guess you can rename your Loader to LoadersName ;)

You probably need to verify the syntax of Houdini's metadata for the camera. There's a Metadata Subview option in the Viewer. I believe this example assumes Vray, so with some luck it's pretty much the same.

Code: Select all

{
	Tools = ordered() {
		CameraFromMetadata = Camera3D {
			CtrlWZoom = false,
			NameSet = true,
			Inputs = {
				Comments = Input { Value = "Gets the camera parameters from metadata of an image sequence.\nDrag and drop the Loader to the Loader's Name field.\n\nby Gregory Chalenko for Parasol Island (math support by Leo Quensel)\nwww.compositing.ru", },
				["Transform3DOp.Translate.X"] = Input {
					Expression = "self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[13]",
				},
				["Transform3DOp.Translate.Y"] = Input {
					Expression = "self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[14]",
				},
				["Transform3DOp.Translate.Z"] = Input {
					Expression = "self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[15]",
				},
				["Transform3DOp.Rotate.X"] = Input {
					Expression = "iif(cos(-asin(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[3])) ~=0, deg(atan2(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[7],self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[11])), deg(atan2(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[5],self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[6]))) - 90",
				},
				["Transform3DOp.Rotate.Y"] = Input {
					Expression = "deg(-asin(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[3]))",
				},
				["Transform3DOp.Rotate.Z"] = Input {
					Expression = "iif(cos(-asin(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[3])) ~=0, deg(atan2(self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[2],self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraTransform[1])),0)",
				},
				AovType = Input { Value = 1, },
				AoV = Input {
					Expression = "self:GetSourceTool(\"LoadersName\").Output.Metadata.cameraFov",
				},
				ResolutionGateFit = Input { Value = FuID { "Width", }, },
			},
			ViewInfo = OperatorInfo { Pos = { -2640.93, -254.4, }, },
			UserControls = ordered() {
				LoadersName = {
					LINKID_DataType = "Image",
					LINKS_Name = "Loader's Name",
					IC_ControlPage = 1,
					INPID_InputControl = "ImageControl",
					INP_Default = 0,
				},
			},
		},
	},
}

User avatar
Kristof
Fusionista
Posts: 493
Joined: Thu Aug 07, 2014 1:30 pm
Been thanked: 10 times

Re: Extract camera from EXR metadata

#3

Post by Kristof » Tue Jan 20, 2015 5:10 am

Is this broken in Fu 7.6? Looks that way as I'm no longer able to use the setup I made in 6.4.

User avatar
Kristof
Fusionista
Posts: 493
Joined: Thu Aug 07, 2014 1:30 pm
Been thanked: 10 times

Re: Extract camera from EXR metadata

#4

Post by Kristof » Tue Jan 20, 2015 6:20 am

Anyway, I'd like to push this topic further. Recently tried this workflow and it has proven to be very useful, but there is room for improvement. Keep in mind that this was in Fu 6.4 and maybe some of the issues have been resolved in more recent builds.

I'm trying to build a prototype to address some of the problems we were having in production.

The biggest advantage is that you will always have a camera that is in synch with the render you are loading in Fusion. Nobody has to export the cam for you, or do it again to keep up--it's there automatically. Downside, it can be slow--depending on the size of the files--and you won't have motion blur. You can bake the camera in Fusion, but then you lose the live connection between your render and the Fusion camera. Also, when the camera zooms, the field of view expression bugs like hell. Again, the only solution is to bake.

So basically you want to have the best of both worlds: switch between live connections or a baked one.

First of all, I modified Tilt's nice switch fuse to work in 3D space, a very straightforward tweak as you just need to change the datatype to "DataType3D" and you're all set. So I figured that a macro with two 3d transforms and a 3D switch under the hood could do it. The basic idea works, but I'm stuck at baking the live transform to another transform while keeping the live one untouched. All this at the push of a button on the macro.

Any ideas on how to do that? Baking transform A to transform B?

I was working on said prototype to demonstrate what I'm on about, but helas, neither do I have proper renders with metadata at hand and apparently the syntax has changed since 6.4 and I'm stuck at home with a free version of Fusion 7.6

User avatar
SecondMan
Site Admin
Posts: 3239
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 1
Location: Vancouver, Canada
Been thanked: 51 times
Contact:

Re: Extract camera from EXR metadata

#5

Post by SecondMan » Tue Jan 20, 2015 9:30 am

Kristof wrote:I'm stuck at baking the live transform to another transform while keeping the live one untouched. All this at the push of a button on the macro.

Any ideas on how to do that? Baking transform A to transform B?
Could you make a straight copy of the live one in your script and then work from the copy to bake B?

User avatar
SecondMan
Site Admin
Posts: 3239
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 1
Location: Vancouver, Canada
Been thanked: 51 times
Contact:

Re: Extract camera from EXR metadata

#6

Post by SecondMan » Tue Jan 20, 2015 10:02 am

Kristof wrote:Downside, it can be slow--depending on the size of the files--and you won't have motion blur.
I wonder if you couldn't generate a vector pass from those metadata values and use vector blur?

User avatar
Kristof
Fusionista
Posts: 493
Joined: Thu Aug 07, 2014 1:30 pm
Been thanked: 10 times

Re: Extract camera from EXR metadata

#7

Post by Kristof » Tue Jan 20, 2015 11:41 am

SecondMan wrote:
Kristof wrote:Downside, it can be slow--depending on the size of the files--and you won't have motion blur.
I wonder if you couldn't generate a vector pass from those metadata values and use vector blur?
Hmmm, don't think so. Because you need to know the screen coordinates a 3d position occupies in screen space from one frame to the other. So it is a combination of factors. Not just the camera motion only.

I did play around with displacing an image plane using the WPP and then generate vectors that way (I did bake the cam in that test too).

User avatar
SecondMan
Site Admin
Posts: 3239
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 1
Location: Vancouver, Canada
Been thanked: 51 times
Contact:

Re: Extract camera from EXR metadata

#8

Post by SecondMan » Tue Jan 20, 2015 12:24 pm

Yes, I was thinking of camera blur only.

Which could still be useful, mind you.

User avatar
French_Fry
Fusionista
Posts: 322
Joined: Tue Aug 05, 2014 8:04 am
Location: Nashville, TN
Contact:

Re: Extract camera from EXR metadata

#9

Post by French_Fry » Thu Feb 19, 2015 10:07 am

The devs over at Redshift 3D are looking into implementing exr metadata for their renderer. They had a few technical questions regarding camera matrix / FOV handling in regards to comp apps. If there are any redhsift / Fusion users that want to chime in and help push things forward, you can find the thread here:
https://www.redshift3d.com/forums/viewthread/2603/

User avatar
Kristof
Fusionista
Posts: 493
Joined: Thu Aug 07, 2014 1:30 pm
Been thanked: 10 times

Re: Extract camera from EXR metadata

#10

Post by Kristof » Tue Feb 24, 2015 2:36 am

Kristof wrote:Is this broken in Fu 7.6? Looks that way as I'm no longer able to use the setup I made in 6.4.
Works again in 7.7. So another entry to that changelog :)

User avatar
French_Fry
Fusionista
Posts: 322
Joined: Tue Aug 05, 2014 8:04 am
Location: Nashville, TN
Contact:

Re: Extract camera from EXR metadata

#11

Post by French_Fry » Fri Mar 20, 2015 8:14 am

I was wondering if someone could nudge me in the right direction.
I'm trying to pull the metadata from a Redshift render for my camera setup. I have the exr file piped into the camera,and I'm setting up the expressions to access the data. I have the FOV working because it's just one number, but I can't figure out how to get any of the transforms working. It's a string of 16 different numbers for the RSCameraTransform, but I can't figure out the expression to pull them.
Any help, advice would be appreciated! thanks! :)

RS_cameraMetaData.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
Chad
Fusionator
Posts: 1362
Joined: Fri Aug 08, 2014 1:11 pm
Been thanked: 8 times

Re: Extract camera from EXR metadata

#12

Post by Chad » Fri Mar 20, 2015 10:09 am

The transform is likely a transform matrix, accessed as a table. Do you know what format Redshift is using for the matrix? Like the standard 3ds max EXR output would use [13,14,15] for XYZ translation. Rotation you need to calculate, like for X it would be something like iif(cos(-asin(ImageInput.Metadata.cameraTransform[3])) ~=0, deg(atan2(ImageInput.Metadata.cameraTransform[7],self.ImageInput.Metadata.cameraTransform[11])), deg(atan2(ImageInput.Metadata.cameraTransform[5],self.ImageInput.Metadata.cameraTransform[6]))). It depends on the format of the transform matrix though.

User avatar
French_Fry
Fusionista
Posts: 322
Joined: Tue Aug 05, 2014 8:04 am
Location: Nashville, TN
Contact:

Re: Extract camera from EXR metadata

#13

Post by French_Fry » Fri Mar 20, 2015 2:50 pm

Thanks for responding Chad...I think you got me though! :) I'm in over my head... I can check with the Redshift guys, they're usually pretty responsive, but honestly, I don't think I'll know what to do with the answer. :)
I'll be back if I get a lead!

User avatar
Chad
Fusionator
Posts: 1362
Joined: Fri Aug 08, 2014 1:11 pm
Been thanked: 8 times

Re: Extract camera from EXR metadata

#14

Post by Chad » Sat Mar 21, 2015 7:53 am

Wow, going back to my archives, I did this in 2009. Hope this works. Anyway, give it a shot, if there's a problem it's most likely one of two things... They either transpose the matrix or they have a different coordinate system.

Code: Select all

{
	Tools = ordered() {
		Transform3D1 = Transform3D {
			CtrlWZoom = false,
			Inputs = {
				SceneInput = Input {
					SourceOp = "Camera3D1",
					Source = "Output",
				},
				["Transform3DOp.Rotate.X"] = Input { Value = -90, },
				["Transform3DOp.PivotNest"] = Input { Value = 1, },
			},
			ViewInfo = OperatorInfo { Pos = { -220, 181.5, }, },
		},
		Camera3D1 = Camera3D {
			NameSet = true,
			Inputs = {
				["Transform3DOp.Translate.X"] = Input {
					Value = 93.1510467529297,
					Expression = "ImageInput.Metadata.cameraTransform[13]",
				},
				["Transform3DOp.Translate.Y"] = Input {
					Value = 62.9016571044922,
					Expression = "ImageInput.Metadata.cameraTransform[14]",
				},
				["Transform3DOp.Translate.Z"] = Input {
					Value = 54.0075798034668,
					Expression = "ImageInput.Metadata.cameraTransform[15]",
				},
				["Transform3DOp.Rotate.X"] = Input {
					Value = 64.335947857655,
					Expression = "iif(cos(-asin(ImageInput.Metadata.cameraTransform[3])) ~=0, deg(atan2(ImageInput.Metadata.cameraTransform[7],self.ImageInput.Metadata.cameraTransform[11])), deg(atan2(ImageInput.Metadata.cameraTransform[5],self.ImageInput.Metadata.cameraTransform[6])))",
				},
				["Transform3DOp.Rotate.Y"] = Input {
					Value = -8.53773646251594e-007,
					Expression = "deg(-asin(ImageInput.Metadata.cameraTransform[3]))",
				},
				["Transform3DOp.Rotate.Z"] = Input {
					Value = 124.02980571777,
					Expression = "iif(cos(-asin(ImageInput.Metadata.cameraTransform[3])) ~=0, deg(atan2(ImageInput.Metadata.cameraTransform[2],self.ImageInput.Metadata.cameraTransform[1])),0)",
				},
				["Transform3DOp.PivotNest"] = Input { Value = 1, },
				PerspNearClip = Input { Value = 10, },
				PerspFarClip = Input { Value = 900, },
				PerspAdaptiveClip = Input { Value = 0, },
				AovType = Input { Value = 1, },
				AoV = Input { Expression = "ImageInput.Metadata.cameraFov", },
				FLength = Input { Value = 11.5743433714072, },
				PlaneOfFocus = Input { Value = 150, },
				["Stereo.EyeSeparation"] = Input { Value = 5.2596685, },
				["Stereo.ConvergenceDistance"] = Input { Value = 150, },
				FilmGate = Input { Value = FuID { "HD", }, },
				ApertureW = Input { Value = 0.3775, },
				ApertureH = Input { Value = 0.2123, },
				ResolutionGateFit = Input { Value = FuID { "Inside", }, },
				IDepth = Input { Value = 900, },
				["SurfacePlaneInputs.SubdivisionWidth"] = Input { Value = 1, },
				["SurfacePlaneInputs.ObjectID.ObjectID"] = Input { Value = 1, },
				["MtlStdInputs.MaterialID"] = Input { Value = 1, },
				ProjectionMode = Input { Value = 2, },
				Comments = Input { Value = "Pulls Transform from the input image metadata.\r\n\r\nThe transform matrix is Z up, not Y up, so a 3Xf rotating in X by -90 can be used after the 3Cm to compensate.\r\n\r\nThe FoV in the EXR metadata is Horizontal", },
			},
			ViewInfo = OperatorInfo { Pos = { -715, 181.5, }, },
		},
	},
	ActiveTool = "Transform3D1",
}

User avatar
svenneve
Fusioneer
Posts: 92
Joined: Fri Sep 12, 2014 1:47 am
Been thanked: 1 time
Contact:

Re: Extract camera from EXR metadata

#15

Post by svenneve » Mon Mar 30, 2015 9:59 am

I might have to give this a try, as our old fuse and script (and others) depends on Matrix4 , which is broken in Fusion7.

Thanks for posting