Automating Fusion via Houdini TOPs

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Automating Fusion via Houdini TOPs

#1

Unread post by AndrewHazelden »

If you're a Fusion TD* who has a copy Fusion Studio/Fusion Render Node and Houdini Apprentice/Indie installed on your system, you might enjoy working your way through the following workflow guide. You will then be able to make your own wacky "Rube Goldberg" automation machines like never before:

Render Fusion Comps in Houdini TOPs

Part 1 Overview

The following guide lists the essential steps required to set up a new Houdini TOPs "Task" that renders a folder of Fusion .comp files. This rendering process is done using command-line approaches that rely on Blackmagic Design's Fusion Studio and the included Fusion Render Node executable.

Houdini TOPs based node graphs provide an exciting new way to tame complex workflows that require multiple linked dependent tasks. It has the potential of being the ultimate workflow automation "glue" to unify post-production tasks for processes running across artist workstations, local render farms, cloud computing systems, and even for assisting with general purpose task scheduling.

Image

Part 2 Overview

Part 2 will expand on the initial concepts presented, and explain at an overview level, how it is possible to create several more TOPs nodes that will remotely control a Fusion Studio compositing session using FuScript and the "fusion:DoAction()" and "comp:DoAction()" functions. This is an interesting concept as it allows you to run Fusion actions external to the app.

Image

Resolve Support
*The same base concepts can also be used to automate Resolve Studio workflows from Houdini TOPs via FuScript too... if you specify the Resolve SubType when connecting to the app using FuScript, and access the Resolve API functions via Lua or Python scripting. :cheer:
Last edited by AndrewHazelden 3 years ago, edited 1 time in total.

User avatar
UserNoah
Fusionista
Posts: 294
Joined: 5 years ago
Been thanked: 1 time
Contact:

Re: Automating Fusion via Houdini TOPs

#2

Unread post by UserNoah »

Andrew! :shock: This is amazing!

This should also make it possible to use the Deadline scheduler to send Resolve into the Deadline queue. A feature that I'm missing (it's probably already possible but I can't seem to understand how to create real submission scripts for Deadline).

Thank you very much for writing this introduction!

User avatar
UserNoah
Fusionista
Posts: 294
Joined: 5 years ago
Been thanked: 1 time
Contact:

Re: Automating Fusion via Houdini TOPs

#3

Unread post by UserNoah »

I've played around with Resolve scripting and TOPs. I’ve been less successful on my actual goal but now I have a crude render manager for resolve. Providing the ability to start renders from several projects. A functionality I actually missed a few weeks back when having to render several longer clips from different projects.

Starting a render wasn't particularly easy since as far as I know, Resolve does not feature a true command line renderer where you specify a file and it renders.
I don't actually think this is possible due to the Database approach.
Instead, you need to find and open a project, find the render jobs and start rendering.
For the rendering, I copied several functions out of the scripting examples that come with resolve. They ensure that the script will wait on Resolve to finish rendering before starting the next render job. Without this waiting, I experienced short freezes and ultimately it would error out on the next project to render. This is definitely something to keep in mind when creating your own Resolve render scripts.

I also haven't found a way to have an external script use any of the attributes that you can set in TOPs.
I assume this needs to be split up into individual TOPs.
But the intermediate result right now is this:

Disclaimer: This is very much alpha and while functional I wouldn't rely on this for now.


Create a renderlist:
Image

Optional TOPs network:
Image
Image
Image
Image


It's essentially just starting a script that also works on its own. You don't need to start it through TOPs. The script will read a csv list of projects you have specified and render all render jobs in these projects that haven't been rendered yet.

In theory, this should be platform agnostic but I haven’t tested this on any other machine than my own windows pc.

Usage:

First, you need to add these scripts to resolve like any other script. Place them in your preferred userdata path. I placed them under the “Deliver” folder.

Then you need to start the script Edit_Renderlist.
Image

This script features a UI that allows you to add and remove projects to a csv list. This is being created at the default userdata path. You can check the location in the console when the script is opened.
Deleting entries can be buggy for now so you might need to double click twice.

Save the list.

Next, you can start the second script called Resolve_Render_From_ProjectList
Either start it through Resolve, the fuscript command line (or TOPs which is just starting through the command line, too)

This script does not employ any UI.
It will simply go through the projects it finds in the Renderlist file and renders the desired render jobs.

As of right now this only works in the last opened database. It’s definitely possible to take databases into account but that’s a feature that I would add if I personally need it or if there is a general interest in a script like this.

More information:
  • This doesn't work with my initial goal of using the deadline TOPs scheduler. But not because of any error in the script, the deadline scheduler doesn't manage to open Resolve in the first place.
  • The TOPs functionality, in general, is limited. My example shows how to launch a script but as it is right now, you don't have any benefits to simply starting this in Resolve. (If you use hqueue this might be a different story)
  • I chose a csv list compared to a simple textfile because I initially hoped to use this file with the CSV Import TOP. At least now there is an example of how to write a csv using Lua/Fusion.
  • The TOPs network is very crude. Starting Resolve, waiting 20 seconds, and then starting fuscript to start the script.
  • I imagine it would be easier to use the same or a similar script to submit this to deadline using a custom submitter script but as mentioned above I can't seem to wrap my head around the deadline documentation for now.
  • The final function of quitting Resolve is disabled in the script. This is simply because for testing we probably don't want to open Resolve all the time
  • The code is a mess. I need to read some "best practices" the next time. But the scripts are functional.
  • I'm all ears if somebody has ideas to improve this!
Big thanks to you @AndrewHazelden. Your initial post made me look into Resolve scripting for the first time and I've learned a bunch! Also, controlling Fusion through TOPs will come in handy!

Posts I've used:
Resolve Scripting Essentials
Building GUIs With Fusion's UI Manager
UITree method FindItems() is broken [FIXED]

Edit_Renderlist.lua
Resolve_Render_From_ProjectList.lua
You do not have the required permissions to view the files attached to this post.
Last edited by UserNoah 3 years ago, edited 2 times in total.

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: Automating Fusion via Houdini TOPs

#4

Unread post by AndrewHazelden »

Wow! You've really expanded on the idea and done some neat scripting work there. Very cool.

Added in 2 minutes 45 seconds:
I've been meaning to dig into Resolve's "Workflow Integration" scripting features, along with the ability to make custom delivery page exporter plugins. Those extra layers of customization could take things to a better level of control since the Resolve project rendered results could then check themselves into ftrack or Shotgun automatically.

My long term goal is to try and make a Fusion Server and Resolve Server like bi-directional integration node for TOPs like Nuke and Maya have.

Examples from docs:
https://www.sidefx.com/docs/houdini/nod ... erver.html

https://www.sidefx.com/docs/houdini/nod ... erver.html

User avatar
cdordelly
Posts: 22
Joined: 7 years ago

Re: Automating Fusion via Houdini TOPs

#5

Unread post by cdordelly »

Thanks a lot for this guide Andrew!

I have some problems with the exercise of Part Two, I hope that you can help me to find out what I'm doing wrong or I don't know if probably now there it can be some problems with whits workflow using H19.5 + F18.

You can find the .hip file that I'm using here:
fusion_encoder_top_v02_basic.rar
houdini_zvDW5hdHoL.png
The first problem is that the .comp file is not saved by the "Comp_Save" action, and after executing the graph and waiting for a few seconds or by doing something like selecting another node, my Houdini session crashed.

If I try to execute the "blur" example that you shared on the other thread, I receive this message:

Code: Select all

[string "???"]:1: attempt to index global 'fu' (a nil value)
You mentioned that probably that message can appear if the 'fuscript.exe' is blocked by the firewall but I have allowed everything from Fusion:
mmc_5DtbuBJQmz.png
In case the attached .hip file works on your end, can you suspect what other thing I missed here? I have almost 0 experience with the technical part of Fusion, unfortunately.


The other interesting thing that I discover from Part 1, is if I execute the render from one .comp file where the Saver node will save a .mov file with ProRes 4444 XQ, if I use the Fusion Render Node for the process, the .mov will be exported using DNX Codec, but if I target the common fusion.exe GUI on the Generic Generator, the .mov is saved correctly as ProRes 4444 XQ, maybe the Fusion Render Node can't render ProRes Codecs?

Thanks a lot again for the guide and for your help.

Cheers!
You do not have the required permissions to view the files attached to this post.

User avatar
Midgardsormr
Fusius Of Borg
Posts: 2848
Joined: 10 years ago
Location: Los Angeles, CA, USA
Real name: Bryan Ray
Mood:
Has thanked: 1 time
Been thanked: 19 times
Contact:

Re: Automating Fusion via Houdini TOPs

#6

Unread post by Midgardsormr »

I can't speak to most of it, but your guess about ProRes is correct: The license only covers interactive sessions because render nodes are free. If BMD charged for render nodes, maybe something could be worked out, but under their model, there's no way Apple would agree.

User avatar
AndrewHazelden
Fusius Of Borg
Posts: 2598
Joined: 10 years ago
Location: West Dover, Nova Scotia, Canada
Has thanked: 2 times
Been thanked: 7 times
Contact:

Re: Automating Fusion via Houdini TOPs

#7

Unread post by AndrewHazelden »

cdordelly wrote: 2 years agoI have some problems with the exercise of Part Two, I hope that you can help me to find out what I'm doing wrong or I don't know if probably now there it can be some problems with whits workflow using H19.5 + F18.
Hi @cdordelly,

Thanks for checking out the Houdini TOPs + Fusion workflow efforts.

I suspect the issue might be related to either pre-existing environment variables from the Houdini Hython session causing problems, or the FuScript scriptapp binding process might have gotten tripped up if any two of Fusion Studio, Fusion Render Node, and Resolve Studio were running at the same moment. In the time since this thread was originally created I've done a little more explorations into scriptapp based UUID and sub-type usage for more precise inter-app communication, thanks in part to reference information shared by WSL's @danelll.

I'm on a holiday schedule at the moment so it might take me up to a ~week to get the time available to try out your RAR attachment and try to track down the issue. I am interested in seeing people's use of these workflows so am very encouraged by your explorations. Thanks for sharing your notes, bug report, and a glimpse into your efforts with Fusion + Houdini. :)