Welcome to WSL!

Make yourself at home, but before posting, please may I ask you to read the following topics.


Posting 101
Server space, screenshots, and you

Thank you!

PS. please pretty please:


Image

Batch loader file path change? (Or alternatives)

Moderator: SecondMan

User avatar
jedmitchell
Fusioneer
Posts: 54
Joined: Mon Feb 27, 2017 5:11 pm
Been thanked: 2 times

Batch loader file path change? (Or alternatives)

#1

Post by jedmitchell » Sat May 09, 2020 11:33 am

Pretty sure I already know the answer is "suck it up, do it by hand" but is there a way to change many loader file paths at the same time?

This feels like a tale as old as Fusion, but I'm doing some CG comps with many, maaaaaany versions that all use roughly the same node tree and I'm manipulating individual lighting channels, so each shot might have a dozen loaders feeding different passes into the comp. The hos_splitEXR script is a fine way to start that process, but when I have to do it for 40 more instances of the same tree, changing each one of a dozen loaders with each version...

Tried the batch change parameter script in Reactor but that doesn't appear to work with file paths (or at least I can't find them as an option in the dropdown. I guess that's not surprising considering you also can't manipulate the file path with expressions (from what I know / have tried).

So is there any way to do this in Fusion that I'm not thinking of? Or just a better way to handle EXRs with lots of custom channels? Like a secret Channel Booleans+ node I don't know about that recognizes custom channels? In the past I've just tolerated doing this by hand but there are so many loaders in this job I feel like I'm going to throw my computer out the window (or sell all my possessions and rent a Nuke license in spite).

Edit: Actually, tried some different search terms here on the forum found this thread from Brian: viewtopic.php?f=6&t=1854

That might contain enough information to figure it out but holy shit that's a lot to go through just to come out the other side with some basic functionality...

User avatar
Midgardsormr
Fusionator
Posts: 1757
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 14
Location: Los Angeles, CA, USA
Been thanked: 89 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#2

Post by Midgardsormr » Sat May 09, 2020 3:03 pm

Ha! Yeah, that's a pretty elaborate solution. It's turned into a great and useful script, and I've even extended it to police comps for unused renders and prepare for archival. But there's a much more straightforward script already available to you: In the Eyeon Essentials script collection on Reactor, you'll find the Change Paths script, which does a simple search-and-replace on Loaders and/or Savers.

User avatar
Kristof
Fusionista
Posts: 760
Joined: Thu Aug 07, 2014 1:30 pm
Answers: 2
Been thanked: 15 times

Re: Batch loader file path change? (Or alternatives)

#3

Post by Kristof » Sun May 10, 2020 10:16 am

Nah, you don't need to that by hand. That's what a composting pipeline is for. Fusion, Nuke, whatever. But depending on how dynamic you want it all, you can get by even without any code. Published files, proper naming convention and filepaths relative to the comp.

This question keeps popping up and somehow people think that Fusion is the limitation. Or makes your life difficult. That's not the case.

User avatar
jedmitchell
Fusioneer
Posts: 54
Joined: Mon Feb 27, 2017 5:11 pm
Been thanked: 2 times

Re: Batch loader file path change? (Or alternatives)

#4

Post by jedmitchell » Sun May 10, 2020 12:06 pm

Midgardsormr wrote:
Sat May 09, 2020 3:03 pm
In the Eyeon Essentials script collection on Reactor
Oh thanks for that -- I didn't have that atom installed so didn't realize what was in it. For whatever reason that didn't come up when I searched the forums so I thought Batch Change Parameters was my best option.
Kristof wrote:
Sun May 10, 2020 10:16 am
This question keeps popping up and somehow people think that Fusion is the limitation. Or makes your life difficult. That's not the case.
Part of it is just frustration talking (which I should be more upfront about), but I think Fusion is the problem sometimes.

Looking over the more advanced experiments posted to WSL it's clear that (almost) anything can be done in Fusion, but for even moderately advanced users the process is often onerous, poorly documented and doesn't feel like it went through UX refinement. I don't have much trouble maintaining a small freelancer's pipeline in Houdini, used to do so in Nuke / Hiero, and in a former life I implemented Maya pipeline tools for a mid-size studio.

That's not to say I have some kind of privileged point of view or that I know better than you guys do -- I don't -- but I've been banging away at Fusion long enough and frequently enough to feel like there are some real differences. There's lots to learn with all those tools but the starting point for learning is often easy to find and the concepts are generally transportable from one situation to another. I find that to be much less true in Fusion.

The most modern, well thought out & best documented tools in Fusion seem to come from the small group of experts here on WSL (by the way @Kristof, thank you for the cryptomatte plugin!). That's not a good sign to me. User created tools are a huge part of the industry but Fusion has a smaller user base, generally smaller customers, and a price point an order of magnitude lower than it's competition. None of that makes me think it would see the same level of development or documentation, but knowing the source of the problem hasn't made my experience of it any more pleasant.

User avatar
Kristof
Fusionista
Posts: 760
Joined: Thu Aug 07, 2014 1:30 pm
Answers: 2
Been thanked: 15 times

Re: Batch loader file path change? (Or alternatives)

#5

Post by Kristof » Sun May 10, 2020 12:58 pm

I can totally understand the frustration. The fragmented pieces of information and examples scattered around doesn't make it easy. Your reply does accurately sum up how it often feels like.

I'm no expert either and only recently did I learn about how to parse fusion comps outside of a Fusion instance, using Python or Lua. Which is a very cool feature if you want to modify comps quickly.
jedmitchell wrote:
Sat May 09, 2020 11:33 am
I guess that's not surprising considering you also can't manipulate the file path with expressions (from what I know / have tried).
You can't, as you discovered. If you run ==YourLoader.Clip:GetAttrs() you'll see that INPB_External equals "false", which means you're not allowed to. I modified @AndrewHazelden's GetFrame fuse to allow it so you might want to look at that alternative loader to experiment with building filepaths.

These fuses really help:

https://github.com/Spicy-Acorn/fusiontext

User avatar
Midgardsormr
Fusionator
Posts: 1757
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 14
Location: Los Angeles, CA, USA
Been thanked: 89 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#6

Post by Midgardsormr » Sun May 10, 2020 3:00 pm

jedmitchell wrote:
Sun May 10, 2020 12:06 pm
... so I thought Batch Change Parameters was my best option.
FWIW, I'd like to see if I can overcome that limitation in Batch Change Parameters, too. It's unintuitive that it doesn't work. I fell into the same trap just a few weeks ago, and I wrote the danged thing! (Well, rewrote it; I didn't build it from scratch.)

User avatar
thomastraum
Posts: 10
Joined: Sat May 02, 2020 4:16 pm

Re: Batch loader file path change? (Or alternatives)

#7

Post by thomastraum » Sun May 10, 2020 3:05 pm

Kristof wrote:
Sun May 10, 2020 10:16 am
you can get by even without any code. Published files, proper naming convention and filepaths relative to the comp.
Would love to have more info on this how you would set this up on a basic level. Maybe asking for too much but even jsut a pointer would help

User avatar
jedmitchell
Fusioneer
Posts: 54
Joined: Mon Feb 27, 2017 5:11 pm
Been thanked: 2 times

Re: Batch loader file path change? (Or alternatives)

#8

Post by jedmitchell » Tue May 12, 2020 12:52 am

Kristof wrote:
Sun May 10, 2020 12:58 pm
only recently did I learn about how to parse fusion comps outside of a Fusion instance, using Python or Lua
Yeah, isn't that cool? I did that for some dumb tools I made a few years ago for turning clone strokes into alpha mask strokes. Wound up bundling it into an internal python script eventually instead but it's neat you can do it that way. Unfortunately the scripting reference for strokes was so vague I figured it out largely by trial and error, but it got done...

Anyway, I've been playing with the Eyeon "change paths" script, and I must be missing something because this seems to have a couple of pretty crippling limitations.

+ It seems to replace *all* loaders in a comp that match a given search string (probably because it's just parsing the comp file in plain text, not the active state?), which renders it useless when I'm trying to copy a node tree then change just the copied instances and leave the prior version intact. Am I missing something there? Selecting the target nodes seems to have no effect.

+ It doesn't give the option to preserve loader trim / extents settings the way a manual file path change does, so if I've got a shorter frame range than the copy source (i.e. I'm doing a limited test run of frames from Houdini to check lighting changes before committing to the full render) I lose my in/out points.

+ The lack of a file browser interface is also pretty aggravating, you could really just do this yourself in Notepad++. It's kind of a moot point if the above are true limitations though.

...eh, people should probably ignore the rest of this as it is completely unproductive but I'm annoyed and posting on the official BMD Fusion forum is like talking to a brick wall:

My needing to dig through barely documented scripting references to get some alpha channels from a paint node or spending 2 days on the forums getting the best help from extremely knowledgeable users only to realize there's no good way to change multiple loader paths? That's not great.

I guess this is why I get frustrated with BMD: these aren't exotic tasks here, this is basic usability. There is a healthy debate to be had about where responsibility falls in determining the power of DCC software: does it lie with the developers in creating the platform & the tools on it? Or the users in finding ways to build on that platform in new ways that push the limits? People will draw that line in different places, but I put the responsibility for things at this low of a level on the developer's side of that relationship, and it's a pretty consistent shortcoming of Fusion.

Fusion has a lot of power, but so does a Ferrari with the seats installed backwards...

Anyway, that's not the fault or responsibility of anyone here -- I appreciate the help you guys give everyone on WSL. Sometimes I think WSL collectively knows more about Fusion than BMD does.

User avatar
theotheo
Fusionista
Posts: 347
Joined: Thu Aug 07, 2014 8:35 am
Answers: 2
Been thanked: 8 times

Re: Batch loader file path change? (Or alternatives)

#9

Post by theotheo » Tue May 12, 2020 2:39 am

Sometimes I think WSL collectively knows more about Fusion than BMD does.
When it comes to use case scenarios absolutely. No doubt about it. I think thats the main reason why WSL exists.

User avatar
Midgardsormr
Fusionator
Posts: 1757
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 14
Location: Los Angeles, CA, USA
Been thanked: 89 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#10

Post by Midgardsormr » Tue May 12, 2020 7:31 am

jedmitchell wrote:
Tue May 12, 2020 12:52 am

Anyway, I've been playing with the Eyeon "change paths" script, and I must be missing something because this seems to have a couple of pretty crippling limitations.
Rather than complaining about it, could you please phrase it as constructive criticism? :) Although the original script was distributed by Eyeon, maintenance has since been taken over by yours truly. I'm happy to make updates, but I'm less motivated to do so if it doesn't come with some sugar.
+ It seems to replace *all* loaders in a comp that match a given search string (probably because it's just parsing the comp file in plain text, not the active state?), which renders it useless when I'm trying to copy a node tree then change just the copied instances and leave the prior version intact. Am I missing something there? Selecting the target nodes seems to have no effect.
That's a fairly easy fix. If you're in a hurry, open the script and replace line 108 with the following:

Code: Select all

toollist = composition:GetToolList(true)
if table.getn(toollist) < 1 then toollist = composition:GetToolList(false) end
If you have any nodes selected, it will only operate on those nodes, but if you have zero nodes selected, it will operate on every Loader and Saver. It's a bit less intuitive than the global modification for most users, so the next version in Reactor will probably have a check box to choose the mode.
+ It doesn't give the option to preserve loader trim / extents settings the way a manual file path change does, so if I've got a shorter frame range than the copy source (i.e. I'm doing a limited test run of frames from Houdini to check lighting changes before committing to the full render) I lose my in/out points.
That's a bug. I thought I'd fixed it, but I may have to do some more testing. Which version of Fusion are you using? The process is a little fiddly and delicate, and it wouldn't surprise me if BMD broke something in an update.
+ The lack of a file browser interface is also pretty aggravating, you could really just do this yourself in Notepad++. It's kind of a moot point if the above are true limitations though.
I'll need a user story for this one. How would that work in the context of this script?

User avatar
jedmitchell
Fusioneer
Posts: 54
Joined: Mon Feb 27, 2017 5:11 pm
Been thanked: 2 times

Re: Batch loader file path change? (Or alternatives)

#11

Post by jedmitchell » Tue May 12, 2020 11:42 am

Midgardsormr wrote:
Tue May 12, 2020 7:31 am
Rather than complaining about it, could you please phrase it as constructive criticism? :) Although the original script was distributed by Eyeon, maintenance has since been taken over by yours truly. I'm happy to make updates, but I'm less motivated to do so if it doesn't come with some sugar.
Hey sorry about that -- I wasn't intending to criticize, only to say that I must either be missing something or that maybe it just isn't built to do what I'm looking for. Doesn't make it a bad tool, just possibly not the solution to this. I apologize that it came off sounding critical of the author.
Midgardsormr wrote:
Tue May 12, 2020 7:31 am
That's a fairly easy fix. If you're in a hurry, open the script and replace line 108 with the following:

Code: Select all

toollist = composition:GetToolList(true)
if table.getn(toollist) < 1 then toollist = composition:GetToolList(false) end
If you have any nodes selected, it will only operate on those nodes, but if you have zero nodes selected, it will operate on every Loader and Saver. It's a bit less intuitive than the global modification for most users, so the next version in Reactor will probably have a check box to choose the mode.
Checkbox sounds like a good idea there, but this will work in a pinch and it's easy to modify. Thanks!
Midgardsormr wrote:
Tue May 12, 2020 7:31 am
That's a bug. I thought I'd fixed it, but I may have to do some more testing. Which version of Fusion are you using? The process is a little fiddly and delicate, and it wouldn't surprise me if BMD broke something in an update.
I'm on 16.2 right now.
Midgardsormr wrote:
Tue May 12, 2020 7:31 am
I'll need a user story for this one. How would that work in the context of this script?
The user experience I imagine would be having two new UI elements: a toggle on the search pattern field to disable it (setting the scope to all selected loaders) + a "Browse" button on the replacement path field to allow you to use the standard file browser to navigate to your new path. I've got files on different drives with different root paths below the "job" directories so switching from one drive to another involves typing in the path from the root. Not impossible, just a bit cumbersome. Actually, maybe a "Browse" button on both the search & replace sides, sometimes it's faster to just grab a path from the recent folders in the browser than copy/paste from the inspector.

Manually entering a partial path seems like it has additional utility for changing, say, just a version number, but I would consider a browser option if you're planning to do more work on it.


Just to make sure it's clear because I know tone is hard to get across in ranty forum text: I think Reactor & everything in it is a gift of immeasurable value. Without your contributions Fusion would probably degenerate to nothing but Resolve's title tool (which I suspect is close to what BMD is really interested in it for).

The future of Fusion as a studio tool is defined by this community, so I'm always grateful to you guys and I apologize if it sounds otherwise. I'm just frequently pissed at BMD for the lack of base level improvements since buying Eyeon & for painting themselves into a corner with software pricing that deprioritizes more complex use cases.

Added in 24 minutes 46 seconds:
Oh, one more thing I noticed about the script that I don't really understand:

After using it to change some paths, if I subsequently try to change one of them manually using the "Browse" button it doesn't work the first time I do it. The browser opens, I select a new file path, and the text in the Loader field updates but the output to the viewer is the old image. If I do the same thing a 2nd time it works.

User avatar
Midgardsormr
Fusionator
Posts: 1757
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 14
Location: Los Angeles, CA, USA
Been thanked: 89 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#12

Post by Midgardsormr » Tue May 12, 2020 1:06 pm

I don't think that has anything to do with the script, necessarily, although it's possible that the script sets up an environment for it to happen. Fusion sometimes seems to have a hard time letting go of Loader caches. I've noticed that with our comp generation script, too, but I haven't taken the time to really investigate it to see if I can figure out what's happening.

Anyway, I'm hopeful that I'll have a chance to make the updates this week sometime, but there are a few other things in our pipeline that are higher priority.

User avatar
Movalex
Sir Requestalot
Posts: 228
Joined: Fri Nov 03, 2017 5:36 am
Answers: 3
Been thanked: 36 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#13

Post by Movalex » Tue May 12, 2020 2:43 pm

I use this script from time to time. it definitely deserves complete rewrite at least to a new UI.
Until then, I just added a couple of new features to it:
  • It has a checkbox to process only selected nodes
  • Search/replace GeoLoaders paths
  • Pattern search option
With pattern search you can replace part of the filename or completely reconstruct the name with wildcards. After all, this is just a string replacement, you can go wild here.
Say, you need to change the drive letter, then take version number and put it in front of the file name:
D:/RAID/company/comp_name/prerenders/v023/company_cheese_mask_0001.exr
So you put in search field:
^D:(.*)/(v%d+)/(.*)$
In replace put:
E:%1/%2/%2_%3
And you'll get:
E:/RAID/company/comp_name/prerenders/v023/v023_company_cheese_mask_0001.exr

Easy-peasy.

Here's the script, check it out
ChangePath.lua
You do not have the required permissions to view the files attached to this post.

User avatar
jedmitchell
Fusioneer
Posts: 54
Joined: Mon Feb 27, 2017 5:11 pm
Been thanked: 2 times

Re: Batch loader file path change? (Or alternatives)

#14

Post by jedmitchell » Tue May 12, 2020 3:00 pm

Midgardsormr wrote:
Tue May 12, 2020 1:06 pm
Fusion sometimes seems to have a hard time letting go of Loader caches.
Hah, yeah that is quite the understatement. Anyway it's not a big deal just thought you should know.

Thanks for the updated script @Movalex!

User avatar
Midgardsormr
Fusionator
Posts: 1757
Joined: Wed Nov 26, 2014 8:04 pm
Answers: 14
Location: Los Angeles, CA, USA
Been thanked: 89 times
Contact:

Re: Batch loader file path change? (Or alternatives)

#15

Post by Midgardsormr » Thu May 21, 2020 4:21 pm

Well, I didn't do anything to Change Paths yet, but I did update Batch Change Parameters to handle the Clip inputs of Loaders and Savers. Beta testing happening here:

viewtopic.php?f=45&t=4131