Welcome to WSL!

New to the forum? Please read this and this.

[RELEASED] [Fuse] AudioWaveform

Where the future is being made, today.

Welcome to the WSL development corner!

In this forum, please post your development projects. You get kudos and feedback here.
Topics ideally have preset prefixes, and this is what they (might) mean:

  • [DEV] - very much work in progress, don't build a business on this, could go anywhere
  • [BETA] - should kinda do what it's supposed to do, please test, give feedback
  • [RC] - this may end up in Reactor soon, polishing up, now's the time for last minute thoughts
  • [ABD] - died a premature death, sadness, will not see the light of day ever (unless someone picks up the scraps)

Once a development project has been released (hurray), topics can be marked as - you guessed it - [RELEASED] :cheer:

Development topics only, please. For generic questions, how-to's, questions and inquiries about existing tools etc, please go to the appropriate other forums.
User avatar
atmosfar
Posts: 17
Joined: Tue Aug 11, 2020 4:13 pm

Re: [RC] [Fuse] AudioWaveform

#76

Post by atmosfar »

Thanks @JiiPii! I had sort of managed to make it work using 2 copies of AudioWaveFrom and masking them. Although I had to use a Merge node for the mask because my Rectangle mask didn't seem to be doing anything directly as the mask input to the AWF node ... maybe a bug?Image

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RC] [Fuse] AudioWaveform

#77

Post by JiiPii »

Hello @atmosfar
the mask in Fusion refers to the source image. I "built" the fuse in such a way that it doesn't need a source image, but then unfortunately the mask function doesn't work either. So a background node with alpha = 0 is enough

Image

User avatar
atmosfar
Posts: 17
Joined: Tue Aug 11, 2020 4:13 pm

Re: [RC] [Fuse] AudioWaveform

#78

Post by atmosfar »

Thanks for the explanation. I used the fuse to make a trailer clip for a podcast :)




User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#79

Post by JiiPii »

@atmosfar Cool - it's always interesting to see what is being done with the fuse. Thanks for that.

I was currently throwing myself into creating DCTL fuses and had just found some great rain toys, so @Shem Namo (LearnNowFX) showed me a toy that I couldn't resist. The challenge was to get the FFT data into the DCTL part. Shadertoy uses a texture for this.
I have now finished the fuse so that you can play around with it and test it.

Image

There is now a Frequency Visualization switch in the spectrum page. When switching on, the generated FFT is displayed by the shader. This can be scaled and there is the option to soften the dynamics. With a switch you can choose a static or dynamic operation. The threshold for this must then be adjusted. Please bear in mind that changes to the parameters in the still image are NOT implemented directly in the DCTL. The timeline must be played for this.
I took the opportunity and added another feature to try out: decompression. The idea is to spread the values ​​apart with a threshold and limit the high values ​​with a highline. In any case, a wide playing field.
As a small by-product, I noticed that a fuse with DCTL code can only be used once in a composition because the fuse loads the same DCTL code into the GPU in its copy. But this can be fixed. The template for this was the FragmentShader by Chris Ridings. You could now use several audio waveforms with frequency visualization in parallel in one comp. Okay - but it doesn't happen that often.
I wish you a lot of fun and, as always, look forward to your feedback
AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
Shem Namo
Fusionista
Posts: 677
Joined: Sun Oct 06, 2019 9:15 pm
Answers: 1
Location: North Israel
Real name: David Kohen
Been thanked: 8 times

Re: [RELEASED] [Fuse] AudioWaveform

#80

Post by Shem Namo »

Wow @JiiPii, this is really cool!!
You managed to take one of the best fuses ever written and make it even better!!

The fuse didn't work on OpenCL, but don't feel bad about it.
Since you compiled on CUDA, it's very hard to know if it will work on OpenCl and Metal.
Even if you know all 3 APIs inside out and outside in, DCTL will always find a way to surprise you. :)

There were only 3 issues:
1. line 187 was missing an f after a floating point number.
2. Apparently, DCTL running on OCL doesn't support static float values(Who knew? :oops: ).
3. This is the most important one, in DCTL you can't create a function with __DEVICE__ if that function already exists.
In this case you made a function called sign(), that function already exists in DCTL, even though it doesn't really work.

The only way to change a function that already exists is with #define, like we do with fract().

So that basically leaves you 2 options, you can either make a function with __DEVICE__ and give it a different name,
or you can do something like this:
  1. #define sign(a) if(a>0.0f){ a = 1.0f;} elseif(a < 0.0f){ a = -1.0f;}

I didn't try the second one to see if the syntax works, but in theory it should work fine and should be slightly faster than the first option.

Here's a version that works on OpenCL, but I'm not sure if it will work on CUDA.

Thanks again for this great fuse Gerald,
It's really useful and fun to use!!

Cheers,
David
AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#81

Post by JiiPii »

@Shem Namo Thank you very much for your feedback. That helps me a lot to get better. I had problems with the define at one point and so I switched to the function for the sake of simplicity, but I actually like the define solution better. It's a shame that OpenCl doesn't support static variables. The feature is really cool. The changes also run under Cuda. For the static variables I would have to query the current system and adapt the DCTL code. That should be possible.
Is there a good site on WebGL / OpenGL where you can learn about lod (level of design), dFdx / dFdy, fwidth and the emulation of mat3 / mat4? I've already worked a bit with mat3, but if there was a small emulation library it would be great. Totally cool that the fuse is fun.
all the best
Gerald

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#82

Post by JiiPii »

I've implemented the changes and the improved workaround for the memory leak. The DCTL code contains a query for Cuda so that the static variables do not result in an error.

all the best
AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#83

Post by JiiPii »

DR17 / Fusion17 creates an ugly "return stripe" when displaying the waveform and the spectrum in rough and smooth appearance.
Image

This arises obviously because a MoveTo is not completely color-free. So an update was necessary. And then I had this idea to make the Spectrum a little smoother. I had already tried out the idea at Frequency Visualization, but there as DCTL code. So now there is a smooth factor with which you can delay the falling edge of the spectrum display, so you can play with it.



Have fun
AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
SecondMan
Site Admin
Posts: 4857
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 31
Location: Vancouver, Canada
Been thanked: 48 times
Contact:

Re: [RELEASED] [Fuse] AudioWaveform

#84

Post by SecondMan »

Hey Gerald,

You may want to start a new [DEV] topic for a new version - I realise this gets a bit confusing when there are new developments in a topic that says [RELEASED]. Feel free to continue in this topic as well, but perhaps you could change the topic title to include the new version number and set it back to [DEV]?

Cheers!

User avatar
X-Raym
Posts: 2
Joined: Sun Jan 10, 2021 10:29 pm

Re: [RELEASED] [Fuse] AudioWaveform

#85

Post by X-Raym »

HI,

Many thanks for this this definitely open new possibilities, I don't see any other way to have dynamic spectrum in Fusion... ! Good work !

Question:
- It seems that the last version is v1.5 but last in Reactor is v1.2, how come ?
- is this last version still limited to wav 16 bit under 50 MB ?

Feature Request:
A common setting in spectrum visualizer is Slope, which is often set at default to 4.5, and which is a coefficient to compensate the energy of the low frequency compared to the high one.
Here is a demo with span VST visualizer:

Image

This would be even more useful than the actual compression setting IMHO to have good flat vizualisation. :)

Would it be possible to add this ? This would be very appreciated !

Thanks for your consideration :)

ps: first post here, I hope I didn't miss any rules.

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#86

Post by JiiPii »

@X-Raym Thank you for the praise. The difference in the version number is explained by the fact that I first collect changes before I submit a finalized version to the Reactor. So I hope to collectively implement bugs and improvements.
With regard to the 50 MB it would mean that I would have to reprogram a lot in the base. I tried to find a way to completely redesign the reading in of the audio data (then mp3 data could have been loaded). Unfortunately, I didn't get any further on this construction site (I'm not a professional programmer and this FFMPEG topic was a size too big for me, but there may be another possibility)
My current mini topic is the generation of the elongation with smooth parameters, i.e. the rise and fall of the change variables are filtered. I will then start a new post as recommended by @SecondMan , so that the versions are more clearly distinguishable.

I find the suggestion with the feature shown very exciting. The mathematical background is not yet clear to me. It will certainly not be a gain curve versus frequency (but maybe this is a prestudy). If a standard is required for the calculation, then the question arises again whether the FFT must be completely generated at the beginning (when loading the audio data). If you still have information about this slope-function, I would be very grateful.

And thank you for your suggestion

User avatar
X-Raym
Posts: 2
Joined: Sun Jan 10, 2021 10:29 pm

Re: [RELEASED] [Fuse] AudioWaveform

#87

Post by X-Raym »

Hi !

Thanks for your quick answer !

Slope is dB per octave, the most common value in audio plugins in 4.5db per octave to have nice flat bass and highs, but fully adjustable is good.
It is sometimes called Tilt. Here are some infos.

There is a free open source JSFX audio effects for reaper called ReSpectrum which has tilt setting, so you can access formula (note that JSFX give access to decoded audio). https://forum.cockos.com/showthread.php?t=213501 (zip in first post). You will see some TIlt variable in the code. Of course testing it means getting basic knowledge about reaper if you want to test the code (installing th FX, importing an audio, adding the fx to a track) but hopefully you will find infos easily. :P

I'll have to find another solutions for my current project which needs to be done quickly, but I hope these resources will make you push this script further, as there is no sign that Resolve or other third party plugin pack will try to do something about this kind of thing :P (ignite pro has spectrum plugin but only for after effects...).

Good luck, and thanks again for your consideration ! :)

User avatar
functionform
Posts: 1
Joined: Sun Apr 11, 2021 9:23 am

Re: [RELEASED] [Fuse] AudioWaveform

#88

Post by functionform »

Hello, I hope I'm not placing this in the wrong location, but I followed several tutorials online to load a waveform into AW and it doesn't appear to take. Just to be I used the correct format (and because it was too big for the 50 mb limitation) I put my audio through Audacity (3.0.0) , converted it to mono. and generated 3 versions, Microsoft WAV, 16, 24 and 32 bit. They came in at 24mb, 37mb, 49mb respectively.

None of the 3 files loaded successfully in the plugin. Resolve 17.1.1 Build 9, Windows 10. No matter what I did, nothing showed up in visualization panels at the top, and the plugin on the right says "No audio loaded" at the bottom. I enabled the Resolve console and I didn't see any messages there either, but that could be because I'm not sure how to use it properly.

Any tips on what could be wrong? I was following this tutorial:

Thanks for your help and apologies if this is in the wrong place!

User avatar
JiiPii
Fusioneer
Posts: 206
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 19 times

Re: [RELEASED] [Fuse] AudioWaveform

#89

Post by JiiPii »

@functionform
It is absolutely the right place. I got it right: AudioWaveform does not give an "error message" on the screen?
I always think that the "easiest" method to get wave files is to split the audio into corresponding parts in the edit page and put a fusion composition over it (i.e. in a pair: audio track, video track). Then you can use the renderer to use the Select the setting only audio (format: 16-bit PCM wave) and separate files and get a wave file for each section. This is then to be loaded into the respective fusion composition in the AudioWaveform. So you have the waves in fusion.

If it were possible, I would be grateful for a test wave with which I can understand the error. Many Thanks