Welcome to WSL!

New to the forum? Please read this and this.

[RELEASED] [Fuse] AudioWaveform

Where the future is being made, today.

Welcome to the WSL development corner!

In this forum, please post your development projects. You get kudos and feedback here.
Topics ideally have preset prefixes, and this is what they (might) mean:

  • [DEV] - very much work in progress, don't build a business on this, could go anywhere
  • [BETA] - should kinda do what it's supposed to do, please test, give feedback
  • [RC] - this may end up in Reactor soon, polishing up, now's the time for last minute thoughts
  • [ABD] - died a premature death, sadness, will not see the light of day ever (unless someone picks up the scraps)

Once a development project has been released (hurray), topics can be marked as - you guessed it - [RELEASED] :cheer:

Development topics only, please. For generic questions, how-to's, questions and inquiries about existing tools etc, please go to the appropriate other forums.
User avatar
Millolab
Fusionista
Posts: 824
Joined: Wed Oct 24, 2018 6:26 am
Answers: 6
Been thanked: 29 times
Contact:

Re: [Fuse] AudioWaveform

#31

Post by Millolab »

chrisr wrote: Tue Jul 14, 2020 8:16 am I mostly read this forum, but wanted to comment. For me it's kind of uncomfortable seeing people publically called out. I guess I'm old, but surely the best option is to assume it is unintentional and deal with things privately first.
I think it's not calling someone out. It's more asking to be part of a discussion.
chrisr wrote: Tue Jul 14, 2020 8:16 am I agree very much with the idea of credit and authorship rights. But there's a level of subjectivity in there and we should always assume a mistake and give people respect first, to be honest, WSL is great but not exactly squeaky clean in this area.
Well, I think that's why we are having this conversation (which could be helpful for the whole community).
And honestly I can't see where people has been disrespectful.

chrisr

Re: [Fuse] AudioWaveform

#32

Post by chrisr »

@Millolab - it's about the fact that discussion is public and there's no indication that there's been a private one. The idea that it was a mistake wasn't considered - in a scenario where all parties are fallible. But I also think it's the best way to sort things out. June has offered credit and it's reasonable to think he would've in a private conversation too. Maybe I'm really old and out of touch, or maybe it's cultural, but essentially courtesy is being asked for from someone, and should be shown to them too. It's also kind of the easiest way to get things most of the time too. If somebody doesn't respond to that then by all means stick your tongue out, throw orange juice over them, etc.

That said @JiiPii, I forgot to say too - that your tool looks great. I would worry less about other people/things if I was you - you made it, it has bigger distribution, you understand it best, you can fix bugs more easily, etc, etc, it's always going to be better. Programmers use a whole load of references, and it's actually not that unusual for people to forget when they're deep into solving some problem and perspectives of how important something is (specification/technical input/functions) vary. So if you want something specific - it's generally best to write it in the comments. It's under a MIT license if I'm not mistaken, so technically people can do what they want at long as they repeat the license in the code. Nice people will do what you ask.

User avatar
SecondMan
Site Admin
Posts: 4900
Joined: Thu Jul 31, 2014 5:31 pm
Answers: 31
Location: Vancouver, Canada
Been thanked: 52 times
Contact:

Re: [Fuse] AudioWaveform

#33

Post by SecondMan »

@chrisr
chrisr wrote: Tue Jul 14, 2020 8:16 amI agree very much with the idea of credit and authorship rights. But there's a level of subjectivity in there
Well, we are subjective beings. All of our experience is subjective, and that's - I think - what this is mostly about. That's also why I was careful with my words, not to try to state just facts or legal requirements, but to express my personal feelings about it. I wonder what in particular makes you think that "the idea that it was a mistake wasn't considered"?

I really don't see any reason why this has to be a private conversation. To me this is no different than giving feedback on someone's work in a collaborative environment. It's the very thing this forum is for. Nobody is pointing fingers, or laying blame on, or trying to shame someone here. How to code is something that requires learning, and it comes with its own protocol and social aspect as well, as you say a subjective dimension of best practices. So there is real value in doing this publicly if done respectfully. I mean, if you feel strongly about that, you could have also chosen to send me a PM about it... ;)

This is all well-intended constructive criticism, and I for myself am trying to do it in as few words as possible, because I know June's first language is not English, not because I'm trying to be particularly terse.
chrisr wrote: Tue Jul 14, 2020 8:16 amThat original fuse at some point needed knowledge from a technical description or specification that would have been exceptionally helpful both to programming it and to future programmers, there's no credit for that.
Ah yes, guilty as charged :oops: - and I would probably do this differently today (not in the least because I didn't save my original sources anywhere, and when @JiiPii did a couple of code adjustments to cater for another input flavour, I couldn't immediately verify against them). But this also argues for having these discussions in public, because this is exactly what this learning process is; I never coded anything like that before, and had little experience documenting what I was doing.

However, there are also a few important differences to consider here. Firstly, my sources were the publicly available specifications from Microsoft, the sole purpose of which is to reference them when dealing with the structure of WAV files, plus a few mathematical concepts I needed to understand, like "two's complement" which is noted in the comments and is a generic operation, almost like addition, not a prior "creative" use of something (I am aware of the fuzziness).

Crucially, at no point did I build on or copy prior art, as in code - or even pseudo-code - in that original Fuse. All the code is my own, with some helpful tips to improve the implementation into Fusion. All of that is public.

Oh and also...
chrisr wrote: Tue Jul 14, 2020 8:16 am It also includes a license agreement that is not included in JiiPii's fuse, if it was things would have been clearer for June.
As I understand it, licenses like that are permissive, not prohibitive. So my license basically says "do whatever it is you like with this as long as you credit me for using a substantial functional part for your own whatever it is". Which JiiPii did. When there is no such license, as in the case of AudioWaveform, the art falls under copyright. So what we're potentially looking at here, if we forget the subjective for a second, is the unlicensed sale of copyrighted works. If that's not worth discussing in public, I don't know what is... :)

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#34

Post by JiiPii »

AudioWaveform.fuse
I have to admit, I bit my teeth into the AudioWaveformfuse. But it was a lot of fun. Even if I have explored some dead ends :-) It was also great to find out the university knowledge that had long been forgotten. I have now completed the implementation of the amplitude spectrum. So there is now a "correct" FFT (with 2048 values) and I have added equidistant points.

Image

I made a short explanatory video (please excuse the language): https://www.youtube.com/watch?v=06g_rY3-0Cg

Image

I observed a "peculiarity" when using the elongation. If I start playing "too quickly" after switching from the Fusiontab to the Edittab, DR cannot cache the composition, every time I try to play it, the previously rendered area is discarded and cached again.

I look forward to your comments and or suggestions.

AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
SteveWatson
Fusioneer
Posts: 165
Joined: Sat Oct 12, 2019 6:03 am
Answers: 3
Been thanked: 6 times

Re: [Fuse] AudioWaveform

#35

Post by SteveWatson »

Hi,

impressive work. (I watched it with auto generated subtitles in english which allowed me to get the gist of what you were explaining(with a few strange translations - "duck spectrum" "no gropes in the slippers" which is google's fault(though amazing we live in a time of real babel fish)).)

Nicely done.

User avatar
SteveWatson
Fusioneer
Posts: 165
Joined: Sat Oct 12, 2019 6:03 am
Answers: 3
Been thanked: 6 times

Re: [Fuse] AudioWaveform

#36

Post by SteveWatson »

Hi,

I've been playing around with this a bit, a few observations. It seems that if you don't have the audiowaveformwork node connected somehow to the media out(though a merge) then any modification you use the elongation for doesn't seem to work in the edit page.(Resolve Studio 16.3). This seem to happen even if you use an expression rather than connect to(just noticed it applies in fusion page too, ie if not connected through a merge then it doesn't work)

In this example I used 2 audiowaveforms with different freq. band to control x and y of a text node

Code: Select all

{
	Tools = ordered() {
		Merge2 = Merge {
			Inputs = {
				Background = Input {
					SourceOp = "Wave1",
					Source = "Output",
				},
				Foreground = Input {
					SourceOp = "Text1",
					Source = "Output",
				},
				PerformDepthMerge = Input { Value = 0, },
			},
			ViewInfo = OperatorInfo { Pos = { 440, 82.5 } },
		},
		Wave1 = Fuse.AudioWaveformWork {
			NameSet = true,
			Inputs = {
				GlobalOut = Input { Value = 508, },
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				UseFrameFormatSettings = Input { Value = 1, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				WaveFile = Input { Value = "/Volumes/Disk 3/Audio Test/test1.wav", },
				Proxy = Input { Value = 1, },
				Zoom = Input { Value = 1, },
				Resolution = Input { Value = 1, },
				Thickness = Input { Value = 0.00089, },
				Spectrum = Input { Value = 1, },
				EquiCuts = Input { Value = 1, },
				Steps = Input { Value = 3, },
				Limitation = Input { Value = 1, },
				Threshold = Input { Value = 0.7848, },
				Compression = Input { Value = 2, },
				ElongationOn = Input { Value = 1, },
				Elongation = Input {
					Value = 0.0752618110868599,
					Disabled = true,
				},
				RecOffset = Input { Value = 0.024, },
				ElongationAmplify = Input { Value = 0.126, },
				Font = Input { Value = "Open Sans", },
				Style = Input { Value = "Regular", },
			},
			ViewInfo = OperatorInfo { Pos = { 220, 82.5 } },
			Version = 6
		},
		Text1 = TextPlus {
			Inputs = {
				GlobalOut = Input { Value = 508, },
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				UseFrameFormatSettings = Input { Value = 1, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				Center = Input {
					Value = { 0.57526181108686, 0.505446519106789 },
					Expression = "Point(0.5 + Wave1.Elongation, 0.5 + Wave2.Elongation )",
				},
				StyledText = Input { Value = "TEXT", },
				Font = Input { Value = "Open Sans", },
				Style = Input { Value = "Bold", },
				Size = Input { Value = 0.154809312822483, },
				VerticalJustificationNew = Input { Value = 3, },
				HorizontalJustificationNew = Input { Value = 3, },
				ManualFontKerningPlacement = Input {
					Value = StyledText {
						Array = {
						},
						Value = ""
					},
				},
			},
			ViewInfo = OperatorInfo { Pos = { 550, -16.5 } },
		},
		Merge3 = Merge {
			Inputs = {
				Background = Input {
					SourceOp = "Wave2",
					Source = "Output",
				},
				Foreground = Input {
					SourceOp = "Merge2",
					Source = "Output",
				},
				PerformDepthMerge = Input { Value = 0, },
			},
			ViewInfo = OperatorInfo { Pos = { 495, 148.5 } },
		},
		Wave2 = Fuse.AudioWaveformWork {
			NameSet = true,
			Inputs = {
				GlobalOut = Input { Value = 508, },
				Width = Input { Value = 1920, },
				Height = Input { Value = 1080, },
				UseFrameFormatSettings = Input { Value = 1, },
				["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
				WaveFile = Input { Value = "/Volumes/Disk 3/Audio Test/test1.wav", },
				Proxy = Input { Value = 1, },
				Zoom = Input { Value = 1, },
				Resolution = Input { Value = 1, },
				Thickness = Input { Value = 0.00089, },
				Spectrum = Input { Value = 1, },
				EquiCuts = Input { Value = 1, },
				Steps = Input { Value = 3, },
				Limitation = Input { Value = 1, },
				Threshold = Input { Value = 0.7848, },
				Compression = Input { Value = 2, },
				ElongationOn = Input { Value = 1, },
				Elongation = Input {
					Value = 0.00544651910678889,
					Disabled = true,
				},
				RecOffset = Input { Value = 0.646, },
				Font = Input { Value = "Open Sans", },
				Style = Input { Value = "Regular", },
			},
			ViewInfo = OperatorInfo { Pos = { 275, 148.5 } },
			Version = 6
		},
		MediaOut1 = MediaOut {
			Inputs = {
				Index = Input { Value = "0", },
				Input = Input {
					SourceOp = "Merge3",
					Source = "Output",
				},
			},
			ViewInfo = OperatorInfo { Pos = { 660, 148.5 } },
		}
	}
}
This works in the edit page. But if I connect the text node directly to the media out it doesn't(even though the 2 audiowaveform nodes are still in the flow). This becomes an issue if you want to use the waveform to drive something without actually seeing the waveform itself. This might be a side effect of the node's working method but I'm used to using node properties even if they are not directly connected to the output.

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#37

Post by JiiPii »

Hallo SteveWatson,

When programming the elongation I once had the phenomenon that it no longer worked, too. At some point I probably either integrated the fuse again, or, funnily enough, displayed it in the viewer. Because the fuse does not have to output "data", then the "Process" is not called, but "only" the PreCalc. I did a quick test with one change and it was successful. I'm still testing it and if it fixes the problem upload it here.

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#38

Post by JiiPii »

I tried with REG_NoPreCalcProcess = true (call Process for precalc requests (instead of PreCalcProcess)) to only ever call the process. Unfortunately, that doesn't work with my fuse. I think that the PreCalc cannot prevent itself with elongation (i.e. with output of non-image data). So I will have to call the Process in PreCalc. I can use req.IsPreCalc to exclude the image outputs. Or is there a more elegant option?

User avatar
Shem Namo
Fusionista
Posts: 686
Joined: Sun Oct 06, 2019 9:15 pm
Answers: 1
Location: North Israel
Real name: David Kohen
Been thanked: 9 times

Re: [Fuse] AudioWaveform

#39

Post by Shem Namo »

Hi @JiiPii, I have a question about your fuse.
What king of audio file does it need to be?

I have a couple .wav files that work with the Suck Less Modifier, but it doesn't seem to work in your AudioWaveformWork fuse.
I converted these audio file from mp3 using FFmpeg with this command ffmpeg -i song.mp3 song.wav.
On the other hand, I have a few other .wav files that I didn't make, and the work with your fuse perfectly.

So I'm not sure what format or codec the file needs to be to work.
Thank you so much for this fuse Gerald, it is really amazing!!

Thanks again,
David

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#40

Post by JiiPii »

Hi Shem Namo,

I have tested a mp3 with the conversion of ffmeg. The format of the wav-file is the same as from DR and it worked. I suspect the "problem" is the size of the WAV file. Since I cache the audio data, I have built a limit of 10MB for a WAV file. I assumed that the fusion composition in the editing tab should be synchronous and a fusion composition should be very short, i.e. not longer than a minute. Then you would render the audio snippet through DR and load the generated wav file into the fuse. I have tested also WAV-Files with 50 MB and it worked for me, so maybe I can raise the limit.
I can admit the compliment directly, your nova fuse is absolutely great.

Thank you too

Gerald

User avatar
Shem Namo
Fusionista
Posts: 686
Joined: Sun Oct 06, 2019 9:15 pm
Answers: 1
Location: North Israel
Real name: David Kohen
Been thanked: 9 times

Re: [Fuse] AudioWaveform

#41

Post by Shem Namo »

JiiPii wrote: Tue Jul 21, 2020 2:27 pm I suspect the "problem" is the size of the WAV file.
Thanks Gerald, that's probably it, since the files that worked were pretty short and the files that didn't were long.

It's probably better to raise the cache limit if that's possible,
because I would suspect that a lot of people would want to use this fuse for music videos and remixes,
and most of those are usually longer than 3 minutes.
It would be even better to add a cache limit slider to the UI if that's possible. :)
JiiPii wrote: Tue Jul 21, 2020 2:27 pm I can admit the compliment directly, your nova fuse is absolutely great.
Thank you so much, I'm really glad that you like it!!

Thanks again,
David

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#42

Post by JiiPii »

I've fixed the bug. Back then I was a little surprised that the PreCalc function was absolutely necessary, because I assumed that the process routine would follow anyway. Thought it had internal synchronization reasons. Now I know better :-)
I have now taken the error messages "File too big" and "Unsupported" into the display window. I think that avoids a lot of frustration. I should have done that right away.
Image
AudioWaveform.fuse
You do not have the required permissions to view the files attached to this post.

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#43

Post by JiiPii »

Hello,
I have revised the subtitles in my explanatory video on Youtube and now there are no more "duck spectrum" or "no gropes in the slippers". I hope it is understandable now.

Have fun

User avatar
SteveWatson
Fusioneer
Posts: 165
Joined: Sat Oct 12, 2019 6:03 am
Answers: 3
Been thanked: 6 times

Re: [Fuse] AudioWaveform

#44

Post by SteveWatson »

JiiPii wrote: Wed Jul 22, 2020 4:00 pm Hello,
I have revised the subtitles in my explanatory video on Youtube and now there are no more "duck spectrum" or "no gropes in the slippers". I hope it is understandable now.

Have fun
Hi,

thanks for doing that. I watched it again and it made how to use it much clearer. (I'm sure anything I said that was auto translated into german would give similarly odd results).

User avatar
JiiPii
Fusioneer
Posts: 211
Joined: Tue Mar 03, 2020 8:34 am
Answers: 1
Been thanked: 23 times

Re: [Fuse] AudioWaveform

#45

Post by JiiPii »

I have almost finished a new update (there is already a small preview https://www.youtube.com/watch?v=6lN8nnqnErM ) I am just testing and came across a great fuse called Append https://www.steakunderwater.com/wesuckl ... =33&t=3707 (this post may be in the wrong place). I wanted to change data for an input / button calculate the AudioWave and came across the following lines with the append fuse:
  1. function NotifyChanged(inp, param, time)
  2.  
  3.     reCalcFootage = true
  4.  
  5. end
and in Process()
  1.     ------- Get dynamically generated inputs
  2.     if reCalcFootage then
  3.         reCalcFootage = false
  4.  
  5.         --print("Recalced!")
  6.         fusion = getFusion()
  7.         comp = fusion.CurrentComp
  8.     end
I then tried that and failed - then examined the append fuse, but there, too, you cannot exchange data across threads. Isn't there a way to do that? :cry: