Reading Audio Data

16 posts (showing 1-16)
FGL_Porter

FGL AdminCommunity Level 16
427 posts

I'm extremely interested in working with audio in game development. That being said, the few times I've dealt with reading audio data, I wasn't too successful, but I think my programming skills have increases significantly since then, and I'd like to try again.

Does anyone know much about, or have any useful links on reading audio data? Just to be clear, I'm referring to detecting bass, melody, the beat, etc, as seen in audio visualizers, and used in rhythm games. Anything for AS3, Haxe, or Unity (C#) would be great! If anyone has any source for this kind of thing, I'd love to look through it!

posted 2015-09-15T12:24:21-07:00 | edited 2015-09-15T12:30:05-07:00
OceanBreezeGames

Market Level 7Community Level 2
119 posts

Maybe you want to look at this for some of that:

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/media/SoundMixer.html#computeSpectrum()

The computeSpectrum part...

posted 2015-09-15T13:57:39-07:00 | edited 2015-09-15T13:58:18-07:00
FGL_Dave

FGL AdminCommunity Level 4
318 posts

Have a read about Fast Fourier Transforms and their applications in audio processing :)

posted 2015-09-16T12:25:30-07:00 | edited 2015-09-16T12:27:27-07:00
FGL_Porter

FGL AdminCommunity Level 16
427 posts

Will do! I've never heard of them, and they sound fancy, so they should assist!

posted 2015-09-16T12:45:57-07:00
b10b

Market Level 4Community Level 7
970 posts

Dave, have a read about how FFTs can be done with light and lenses.

posted 2015-09-16T19:10:26-07:00
maulidanBee

Market Level 5Community Level 5
601 posts

My studio so far has been researching on voice data processing and make it implemented on our game as a action function. Something like Flash AS3 speech recognition based game.

https://www.youtube.com/watch?v=fN_0Mq0bnKM

posted 2016-01-04T08:53:01-08:00
lasha

Market Level 0Community Level 0
1 posts

in as3

I need to stop sound in case of hitting object. and start another sound. I don't know how to add it . Could you help me??

posted 2016-03-27T11:12:07-07:00
JohnBlackburne

Market Level 3Community Level 10
2188 posts

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/media/Sound.html

Just follow the link and read through the documentation. All of it, including the examples. Playing sounds in Flash is simple. Load the sound. Use play() to play when you want. Use stop() on the returned SoundChannel to stop it when you want. That‘s all you need and all most games do as more complex tasks are beyond Flash’s capabilities.

posted 2016-03-27T18:15:09-07:00
lasha

Market Level 0Community Level 0
1 posts

Ok.Thanks again ))

posted 2016-03-28T12:13:51-07:00
grado

Market Level 0Community Level 0
1255 posts

I would like to do some dynamic music having a few loops that you could fade in/out. So could add drums etc as the action picks up.

Ive tried to do it in the past using SampleData (I think), but was a bit above my head. I wonder if anyone else here has done something like this?

Before anyone says, just using play and events on sound end doesnt get a nice smooth loop

EDIT: Some interesting things here: http://www.as3gamegears.com/sound/

posted 2016-03-31T12:29:56-07:00 | edited 2016-03-31T12:52:33-07:00
b10b

Market Level 4Community Level 7
970 posts

@djstatika, although I've moved on from Flash now, dynamic sound and music linked to in-game progression was something I often experimented with during my Flash days.

SampleData is very powerful and much simpler than it first looks.  Think of it the same way you would manually do bitmap filters / blendmodes etc - i.e. scanning through one pixel at a time, applying the filter to it or its near neighbours.

It's certainly possible to near-real-time mix multiple parts and even add filters (e.g. cutoff for underwater, or reverb for ear ringing damage).  However - and this was a show-stopper for me - the CPU usage to do so is significant.  Because I was rendering in CPU I had little to spare, and mixing multiple audio channels together required anything from 25%-100% of available CPU cycles alone - so the net result was awesome sound, but poor graphics.  I briefly experimented with using PixelBender to do it, but then Adobe changed their license on that.  Cutting room floor!

These days that might all be different - e.g. by using GPU for rendering the CPU may be sufficiently relaxed to do near-real-time sound processing?

I found the happy medium was to use SoundTransforms to create dynamic crossfades.  For example say we have three channels (drums, bassline, melody) - we create a seamless loop for each of the exact same length and tempo, then start all of them at the same time.  From there, we only adjust the volume of each channel based on in-game events.  It's quite a versatile approach when used to crossfade more subtle things (e.g. filtered bassline, vs non-filtered bassline).  Combining it with a timer is good enough to synchronise back to in-game events (e.g. for rhythm games), however, as you know, the event will trigger on the nearest enterFrame after the timer, which is not usually good enough for high precision.  Which reminds me, even using SampleData has a lag.

posted 2016-03-31T15:49:04-07:00 | edited 2016-03-31T15:50:32-07:00
grado

Market Level 0Community Level 0
1255 posts

Thanks b10b, interesting stuff, probably saved me a lot of time

Is it possible to "prerender" audio at runtime, i.e. construct a full music track out of some loops

posted 2016-03-31T16:05:38-07:00
lasha

Market Level 0Community Level 0
1 posts

Thanks.It's very useful information. May I ask you is it possible to convert swf flash as3 file for ios or other platforms? Maybe it is very unreal I don't know ??

posted 2016-03-31T17:50:32-07:00
DelaniStudio

Market Level 2Community Level 5
370 posts

lasha


1) Download Flashdevelop ...
2) Create as3 Mobile Project..
3) open Readme.txt in that project ..
4) follow instructions...
5) Profit.....

posted 2016-03-31T19:22:37-07:00 | edited 2016-03-31T19:23:21-07:00
b10b

Market Level 4Community Level 7
970 posts

djstatika said:

Is it possible to "prerender" audio at runtime, i.e. construct a full music track out of some loops

Good question.  Yes prerendering the byte collection is viable and it could be played back later through SampleDataEvent as a stream.  My guess is that performance will be slower and more memory hungry than using conventional playbacks - but I don't know for sure.

posted 2016-03-31T21:37:41-07:00
grado

Market Level 0Community Level 0
1255 posts

On Mar 31, 2016, lasha said:

Thanks.It's very useful information. May I ask you is it possible to convert swf flash as3 file for ios or other platforms? Maybe it is very unreal I don't know ??

Yes, using air sdk. You can use flashdevelop, or flashbuilder to do this for you or you can do it via a command prompt. More info here: http://help.adobe.com/en_US/air/build/WS901d38e593cd1bac1e63e3d128cdca935b-8000.html

Good question. Yes prerendering the byte collection is viable and it could be played back later through SampleDataEvent as a stream. My guess is that performance will be slower and more memory hungry than using conventional playbacks - but I don't know for sure.

It sounds like a fun thing to attempt some time, i might take a look at the weekend. ITs not quite as exciting as generating a music track on the fly, but might be a good way of having long music tracks in flash. Thanks for the info

posted 2016-04-01T11:49:29-07:00