Skip to content

Sound Visualization and Flash

March 25, 2010

Using sound data in Flash can be accomplished using the SoundMixer.computeSpectrum Class. Essentially, the computeSpectrum class takes values of a soundwave in at the moment when computeSpectrum is called. 512 values are returned in a byte array: 256 values from the left stereo channel, followed by 256 values from the right. Note that computeSpectrum cannot be applied to individual sounds (ie. if two mp3s are playing at the same time, computeSpectrum would not be able to analyze these sounds seperately). Instead it reads the overall sound output at runtime.

Reading Sound Data

Each of the 512 values in the byte array is a number between -1 and 1. Values from the byte array can be retrieved using the readFloat() method in a for loop. readFloat will return the next read the next value in the byteArray each time it is a called:

for(var i:uint=1; i<=256; i++) //only first 256 points (left channel)
{
    value = myByteArray.readFloat();
}

Using Sound Data

The data you get back from computeSpectrum could be used for a variety of purposes. I actually saw an example where it was used with some success for a simplified lip syncing! A common way to make use of the data is to use each of the value points an applying them to some other property;  such as the size, color or position of a movieclip. A how bunch of effects can be achieved in this way.

One API we were introduced to in class is called Woodpecker (http://woodpeckerflash.wordpress.com/) and is meant to simplify using computeSpecturum. It does this by allowing you assign how many values you deal with from each computeSpectrum (they call them beats). It also allows you to assign a different movieclip to each beat, change properties based on the beat values. It also deals with some normalization calculation.

Actually,  I tried creating a similar class that would allow me a little more control over what the beat values were used for. Essentially, it just averages points and returns back a specified number of beat values. I’ve add the code as “Beatmapper” at the bottom of this post. I guess BeatMapper is a misnomer, i did try to detect beats using some more code in additonal, but it was only had moderate to low success:

Attempt at Beat Detection:


function DetectBeat()
 {
 for each(var beat in myBeatMapper.beatValue)
 {
 sum += Math.abs(beat);
 } //for

 trace(sum-prevSum);

 if(sum > prevSum*1.7){glowy.play();}  //Using frequency, use *2 if spectrum?
// Glowy Movieclip plays when a beat is detected.

 prevSum=sum;
 sum=0;
 }

Reducing Computational Load

One way to reduce the computational load of your code is to use only one channel from your sound output(either the first or last 256 points). The sound is fairly similar on both channels, this shouldn’t make much of a difference.

Issues with Security Sandbox

This is what the error looks like:

SecurityError: Error #2121: Security sandbox violation: SoundMixer.computeSpectrum: http://blahblahblah.swf cannot access blahblahblah. This may be worked around by calling Security.allowDomain.

I’ve found that I occasionally got this error while testing my swf locally. There is a restriction with accessing a local sound files, such as I did to when I loaded the mp3 I wished to use. I found I could solve the problem in this case by hosting the mp3 on my server and referring to it by it’s URL when loading it into flash, this seemed to work out fine.

This error also comes up sometimes when hosting your swf on a website. Unfortunately, the reason for the error and the options available were not very pleasing. I found this blog post that seems to explain it correctly. Apparently, if flash is running more than one swf using audio, then computeSpectrum will be unable to work and the error will come up.

Issue with frequency spectrum

You may have noticed that if you set computeSpectrum to return frequencies instead of soundwaves; it appears that the higher frequencies seem under emphasized. Smarter people then me have given us an explanation at this blog post, “computeSpectrum and Math”:http://blog.benstucki.net/?p=60 . Apparently, it has something to do with a linear distribution instead of a logarithmic one when applying the Fourier Transformation.

Other Links

ByteArray.org Equalizers (http://www.bytearray.org/?p=9): Here is an example of a class used to make an equalizer with some nice visual effects.

ByteArray.org Sound.extract() ( http://www.bytearray.org/?p=329): apparently with flashplayer 10 it is possible to analyze sounds before they are played.

Hype, soundanalyser: http://hype.joshuadavis.com/02_examples/soundanalyzer/content/01_soundanalyzer/

BeatMapper code:

/*
To use, call a new instance of BeatMapper:
 var myBeatMapper= new BeatMapper(beatNum:Number, isFreq:Boolean);

beatNum= a positive number between 1 and 256. This number of divisions you would like soundwave to split into.

isFreq= false returns a soundwave, true returns a frequency spectrum

All, the beat values (which range from -1 to 1) are stored in the the array named beatValue. You can call on the value of any of the beats in they array by it's index number. In the example below, all the beats value are accessed through a for loop:

 for (var j:uint=0; j<myBeatMapper.beatNum; j++)
 {
 //do whatever you want with beatValue[j];

 beatX= j*spacing + 15;
 beatY= myBeatMapper.beatValue[j]*maxAmp+startPos;
 graphics.drawCircle(beatX, beatY, 5);

 }//for

*/

package soundSpectrumCode
{
 import flash.utils.ByteArray;
 import flash.media.SoundMixer;
 import flash.events.Event;
 import flash.display.MovieClip;

 public class BeatMapper extends MovieClip
 {
 public var beatNum:Number; // 1-256 . Splits up wave into bars.
 public var isFreq:Boolean; //false give raw wave outpout, true gives frequency
 public var ba:ByteArray= new ByteArray();
 public var beatValue:Array=new Array;

 private var divisor:Number;
 private var remainder:Number;

 public function BeatMapper(numberOfBeats:Number, isFrequency:Boolean)
 {

 trace("hi from Soundbeat");
 beatNum= numberOfBeats;
 isFreq= isFrequency;

 divisor =Math.floor(256/beatNum);
 remainder= 256%divisor;

 addEventListener(Event.ENTER_FRAME, loop);

 } //SoundBeat

 private function loop(e:Event)
 {
 SoundMixer.computeSpectrum(ba, isFreq); //512 pieces, 256 each left and right. Each value btwn -1 and 1

 var oneBeat:Number= 0;
 var whichBeat:uint= 0;

 for(var i:uint=1; i<=256; i++) //only first 256 (left channel)
 {
 oneBeat += ba.readFloat();

 if(whichBeat==beatNum-1)
 {
 if(i==256)
 {
 beatValue[whichBeat]= oneBeat/(divisor+remainder);
 }// if if
 }
 else if(i%divisor == 0)
 {
 beatValue[whichBeat]= oneBeat/(divisor);
 oneBeat= 0; //new start of one beat
 whichBeat++;
 } //if else if

 }//for
 } //loop

 } //class
} //package
Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: