MIXLAB GUI

E177Group 5

Chris Jones

David Lovell

Drew McPherson

Contents

1 Abstract 1

2 Description, Purpose and Functionality 2

3 Description of Classes 23.1 AudioSample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.2 Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.3 Effects.Echo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.4 Effects.Fade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.5 Effects.Flange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.6 Effects.Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.7 Effects.Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.8 MIXLAB GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.9 TimeLine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.10 TrackSamples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.11 graphDisp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.12 uiPlay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Manual of Operation 8

5 Work Allocation 10

1 AbstractMIXLAB GUI is sound editing software which was designed and built using MATLAB object oriented programming. To accomplish

this task, a new sound class called AudioSample was constructed that builds upon the MATLAB provided class, audioplayer. An interactiveGUI allows for these AudioSample objects to be imported, trimmed, and modified through the application of custom Effects objects. Thesealtered AudioSample objects can then be combined through a visual interface and finally exported as a single AudioSample object to a playand display feature.

2 Description, Purpose and FunctionalityMIXLAB GUI allows users to import and modify audio samples through a graphical user interface. These modified audio samples are

then easily combined through a visual interface an can be exported to an audio display feature.

s

Figure 1. UML diagram of the MIXLAB GUI code structure

3 Description of Classes3.1 AudioSample

AudioSample was built upon the functionality of the MATLAB provided class audioplayer. This newly constructed class allows foralterations of audio files that were not otherwise possible with audioplayer. AudioSample has five private SetAccess properties; BaseAu-dioSample, SampleRate, Effects, SampleName and Volume, that can be viewed publicly but only set internally. There are also two fullyprivate properties, AudioPlayerObj and AudioBuffer that can only be access internally. There are also two dependent properties, Length andLengthSamples that are calculated for every call.

PropertiesThe constructor of AudioSample is set for a variable number of arguments to handle a single input of an audio file path, two arguments

in the form of data with a specified sample frequency or a null condition that builds an empty object.

BaseAudioSample: array This is the audio data that is passed into the AudioSample object.

SampleRate: double The sample rate is given in units of samples per second.

Effects: Effect The Effects property is an array that holds all of the Effect objects attached to an AudioSample object.

SampleName: string A string which describes the object.

AudioPlayerObj: audioplayer audioplayer is a built-in MATLAB class which provides the ability to load an audio filesuch as .mp3 or .wav and gives basic play/pause/resume functionality.

AudioBuffer: array The array which is used to store the audio with effects applied. This data is then used to create an AudioPlayerObjto play it back.

Length: double The length of the audio clip, in seconds.

LengthSamples: double The number of samples in the audio clip.

CurrentSample: double Returns the sample that the audioplayer object is paused on.

Methodsplay(start,end) The play method starts playback of the AudioSample object. It calls the play method from the internal AudioPlayerObj.The method is overloaded for three different input argument types: With no arguments, it plays the whole track. One argument specifies thestart time, and two arguments specifies a start and stop time.

pause() The pause function calls the pause method from the internal audioplayer object.

resume() Resume calls the resume method from the internal audioplayer object.

addEffect(Effect) The addEffect method adds Effect objects to the AudioSample Effects list.

updateAudioBuffer() The updateAudioBuffermethod applies the Effect objects to the base audio track in the AudioSampleobject. It ensures quicker playback when the user hits play.

getModifiedAudioData() This method gets the audio data array and sample rate for the sample with all the effects applied to it.

disp() The display method is overloaded to show a plot of the music’s amplitude and frequency spectrum.

getAudioSegment(start,end) This method returns a new AudioSample object between the start and end time specified.

getAudioSegmentNormalized(start,end) This method is similar to getAudioSegment, but the input arguments are takenas fractions of the length of the track. This method makes trimming the AudioSample clips using the slider bars simpler.

add(AudioSample,StartTime) This method adds two AudioSample objects together, and has is overloaded to take a start timeargument if desired. The default case adds both operand objects at t=0. This method also makes sure to up-sample the operand with thelower sample rate (if they are different). It returns a new AudioSample object. This method is used to distill all of the TimeLine’sAudioSample objects into one object for playback.

get.Length This is a getter for the dependent property Lengthwhich returns the length of the sample (with effects applied) in seconds.

get.LengthSamples This is a getter for the dependent property LengthSamples which returns the number of samples in the audiodata.

3.2 EffectAn abstract superclass used to provide the basic structure and required methods for the derived effects. It has no constructor or

properties, but defines two important abstract methods required for the function of the effects.

Methodsdisp() Overloaded display operator, to be redefined in each derived class.

loadGUI(handle) A static method which is redefined for each implementation to populate the effect parameters panel in the mainGUI. This is necessary because each Effect has different input parameters. Each subclass has similar code to define this panel - slidersare created with editable text boxes using loops to ensure even spacing and clean formatting. Also, minimum and maximum values areenforced to ensure the effects are configured properly and error-free. This method contains a few callback sub-functions which update thetext boxes with the slider values as well as allow the user to input specific values, which are sanitized to prevent error. This method returnsa cell array of slider uicontrol handles which allows the main GUI to interface with the panel and apply the Effect objects to theAudioSample objects.

Apply(data,rate) Each Effect has this method which performs the manipulation on the audio data arrays. The method returns anarray of the modified audio data and the rate to play back the new audio.

3.3 Effects.EchoThe Echo effect works by multiplying the input audio data array with an exponentially decaying coefficient array. The time constant

used in the exponent is calculated using the desired FadeTime.

PropertiesAmplitude This parameter controls the attenuation of the data which is added back to the original sound data. Its value can be between[0,1].

Delay Controls the amount of time (in seconds) to delay the audio data.

MethodsApply(data,rate)

3.4 Effects.FadePropertiesFadeTime The fractional amount of the track to apply the fade effect over. Input must be within [0,1].

FadeType Denotes the type of fade, which is in, out, or both.

MethodsApply(data,rate) This implementation uses the following discrete formula

Y [n] =12(X [n]+X [n+delay]) (1)

3.5 Effects.FlangeThe Flange effect creates an in/out ’wooshing’ effect which is essentially an echo effect applied periodically.

PropertiesDelay Delay time in seconds.

Frequency Frequency to sweep, in 1/s.

Range Time range of the sweeping amplitude in seconds.

MethodsApply(data,rate) The implementation of this effect uses the following discrete formula. Using a for loop to implement this effectwas extremely slow and significant speed gains were found by vectorizing the operations.

Y [n] =12

X [n]+12

X

Range⇤ sin(n

2pi f

rate

)

�(2)

3.6 Effects.SpeedThe speed effect speeds up and slows down the audio data. Instead of changing the rate, the audio data is re-sampled for maximum

quality and ease of handling elsewhere in the code base.

PropertiesMultiplier The speed multiplier. Values less than 1 reduce the speed of the audio and values greater 1 speed it up.

MethodsApply(data,rate) The Apply method implements this by re-sampling the audio data. The re-sampling is performed at a rationalnumber of precision 0.01 from the input multiplier.

3.7 Effects.VolumeThe Volume effect is straightforward. It works by increasing or decreasing the amplitude of the audio data by multiplying each value

by the Amount property.

PropertiesAmount The multiplier to apply to the audio data.

MethodsApply(data,rate) A simple multiplication of the audio data.

3.8 MIXLAB GUIThe MIXLAB GUI class creates the interactive GUI for the user to mix, modify and build AudioSample objects. The majority of

the functions within the class are callBacks from the various uicontrol objects that reside in the GUI.

PropertiesFig: figureHandle The Fig property holds the parent figure that the MIXLAB GUI resides upon. This handle is passed to severalof the other properties as well as methods and functions, which allows for the various child panels of the figure to determine their correctsizing and placement.

Handle Waveform: uicontrol panel This property holds the handle to the uicontrol that in turn holds the axes for editingthe AudioSample objects. This handle is stored as a property so that its scope is not limited to the constructor.

Handle StartSlider: uicontrol slider This property holds the handle to the uicontrol slider that indicates where theuser wants to start a trim on an AudioSample object. This handle is stored as a property so that its scope is not limited to the constructor.

Handle EndSlider: uicontrol slider This property holds the handle to the uicontrol slider that indicates where theuser wants to end a trim on an AudioSample object. This handle is stored as a property so that its scope is not limited to the constructor.

RadioButtons: uicontrol radio This property holds the handle to the uicontrol radio buttons that indicate the samplethat the user is currently loading, editing or adding to the TrackPanel object. This handle is stored as a property so that its scope is notlimited to the constructor.

SoundButtons: uicontrol push This property holds the handle to the uicontrol push buttons where the users stores AudioSampleobjects for editing or adding to the TrackPanel object. This handle is stored as a property so that its scope is not limited to the constructor.

ButtonGroup: uicontrol group This property holds the handle to the uicontrol buttons group that holds both the radioand push buttons. This handle is stored as a property so that its scope is not limited to the constructor.

LastSample: AudioSample This property holds the most recently added AudioSample object to the Handle Waveform.This handle is then accessed when the copy callback is called. This is stored as a property so that its scope is not limited.

EffectSlider: uicontrol slider This property holds the handle to the uicontrol slider in the Handle EffectsPanel.This is stored as a property so that its scope is not limited to the constructor.

EffectMenu: uicontrol edit This property holds the handle to the uicontrol edit box in the Handle EffectsPanel.This is stored as a property so that its scope is not limited to the constructor.

MethodsMIXLAB GUI (constructor) The constructor for MIXLAB GUI contains the majority of the functions for the class. The constructorfirst builds the various panels, axes and uicontrols for the GUI. Many of the tasks are then allocated to sub-functions for the purpose ofmaking the code more modular. The TimeLine is built by the class of the same name by a call that passes the parent panel to it. The restof the functions act as callBacks for the uicontrols throughout the GUI. Many of these callBacks utilize a creative manipulationof logic to work around the restrictions imposed by MATLAB uicontrols. The PlayWhenPushed callBack is of particular note.The callBack contains a callBack to the parent figure. This enables the button to trigger the playback of music when it is pushed andto stop playback when the button is released.

getSelectedRadioButtonInd() This is a simple method that returns the selected radio button for use in various sub-functions ofthe class.

updateWaveform() This method handles all of the graphical modifications involved in trimming the AudioSample objects. Itdynamically adjusts the scale of the plot to the length of the track and re-sizes the plot image. The sliders are also defaulted to the beginningand end of the trim window during each update.

3.9 TimeLineThe TimeLine class controls the functionality of the time line feature of the MIXLAB GUI, stores the AudioSamples passed to it,

and builds the AudioSample track that the system outputs. There are two main aspects of the class, the graphical output and the passing of

data to the TrackSamples class. For the graphical output, TimeLine uses built-in MATLAB uicontrol objects. The layer of the trackis selected by an array of radio buttons on the left of the screen, and the additional control is conducted through a uicontrol edit barand push buttons. These buttons trigger callback functions that process the desired tasks. A unique feature of the TimeLine class is theutilization of uicontrol buttons for the visual display of the track samples. These buttons Enable property is set to Inactive. This allowsthem to be viewed as blocks rather than buttons. Because they are inactive, the ButtonDownFcn callback is used to indicate the selectionof that sample. This callback places the handle of a selected sample as the ActiveButton of the class. The ActiveButton can bemoved and deleted through callback functions from other buttons labeled accordingly. The buttons call functions that alter the Positionproperty of the selected sample. TimeLine also serves to pass the AudioSamples to the TrackSamples linked list. TimeLinesends appropriate commands to the TrackSamples linked list to store, delete and to compile the output track.

Properties

AcitveButton: uicontrol button Selected by the user clicking on sample objects, the ActiveButton property holds thehandle to the selected object. The handle is then manipulated through the user clicking on other buttons and their associated callbacks.

TrackPanel: uicontrol panel This property holds the uicontrol panel that the constructed track is built upon. This infor-mation is stored as a class property so that tasks can be allocated to functions outside of the constructor while still having the data withinscope.

Move: uicontrol edit The Move property holds the uicontrol edit bar. This allows the user to select the amount of time inseconds that they would like for the samples to be moved. This value is stored as a property so that it is accessible by multiple functions andso that it will be up to date for each call.

Panel: uicontrol panel The Panel property holds the parent figure panel that the Timeline resides upon.

Samples: TrackSamples Samples is a linked list that holds the track samples that the user adds to the track panel. The informa-tion is contained as a linked list because the order of the data is not important. It is necessary, though, for the data to be transversed easilyand deleted without causing holes. The linked list is also ideal for this data storage because it allows data of different types to be stored in asingle container without the need for indexing. In this class a figure handle, AudioSample, and start time are grouped together.

RadioButtons: uicontrol radio The radio buttons on the MIXLAB GUI are stored in the property of the same name. Thisallows the radio button status to be known throughout the functions and methods of the class. This is vital for the placement of the trackswithin layers.

Track: AudioSample The Track property holds the output track built by the user. The TimeLine class can then play back theTrack AudioSample or then pass the object to another class that visualizes and plays the music.

Methods:makeLayerButtons() This method builds the button layout for the radio button group and the associated text box labels. The methodpulls the Panel property of the class object and determines the proper sizing and spacing of the buttons.

makeTrackSample(AuidoSample,index) This method id responsible for building the visual representations of the samples asthey are added to the track and for adding the new sample to the Samples property. makeTrackSample pulls the radio button status fromthe class objects RadioButton property to determine which layer of the track to display the sample on the TrackPanel. Additionally,it makes the most recently created sample the new ActiveButton for ease in moving the sample. This method also contains the callbackfunction MoveWhenPushed that is responsible for setting the sample to the ActiveButton when the user clicks on it.

3.10 TrackSamplesTrackSamples is a linked list class that is dedicated to the storage and management of AudioSamples that are placed on the

TimeLine by the user. The linked list feature is enabled by a call to a modified dlnode that holds an AudioSample, uicontrolhandle, and a double in each node. TrackSamples does not use indexing and adds each new node to the end of the list. This isbecause the order of the list is inconsequential. Since the list does not use an index, searches are done by comparing the uicontrolhandles.

PropertiesNode1: dlNodeSample Holds the first Node in the list to allow the list to be searched.

NodeEnd: dlNodeSample Holds the last Node in the list to allow the list to be searched.

Size: double (Dependent) Size is a dependent property so that the length of the list is calculated each time a call is made tothe property.

MethodsTrackSamples (constructor) A basic constructor that initiates the three properties to create a object of the class.

add(uicontrol,AudioSample,double) add places the new Node at the end of the list and updates the Node1 and NodeEndproperties accordingly.

updateStart(uicontrol,double) This method searches the list for a matching uicontrol handle and then updates the Startproperty of that Node. This method also calls the updateLength method to re-size all of the samples in the list.

size() This is a dependent method and calculates the length of the list by traversing through each Node.

makeAudioSample() This method creates a new AudioSample object by traversing the list and calling the AudioSample add()method for each Node.

getNode() This method searches and returns from the list the Node with the matching uicontrol object.

getStart() This method returns a 1xn array of doubles with the Start property of each Node.

getMusicLength() This method returns a 1xn array of doubles with the length of each music sample in the list.

updateLength() This method calculates the longest run time of any of the samples in the list. The list is then traversed and the sizeof each uicontrol object is updated to be the appropriate portion of the TrackPanel.

deleteNode(uicontrol) This method searches the list for a Node with the matching uicontrol handle. This Node is thenremoved from the list and the Obj.updateLength method is called to re-size all of the remaining uicontrol handles in the list.

isempty() This is a simple method that determines if the list is empty to add simplicity to the logic of other methods.

3.11 graphDispThis class was created to be the primary playback and visualization class once the user had finished composing the track in the

MIXLAB GUI. The graphDisp takes in the AudioSample object which is used to derive several of the properties for both graphingand playback.

PropertiesAudioSample This property is simply assigned the handle to the AudioSample object which is passed in for playback and graphing.

uiPlay This property is assigned the handle to a uiPlay class object which can either be passed in or is initialized in the constructor.

polAx This property contains the figure handle on which the polar graph visualization is plotted.

linAx This property contains the figure handle on which the spectrogram graph visualization is plotted.

currentGraphSample This property contains a double corresponding to the initial index for audio and graph playback when pausedor restarted.

MethodsplotPlay This method is the primary method of the graphDisp class in which all calculations are done and both the polar andspectrogram graphs are plotted. A key feature to this method is that it must be interrupted if the user selects to pause or stop playback of theAudioSample object.

StopCloseEvent callback This method is called from the uiPlay object to terminate the playback and close the GUI.

PauseEvent callback This method is called from the uiPlay object to pause the visual and audio playback.

PlayResumeEvent callback This method is called from the uiPlay object to resume the visual and audio playback either fromthe beginning or at its stopped point.

3.12 uiPlayThis is a handle class with its primary property a figure handle upon which the uiPlay GUI is built. This GUI functions as a

visualization for when the user chooses to play back the audio sample. This GUI allows for minimal user interface with simple playbackfunctions including play, resume, stop and pause. However, most of the functionality of this occurs in the graphDisp object while thisclass primarily lays out the display.

PropertiesgraphDisp The graphDisp property contains the parent graphDisp object for which the uiPlay instance is created.

gufig This property is assigned the figure handle upon which the GUI is built. This figure handle is then referenced by the graphDispclass,

polAx This property contains the handle to the polar axes plotted on by the graphDisp.plotPlay method. This graph is a real-timevisualization of the magnitude of the audio sample graft around the polar axis.

linAx This property contains the handle to the Cartesian axes plotted on by the graphDisp.plotPlay method. On this axes a colormap is produced of the spectrogram containing the fast Fourier transform of the sample data separating out the different frequencies. Thisis often used for signal processing of audio files but in our case it is more of a visualization.

MethodsuiPlay This method is the constructor in which the figure is divided into three separate uipanels. The first panel contains the polaraxes for the polar visualization. The second panel contains the axes for the spectrogram and the last is a button group.

Stop callback This method is called when the user presses the stop button which in turn calls the method in graphDisp to stopplayback of both the graphs and audio.

Play callback This method is called when the user presses the play/Resume button which in turn calls the method in graphDisp torestart or resume playback of both the graphs and audio.

Pause callback This method is called when the user presses the pause button which in turn calls the method in graphDisp to pauseplayback of both the graphs and audio.

4 Manual of OperationTo start MIXLAB GUI the user simply has to place the entire software package in their active file path of MATLAB 2016b or higher

and then run MIXLAB GUI.m. A figure will appear. In the top half of the figure, near the middle, is a button labeled, Load Clip. Pushthis button and then select an audio file, (eg. .WAV or .mp3). The music file will be imported to the top indicator labeled 1 and itwill turn from red to green. Now that the sample indicator is green, you can click and hold to play the audio stored in the indicator.

s

Figure 2. Sample indicator 1 has been loaded with the audio file that is displayed in the edit window.

To trim the clip, adjust the sliders in the edit window to the desired location and then push the button, Trim. This will retain the audiothat is between the two slider positions as shown below. To copy the new clip to another sample indicator, select a radio button next to anempty indicator (red color) and push Copy. The result is shown below.

s

Figure 3. After the audio file was trimmed, the radio button next to track indicator 2 was selected and the Copy Clip button was pressed.

To add an effect to an audio track, select an effect from the drop down list on the right of the button panel. This will change the displayof the panel on the top right. Adjust the sliders to produce the desired effect and push Add Effect.

s

Figure 4. Fade was selected from the Effects drop down menu and then added to the track by pressing Add Effect.

To add this audio sample to the main track below, simply select a radio indicate the desired layer by selecting a radio button along thebottom left, and then press Add to Track. This will add the sample to the beginning of the track and the window will normalize to thelength of the sample. Note: the window will always normalize to the length of the maximum runtime of the track.

s

Figure 5. The Add to Track button was pressed when layer button 7 was selected. The sample was added to the track and the track window normalized to

the sample length. The number 2 indicates what sample was loaded.

To move a sample along the main track, first click on the sample, then enter the amount of time in seconds you wish to move it, andfinally press the Shift Left or Shift Right buttons. A track can be deleted in the same way. Note: the most recently added trackwill always be the active sample. To continue to add samples to the track repeat the above steps. A result of this process is show below.

s

Figure 6. Several samples were loaded, edited and then added to the main track. The shift left and right buttons were employed to place them in the desired

locations.

At any point after a sample has been added to the track, the Create button can be pressed. This will import all of the samples into asingle track which can be played and stopped with the buttons at the bottom of the figure.

s

Figure 7. Visualization of the created track.

5 Work AllocationChris Jones wrote the entirety of the AudioSample class and all of the Effect classes, as well as co-wrote MIXLAB GUI.

David Lovell wrote TimeLine, TrackSamples, altered dlnodeSample, and co-wrote MIXLAB GUI.

Drew McPherson wrote dispGraph and uiPlay.