FYP Final Presentation: Distributed Audio Mixing Daire O'Neill, Final Year Electronic Engineering...
-
Upload
cornelius-harvey -
Category
Documents
-
view
218 -
download
5
Transcript of FYP Final Presentation: Distributed Audio Mixing Daire O'Neill, Final Year Electronic Engineering...
FYP Final Presentation:FYP Final Presentation:Distributed Audio MixingDistributed Audio Mixing
Daire O'Neill, Final Year Electronic Daire O'Neill, Final Year Electronic EngineeringEngineering
Project Supervisor: Dr Peter CorcoranProject Supervisor: Dr Peter Corcoran
Co-Supervisor: Frank CallalyCo-Supervisor: Frank Callaly
Structure of presentationStructure of presentation
Project OverviewProject Overview
How I approached this projectHow I approached this project
Issues encounteredIssues encountered
Implementation of final systemImplementation of final system
Objectives achievedObjectives achieved
ConclusionsConclusions
Project OverviewProject Overview
The ultimate goal of this project was to design The ultimate goal of this project was to design and implement a client-server system on Linux and implement a client-server system on Linux that allowed mixing of audio files from different that allowed mixing of audio files from different sources, over a networksources, over a network
A server was needed that could receive audio A server was needed that could receive audio data from multiple clients and mix the audio, as data from multiple clients and mix the audio, as in a traditional recording studioin a traditional recording studio
This system would allow musicians in different This system would allow musicians in different locations to record together, as if they were in locations to record together, as if they were in the same roomthe same room
GoalsGoals
The goals for this project were as follows:The goals for this project were as follows:– To install Linux on a desktop PC and become To install Linux on a desktop PC and become
familiar with the basics of using Linuxfamiliar with the basics of using Linux– To install and use ALSA to capture and To install and use ALSA to capture and
playback audio on a Linux machineplayback audio on a Linux machine– To implement a server capable of receiving To implement a server capable of receiving
audio data from clients and mixing the audioaudio data from clients and mixing the audio– To implement a method of synchronising To implement a method of synchronising
separate audio filesseparate audio files
How I approached the How I approached the projectproject
My first task was to install Linux on a PC. I My first task was to install Linux on a PC. I experimented with both Fedora and Ubuntu experimented with both Fedora and Ubuntu distributions, and found Ubuntu easier to usedistributions, and found Ubuntu easier to useAfter becoming familiar with using Linux, I After becoming familiar with using Linux, I began to search for a suitable application on began to search for a suitable application on which to base my project, as it was clear that I which to base my project, as it was clear that I could not hope to write an entire multitrack could not hope to write an entire multitrack recording program by myselfrecording program by myselfI needed a multitrack recording application that I I needed a multitrack recording application that I could configure to use files that were received could configure to use files that were received over a networkover a network
How I approached the How I approached the projectproject
I initially downloaded and installed I initially downloaded and installed multitrack programs such as Ardour and multitrack programs such as Ardour and Audacity, hoping to modify the source code Audacity, hoping to modify the source code such that these programs could connect to such that these programs could connect to a server to receive audio dataa server to receive audio data
However these programs are sophisticated However these programs are sophisticated (and therefore complex), so the source (and therefore complex), so the source code was difficult for me to understandcode was difficult for me to understand
How I approached the How I approached the projectproject
To go back to basics, I then concentrated on To go back to basics, I then concentrated on writing Python scripts to capture and playback writing Python scripts to capture and playback audioaudio
I successfully completed this by using Pyaudio, I successfully completed this by using Pyaudio, a Python module that provides Python bindings a Python module that provides Python bindings for the Linux PortAudio applicationfor the Linux PortAudio application
I then gained understanding of the actual code I then gained understanding of the actual code used in the recording and playback of audioused in the recording and playback of audio
However I still needed an application that could However I still needed an application that could provide multitrack playback of audio files...provide multitrack playback of audio files...
EcasoundEcasoundEcasound proved to be the solution to these Ecasound proved to be the solution to these problemsproblems
As it is a command line based recording As it is a command line based recording application, I could control it the way I wanted by application, I could control it the way I wanted by using shell scripts, eliminating the need to modify using shell scripts, eliminating the need to modify the actual source codethe actual source code
Ecasound can easily be automated by writing Ecasound can easily be automated by writing shell scripts to execute audio playback and shell scripts to execute audio playback and recording functionsrecording functions
Many complex functions can be implemented in Many complex functions can be implemented in Ecasound by passing the required arguments Ecasound by passing the required arguments from the command linefrom the command line
FTP ServerFTP Server
My next task was to implement a server that My next task was to implement a server that could receive audio files from multiple clients and could receive audio files from multiple clients and mix themmix them
The server I used was VSFTPD. I chose this The server I used was VSFTPD. I chose this because of it's stability and ease of usebecause of it's stability and ease of use
I then wrote a series of shell scripts that could I then wrote a series of shell scripts that could upload and download audio data from the server, upload and download audio data from the server, and provide recording and mixing functions, thus and provide recording and mixing functions, thus implementing the final systemimplementing the final system
The Finished SystemThe Finished System
By executing the appropriate shell script, By executing the appropriate shell script, the user (client) can:the user (client) can:– Download an audio track (in wave file format) Download an audio track (in wave file format)
from the serverfrom the server– Playback this audio track and mix their own Playback this audio track and mix their own
audio input during playbackaudio input during playback– Record the desired mix of the downloaded Record the desired mix of the downloaded
track and their own track as a wave filetrack and their own track as a wave file– Upload the new mix to the serverUpload the new mix to the server
The system was set up for use by 2 client The system was set up for use by 2 client PCs, both running Linux PCs, both running Linux
System FlowchartSystem Flowchart
The Finished SystemThe Finished SystemThe server retrieves the wave files uploaded by the The server retrieves the wave files uploaded by the clients from their directories, mixes the audio using clients from their directories, mixes the audio using Ecasound and returns the resulting wave file to the Ecasound and returns the resulting wave file to the clients directoriesclients directories
Each client can then download the new audio file (a Each client can then download the new audio file (a mix of both clients audio tracks), and add more mix of both clients audio tracks), and add more tracks as desiredtracks as desired
In this way, audio tracks from multiple clients are In this way, audio tracks from multiple clients are layered to create a finished audio mix, as in a layered to create a finished audio mix, as in a recording studiorecording studio
For example, Client 1 records a vocal track, then For example, Client 1 records a vocal track, then Client 2 mixes a drum track with this, Client 1 then Client 2 mixes a drum track with this, Client 1 then adds a bass track, etcadds a bass track, etc
Using EcasoundUsing Ecasound
All audio files used in this project are All audio files used in this project are recorded using PCM (uncompressed) wave recorded using PCM (uncompressed) wave file format, sampled at 44.1kHz with a file format, sampled at 44.1kHz with a resolution of 16 bitsresolution of 16 bits
Although this is the default for Ecasound, Although this is the default for Ecasound, this can easily be changed for different this can easily be changed for different sampling ratessampling rates
Using EcasoundUsing EcasoundA fundamental principle of Ecasound is the use of A fundamental principle of Ecasound is the use of “chains”“chains”
Chains can be considered as virtual “audio Chains can be considered as virtual “audio cables”, with a specified input and output. They cables”, with a specified input and output. They are used to route audio in Ecasound.are used to route audio in Ecasound.
In the following example syntax, a chain is set up, In the following example syntax, a chain is set up, the input set as a wave file (test.wav) and the the input set as a wave file (test.wav) and the output routed to the soundcard (/dev/dsp) :output routed to the soundcard (/dev/dsp) :– ecasound -a:1 -i test.wav -o ecasound -a:1 -i test.wav -o /dev/dsp/dev/dsp
This results in playback of the audio file This results in playback of the audio file “test.wav”.“test.wav”.
Example scriptsExample scripts
The next slide shows an example of one of the The next slide shows an example of one of the scripts written to control Ecasound for this scripts written to control Ecasound for this project, called “monitor+record”project, called “monitor+record”
When executed, this script starts Ecasound and When executed, this script starts Ecasound and plays back the latest version of the “current mix” plays back the latest version of the “current mix” as downloaded from the server, and also as downloaded from the server, and also records any audio that is input from the records any audio that is input from the soundcardsoundcard
Monitoring the output allows the user to record Monitoring the output allows the user to record in time with the backing track. The mixed audio in time with the backing track. The mixed audio data is written to a wave file (“client2.wav”).data is written to a wave file (“client2.wav”).
““monitor+record”monitor+record”
““servermix” scriptservermix” script
The next slide shows the “servermix” script The next slide shows the “servermix” script used in this projectused in this project
This script copies the uploaded files from This script copies the uploaded files from each client into a common directory, uses each client into a common directory, uses Ecasound to mix the audio files and returns Ecasound to mix the audio files and returns the resulting mix to the directory of each the resulting mix to the directory of each clientclient
The new audio data can then be The new audio data can then be downloaded by the clientsdownloaded by the clients
““servermix”servermix”
Configuring the FTP Configuring the FTP serverserver
Audio files are uploaded and downloaded from Audio files are uploaded and downloaded from the FTP server by using “put” and “get” the FTP server by using “put” and “get” commandscommands
Seperate directories were set up on the server for Seperate directories were set up on the server for both clients, each containing “incoming” and both clients, each containing “incoming” and “outgoing” folders“outgoing” folders
Files are uploaded by each client to their Files are uploaded by each client to their “incoming” folder, and downloaded from their “incoming” folder, and downloaded from their “outgoing” folder“outgoing” folder
This allows the server to mix audio data from both This allows the server to mix audio data from both clientsclients
Objectives AchievedObjectives Achieved
The required objective of implementing a The required objective of implementing a networked recording system has been achievednetworked recording system has been achieved
Using this system, musicians can collaborate to Using this system, musicians can collaborate to produce a finished audio track containing produce a finished audio track containing contributions from many different userscontributions from many different users
Although a solution to the issue of Although a solution to the issue of synchronisation between different audio files synchronisation between different audio files (due to clock timing differences on different (due to clock timing differences on different PCs) has not been implemented, the finished PCs) has not been implemented, the finished system still allows for server-side mixing of system still allows for server-side mixing of independently captured audio sources.independently captured audio sources.
Objectives AchievedObjectives Achieved
I also learned a lot about using Linux, and I also learned a lot about using Linux, and particularly about audio applications in Linuxparticularly about audio applications in Linux
As I have a keen interest in audio technology, I As I have a keen interest in audio technology, I feel that this project has been of great benefit to feel that this project has been of great benefit to me, and am glad to have been given the me, and am glad to have been given the opportunity to undertake such a project.opportunity to undertake such a project.