JFugue and Log4JFugueMusic for Fun and Productivity
David Koelle Brian TarboxAuthor of JFugue Author of Log4JFugueSenior Software Engineer Principal Staff EngineerCharles River Analytics Inc. Wabi Sabi Software
2
Overview> Learn about JFugue API to program music in Java
> Discover how to use music to increase your productivity
3
Introduction to JFugue> JFugue is an open-source API for programming
music in JavaTM code Generates MIDI music using Java Sound Provides a high-level interface to Java Sound’s
low-level API> Allows you to specify music naturally
player.play(“C D E F G A B”);
4
Programming Music with JavaSound// Play a Middle-CSequencer sequencer = MidiSystem.getSequencer();Sequence sequence = sequencer.getSequence();Track track = sequence.createTrack();ShortMessage onMessage = new ShortMessage();onMessage.setMessage(ShortMessage.NOTE_ON, 0, 60, 128);MidiEvent noteOnEvent = new MidiEvent(onMessage, 0);track.add(noteOnEvent);ShortMessage offMessage = new ShortMessage();offMessage.setMessage(ShortMessage.NOTE_OFF, 0, 60, 128);MidiEvent noteOffEvent = new MidiEvent(offMessage, 200);track.add(noteOffEvent);sequencer.start();try { Thread.sleep(track.ticks());} catch (InterruptedException e) { Thread.currentThread().interrupt();}
6
Introduction to JFugue> JFugue is a programmer’s tool for musical
innovation Parsing and transforming musical patterns Creating rhythms Creating microtonal music Capturing from, or sending to, external devices
> JFugue enables interaction with other music tools and formats Reads and writes MIDI, MusicXML Extensible parser/renderer architecture
Writing Music in JFugue
Pattern pattern = new Pattern(“C D E F G A B”);Player player = new Player();player.play(pattern)
Music StringPattern
9Free sheet music from http://www.classical-scores.com`
Vivaldi’s “Spring” in JFugue
Pattern p1 = new Pattern("G6i G6i G6i F6s E6s B6q. B6s A6s");
G note, 6th octave, 1/8 duration
w Whole Durationh Halfq Quarteri Eighth
s Sixteentht 32nd x 64th o 128th
11
Vivaldi’s “Spring” in JFugue
p1 p1 p2
p1 p1 p2
p3 p3
p4 p4
p5 p5
p6
p7
p8
Pattern p1 = new Pattern("G6i G6i G6i F6s E6s B6q. B6s A6s");Pattern p2 = new Pattern("G6i A6s B6s A6i G6i F6i D6i B5i E6i");Pattern violin1 = new Pattern(p0, p1, p1, p2);
Pattern p0 = new Pattern("T[Allegro] V0 I[Violin] KEmaj");
Player player = new Player();player.play(violin1);
T TempoV VoiceI InstrumentL Layer
K Key Signature& Pitch Wheel+, * Pressure EventsX Controller Event
13
Making Music Sound Great> MIDI gets a bad rap for creating dinky music> The real problem is the soundbank> Soundbanks are now replaceable
Thanks to the Audio Synthesis Engine Project!http://openjdk.java.net/projects/audio-engine
> Steps to better music: Use Gervill as your synthesizer http://gervill.dev.java.net Get a great soundbank http://www.sonivoxrocks.com Load soundbank instruments into the synthesizer
14
Making Music Sound Great// Place gervill.jar in your classpath to get an instance// of the Gervill SynthesizerSoundbank soundbank = MidiSystem.getSoundbank(soundbank file));// Load instruments from the soundbank into the synthesizerSynthesizer synth = MidiSystem.getSynthesizer();synth.loadAllInstruments(soundbank);// Create a JFugue Player object that is attached// to the synthesizer with the new instrumentsPlayer player = new Player(synth);// Now play your music with the new soundbank!player.play(your music here);
15
> Other features…
> Not all synthesizers support all of these features!
Other Features of the MusicString> Chords are easy to specify
Cmaj Amin7^^(or Amin7^E)
Attack and Decay Channel PressureController Events
Interval NotationPitch Wheel
Triplets and TupletsTies
Microtonal music
16
Rhythms> Build rhythms in a natural manner
#.##...#.###
> Easily re-assign instruments to notes> Combine chromatic notes with percussions
Create riffs within your beats!
17
RhythmsRhythm rhythm = new Rhythm();rhythm.setLayer(1, "OO..oO..OO..8OO.");rhythm.setLayer(2, "..*...*...*...*.");rhythm.setLayer(3, "......B.......B!");rhythm.setLayer(4, "..C.......C...CC");rhythm.setLayer(5, "^.^.^.^.^.^.^.^.");rhythm.addSubstitution('.', "Ri");rhythm.addSubstitution('O', "[BASS_DRUM]i");rhythm.addSubstitution('o', "Rs [BASS_DRUM]s");rhythm.addSubstitution('8', "[BASS_DRUM]s [BASS_DRUM]s");rhythm.addSubstitution('*', "[ACOUSTIC_SNARE]i");rhythm.addSubstitution('B', "[ACOUSTIC_BASS_DRUM]i");rhythm.addSubstitution('C', "[HAND_CLAP]i");rhythm.addSubstitution('^', "[PEDAL_HI_HAT]s Rs");rhythm.addSubstitution('!', "[CRASH_CYMBAL_1]s Rs");
Pattern pattern = rhythm.getPattern();pattern.repeat(4);Player player = new Player();player.play(pattern);
OO..oO..OO..8OO...*...*...*...*.......B.......B!..C.......C...CC^.^.^.^.^.^.^.^.
X ..xl...Y ..ym......Z ..zn
18
Rhythms with RiffsRhythm rhythm = new Rhythm();// Keep the code we just created, and add some hornsrhythm.setVoice(1, "X ..xl", "I[Trombone]");rhythm.setVoice(2, "...Y ..ym", "I[Alto_Sax]");rhythm.setVoice(3, "......Z ..zn", "I[Tenor_Sax]");rhythm.setVoiceOffset("................", 1);rhythm.addSubstitution('X', "Dwh<50");rhythm.addSubstitution('Y', "Fwi<50");rhythm.addSubstitution('Z', "Ahq<50");rhythm.addSubstitution('x', "Di"); rhythm.addSubstitution('l', "Ei");rhythm.addSubstitution('y', "Ai"); rhythm.addSubstitution('m', "Gi");rhythm.addSubstitution('z', "Fi"); rhythm.addSubstitution('n', "Bi");
19
Connecting to Musical Devices> Sending music to an external device is easy:DeviceThatWillReceiveMidi device = new DeviceThatWillReceiveMidi(MidiDevice.Info); sequence = player.getSequence(pattern);device.sendSequence(sequence);
> Reading music from an external device is easy:DeviceThatWillTransmitMidi device = new DeviceThatWillTransmitMidi(MidiDevice.Info);device.listenForMillis(5000);Pattern pattern = device.getPatternFromListening();
> Each of these involves lots of JavaSound code!
20
Using JFugue as “Musical Glue”> JFugue uses event-driven parsers and listeners
Current set: MusicString, MIDI, MusicXML> A Parser knows how to convert data into musical
events> A ParserListener (also known as a Renderer) knows how to
handle or render musical events
> Code to connect Parsers and ParserListeners:YourParser parser = new YourParser();YourParserListener listener = new YourParserListener();parser.addParserListener(renderer);parser.parse(whatever object the parser can parse);
21
Fun Things You Can Do With JFugue> Re-mix Johann Sebastian Bach
Read existing MIDI files into JFugue Patterns Perform Pattern Transformations on the music Recombine to come up with mutated Bach music
> Write “Piano Hero” that talks to a MIDI keyboard> Make a Virtual Instrument that creates microtones> Create a 3D animation that acts in anticipation of
upcoming music> Write a spam parser to turn junk email into music
and renders a light show
22
Music for Everyday Tasks> Having trouble keeping track of time?
GrandfatherClockp1 = "T110 I[TUBULAR_BELLS] E5qd120 D5qd120 C5qd120 G4h.d120";p2 = "T110 I[TUBULAR_BELLS] C5qd120 E5qd120 D5qd120 G4h.d120";p3 = "T110 I[TUBULAR_BELLS] C5qd120 D5qd120 E5qd120 C5h.d120";p4 = "T110 I[TUBULAR_BELLS] E5qd120 C5qd120 D5qd120 G4h.d120";p5 = "T110 I[TUBULAR_BELLS] G4qd120 D5qd120 E5qd120 C5h.d120";quarterPattern = new Pattern(p1);halfPattern = new Pattern(p2, p3);threeQuarterPattern = new Pattern(p4, p5, p1);hourPattern = new Pattern(p2, p3, p4, p5);hourGong = new Pattern("T110 I[TUBULAR_BELLS]
C5hqa100d120 Rs");
23
Music for Everyday Tasks
import org.apache.tools.ant.module.spi.AntLogger;public class BuildLogger extends AntLogger { @Override public void buildFinished(AntEvent event) { Player player = new Player(); Throwable t = event.getException(); if (t != null) { player.play("I[String_Ensemble_1] B3q Bb3q. G3i F3h"); } else { player.play("I[French_Horn] As E6h As E6i Rt As E6h"); } }}
> Want to know if your Ant build was successful?http://blogs.sun.com/geertjan/entry/ode_to_build_scripts
2525
Log4JFugue> Enterprise scale applications are large and
complex
> Knowing what an application is doing is getting harder
> Many of us have spent sleepless nights looking at enormous log files trying to figure out the cause of a customer problem
2626
> Most efforts to make a program’s behavior more apparent involve Application Visualization
> That’s fine but we have more senses than that
> Use more of your natural senses, not just vision
> Convert your program’s log4j output into sound
Application sonification
2727
Sample application: Video on Demand> “Streams” refers to video content that is being
played
> Interesting events:
stream createstream failureTrick a stream
stream destroyAdd contentEmit SNMP trap
2828
Mapping stream events to music> Each interesting log message is associated with a
particular musical instrument
stream create [ACOUSTIC_BASS_DRUM]sstream destroy [ACOUSTIC_SNARE]sstream failure [CLOSED_HI_HAT]s
2929
Counting occurrences of stream events> Increment a counter for an instrument each time
an event is found in a log message
> Use double buffering to build one Pattern as another one is being played
> Count-based music lends itself naturally to rhythms
30
Log4JFugue architecture
Message scanner
Log msg listener
Pattern builder
JFugue player
Instrument counts
Instrument counts
Create and play music
Instrument count double
buffer
Parse log files
3131
Message counts become beat patterns> Higher message counts equate to faster beats> Each second of play is based on a 16-beat
measure
> Beat for a message that occurred once:rhythm.setLayer(1, ".......O........");
> Beat for a message for occurred four timesrhythm.setLayer(2, "..O...O...O...O.");
34
Why use rhythms?> Tonal music varies in both duration and pitch
But Log4JFugue’s count-based approach provides only one changing value
Tried making all notes the same pitch and vary the duration (whole / half / quarter notes)
Tried changing the pitch and duration based on the count
> Until we figure out how to generate pitch and duration from a single count variable we’ll stay with rhythms
34
3535
Performance considerations> Log4JFugue isn’t free
Performance overhead
> Every log message needs to be compared to every target string
> Uses a highly optimized MIT-based scanner
> Can run on a remote box or after the fact
3636
Main classes> DataGetter – gets the messages
SocketDataGetter, FileDataGetter, LogDelay> MessageReceiver – processes the messages
Music map High-speed string scanner from MIT
> SoundBuilder – builds the music string> Player – wrapper around JFugue’s Player
PretendPlayer – writes music to a file, for playing later
3737
Spring-based, loosely coupled> Log4JFugue is build as a set of very loosely
coupled, extensible classes
> Configure where to get your data Live application, existing log file, replay speed
> Configure which strings to search for and which instruments to associate with them
3838
Spring-based, loosely coupled> Configure how to build the music
Chromatic or achromatic
> Almost no glue code in main, the application is assembled via Spring
39
Setting up Log4JFugue> We assume your application already uses Log4j
> Change your appender to use SocketAppender
log4j.appender.SOCKET=org.apache.log4j.net.SocketAppender
log4j.appender.SOCKET.Port=4445log4j.appender.SOCKET.RemoteHost=192.168.1.3
39
40
Setting up Log4JFugue> Configure Log4JFugue to listen on the socket
<bean id="dataGetter" class="org.log4jfugue.SocketDataGetter">
<property name="writer" ref="writer"/> <property name="reader" ref="reader"/> <property name="port" value="4445"/></bean>
40
41
Setting up Log4JFugue> Configure Log4JFugue to play an existing log file
<bean id="dataGetter" class="org.log4jfugue.FileDataGetter">
<property name="writer" ref="writer"/> <property name="reader" ref="reader"/> <property name="fileName“ value="server.log"/>
</bean>
41
42
Configuring for different MusicString> Edit the spring.xml file> Rerun the program
<prop key="OPEN_HI_HAT">sendTrap</prop> <prop key="MARACAS">stream create</prop> <prop key="HI_WOOD_BLOCK">destroyImpl</prop>
42
43
Log4JFugue Conclusion> Log4JFugue allows you to turn a dull and difficult
task into something easy and fun Understand your own application’s behavior Solve problems faster Actually get to sleep at night!
> Highly configurable> Lots of room for improvement
44
Conclusion> JFugue makes music programming easy and fun
Create exciting new musical things! Get your kids interested in programming! Impress your co-workers! Rekindle your joy of programming!
> Project websites JFugue – http://www.jfugue.org Log4JFugue – http://www.log4jfugue.org
45
David Koellehttp://[email protected]
Brian Tarboxhttp://www.Log4JFugue.orghttp://[email protected]
47
Making Music Sound Great// Place gervill.jar in your classpathSoundbank soundbank = MidiSystem.getSoundbank(soundbank file);Pattern pattern = new Pattern(your music here);Patch[] instruments = pattern.getPatches();// Load instruments from the soundbank into the synthesizerSynthesizer synth = MidiSystem.getSynthesizer();synth.loadInstruments(soundbank, patches); // was “loadAllInstruments” // Create a JFugue Player object that is attached// to the synthesizer with the new instrumentsPlayer player = new Player(synth);// Now play your music with the new soundbank!player.play(pattern);
48
JFugueUI> New side project!> User interface components for creating music> Encourages the development of Virtual Instruments
VirtualInstrumentListener
> Currently in development: JFPiano
49
JFugue Musical Intelligence> Another new side project!> Heuristics for creating music
Suggestions for counterpoint Procedural generation of musical phrases Generation of chord progressions using common
chord leading rules> An open-ended experiment in programming music> Participation welcome!
Just try ideas and see what works
5050
A Groovy side note> Most of Log4JFugue is written in Java…> …But some of it is written in Groovy
> For now there is no compelling reason that the Groovy modules had to be Groovy, basically did it because I could
> On the other hand, this will keep my eyes open to further use of Groovy in the future
Top Related