Soundulous A better music visualizer
description
Transcript of Soundulous A better music visualizer
![Page 1: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/1.jpg)
Soundulous
A better music visualizer
Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan
![Page 2: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/2.jpg)
Concept
Streaming music and visualizer to mobile device
Interactive visualizer with touches Sync to multiple phones Two user functions:
New: a user (master) creates a session Join: a user joins an existing session
![Page 3: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/3.jpg)
What Is It Good For?
Audience: teens, young adults, other bored people
Fun group activity Parties! Bar Mitzvahs! Weddings!
![Page 4: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/4.jpg)
Technical Challenges
Streaming images to each phone Combining touch events from
various phones Integrating touches into visualizer
quickly Network Options
TCP/HTTP UDP Some combination of both
![Page 5: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/5.jpg)
Technical Challenges
The Visualizer! Need to do audio parsing for volume,
frequency detection FFT, amplitude, etc.
Map touch events to changes in volume, or frequency, or something else
Make it fast Make it cool
![Page 6: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/6.jpg)
The Client Side
Listening to touch events Mixing events Differentiating between “touches”
and “swipes”
Rendering the bitmap images successfully
![Page 7: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/7.jpg)
The Server Side
Getting packets and transferring to input of visualizer
So far: one thread Getting data via UDP from client
![Page 8: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/8.jpg)
The Server Side
Using Processing (processing.org) Making visualizer based on inputs So we need three more threads:
One for doing the actual rendering One for receiving video data from
processing One for sending video data to client(s)
Total threads: 4! Wow!
![Page 9: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/9.jpg)
Workflow (creating session) Phone posts to /new Server responds with a port number
(eg. 10001) Server starts sending visualization data
(compressed as JPG) on 10001 Phone opens an HTTP connection on
10001, and renders the incoming JPG data
Whenever the phone detects a touch event, it sends the event via UDP
![Page 10: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/10.jpg)
Workflow (joining session) Phone posts to /join with the port
number Server responds to confirm. Exact same workflow as with
creating!
![Page 11: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/11.jpg)
Demo
![Page 12: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/12.jpg)
Future Considerations
Making the full-scale app Limiting number of people/session Viewers/Enablers? Private/Public sessions?
More options Multiple visualizers Different touch functionalities
![Page 13: Soundulous A better music visualizer](https://reader036.fdocuments.in/reader036/viewer/2022062410/56816456550346895dd6236e/html5/thumbnails/13.jpg)
Thanks!