Programming with Python: Math, Music and Art in Web

Based EnvironmentsGISA 2016

Session 233Marist School

Christopher Michaud

Why Computer Science and Art?

• Intersection between the process of creating Art

• Language of Mathematics

• Tools of Computer Science

Process of Creating Art

• Iterative interaction between the Artist and the Materials of Art• “What If . . . .?”• Artist creates art to embody non-verbal ideas, motion, flow• Artist reflects and refines the art during creation

• From Music Education• A Philosophy of Music Education, Bennett Reimer

Language of Mathematics

• Spatial Relationships in 1D, 2D, 3D spaces• Temporal Thinking – how do properties change through time (rate of

change, derivative . . .)• Sequential Flow of Logic• Dynamics – how are parts of the system changing? How can we

control the change?

Tools of Computer Science

• Data types• Variables• Arrays• Control• Functions• Classes• Recursion

What is Programming?Allows humans to encode, refine, reflect, and share logical

thought in a dynamic moving system that changes over time.

Music and Programming (Computational Thinking) are the same Human Skill expressed to two different domains: nonverbal feelings in art/music and logical/motion thought in Programming or Computational Thinking

Part One: Music Mixing with Python and EarSketch

Large population of students through which music is the primary vehicle for personal expression.

In CS – we tend to focus on:Graphics / Games / VisualText Processing / DatabaseLogic / Algorithmic / Mathematical Robotics and Kinesthetic

EarSketch seeks to engage students in CS through the art of music.

Description of EarSketch•Programming Environment•Python Based•Web Based App or Installed System of Software•API built in Python for Music Mixing•NSF Funded project to encourage computational interest through the mixing and sharing of music.

•Curriculum and Social Media Site•http://earsketch.gatech.edu

AuthenticRelevantExpressive

EarSketch Website: Web based IDE and DAW

Coding Window: Programming

Music View: Display and PlaybackDocumentation and CurriculumSound Library

Toggle Views

What is Python?•General purpose Computer Programming Language•Web development: Google and Yahoo•Game Development, Science, Graphics•In Business: National Weather Service, NASA, IBM, Disney, and Nokia

•Used in Development of the Google Car and other Robotic Systems

How is Music Organized? How do we encode musical clips?

Tracks: Vertical Layers of Music

Layers: Music exists through time

Data Types and Variables

EarSketch Python FunctionsEarSketch Function Description

init() Start new Earsketch Mix and clear the DAW

setTempo(120) Sets the beats per minute (speed) of the Mix

println(“Hello”) Prints message to console

EarSketch Python FunctionsMusic Mixing Functions

fitMedia(file, track, start, end)

makeBeat(file, track, measure, beatString)

setEffect(track, effect, parameter, sV, sM, eV, eM)

rhythmEffects(track, effect, parameter, list, measure)

“fitMedia” FunctionfitMedia(file, track, start, end)

Location of MediaSound

Which Track in Reaper

Start measure.

End Measure

fitMedia(HIP_HOP_DRUMS1_2M, 1, 1, 9)

Example:

Setting Volume EffectssetEffect(track, VOLUME, GAIN, level, start, level2, end)

• Example

setEffect(1, VOLUME, GAIN, -40, 1, 10, 5)

Selected List of Effects and Parameters

Effect Parameter Min to Max ValuesVOLUME GAIN -60 to 12

DELAY DELAY_TIME 0 to 300.0

CHORUS CHORUS_LENGTH 1.0 to 15.0

CHORUS CHORUS_NUMVOICES 1.0 to 8.0

DISTORTION DISTO_GAIN 0.0 to 50.0

FILTER FILTER_FREQ 20.0 to 20000.0

PAN LEFT_RIGHT -100 to 100 (Left to Right)

Complete Effect list at: http://earsketch.gatech.edu/category/learning/reference/every-effect-explained

“makeBeat” MethodmakeBeat(file, track, measure, BeatString)

Location of MediaSound

Which Track in Reaper

What measure.

Example: “0+++0+++0+0+0+++”

makeBeat(drums, 1, 1, “0+0+0+++00-00+++”)

Example:

Beat String notation“0, 1, 2, 3 . . . “ = Which Media Sound you want for the segment

of beat. Correspond to placement in a List that is one based.

Note: 0 will refer to a sound if it is the only media file in the argument.

“+” Means extend or loop the Media sound 1/16th of a measure.

“-” Means 1/16th measure of rest.

For Loops: Skip CountingfillDrum = HIP_HOP_SYNTHDRUMS2_2Mbeat = "0+++0+++0-000+00"

for measure in range(1, 9, 2):makeBeat(fillDrum, 1, measure, beat)

measure is the “index variable” = assigned values from therange()

(1, 9, 2) means start counting at 1, end before 9 [meaning 8] and skip count by 2:

(1, 3, 5, 7)

Functions: Recycle and Reuse!

Now I can use this section anywhere!sectionA(1, 9)sectionA(17, 25)

Exercises for Classroom

•Create EarSketch Account•Mix1: (AB Section Exercise)

• Use fitMedia and setEffect• Music in sections

•Mix2: (makeBeat and For Loop Exercise)• makeBeat• For Loop Structure

•Mix3: (Defining Functions Exercise)•Final Mix

Control and Data StructuresIf Statement:

Lists

Part Two: Programming with Java and Processing

What is Processing?

● Java based programming environment (IDE) from MIT● Free!!!● Designed for creating interactive Art● Data Visualization● Prototyping Tool● Many libraries for interfacing with other hardware● Very High Ceiling and Clean interface● http://nebomusic.net/processinglessons/

Screenshots: IntelliJ compared to Processing

Processing Java Topics•Data Types•Two Functions in Processing•Comments•Canvas•Shape Drawing•Pen•Colors•Control Structures

A Side Note: Comments

•Comments are for humans

•Computer ignores comments

•Use lots of them in programming

–Graded on Comments

–Good style

–Allows code to be used by others

•// Single Line Comment

•/* Multi line comments

•*/

Java Data Types

•int: Whole Numbers•float: Decimal Point Numbers

•String: Characters and Letters

•boolean: 1 Bit value storing ‘true’ or ‘false’ (Sometimes ‘1’ or ‘0’)

•char: 16 Bit Unicode character ranging from \u0000 to \uffff (Hexidecimal)

–This is a single Letter or character•String: (This is actually a class)–java.lang.String–‘string’ of characters: like “Mr.

Michaud”

Operators and Logic•String name = “Bob”; // Used to Assign a value•name == “Bob” // ‘is equal to’•4 * 5 // multiply•20 / 4 // Divide•4 + 5 // Addition•4 – 5 // Subtraction•5 % 4 // Modulo•5 > 4 •4 < 5 •5 >= 4•4 <= 5

Logic:

&& // Means “and”|| // Means “or”

Two Functions in Processing

•All Processing Programs must have these two functions:

–void setup()–void draw()

setup() and draw()

•void setup()–Runs one time–Initialize Objects–Set Canvas size and Color–Still Images

•void draw()–Runs in a loop–Used in Animation–Moving Images by changing

properties over time

Canvas

Canvas Functions•size(width, height); // Sets the

Size of the Canvas•background(red, green, blue);

// Sets the color of the canvas

2D Shape Functionsline(x1, y1, x2, y2);

rect(x, y, width, height);

ellipse(x, y, width, height);

triangle(x1, y1, x2, y2, x3, y3);

quad(x1, y1, x2, y2, x3, y3, x4, y4);

arc(x, y, width, height, start, stop);

Note: Arc start and stop are in RADIANSUse radians(deg) to convert.

Color and Pen in Processingfill(red, green, blue); // inside of shape

fill(red, green, blue, alpha); // inside of shape

stroke(red, green, blue); // outside of shape

stroke(red, green, blue, alpha); // outside of shape

strokeWeight(10); // Thickness of shape

Color Representations in Graphic Systems• Red, Green , Blue (Standard)

– Values from 0 to 255 (Base 10)• 0 is Dark (no light), 255 is maximum

– Values of 00 to ff (Hexadecimal)• #ff0000 is Red or same as (255, 0, 0)

– Mix Red, Green, and Blue to Create all Colors

• Standard in Computer Graphics (Apple, Android, Java, Maya . . )

Color in Processing: H, S, B

• Hue, Saturation, Brightness• Device independent• Available in Processing

http://www.tomjewett.com/colors/hsb.html

Functions in Java to Create Complex Shapes• Return types:

– void: Does not return a Value. Performs an action– int: Returns an integer value– float: Returns a float value– Any data type or object can be a function return type

• Parameters: Values passed into the function• Brackets {}

– Define regions of the Function

Example Functions

Sample student created functions

Objects in Processing: Modeling with Java ClassesWhat is a Class?Collection of Variables and Functions that model an object.Blueprint through which we build instances.

Remember:Variables store valuesFunctions store sequences of directionsClasses store models of objects with variables and functions.

Parts of a ClassNameFields

-Variables and Data structures that store the properties of the object

Functions-Code that gives the Class the ability to perform actions-Accessors, Modifiers, Void(Getters, Setters, and Action)

Constructor-Builds the instance and returns it to the program

The Actor ClassImplements the idea of the Sprite from Scratch.Models a 2D Character / Agent in the Animated World.Automates movements, collision, edge interaction, colors, rotation, scaling,

and polygons.Code is provided to students as a library.Actor code is open source to students so they can modify if desired.

http://nebomusic.net/processinglessons/Actor.txt

Actor Class Fields and Functions

For Loops•Repeats section of code while counting up or down with an index

variable•Example

for (int i = 0; i < 10; i++) {System.out.println(i);

}

for (int i = 0; i < 10; i++) {println(i);

}•i++ means "i = i + 1"•int i means "integer i"•for (int i = 0; i < 10; i++) means "For index variable i starting at 0, while i is less than 10, count be 1."

Conditional Statements

•‘if statement’: Checks if a given statement or expression is true and then executes a section of code

if (score > 9) {textScore.setText(“You Win”);

}

User Input and Fields

Overview of Computational Media Curriculum

Media: File Extensions - How computers store dataGraphics:

PhotoshopSketchUp

Programming:Processing: 2D Graphics

StillAnimation

EarSketch: MusicCode.org JavaScript Lessons in App Studio

Sequence of Lessons Computational Media EarSketch

• AB Exercise: http://nebomusic.net/earsketchlessons/A_B_Exercise_EarSketch2.pdf

• makeBeat Exercise: http://nebomusic.net/earsketchlessons/makeBeat_practice_EarSketch_2.pdf

• Functions Exercise: http://nebomusic.net/earsketchlessons/Directions_EarSketch2_Function_Exercise.pdf

• List Exercise: http://nebomusic.net/earsketchlessons/Directions_EarSketch2_Lists.pdf

• Final Project: http://nebomusic.net/earsketchlessons/EarSketch_Final_Project_v2.pdf

Sequence of Lessons Used in Computational Media Processing

• Boat Exercise: http://nebomusic.net/processinglessons/Processing_Exercise_03a_Functions_Written.pdf

• Function Exercise: http://nebomusic.net/processinglessons/Processing_Exercise_01a_Functions.pdf

• Actor Class Project: http://nebomusic.net/processinglessons/Class_Based_Exercise_Requirements_v4.pdf

• Simple Game Template: http://nebomusic.net/processinglessons/SimpleGameProcessing.pdf

• Particle Simulation (Make it Rain): http://nebomusic.net/processinglessons/Arrays_Loops_Act.pdf

• Final Design Process: http://nebomusic.net/processinglessons/Process_Processing_Final_Project.pdf

References

•Key Java Vocabulary: (http://docs.oracle.com/javase/tutorial/java/nutsandbolts/index.html)

•Processing Official Reference:http://processing.org/reference/