March 17, 2020 © Sam Siewert

CS 332

Programming Language Concepts

Lecture 12 – Alternative Language

Examples

(API Concurrency vs. Built-in

Language Concurrency)

Tell your Story about Programming

Use the oldest methods to make convincing

science and technical arguments …

© Sam Siewert

2

Common PL vs. Alternative PL StoryPick and adventure - Very simple algorithm or app of interest

Research PL that might give advantage over common

Implement and test algorithm/app in both PLs

– 3+ Objective Metrics - measurable and quantifiableSpeed of execution

Size (compiled image, SLOC, assembly statements)

Complexity (e.g. McCabe cyclomatic)

Memory used for common algorithm (global, heap, stack)

CPU loading for common algorithm

Speed of compilation or interpretation

– 3+ Subjective Observations - Majority would agree when viewed side by sideTwo blocks, functions, modules compared - readability

Syntax equivalents

Semantic equivalents

Testimonials by experts

Number of people using, size of largest project, number of projects

Ease of debugging

Ease of learning

Built-in features vs. need to use libraries (paraphrase extension)

Moral of the Story - Stick with Common, or Use Alternative

© Sam Siewert 3

Final Presentation – the ObjectiveDefine terminology and concepts, #1

Use references (expert voice) to support hypothesis, #2, #3

Present data (metrics) and how you gathered, #4

Provide examples (code side by side) and summary of results, #5, your conclusions, #6

Tell a story using Scientific method1. Play with and learn primary and alternate PLs to make

observations

2. Choose a problem to solve and research

3. Form a hypothesis about which PL will help you solve the problem (and related problems) best

4. Compose and complete tests to verify / refute your hypothesis

At least 3 objective metrics

At least 3 qualitative (subjective)

5. Analyze and summarize you results

6. Present final conclusions (positive, negative, mixed outcome)

© Sam Siewert 4

Definition Testimonial

Example Data

Key terms

Concepts,

Theory

Expert

References

Tell a

Story!

Relate

Data

(graphic)

Telling a Research Story

Ask a Question

Do research (what

other experts think)

Hypothesis

Test with experiment

– Objective measures

– Subjective

– Arguments or Proofs

(inductive, deductive,

transductive)

Analyze / Summarize

Conclusion -

Positive/Negative

Outcome

© Sam Siewert 5

Example – Part 1

Does Lisp/Prolog/R/Scheme offer an advantage for

problems I want to solve?

– What problem do I want to solve?

– What algorithm, application, technology interests me?

– What alternate (new) programming language interests me?

What do experts say regarding an alternate

programming language? Problem? Algorithm?

– Start easy – Rosetta Code

– Explore deeper – google.com/scholar

– Go real deep – Hazy library, ask and expert on Research Gate

My hypothesis is – PL A has advantage over B, we

should switch (port) from B to A

© Sam Siewert 6

Example – Part 2Design tests and experiments

– Objective (by measure)

How big (output code size)

How fast (execution time)

Size of source input (lines of code)

Correctness (by induction, deduction, transduction)

– Subjective (by example)

How easy (to write, read, understand)

Debugging

Support documentation

Number of users and trends of use

Arguments and Proofs

– What does each measure and example mean?

– Why? How do you know?

© Sam Siewert 7

Analyze and Summarize

Big picture from each result – overall

– E.g. Scoring and weighting

– Visualizations

– Summary – experts were right/wrong, I was right/wrong, results

are conclusive (positive, negative), inconclusive (mixed), majority

(positive, negative)

Conclusion

– For problem studied PL A is better than B

– I plan to use PL A

– You should also use PL A for the same types of problem (class

of problem – e.g. sorting)

– Exceptions

– Future work

© Sam Siewert 8

Using Rosetta Code – To ExploreSearch for a Common

problem, task, or algorithm

Compare 2 Programming

Language Contributed

Solutions

© Sam Siewert 9

The Problem

Uses a library, “algorithms”, but common

Uses “sort” function, but common

Dig down one more level – to

sort and algorithms

Rosetta Code for Sort

Choose an algorithm (research category)

– Quick sort

– Insertion sort

– Heap sort

– Bubble sort

– Etc.

E.g. Quick sort of keys (verify contributed solutions)

© Sam Siewert 10

Verification – Objective Measures

1. Correct Implementation?

2. How fast can it run on common test hardware?

3. How big is the generated code?

4. How many lines of code for each total?

5. Can the program scale? [Limits – e.g. how many

records can it sort, divide and conquer, parallel?]

© Sam Siewert 11

Assessment - Subjective

1. Is the correct implementation hard to understand?

a. Side by side comparison

b. Libraries used [how much abstraction provided] – fair

comparison?

2. Does the programming language have a good compiler

or interpreter and IDE?

3. Does the PL and IDE provide good debugging?

4. How many people are using the PL?

5. Does it have good documentation and support?

© Sam Siewert 12

Interesting Google Trends – Subjective

Metrichttps://www.google.com/trends/ - Search Trends

Measure of Interest, Affinity (Like) or Confusion?

© Sam Siewert 13As of 4/3/2017

Concurrency - Terminology

Multiple CPU Cores (Flynn’s Taxonomy)– Multi-Core (MIMD, MPMD)

– Vector Processing (SIMD)

– Micro-Parallelism (SIMT)

– Co-processors (SPMD)

Multiple Software Execution Contexts

Concurrent

Parallel

Multi-programming

Multi-threaded

© Sam Siewert 14

Programming Languages with

Built-in Concurrency

Why not all?

Are libraries and OS better?

© Sam Siewert

15

Major ConceptsConcurrent Processing– Processes, Tasks, Threads

1. For Multi-programming to Overlap I/O Blocking or Leverage Multi-core with Shared Memory – POSIX Threads

2. For Multi-core and Multiple CPU Systems for Services and Parallel Applications (Shared Memory) - OpenMP

3. For HPC Clusters to Scale Processing – MPI, OpenMPI

For Tasking with Shared Memory (Threads) #1 and #2, Alternative is Built-in Language Concurrency

1. Ada95 Tasking, Using Linux GNAT, “sudo apt-get install gnat”

2. Java Threads, DrJava or Eclipse on Linux for Java

3. C++ 2011 – FAQ, New C++11 Design Goals, C++11 Features

© Sam Siewert 16

Java Example

© Sam Siewert 17

public class simpleThreads {

private static int gsum=0;

private static class incrementThread implements Runnable {

public void run() {

for(int i=0; i < 100; i++) {

gsum=gsum+1;

System.out.format("incrementThread: gsum=%d\n", gsum); } } }

private static class decrementThread implements Runnable {

public void run() {

for(int i=0; i < 100; i++) {

gsum=gsum-1;

System.out.format("decrementThread: gsum=%d\n", gsum); } } }

public static void main(String args[]) throws InterruptedException {

System.out.println("Starting increment and decrement threads");

Thread inct = new Thread(new incrementThread());

Thread dect = new Thread(new decrementThread());

inct.start();

dect.start();

System.out.println("Main will loop until it can join both threads");

// Option 1 - just wait indefinitely

inct.join();

dect.join();

System.out.println("TEST COMPLETE\n");}}

Java Output

Fairly Deterministic for

10 iterations each

For 100, More Variation

(Chance for Context

Switching)

Are the Outputs Even in

the Right Order? (Check

with Timestamps)

Are the Outputs Valid?

© Sam Siewert 18

Welcome to DrJava.

> run simpleThreads

Starting increment and decrement threads

Main will loop incrementThread: gsum=1

incrementThread: gsum=1

incrementThread: gsum=2

incrementThread: gsum=3

incrementThread: gsum=4

incrementThread: gsum=5

incrementThread: gsum=6

incrementThread: gsum=7

incrementThread: gsum=8

incrementThread: gsum=9

decrementThread: gsum=0

decrementThread: gsum=8

decrementThread: gsum=7

decrementThread: gsum=6

decrementThread: gsum=5

decrementThread: gsum=4

decrementThread: gsum=3

decrementThread: gsum=2

decrementThread: gsum=1

decrementThread: gsum=0

TEST COMPLETE

>

C Pthreads Example

© Sam Siewert 19

int gsum=0;

void *incThread(void *threadp){

int i; threadParams_t *threadParams = (threadParams_t *)threadp;

for(i=0; i<COUNT; i++) gsum++;}

void *decThread(void *threadp){

int i; threadParams_t *threadParams = (threadParams_t *)threadp;

for(i=0; i<COUNT; i++) gsum--; }

int main (int argc, char *argv[])

{

int rc;

int i=0;

threadParams[i].threadIdx=i;

pthread_create(&threads[i],(void *)0,incThread, (void *)&(threadParams[i]) );

i++;

threadParams[i].threadIdx=i;

pthread_create(&threads[i], (void *)0,decThread, (void *)&(threadParams[i]));

for(i=0; i<2; i++)

pthread_join(threads[i], NULL);

printf("TEST COMPLETE\n");

}

C Output

Which Goes First

Appears to Go to

Completion

Could we Count on a

Specific Sequence

We can Count on 0 at

End?

© Sam Siewert 20

$ ./pthread

Decrement thread idx=1, gsum=-1

Decrement thread idx=1, gsum=-2

Decrement thread idx=1, gsum=-3

Decrement thread idx=1, gsum=-4

Decrement thread idx=1, gsum=-5

Decrement thread idx=1, gsum=-6

Decrement thread idx=1, gsum=-7

Decrement thread idx=1, gsum=-8

Decrement thread idx=1, gsum=-9

Decrement thread idx=1, gsum=-10

Increment thread idx=0, gsum=-9

Increment thread idx=0, gsum=-8

Increment thread idx=0, gsum=-7

Increment thread idx=0, gsum=-6

Increment thread idx=0, gsum=-5

Increment thread idx=0, gsum=-4

Increment thread idx=0, gsum=-3

Increment thread idx=0, gsum=-2

Increment thread idx=0, gsum=-1

Increment thread idx=0, gsum=0

TEST COMPLETE

C Output on Multi-Core

Multi-Core has More

Switching

Less of a Sequence

Shows This Really is

Non-Deterministic

Result is Still 0

© Sam Siewert 21

[ssiewert@everest]$ ./pthread

Increment thread idx=0, gsum=1

Increment thread idx=0, gsum=1

Increment thread idx=0, gsum=2

Increment thread idx=0, gsum=3

Increment thread idx=0, gsum=4

Increment thread idx=0, gsum=5

Decrement thread idx=1, gsum=0

Decrement thread idx=1, gsum=5

Decrement thread idx=1, gsum=4

Decrement thread idx=1, gsum=3

Decrement thread idx=1, gsum=2

Decrement thread idx=1, gsum=1

Decrement thread idx=1, gsum=0

Increment thread idx=0, gsum=6

Increment thread idx=0, gsum=0

Increment thread idx=0, gsum=1

Increment thread idx=0, gsum=2

Decrement thread idx=1, gsum=-1

Decrement thread idx=1, gsum=1

Decrement thread idx=1, gsum=0

TEST COMPLETE

More on ConcurrencyCovered in Operating Systems – Fair threading on one Multi-core Platform

Real-Time Operating Systems– AMP – Asymmetric Multi-core Processing (Unbalanced)

– SMP – Symmetric Multi-core Processing (Balanced)

Parallel and High Performance Computing– NUMA vs. UMA (Uniform, Non-Uniform Memory Access)

– Message Passing vs Shared Memory

CUDA – Thread Grids for Data Transformation

MapReduce - Hadoop

© Sam Siewert 22

Computer Architecture TrendsMulti-Core, Many-Core

– E.g. Intel Xeon Phi MICA, not MICA wearable

Vector Processing Instructions – ARM NEON

– Intel SSE

– POWER Architecture ALTIVEC

Vector Co-Processing– GP-GPU (E.g. NVIDIA Tegra X1, Tesla/Kepler/Fermi)

– AMD ATI StreamProc

HP – “The Machine”, Memristor, Optical, Collapsed Memory Hierarchy

IBM – Power8 Architecture

ARM – SoCs [Altera Cyclone V SoC, Qualcomm, NVIDIA, Apple, Texas Instruments, Marvel, …]

General Commodity Software-Defined Clusters [Google, Facebook, Hadoop, Seamicro (Now Defunct)]

Clear Trend toward Concurrency, Threading, Message Passing, MapReduce

© Sam Siewert 23

Discussion

More Latest PL Revisions Include Concurrency as a

Built-in Language Feature – Good? Bad? Don’t-Care?

Should Concurrency Be an Operating System Feature?

Should it be a PL Run-time Feature?

Should it Be Provided by a Library?

Why is Concurrency a Contemporary Issue in Computer

Engineering, Software Engineering and PL Design?

© Sam Siewert 24