Chapter 5: Synchronization !1smartynk/Resources/CMPUT 379/Notes/Le… · Chapter 5: Synchronization...

Chapter 5: Synchronization ��1

Chapter 5: Synchronization

Start of Lecture: February 3, 2014

��2


Reminders

• Put up Exercise 2 for Chapters 3-5; due February 12

• Put up Assignment 2; due March 7

• Assignment 2 is significantly harder than Assignment 1, so get working on it early; for this assignment, it really won’t work to wait until a couple of days before, it really need to be weeks before (particularly as its much more code as well)

• Any questions or comments?

��3


Let’s step back and re-evaluate

• Why are we learning this stuff about operating systems?

• Process control block? So what?

• What am I supposed to get out of my comp sci degree?

• of course, this is subjective

• One of the main goals: be able to understand computing

• all the way from high-level (algorithms, theory of computation) to

• low-level (how instructions represented in binary, how they executed on a CPU with switches, maybe in future quantum computers, etc.)

• with in-between being varying levels of abstraction in implementation and algorithm development

��4


Most courses try to teach algorithmic thinking and how to implement those solutions

• Operating systems course closer to the low-level side of the understanding

• for some, the specific knowledge will be useful for future projects

• others will stay closer to high-level algorithms side, where much of the low-level is abstracted away, making low-level knowledge less relevant

• The overarching theme, however, is taking a problem and

• Step 1. Abstract away the details to find a higher-level representation and understanding of the problem to simplify finding a solution

• Step 2. Take your high-level, abstract solution and turn in into an implementation (something that runs and finds the “answer”)

��5


Tackling Daunting Projects

• You have finished Assignment 1, and maybe you spent some time staring at a screen wondering where to begin

• This will be even more true for Assignment 2

• A core skill in Computing Science is to be able to take your plan (and/or a given specification) and convert it into an implementation; each assignment tries to make you better at this

��6


Main components to a project

• Understand the problem to be solved, or understand requirements for the software application, or …

• Research (i.e. find) or develop algorithms to solve the problem; usually there are many tools available

• e.g. max-flow algorithm to solve airline scheduling

• e.g. design patterns for large software projects

• Concisely write down the high-level design for solving the problem, to crystallize your plan and understanding

• Convert this plan into an implementation

��7


Stuck on the implementation?

• One issue might be that you have not crystallized your plan or thought about how to divide and conquer

• Think about how your code can be modularized, and possibly write down the sub-solutions that need to be implemented to solve the larger solution

• e.g. dot product thread function (nice sub piece for larger problem)

• Once you have a plan, don’t stare at the screen; start writing!

• “Writing a novel is like driving a car at night. You can see only as far as your headlights, but you can make the whole trip that way” – E. L. Doctorow

��8


Advice for completing your plan

• Pick low hanging fruit: you might feel immobilized by the larger problem, but there are usually parts you can easily complete e.g. function that sets up signal handler

• Copy-and-paste: re-use code you know works

• some programmers frown on this, but it nots black-and-white; can be a good starting point if you use it correctly

• if you thoroughly understand pasted code and properly simplify and minimize what is copied for your new application, then can speed implementation and save time

• e.g. reading through a file and obtaining lines correctly

• e.g. do non-blocking I/O correctly through sockets

��9



• Keep it simple and don’t be too much of a perfectionist

• Don’t design your code for the most general setting, as that gets too complicated and you don’t really know what you’ll need later; likely, you’ll end up scrapping your original implementation and writing it for the new (more general) setting later anyway

• Obviously, don’t hardcode everything either, but do try to avoid grand re-designs and refactoring to make your code general (rabbit hole)

• Small, clear wins in 25 minute bursts

• modularize your code properly so that you can sit down and complete a part, instead of leaving half-way through an overwhelming piece

��10



• Proper reconnaissance: use the resources around you

• look at really nicely implemented code to learn how other people factor their code and best practices in a language

• search the internet for similar problems and implementation ideas; letting these ideas percolate will let you better formulate a plan

• pairs programming: discuss problem with other people to brainstorm; when you get stuck, a second pair of eyes helps get out of the hole

• Make a small end-to-end prototype or skeleton that touches all major components after reconnaissance

• laying out your plan, even as a skeleton, is a big achievement

• can tackle unknown, more difficult cases/parts later

��11


Remember that this is still a Science

• This means more of a focus on problems and algorithms

• its not about learning any specific language; you’ll learn many in your career, because a language is a tool to solve problems

• programming in C is to show you how to manage resources, like memory, that other languages might abstract away; C is the tool for you to have closer control with the hardware, not the end goal

• This means being analytical when evaluating different algorithms

• investigate possible solutions; hypothesize which algorithm(s) more appropriate; then run a controlled (research) experiment with enough samples to make claims about significant differences in accuracy/speed between your implementations. Yes, your stats class is useful (and besides, data is beautiful)

• Maybe surprisingly, scientific exploration is not cut-and-dry; its about discussion and being creative (not just clean analysis)

��12


Once you’re done undergrad and programming a big project

• You will hopefully have obtained a tool belt of algorithms and solution techniques that will help guide your search towards the right set of algorithms for your problem

• You will hopefully have a better understanding of how to tackle big problems and divide-and-conquer (modularize)

• You will hopefully have cultivated an analytical mind

• You will hopefully have learned (more) how to learn

• But likely you won’t be an expert in any language; you will have to teach yourself and at that time become an expert in the required language (e.g. Objective C for an app)

��13


Simple Example: Assignment 1

• Needed to implement a function, get_mem_layout, to scan virtual address space

• Needed to test that function by writing test code

• Needed to separate (modularize) get_mem_layout into its own file, to be included in other code that wants to be able to scan its address space

��14


One possible plan for Assignment 1• Write test code, assuming get_mem_layout is

implemented; separate into header file and make sure everything is compiling correctly with get_mem_layout temporarily a skeleton with no functionality

• Start tackling main components of get_mem_layout

• write skeleton for check_page function to check if memory location is R/W: check_page(char * address) { }

• write skeleton code in get_mem_layout that loops over address space and calls check_page on each address

• write skeleton for function that setups signal handler to catch segmentation faults from check_page

• write skeleton for function that set the values in a chunk

��15


static void setup_handler() {}!static void check_page(char * address) {}!static void fill_chunk(struct memchunk *chunk) {}!int get_mem_layout { setup_handler(); while (address < MAX_ADDRESS) { /* Determine readable-writable */ check_page(address); /* After checking, we should store info */ fill_chunk(current_chunk); }}

Initial non-compiling layout to start tackling the assignment


One possible plan for Assignment 1

• Start picking off low-hanging fruit

• Fill in code for setting up a signal handler (copy-paste from sig2.c)

• Fill in code to check if a page is readable or writable (start to understand why you need the signal handler function)

• Fill in code that sets the values in a chunk, like RW (start seeing what global variables you might need)

• Determine PAGE_SIZE so can increment address variable

• Start thinking about more difficult procedure of where to jump after signal is caught, and so the order that functions should be called in the main loop

• But now this difficult part is simpler because other simpler components separated from this task, and you’ve identified the difficult part

��17


What are your approaches?• Going from high-level abstraction to implementation is

often quite different for each person; in teams, often iterate discussion to brainstorm a plan and implementation

• and in teams often do code reviews of each others code to sync

• What approaches do you use? What did you do to complete Assignment 1?

• Anyone use pairs programming to help find bugs? Some typical bugs I saw were

• using int instead of unsigned long, and getting overflow (infinite loops)

• overwriting memory when checking if could write, and not restoring to the original value if the write was successful

��18


Video Break: brought to you by a wonderful classmate

��19


More about threads and modularizing

• Pretend someone asks you to implement a server, which accepts connections and hands them off to threads

• You know about pthreads, but need to think about how you are going to design and modularize your system

• Where do you start?

��20


What are some of the first things you might think about when approaching this problem?

• How do you write a server to accept connections? Is there some code out there I can look at that I trust?

• What is the core thread functionality? Are there different group of threads doing different things?

• What information are the threads sharing? Will there be issues? I remember something about semaphores for synchronization from that CMPUT 379 class, maybe I should look into those again?

��21


Demo: server.c

• Let’s look at server.c to start with code we trust

��22


Start of Lecture: February 5, 2014

��24


Reminders

• You should be reading Chapters 4, 5 and 6; we are starting into CPU scheduling today

• It’s a good idea to stay on top of the readings, because the final exam will draw a lot from the book, so you want to know the material

• you’re expected to know the readings in the book, even if I don’t explicitly lecture on it

• Exercise 2 due next Wednesday

��25


Demo: server.c

• Let’s finish off a few things with server.c

• Friday, Jan. 14, Ankush Roy will discuss select_server.c and how to properly do non-blocking I/O

• due to your understanding of busy waiting versus performing a self-block to context-switch off the processor, this distinction will be much simpler for you (yah for understanding what you’re coding!)

��26


How to implement a server with fork

• server.c shows how to set up a socket, listen for connections

• server.c gives you some idea of what you need to do when you fork off children to handle connections

• including making sure children are not left as zombies

• server.c only needs to write a fixed buffer (write a response)

• What might child have to do in addition for HTTP server?

• needs to read client request: GET filename HTTP/1.1

• needs to try to locate file and send back a tailored response (either the file or a not found response, or an error, etc.)

• needs to write event to server log

��27


How to implement a server with threads?

• How might the separated code that handles a new accepted client connection be different for threads?

• Whenever you think threads, you think ease in sharing memory and variables

• Whenever you think sharing memory and variables, you think mutexes and semaphores

��28


Some best practices in C

• Use strncpy instead of strcpy to prevent buffer overflow

• same is true of using fgets instead of gets, snprintf instead of sprintf

• Use portable functions (probably have to look it up)

• e.g. use C library for system calls, rather than directly calling system call

• Reasonably optimize code, as cannot always know what compiler might make more optimal

��29

char *buff = “Hammer time!”;while (/* Some condition */) { … if(len > strlen(buff)) /* Do some stuff */ …}

char *buff = “Hammer time!”;int buff_len = strlen(buff);while (/* Some condition */) { … if(len > buff_len) /* Do some stuff */ …}


Some best practices in C!

• Do not typedef structs — obfuscates that the variable is a struct and pollutes namespace

• Generally avoid globals variables; but, otherwise, preface non-extern globals and private functions with static

• not too important, but can prevent someone from incorrectly changing scope

• Use const for function arguments so variable need not be copied (code explicitly states that will not modify it)

��30

static int check_state(const int var) { if (var > 0) return 1; return -1;}

/* This won’t compile! */static int check_state(const int var) { var = 5; return -1;}


Some best practices in C• Write clear code — don’t obfuscate to get fewer lines

• Headers are for encapsulation — should only contain functions you want exposed to outside world; prototype private functions inside c file

• Use structs for nice encapsulation (like object-oriented programming, can be good for abstraction)

• Handle the case that a header is included multiple times

��31

#ifndef MY_UTILS_H#define MY_UTILS_H/* Header interface for all functions you want to make available */#endif /* MY_UTILS_H */


Review of last 5 chapters

• Services provided by the OS: system calls

• to run in kernel mode, need to ask kernel to run a system call

• Multi-programming and concurrency: switching between tasks to let all processes make progress

• key notion of process as entity with states (e.g. ready, running, blocked)

• context-switching generally initiated by an interrupt, involves storing and loading state of process and involves scheduling processes

• Multi-processing and parallel computing

• determining how to break down problem in subtasks, with data parallelism (e.g. processing separate parts of data) and task parallelism (e.g. searching over many possible configurations)

��32


Review of last 5 chapters• Process tree in Linux, children created with fork, with

parents responsible for resource use (etc.) of children

• Inter-process communication: shared memory and message passing (such as signals, sockets)

• with a focus on synchronous (blocking) versus asynchronous (non-blocking) communication

• Threads as a light-weight unit of work, difference with processes created by fork()

• Synchronizing cooperating processes

• mutexes (spin locks) and binary semaphores (blocking wait) for locking and counting semaphores for synchronizing work and resources

• monitors for guaranteed mutual exclusion

��33


A threads review questions

• Parallel max: finding the max value in an array of size N using M threads.

• Fill in the pthread function; don’t forget to use whatever mutexes you might need to lock critical sections

• Code comments describing what code should be there; left out most of code in main, so mentally fill it in

��34

int max_val;int array[100];pthread_mutex_t mutex1, mutex2; /* or more */ !int pthread_fcn(void *threadarg) { /* What is the required threadarg? */ /* What needs to be locked? */}!int main() { pthread_t threads[10]; pthread_mutex_init(&mutex1, NULL); /* Fill array with random values */ max_val = array[0]; /* init max*/ /* Create threads */ /* * Join on the threads, since have * to wait for them to complete */ printf(“The max value is: %d\n”, max_val); pthread_exit(NULL); }

Chapter 5: Synchronization !1smartynk/Resources/CMPUT 379/Notes/Le… · Chapter 5: Synchronization...

Documents

Transcript of Chapter 5: Synchronization !1smartynk/Resources/CMPUT 379/Notes/Le… · Chapter 5: Synchronization...