theory 1.docx

7/28/2019 theory 1.docx

1/6

[next][tail][up]

6.1 Error-Free Probabilistic Programs

Randomization is an important programming tool. Intuitively, its power stems from choice. The

ability to make "random choices" can be viewed as a derivation of the ability to make"nondeterministic choices."

In the nondeterministic case, each execution of an instruction must choose between a number ofoptions. Some of the options might be "good", and others "bad." The choice must be for a good

option, whenever it exists. The problem is that it does not generally seem possible to make

nondeterministic choices in an efficient manner.

The options in the case for random choices are similar to those for the nondeterministic case,

however, no restriction is made on the nature of the option to be chosen. Instead, each of the

good and bad options is assumed to have an equal probability of being chosen. Consequently, the

lack of bias among the different options enables the efficient execution of choices. The burden ofincreasing the probability of obtaining good choices is placed on the programmer.

Here random choices are introduced to programs through random assignment instructions of the

form x := random(S), where S can be any finite set. An execution of a random assignment

instruction x := random(S) assigns to x an element from S, where each of the elements in S isassumed to have an equal probability of being chosen. Programs with random assignmentinstructions, and no nondeterministic instructions, are calledprobabilistic programs.

Each execution sequence of a probabilistic program is assumed to be a computation . On a giveninput a probabilistic program might have both accepting and nonaccepting computations.

The execution of a random assignment instruction x := random(S) is assumed to take one unit oftime under the uniform cost criteria, and |v| + log |S| time under the logarithmic cost criteria. |v| is

assumed to be the length of the representation of the value v chosen from S, and |S| is assumed todenote the cardinality of S.

A probabilistic program P is said to have an expected time complexity (x) on input x if (x) is

equal to p0(x) 0 + p1(x) 1 + p2(x) 2 + . The function pi(x) is assumed to be the probability for

the program P to have on input x a computation that takes exactly i units of time.

The program P is said to have an expected time complexity (n) if (|x|) (x) for each x.

The following example shows how probabilism can be used to guarantee an improved behavior

(on average) for each input.

Example 6.1.1 Consider the deterministic program in Figure6.1.1
http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.html


2/6

call SELECT(k, S)

procedure SELECT(k,S)

x := first element in S

S1 := { y | y is in S, and y < x }

n1 := cardinality of the set stored in S1

S2 := { y | y is in S, and y > x }

n2:= cardinality of the set stored in S

2

n3 := (cardinality of the set stored in S) - n2

case

k n1: SELECT(k, S1)

n3 < k : SELECT(k - n3, S2)

n1 < k n3: x holds the desired element

end

end

Figure 6.1.1A program that selects the kth smallest element in S.

(given in a free format using recursion). The program selects the kth smallest element in any

given set S of finite cardinality.

Let T(n) denote the time (under the uniform cost criteria) that the program takes to select an

element from a set of cardinality n. T(n) satisfies, for some constant c and some integer m < n,

the following inequalities.

From the inequalities above

T(n) T(n - 1) + cn

T(n - 2) + c

T(n - 3) + c

T(1) + c

cn .

That is, the program is of time complexity O(n2).

The time requirement of the program is sensitive to the ordering of the elements in the sets inquestion. For instance, when searching for the smallest element, O(n) time is sufficient if the

elements of the set are given in nondecreasing order. Alternatively, the program uses O(n2) time

when the elements are given in nonincreasing order.


3/6

This sensitivity to the order of the elements can be eliminated by assigning a random element

from S to x, instead of the first element ofS. In such a case, the expected time complexity (n)of the program satisfies the following inequalities, for some constant c.


(n) + cn

+ + + cn

+ c(1 - ) + cn

+ c + cn

+

+ + c + cn

+ 2c + cn

+ 2c + cn

+ 3c + cn

(1) + (n - 1)c + cn

2cn.

That is, the modified program is probabilistic and its expected time complexity is O(n). Forevery given input (k, S) with S of cardinality |S|, the probabilistic program is guaranteed to find

the kth smallest element in S within O(|S|2) time. However, on average it requires O(|S|) time for

a given input.

[next][front][up]

[next][tail][up]

6.1 Error-Free Probabilistic Programs

Randomization is an important programming tool. Intuitively, its power stems from choice. The

ability to make "random choices" can be viewed as a derivation of the ability to make"nondeterministic choices."
http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk.html#Q2-70001-2http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#tailtheory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.html


4/6

In the nondeterministic case, each execution of an instruction must choose between a number of

options. Some of the options might be "good", and others "bad." The choice must be for a good

option, whenever it exists. The problem is that it does not generally seem possible to make

nondeterministic choices in an efficient manner.

The options in the case for random choices are similar to those for the nondeterministic case,however, no restriction is made on the nature of the option to be chosen. Instead, each of the

good and bad options is assumed to have an equal probability of being chosen. Consequently, the

lack of bias among the different options enables the efficient execution of choices. The burden ofincreasing the probability of obtaining good choices is placed on the programmer.

Here random choices are introduced to programs through random assignment instructions of the

form x := random(S), where S can be any finite set. An execution of a random assignment

instruction x := random(S) assigns to x an element from S, where each of the elements in S isassumed to have an equal probability of being chosen. Programs with random assignment

instructions, and no nondeterministic instructions, are calledprobabilistic programs.

Each execution sequence of a probabilistic program is assumed to be a computation . On a given

input a probabilistic program might have both accepting and nonaccepting computations.

The execution of a random assignment instruction x := random(S) is assumed to take one unit oftime under the uniform cost criteria, and |v| + log |S| time under the logarithmic cost criteria. |v| is

assumed to be the length of the representation of the value v chosen from S, and |S| is assumed to

denote the cardinality of S.

A probabilistic program P is said to have an expected time complexity (x) on input x if (x) is

equal to p0(x) 0 + p1(x) 1 + p2(x) 2 + . The function pi(x) is assumed to be the probability for

the program P to have on input x a computation that takes exactly i units of time.

The program P is said to have an expected time complexity (n) if (|x|) (x) for each x.

The following example shows how probabilism can be used to guarantee an improved behavior

(on average) for each input.

Example 6.1.1 Consider the deterministic program in Figure6.1.1

call SELECT(k, S)procedure SELECT(k,S)

x := first element in S

S1 := { y | y is in S, and y < x }


S2 := { y | y is in S, and y > x }


n3 := (cardinality of the set stored in S) - n2

case
http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.html#70001-2007r6.1.1


5/6

k n1: SELECT(k, S1)

n3 < k : SELECT(k - n3, S2)

n1 < k n3: x holds the desired element

end

end

Figure 6.1.1A program that selects the kth smallest element in S.

(given in a free format using recursion). The program selects the kth smallest element in anygiven set S of finite cardinality.

Let T(n) denote the time (under the uniform cost criteria) that the program takes to select anelement from a set of cardinality n. T(n) satisfies, for some constant c and some integer m < n,

the following inequalities.


T(n) T(n - 1) + cn

T(n - 2) + c

T(n - 3) + c

T(1) + ccn .

That is, the program is of time complexity O(n2).

The time requirement of the program is sensitive to the ordering of the elements in the sets inquestion. For instance, when searching for the smallest element, O(n) time is sufficient if the

elements of the set are given in nondecreasing order. Alternatively, the program uses O(n2) time

when the elements are given in nonincreasing order.

This sensitivity to the order of the elements can be eliminated by assigning a random element

from S to x, instead of the first element ofS. In such a case, the expected time complexity (n)of the program satisfies the following inequalities, for some constant c.



6/6

(n) + cn

+ + + cn

+ c(1 - ) + cn+ c + cn

+

+ + c + cn

+ 2c + cn

+ 2c + cn

+ 3c + cn

(1) + (n - 1)c + cn

2cn.

That is, the modified program is probabilistic and its expected time complexity is O(n). For

every given input (k, S) with S of cardinality |S|, the probabilistic program is guaranteed to find

the kth smallest element in S within O(|S|2) time. However, on average it requires O(|S|) time for

a given input.

[next][front][up]
http://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-six.html#theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse1.htmlhttp://www.cse.ohio-state.edu/~gurari/theory-bk/theory-bk-sixse2.html

theory 1.docx

Documents

Transcript of theory 1.docx