Post on 11-May-2015
Theory of Computation
Mathematical Preliminaries
Vladimir KulyukinDepartment of Computer Science
Utah State University
www.vkedco.blogspot.com
Outline
● What is Theory of Computation?● Alphabets, Strings, Languages● Numbers, Sets, Set Formers● Proofs
www.vkedco.blogspot.com
What is Theory of Computation?
● Theoretical Computer Science can be broadly divided into algorithms and computability (aka theory of computation)
● The field of algorithms answers the question – how something can be computed?
● The field of computability answers the question – can something be computed?
www.vkedco.blogspot.com
Basic Methodology
● Abstraction of hardware details● Focus on what can be solved, not on how it can
be solved● Problem analysis in terms of devices (aka
automata) and inputs (aka strings, languages)
www.vkedco.blogspot.com
Basic Methodology
● Abstraction of hardware details● Focus on what can be solved, not on how it can
be solved● Problem analysis in terms of devices (aka
automata) and inputs (aka strings, languages)
www.vkedco.blogspot.com
Abstraction of Hardware Details
Junun Mark III Robot
SmartphonePersonal Computer
While these devices have very different hardware, they have the same computational model
www.vkedco.blogspot.com
Focus on What, Not How
● Problem: sorting a sequence of numbers from smallest to highest
● Algorithmic answer: merge sort, heap sort, quick sort, etc
● Computability answer: sorting is primitive recursive
www.vkedco.blogspot.com
Automata & Languages
● Finite State Automata/Regular Expressions – Regular Languages
● Push Down Automata/Stack Machines – Context-Free Languages
● Linear Bounded Automata – Context Sensitive Languages
● Turing Machines/Universal Programs – Recursively Enumerable Languages
www.vkedco.blogspot.com
Alphabets
● An alphabet is a finite set of symbols● The Greek letter Σ is typically used to denote an
alphabet● Examples: Σ1 = {a, b}, Σ2 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.● The symbols in the alphabet do not have any
meaning in and of themselves
www.vkedco.blogspot.com
Strings
● A string is a finite sequence of symbols● ε is the empty string; ε is not a symbol in any
alphabet; it denotes the string with zero symbols● In the book (Davis et al.), the empty string is
denoted as 0● In formal language theory, strings are typically
written without quotation marks: aab and 010001 instead of “aab” and “010001”
www.vkedco.blogspot.com
Strings
● The length of a string is the number of symbols/characters in it
● The length of a string is denoted with a pair of matching vertical lines around it
● Examples: – if x = aab, then |x| = 3; – if x = ε, then |x| = 0
www.vkedco.blogspot.com
String Concatenation
● The concatenation of two strings x and y is the strings containing the symbols of x followed by the symbols of y
● Examples: if x = ab and y = 100, then xy = ab100; if x = ε and y = abc, then xy = abc
● For any string, xε = εx = x
www.vkedco.blogspot.com
Power Notation in String Concatenation
● When a natural number n is used as an exponent on a string, it denotes the concatenation of that string with itself n times
● Examples: – x0 = ε – x1 = x – x2 = xx – (ab)3 = ababab
www.vkedco.blogspot.com
Languages
● A language is a set strings over a alphabet● Note that we can define multiple languages over
the same alphabet● Example: Σ = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}. L1 = the set
of all strings over Σ that end in 0 (e.g., 0, 110, 213450, etc.); L2 = the set of all strings over Σ that end in 1 (e.g., 1, 01, 0001, 91, etc.)
www.vkedco.blogspot.com
Kleene Closures of Alphabets
● The Kleene closure of an alphabet is the set of all strings over it
● If Σ is an alphabet, its Kleene closure is written as Σ*
● Example: Σ = {a, b}. Σ* = the set of all strings consisting of a's and b's, including the empty string
www.vkedco.blogspot.com
Natural Numbers
● N is a set of natural numbers.● N includes 0, 1, 2, 3, …● Some texts exclude 0 from the set of natural
numbers, but we will keep it in● In the texts, the word number refers to natural
number
www.vkedco.blogspot.com
Set Former Notation
{ }{ } { }
{ } { }
{ } { }{ }.or by followed is or
wherestrings ofset theis ,,,|
s.' ofnumber the
toequal is s' ofnumber theand s' precede s'that
such ,over stringsempty -non ofset theis 1|
3.or 2, 1, 0, islength whose
,over strings all ofset theis 3|, *
ccaaba
ccaaybaxxy
b
aba
banba
baxbax
nn
∈∈
≥
≤∈
www.vkedco.blogspot.com
Subsets
.
:setevery ofsubset a isset empty The
. and if)only and (if iff
R
RSSRSR
⊆∅
⊆⊆=
www.vkedco.blogspot.com
Set-Theoretic Equalities
s.complement theofunion theis
onintersecti theof complement thei.e. ,
s.complement theofon intersecti the
isunion theof complement thei.e. ,
).(
. and ofon intersecti -
SRSR
SRSR
SRRSR
SRSR
∪=∩
∩=∪
∩−=−∩
www.vkedco.blogspot.com
Sets and N-Tuples
( ) ( ) ( ).,,,,,,
:matter does sequences ain elements oforder The
}.,,{},,{},,{
:matternot doesset ain elements oforder The
set. a is },...,,{ 21
bacacbcba
cabbcacba
aaa n
≠≠
==
www.vkedco.blogspot.com
Sets and N-tuples
}.,...,,|),...,,{(
...
:follows as defined isset theseofproduct
Cartesian Then the sets. are ,...,,Let
221121
21
21
nnn
n
n
SaSaSaaaa
SSS
SSS
∈∈∈=×××
www.vkedco.blogspot.com
Predicates
.0)(or 1)(or )(or )(either
each for such that on function
valued-Boolean totala is predicate A
====∈
aPaPFaPTaP
SaS
P
www.vkedco.blogspot.com
Predicates
R. offunction sticcharacteri a is )(
}1)(|{
Then
if 0
if 1)(
set. a be Let
xP
xPxR
Rx
RxxP
R
==
∉∈
=
www.vkedco.blogspot.com
Proof Methods
● In CS, there are, broadly speaking, two methods of proving things: formal and empirical
● Formal methods are used in theory of computation, algorithms, operations research, etc.
● Empirical methods are used in many applied branches of CS
● Many R&D projects combine formal and empirical methods
www.vkedco.blogspot.com
Mathematical Proofs
● The corner stone of the formal method is the mathematical proof
● Many online and printed CS materials contain proofs● It is of vital importance for a CS practitioner to read at
least some proofs● The good news is that reading proofs is significantly
easier than doing them
www.vkedco.blogspot.com
Proof Techniques
● Proof techniques are independent of their subject matter: valid proofs in calculus use the same proof techniques as valid proofs in algorithms or theory of computation
● Common proof techniques can be identified● Learning to identify common proof techniques will enable you to
study many areas of CS independently● The ability to identify proof techniques is based on your ability to
understand how the technique works and when it is likely to be applicable
● In CS, a prominent proof technique is induction
www.vkedco.blogspot.com
Learning to Love the P-Word
● General advice: Do not be afraid of proofs; one can be a mediocre theorem prover but a very good proof reader
● The first step in mastering the art of mathematical proof is to read and do proofs of known facts; do not think of it as a waste of time
● When you read some CS material, do not shy away from it, if it contains proofs
www.vkedco.blogspot.com