Lecture 07 Analog-Digital
Computation
1Discrete Structures - 2015
Alan Turing, 1912-1954
Claude Shannon,1916-2001
The Most Important Man you have Never Heard of
2Discrete Structures - 2015
Claude Elwood Shannon 1916 - 2001
Claude Shannon’s Master Thesis at MIT (1937)
3Discrete Structures - 2015
• A switch between two terminals is a device that is either closed (connecting the terminals), or open (disconnecting the terminals)
- For example, a light switch opens and closes the flow of current from a source to your light bulb
Basic Problem Studied in the Thesis
4Discrete Structures - 2015
Closed switch.Current flows.
Open switch.Current doesn’t flow.
• A switch network is a network of switches between a set of input terminals and a set of output terminals
- The switches are set from the outside
• Example: two light switches for the same light bulb, so that the light is on if at least one of the switches is on
Basic Problem Studied in the Thesis
5Discrete Structures - 2015
• Light bulb should be on iff exactly one of the switches is on.
Another Example
6Discrete Structures - 2015
Evil twin of the original. Does exactly the opposite of what its counterpart switch does.
Basic Problem Studied in the Thesis
• Such switch networks are not restricted to “light bulbs”
- Telephone switches
- Industrial control circuits
- Or… on any processor in any of your devices
• Shannon asked himself:
- How to “analyze” such circuits efficiently➡ Given a circuit, how do we make sure that it “computes”
what it is supposed to
- How to “synthesize” such circuits efficiently➡ Given a computation problem, how do we come up with
such circuits
• … And provided solutions in his thesis.
7Discrete Structures - 2015
• Such circuits are essentially the same as compound propositions
• Any circuit can be composed of the AND, the OR, and the NEGATION circuits
• Representations are not unique, and can be optimized to include a small number of such elementary “gates”
Basic Insights
8Discrete Structures - 2015
X
Y
X _ Y
X Y
X ^ Y
X
¬X
Switch is ON = True = 1Switch is OFF = False = 0
• Any truth table can be implemented with - NOT:
- AND:
- OR:
• Hand-waving proof is on the right
• Example suffices for now - For every row with value True,
create formula that is True in that row, and False everywhere else
- Take the “OR” of the formulas
• Enormously important in computer architecture
Logical Completeness
9Discrete Structures - 2015
¬^_
?T T T F
T T F T
T F T T
T F F F
F T T F
F T F T
F F T T
F F F F
p q r
(p ^ q ^ ¬r)(p ^ ¬q ^ r)
(¬p ^ q ^ ¬r)(¬p ^ ¬q ^ r)
True if and only if p is True, q is True, r is False. Why?
(p ^ q ^ ¬r) _ (p ^ ¬q ^ r) _ (¬p ^ q ^ ¬r)
_(¬p ^ ¬q ^ r)
• is complete!- Suffices to write the AND and the
OR in terms of NAND and the negation
NAND and NOR
10Discrete Structures - 2015
NAND(x, y) = ¬(x ^ y)
NOR(x, y) = ¬(x _ y)
(NOR,¬)
(NAND,¬)
• is complete!- Suffices to write the AND and the
OR in terms of NAND and the negation
x _ y = ¬((¬x) ^ (¬y)) = NAND(¬x,¬y)x ^ y = ¬(¬(x ^ y)) = ¬NAND(x, y)
x ^ y = ¬((¬x) _ (¬y)) = NOR(¬x,¬y)
x _ y = ¬(¬(x _ y)) = ¬NOR(x, y)
You will learn this concept much more precisely in your “Computer Architecture” classes (CS-208, CS-307)
Logic Gates
11Discrete Structures - 2015
NOT
AND
NAND
NOR
XOROR
Addition of two one-bit integers: a + b = v + 2*c
Synthesis, According to Shannon
12Discrete Structures - 2015
0 + 0 = 0 + 2*00 + 1 = 1 + 2*01 + 0 = 1 + 2*01 + 1 = 0 + 2*1
a b v c
0 0 0 0
0 1 1 0
1 0 1 0
1 1 0 1
v = (¬a ^ b) _ (a ^ ¬b)
= NAND(NAND(a,¬b),NAND(¬a, b))
c = a ^ b
= NOR(¬a,¬b)
NAND
NAND
NOT
NOT
NAND
NOR
a
b
v
c
(a+2b)+(c+2d) = u + 2v + 4w = (a+c) + 2(b+d)
Addition of two 2-bit Integers
13Discrete Structures - 2015
a
b
uc
d
NAND
NAND
NOT
NOT
NAND
NOR
NAND
NAND
NOT
NOT
NAND
NOR
NAND
NAND
NOT
NOT
NAND
NORNAND
NAND
NOT
NOT
NAND
v
w
See how the same block is repeated?
Triode Vacuum Tubes
14Discrete Structures - 2015
https://en.wikipedia.org/wiki/Vacuum_tube
Lee De Forest, 1906
Triode Vacuum Tubes as Switches
15Discrete Structures - 2015
Cathode
Plate
Gate
Negative Voltage
Electrons repelledby gate and don’t arrive at plate.Switch is off.
Positive Voltage
Electrons attractedby gate and pass to plate. Switch is on.
Triode Vacuum Tubes as Switches
16Discrete Structures - 2015
Lowvoltage
Switch is off
Highvoltage
Switch is on
Triode Vacuum Tubes as Switches
17Discrete Structures - 2015
0
Switch is off
1
Switch is on
Implementation of an Inverter
18Discrete Structures - 2015
0
1
I
R V=I*R
1
0
I
R V=0
http://www.vintchip.com/mainframe/BENDIXG-15/BENDIXG15.html
Buffer inverter from Bendix G15 Computer
NOT
Implementation of NAND and NOR
19Discrete Structures - 2015
NAND(X,Y )
NOR(X,Y )XX
YY
There is no “clean” 0 or “clean” 1
The Reality
20Discrete Structures - 2015
0 1Forbidden regime
The circuitry has to be designed so a voltage in the forbidden regime occurs with very very very low probability (for example 10-30)
Division of labor• Circuit designers: guarantee 0 and 1• Digital designers: Design efficient digital circuits that can compute complex
functions
Distribution of voltage values
$ $ $ $ $ $
$ $ $ $ $ $ 0 1
This is one (of the many, many) reason(s) you need to learn probability theory and statistics (MATH-232)
Problems with Vacuum Tubes
• Bulky, big
• Can’t switch from a 0 to a 1 very fast
• Static current- For example, inverter needs to consume current to keep itself at “0”
- Power consumption can be high
• In general, high power consumption because of size
• Complex circuitry is very difficult to build with them
21Discrete Structures - 2015
• Founded in 1925
- As a subsidiary of AT&T and Western Electric
• The name comes from Alexander Graham Bell, the inventor of the telephone
- Though the correct assignment of inventors has a long and complicated history….
• Bell Laboratories was mainly concerned with research on all aspects of long distance telephony
Bell Laboratories
22Discrete Structures - 2015
Aerial view of Bell Labs in Murray Hillhttp://users.ece.gatech.edu/~juang/B%20JUANG%20Georgia%20Tech%20Pictures.html
Partial List of Bell Labs Innovations
• First Fax Machine (1920’s)
• Long distance television transmission (1920’s)
• Radio astronomy (1930’s)
• Relay computer (1930’s)
• Transistor (1947)
• Error-correcting codes (1940’s)
• Information Theory (foundation of modern communication - 1948)
• Signal processing fundamentas (Nyquist)
• Calculators (1940’s-50’s)
• Direct distance dialing (1950’s)
• First modem (1960)
• Communication satellite (1962)
• Laser (1964)
• Cosmic background radiation (1965)
• Computer animation (1960’s)
• UNIX (1969)
• C programming language (1972)
• Fiber optics systems
• HDTV
• TDMA/CDMA (for wireless multiple access systems)
• Wireless telephony (fundamentals by Bell Labs, first one by Motorola in 1973)
• C++ programming language (1986)
• And a LOT more…..
23Discrete Structures - 2015
Transistor
• Bell Labs was interested in a telephone switching system that could handle calls made within the entire United States, and later, around the world- Switching systems based on vacuum tubes were bulky, and
consumed too much power
- Bell Labs wanted a solid-state equivalent that was smaller, faster, and more efficient
• They set out to “invent” the transistor- Remarkable achievement, since they didn’t know whether such a
thing existed at all
- Several years of work
- Final discovery was somewhat “accidental”
24Discrete Structures - 2015
Invention of the Transistor (End of 1947)
25Discrete Structures - 2015
Bardeen, Shockley, and Brettain, 1948Received the Nobel Prize in Physics in 1956
First Transistor
Bipolar Junction Transistor (npn)
26Discrete Structures - 2015
NMOS Transistors
27Discrete Structures - 2015
Low voltage
High voltage
You will learn this concept much more precisely in your “Electronics” classes (EE-202)
Implementation of the Negation Gate
28Discrete Structures - 2015
01 0 1
Current “bleeds” to ground.Static power dissipation.
Transistors allow for much better implementation
PMOS and NMOS Transistors
29Discrete Structures - 2015
High voltage
Low voltage
Low voltage
High voltage
NMOS transistors PMOS transistors
You will learn this concept much more precisely in your “Electronics” classes (EE-202)
Doesn’t have analog device with vacuum tubes.
Implementation of the Negation Gate
30Discrete Structures - 2015
1 0•
•
Closed
Open
1
•
•Open
Closed
0
Doesn’t bleed to ground!
Implementation of the Negation Gate
31Discrete Structures - 2015
NOTX ¬X
High voltage
Low voltage
Closed
Open
High voltage
Low voltage
Open
Closed
X ¬X1 00 1
Implementation of the NOR Gate
32Discrete Structures - 2015
Low
LowHigh
LowLow
HighHigh
HighLow
Low
Low
High
NORX
Y
Closed Open
X Y ¬(X ^ Y )
1 1 0
1 0 0
0 1 0
0 0 1
¬(X _ Y )
Implementation of the NAND and the AND Gates
33Discrete Structures - 2015
HHLLHLHL
LHHH
NAND
X
Y
¬(X ^ Y )
HHLLHLHL
HLLL
X
Y
AND (X ^ Y )
Note how an inverter is addedin order to produce AND from NAND
You will learn more about digital systems design in your course CS-173
• Discrete transistors were difficult to connect
- Too many wires
•The next big idea was the use of “integrated circuits
- Put all the electronic circuits onto one small plate of semiconductor material (typically silicon)
- The entire circuit can be made much smaller
• Today manufacturing uses the process of optical lithography
Integrated Circuits
34Discrete Structures - 2015
http://www.just2good.co.uk/cpuSilicon.php
ARMv6 processor
• Billions of transistors on a single chip
• Reduction of size from generation to generation- Less power consumption
- More transistors ➡ Higher functionality of the
end device
- Lower cost
• We are using transistors with a pitch of 28nm in our devices today, moving towards 10nm and below in the future
The Result
35Discrete Structures - 2015
Processor Transistor count Manufacturer Transistor pitch
ARM-1 25,000 ARM Holdings 3000 nm
Intel i-960 250,000 Intel 600 nm
Pentium Pro 5,500,000 Intel 500 nm
Pentium III 45,000,000 Intel 130 nm
Pentium 4 184,000,000 Intel 65 nm
Apple A7 1,000,000,000 Apple 28 nm
Apple A8 2,000,000,000 Apple 20 nm
Xeon Haswell 5,560,000,000 Intel 22 nm
Fiji (GPU) 8,900,000,000 AMD 28 nm
iPhone 6
iPhone 5S
PC’s
Gaming
Computers
• Today’s computing era is (at a technical level) dominated by - Shannon’s thesis (digital design)
- The transistor (Bell Labs)
- Integrated circuits (Texas Instruments, Intel)
- Manufacturing processes (Intel, IBM, TSMC, etc.)
• But this is only one side of the story…
- What can (and can’t) be done with computers?
36Discrete Structures - 2015
Alan Turing
37Discrete Structures - 2015
Alan Turing 1912 - 1954
What is an Algorithm?
• A recipe?
- Algorithm for cooking Spaghetti Bolognese, for example
- Not satisfactory, though➡ Not completely reproducible…
• According to Wikipedia “a self-contained step-by-step procedure to be performed”
- Still not satisfactory
- Not precise enough to let us argue about it
• Another definition (Wikipedia) “An effective method that can be expressed within a finite amount of space and time and in a well-defined formal language for calculating a function”
- Oh well…. 38Discrete Structures - 2015
Check out your “Algorithms” class (CS-250) next year!
Algorithm Concept
• Concept has existed for a long time
- The name comes from the Persian mathematician “Al-Khawrizmi” from his book “On the calculation with Hindu Numerals” (825 AD)
• However, formalization happened with an attempt to solve Hilbert’s “Entscheidungsprobleme” (decision problems) he posed in 1928
- Difficult to even say what these are at this stage of your studies
• A precise formulation and investigation of the problem requires the definition of “computability” and that of algorithms.
• This was done independently by Alonzo Church and Alan Turing.
- One invented the notion of the lambda-calculus
- One invented what is today called a “Turing Machine”39Discrete Structures - 2015
The “Paper”
40Discrete Structures - 2015
Proceedings of the London Mathematical Society (Ser. 2, vol. 42, 1937)
• Consist of
- an infinite tape (memory) from which they can read and on which they can write
- A “control” which upon reading the content of a cell performs some operation, and possibly writes another content on the same cell, and moves its head to the left or to the write
• The “program” is essentially the control, i.e., the behavior of the machine upon reading the contents of cells
• An “algorithm” is generally accepted to be the same as a Turing machine
• How “universal” are algorithms?
Turing Machines
41Discrete Structures - 2015
http://physics.kenyon.edu/coolphys/thrmcmp/newcomp.htm
You will learn this concept much more precisely in your “Theory of Computation” class (CS-251)
You need set theory and the concept of functions/relations for a precise definition.
Halting Turing Machines
• A Turing Machine can “halt” on some input (=content of tape at start of computation)
• Or it can go into an “infinite loop” (does not halt)- For example, the Turing Machine implementing the code on the right
does not halt➡ Caveat: this is NOT a real piece of computer code
• Halting problem: Does there exist a Turing Machine (call it a “Universal Turing Machine”) which accepts as inputs ➡ A description of a Turing Machine M➡ An input x to the Turing Machine M
- And decides whether M halts on input x?
42Discrete Structures - 2015
i=0while (i≥0)
i = i+1end while
Trivial Algorithm (does not work)
• The answer should be easy, no?- Run M on input x and wait until it halts.
• But…..- What if M never halts on x?
- How can we distinguish between ➡ M never halting on x, and ➡ M taking a very very very long time to finish its computation on x?
• In fact, we cannot!
• Turing proved that there is NO Universal Turing Machine- Proof uses the concept of countability and Cantor’s diagonal principle
• It gives a negative answer to the “Entscheidungsproblem” posed by Hilbert
43Discrete Structures - 2015
Turing Machines
• Turing machines allow us to also argue about the “efficiency” of algorithms that always halt- How much time do they need on their inputs before halting?
- How much space to they need to finish their computation?
• This allows to sub-divide computational problems into “efficient” and “inefficient” ones- PTIME (or simply P) for those that use “little” time to compute
- PSPACE for those that use “little” space to compute
- And a LOT more
• This leads to one of the biggest achievements of Theoretical Computer Science in the 20th century (NP-complete problems) and to one of the most notorious open problems- Is P = NP? (1 million dollar prize problem)
44Discrete Structures - 2015
You will learn this concept much more precisely in your “Theory of Computation” class (CS-251)
Turing Machines in Nature?
• We generally accept Turing Machines as the equivalent of algorithms or computing devices
• But it is not clear that computing going on in nature is covered by Turing Machines- Is our brain a Turing Machine?
- How about the Nature in general➡ E.g., quantum effects?
• If they are not, then there is a vast source of computational power that we can tap into- And solve problems that are generally considered very hard using
traditional Turing Machines
• This is, however, purely speculative.
45Discrete Structures - 2015
What Happened to Shannon Afterwards?
• Shannon finished his PhD at MIT in 1940 on “An Algebra for Theoretical Genetics”
• Joined Bell Labs
- During the heydays of research on transistors, though he was not affiliated with it
• And.. Founded “Information Theory” in 1948
- The theory that underlies cellular communication, image recognition/compression, hard disks, flash drives, and practically every device in our digital life➡ Arguably his biggest achievement
• After Bell Labs, he joined the Faculty at MIT in 1956
• He died in 2001 (Alzheimer’s disease) at the age of 84
46Discrete Structures - 2015
You will learn more about Information Theory in “Information Sciences” COM-101, and COM-302,
What Happened to Turing?
• He went to Princeton University in 1936 to obtain his PhD - His paper on Turing Machines was written there, while he was a
graduate student!
• He obtained his PhD in 1938 under Alonzo Church
• He joined GC&CS (predecessor of GCHQ) in 1938, a British code-breaking organization, and developed the “Bombe” an electromechanical machine to break Germany’s Enigma- Much of his work was classified up until 2012
- Visited Bell Labs in 1942 as part of a trip to the US to work with US Navy Cryptanalysts (secure voice communication)
• In 1948 he joined the Faculty of the University of Manchester
• He was convicted of indecent behavior in 1952 because of his sexual orientation and was given the choice of a lengthy prison sentence or a hormonal therapy. - He chose the latter.
• He committed suicide in 1954 at the age of 42.
47Discrete Structures - 2015
Top Related