Post on 19-Jun-2020
IST 4 Information and Logic
mon tue wed thr fri 28 M1 1 4 M1
11 1 2 M2
18 2
25 M2
2 3 9 3 4 16 4 5 23
30 5
x= hw#x out
x= hw#x due
Mx= MQx out
Mx= MQx due
midterms
oh oh
oh oh
oh oh
oh
oh
oh oh
oh
oh
oh
oh
oh
oh
oh
oh = office hours oh
T
= today T oh
Last Lecture - Stochastic chemical networks
Stochastic logic design The B- algorithm Duality
- Molecular switches DNA strand displacement
- Stochastic flow networks Feedback helps!
21
21
21
21
32
31
A relay circuit is a physical system for syntax manipulation
Relay circuits are not the only option!
OR gate AND gate
Circuits with Gates AON: AND, OR, Not
Efficiency and complexity
Feasibility
If feasible, how many blocks are needed? Algorizm?
Questions about building blocks?
Given a set of building blocks: What can/cannot be constructed?
AND, OR and NOT (AON)
a b
a b
a
b
What is the function computed by this circuit?
longest path from input to output – counting the number of gates
total number of gates in the circuit
2
3
Every 0-1 Boolean Function Can be Implemented Using A Depth Two AON Circuit
Implement the DNF representation: OR of many ANDs
abc XOR(a,b,c) 000 001 010 011
0 1 1 0
100 101 110 111
1 0 0 1
XOR of 3 Variables
> >
>
> > a
b c
a b c
a b c
a b c
Depth = 2
Size = 5
is the complement
XOR of 3 Variables
How many gates in a depth-2 circuit for XOR of n variables with AON?
XOR of More Variables?
Surprisingly, this is the optimal size for depth-2
Depth-2 AON Circuit for XOR
Theorem: An optimal size depth-2 AON circuit for has gates
Proof: The construction follows from the DNF representation: normal terms + one OR gate
The lower bound: WLOG >>
>
>>
abc
abc
abc
abc
>>>>
>>
>>>>
abc
abc
abc
abc
(i) Every AND gate must have all n inputs (ii) Every AND gate computes a normal term DNF is a representation, hence, there are AND gates
???
Without Loss Of Generality??
Depth-2 AON Circuit for XOR
Proof (cont): Need to prove: (i) Every AND gate must have all n inputs
By contradiction: Assume that there is a gate G with n-1 inputs . Say x1 is missing from G
Assume that:
Hence, the output of the circuit is 1 OR gate has input of 1
Making G=1 ?
>>
>
>>
abc
abc
abc
abc
>>>>
>>
>>>>
abc
abc
abc
abc
set a variable to 1 and a complement to 0
Theorem: An optimal size depth-2 AON circuit for has gates
> a b c
0 1 0
1
Note that the following two assignments force the output of the circuit to be 1:
Depth-2 AON Circuit for XOR
Proof (cont): Assume that:
Hence, the output of the circuit is 1 (OR gate has input of 1)
Contradiction!!
Q Those assignments have different parities
So what?
Theorem: An optimal size depth-2 AON circuit for has gates
How many gates in a depth 2 circuit for XOR of n variables with AON?
It is optimal size for depth-2 n=4, depth 2, size 9
Q: for n=4, arbitrary depth, suggest a circuit for XOR with size less than 9?
Size 8 AON Circuit for XOR of Four Variables
XOR(x,y,z) b c
a XOR(x,y)
d
size 5 size 3
XOR(a,b,c,d)
Idea: Compute a large XOR by using a circuit of small XOR gates
Arbitrary depth circuit for XOR of n variables with AON?
AON Circuit for XOR
Idea: Compute a large XOR by using a circuit of small XOR gates
XOR
8 variables
Tree
edge = wire
in-degree = 2 leaf = input edge
node = XOR gate
Idea: Compute a large XOR by using a circuit of small XOR gates
XOR
8 variables Circuit size in AON gates?
Size = Node size X number of nodes
3 X 7 = 21
Q: Can we do better for 8 variables?
Note that we need size 129 in depth-2…
Idea: Use a larger in-degree?
9 variables
Size = Node size X number of nodes
5 X 4 = 20
XOR Note that we need size 21 with in-degree 2
Size 18 for 8 variables
Q: Can we do better for 8 variables?
Idea: Use a larger in-degree?
9 variables
Size = Node size X number of nodes
5 X 4 = 20
XOR Note that we need size 21 with in-degree 2
Size 18 for 8 variables
In general, we can prove that degree-3 XOR trees are the best! Size is
AON Constructions for XOR
circuit kind size
AON, d-2
AON
optimal
maybe optimal
n=4 9 8
lower bound:
AON Circuit for XOR
We have a construction of size we know how to prove a lower bound of 2n-1
2 3 4 5 6 7 8
3 5 7 9
11 13 15
3 5 8 10 13 15 18
Matt Cook proved that an AON circuit of size 7 for XOR does not exist he used a computer search
AON Circuit for XOR
We have a construction of size we know how to prove a lower bound of 2n-1
2 3 4 5 6 7 8
3 5 8 10 12 14 16
3 5 8 10 13 15 18
Matt Cook proved that an AON circuit of size 7 for XOR does not exist he used a computer search
Matching upper/lower bounds = MSc in CS
next gap
The problem:
Most functions require a large circuit size - in the number of inputs
4x(3-1)=8
Size: total number of gates in the circuit
Show a function that requires
circuit size!
The circuit complexity problem:
While most functions require a large circuit size - in the number of inputs
Currently we can only prove lower bounds...
Circuits with Gates
LT: Linear Threshold
Neuron – Neural Gate LT: Linear Threshold
-2 1
1 0 0 0 1
1 0 1 1
-2 -1 -1 0
0 0 0 1
LT: Linear Threshold What is the function computed by this gate?
Neural Circuits feasibility
2 input Linear Threshold (LT) gate
Q: Are LT gates magical?
LT: Linear Threshold
Q: Are LT gates magical?
LT: Linear Threshold
Idea: A Linear Threshold is Magical
Can compute AND, OR and NOT
We showed that we can compute the AND function with an LT gate
-2 1
1 0 0 0 1
1 0 1 1
-2 -1 -1 0
0 0 0 1
Can We Compute an OR Function with an LT Gate?
-1 1
1 0 0 0 1
1 0 1 1
-1 0 0 1
0 1 1 1
Can We Compute a NOT with an LT Gate?
1 -2
Can we compute NOT without sgn?
More Variables for AND?
Hence is an AND
More Variables for OR?
Hence is an OR
Circuits Efficiency and complexity
The Functions of the Adder
carry
2 symbol adder c
s
d1 d2
c
sum
XOR with a Single LT Gate
Is it possible to compute with a single LT gate?
Idea: Find weights w0, w1 and w2 such that:
2 symbol adder c
s
d1 d2
c
Is it possible to compute with a single LT gate?
Answer : NO Proof: By contradiction
assume it is possible and reach a contradiction
Q
2 symbol adder c
s
d1 d2
c XOR with a Single LT Gate
XOR with More Variables?
Is it possible to compute with a single LT gate?
Idea: suppose that it is possible, and reach a contradiction
However,
And,
Contradiction
2 symbol adder c
s
d1 d2
c
Need LT circuits for XOR!
MAJ with a Single LT Gate
Is it possible to compute with a single LT gate?
|X| MAJ
0 0 1 0
2 1
3 1
2 symbol adder c
s
d1 d2
c
AND, OR, XOR and MAJ are symmetric functions
|X| AND OR XOR MAJ
0 0 0 0 0
1 0 1 1 0
2 0 1 0 1
3 1 1 1 1
LT1 = the class of Boolean functions that can be realized by a single LT gate.
LT1 LT1 LT1 not LT1
Q: Which symmetric functions are in LT1?
|X| AND OR XOR MAJ
0 0 0 0 0
1 0 1 1 0
2 0 1 0 1
3 1 1 1 1
Definition: A symmetric Boolean function is in TH if it has at most a single transition in the symmetric function table
= a transition
Not in TH In TH
The Class TH
The Class TH - Single Transition
|X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3
0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1
2 1 1 1 0 0 0 0 1
3 1 1 1 1 0 0 0 0
Q: what is |TH| ? the number TH functions...
A: 2n+2
= a transition
Claim:
Q
0
1
Proof:
|X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3
0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1
2 1 1 1 0 0 0 0 1
3 1 1 1 1 0 0 0 0
The Class TH is in LT1
AON and Linear Threshold Circuits
XOR example
Need LT circuits for XOR!
XOR of Three Variables
> >
>
> > a
b c
a b c
a b c
a b c
Depth = 2
Size = 5
Size 5 is optimal for AON depth 2
is the complement
Size 4 LT depth 2
LT gates are MORE Powerful
1
1
1
-1
-1
-1
1
1
1
1
-1
-3
1
1
1
-2
FOR XOR: Size 5 is optimal for AON depth 2
1
1
1
-1
-1
-1
1
1
1
1
-1
-3
1
1
1
-2
A
B
C
0 1
2 3
0 1 1 1
0 0 0 1
A B C A+B+C -2+A+B+C
1 1 0 0
Can take the sgn or add 1
1 2 1 2
-1 0
-1 0
LT gates are MORE Powerful
LT-l = LT layered inputs go to first layer only
TH functions
XOR Function: Size of LT vs AON in Depth 2
5 4
AON LT-l
*
*
* = it is optimal Exponential gap in size
5 4
AON LT-l
General construction for symmetric functions
Linear Threshold Circuits
symmetric functions
LT Depth-2 Circuits
+ -1
TH1
TH2 |X| TH1 TH2 TH1+TH2-1
0 0 1 0 1 1 1 1 2 1 0 0
???
|X| f(x) 0 0 1 1
2 1 3 0 4 0
Generalization
|X| f(x) 0 0 1 1
2 1 3 0 4 0
Generalization
|X| f(x) TH1
0 0 0 1 1 1
2 1 1
3 0 1
4 0 1
Generalization
|X| f(x) TH1 TH3
0 0 0 1 1 1 1 1
2 1 1 1
3 0 1 0
4 0 1 0
Generalization
|X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1
2 1 1 1 1
3 0 1 0 0
4 0 1 0 0
Generalization
|X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1
2 1 1 1 1
3 0 1 0 0
4 0 1 0 0
+ -1
Generalization to SYM
Q: What is the generalization to arbitrary symmetric functions?
Generalization to SYM
Q: What is the generalization to arbitrary symmetric functions?
A: Consider the symmetric function table, it is a sum of non-overlapping 1-intervals
0
0
1
1 Sum of two TH functions
Back to XOR
0
1
2
3
4
5
0
1
0
1
0
1
n TH gates for XOR of n variables
LT-l Circuit Design Algorithm for SYM
0
1
2
3
4
5
1
1
0
1
1
0
f(X)
6 7
1 1
Subtract 1 for every isolated 1-block
The Layered Construction for SYM Some History
Saburo Muroga 1925- 2009
1959
Was born in Japan PhD in 1958 from Tokyo U, Japan 1960-1964: Researcher at IBM Research, NY 1964-2002: professor at the University of Illinois, Urbana-Champaign
Majority Decision
(6,2,0,2)
(4,2,2,3)
LT1 = Can be computed by a single LT gate