IST 4 Information and Logicparadise.caltech.edu/ist4/lectures/lect 1616 p.pdf · Algorizm?...

IST 4 Information and Logic

mon tue wed thr fri 28 M1 1 4 M1

11 1 2 M2

18 2

25 M2

2 3 9 3 4 16 4 5 23

30 5

x= hw#x out

x= hw#x due

Mx= MQx out

Mx= MQx due

midterms

oh oh

oh oh

oh oh

oh

oh

oh oh

oh

oh

oh

oh

oh

oh

oh

oh = office hours oh

T

= today T oh

Last Lecture - Stochastic chemical networks

Stochastic logic design The B- algorithm Duality

-  Molecular switches DNA strand displacement

- Stochastic flow networks Feedback helps!

21

21

21

21

32

31

A relay circuit is a physical system for syntax manipulation

Relay circuits are not the only option!

OR gate AND gate

Circuits with Gates AON: AND, OR, Not

Efficiency and complexity

Feasibility

If feasible, how many blocks are needed? Algorizm?

Questions about building blocks?

Given a set of building blocks: What can/cannot be constructed?

AND, OR and NOT (AON)

a b

a b

a

b

What is the function computed by this circuit?

longest path from input to output – counting the number of gates

total number of gates in the circuit

2

3

Every 0-1 Boolean Function Can be Implemented Using A Depth Two AON Circuit

Implement the DNF representation: OR of many ANDs

abc XOR(a,b,c) 000 001 010 011

0 1 1 0

100 101 110 111

1 0 0 1

XOR of 3 Variables

> >

>

> > a

b c

a b c

a b c

a b c

Depth = 2

Size = 5

is the complement

XOR of 3 Variables

How many gates in a depth-2 circuit for XOR of n variables with AON?

XOR of More Variables?

Surprisingly, this is the optimal size for depth-2

Depth-2 AON Circuit for XOR

Theorem: An optimal size depth-2 AON circuit for has gates

Proof: The construction follows from the DNF representation: normal terms + one OR gate

The lower bound: WLOG >>

>

>>

abc

abc

abc

abc

>>>>

>>

>>>>

abc

abc

abc

abc

(i)  Every AND gate must have all n inputs (ii) Every AND gate computes a normal term DNF is a representation, hence, there are AND gates

???

Without Loss Of Generality??


Proof (cont): Need to prove: (i) Every AND gate must have all n inputs

By contradiction: Assume that there is a gate G with n-1 inputs . Say x1 is missing from G

Assume that:

Hence, the output of the circuit is 1 OR gate has input of 1

Making G=1 ?

>>

>

>>

abc

abc

abc

abc

>>>>

>>

>>>>

abc

abc

abc

abc

set a variable to 1 and a complement to 0


> a b c

0 1 0

1

Note that the following two assignments force the output of the circuit to be 1:


Proof (cont): Assume that:

Hence, the output of the circuit is 1 (OR gate has input of 1)

Contradiction!!

Q Those assignments have different parities

So what?


How many gates in a depth 2 circuit for XOR of n variables with AON?

It is optimal size for depth-2 n=4, depth 2, size 9

Q: for n=4, arbitrary depth, suggest a circuit for XOR with size less than 9?

Size 8 AON Circuit for XOR of Four Variables

XOR(x,y,z) b c

a XOR(x,y)

d

size 5 size 3

XOR(a,b,c,d)

Idea: Compute a large XOR by using a circuit of small XOR gates

Arbitrary depth circuit for XOR of n variables with AON?

AON Circuit for XOR


XOR

8 variables

Tree

edge = wire

in-degree = 2 leaf = input edge

node = XOR gate


XOR

8 variables Circuit size in AON gates?

Size = Node size X number of nodes

3 X 7 = 21

Q: Can we do better for 8 variables?

Note that we need size 129 in depth-2…

Idea: Use a larger in-degree?

9 variables


5 X 4 = 20

XOR Note that we need size 21 with in-degree 2

Size 18 for 8 variables

Q: Can we do better for 8 variables?

Idea: Use a larger in-degree?

9 variables


5 X 4 = 20

XOR Note that we need size 21 with in-degree 2

Size 18 for 8 variables

In general, we can prove that degree-3 XOR trees are the best! Size is

AON Constructions for XOR

circuit kind size

AON, d-2

AON

optimal

maybe optimal

n=4 9 8

lower bound:

AON Circuit for XOR

We have a construction of size we know how to prove a lower bound of 2n-1

2 3 4 5 6 7 8

3 5 7 9

11 13 15

3 5 8 10 13 15 18

Matt Cook proved that an AON circuit of size 7 for XOR does not exist he used a computer search

AON Circuit for XOR

We have a construction of size we know how to prove a lower bound of 2n-1

2 3 4 5 6 7 8

3 5 8 10 12 14 16

3 5 8 10 13 15 18

Matt Cook proved that an AON circuit of size 7 for XOR does not exist he used a computer search

Matching upper/lower bounds = MSc in CS

next gap

The problem:

Most functions require a large circuit size - in the number of inputs

4x(3-1)=8

Size: total number of gates in the circuit

Show a function that requires

circuit size!

The circuit complexity problem:

While most functions require a large circuit size - in the number of inputs

Currently we can only prove lower bounds...

Circuits with Gates

LT: Linear Threshold

Neuron – Neural Gate LT: Linear Threshold

-2 1

1 0 0 0 1

1 0 1 1

-2 -1 -1 0

0 0 0 1

LT: Linear Threshold What is the function computed by this gate?

Neural Circuits feasibility

2 input Linear Threshold (LT) gate

Q: Are LT gates magical?


Q: Are LT gates magical?


Idea: A Linear Threshold is Magical

Can compute AND, OR and NOT

We showed that we can compute the AND function with an LT gate

-2 1

1 0 0 0 1

1 0 1 1

-2 -1 -1 0

0 0 0 1

Can We Compute an OR Function with an LT Gate?

-1 1

1 0 0 0 1

1 0 1 1

-1 0 0 1

0 1 1 1

Can We Compute a NOT with an LT Gate?

1 -2

Can we compute NOT without sgn?

More Variables for AND?

Hence is an AND

More Variables for OR?

Hence is an OR

Circuits Efficiency and complexity

The Functions of the Adder

carry

2 symbol adder c

s

d1 d2

c

sum

XOR with a Single LT Gate

Is it possible to compute with a single LT gate?

Idea: Find weights w0, w1 and w2 such that:

2 symbol adder c

s

d1 d2

c


Answer : NO Proof: By contradiction

assume it is possible and reach a contradiction

Q

2 symbol adder c

s

d1 d2

c XOR with a Single LT Gate

XOR with More Variables?


Idea: suppose that it is possible, and reach a contradiction

However,

And,

Contradiction

2 symbol adder c

s

d1 d2

c

Need LT circuits for XOR!

MAJ with a Single LT Gate


|X| MAJ

0 0 1 0

2 1

3 1

2 symbol adder c

s

d1 d2

c

AND, OR, XOR and MAJ are symmetric functions

|X| AND OR XOR MAJ

0 0 0 0 0

1 0 1 1 0

2 0 1 0 1

3 1 1 1 1

LT1 = the class of Boolean functions that can be realized by a single LT gate.

LT1 LT1 LT1 not LT1

Q: Which symmetric functions are in LT1?

|X| AND OR XOR MAJ

0 0 0 0 0

1 0 1 1 0

2 0 1 0 1

3 1 1 1 1

Definition: A symmetric Boolean function is in TH if it has at most a single transition in the symmetric function table

= a transition

Not in TH In TH

The Class TH

The Class TH - Single Transition

|X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3

0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1

2 1 1 1 0 0 0 0 1

3 1 1 1 1 0 0 0 0

Q: what is |TH| ? the number TH functions...

A: 2n+2

= a transition

Claim:

Q

0

1

Proof:

|X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3

0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1

2 1 1 1 0 0 0 0 1

3 1 1 1 1 0 0 0 0

The Class TH is in LT1

AON and Linear Threshold Circuits

XOR example

Need LT circuits for XOR!

XOR of Three Variables

> >

>

> > a

b c

a b c

a b c

a b c

Depth = 2

Size = 5

Size 5 is optimal for AON depth 2

is the complement

Size 4 LT depth 2

LT gates are MORE Powerful

1

1

1

-1

-1

-1

1

1

1

1

-1

-3

1

1

1

-2

FOR XOR: Size 5 is optimal for AON depth 2

1

1

1

-1

-1

-1

1

1

1

1

-1

-3

1

1

1

-2

A

B

C

0 1

2 3

0 1 1 1

0 0 0 1

A B C A+B+C -2+A+B+C

1 1 0 0

Can take the sgn or add 1

1 2 1 2

-1 0

-1 0

LT gates are MORE Powerful

LT-l = LT layered inputs go to first layer only

TH functions

XOR Function: Size of LT vs AON in Depth 2

5 4

AON LT-l

*

*

* = it is optimal Exponential gap in size

5 4

AON LT-l

General construction for symmetric functions

Linear Threshold Circuits

symmetric functions

LT Depth-2 Circuits

+ -1

TH1

TH2 |X| TH1 TH2 TH1+TH2-1

0 0 1 0 1 1 1 1 2 1 0 0

???

|X| f(x) 0 0 1 1

2 1 3 0 4 0

Generalization

|X| f(x) TH1

0 0 0 1 1 1

2 1 1

3 0 1

4 0 1

Generalization

|X| f(x) TH1 TH3

0 0 0 1 1 1 1 1

2 1 1 1

3 0 1 0

4 0 1 0

Generalization

|X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1

2 1 1 1 1

3 0 1 0 0

4 0 1 0 0

Generalization

|X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1

2 1 1 1 1

3 0 1 0 0

4 0 1 0 0

+ -1

Generalization to SYM

Q: What is the generalization to arbitrary symmetric functions?

Generalization to SYM

Q: What is the generalization to arbitrary symmetric functions?

A: Consider the symmetric function table, it is a sum of non-overlapping 1-intervals

0

0

1

1 Sum of two TH functions

Back to XOR

0

1

2

3

4

5

0

1

0

1

0

1

n TH gates for XOR of n variables

LT-l Circuit Design Algorithm for SYM

0

1

2

3

4

5

1

1

0

1

1

0

f(X)

6 7

1 1

Subtract 1 for every isolated 1-block

The Layered Construction for SYM Some History

Saburo Muroga 1925- 2009

1959

Was born in Japan PhD in 1958 from Tokyo U, Japan 1960-1964: Researcher at IBM Research, NY 1964-2002: professor at the University of Illinois, Urbana-Champaign

Majority Decision

(6,2,0,2)

(4,2,2,3)

LT1 = Can be computed by a single LT gate

IST 4 Information and Logicparadise.caltech.edu/ist4/lectures/lect 1616 p.pdf · Algorizm?...

Documents

Transcript of IST 4 Information and Logicparadise.caltech.edu/ist4/lectures/lect 1616 p.pdf · Algorizm?...