Juris Hartmanis, Richard Stearns, and Computational Complexity.

March 24, 2003Josef Pohl

Outline of topics:

1. Biograhpical Sketches of Hartmanis and Stearns2. Collaborative and Turing Ventures.3. What is Computational Complexity?

Examples and starting points.4. What did Hartmanis and Stearns Contribute?

Ideas, Theorems, and Extensions.5. Where did things go from there?

Making Complexity more complex6. Open problems and the future of Computational

Complexity. (aka. How to become rich and famous.)

Juris Hartmanis

- Born in Latvia

- Moved to “displaced persons” in Germany during WWII.

- Phillips Univ. in Marburg (Studied physics)

- Relocated to Kansas City and attended the University of Kansas City where he received a Masters in Mathematics.

- PhD in pure mathematics (Lattice Theory) at Cal Tech 1955.

- Faculty at Cornell

- General Electric Research, Schenectady NY (summer job)

Richard Stearns

- Undergraduate work at Carlton College (1958)

- Graduate degree from Princeton in mathematics

“Three-Person Cooperative Games without Side Payments”

Game Theory

Completed in 1961

- Prior to completion of his degree, Stearns took a summer job at GE.

- Became a permanent employee in 1961.

So began their work together...

- Hartmanis and Stearns met in the Information Studies Section (a subgroup of the Electron Physics group)

- Began work on decomposition of sequential machines.

- Published several papers covering computation on sequential machines and a book:

“Algebraic Structure Theory of Sequential Machines.” - published in 1966

- The text describes how sequential machines can be described in terms

of smaller machines, how these machines must be interconnected, and

how “information” flows through them.

Turing award work

- Hartmanis and Stearns won their Turing award for their foundation

work in Computational Complexity.

-Citation:

“In recognition of their seminal paper which

established the foundations for the field of

computaitional complexity theory”

- Their paper laid the foundation for much of the later work in

Complexity Theory and theoretical computer science as a whole.

- Also coined the phrase “Computational Complexity”.

So what is Computational Complexity?

Of the fundamental questions that Computer Scientists ask, computational complexity answers the question: How difficult is to compute something that we know to be computable?

e.g. What resources (time, space, machines, ...) will it take to get a result?

– Based on input size and the basic operations in a given algorithm.

–

Examples you probably have seen:

MergeSort O(n*log(n))

Fast Fourier Transform: O(n*log(n))

Multiplication: O( n^2 ) or O( n^1.59 ) or O(n*log(n)*loglog(n))

Matrix Multiplication: O( (n^2)*(2n-1) ) or O(4.7*n^2.81 )

Prime Number determination: O(log^12 (n))

(mostly from Cook 1983)

What was the state of the science?

● Turing:

Foundations on algorithmically computable functions.● Rabin:

Investigated the question of what makes one function more difficult to compute than another.

● Yamada:

Looked into the question of real-time computation on multi-tape turing machines.

On the computational complexity of algorithms.

- Published in 1965 (Transactions of the American Mathematical Society).

- The paper developed from the search for a “robust computing model and an intuitively satisfying classification of the complexity of problems” (Hartmanis 1994)

- Introduced the concepts of

– Classification (classes)– “speed-up”– Hierarchy theorems

What exactly is a complexity class?

If T(n) is some function of the size of the input (length), then a class, Time[T(n)] is defined to be all the problems that can be solved in T(n) steps.

For example.

T(n) = n^2

Then the class Time[ n^2 ] are all the problems that

can be solved in n^2 steps.

Speedup

The speed up theorem basically dictates that membership in a class is dependant upon the order of the function.

Time(T(n)) = Time(c * T(n))

where c > 0.

ie the constants are not important (sort of) in the determination of class membership.

Heirarchies of Classes

Several theorems which defined relationships between classes.

All based on the limit as the size of the input approached infinity.

Allowed determination of whether or not a class was a subset (proper or not) of another.

The general form is:

L= lim T(n)*log(T(n))/U(n)n->infinity

Where U(n) is a function of n.

If L = ∞ then T(n) is a superset of U(n)

If 0 < L < ∞ then T(n) is in the class of U(n)

If L = 0 then T(n) is a subset of U(n)

A few other results

The intersection of two classes is also a class.

These results are machine dependent.– the complexity of computation shown using these

theorems is dependent on the type of machine that is used. (single tape vs multi-tape turing machines)

Paper summary

“The important notion of complexity measure defined by the computation time on multitape Turing machines was introduced and hierarchy theorems were proved” (Cook 1983)

Those who followed

Hartmanis and Stearns paper provided the notions with which others could further define the concepts of classes and complexity.

Manuel Blum– Thesis developed an axiomatic approach to

computaion

Stephan Cook– Defined NP-completeness

Hartmanis and Stearns

Manuel Blum

Probably the most used notion of complexity today.

Created a machine independent version of many of the same theorems which Hartmanis and Stearns developed

Showed that all complexity measures are recursively related.

Stephan A. Cook

Won Turing award in 1983 for his work in Computational Complexity.

Introduced the concept of NP-completeness in 1971.

Meaning he showed that there was a class of problems that do not have a deterministic polynomial time algorithm.

And if a decent algorithm existed for a given set of problems then a decent algorithm existed for the entire set.

NP complete problems (identifying and analyzing) have generally been the focus of much of the work in complexity theory since the concept was introduced mostly due to the practical nature of someof the problems.

Hartmanis and/or Stearns

Together they did more work on developing the concept of hierarchies and extended the idea to space complexity. They also revised their work slightly to include that of Blum. (machine independence)

Stearns developed the concept of a power index which is roughly defined as the greatest lower bound on a set. (work done with H.B. Hunt)

Stearns also went back to his original area of game theory.

Hartmanis continued to work in Complexity Theory, most notably trying to relate classes P to NP.

Future

Some areas where work is going on:

Computation on the set of real numbers?

Probabilistic Algorithms?

Open Problems:

P=NP ?

sqrt(2) computable in real time?

“When you can measure what you are speaking about and can express it in numbers, you know something about it. But when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind”

-Lord Kelvin

“From these and other considerations grew our deep conviction that there must be quantitative laws that govern the behavior of information and computing”

-Juris Hartmanis (1994)

References

Blum, M. “A machine-independent theory of the complexity of recursive functions. “ Journal of the ACM, Vol 14, Num 2, April 1967.

Cook, S. A. “An Overview of Computational Complexity.”. Communications of the ACM, Vol 26, Num 6, June 1983.

Hartmanis, J. , “On Computational Complexity and the Nature of Computer Science.”Communications of the ACM, Vol 37, Num 10, October 1994.

Hartmanis, J., “Computational Complexity and Mathematical Proofs.” Informatics -10 Years Back, 10 Years Ahead. Reinhard Wilhem, ed. Springer Verlag, 2001.

Hartmanis, J., and R. E. Stearns, “On the Computational Complexity of Algorithms.” Trans. Amer. Math. Soc. 117, May 1965.

Hartmanis, J., and R. E. Stearns, “Algebraic Structure Theory of Sequential Machines.” Prentice Hall, Engelwood Cliffs, NJ, 1966.

Hartmanis, J and J. E. Hopcroft, “An Overview of the Theory of Computational Complexity.” Journal of the ACM, Vol 18, Num 3, July 1971.