ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF...
Transcript of ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF...
![Page 1: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/1.jpg)
ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW
Complex Materials in Physics and BiologyVarenna, 29 June - 9 July 2010
Luciano PietroneroMatthieu Cristelli, A. Zaccaria
Institute of Complex Systems, CNR, Rome ItalyUniversity of Rome Sapienza and Centro Fermi, Rome
(WEB page: http://pil.phys.uniroma1.it)
![Page 2: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/2.jpg)
Back to Liquidity problem
Liquidity: seems more important than volume or news for price changes
Microscopic model for the order book& finite liquidity (crisis).
This should be included in a realistic ABM
![Page 3: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/3.jpg)
Order Book & ABM
Order book model
In a typical Agent-Based Model (ABM) the price evolution is a coarse-grained clearing/adjustment mechanism that does not take into account the liquidity of the market
real markets
Therefore we need to investigate the microscopic mechanisms for price formation in order to find β(g)
![Page 4: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/4.jpg)
![Page 5: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/5.jpg)
Order Book in a nutshell
The elementary mechanism of price formation is a double auction where traders submit orders to buy or sell. The complete list of all orders is called the order book.
Two classes of ordersmarket orders (impatient traders)
limit orders (patient traders)
•Market orders correspond to the intention of immediately purchase or sale at the best price (quote) available at that time
•The limit ones instead are not immediately executed since they are orders to buy or sell at a certain quote which is not necessary the best one.
![Page 6: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/6.jpg)
The prices of placement of orders, called “quotes”, are not continuous but quantized in unit of ticks whose size is an important parameter of an order book.
best bid b(t): order of buying with the highest price
best ask a(t): order of selling with the lowest price
The non-zero difference between a(t) and b(t) is defined as the spread s(t) = a(t) − b(t)
![Page 7: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/7.jpg)
price
askbid
b(t) a(t)
s(t)
Patient traders: limit ordersvo
lum
e
![Page 8: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/8.jpg)
price
askbid
b(t) a(t)
s(t)
Patient traders: limit ordersvo
lum
e
![Page 9: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/9.jpg)
price
askbid
b(t) a(t)
s(t)
Patient traders: limit ordersvo
lum
e
![Page 10: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/10.jpg)
price
askbid
b(t) a(t)
s(t)
Patient traders: limit ordersvo
lum
e
![Page 11: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/11.jpg)
price
askbid
b(t) a(t)
s(t)
Patient traders: limit ordersvo
lum
e
![Page 12: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/12.jpg)
price
askbid
b(t) a(t)
Patient traders: limit ordersvo
lum
e
![Page 13: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/13.jpg)
price
askbid
b(t) a(t)
Patient traders: limit ordersvo
lum
e
![Page 14: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/14.jpg)
price
askbid
b(t) a(t)
Impatient traders: market ordersvo
lum
e
![Page 15: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/15.jpg)
price
askbid
b(t) a(t)
Impatient traders: market ordersvo
lum
e
![Page 16: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/16.jpg)
price
askbid
b(t) a(t)
Impatient traders: market ordersvo
lum
e
![Page 17: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/17.jpg)
price
askbid
b(t) a(t)
Changing opinion: cancellationsvo
lum
e
![Page 18: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/18.jpg)
price
askbid
b(t) a(t)
Changing opinion: cancellationsvo
lum
e
![Page 19: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/19.jpg)
price
askbid
b(t) a(t)
Changing opinion: cancellationsvo
lum
e
![Page 20: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/20.jpg)
price
askbid
b(t) a(t)
Changing opinion: cancellationsvo
lum
e
![Page 21: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/21.jpg)
Order Book in a nutshell(Microscopic version of Farmer’s zero intelligence model)
p
Δp
askbid
bestbid b(t)
best ask a(t)
limit order
market order
spread s(t)
limit order bid può cadere tra [-∞,a(t)]
limit order ask può cadere tra [b(t),+∞]
V
p
Δq
askbid
bestbid b(t)
best ask a(t)
incominglimit order
incomingmarket order
spread s(t)
a buy limit order can be placed in the interval [-∞,a(t)]
a sell limit order can be placed in the interval [b(t),+∞]
ω
p(t)
![Page 22: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/22.jpg)
Order Book regimes
We can identify two different regimes in the order book dynamics
Very liquid market
• Small price variations• Behavior similar to a continuous system
Illiquid market: discreteness • Large price variations• The discreteness of the system is crucial
![Page 23: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/23.jpg)
High frequency data (from 1991 until now)
Order Book Stylized Facts
The amount of information contained by high frequency data is similar to the one from biology or from a modern particle collider such as LHC
In some sense the order book problem is scientific: some Laws exist.
We need new statistical tools to extract information in such a huge amount of data: data mining, ...
Model/theory/method/statistical regularities are now investigable in a way similar to the research in physics
Models can be tested
![Page 24: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/24.jpg)
Order book profile
Wild fluctuations
![Page 25: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/25.jpg)
Cancellation rate
Evidence of automated trades
![Page 26: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/26.jpg)
We can identify two different regimes in the order book dynamics
•Very liquid market•Small price variations•Behavior similar to a continuous system
•Illiquid market: discreteness•Large price variations•The discreteness of the system is crucial
Order Book regimes
What is liquidity ?It is the market capability for converting money into stocks or viceversa.
It defines how large the impact of an order is.
![Page 27: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/27.jpg)
Liquid market
pΔp
bid
bestbid b(t)
best ask a(t)
market order
spread s(t) Vp
askbid
b(t) a(t)
market order
p(t)
a(t+1) p(t+1) p(t+1)-p(t) = fraction of a tick
![Page 28: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/28.jpg)
Illiquid market
p
Δp
bid
bestbid b(t)
best ask a(t)
market order
spread s(t) V
p
askbid
b(t) a(t)
market order
p(t)
p(t+1) a(t+1)p(t+1)-p(t) = several ticks
![Page 29: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/29.jpg)
Price Impact Function
ω
Δp
![Page 30: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/30.jpg)
Price Impact Function
The Price Impact Function (PIF) can be considered as the response function of a stock, that is ...If an agent submit a “virtual” market order of volume ω at time t, what will be the average price change at time t+τ?
The Price Impact Function of real market is a concave function with respect to the order volume
![Page 31: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/31.jpg)
The response is concave (this is a sure feature) but the shape depends on the market, on the order size, on the liquidity, ...
Farmer, Lillo, Mantegna
Stanley, et al.
Almgren
Bouchaud, Potters, et al.
NYSE, LSE
NYSE
Citygroup
Markets are not in a linear response regime
Paris Bourse
small/medium order size
large order size
large order size
small/medium order size
Concavity is due to the shape of the profile, to the response of the investors to a liquidity demand (liquidity provider), ...
![Page 32: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/32.jpg)
![Page 33: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/33.jpg)
Price Impact Surface
We want to study the role played by liquidity/granularity in price response but the normal PIF is calculated averaging on order book configurations with different liquidity/granularity
where g is a measure of liquidity/granularity
We define the Price Impact Surface (PIS) which is instead a function of volume and liquidity/granularity
![Page 34: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/34.jpg)
Price Impact Surface - 1
is concave as one measured in real order book
Model result for τ=400 and k=4
![Page 35: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/35.jpg)
Equity Market Impact Function: Different results ???• R. Almgren et al + F. Abergel (Paris): exponent = 0.6• Lillo, Farmer, Mantegna: exp = 0.2 - 0.4• J.P. Bouchaud et al: log behavior
Present Model:0.6 seems to be naturalbut prefactor is alsoimportant
More Liquid >
![Page 36: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/36.jpg)
If we rescale the PIS with the average impact function that is proportional to we observe a quasi-collapse in a unique curve (in particular for small values of the order volume)
Therefore the PIS can be approximately factorized as:
Price Impact Surface - 2
Relation to ABM:Small N leads to moreSparse orders and moregranularity.N dependent price formation
![Page 37: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/37.jpg)
Stylized Facts of Order Book
![Page 38: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/38.jpg)
Key Concepts:
TO IDENTIFY FROM REAL DATA
• Market sentiment, stabilizing vs destabilizing• The effective independent agents N* in a market • Analysis of Herding, Contagion, Correlations• Liquidity analysis of order book• Network oriented approach - Direct interaction vs global Trust.• Coherence problem, similar behavior
BASIC STRATEGIC PROBLEM• Efficiency vs. Robustness
![Page 39: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/39.jpg)
Forecasting Financial Crisis:Measurements, Models and Predictions
(ISC-FET Open Call 2010-2014)
• ISC-CNR, Italy (G. Caldarelli, S. Leonardi and LP)• Univ. delle Marche, Italy (M. Gallegati, D. Delli Gatti)• ETH Zurich (F. Schweitzer, S. Battiston)• City Univ. London, UK (G. Iori, A. Banal-Estanol, S. Jafarely)• Univ. of Oxford, UK (F. Reed-Tsochas, R. May, E. Lopez)• Yahoo Research, Barcelona, Spain (R. Baeza-Yatez)• European Central Bank, Frankfurt, D
![Page 40: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/40.jpg)
![Page 41: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/41.jpg)
The Mystery of Zipf’s Law and Rank-Size Laws
![Page 42: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/42.jpg)
Outline
Introduction Rank-Size Law (i.e. cities) Zipf’s Law (i.e. word frequency) Details matter (works for men but not for women)
• Models for city sizes• Simon Model• Multiplicative process+ reflecting boundary• Inverse square law• Coherence and correlations in the sample
• Microscopic origin of the inverse square law• Widespread occurrence of the inverse square distribution• Aggregation processes
![Page 43: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/43.jpg)
Zipf’s Law
Plot of word frequency in Wikipedia (2006) - source: Wikipedia
![Page 44: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/44.jpg)
Zipfs Law (frequency)Linux package download
package download
page viewssite visits
![Page 45: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/45.jpg)
Zipfs Law (Rank-Size)
GDP
oil reserves
![Page 46: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/46.jpg)
Zipf’s Law ≠ Rank-Size Law
Zipf’s Law is an empirical law which states the frequencies of many types of data follow a discrete power law probability distribution
Rank-size rules describe the regularities observed in the size of social phenomena when the size is plotted versus the rank
In both cases k represents the rank but
is a probability is a size, a lenth, etc
![Page 47: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/47.jpg)
Zipf’s Law ≠ Rank-Size Law
Zipf’s Law is an empirical law which states the frequencies of many types of data follow a discrete power law probability distribution
Rank-size rules describe the regularities observed in the size of social phenomena when the size is plotted versus the rank
We focus our attention on Rank-Size Law
In both cases k represents the rank but
is a probability is a size, a lenth, etc
![Page 48: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/48.jpg)
Non-universality of 1/k behavior: details matter
If x(k)=1/k holds on a set, the rule cannot hold on a subset or in general on a larger set
The dataset details matter, 1/k is not a universal behavior
For instance on an ordered subset we obtain:
![Page 49: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/49.jpg)
Some pratical examples: USA
x(k)=1/k approximately holds for United States of America but the rule does not hold in each state
USA are the “right” set
New York Illinois Alaska
![Page 50: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/50.jpg)
Some pratical examples: EU
x(k)=1/k approximately holds for each european country, the power law behavior is no more observed once the countries are aggregated in the EU as expected
Italy France Spain
Replica effectFrance, Italy, Spain, etc, are the “right” setEU and USA have a difference development
![Page 51: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/51.jpg)
Models
Two mainstreams
Simon Model
Gibrat’s Law
Multiplicative process repelled from the origin
Simon growth model can be interpreted in the framework of coagulation processes
![Page 52: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/52.jpg)
An argument for Rank-Size Rules 1/kL. Pietronero, E. Tosatti, V. Tosatti, A. Vespignani, Physica A, 293, pp 297-304 (2001)
Hp: rank-size rule can be interpreted in terms of the exponent of the underlying pdf for city size
drawn from
The rank k of the x(k) number is given by all the numbers whose value is between x(k) and xmax
but....
![Page 53: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/53.jpg)
An argument for Rank-Size Rules 1/k
xmax cannot be neglected:
x(k) fails to reproduce the 1/k behavior for small values of k (i.e. big cities)
![Page 54: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/54.jpg)
Backward problemDirect problem: given the underlying pdf we search for the rank-size rule
Backward problem: given the rank-size rule 1/k we want to find the pdf for x
The direct problem is meaningful from a mathematical point of view since the range of x is fixed by the pdf range.
The backward problem fails since x(k) has not a priori an inferior bound
The pdf must depends on the set size and must know the values of the previous draws → conditioned draws
![Page 55: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/55.jpg)
Model for Rank-Size Rules: conditioned draws
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 56: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/56.jpg)
Model for Rank-Size Rules: conditioned draws
1. is drawn from f(x)
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 57: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/57.jpg)
Model for Rank-Size Rules: conditioned draws
1. is drawn from f(x)
2.
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 58: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/58.jpg)
Model for Rank-Size Rules: conditioned draws
1. is drawn from f(x)
2.
3.
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 59: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/59.jpg)
4. is drawn from f2(x)
Model for Rank-Size Rules: conditioned draws
1. is drawn from f(x)
2.
3.
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 60: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/60.jpg)
4. is drawn from f2(x)
Model for Rank-Size Rules: conditioned draws
1. is drawn from f(x)
2.
3.
N. is drawn from fN(x)
...
0.
Big cities have a strong screening effect
Unconditioned and conditioned draws nearly coincide for small cities
![Page 61: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/61.jpg)
Model for Rank-Size Rules: some results
![Page 62: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/62.jpg)
Cities around the worldData from Mathematica online database
Nigeria!
We perform an average over all the countries in order to compare real countries with the results of our model
![Page 63: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/63.jpg)
A model for Rank-Size Rules: some results
![Page 64: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/64.jpg)
Rank-size Laws vs timeM. Batty, Nature 444, pp 592-596
Today rank-size rules for cities approximately hold? Is this still true in the past?
![Page 65: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/65.jpg)
Rank-size Laws vs time: US urban system 1790-2000M. Batty, Nature 444, pp 592-596
The r ank - s i z e r u l e appear to be stable in time ...
... but the rank of each city is not constant
The time evolution is not trivial
![Page 66: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/66.jpg)
Rank-size Laws vs time: UK and world urban systemM. Batty, Nature 444, pp 592-596
UK 1901-2001
WORLD 430B.C. -2000 A.D.
![Page 67: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/67.jpg)
Widespread occurence of the inverse square lawG. Caldarelli, C. C. Cartozo , P. De Rios, V. Servedio, Phys. Rev. E, 69, (2004)
Super-critical tree/network Random division of the unit interval Phenomena organized in hierarchical subgroup
• taxonomy trees• drainage basins of rivers
• Simon and Yule processes• Aggregation processes - Smoluchowski’s coagulation
equation• cluster-cluster aggregation• DLA cluster cluster aggregation
• ?Matter density?
All these phenomena are characterized by aggregation growth or by successive fragmentations
![Page 68: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/68.jpg)
Microscopic origin of the inverse square law
Smoluchowski’s coagulation equation = mean field equation for cluster-cluster aggregation
If there are not any sinks and sources for particles, the system does not have a stationary states
W. H. White, Journal of Colloid and Interface Science, 87, (1982)We can add sinks and sources
Simon’s model is very similar: monodisperse source and no sinks
the stationary state exists
![Page 69: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/69.jpg)
Conclusion and perspectives
Role of the set/subset selection cities other phenomena
Role of the definition of the border of a city Analysis of rank-size plot versus time: dynamics Microscopic origin of the inverse square law
aggregation/fragmentation processes Smoluchoswki coagulation equation Is 1/x2 an attractor for aggregation processes? how strong? for which regime?aggregation/fragmentation processes in 2D embedded in the country “topology” (distance, radius of influence of a city, mountain, plain, etc)
![Page 70: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/70.jpg)
![Page 71: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/71.jpg)
![Page 72: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/72.jpg)
![Page 73: ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAWstatic.sif.it/SIF/resources/public/files/va2010/pietronero_0702b.pdf · ORDER BOOK MODELS AND THE MYSTERIES OF ZIPF LAW Complex Materials](https://reader030.fdocuments.in/reader030/viewer/2022041104/5f047b717e708231d40e302b/html5/thumbnails/73.jpg)