Social Learning and Diffusion over Pervasive Products: An Empirical Study of an African App-store's...

Social Learning and Diffusion over Pervasive Products:An Empirical Study of an African App-store’s Big Data

Meisam Hejazi Nia

University of Texas at Dallas (UTD)

April 30, 2016

Meisam Hejazi Nia (UTD) Social Learning over Pervasive Products April 30, 2016 1 / 37

Mobile App-Stores as a Two-sided Platform

$77 billion revenue forecast, from over 268 billiondownloads for 2017 (entrepreneur.com)

Connect mobile-app publishers with consumers

Platform revenue sources:

Advertising and revenue share from in-app purchaseson the freemium mobile appsRevenue share from paid mobile apps

Platform objective: estimate and maximize mobile appadoptions/demand

How to target promotion policies:

Viral marketing by giving bonus (gift) to users whoinvite othersTrial suggestion and giving bonus (gift) to users whowant to use

Data Concerns:

Long tail distribution of consumers’ app choices(unknown number of segments)Measure for social influenceBias sources: potential correlation between choices,omitted variables, measurement error, simultaneity ofadoptions


Mobile Apps and Consumers’ choices

Pervasive Goods (Watches, Smart-phones, Googleglasses?)

Hedonistic mobile apps versus Utilitarian ones

Consumers’ preferences’ interdependence:

Psychological benefits (SocialIdentification/Learning/Inclusion)Utilitarian benefits (Network Externalities)

Local or Global social influence?

Consumers behavior heterogeneity: Influentials orImitators?

Which product category in classical economy hassimilar adoption pattern?


Research Questions

How can I design a targeting approach for an app-store platform?

How does the social learning process of the mobile apps’ customers differfrom the social learning of the classical economy products, such as a colorTV?

How can an app-store platform capture the heterogeneity of its customersand the variation in the mobile apps to customize its marketing actions?

What are the key elements of the consumers’ utility of adopting a mobileapp that allows an app-store platform to group and target its potentialcustomers?


Why these questions in this context are significant?

“iTunes App Store Reaches 1.3 MillionMobile Applications.”

— techcrunch.com(2014)

“GARTNER: Mobile Apps Will HaveGenerated $77 Billion In Revenue By2017 ” — businessinsider.com(2014)

“African mobile penetration hits 80%(and is growing faster than anywhereelse) ” — venturebeat.com(2013)

“Global marketing isn’t dead: Whymarketing localization matters nowmore than ever” — sajan.com/blog(2013)

“Data Scientist Lesson:Micro-to-Macro Analytics Modeling byBill Schmarzo (20 years Big data Expert)”

— infocus.emc.com(2013)

Long Tail Distribution in Onlineworld, where shelf space is not scarcecommodity resource anymore


Overview of this study

Data

Individual daily app-choices of 20K consumers (30 global cities),1,300 local with 1,473 downloads, for 200 days (2013-2014): calledAggregated sample

A sample of 1,258 users (in a city in Africa) weekly choice over 10mobile app categories for 20 weeks (only one instance of non outsideoption)

Methodology

Model Consumers’ Choices by Hierarchical Dirichlet Process (DP)Multinomial Logit Model (estimated by MCMC)

Control variation in price, advertising, and product quality by a FactorModel (on stratified sample)

Extracted a social influence measure by simultaneous social learningdiffusion models (on the aggregate samples), estimated by

Discrete-NL State Space estimated with SUR, UKF, MCEM, MAP, GAapproaches


Overview of Results

Results

Social influence drives customers’ mobile app choices

Social influence at offline world (in contrast to online world) drivecustomer choices

Ignoring social influence creates biased estimates for mobile app’sdemand

Among many different classical goods, mobile app adoption pattern isvery similar to music CD’s adoption pattern

Counterfactual analysis suggests a policy that can increase mobile appadoption by 13.6%

Contributions

Estimate dependent mobile app store demand for targeting

Proposed an approach to combine macro and micro analysis(prescriptive analytics)

Benchmarked social learning parameters of mobile apps againstclassical economics products such as TV

Showed significance of social influence

Showed that ignoring social influence causes bias estimates

Identified social influence policy that increases mobile app diffusion by13.6%


App-store in Africa

Position of This Research in Literature


App categories Statistics

Index Category Total Downloads within local city

1 Dating 272 eBook 4143 Education & Learning 244 Health/Diet/Fitness 425 Internet & WAP 526 Movie/Trailer 5977 POI/Guides 228 Reference/Dictionaries 559 TV/Shows 135

10 Video & TV 105

Each of the 1, 258 customers under the study adopts only one of themobile apps during the course of study

The number of available apps within the categories varies between 12to 141

Number of free apps within the category varies between 7 to 120

Variance of prices of apps within the app categories locate at 0.51with variance of 1 (across days)


Diffusion and Popularity of mobile App Categories

Entertainment and Education app categories have top share ofpopular mobile apps

Diffusion of apps downloaded within categories follow sigmoidfunctional form


Free mobile apps versus paid mobile apps

Most of the mobile apps are free, so key cost is learning from others(e.g. friends)

Categories of mobile apps:

Investment or Utilitarian (Paid): Device tools, Health and Diet,Internet/WAPEntertainment or Hedonistic (Free): eBook, Games, Humor/Jokes,Social Net


App-Store: A Two Sided Market


Micro Choice Model

Consumers’ choice modelUtility of consumer i from down-loading an app in category j at week t︷︸︸︷

uijt =Preference of consumer i for mobile app category j︷︸︸︷

αij +αi11

App Download History (Control for Dependence)︷︸︸︷sit +

Control for variation in quality, advertising, price, assortment︷︸︸︷αi13F1jt + αi14F2jt + αi15F3jt

+αi12

Social Influence︷︸︸︷c immjt +

idiosyncratic error︷︸︸︷εijt

Purchase Probability︷︸︸︷pijt = e

vijt

1+∑J

j=1 evijt

,Outside Option Download︷︸︸︷

u0it = 0

Customer Specific Choice Parameters︷︸︸︷Ai = (αij , αi11, ..., αi15)

Mixture Normal Distribution, and DP when K1 =∞︷︸︸︷p(Ai −∆zi |π1, {µk1,Σk1}) =

K1∑k1=1

πk1φ(Ai −∆ zi︸︷︷︸CustomerTenure

|µk1,Σk1)


Mobile app information

Factor ModelMobile app category quality, price, assortment size, featured mean and variance︷︸︸︷

xjt =Factor Loading Matrix.︷︸︸︷

b

Vector of Latent Factors︷︸︸︷Fjt

+

Error Term Vector︷︸︸︷ejt , ekjt ∼ N(0,E )


Social Learning about Mobile Apps: State Space

Social Influence Model: Van den Bulte and Joshi (2007)observed cumulative downloads of mobile app j at day t︷︸︸︷

yjt

=

influential segment size︷︸︸︷θInfj

Cum. Num. Influentials︷︸︸︷c infjt +(1− θj)

Cum. Latent Num. Imitators︷︸︸︷c immjt +vjt ,

Observation Error︷︸︸︷vjt ∼ N(0,Vj)

c infjt = (

independent adoption rate︷︸︸︷pinfj +

dependent rate of adoption︷︸︸︷qinfj (

c infjt−1

M infj

))(

Market Size︷︸︸︷M inf

j −c infjt−1) + e infjt︸︷︷︸Bass Diffusion for Influential Segment

,

c immjt = (

independent adoption rate︷︸︸︷pimmj +

dependent rate of adoption︷︸︸︷qimmj (wj

c infjt−1

M infj

+

(1−Relative influence of influentials on imitators︷︸︸︷

wj )c immjt−1

M immj

))(M immj − c infjt−1) + e imm

jt ,

State Eq. Errors (full covariance matrix)︷︸︸︷ejt ∼ MVN(0,W )︸︷︷︸

Adoption for Imitator Segment

Parameters of Adoption Model︷︸︸︷Φj = (pinfj , pimm

j , qinfj , qimmj ,M inf

j ,M immj , θj ,wj), Φj ∼ N(µ,Σ)


Identification

Choice Model

Choice model identified by random coefficient logit (fixed diagonalscale)

Location of utility is set by normalizing the utility of outside option tozero

Endogeneity (omitted variable): control for potential correlations b/wchoices by explicitly modeling the inter-temporal interdependency ofchoices (choice history)

Control for potential confounding effects of price, advertising, productquality by including latent variation in these variables in choice model

Control potential measurement error in social influence by KalmanFilter

Control for heterogeneity and over-fitting by Dirichlet Process prior(Bayesian semi-parametric)

Avoid independence from irrelevant alternatives (IIA) by randomcoefficient model

Avoid label switching by limiting inference to the joint distributionrather than segment assignment

Social Influence Model

Control for dependence of social diffusion (simultaneity) by SeeminglyUnrelated Regression

Identify adoption parameters through prior over category parameters


Estimation Practices and their fit (1 from 2)

MCMC: Hierarchical Dirichlet Process Multinomial Logit Model

Gibbs sampling by iteratively sample segment membership of a usergiven segment membership of other customers (in contrast tocollapsed Gibbs sampling)

Metropolis Hasting Sampling with Random Walk by weightedlikelihood to generate proposal density

UKF + SUR: Discretized Social Learning Model

Robust to Process and Measurement Error, characterize uncertainty

Does not require Analytical Solution for simultaneous diffusion;Efficient, general and flexible, Underlying interdependency of mobileapp adoptions

Third Order polynomial approximation, unlike EKF which has firstorder approximation (Wan and van der Merve 2001, Julier andUhlmann 1997, Robotics literature)

Granular sparse data, mollifies interval bias, better in/out sampleprediction fit (Xie et al. 1997)


Estimation Practices and their fit (2 from 2)

MCEM + MAP: Social Learning Model

Monte Carol Expectation Maximization quicker than MCMC, somore appropriate for big data volume and velocity (de Valpine 2012)

Maximum A Posteriori account for heterogeneity, machine learningapproach, quicker unlike MCMC (also no problem of mixing and slowconvergence)

Genetic Algorithm: Social Learning Model

Stochastic surface search, which fits for problems with multiple localmaxima (like mixture models)

Global optimization Vankatesan et al. (2004)


Factor Model Results

Loadings/Components C1 C2 C3

Average File size (a proxy for appquality)

0.77 -0.07 -0.09

Dummy variable of Is Featured 0.82 0.3 0.01Average Price -0.06 0.94 -0.28Variance of Price -0.05 0.94 -0.19Number of Paid app Options 0.97 -0.09 -0.09Number of Free app Options 0.96 -0.15 -0.03Fraction of Free apps to Paid Apps -0.09 -0.25 0.87Average Tenure (time from creation) -0.08 0.67 0.48Total number of app Options 0.96 -0.14 -0.03

Supply Name Red Ocean app categories Paid app categories Free app categoriesDemand Name Popular Apps Investment Apps Freemiums


Model Performance: LL, MAD, MSE

Model Description Number ofobs.

Log Lik.

1 Local Adoption (aggregate sample) 2,000 -20,724.162 Global Adoption (aggregate sample) 2,000 -21,649.323 Choice Explained by Local Imitators Signals (micro sample) 22,644 -25,921.924 Choice Explained by Global Imitators Signals (micro sample) 22,644 -38,310.495 Choice Explained by Local Adopters Signals (micro sample)* 22,644 -12,252.856 Choice Explained by Global Adopters Signals (micro sample) 22,644 -15,275.207 Choice with No social influence measure (micro sample) 22,644 -15,977.04

Model Description MAD MSE

Local Adoption (Social Learning) 0.64 1.48Global Adoption (Social Learning) 0.03 0.12

LL favors choice model with social learning process modeled by thenumber of local adopters


Global Social Learning Parameters Estimates

Mobile App Categories: pinf qinf pimm qimm M inf M imm w θ

Device Tools 0.024 0 0.278* 0.192* 50* 580* 0.010* 0.039*eBooks 0.024 0 0.274* 0.189* 260* 4540* 0.007* 0.044*Games 0.026 0 0.293* 0.202* 80* 1150* 0.009* 0.046*Health/Diet/Fitness 0.025 0 0.288* 0.199* 100* 1600* 0.008* 0.048*Humor/Jokes 0.026 0 0.297* 0.205* 100* 1410* 0.008* 0.043*Internet/WAP 0.026 0 0.296* 0.204* 100* 1580* 0.010* 0.039*Logic/Puzzle/Trivia 0.024 0 0.275* 0.190* 90* 1440* 0.009* 0.039*Reference/Dictionaries 0.026 0 0.296* 0.204* 90* 1440* 0.007* 0.048*Social Networks 0.026 0 0.297* 0.205* 40* 390* 0.008* 0.044*University 0.025 0 0.281* 0.193* 130* 2080* 0.009* 0.046*

∗p ≤ 0.05

• Note: Global means across 30 cities the app store platform performs

• Rate of dependent and independent adoption is close to the same rates for music CDs


Local Social Learning Parameters Estimates

Mobile App Categories: pinf qinf pimm qimm M inf M imm w θ

Device Tools 0.025 0 0.282* 0.194* 5* 80* 0.100* 0.046*eBooks 0.024* 0 0.278* 0.191* 103* 1952* 0.032* 0.044*Games 0.024 0 0.275* 0.189* 3* 56* 0.246* 0.038*Health/Diet/Fitness 0.025 0 0.282* 0.194* 7* 120* 0.782* 0.038*Humor/Jokes 0.026 0 0.299* 0.206* 6* 99* 0.506* 0.041*Internet/WAP 0.025 0 0.285* 0.197* 11* 200* 0.738* 0.041*Logic/Puzzle/Trivia 0.025 0 0.282* 0.194* 6* 113* 0.344* 0.043*Reference/Dictionaries 0.026 0 0.299* 0.206* 12* 225* 0.940* 0.042*Social Networks 0.025 0 0.281* 0.193* 3* 48* 0.658* 0.040*University 0.025 0 0.281* 0.194* 6* 113* 0.555* 0.042*

∗p ≤ 0.05

• Note: Local means across the local city in Africa

• Rate of dependent and independent adoption is close to the same rates for music CDs


Model Performance: A Step Ahead Adoption Forecast

One step ahead forecast indicates model ability to capture the phenomena


Choice Model Parameter Vector estimate (Local Adopters)

Estimate Std. Dev. 2.5th 97.5th

Category specific preference:Device Tools -6.22* 5.04 -14.327 -2.669

eBooks -11.34* 3.14 -15.29 -6.381Games -4.35* 3.76 -11.222 -2.296Health/Diet/Fitness -4.1 2.18 -5.939 2.982Humor/Jokes -16.32* 5.85 -22.097 -9.715Internet/WAP -5.41* 2.29 -8.021 -3.021Logic/Puzzle/Trivia -14.2* 3.49 -18.122 -8.332Reference/Dictionaries -8.48* 1.92 -11.092 -4.547Social Networks -10.54 3.47 -15.53 0.076University -5.78* 1.39 -7.791 -2.916

States:Individual download history State -27.27* 5.46 -34.35 -13.821Latent imitation level 0.02* 0.01 0.011 0.035

App category characteristics (factors):Popularity of app category 1.32 0.63 -0.83 1.767Investment apps category 5.34 1.75 -0.922 7.23Hedonic apps category 7.13 4.28 -6.606 10.33

Health and Game app categories are more preferred than other appcategories (targeting)

Significant negative individual download history effect (usability ofapp-store)

Social influence has significant effect on choice

Ignoring social influence causes biased estimates


Effect of Tenure on Choice Parameters (Local Adopters):Targeting

Estimate Std. Dev. 2.5th 97.5th

Category specific preference:Device Tools -0.00044* 1.01E-04 -0.0006 -0.0002eBooks -0.00048* 2.63E-04 -0.0009 -6E-05Games -0.00041* 4.46E-05 -0.0005 -0.0003Health/Diet/Fitness -0.0008* 7.30E-05 -0.0009 -0.0006Humor/Jokes -0.00091* 2.49E-04 -0.0013 -0.0005Internet/WAP 0.00011 7.58E-05 -2E-05 0.00025Logic/Puzzle/Trivia -0.00056* 1.29E-04 -0.0008 -0.0004Reference/Dictionaries -0.00028 1.50E-04 -0.0005 0.00002Social Networks -0.00001 9.45E-05 -0.0002 0.0002University 0.00018 1.36E-04 -7E-05 0.00034

States:Individual download history State -0.00136* 3.33E-04 -0.0019 -0.0008Latent imitation level 0.00006* 7.64E-06 0.00004 0.00007

App category characteristics (factors):Popularity of app category 0.00001 2.06E-05 -3E-05 0.00005Investment apps category -0.00006 6.77E-05 -0.0002 0.00007Hedonic apps category 0.00021 1.35E-04 -3E-05 0.00043

Those who register early to the app-store (potentially with innovatorpersonality) have higher preference for mobile apps in Internet/WAPand university mobile app categories

Mobile innovators might be more interested in improving theirperformance oriented apps


Choice Model Parameters: Population Heterogeneity

Long tail distribution of preferences in online app store platform


Counterfactual Analysis

Viral Marketing and Sharing Button

max{c immjt }

∑Tt=1

∑Jj=2

∑Ii=1

exp(uijt)

1+∑J

j=1 exp(uijt)

s.t. cimmjt−1 ≤ c imm

jt , ∀t, j

Searched for the time varying shocks for social influence

(Viral Marketing or Sharing with friends and family Button/ campaign)


Counterfactual Analysis: Change in adoption level byintervening social influence

Category specificcounterfactualresults:

originalexpectedadoption

shut downsocial influ-ence

1% moresocial influ-ence

1%lesssocial influ-ence

An optimalsocial influ-ence

Device Tools 875.83 -57% 0.80% -0.70% 55.80%eBooks 189.45 -1% 0.00% 0.00% 2.50%Games 187.51 19% -0.30% 0.30% 58.60%Health/Diet/Fitness 22.21 0% 0.00% 0.00% 6.10%Humor/Jokes 255.09 0% 0.00% 0.00% 0.60%Internet/WAP 1042.2 23% -0.40% 0.50% 14.10%Logic/Puzzle/Trivia 249.12 25% -0.60% -0.20% 109.10%Reference/Dictionaries1262.09 16% -0.40% 0.30% -36.20%Social Networks 21.66 0% 0.00% 0.00% 0.30%University 18.08 -1% 0.00% 0.00% 1.50%Total improvement 4123.25 1% -0.10% 0.10% 13.60%

Surprisingly shutting down the social influence improves the totalexpected adoptions of mobile apps on this app-store platform

An optimal social influence policy shows 13.6% increase in totalexpected adoptions of the platform (new allocation policy based onpopularity)


Conclusion

Proposed an approach to combine macro and micro modeling to allowapp-stores to target their customers

Proposed Dirichlet Process to model customers’ heterogeneity, andUnscented Kalman Filter to estimate social influence measure

Showed the importance of social influence in determining mobile appadoption choices of customers

Showed that ignoring social influence in modeling customers’ adoption canbias the demand parameter estimates


Managerial Take Away

Social influence on mobile app adoption choices is effective locally (withinthe city of the study) rather than globally (over the internet)

The approach allows the mobile app store to design targeted policies

Mobile app adoption process is similar to that of music CD’s

If this app store runs viral marketing campaign focusing on more popularmobile app categories, it can increase its total adoption by 13.6%.


Thank You


Counterfactual Analysis: Optimal Social influenceexplanation by popularity

Coefficients Standard Error p-value t Stat

Intercept -0.185 0.147 0.245 -1.254Category popularity rank 0.050* 0.015 0.01 3.388

More popular mobile app categories have more improvement underoptimal policy


Probabilistic Graphical Model


Unscented Kalman Filter

UKF Recursionyk = H(xk) + ηk , ηk ∼ MVN(0, I )

xk = F (xk−1) + νk , νk ∼ MVN(0,Y ),k∈ {1, ...∞}

0 = E [x0], P0 = E [(x0 − x0)(x0 − x0)]

Wm0 = λ

L+λ , Wc0 = λ

L+λ + (1− α2 + β)

Wmi = W c

i = 12(L+λ) , i=1,...,2L,β = 2

Drawing Sigma Points: Ξ = (xk−1, xk−1 ±√

(L + λ)Pk−1

Updating Time:

Ξk|k−1 = F (Ξk|k−1), x−k =∑2L

i=0 Wmk Ξik|k−1

P−k =∑2L

i=0 Wci [Ξik|k−1 − x−k ][Ξik|k−1 − x−k ]T + I

Γk|k−1 = H[Ξk−1], y−k =∑2L

i=0 Wmi Γik|k−1

Updating Measurement:

Pyk =∑2L

i=0 Wci [Γk|k−1 − y−k ][Γk|k−1 − y−k ]T + Y

Pxkyk =∑2L

i=0 Wci [Γik|k−1 − x−k ][Γk|k−1 − y−k ]T

K=PxkykP−1yk, x−k + K (yk − y−k ),P−k − KPykK

T


Conditional distribution of micro choice model

Mixture Normal Hierarchical Multinomial Logit (MNHML)

αik |y jit , sjit , ind

1i , µ

1indi, σ1

indi, c imm

jt ,Fjt

ind1, π1, {µ1k1, σ

1k1}|{αi}Ii=1

π1 ∼ Dirichlet(a1), µk1|Σk1 ∼ N(µ1, a−1µ1

Σk1,Σk1 ∼ IW (ν1, nu1I )

a1 = ( 0.5k1 , k1), aµ1 = 1

16 , ν1 = dim(αi ) + 3

αi |indi , {µk1,Σk1} ∼ φ(αi |µind , σindi ), µindi ∼ MVN(π1), π1, {µk1,Σk1}|b1


Why Stochastic Time Varying Method, Kalman Filter?

UKF: Discretized

Robust to Process and Measurement Error, characterize uncertainty

Does not require Analytical Solution for simultaneous diffusion;Efficient, general and flexible

Granular sparse data, mollifies interval bias, better in/out sampleprediction fit (Xie et al. 1997)

OLS Inference Problem

Biased Estimates (Putsis and Srinivasan 1999)

No Direct way to find Standard Errors

MLE Inference Problem

Requires closed form solution

Downward biased Standard Errors (Van den Bulte and Lilien 1997)

NLLS

Requires closed form solution

Not robust to specification errors


Social Learning and Diffusion over Pervasive Products: An Empirical Study of an African App-store's...

Data & Analytics

Transcript of Social Learning and Diffusion over Pervasive Products: An Empirical Study of an African App-store's...