COMMUNICATION

Tapas of Algebraic StatisticsCarlos Améndola, Marta Casanellas, and Luis David García Puente

What is Algebraic Statistics?Algebraic statistics is an interdisciplinary field that usestools from computational algebra, algebraic geometry,and combinatorics to address problems in statistics andits applications. A guiding principle in this field is thatmany statistical models of interest are semialgebraicsets—a set of points defined by polynomial equalities andinequalities.Algebraic statistics isnotonly concernedwithunderstanding the geometry andalgebra of theunderlyingstatistical model, but also with applying this knowledgeto improve the analysis of statistical procedures, and todevise new methods for analyzing data.

A well-known example of this principle is the modelof independence of two discrete random variables. Twodiscrete random variables 𝑋,𝑌 are independent if theirjoint probability factors into the product of the marginalprobabilities. Equivalently, 𝑋 and 𝑌 are independent ifand only if every 2 × 2-minor of the matrix of their jointprobabilities is zero. These quadratic equations, togetherwith the conditions that the probabilities are nonnegativeand sum to one, define a semialgebraic set.

In 1998, Persi Diaconis and Bernd Sturmfels showedhow one can use algorithms from computational algebraicgeometry to sample from conditional distributions. Thiswork is generally regarded as one of the seminal works ofwhat is now referred to as algebraic statistics. However,algebraic methods can be traced back to R. A. Fisher, whoused Abelian groups in the study of factorial designs,and Karl Pearson, who used polynomial algebra to studyGaussian mixture models.

Carlos Améndola is postdoctoral researcher in the department ofmathematics at Technische Universität München. His email ad-dress is carlos.amendola@tum.de.Marta Casanellas is associate professor and vice director of re-search in the department of mathematics at Universitat Politèc-nica de Catalunya. Her email address is marta.casanellas@upc.edu.Luis David García Puente is associate professor and assistantdepartment chair in the department of mathematics and sta-tistics at Sam Houston State University. His email address islgarcia@shsu.edu.

For permission to reprint this article, please contact:reprint-permission@ams.org.DOI: http://dx.doi.org/10.1090/noti1713

Algebraic statistics is a broad field actively expand-ing from discrete statistical models, contingency tableanalysis, and experimental design to Gaussian models,singular learning theory, and applications to phylogenet-ics, machine learning, and biochemical reaction networks.In this note, we will address two recent contributions tothis field: an extension of Pearson’s work on Gaussianmixtures and some recent results in phylogenetics.

Algebraic Statistics of Gaussian MixturesIn 1894 the famous statistician Karl Pearson [5] wantedto explain the asymmetry observed in data measuredfrom a population of Naples’ crabs, believing it waspossible that two subpopulations of crabs were presentin the sample. The corresponding statistical model isknown as a Gaussian mixture; in this case a mixture oftwo univariate Gaussian distributions, each with its ownmean and variance. In order to recover the parametersfrom the sample, Pearson introduced the method ofmoments, matching the density moments to the samplemoments. He obtained the following systemof polynomialequations in the means 𝜇1 and 𝜇2, variances 𝜎2

1 and 𝜎22 ,

and mixture proportions 𝛼1 and 𝛼2:

𝛼1+𝛼2 =1𝛼1𝜇1+𝛼2𝜇2 =0

𝛼1(𝜇21+𝜎2

1 )+𝛼2(𝜇22+𝜎2

2 ) =𝑚2

𝛼1(𝜇31+3𝜇1𝜎2

1 ) + 𝛼2(𝜇32+3𝜇2𝜎2

2 ) =𝑚3(1)

𝛼1(𝜇41+6𝜇2

1𝜎21 +3𝜎4

1 )+𝛼2(𝜇42+6𝜇2

2𝜎22 +3𝜎4

2 ) =𝑚4

𝛼1(𝜇51+10𝜇3

1𝜎21 +15𝜇1𝜎4

1 )+𝛼2(𝜇52+10𝜇3

2𝜎22 +15𝜇2𝜎4

2 ) =𝑚5.

After considerable effort and cleverness, Pearson man-aged to eliminate variables to obtain a ninth degreepolynomial relation in the single unknown 𝑥 = 𝜇1𝜇2,

24𝑥9 − 28𝜆4𝑥7 + 36𝑚23𝑥6 − (24𝑚3𝜆5 − 10𝜆2

4)𝑥5 − (148𝑚23𝜆4 + 2𝜆2

5)𝑥4+(288𝑚4

3−12𝜆4𝜆5𝑚3−𝜆34)𝑥3+(24𝑚3

3𝜆5−7𝑚23𝜆2

4)𝑥2+32𝑚43𝜆4𝑥−24𝑚6

3 =0,

(2)

where 𝜆4 = 9𝑚22 − 3𝑚4 and 𝜆5 = 30𝑚2𝑚3 − 3𝑚5. After

substituting his numerical moment estimates 𝑚𝑖, hefound the real roots of this nonic and determined if theycould correspond to a solution for the mixture model. We

936 Notices of the AMS Volume 65, Number 8

COMMUNICATION

see his approach as one of the first instances of algebraicstatistics. Pearson’s work leads to natural questions:

Problem 1. Can Pearson’s method be generalized for amixture of 𝑘 Gaussians? How many moments are neededto recover the parameters? Is there an analogous polyno-mial to (2)? What is its degree? What about Gaussians inhigher dimensions?

Recovering the parameters from data drawn from aGaussian mixture is an important problem in statistics,computer science, and machine learning. Answers tothe above questions shed light on the computationalcomplexity and the effectiveness of several algorithmsproposed in these areas. The key point is that all themoments of a mixture of Gaussians are polynomials inthe parameters, so they define moment varieties that canbe studied algebraically.

Recent progress with this approach has been made byAméndola et al. [2,3], with partial answers. For example,it was shown [3] that considering all the moments upto order 3𝑘 − 1 will yield generically a finite numberof Gaussian mixture densities with the same matchingmoments. In other words, the polynomial moment systemgeneralizing (1) will generically have a finite number ofsolutions for the 3𝑘 unknown parameters 𝜇𝑖,𝜎𝑖, 𝛼𝑖 for1 ≤ 𝑖 ≤ 𝑘. For 𝑘 = 2 this is Pearson’s number 9. For𝑘 = 3 it was found [2] that the corresponding degreeis 225. In contrast, perhaps shockingly, the system of20 polynomial equations in 20 unknowns correspondingto the moments up to order three of mixtures of twoGaussians in 3-dimensional space ℝ3 will have genericallyinfinitely many solutions. This means that one needs toconsider higher order moments in order to recover theparameters. A complete classification of such defectivecases is still open.

Algebraic Statistical PhylogeneticsAlgebraic statistics has been also used in phylogenetics.Phylogenetics seeks to explain the ancestral relationshipsamong a group of living species. These relationships areusually represented in a phylogenetic tree as in Figure 1,where the leaves are in bijection with the living species,the interior nodes represent ancestral species, the rootis the common ancestor to all the species in the tree,and the edges represent an evolutionary process that ledfrom one ancestral species to the next. Figure 1 showsthree possible phylogenetic trees that could explain theevolution of human, gorilla, lemur, and macaque.

In order to infer the phylogenetic tree that best explainsthe evolution of the species, one uses the genome of theliving species and models the substitution of nucleotidesusing a Markov process on trees, assuming that eachposition in the genome evolves in the same way andindependently of the others. We denote the set of fournucleotides by {𝐴,𝐶,𝐺,𝑇}. A discrete random variabletaking values in this set is assigned to each node ofthe tree. For each edge, the probabilities of substitutionof nucleotides between the two species at the ends ofthe edge are recorded in a transition matrix (a Markov

matrix). The entries of these matrices, together with thedistribution of nucleotides at the root of the tree, formthe parameters of the model. Then, the probability ofobserving a certain pattern of nucleotides at the leaves ofthe tree can be written in terms of these parameters, byassuming that the evolutionary processes of two edgesincident at a node 𝑣 is independent given the observationsat 𝑣.

Figure 1. Three phylogenetic trees representing threepossible evolutionary histories of human (h), gorilla(g), macaque (m), and lemur (l).

Again, the key point is that for each phylogenetic tree𝜏, the map 𝜑𝜏 that sends each set of parameters to thevector of probabilities of patterns 𝐴𝐴…𝐴, 𝐴𝐴…𝐶, … ,𝑇𝑇…𝑇 at the leaves is a polynomial map, and henceits image is (almost) an algebraic variety 𝑉𝜏. Differenttrees (as the ones in Figure 1) lead to different algebraicvarieties, and the goal is to use the equations that definethese algebraic varieties in order to decide, given a datapoint (that is, a sequence of nucleotides for each speciesat the leaves), to which variety is closest (in some sense).

The idea of using polynomial equations in phyloge-netics is not due to mathematicians but to biologists.Indeed, in the late 1980s, biologists James A. Cavender,Joseph Felsenstein, and James A. Lake already realizedthat the equations satisfied by the pattern probabilitieson a phylogenetic tree could help in inferring the treewithout having to estimate the parameters of the model.It is precisely this, the fact of not having to estimate theparameters, that makes algebraic statistics potentiallyuseful in phylogenetics. However, selecting a set of equa-tions that define the algebraic variety cannot be donein a canonical way. Moreover, the codimension of thesevarieties grows exponentially in the number of leaves, sousing them directly may not be a practical choice.

A recent approach to indirectly using these algebraicvarieties is based on the following result due to ElizabethAllman and John Rhodes: Assume the vector of probabili-ties 𝑝 = (𝑝𝐴𝐴…𝐴, 𝑝𝐴𝐴…𝐶,… ,𝑝𝑇𝑇…𝑇) belongs to the imageof 𝜑𝜏. Any edge of 𝜏 splits the set of leaves into twosubsets 𝑎 and 𝑏, giving rise to a matrix 𝑀𝑎,𝑏 whose rows(resp. columns) are labeled by the states at the leavesin 𝑎 (resp. 𝑏) and whose entries are the correspondingprobabilities in 𝑝. Then the matrix 𝑀𝑎,𝑏 has rank at most4. This result leads to equations satisfied by the pointsof the variety (the 5 × 5 minors must vanish), and it alsogives the possibility to test candidate phylogenetic treesby checking how far certain matrices are from the set ofrank 4 matrices. This distance can be easily computedusing singular value decomposition. This approach hasbeen recently exploited [1,4] with great success on bothsimulated and real data. As a consequence, algebraic

September 2018 Notices of the AMS 937

COMMUNICATION

tools have finally attracted the attention of biologists andhave been implemented in some widely used packages ofphylogenetic inference.

There are several books and even a journal dedicated toalgebraic statistics. The R package algstat contains manycomputational algebraic statistics tools including thestate of the art implementation of the Diaconis-Sturmfelssamplingmethod. The upcoming bookAlgebraic Statisticsby Seth Sullivant is a great resource for graduate studentsand researchers interested in learning more about thisexciting field.

References[1] Elizabeth S. Allman, Laura S. Kubatko, and John A.

Rhodes, Split scores: A tool to quantify phylogenetic signalin genome-scale data, Systematic Biology, 66:620–636, 2016.

[2] Carlos Améndola, Jean-Charles Faugère, and BerndSturmfels, Moment varieties of Gaussian mixtures, J. Algebr.Stat., 7(1):14–28, 2016. MR3529332

[3] Carlos Améndola, Kristian Ranestad, and Bernd Sturm-fels, Algebraic identifiability of Gaussian mixtures, Interna-tional Mathematics Research Notices, 2017.

[4] Jesús Fernández-Sánchez and Marta Casanellas, Invari-ant versus classical quartet inference when evolution isheterogeneous across sites and lineages, Molecular Biologyand Evolution, 65:280–291, 2016.

[5] Karl Pearson, Contributions to the mathematical theory ofevolution, Philosophical Transactions of the Royal Society ofLondon A: Mathematical, Physical and Engineering Sciences,185:71–110, 1894. MR541179

Image CreditsFigure 1 created by the authors.Author headshots courtesy of the authors.

Carlos EnriqueAméndola Cerón

ABOUT THE AUTHORS

Carlos Enrique Améndola Cerónwants everyone to know that al-gebraic statistics is a very coolsubject. Inspired by his PhD advi-sor Bernd Sturmfels, he believesapplied algebraic geometry topicshave enormous research potential.Additionally, he enjoys playingboard games, traveling around theworld, and dreaming about far, faraway galaxies.

Marta Casanellas

Marta Casanellas did her PhD inalgebraic geometry. After a post-doc at UC Berkeley, she shiftedher interest towards the applica-tions of algebraic techniques inphylogenetics.

Luis DavidGarcía Puente

Luis David García Puente worksin applied and computational al-gebraic geometry, algebraic statis-tics, and algebraic combinatorics.He has devoted much of his pro-fessional energy to broadeningparticipation in the mathematicalsciences through directing under-graduate research. He also enjoyssalsa dancing with his daugh-ters, coaching soccer, and playingracquetball.

938 Notices of the AMS Volume 65, Number 8