Lopez_Aldana_u3095134_PhD Thesis.pdfEVALUATION OF METHODS AND APPLICATIONS.
Maria Angelica Lopez Aldana.
Bachelor of Science – Biology,
Specialist in Statistics (Universidad Nacional de Colombia)
A thesis in fulfilment of the requirements for the degree of
Doctor of Philosophy
i
Abstract.
Ecological niche models (ENMs), which are created by linking species’ occurrence data with
environmental envelopes, are popular tools to answer fundamental questions in ecology and
evolution. Often, due to data availability, ENMs must rely on museum and herbarium data (or
PO data). As with any other type of model, ENMs’ performance will depend on the specific
model conditions such as model assumptions, data points quality and quantity and covariates
availability. Moreover, due to the lack of species’ absence information, ENMs utilizing PO
data should face additional caveats. The two main problems are; sampling bias and difficulty
in the estimation of species occurrence probability. This thesis aims to address some unresolved
issues in the ENM-PO field by using different datasets and conditions.
Whether it is possible to accurately predict species occurrence probability, when working with
presence only data, is the main question of the first part of this study. Specifically, I compared
two ENMs (i.e. Maxlike & MaxEnt) using a generous set of Acacia species. A strong
relationship between quantity of data and Maxlike model performance was described, and
further implications were discussed.
The consequences of using these methods to predict areas of distribution, rank covariates and
forecast climate scenarios were analyzed using the freshwater turtle Emydura Macquarii as a
case study. Although, similarities among methods were evident when ranking covariates and
predicting E. macquarii distribution. Predictions over time were particularly heterogeneous
among models, which prevents users from applying these methods in an interchangeable
manner.
The second part of the thesis moves beyond ENM methodologies, using ENMs to evaluate
phylogenetic niche conservatism (PNC) in Australia grasses, thereby tackling the question of
whether environmental niches are conserved among related species. Novel approaches were
ii
developed to evaluate niche conservatism, and more importantly, a relationship among niche
conservatism and phylogenetic clustering was identified. Specifically, Panicoidea clade
presented a tendency towards niche shifts. Theoretical and practical implications were
discussed.
planning.
v
Challenges of ecological niche modelling for presence only data. ................................... 8
Prediction errors and conservation decisions. ................................................................... 11
Thesis aims and structure. .............................................................................................................. 14
Chapter II. Comparing the performance of the Species Distribution modelling approaches
Maxlike and MaxEnt using distribution data of Australian Acacias as a case study. ............... 17
Abstract ............................................................................................................................................ 17
Introduction ...................................................................................................................................... 18
Methods ........................................................................................................................................... 21
Conclusions and Recommendations ................................................................................................ 41
Chapter III. A practice-oriented assessment of Maxlike and MaxEnt for modelling Emydura
macquarii. ........................................................................................................................................... 45
Abstract ............................................................................................................................................ 45
Introduction ...................................................................................................................................... 46
Methods ........................................................................................................................................... 49
Model fitting and evaluation ....................................................................................................... 50
Applied conditions ....................................................................................................................... 52
Variable contribution ............................................................................................................ 52
Discretized predictions ......................................................................................................... 53
Discussion. .......................................................................................................................................... 64
Conclusions ......................................................................................................................................... 70
Chapter IV. Phylogenetic niche conservatism in the spread of invasive grasses in
Australia .............................................................................................................................................. 73
Abstract ............................................................................................................................................ 73
Introduction ...................................................................................................................................... 74
Methods ........................................................................................................................................... 78
Conservatism in each genus ....................................................................................................... 81
Contrasting phylogeny and niche overlap ................................................................................ 82
Results .............................................................................................................................................. 83
Phylogeny, metabolism and overlap measures ........................................................................ 90
Discussion. .......................................................................................................................................... 92
Conclusions ......................................................................................................................................... 96
Appendix. ........................................................................................................................................ 107
References. ...................................................................................................................................... 129
List of Figures
Figure 1.1. Challenges and applications of ENM using PO data. Left panel, inputs methods
and source of variability (in green) of ENM. Right panel, geographic and environmental
purposes of ENM. Related chapters of this thesis are shown in orange. ................................... 7
Figure 2.1. (a) Box plot displaying one standard deviation around the mean intercept –based
in the 30 repetitions– estimated values for Maxlike model implementing linear and quadratic
features. (b) Maxlike’s maximum estimated probability of occurrence by species. ............... 29
Figure 2.2. Comparison among mean “probability” predictions from (a) Maxlike and (b)
MaxEnt models for a sample of six Acacia species. ............................................................... 31
Figure 2.3. Mean probability of occurrence for evaluation points and its equivalent number of
random selection background points for MaxLike versus MaxEnt models implementing linear
and quadratic features. The plotted probabilities at each point indicate the mean of the
predictors from the 30 models for each species. Percentage of species’ occupancy is displayed
next to the species’ names. ...................................................................................................... 32
Figure 2.4. Box plot displaying the 25th and 75th percentiles around the median predicted
probability for (a) evaluation points, (b) 10000 background points Maxlike and MaxEnt models
implementing linear and quadratic features. ............................................................................ 33
Figure 2.5. (a) Comparison among Maxlike and MaxEnt models implementing linear and
quadratic features using Akaike information criteria and (b) Box plot displaying the 25th and
75th percentiles around the median AUC (Area Under Operator Curve). .............................. 36
x
Figure 2.6. Box plot displaying the 25th and 75th percentiles around the median (a) proportion
of the study area predicted as present using (b) the threshold to correctly predict as present 95%
of test occurrences from Maxlike and MaxEnt models implementing linear and quadratic
features. ................................................................................................................................... 37
Figure 3.1. Maxlike and Maxent linear and quadratic (Maxent_LQ) and default features
(Maxent_DF) distributions. Predicted probability of presences at test locations in blue and
predicted probability of background points in red. ................................................................. 56
Figure 3.2. Importance of predictor variables for a. Maxlike, based on AIC values of jacknifed
models (= build it without the covariate under evaluation). b. Covariate’s importance according
for Maxent using percentage of variable contribution. Error bars = Standard error. ............ 60
Figure 3.3. (a) First column, comparison among predictions from Maxlike, Maxent linear and
quadratic features (Maxent_LQ) and Maxent default features (Maxent_DF). Black dots present
actual occurrences. (b) Predicted occurrences based on four thresholding methods; 10
percentile training data, Maximum Sensitivity-Specificity, Maximum Kappa and Fix threshold
value of 0.5 .............................................................................................................................. 61
Figure 3.4. Comparison of projections for Maxlike, Maxent linear and quadratic (Maxent_LQ)
and default features (Maxent_DF). CCSM4 model for past climate (i.e. Mid Holocene, about
6000 years ago) and future scenario –20170– under “MIROC5” global climate model for
greenhouse gas scenario RCP4. .............................................................................................. 63
Figure. 4.1. (a) Niche changes between native and invaded rage for grammineas’ Genera in
Australia using Schoener’s D values. Extensions above and below zero indicate expansion
(red) and unfilling (green) respectively. (b) Boyce Index evaluation of ENMs calibrated in the
native range and projected onto analogue climates in the invaded range. ............................... 85
xi
Figure. 4.2. (a) Phylogenetic tree of grasses by genera. Based on GPWG-II (2012). (b)
Boxplots of overlap (Schoener’s D) grouped and displayed according with phylogenetic
distance. Group 1: Agrostis, Puccinellia, Poa, Festuca; group 2: Echinochloa, Cenchus, Setaria,
Urochloa, Panicum and Digitaria; group 3: Sorghum and Paspalum; group 4: Eragrostis,
Chloris and Sporobolus. ......................................................................................................... 87
Figure. 4.3. “Distinctiveness index” (DI) by genera for its native and invaded rage. Genera
with asterisk (** 0.05) indicates significant difference in DI. ............................................... 88
Figure 4.4. Geographic distribution and phylogeny of genera grasses in Australia. Native
distribution in blue, invasive distribution in red. ................................................................... 91
xiii
List of Tables
Table 1.1. Comparative table of the main differences among Maxlike and Maxent ENM. .. 10
Table 1.2. Consequences of omission and commission errors in ENM common applications
for conservation. .............................................................................................................................. 13
Table 2.1. Modeled Acacia’s species, number of records reported, number of grid cell occupied
and percentage of occupancy on Australia……………………………………………….… 27
Table 2.2 Comparison of coefficient values among Maxlike and MaxEnt models for 16 species
of Acacias. Where βo is the intercept, β1 is the estimated parameter value for radiation, β2 is
the estimated parameter value for precipitation, and β3 is the estimated parameter value for
temperature. In bold pair values that differ in sign, notice that in these pairs one of their values
is close to zero. ................................................................................................................................. 35
Table. 3.1 Mean and standard deviation of performance measurements for: Maxlike and
Maxent linear and quadratic features (MaxEnt_LQ), and MaxEnt default features
(MaxEnt_DF). Evaluator values are calculated using 25% of dataset -testing data-. Better
performance is related to higher values of AUC, AVI, Threshold_95 and Boyce index and
smaller values of MPA. .................................................................................................................... 55
Table 3.2. Comparison of coefficients among linear and quadratic models of Maxlike and
Maxent. Models are visualized in figure 3.3a. In bold the pair of parameters that not share sign.
.............................................................................................................................................................. 58
Table 3.3. Sensitivity, percentage of correctly predict presences and percentage of occupied
area for the four different thresholding methods. ........................................................................ 62
xiv
Table 4.1. Number of occurrence in native range -Occ_nat-, number of occurrence in invasive
range -Occ_inv-, type of metabolism, niche overlap -Schoener’s D-, niche metrics: Expansion,
Stability and Unfilling. Genera ordered according with overlap-D value. In red genera with
niche shifts (p.value of similarity test > 0.05) ............................................................................... 84
xv
Acknowledgements.
There are many people who, without their support and assistance, this Ph.D would not have
happened.
Firstly, I would like to express my sincere gratitude to my advisor Prof. Bernd Gruber for the
continuous support of my Ph.D study, for his knowledge, patience and motivation. His
guidance played a large role in ensuring the completion of this thesis. Thanks to his expert
direction, I have learned so much about data managements, statistics and writing.
Besides my advisor, I would like to thank the rest of my thesis committee: Prof. Arthur Georges
and Dr. Carlos Gonzalez-Orozco. Thank you, Arthur, for passing on your detailed knowledge,
your generosity in you guidance. Carlos Gonzalez-Orozco, thanks for your ongoing support.
Even after you left the institute you continued supporting me, giving me insightful comments
and encouragement.
To all my committee. I deeply appreciate, not only your help, but also the hard questions which
encouraged me to improve my research from different perspectives.
My sincere thanks also go to Prof. Richard Duncan, who guided me in the last part of the Ph.D.
Thanks for providing new ideas and for promoting new learning spaces for PhD students.
Diverse academic spaces promoted by Richard have facilitated not only the exchange of ideas
but also the consolidation of learning spaces among IAE-PhD students.
The project would not have been possible without the funding provided by MDBfutures
Murray-Darling Basin Futures collaborative research network. I also received valuable
information from the University of Canberra wildlife tissue collection.
xvi
While in UC, I had support from several people. I would like to thank the Institute for Applied
Ecology (IAE) team, specially Barbara Harriss, Jane Ebner, and Ross Thompson for making
the life of students easier in relation to administrative and day-to-day matters and the IAE
students specially, Elodie Modave, Rheyda Hinlo, Margarita Medina, Rakhi Palit, Matthew
Young, Anthony Davidson, Andrew O’reilly, Kyle Hemming, Alexandra Herdenson and
Berenice T. for their support and companionship.
Last but not the least, I would like to thank my close friends and family. Thank you for your
support, tolerance and encouragement. To Mum, Dad, Marcela and Pablo, thank you for
supporting me spiritually throughout writing this thesis and my life in general. And to my
Australian family; to my partner Jonathon Tacey, and my lovely cat “Panela” thank you for all
your tolerance and support.
Chapter I. Introduction.
Definition of niche
A necessary concept to frame the empirical modelling of species distributions is the species
niche concept. There are at least two different concepts of niches. the Grinnellian and Eltonian
niche concept. The Grinellian niche concept focusses on describing the environmental
requirements needed for species’ subsistence without considering immigration (Grinnell 1917),
while Elton (1927) emphasizes the functional role of the species considering aspects of their
interactions with another species (e.g. trophic position). Different elements (i.e. data types,
methods, scale, etc) are used to describe functional (i.e. Eltonian) or geographical (i.e.
Grinnellian) niches (Soberon 2007).
Later, Huntchinson (1957) classified the niche into fundamental and realized niche. The
fundamental niche is set by the conditions and resources that allow a given organism to survive
and reproduce in the absence of biotic interactions (e.g. interspecific competition and
predation). The realized niche is the portion of the fundamental niche that the species occupies
(Araújo & Guisan 2006).
Hutchinson was the first to formally define the niche concept as “the activity range of
each species along every dimension of the environment”. These dimensions are also known as
scenopoetic variables (Peterson et al. 2011) and are measured at coarse spatial resolutions, thus
they are very important in determining the broad aspects of species distributions. These
dimensions are strongly related with the definition of the “Grinellian niche”, describing the
species’ environmental requirements, estimated in their associated geographic and
environmental spaces. On the other hand, the “Eltonian niche” concept focuses on species’
impact to their associated environment, which generally requires finer resolutions of
measurement of the predictors.
2
There are three constraints that determine the presence of a species in a place (Soberon &
Peterson 2005, Soberon 2007): Local environment, species interactions and dispersal capacity.
So, theoretically environmental conditions in the locations where the species is present are
useful to reconstruct the realized Grinellian niche of a species (Hirzel & Le Lay 2008). Realized
Grinellian niche is defined as the noninteractive, no consumable environmental conditions that
describe a species’ distribution in a particular location (Soberon & Nakamura 2009).
Explanations and spatial predictions of the Grinellian niche are the main goals of ENM. In line
with this, in this thesis we consider the Grinellian niche, since our interest lies at the relationship
between environmental predictors and the geographic extent of species.
From Niche theory to Ecological niche models.
Ecological niche, understood as a function that relates an organism’ fitness to its environment,
opened a new field in contemporary ecology: the ecological niche theory (Chase & Leibold,
2003). A recent and popular application of niche theory includes ecological niche modelling
(ENM). ENM typically correlates environmental features with species occurrences to describe
and predict environmental niches (e.g. Stockwell & Peterson 1999). ENM has being applied to
a set of different purposes in ecology, from conservation planning (e.g. reserve selection,
estimate invasive distributions or defined translocation areas), to study climate change effects
(e.g. Peterson et al. 2002), disease transition (e.g. Reed et al. 2008, Levine et al. 2007) and
evolutionary process (Peterson & Nyári 2007). Thus, ENMs provide important links to global
change theory (Chase & Leibold 2003) and potentially deepens our understanding of niches
and how they evolve over time, space and between species.
3
Common facets of niche theory are studied using ENM. Most of them are focused in the
characterization of a single species’ niche, but recently the applications have been extended to
evaluate niche interactions between species or to explore more complex problems such as niche
evolution (e.g. Martinez-Meyer & Peterson 2006, Pollock et al. 2014).
“Measuring” ecological niche: Clarifying terms and methods
Niche model applications
Species are distributed in geographical and environmental dimensions (Colwell & Rangel
2009). Geographic coordinates of species’ location serve as a link among geographic
distribution and the associated environmental conditions. Accordingly, when reviewing the
literature, I can identify two main lines of inquiry into ecological niche modelling (i) Studies
seeking insights, describing or identifying the environmental space of the species (i.e.
fundamental niche) and (ii) studies that focus on describing the geographical distribution of the
species (Fig 1.1).
Identifying fundamental Ecological niche – environmental space applications.
Some applications of ecological niche modelling focus on the explanation or description of the
fundamental niche of species. Early niche ecological studies had a strong ecological focus,
trying to identify the environmental conditions that drive the distribution of species (Mac Nally
2000). Contemporary ecology still uses ENMs for similar purposes in some areas, such as
quantitative ecology (Leathwick & Austin 2001) and evolutionary biology (Graham et al.
2004).
Linking niches with evolutionary process, requires the inclusion of another dimension
of analysis in niche ecology, the evolutionary time. Exploring the change in niche dimension,
through time, allows us to answer key questions of niche evolution, speciation and the
accumulation of ecological diversity within clades (Peterson et al. 2011, Warren et al. 2008).
4
Early studies integrated phylogenetic hypotheses with geographic distribution and ecology of
the species. These studies gave insights into the factors shaping evolution of the species (e.g.
Schneider et al. 1999, Johnson and Cicero 2002). Relatively recent efforts focus on describing
the changes of species’ niches over evolutionary time through to adaptation to climate changes
(i.e. niche evolution) and/or describe the tendency of the species to retain an environmental
niche (i.e. niche conservatism) (e.g. Broennimann et al. 2007, Broennimann et al. 2012).
Mapping predictions - Geographic space applications
Mapping predictions of the geographical distribution of the species has been used for a wide
range of biological applications including; defining the potential distribution of invasive
species (e.g., Peterson 2003, Peterson & Robins 2003), forecasting geographic range shifts
caused by climate change (Araújo et al. 2006, Hijmans & Graham 2006, Peterson et al. 2002,
Thomas et al. 2004) and estimating macro-ecological patterns such as species richness
(Graham & Hijmans 2006).
If predictions are made within the same range of environments sampled and within the
same general time frame, they are called interpolation to un-sampled sites. A typical example
for this type of predictions are the identification of a suitable area for a threatened species (e.g.
Thorn et al. 2009). On the other hand, predictions across time, (i.e. past and future climates) or
across the space (i.e. to un-sampled geographic areas) are commonly called forecasting (e.g.
Thomas et al. 2004). Generally, forecasting is more problematic and challenging than
interpolation. Space and time extrapolation are complicated because the correspondence
among climatic and non-climatic factors change across time and space (Pearson et al. 2006),
and the collinearity structure among covariates may change across time and space (Dormann
et al. 2012).
Species niche models, species distribution models, niche-theory models or even climatic
models are terms that have been used to describe ENMs. Although some authors have called
for concept differentiation (e.g. Peterson & Soberon 2012) these terms are often
interchangeable in the literature. I will call ENMs all methods that seek to describe a species’
ecological niche. This includes not only the most common correlative methods - species
distribution models (SDMs)- but also other methodological approximations such as principal
component analysis to compare species’ environmental distribution. Thus, species distribution
model (SDM) is a term used to refer exclusively to correlative methods of covariates seeking
to describe species’ probability of occurrence and their main purpose is being linked to map
predictions (Fig. 1.1).
Types of data of ENMs
ENM methods can be divided, according to the source of the data between presence/absence
(PA) or presence only (PO) methods. In the first case the area to be sampled (and mapped) is
defined prior and the probability of prevalence is estimated using classical statistical
approaches (see. Bailey et al. 2014). In contrast, when the model is based on presences only,
the expected prevalence is not directly computable and the spatial extent is not previously
defined. Commonly, the information about the distribution of a species is not available under
a PA scheme, but instead is compiled from several different resources (e.g. Herbarium and
museum data), resulting in a PO data set. ENM-PO methods require the use of surrogates to
get information about the possible environmental conditions where the species is absent. The
most commonly used approach when working with PO data is to select random points over the
landscape (also called background points) to contrast the environmental conditions among
presences and simulated absences (e.g. Elith et al. 2011, Renner et al. 2015).
6
Ideally, background data should be larger than the presence dataset. It should provide a good
representation of the environmental conditions in the landscape (Franklin 2009). Background
data are usually drawn at random from the research area, whereas occurrence data is often
biased towards easy and accessible areas. Since spatial bias could lead to environmental bias,
some authors have proposed to draw background data with the same bias as occurrence data.
This approach is commonly referred as background bias selection (see. Phillips et al. 2009).
PO datasets contain a set of georeferenced presences, a gridded landscape available for the
specie and the environmental data for each grid. PO models assume that the area has been
randomly sampled, then the records occur in proportion to the species’ preferences (Merow &
Silander 2013). Independently of the number of points registered within a grid cell, a 1 is
recorded.
7
Species distributions are the result of complex occurrence-environment relationships, driven
by three factors: environmental conditions, biotic interactions and dispersal (Soberon &
Nakamusa 2009). The first two components can be included into an ENM, allowing prediction
of species’ distributions based on the correlation between species presence records and
environment conditions (Guisan & Zimmermann 2000). To include dispersal process, some
authors have combined ENM with spead models (e.g. Roura-Pascual et al. 2009).
A variety of factors could alter the quality of an ENM. Inputs Characteristics -quality
and quantity of occurrences points, scale and relevance of the covariates and their collinearity-
as well as methods characteristics -algorithm, features and assumptions- are the main source of
ENM variability (Fig. 1.1). Furthermore, some posterior treatments (e.g. thresholding for
binary mapping) alter final predictions and thus utility of ENMs. Moreover, an ENM fitted
with PO data presented two additional challenges. First, PO data does not contain information
related to sampling effort, being more susceptible to sampling bias (Phillip et al. 2009). Second,
the calculation of occurrence probability has been proven to be complicated without absence
information (Hastie & Fithian 2013, Phillips & Elith 2013).
ENM-PO methods: Maxlike and MaxEnt.
MaxEnt is the most popular algorithm for ENM-PO data, however it generates relative
occurrence rates (ROR), indices proportional to habitat suitability, rather than a species
occurrence probability (Phillips et al 2006). MaxEnt is a machine learning model, which argues
that the best approximation of a distribution is determined by maximum entropy, subject to
constraints on their moments (Phillips et al. 2006). Maximum Entropy models seek to find the
distribution that is most spread out (i.e. closest to the uniform) restraint on the average of the
covariates (Elith et al. 2011). In the ENM context, the distribution being estimated is the
9
environmental covariates (which set the constraints) but “chooses” the distribution following
the maximum entropy principle (Elith et al. 2011).
Unlike MaxEnt, Maxlike is estimated by maximum likelihood using standard methods.
This allows Maxlike to apply the standard statistical inference techniques (e.g. hypothesis test,
confidence intervals or model selection based on AIC derivates) (Merow & Silander 2014).
Maxlike uses a logit-linear model which first ensures that the predicted value is a real
probability value and second, incorporates a value for the intercept, which differentiates the
ability to predict occurrence probability (Royle et al. 2012). This intercept represents the
expected prevalence across a landscape, which is assumed to be not identifiable under MaxEnt
formulation, though the prevalence needs to be set. As this quantity is often unknown, it is set
to 0.5 in most Maxent application. Conversely, Maxlike claims to be able to avoid this
assumption. In practical terms, Maxlike is able to derive real probability values of occurrences
but under assumption of random sampling and constant probability of species detection over
the landscape (Royle et al. 2012). A brief summary of the main differences among these
methods is given in Table 1.1.
10
Table 1.1. Comparative table of the main differences among Maxlike and Maxent ENM.
MaxEnt Maxlike
Prevalence is not identifiable Prevalence is identifiable
Estimate relative occurrence rate
prediction of state variable
Parameter estimation and hypothesis
testing easy to apply
Default features: linear, quadratic,
product, hinge and threshold
formula
with a GUI
Implemented in R
Prediction errors and conservation decisions.
To evaluate ENM, usually the dataset is split into a training and test dataset. This subsetting is
often carried out randomly. The training data set is used to derive the model and the testing
dataset is used to evaluate the model performance. Model performance is measured in terms of
omission –real presences predicted as absences – and commission – real absences predicted as
presences – errors. Omission and commission errors calculation requires a threshold value that
categorizes a continuous ENM output to a binary map. The critical point when measuring these
errors with a PO data is that it is wrong to consider that these two errors weight equally.
Presence testing datapoints are known to be actual presences, whereas testing background
datapoints are not known to be absences, hence omissions are true errors, whereas commissions
are not. This difference in the certainty of presences (occurrence points) and absences
(background data) creates the disparity between omission and commission errors. Commission
error is then a difficult measure to estimate when working with PO data. Areas predicted as
suitable for the species but without available species records could be consequence of lack of
sampling and/or detectability. Instead, omission error, is a more quantifiable measure -if we
consider all occurrence points to be real presences-. In short, use of background points inflates
false absences in SDM-PO data.
There are other evaluation methods, which do not require a conversion into presence
absences via a threshold to estimate ENM performance. The most common measure is the Area
Under the Curve (AUC) statistic, which is derived by using all possible thresholds to plot
sensitivity (the probability that a model correctly classifies a presence = 1-omission error)
versus specificity (the probability that a model correctly classifies an absence =1-commission
error). A value of 1 signifies a perfect performance, if above 0.75 the model is rated ‘good’,
12
while a value of 0.5 indicates a model no better than random (Graham & Hijmans, 2006, Lobo
et al. 2007, Newbold et al. 2009).
Additionally, model evaluation for ENM require a more detail analysis, since
committing error type I (commission error) or type II (omission error) have also different
implications in accordance to the target of the model. In table 1.2, I summarize the effect that
these errors could have on some of the most common ENM conservation purposes. For
example, control of commission error is more important when selecting areas for reserve
selection or translocation (i.e. areas to translocations of threatened species), because acquiring
a land and release a species in an unsuitable habitat implies high economic loss. Conversely,
control of omission error is often more important when determining the potential area of
invasive species, a false negative or a species that can establish in an area thought not suitable
could result in elevated costs for future eradications. Similarly, avoidance of omission error is
very important when creating a sampling design, as in this case overestimation of suitable areas
is less dramatic than underestimation (Guisan et al. 2013) (Table 1.2).
13
Thesis aims and structure.
This thesis is framed by the topic of environmental niche models. As ENM are a complex topic
with a variety of applications and challenges, I identified and selected some key research
questions related to its methodology and its applications (Fig 1.1). The thesis can therefore be
divided into two main parts. The first part (Chapter II and III) identifies and explores a
knowledge gap on the controversy of the suitability of two different methodologies to estimate
probability of occurrence. Specifically, we compare the statistical performance of the most
commonly used and popular SDM method -MaxEnt- and a relatively new method -Maxlike-,
which claims to estimate prevalence of a species based on PO data. The second part of the
thesis (Chapter IV), moves beyond ENM methodologies, using them to evaluate phylogenetic
niche conservatism (PNC), tackling the question of whether environmental niches are
conserved among related species.
My specific objectives are:
1) to compare the performance of MaxEnt and Maxlike models to predict species distributions
(Chapter II).
2) to evaluate these methods (i.e. MaxEnt and Maxlike) in an applied context, comparing their
capacity to inform conservation decisions (Chapter III).
3) to investigate the capacity of environmental niche models to evaluate phylogenetic niche
conservatism (Chapter IV).
4) To assess to what extend Grinnellian realized niches are conserved among native and
invasive poaceas’ genera in Australia (Chapter IV).
5) To explore the possible causes that drive phylogenetic niche conservatism among native and
invasive poaceas’ genera in Australia (Chapter IV).
15
In the following paragraphs, I detail each of these specific objectives.
In chapter II, I compare the popular ENM approach, Maxent, with a new method called -
Maxlike- using 30 Acacias species in Australia. MaxEnt is a machine learning method that uses
an exponential model to relate presence with environmental conditions. MaxEnt response
variable is an index of relative occurrence rate. Thus, MaxEnt is not estimating probability of
presence. Conversely, Maxlike is a maximum likelihood method that estimates probabilities of
species occurrence but its performance has not been widely tested. We aim to compare these
methods abilities to predict the distribution of a variety of Acacia species, and clarify the
conditions under Maxlike accurately predict probability of occurrences.
In chapter III, I contrast again these two methods but under a more applied context.
Beyond the statistical comparison among methods, I wanted to explore their differences when
informing common ecological questions. Using a dataset for a widespread turtle E. macquarii.
I aim to describe the differences of these methods when: (i) defining areas of suitability of the
species (ii) ranking covariate importance and (iii) projecting to past and future scenarios.
Finally, Chapter IV explores objective 3 to 5, by studying the relationship between
niches and evolutionary processes. Specifically, I explore how the climatic niche requirements
differ among native and invasive species of grasses within same genera. Exploring if these
requirements remain among close related species (phylogenetic niche conservatism) or change
(niche shifts). We use ENM to test transferability among invasive and native species
distribution, but also applied new methods to explore the degree of conservatism among native
and invasive ranges in the environmental space.
16
The chapters in this thesis (excluding Chapter I and Chapter V) are a set out as a series
of manuscripts; therefore, some repetition is unavoidable. However, I have endeavored to
minimize duplication as much as possible. Each chapter include an Abstract, Introduction,
Methods, Results and Discussion. The literature cited in each chapter has been combined into
a single reference list at the end of the thesis.
17
Comparing the performance of the Species Distribution modelling approaches Maxlike
and MaxEnt using distribution data of Australian Acacias as a case study.
Abstract
MaxEnt is the most commonly used species distribution model (SDM) to analyse presence only
(PO) data. Running Maxent results in estimates of relative habitat suitability which are scaled
between zero and one and therefore are often confused with species occurrence probabilities.
Maxlike, a new method based on a maximum likelihood framework, estimates real probability
values, but requires generous sample sizes. The inclusion of an intercept, term also called
prevalence, allows Maxlike to estimate absolute probabilities of occurrences. Before Maxlike
was developed the prevalence term was commonly assumed to be not identifiable, and its
calculation impossible. Only a few comparisons between both methods exist so far and none
have compared, empirically, their performance in terms of amount of data available and range
of species distribution. We compared the performance of both methods using occurrence data
of 30 native Acacia species distributed over the Australian continent. We identified two main
drawbacks of the Maxlike modelling approach; optimization failure (in 11 of 30 species) and
high variability of the intercept (SD>3; 8 of 30 species). In cases where the Maxlike approach
converges and its intercept estimation is not highly variable, both approaches show similarities
regarding predicted distribution maps, the sign of estimated parameters and AUC values.
However, the range and distribution of predicted values differs remarkably between the two
approaches. Specifically, the mean predicted value for testing points is highly variable for
models based on Maxlike, while values of testing points are remarkably consistent and close
to 0.5 for MaxEnt–based models. Although, both models are able to predict relative differences
in occurrence probability, Maxlike is able to estimate occurrence probability higher than 0.5
18
when the PO data is generous and widely distributed (more than 10% of available grid cells).
We conclude that although both methods are useful to describe species distribution, the high
data requirement of Maxlike could highly restrict its use.
Introduction
Species distribution models (SDMs) –also known as ecological niche models, ENMs– are of
major significance for theoretical and applied ecology (Loiselle et al. 2003). SDMs have been
increasingly implemented on a variety of challenges in conservation biology, biogeography
and environmental management (Guisan & Thuiller 2005, Guisan et al. 2013, Newbold 2010,
Primack 2006, Franklin 2013). SDMs correlate environmental covariates and the presence of
a particular species, seeking to predict its potential spatial distribution (Guisan & Thuiller, 2005
Franklin 2009; Peterson et al. 2011). SDMs can be developed using georeferenced points from
a proper, fine grained sampling design, where spatial extent is often limited and both, presences
and absences are recorded (Cawsey et al. 2002). However, their implementation, especially
over extensive areas, is impractically costly (Ward et al. 2009). Instead, often SDMs are
implemented using presence only (PO) data from large databases (i.e. records from museums
and herbaria) which used to cover larger areas but are collected opportunistically and –most
importantly– lacking information on species’ absences. The lack of absences leads to
difficulties in the estimation of probability of occurrences in SDM–PO. Therefore, these
methods have been relying on suitability indices which, in turn, often have been wrongly
interpreted as estimators of species occurrence (Yackulic et al. 2013).
In recent years, Maxent became the most commonly used method for analyzing PO data
(Yackulic et al. 2013, Phillips et al. 2006). Due to the lack of information about prevalence
(the average of the occurrence probability across the landscape), MaxEnt is not able to predict
occurrence probability. It uses maximum entropy to estimate a set of functions that relate
19
environmental variables and habitat suitability (Phillips et al. 2006). MaxEnt estimates relative
occurrence rates (ROR), relative differences in occurrence rates between cells (Merow et al.
2013), usually called ‘raw output’. However, assuming that the probability of occupancy under
average condition (i.e. prevalence) is 0.5 (Elith et al. 2011) MaxEnt offers an alternative output
called logistic output, which has been commonly called “suitability index”, it is monotonically
related to the raw output, but its values range between zero and one, being commonly confused
with occurrence probability in scientific literature (Yackulic et al. 2013).
To overcome these problems, Royle et al 2012, proposed an alternative PO model using
maximum likelihood approximation, called informally Maxlike. Maxlike estimates absolute
occurrence probability (Royle et al. 2012) using a logit–linear model as a link function, that
differs from the loglinear model used by MaxEnt in the inclusion of prevalence in its
calculations. Unlike MaxEnt, Maxlike estimates an intercept, which determines the total
number of presence records (= prevalence) (Hastie & Fithian 2013). The controversy in the
literature is now focusing on the identifiability of this value, as some authors claim it is not
identifiable (Ward et al. 2009; Phillips et al. 2009; Elith et al. 2011), while Royle et al (2002)
ensures it is estimable using the Maxlike method. However, Hastie and Fithian (2013) pointed
towards the awfully large amount of data that is required to describe prevalence using the
Maxlike method even when presence–absence data are used. Maxlike calculates the probability
of occurrence using several assumptions, namely random sampling and constant probability of
species detection, assumptions that shared with Maxent and other PO methods.
Although PO data is abundant and freely available, it is important to note that the PO
methods are based on several critical assumptions. First, sampling is considered to be either
random or representative across the landscape, therefore sampling should cover the whole
range of covariate values, and second, detection probability should be constant across sites
(Yackulic et al. 2013). These assumptions are not exclusive of PO data; however, PO methods
20
are relatively more sensitive to violations of them than presence/absence methods. Due to the
lack of planned sampling schemes and the variety of collection methods, PO data generally
gather occurrence points from a variety of samplings, so detection probabilities are most likely
not constant across the area, though, both methods (MaxEnt and Maxlike) implicitly assume
random sampling and constant probability of detection.
Royle et al. (2011) and Merow et al. (2013)1 reported a theoretical and empirical
comparison between these methods. Those comparisons were based on simulated data and also
provided an example based on empirical field data of a single species. The empirical
comparison of both papers was based on the same data set – North American Breeding Bird
Survey (BBS) –an extensive presence/absence database. However, both Authors compared
different outputs forms, and therefore derived different conclusions. Royle (2001) compared
outputs from the logistic regression using presence/absence data, Maxlike, and the Maxent
logistic version and reported strong similarities between the first two methods, while MaxEnt
underestimated “occurrence probability”. On the other hand, Merow & Silander (2013),
demonstrated that MaxEnt and Maxlike outputs are alike if cumulative output versions of both
algorithms are used. To my knowledge the only other empirical comparison was using small
datasets of four species of ants (Fitzpatrick & Otelli 2013), but their conclusions about
methods’ performance were inconsistent and depended on the evaluator metric being used.
Despite the different approaches of the comparisons between studies, all the studies found that
Maxlike exhibited high variability in the predicted probability of occurrences estimation when
applied to small samples.
1 For a comprehensive summary of the published comparisons of these methods, please refer to Appendix session – Appendix 2.6 –
21
Although these comparisons gave insight on the relative performance of each model,
we consider it to be necessary to expand the comparison using a larger number of species based
on a robust empirical dataset and the models’ regular outputs (i.e. Maxlike occurrence
probability and MaxEnt in its logistic form), as this is the most likely user scenario of those
methods. Furthermore, the species here included differ in number of data points and range of
distribution, allowing us to clarify the relationship between the amount of available data and
model performance.
We based the comparison of the performance of both methods using a single vegetation
group: Acacias, including linear and quadratic features on both methods in a set of 30 Acacia
species, which differ in expansion range and quantity of available data. We aim to explore the
advantages of each method under the user’s point of view, clarify the conditions under which
Maxlike exceeds MaxEnt performance and discuss the relevance of these methods for applied
ecology purposes.
Acacia database
We used 30 species of Australian Acacia randomly selected from a national scale dataset
(Gonzalez–Orozco et al. 2013). The records were extracted from Australia’s Virtual Herbarium
(AVH) (CHAH 2010); which originally entailed 218388 records. An extensive cleaning
process eliminated doubtful spatial records (e.g. points outside of the continent or located on
small islands), corrected taxonomic names and confirmed Acacia range by species, 18% of the
original records were eliminated (please refer to Gonzalez–Orozco (2013) for further details
on the preparation of the Acacia dataset).
22
In spite of the reduction in the total number of records as a result of the cleaning process,
the resulting 179,730 geographical records cover 99% of the grids cells (i.e. 864 over 868 cells
of the continent 10x10 km grid) (González–Orozco et al. 2013) and gather 1020 species of
Acacia of which we selected 30 for the present comparison.
Environmental covariates.
We used a set of environmental covariates from the WorldClim database (Hijmans et al. 2005)
with a 30 arc–seconds spatial resolution (i.e. about 1 km2). Three environmental covariates
were selected: mean annual radiation, mean annual precipitation and mean annual temperature.
These three covariates were selected because they simultaneously meet three important
conditions, (i) they are biologically relevant for vegetation distribution and population growth,
(ii) they follow the hierarchical framework proposed by Pearson & Dawson 2003, where
climatic covariates are shown to be the most relevant drivers of SDMs on a continental scale
(i.e. 2000 – 10 000 km) and (iii) Scientific literature evidence water and temperature–related
covariates as the most commonly used for building vegetation SDMs (Franklin 2009; Austin
& Van Niel 2011).
Before model fitting, the covariates were standardized to a mean of zero and variance
of one, following Royle et al. 2012 recommendations.
Statistical Modeling.
Model calibration:
Three empirical comparisons between the two modelling approaches Maxlike and MaxEnt
do exist. Two of them used the same set of data (Wren survey data from the BBS) and one is
based on a four species of ants. In order to make a more comprehensive empirical comparison,
this study covers a relatively large number of species, from a single plant genera on a
23
continental scale. Using a plant in distribution model has the clear advantage that they are often
much more directly linked to climatic predictors than animals (Guisan & Thuiller 2005), hence
it is easier to validate their performance, as the general outcome is easier to predict.
We modelled distributions of the 30 species of Acacia using both models (i.e. Maxlike
and MaxEnt). Since our aim is to contrast algorithms’ performance, we fixed the arrangements
including linear and quadratic features for both methods. This allows some complexity and
flexibility to the models and standardized the comparison. Similarly, to make an impartial
comparison, we do not include all features or bias corrections that are available for Maxent
approach.
Additionally, As the present comparison seeks to evaluate method’s performance in an
empirical manner, and considering that the most common output used to describe species
distribution models using MaxEnt is its logistic form, we decided to compare the logistic form
of MaxEnt (i.e. suitability index) against the probability of species occurrences from Maxlike.
All analyses were executed in R 3.1.2, MaxEnt and Maxlike models were calibrated
using “dismo” (Hijmans et al. 2016) and “Maxlike” (Chandler & Royle 2013) packages
respectively. These models were calibrated using a maximum of 10000 iterations to maximize
the respective algorithm (i.e. log likelihood function for Maxlike and Maximun Entropy
function for MaxEnt). We extended and customized Fitzpatrick el al. (2013) code to run our
comparison.
Model evaluation:
To evaluate the models, we used a standard cross–validation procedure. Dividing the records,
two third of the present–data set were used to calibrate the model and the remaining records
were used for evaluation (Sokal & Rohlf 1981, Manly 1991). This process was repeated 30
24
times, and for each run the fitting/testing selected points were identical for both models. The
30 independent replicates allowed the evaluation of the variability between resulting outputs.
A total of 10000 background points were selected to fit and evaluate models. Species
distributions maps were calculated averaging the predicted output of the whole set of iterations.
Similarly, output’s standard deviations were mapped in order to illustrate the variability.
Description of the evaluators:
We compared model performance using a set of descriptors and evaluators. We started with a
simple comparison of descriptors that gave insight about the predictive power of the models,
these were; mean predicted probability of the testing data and mean predicted probability of
the background data. We also used a set of evaluators suitable for PO data which quantify,
under different methods, model’s statistical fit to testing data. These measurements were: Area
Under Operator Curve (AUC), sample size corrected Akaike information criteria (AIC),
Minimum Predicted Area (MPA), Boyce Index (Bb) and Absolute Validation Index (AVI).
AUC – area under the receiver operating characteristic (ROC) curve– (Fielding & Bell
1997) is a threshold based evaluation method (Phillips et al. 2006). AUC is a single value of
discrimination measure that varies from 0 to 1. It is the area under the curve generated when
plotting sensitivity as a function of commission error for different values of the threshold range.
Commonly, it is reading as a proportion of better discrimination (higher values of probability)
in comparison with background points. An AUC of 0.7 means that 70% of the cases, where the
species has been recorded, show a higher value than a randomly selected background point.
AUC values of 0.5 is the expected value for a random model, values between 0.7 to 0.9 are
considered as reasonable predictions and AUC values higher than 0.9 are considered as very
good predictions (Swets 1988).
25
AUC is one of the most popular evaluators used in SDMs because it avoids the selection
of a particular value of threshold to quantify classification measures (e.g. omission and
commission error), describing classification power using a single value. However, AUC
ignores the predicted probability values and the goodness–of–fit of the model (Lobo et al.
2008) and it weights omission and commission errors equally. This problem is accentuated
when using background points instead of true absences in AUC calculation, because the
procedure inflates the number of false absences. AUC does not give information about the
spatial distribution of model errors, and the extent or area which the model is fitting influences
the specificity (i.e. proportion of correctly predict absences) of the model affecting AUC scores
(Lobo et al. 2008)
To evaluate the goodness–of–fit of the model we used sample size corrected Akaike
information criteria (AIC). For MaxEnt this measure is calculated over the standardized raw
values (i.e. all the scores over the extent sum to 1) and calculating likelihood of the data
following Warren and Seifert (2011). For Maxlike, AIC is calculated directly from the
maximized log–likelihood term.
Minimum predicted Area (Engler et al. 2004) identifies the necessary threshold that
yields 95% of the validation points as presences and uses this threshold to calculate the
proportion of the study area predicted as present. Models that show lower values of MPA are
considered superior (Franklin 2009 & Liu et al. 2013). Boyce Index (Boyce et al. 2002) varies
from –1 to 1. Bb, negative values indicated a poor model, values close to zero indicate that the
model is not different from a random model, and positive values of Bb indicate a consistent
model.
26
AVI Absolute validation index (Hirzel et al. 2004), which is the proportion of validation
points falling above a determined threshold; in this paper we set this threshold to 0.5.
Different evaluators measure model accuracy assessing different aspects of the model.
For example, AVI has the advantage to not required absence data. It is a simple and intuitive
measure to quantify the proportions of presence points falling in areas over a specific threshold.
(Li & Guo 2013). However, AVI does not consider commission error (Hirzel et al. 2006).
Conversely, Boyce index is more appropriate to evaluate a model’s ability to predict several
levels of suitability, but it is not useful for binary prediction (Boyce et al. 2002, Hirzel et al.
2006). Boyce index compares predicted and expected frequency of evaluation points by habitat
suitability range classes (see. Hirzel et al 2006).
In summary, a better model will be associated to higher values of AUC, AVI and Boyce index
and lower values of AIC and MPA.
Results
The set of species included in the comparison varies in number of records and occupied grid
cells. The number of records ranged from 236 records for A. latipes to 1254 records for A.
salicina. Occupied grid cells range from 30 for A. halliana to 349 for A. ligulata. (Table 2.1).
The number of records is not always positively correlated with the number of occupied grids.
Some species are highly reported in a limited extent (e.g. A. pulchella) whereas others are
poorly reported over an extended area (e.g. A. sibirica).
27
Table 2.1. Modeled Acacia’s species, number of records reported, number of grid cell
occupied and percentage of occupancy on Australia.
Species Grid cells
% Records
A. halliana 30 3.4 334 A. pruinocarpa 69 7.8 291 A. latipes 31 3.5 236
A. rhodophloia 74 8.4 252
A. microcarpa 33 3.7 331 A. adoxa 75 8.5 356 A. alleniana 35 4.0 247
A. paraneura 77 8.7 250
A. pulchella 35 4.0 1039 A. gonoclada 86 9.8 340 A. euthycarpa 37 4.2 978
A. ayersiana 88 10.0 269
A. hemiteles 37 4.2 320 A. deanei 89 10.1 924 A. lanigera 37 4.2 329
A. strowardii 90 10.2 257
A. triptera 38 4.3 284 A. monticola 128 14.5 709 A. dimidiata 39 4.4 354
A. sibirica 158 17.9 600
A. mucronata 39 4.4 922 A. stenophylla 165 18.7 946 A. floribunda 41 4.6 608
A. holosericea 190 21.5 1086
A. rubida 41 4.6 836 A. salicina 210 23.8 1254 A. terminalis 42 4.8 750
A. ramulosa 215 24.4 1457
A. crassa 43 4.9 556 A. ligulata 349 39.6 2853
28
Overall model convergence
We explored the standard deviation and the maximum probability of occurrence in Maxlike
models. We eliminated species for further analysis where the maximum value of the occurrence
probability was lower than 0.5 and/or the standard deviation of estimate of the intercept across
the 30 repeats were bigger than 3.0 (Fig. 2.1). Specifically, in 14 out of 30 cases Maxlike ran
into at least one of the following issues. First, Maxlike struggled to estimate models’ intercept
– in 8 out of 30 species –, presenting a high standard deviation of intercept estimation values
(Fig. 2.1 group B & C), and/or Maxlike presented optimization failure, it was unable to
converge to acceptable occurrence probability – in 11 out of 30 species – (i.e. maximum
occurrence probability inferior than 0.5) (Fig. 2.1; group A & C). Therefore 14 species were
eliminated from the further comparison, keeping only species for which Maxlike exhibited an
acceptable model (i.e. informative occurrence probability and no extreme large variation in the
intercept).
In general, the value of the estimate of the Maxlike intercept and the number of occupied grid
cells of a species are positively correlated. Species with low coverage tend to score lower values
of the intercept, while species with high coverage tend to score higher intercept values (Fig.
2.1). Contrary to Maxlike, MaxEnt estimated a relative probability of occurrence close to 0.5
for all species (Appendix. 2.1).
29
Figure 2.1. (a) Box plot displaying one standard deviation around the mean intercept –based in
the 30 repetitions– estimated values for Maxlike model implementing linear and quadratic features. (b)
Maxlike’s maximum estimated probability of occurrence by species. Group a. Species whose maximum
probability is inferior to 0.5 but its intercept’s standard deviation is inferior to 3.0; Group b. Species
whose maximum probability is superior to 0.5 but its intercept’s standard deviation is superior to 3.0;
Group c. Species whose maximum probability is inferior to 0.5 and its intercept’s standard deviation is
superior to 3.0 and Group d. Species whose maximum probability is superior than 0.5 and its intercept’s
standard deviation is inferior than 3.0. Note that Acacia species -displayed in the x-axis- are organized
in increasing order according with the number of grids cells occupancy, this number appears next to
the species name.
Comparison among models.
Continuing with the comparison with the remaining 16 species, we contrast three aspects of
the model. First, similarity among model outputs (i.e. visualization, correlation and outputs’
distribution). Second, parameter estimations and third, model performance according with the
chosen evaluators – AIC, AUC and MPA.
Mapped predictions from MaxEnt and Maxlike follow similar patterns (Fig. 2.2.), even
in some cases over predictions are presented in the same areas (e.g. A. euthicarpa –Fig. 2.2a).
Despite of these similarities, MaxEnt output is visually more diffused. For all species, MaxEnt
model has a core area of higher “probability values” that gradually declines toward the edges
of the distribution. Instead, Maxlike usually predicts larger areas of higher probability (Fig 2.2,
Appendix 2.2), so is able to show a higher contrast between core areas and the periphery of the
distribution.
When plotting models outputs for evaluation points, a clear positive relation among
methods is identified (Fig. 2.3). For species with lower coverage, estimated predicted
probability is rarely higher than 0.75 (Fig. 2.3. –first row–). However, Maxlike predictions tend
to reach higher values when the species’ coverage increase. Maxlike’ outputs for validation
points are close to one when the Acacia species is occupying more than 10% of the available
grid cells. On the contrary, MaxEnt rarely show predictions of evaluation points over 0.75 (Fig
2.3).
The mean probability of evaluation points is quite variable for Maxlike models while
MaxEnt estimations tend to be very close to 0.5. Its values are higher for Maxlike for species
with high coverage (>10%) (Fig. 2.4). Maxlike’s mean probability for background points is
slightly inferior or equal than MaxEnt’s for almost all selected species, with exception of A.
31
sibica and A. ramulosa. However, for both models the values were never larger than 0.4
(Fig.2.4).
Figure 2.2. Comparison among mean “probability” predictions from (a) Maxlike and (b)
MaxEnt models for a sample of six Acacia species.
a. b.
1
0
0.
25
0.
5
0.
75
1
33
Figure 2.4. Box plot displaying the 25th and 75th percentiles around the median predicted
probability for (a) evaluation points, (b) 10000 background points Maxlike and MaxEnt models
implementing linear and quadratic features. Species are increasingly ordered by the number of
occupied grid cells, which is displayed next to the species’ name.
34
The second aspect we compared among these methods was parameter estimation. Parameters
values tends to be alike in sign but not in magnitude. Discrepancies between sign values are
only present in cases where one of the parameter values is very close to zero. However,
parameter’s magnitute are very different between models, Maxent parameters tend to be larger
in comparison to Maxlike counterparts (Table 2.2).
Finally, we compared measures of goodness of fit and evaluation values. For all species,
Akaike Information Criterion (AIC) values were lower for Maxlike indicating a better
approximation model (Fig. 2.5a). AUC is slightly higher for Maxlike in almost all cases (with
exception of A. triptera). Independently of the model type, AUC values are higher than 0.9 for
all the species that are reported in less than 50 grid cells. This value decreases when the
occupancy of the species increases (i.e. present in more than 90 grids cells) (Fig. 2.5).
For all Acacia species Maxlike models had slightly smaller values of Minimum Predicted Area
(MPA) than MaxEnt. However, the threshold required to correctly predict 95% of the
evaluation points is higher for MaxEnt with just one exception (i.e. A. sibirica) (Fig. 2.6b). For
species with high coverage (i.e more than 90 grids cells) Maxlike showed slightly less MPA
values and thresholds closer to MaxEnt’s estimations (Fig. 2.6).
35
Figure 2.5. (a) Comparison among Maxlike and MaxEnt models implementing
linear and quadratic features using Akaike information criteria and (b) Box plot displaying
the 25th and 75th percentiles around the median AUC (Area Under Operator Curve). Species
are increasingly ordered by the number of occupied grid cells, which is displayed next to the
species’ name.
37
Figure 2.6. Box plot displaying the 25th and 75th percentiles around the median (a)
proportion of the study area predicted as present using (b) the threshold to correctly predict
as present 95% of test occurrences from Maxlike and MaxEnt models implementing linear
and quadratic features.
Due to the high correlation between evaluators (i.e. among AVI and mean probability
r2=0.92 and Boyce Index and AVI r2=0.81 –Appendix 2.3–) we show results of AVI and
Boyce Index in the appendix session. However, AVI tends to be higher for Maxlike for
species with high coverage. For species with occupancy inferior to 80 gridcells (<10%
occurrence), the proportion of validation points occurring in the predicted core habitat tend
to be inferior for Maxlike than MaxEnt (Appendix 2.4).
38
Continuous Boyce Index was greater than 0.9 for MaxEnt in almost all species with
only one exception, A. sibirica. Maxlike achieved similar satisfactory performance for
species with high coverage contrasting with very small values for species with restricted
distribution (Appendix 2.5). Like AVI, Boyce Index showed a good Maxlike calibration
models for species that are widely distributed.
Discussion.
Although restricted comparisons are present in the literature, this is the first
comparison between both modelling approaches with a generous number of species. In fact,
the two most relevant papers are exemplified with the same dataset (i.e. Royle et al 2012,
Merow & Silander 2013) and only Fitzpatrick et al. (2013) includes a set of six species.
Here, we not only increase the number of species, allowing us to evaluate the effect of
sample size and species range, but also we consider more complex models, including linear
and quadratic features (Appendix 2.6).
Royle et al. 2012 declare that the main advantage of Maxlike over MaxEnt is its ability to
deliver real occurrence probabilities. In this study, Maxlike only partially accomplished this
aim, with approximately half of the species presenting one or both following problems; (i)
inability to find the global optima (i.e. to estimate prevalence, higher than 0.5) and or (ii)
higher variability in probability of occurrence estimations (intercept standard deviation
higher than 3.0). Despite following Royle et al (2012) suggestion to standardize the
covariates prior analysis, Maximum likelihood optimization failed to converge in
approximately a third of species (Fig. 2.1, Appendix 2.2). For these species, mean
occurrence probability calculated across the thirty fitted models, is close to zero.
39
Merow & Silander (2013) pointed the similarities between these methods, making
clear that the mean difference in its formulation resides mainly in the inclusion of the
intercept in Maxlike. The intercept defines the expected prevalence across a landscape (i.e.
proportion of occupied cells), which – in spite of being considered as unidentifiable by some
authors (Ward et al. 2009, Elith et al. 2011) – is included in the logit-linear model of
Maxlike. The loglinear model used by MaxEnt omits the inclusion of the intercept in its
formulation delivering relative occurrence rate (ROR) (also called MaxEnt raw output)
instead of real probability of occurrence (Merrow & Silander 2013). However, as an attempt
to approximate ROR to a more conventional probability value, MaxEnt in its logistic form,
assuming a prevalence of 0.5, gives an estimative of suitability which ranges from 0 to 1.
Basing the comparison to the species where Maxlike models were able to deliver an
acceptable output (maximum probability value was above 0.5 and its estimate of the
intercepts’ standard deviation did not exceed 3.0; Fig. 2.1. –Group D), we found similarities
between models outputs. Maxlike and MaxEnt output values for evaluated points are
correlated (Fig. 2.3). Therefore, maps tend to follow similar patterns (Fig. 2.2), identifying
the same overall areas of high and low “probability”. Furthermore, parameter coefficient
values tend to coincide in sign though they often differ in magnitude. MaxEnt tends to
estimate higher absolute parameters values (Table 2.2). These results reinforce Merow &
Silander (2013) findings about model performance of these models: Maxlike and MaxEnt
are models that can reliably predict species distributions.
Despite of similarity in the resulting maps of Maxlike and MaxEnt, the actual
“probability value” of occurrence in grid cells are considerable different, MaxEnt scores
more frequently medium values (i.e. around 0.5), assigning lower values of “probability” on
grids where the species has been reported and occasionally scoring higher probabilities on
areas were species are not found (Fig. 2.2 & Fig. 2.3.). In fact, MaxEnt mean predicted
40
probability for evaluation points is barely moving away from 0.5 with a very small
variability (Fig. 2.4). These results coincide with Fitzpatrick et al. (2013) and Royle et al.
(2012), showing MaxEnt’s inability to clearly distinguish between areas of high and low
probability of occurrence. This is due to the default value of prevalence in MaxEnt’s
settings, which has been set up to 0.5. This arbitrary default value assumed by Maxent has
been repeatedly criticized. And although prevalence could be modified in MaxEnt
software’s options, it is not a common practice, as overall prevalence is unidentified for most
species.
This strong MaxEnt assumption could provide an argument to favour the use of
Maxlike, as contrary to MaxEnt, Maxlike is able to better discriminate presence points from
background points, scoring higher probability values over areas of observed presences and
lower values over areas without registered presences (Fig. 2.4). In fact, the predicted
performance values for AUC and MPA are slightly better for Maxlike for most of the species
(Fig 2.5 & 2.6) supporting its discriminative power. Similarly, measure of goodness of fit
values (i.e. AIC) are also higher for Maxlike for all species, however Maxlike distribution
models generally showed a higher variance than expected. This overdispersion – when there
is more variability in the data than would be expected from the fitted model – biases AIC
values (Symonds & Moussalli 2011). So, although we compare two methods with the same
number of parameters, lower values of Maxlike AIC could be erroneously associated with
better models.
Although goodness of fit and some evaluation measures (i.e. AIC, AUC and MPA)
tend to support Maxlike models –regardless of the size of species occupancy or grid cells–,
these measurements not include the actual predicted value in its calculations and therefore
they were unable to show Maxlike inability to predict useful (>0.5) probability value when
the species range is narrow. However, AVI clearly show the relationship among Maxlike
41
performance and species range sizes. Widespread species have higher –thus better– AVI
values and Boyce index close to one (Appendix 2.4 & 2.5). Combining the conclusions
related to the predicted values of probability of the models and evaluators measures, we can
say that Maxlike is a useful model to adequately describe the probability of the species,
when the species is widely distributed. In our case, Maxlike successfully describe occurrence
probability for Acacias distributed in more than 80 grids (approx. 800000 km2). Maxlike
drawbacks limited its applications, even when generous sampling sizes are available.
Specifically, when a sample is significant – a demonstrated condition for accurate Maxlike
probability estimation – overfitting is avoided, favoring the implementation of more
complex models such as MaxEnt.
Conclusions and Recommendations.
Based on our study, we found that Maxlike could estimate species occurrence probability if
an Acacia species is present in more than 10% of the grid cells (> 80 grids cells), though not
even under those favorable conditions, it is certain that Maxlike will converge satisfactorily.
In other words, the Maxlike approach fails to converge in some cases even with generous
datasets of widely distributed species. This complication is likely to limits the application of
Maxlike in conservation scenarios. For instance, the distribution of rare and cryptic species,
where the data is limited, will most likely result in a none–converging or highly variable
estimates of parameters, hence it is impossible to accurately estimate distribution of such
species via Maxlike. In constrast, MaxEnt proved approximate distributions of Acacia
species independently of the size of the data set and area occupied. Although MaxEnt’s
output is not a proper probability but a suitability index it provided better estimates when
the geographic distribution of a species is limited and the number of available data points is
small.
42
Implementing SDMs in R, on large continental areas, is computationally expensive.
MaxEnt however, is implemented in a variety of options; in its original Java version and
recently implemented in faster platforms (e.g. cloud–based Biodiversity and Climate
Change Virtual Laboratory (BCCVL); Hallgren et al. 2016). Although, “running” time
should not be a reason to disqualify a model, it is a point to consider specially when
modelling a lot of species – with its replicates – for practical or quick assessments of species
distributions.
However, precise estimates of species probability of occurrence and prevalence
potentially expand and improve SDMs applications. For example, if Maxlike estimates a
prevalence of 0.3, then the expected value of the species in the sampled area is 30%. A solid
estimate of species occurrence probability with areas itself can be really useful to optimize
the conservation and management of a species, as it theoretically allows to incorporate those
probabilities in conservation optimization approaches such as Marxan and Zonation (Ball et
al. 2009, Watts et al. 2009, Moilanen et al. 2005).
Furthermore, because MaxEnt is not a stochastic data model but a machine learning
approach, it is not suitable for hypothesis testing or exploring the explicit predictions of an
actual state variable (Royle et al. 2012). In fact, most of the published papers working with
empirical data and using MaxEnt focus almost exclusively in the resulting map without
making further statements of perform further analysis on the relationships of the covariates
with the response variable (Yackulic et al. 2013).
Most criticisms on MaxEnt are rather related to its interpretations and uses than to
its design. MaxEnt is a user–friendly software which allows to model simple and complex
relationships. This encourages its use, but also increases the danger of misinterpretations.
We conclude that it is a well-established method to describe relative differences between
43
areas of occupancy and its output –suitability index– is appropriate for some SDM purposes.
For example, to support the delineation of a reserve area when the amount of money or land
area is fixed. In contrast, Maxlike is a model that allows a better understanding of the
relationship between covariates on occurrence probability, so it could result in a better
understanding when describing the ecological niche of a species, if this is the focus of the
SDM.
Other limitations should be evaluated when applied SDM–PO methods, especially for model
interpretation. SDM seeks to obtain a value of occurrence probability that reflects the
possibility of a species to be present in a grid cell. Occurrence probability proves to be an
elusive value to obtain especially from presence PO data, because its presences are obtained
without a sampling scheme.
Thus, probability of detection is not only highly variable over the landscape but is also
commonly inferior than one – an issue shared with presence absence datasets- (Graham et
al. 2004, Guillera-Arroita et al. 2015).
Selection of the best SDM–PO method is a complex decision. Every SDM–PO
should be designed in alignment to its purpose, biology of the species, scale and quality of
environmental covariates. Both methods (Maxlike and MaxEnt) were found to be useful to
describe species distribution. However, accurate estimations of occurrence probabilities are
not always estimable with Maxlike model. Therefore, two considerations should be made
before modelling with Maxlike. First, if it is of importance for the study to have estimates
on the probability of occurrence for a species. Second if the available data set is of a decent
size in terms of number of records and third, if the species is widespread in the intended
study area. If any of those question is answered “no”, we have a tendency towards Maxent
as the preferred approach.
A practice-oriented assessment of Maxlike and MaxEnt for modelling Emydura
macquarii.
Abstract
The maximum entropy approach (MaxEnt) is the most common Spatial Distribution Model
(SDM) using presence only (PO) data, but it has been criticised for yielding a spatial index
of species suitability without any associated measure of reliability (e.g. probability). A
promising new method, Maxlike, is able to estimate probability of species occurrence but
requires generous amounts of data. We empirically compare these methods in their
performance and capacity to inform ecological decisions. We explore the methods ability to
(i) estimate the rank of covariate importance; (ii) predict binary species distribution; and (iii)
project past and future distributions under varying climate scenarios. We restricted both
approaches to identical model structures allowing for linear and quadratic features and
allowed for the full feature approach in Maxent to study the effect of complexity. We found
similarities among these models in spatial prediction, ranking covariate importance,
response to thresholding methods and parameters values. However, MaxEnt was superior to
Maxlike when using its full capacity, in that it presented better evaluation values and it was
less influenced by threshold type when converting continuous outputs. Difference among
methods were only evident when comparing projections under past and future scenarios. We
conclude that both methods are valuable for describing current species distribution.
However, temporal projections evidenced the associated uncertainty due to the method used,
which prevents the interchangeable usage of these methods for extrapolations.
46
Introduction
Global scale biological datasets, along with new powerful modelling methods, open new
opportunities to analyse species' distributions to support conservation decisions (Jetz et al.
2012). Spatial Distribution Models (SDMs) use statistical relationships among
environmental conditions and species observations to derive spatial predictions of species'
distributions. Increasing popularity of SDMs is evident in their use in a considerable number
of scientific papers available in the literature (e.g. Newbold 2010, Franklin 2013, Guisan et
al. 2013). For example, Guisan et al. (2013) reports a search of keywords in Web of Science
resulting in hundreds of articles at the beginning of 2000, extending into thousands of
publications in 2010. However, accuracy of SDMs depend on a variety of factors such as
the quality and quantity of data points, which covariates are included as environmental
layers, and the type of method. Different combinations of inputs such as data type and
covariates, as well as model type, could result in diverse predictions, resulting in a different
answer to a particular conservation enquiry (e.g. Peterson et al. 2007, Tsoar et al. 2007).
Usually, global-scale databases, such as records from museums and herbaria, are
collected opportunistically, lacking defensible information of species absences. These data
are known as presence only (PO) data and the models as SDM-PO. The most commonly
used method for analysing PO data is MaxEnt (Yackulic et al. 2013, Phillips et al. 2006).
Despite its popularity, some authors have identified problems with its use and interpretation.
The most common critique is that MaxEnt works with a response variable, commonly
referred to as suitability index, that is not a probability of occurrence that relies on the
assumption of a prevalence of 0.5 (Elith et al. 2011). To overcome these problems, Royle et
al. (2012), proposed an alternative PO model using maximum likelihood approximation,
called Maxlike. Maxlike is capable of estimating the absolute occurrence probability (Royle
47
et al. 2012) but requires a generous amount of data to deliver a reliable output (Hastie et al.
2009 , Chapter II – Lopez et al. 2017 (in prep)).
Statistical comparison between these methods are available in the literature (Merow
& Silander 2013, Royle 2012, Fitzpatrick 2011). Merow & Silander 2013, showed that
MaxEnt and Maxlike outputs are comparable if cumulative output versions of both
algorithms are used. However, most of MaxEnt applications have used its logistic form
(Yackulic et al. 2013), in which case outputs of both methods are correlated but rank
differently (Fitzpatrick 2013, Chapter II, Lopez et al. 2017 (in prep)). Therefore, these
methods could provide conflicting information when applied to common conservation tasks.
Although differences in SDM-PO performance have been widely described, their
applicability and relative utility in a conservation context was not studied yet. Here, we
develop a practice-oriented assessment of the efficiency of SDM-PO models (MaxEnt and
Maxlike) to support conservation decisions. We model the distribution of a widespread
species with adequate data, the freshwater turtle E. macquarii, to evaluate the ability and
consistency of MaxEnt and MaxLike methods when answering common ecological
questions, that of finding relevant environmental drivers, defining critical areas
(thresholding), and predicting the distribution in the past to identify potential refugia and the
future to inform the management of the species.
Globally, turtle populations have been declining sharply because of habitat
degradation, introduction of alien species, unsustainable and illegal harvest, and climate
change (Gibbons et al. 2000). E. macquarii is a wide-ranging species found in the rivers of
eastern Australia, and although not listed under the national Environmental Protection and
Biodiversity Conservation -EPBC Act-, it is listed as vulnerable in South Australia.
Knowledge of climatic factors that favoured E. macquarii distribution as well as predicting
48
its current, past or future distribution will inform conservation decisions on where best to
direct