Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1777–1788Vancouver, Canada, July 30 - August 4, 2017. c©2017 Association for Computational Linguistics

https://doi.org/10.18653/v1/P17-1163

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1777–1788Vancouver, Canada, July 30 - August 4, 2017. c©2017 Association for Computational Linguistics

https://doi.org/10.18653/v1/P17-1163

Neural Belief Tracker: Data-Driven Dialogue State Tracking

Nikola Mrksic1, Diarmuid O Seaghdha2

Tsung-Hsien Wen1, Blaise Thomson2, Steve Young1

1 University of Cambridge2 Apple Inc.

{nm480, thw28, sjy}@cam.ac.uk{doseaghdha, blaisethom}@apple.com

Abstract

One of the core components of modern spo-ken dialogue systems is the belief tracker,which estimates the user’s goal at everystep of the dialogue. However, most currentapproaches have difficulty scaling to larger,more complex dialogue domains. This isdue to their dependency on either: a) Spo-ken Language Understanding models thatrequire large amounts of annotated trainingdata; or b) hand-crafted lexicons for captur-ing some of the linguistic variation in users’language. We propose a novel Neural Be-lief Tracking (NBT) framework which over-comes these problems by building on re-cent advances in representation learning.NBT models reason over pre-trained wordvectors, learning to compose them into dis-tributed representations of user utterancesand dialogue context. Our evaluation ontwo datasets shows that this approach sur-passes past limitations, matching the per-formance of state-of-the-art models whichrely on hand-crafted semantic lexicons andoutperforming them when such lexiconsare not provided.

1 Introduction

Spoken dialogue systems (SDS) allow users to in-teract with computer applications through conversa-tion. Task-based systems help users achieve goalssuch as finding restaurants or booking flights. Thedialogue state tracking (DST) component of anSDS serves to interpret user input and update thebelief state, which is the system’s internal repre-sentation of the state of the conversation (Younget al., 2010). This is a probability distribution overdialogue states used by the downstream dialoguemanager to decide which action the system should

User: I’m looking for a cheaper restaurantinform(price=cheap)

System: Sure. What kind - and where?User: Thai food, somewhere downtowninform(price=cheap, food=Thai,area=centre)

System: The House serves cheap Thai foodUser: Where is it?inform(price=cheap, food=Thai,area=centre); request(address)

System: The House is at 106 Regent Street

Figure 1: Annotated dialogue states in a sample di-alogue. Underlined words show rephrasings whichare typically handled using semantic dictionaries.

perform next (Su et al., 2016a,b); the system actionis then verbalised by the natural language generator(Wen et al., 2015a,b; Dusek and Jurcıcek, 2015).

The Dialogue State Tracking Challenge (DSTC)series of shared tasks has provided a common evalu-ation framework accompanied by labelled datasets(Williams et al., 2016). In this framework, the di-alogue system is supported by a domain ontologywhich describes the range of user intents the sys-tem can process. The ontology defines a collectionof slots and the values that each slot can take. Thesystem must track the search constraints expressedby users (goals or informable slots) and questionsthe users ask about search results (requests), tak-ing into account each user utterance (input via aspeech recogniser) and the dialogue context (e.g.,what the system just said). The example in Figure1 shows the true state after each user utterance ina three-turn conversation. As can be seen in thisexample, DST models depend on identifying men-tions of ontology items in user utterances. Thisbecomes a non-trivial task when confronted withlexical variation, the dynamics of context and noisyautomated speech recognition (ASR) output.

1777

FOOD=CHEAP: [affordable, budget, low-cost,low-priced, inexpensive, cheaper, economic, ...]

RATING=HIGH: [best, high-rated, highly rated,top-rated, cool, chic, popular, trendy, ...]

AREA=CENTRE: [center, downtown, central,city centre, midtown, town centre, ...]

Figure 2: An example semantic dictionary withrephrasings for three ontology values in a restau-rant search domain.

Traditional statistical approaches use separateSpoken Language Understanding (SLU) modulesto address lexical variability within a single dia-logue turn. However, training such models requiressubstantial amounts of domain-specific annotation.Alternatively, turn-level SLU and cross-turn DSTcan be coalesced into a single model to achievesuperior belief tracking performance, as shown byHenderson et al. (2014d). Such coupled modelstypically rely on manually constructed semanticdictionaries to identify alternative mentions of on-tology items that vary lexically or morphologically.Figure 2 gives an example of such a dictionaryfor three slot-value pairs. This approach, whichwe term delexicalisation, is clearly not scalableto larger, more complex dialogue domains. Im-portantly, the focus on English in DST researchunderstates the considerable challenges that mor-phology poses to systems based on exact matchingin morphologically richer languages such as Italianor German (see Vulic et al. (2017)).

In this paper, we present two new models, col-lectively called the Neural Belief Tracker (NBT)family. The proposed models couple SLU and DST,efficiently learning to handle variation without re-quiring any hand-crafted resources. To do that,NBT models move away from exact matching andinstead reason entirely over pre-trained word vec-tors. The vectors making up the user utterance andpreceding system output are first composed into in-termediate representations. These representationsare then used to decide which of the ontology-defined intents have been expressed by the userup to that point in the conversation.

To the best of our knowledge, NBT models arethe first to successfully use pre-trained word vectorspaces to improve the language understanding ca-pability of belief tracking models. In evaluation ontwo datasets, we show that: a) NBT models matchthe performance of delexicalisation-based modelswhich make use of hand-crafted semantic lexicons;

and b) the NBT models significantly outperformthose models when such resources are not avail-able. Consequently, we believe this work proposesa framework better-suited to scaling belief trackingmodels for deployment in real-world dialogue sys-tems operating over sophisticated application do-mains where the creation of such domain-specificlexicons would be infeasible.

2 Background

Models for probabilistic dialogue state tracking, orbelief tracking, were introduced as components ofspoken dialogue systems in order to better handlenoisy speech recognition and other sources of un-certainty in understanding a user’s goals (Bohusand Rudnicky, 2006; Williams and Young, 2007;Young et al., 2010). Modern dialogue managementpolicies can learn to use a tracker’s distributionover intents to decide whether to execute an actionor request clarification from the user. As men-tioned above, the DSTC shared tasks have spurredresearch on this problem and established a standardevaluation paradigm (Williams et al., 2013; Hen-derson et al., 2014b,a). In this setting, the task isdefined by an ontology that enumerates the goals auser can specify and the attributes of entities thatthe user can request information about. Many dif-ferent belief tracking models have been proposedin the literature, from generative (Thomson andYoung, 2010) and discriminative (Henderson et al.,2014d) statistical models to rule-based systems(Wang and Lemon, 2013). To motivate the workpresented here, we categorise prior research accord-ing to their reliance (or otherwise) on a separateSLU module for interpreting user utterances:1

Separate SLU Traditional SDS pipelines useSpoken Language Understanding (SLU) decodersto detect slot-value pairs expressed in the Auto-matic Speech Recognition (ASR) output. Thedownstream DST model then combines this in-formation with the past dialogue context to up-date the belief state (Thomson and Young, 2010;Wang and Lemon, 2013; Lee and Kim, 2016; Perez,2016; Perez and Liu, 2017; Sun et al., 2016; Janget al., 2016; Shi et al., 2016; Dernoncourt et al.,2016; Liu and Perez, 2017; Vodolan et al., 2017).

1The best-performing models in DSTC2 all used both rawASR output and the output of (potentially more than one) SLUdecoders (Williams, 2014; Williams et al., 2016). This doesnot mean that those models are immune to the drawbacksidentified here for the two model categories; in fact, they sharethe drawbacks of both.

1778

Figure 3: Architecture of the NBT Model. The implementation of the three representation learningsubcomponents can be modified, as long as these produce adequate vector representations which thedownstream model components can use to decide whether the current candidate slot-value pair wasexpressed in the user utterance (taking into account the preceding system act).

In the DSTC challenges, some systems used theoutput of template-based matching systems suchas Phoenix (Wang, 1994). However, more robustand accurate statistical SLU systems are available.Many discriminative approaches to spoken dia-logue SLU train independent binary models thatdecide whether each slot-value pair was expressedin the user utterance. Given enough data, thesemodels can learn which lexical features are goodindicators for a given value and can capture ele-ments of paraphrasing (Mairesse et al., 2009). Thisline of work later shifted focus to robust handling ofrich ASR output (Henderson et al., 2012; Tur et al.,2013). SLU has also been treated as a sequencelabelling problem, where each word in an utteranceis labelled according to its role in the user’s intent;standard labelling models such as CRFs or Recur-rent Neural Networks can then be used (Raymondand Ricardi, 2007; Yao et al., 2014; Celikyilmazand Hakkani-Tur, 2015; Mesnil et al., 2015; Penget al., 2015; Zhang and Wang, 2016; Liu and Lane,2016b; Vu et al., 2016; Liu and Lane, 2016a, i.a.).Other approaches adopt a more complex modellingstructure inspired by semantic parsing (Saleh et al.,2014; Vlachos and Clark, 2014). One drawbackshared by these methods is their resource require-ments, either because they need to learn indepen-dent parameters for each slot and value or becausethey need fine-grained manual annotation at theword level. This hinders scaling to larger, morerealistic application domains.

Joint SLU/DST Research on belief tracking hasfound it advantageous to reason about SLU andDST jointly, taking ASR predictions as input andgenerating belief states as output (Henderson et al.,2014d; Sun et al., 2014; Zilka and Jurcicek, 2015;Mrksic et al., 2015). In DSTC2, systems whichused no external SLU module outperformed all sys-tems that only used external SLU features. Jointmodels typically rely on a strategy known as delex-icalisation whereby slots and values mentioned inthe text are replaced with generic labels. Oncethe dataset is transformed in this manner, one canextract a collection of template-like n-gram fea-tures such as [want tagged-value food]. To per-form belief tracking, the shared model iterates overall slot-value pairs, extracting delexicalised featurevectors and making a separate binary decision re-garding each pair. Delexicalisation introduces ahidden dependency that is rarely discussed: howdo we identify slot/value mentions in text? Fortoy domains, one can manually construct semanticdictionaries which list the potential rephrasings forall slot values. As shown by Mrksic et al. (2016),the use of such dictionaries is essential for the per-formance of current delexicalisation-based models.Again though, this will not scale to the rich varietyof user language or to general domains.

The primary motivation for the work presentedin this paper is to overcome the limitations thataffect previous belief tracking models. TheNBT model efficiently learns from the avail-

1779

able data by: 1) leveraging semantic informa-tion from pre-trained word vectors to resolve lex-ical/morphological ambiguity; 2) maximising thenumber of parameters shared across ontology val-ues; and 3) having the flexibility to learn domain-specific paraphrasings and other kinds of variationthat make it infeasible to rely on exact matchingand delexicalisation as a robust strategy.

3 Neural Belief Tracker

The Neural Belief Tracker (NBT) is a model de-signed to detect the slot-value pairs that make upthe user’s goal at a given turn during the flow ofdialogue. Its input consists of the system dialogueacts preceding the user input, the user utteranceitself, and a single candidate slot-value pair thatit needs to make a decision about. For instance,the model might have to decide whether the goalFOOD=ITALIAN has been expressed in ‘I’m look-ing for good pizza’. To perform belief tracking, theNBT model iterates over all candidate slot-valuepairs (defined by the ontology), and decides whichones have just been expressed by the user.

Figure 3 presents the flow of information in themodel. The first layer in the NBT hierarchy per-forms representation learning given the three modelinputs, producing vector representations for theuser utterance (r), the current candidate slot-valuepair (c) and the system dialogue acts (tq, ts, tv).Subsequently, the learned vector representationsinteract through the context modelling and seman-tic decoding submodules to obtain the intermediateinteraction summary vectors dr,dc and d. Theseare used as input to the final decision-making mod-ule which decides whether the user expressed theintent represented by the candidate slot-value pair.

3.1 Representation Learning

For any given user utterance, system act(s) and can-didate slot-value pair, the representation learningsubmodules produce vector representations whichact as input for the downstream components ofthe model. All representation learning subcompo-nents make use of pre-trained collections of wordvectors. As shown by Mrksic et al. (2016), special-ising word vectors to express semantic similarityrather than relatedness is essential for improvingbelief tracking performance. For this reason, weuse the semantically-specialised Paragram-SL999word vectors (Wieting et al., 2015) throughout thiswork. The NBT training procedure keeps these

vectors fixed: that way, at test time, unseen wordssemantically related to familiar slot values (i.e. in-expensive to cheap) will be recognised purely bytheir position in the original vector space (see alsoRocktaschel et al. (2016)). This means that theNBT model parameters can be shared across allvalues of the given slot, or even across all slots.

Let u represent a user utterance consisting ofku words u1, u2, . . . , uku . Each word has an asso-ciated word vector u1, . . . ,uku . We propose twomodel variants which differ in the method used toproduce vector representations of u: NBT-DNNand NBT-CNN. Both act over the constituent n-grams of the utterance. Let vn

i be the concatenationof the n word vectors starting at index i, so that:

vni = ui ⊕ . . .⊕ ui+n−1 (1)

where⊕ denotes vector concatenation. The simplerof our two models, which we term NBT-DNN, isshown in Figure 4. This model computes cumula-tive n-gram representation vectors r1, r2 and r3,which are the n-gram ‘summaries’ of the unigrams,bigrams and trigrams in the user utterance:

rn =

ku−n+1∑

i=1

vni (2)

Each of these vectors is then non-linearly mappedto intermediate representations of the same size:

r′n = σ(W snrn + bsn) (3)

where the weight matrices and bias terms map thecumulative n-grams to vectors of the same dimen-sionality and σ denotes the sigmoid activation func-tion. We maintain a separate set of parameters foreach slot (indicated by superscript s). The threevectors are then summed to obtain a single repre-sentation for the user utterance:

r = r′1 + r′2 + r′3 (4)

The cumulative n-gram representations used bythis model are just unweighted sums of all wordvectors in the utterance. Ideally, the model shouldlearn to recognise which parts of the utterance aremore relevant for the subsequent classification task.For instance, it could learn to ignore verbs or stopwords and pay more attention to adjectives andnouns which are more likely to express slot values.

1780

Figure 4: NBT-DNN MODEL. Word vectors of n-grams (n = 1, 2, 3) are summed to obtain cumulativen-grams, then passed through another hidden layer and summed to obtain the utterance representation r.

Figure 5: NBT-CNN Model. L convolutional filters of window sizes 1, 2, 3 are applied to word vectorsof the given utterance (L = 3 in the diagram, but L = 300 in the system). The convolutions are followedby the ReLU activation function and max-pooling to produce summary n-gram representations. These aresummed to obtain the utterance representation r.

NBT-CNN Our second model draws inspirationfrom successful applications of Convolutional Neu-ral Networks (CNNs) for language understanding(Collobert et al., 2011; Kalchbrenner et al., 2014;Kim, 2014). These models typically apply a num-ber of convolutional filters to n-grams in the inputsentence, followed by non-linear activation func-tions and max-pooling. Following this approach,the NBT-CNN model applies L = 300 differ-ent filters for n-gram lengths of 1, 2 and 3 (Fig-ure 5). Let F s

n ∈ RL×nD denote the collectionof filters for each value of n, where D = 300 isthe word vector dimensionality. If vn

i denotes theconcatenation of n word vectors starting at indexi, let mn = [vn

1 ;vn2 ; . . . ;v

nku−n+1] be the list of

n-grams that convolutional filters of length n runover. The three intermediate representations arethen given by:

Rn = F sn mn (5)

Each column of the intermediate matrices Rn isproduced by a single convolutional filter of lengthn. We obtain summary n-gram representationsby pushing these representations through a recti-

fied linear unit (ReLU) activation function (Nairand Hinton, 2010) and max-pooling over time(i.e. columns of the matrix) to get a single featurefor each of the L filters applied to the utterance:

r′n = maxpool (ReLU (Rn + bsn)) (6)

where bsn is a bias term broadcast across all filters.Finally, the three summary n-gram representationsare summed to obtain the final utterance represen-tation vector r (as in Equation 4). The NBT-CNNmodel is (by design) better suited to longer utter-ances, as its convolutional filters interact directlywith subsequences of the utterance, and not justtheir noisy summaries given by the NBT-DNN’scumulative n-grams.

3.2 Semantic Decoding

The NBT diagram in Figure 3 shows that the ut-terance representation r and the candidate slot-value pair representation c directly interact throughthe semantic decoding module. This compo-nent decides whether the user explicitly expressedan intent matching the current candidate pair

1781

(i.e. without taking the dialogue context into ac-count). Examples of such matches would be ‘Iwant Thai food’ with food=Thai or more de-manding ones such as ‘a pricey restaurant’ withprice=expensive. This is where the use ofhigh-quality pre-trained word vectors comes intoplay: a delexicalisation-based model could dealwith the former example but would be helpless inthe latter case, unless a human expert had provideda semantic dictionary listing all potential rephras-ings for each value in the domain ontology.

Let the vector space representations of a candi-date pair’s slot name and value be given by cs andcv (with vectors of multi-word slot names/valuessummed together). The NBT model learns to mapthis tuple into a single vector c of the same dimen-sionality as the utterance representation r. Thesetwo representations are then forced to interact in or-der to learn a similarity metric which discriminatesbetween interactions of utterances with slot-valuepairs that they either do or do not express:

c = σ(W s

c (cs + cv) + bsc)

(7)

d = r⊗ c (8)

where ⊗ denotes element-wise vector multiplica-tion. The dot product, which may seem like themore intuitive similarity metric, would reduce therich set of features in d to a single scalar. Theelement-wise multiplication allows the downstreamnetwork to make better use of its parameters bylearning non-linear interactions between sets offeatures in r and c.2

3.3 Context ModellingThis ‘decoder’ does not yet suffice to extract intentsfrom utterances in human-machine dialogue. Tounderstand some queries, the belief tracker must beaware of context, i.e. the flow of dialogue leadingup to the latest user utterance. While all previoussystem and user utterances are important, the mostrelevant one is the last system utterance, in whichthe dialogue system could have performed (amongothers) one of the following two system acts:

1. System Request: The system asks the userabout the value of a specific slot Tq. If thesystem utterance is: ‘what price range would

2We also tried to concatenate r and c and pass that vectorto the downstream decision-making neural network. However,this set-up led to very weak performance since our relativelysmall datasets did not suffice for the network to learn to modelthe interaction between the two feature vectors.

you like?’ and the user answers with any, themodel must infer the reference to price range,and not to other slots such as area or food.

2. System Confirm: The system asks the userto confirm whether a specific slot-value pair(Ts, Tv) is part of their desired constraints. Forexample, if the user responds to ‘how aboutTurkish food?’ with ‘yes’, the model must beaware of the system act in order to correctlyupdate the belief state.

If we make the Markovian decision to only con-sider the last set of system acts, we can incorporatecontext modelling into the NBT. Let tq and (ts, tv)be the word vectors of the arguments for the sys-tem request and confirm acts (zero vectors if none).The model computes the following measures ofsimilarity between the system acts, candidate pair(cs, cv) and utterance representation r:

mr = (cs · tq)r (9)

mc = (cs · ts)(cv · tv)r (10)

where · denotes dot product. The computed similar-ity terms act as gating mechanisms which only passthe utterance representation through if the systemasked about the current candidate slot or slot-valuepair. This type of interaction is particularly usefulfor the confirm system act: if the system asks theuser to confirm, the user is likely not to mention anyslot values, but to just respond affirmatively or neg-atively. This means that the model must considerthe three-way interaction between the utterance,candidate slot-value pair and the slot value pair of-fered by the system. If (and only if) the latter twoare the same should the model consider the affirma-tive or negative polarity of the user utterance whenmaking the subsequent binary decision.

Binary Decision Maker The intermediate repre-sentations are passed through another hidden layerand then combined. If φdim(x) = σ(Wx+ b) is alayer which maps input vector x to a vector of sizedim, the input to the final binary softmax (whichrepresents the decision) is given by:

y = φ2(φ100(d) + φ100(mr) + φ100(mc)

)

4 Belief State Update Mechanism

In spoken dialogue systems, belief tracking modelsoperate over the output of automatic speech recog-nition (ASR). Despite improvements to speech

1782

recognition, the need to make the most out of im-perfect ASR will persist as dialogue systems areused in increasingly noisy environments.

In this work, we define a simple rule-basedbelief state update mechanism which can beapplied to ASR N -best lists. For dialogue turn t,let syst−1 denote the preceding system output, andlet ht denote the list of N ASR hypotheses hti withposterior probabilities pti. For any hypothesis hti,slot s and slot value v ∈ Vs, NBT models estimateP(s, v | hti, syst−1), which is the (turn-level)probability that (s, v) was expressed in the givenhypothesis. The predictions for N such hypothesesare then combined as:

P(s, v | ht, syst−1) =N∑

i=1

pti P(s, v | hti, syst

)

This turn-level belief state estimate is then com-bined with the (cumulative) belief state up to time(t− 1) to get the updated belief state estimate:

P(s, v | h1:t, sys1:t−1) = λ P(s, v | ht, syst−1

)

+ (1− λ) P(s, v | h1:t−1, sys1:t−2

)

where λ is the coefficient which determines therelative weight of the turn-level and previous turns’belief state estimates.3 For slot s, the set of itsdetected values at turn t is then given by:

V ts = {v ∈ Vs | P

(s, v | h1:t, sys1:t−1

)≥ 0.5}

For informable (i.e. goal-tracking) slots, the valuein V t

s with the highest probability is chosen as thecurrent goal (if V t

s 6= {∅}). For requests, all slotsin V t

req are deemed to have been requested. Asrequestable slots serve to model single-turn userqueries, they require no belief tracking across turns.

5 Experiments

5.1 Datasets

Two datasets were used for training and evalua-tion. Both consist of user conversations with task-oriented dialogue systems designed to help usersfind suitable restaurants around Cambridge, UK.The two corpora share the same domain ontology,which contains three informable (i.e. goal-tracking)slots: FOOD, AREA and PRICE. The users can spec-ify values for these slots in order to find restaurants

3This coefficient was tuned on the DSTC2 developmentset. The best performance was achieved with λ = 0.55.

which best meet their criteria. Once the system sug-gests a restaurant, the users can ask about the valuesof up to eight requestable slots (PHONE NUMBER,ADDRESS, etc.). The two datasets are:

1. DSTC2: We use the transcriptions, ASR hy-potheses and turn-level semantic labels pro-vided for the Dialogue State Tracking Chal-lenge 2 (Henderson et al., 2014a). The of-ficial transcriptions contain various spellingerrors which we corrected manually; thecleaned version of the dataset is avail-able at mi.eng.cam.ac.uk/˜nm480/dstc2-clean.zip. The training data con-tains 2207 dialogues and the test set consistsof 1117 dialogues. We train NBT models ontranscriptions but report belief tracking perfor-mance on test set ASR hypotheses providedin the original challenge.

2. WOZ 2.0: Wen et al. (2017) performed a Wiz-ard of Oz style experiment in which AmazonMechanical Turk users assumed the role ofthe system or the user of a task-oriented dia-logue system based on the DSTC2 ontology.Users typed instead of using speech, whichmeans performance in the WOZ experimentsis more indicative of the model’s capacity forsemantic understanding than its robustness toASR errors. Whereas in the DSTC2 dialoguesusers would quickly adapt to the system’s(lack of) language understanding capability,the WOZ experimental design gave them free-dom to use more sophisticated language. Weexpanded the original WOZ dataset from Wenet al. (2017) using the same data collectionprocedure, yielding a total of 1200 dialogues.We divided these into 600 training, 200 vali-dation and 400 test set dialogues. The WOZ2.0 dataset is available at mi.eng.cam.ac.uk/˜nm480/woz_2.0.zip.

Training Examples The two corpora are used tocreate training data for two separate experiments.For each dataset, we iterate over all train set utter-ances, generating one example for each of the slot-value pairs in the ontology. An example consistsof a transcription, its context (i.e. list of precedingsystem acts) and a candidate slot-value pair. Thebinary label for each example indicates whether ornot its utterance and context express the example’scandidate pair. For instance, ‘I would like Irish

1783

food’ would generate a positive example for candi-date pair FOOD=IRISH, and a negative example forevery other slot-value pair in the ontology.

Evaluation We focus on two key evaluation met-rics introduced in (Henderson et al., 2014a):

1. Goals (‘joint goal accuracy’): the proportionof dialogue turns where all the user’s searchgoal constraints were correctly identified;

2. Requests: similarly, the proportion of dia-logue turns where user’s requests for infor-mation were identified correctly.

5.2 Models

We evaluate two NBT model variants: NBT-DNNand NBT-CNN. To train the models, we use theAdam optimizer (Kingma and Ba, 2015) with cross-entropy loss, backpropagating through all the NBTsubcomponents while keeping the pre-trained wordvectors fixed (in order to allow the model to dealwith unseen words at test time). The model istrained separately for each slot. Due to the highclass bias (most of the constructed examples arenegative), we incorporate a fixed number of posi-tive examples in each mini-batch.4

Baseline Models For each of the two datasets,we compare the NBT models to:

1. A baseline system that implements a well-known competitive delexicalisation-basedmodel for that dataset. For DSTC2, the modelis that of Henderson et al. (2014c; 2014d).This model is an n-gram based neural net-work model with recurrent connections be-tween turns (but not inside utterances) whichreplaces occurrences of slot names and val-ues with generic delexicalised features. ForWOZ 2.0, we compare the NBT models to amore sophisticated belief tracking model pre-sented in (Wen et al., 2017). This model usesan RNN for belief state updates and a CNNfor turn-level feature extraction. Unlike NBT-CNN, their CNN operates not over vectors,

4Model hyperparameters were tuned on the respective val-idation sets. For both datasets, the initial Adam learning ratewas set to 0.001, and 1

8th of positive examples were included

in each mini-batch. The batch size did not affect performance:it was set to 256 in all experiments. Gradient clipping (to[−2.0, 2.0]) was used to handle exploding gradients. Dropout(Srivastava et al., 2014) was used for regularisation (with 50%dropout rate on all intermediate representations). Both NBTmodels were implemented in TensorFlow (Abadi et al., 2015).

but over delexicalised features akin to thoseused by Henderson et al. (2014c).

2. The same baseline model supplemented witha task-specific semantic dictionary (producedby the baseline system creators). The twodictionaries are available at mi.eng.cam.ac.uk/˜nm480/sem-dict.zip. TheDSTC2 dictionary contains only three rephras-ings. Nonetheless, the use of these rephras-ings translates to substantial gains in DST per-formance (see Sect. 6.1). We believe thisresult supports our claim that the vocabu-lary used by Mechanical Turkers in DSTC2was constrained by the system’s inability tocope with lexical variation and ASR noise.The WOZ dictionary includes 38 rephrasings,showing that the unconstrained language usedby Mechanical Turkers in the Wizard-of-Ozsetup requires more elaborate lexicons.

Both baseline models map exact matchesof ontology-defined intents (and their lexicon-specified rephrasings) to one-hot delexicalised n-gram features. This means that pre-trained vectorscannot be incorporated directly into these models.

6 Results

6.1 Belief Tracking PerformanceTable 1 shows the performance of NBT modelstrained and evaluated on DSTC2 and WOZ 2.0datasets. The NBT models outperformed the base-line models in terms of both joint goal and requestaccuracies. For goals, the gains are always statis-tically significant (paired t-test, p < 0.05). More-over, there was no statistically significant variationbetween the NBT and the lexicon-supplementedmodels, showing that the NBT can handle seman-tic relations which otherwise had to be explicitlyencoded in semantic dictionaries.

While the NBT performs well across the board,we can compare its performance on the two datasetsto understand its strengths. The improvement overthe baseline is greater on WOZ 2.0, which cor-roborates our intuition that the NBT’s ability tolearn linguistic variation is vital for this datasetcontaining longer sentences, richer vocabulary andno ASR errors. By comparison, the language ofthe subjects in the DSTC2 dataset is less rich, andcompensating for ASR errors is the main hurdle:given access to the DSTC2 test set transcriptions,the NBT models’ goal accuracy rises to 0.96. This

1784

DST Model DSTC2 WOZ 2.0Goals Requests Goals Requests

Delexicalisation-Based Model 69.1 95.7 70.8 87.1Delexicalisation-Based Model + Semantic Dictionary 72.9* 95.7 83.7* 87.6NEURAL BELIEF TRACKER: NBT-DNN 72.6* 96.4 84.4* 91.2*NEURAL BELIEF TRACKER: NBT-CNN 73.4* 96.5 84.2* 91.6*

Table 1: DSTC2 and WOZ 2.0 test set accuracies for: a) joint goals; and b) turn-level requests. Theasterisk indicates statistically significant improvement over the baseline trackers (paired t-test; p < 0.05).

indicates that future work should focus on betterASR compensation if the model is to be deployedin environments with challenging acoustics.

6.2 The Importance of Word Vector SpacesThe NBT models use the semantic relations em-bedded in the pre-trained word vectors to handlesemantic variation and produce high-quality inter-mediate representations. Table 2 shows the per-formance of NBT-CNN5 models making use ofthree different word vector collections: 1) ‘random’word vectors initialised using the XAVIER initiali-sation (Glorot and Bengio, 2010); 2) distributionalGloVe vectors (Pennington et al., 2014), trainedusing co-occurrence information in large textualcorpora; and 3) semantically specialised Paragram-SL999 vectors (Wieting et al., 2015), which are ob-tained by injecting semantic similarity constraintsfrom the Paraphrase Database (Ganitkevitch et al.,2013) into the distributional GloVe vectors in orderto improve their semantic content.

The results in Table 2 show that the use of seman-tically specialised word vectors leads to consider-able performance gains: Paragram-SL999 vectors(significantly) outperformed GloVe and XAVIER

vectors for goal tracking on both datasets. Thegains are particularly robust for noisy DSTC2data, where both collections of pre-trained vec-tors consistently outperformed random initialisa-tion. The gains are weaker for the noise-free WOZ2.0 dataset, which seems to be large (and clean)enough for the NBT model to learn task-specificrephrasings and compensate for the lack of seman-tic content in the word vectors. For this dataset,GloVe vectors do not improve over the randomlyinitialised ones. We believe this happens becausedistributional models keep related, yet antonymouswords close together (e.g. north and south, expen-sive and inexpensive), offsetting the useful seman-tic content embedded in this vector spaces.

5The NBT-DNN model showed the same trends. Forbrevity, Table 2 presents only the NBT-CNN figures.

Word Vectors DSTC2 WOZ 2.0Goals Requests Goals Requests

XAVIER (No Info.) 64.2 81.2 81.2 90.7GloVe 69.0* 96.4* 80.1 91.4

Paragram-SL999 73.4* 96.5* 84.2* 91.6

Table 2: DSTC2 and WOZ 2.0 test set performance(joint goals and requests) of the NBT-CNN modelmaking use of three different word vector collec-tions. The asterisk indicates statistically significantimprovement over the baseline XAVIER (random)word vectors (paired t-test; p < 0.05).

7 Conclusion

In this paper, we have proposed a novel neuralbelief tracking (NBT) framework designed to over-come current obstacles to deploying dialogue sys-tems in real-world dialogue domains. The NBTmodels offer the known advantages of couplingSpoken Language Understanding and DialogueState Tracking, without relying on hand-craftedsemantic lexicons to achieve state-of-the-art perfor-mance. Our evaluation demonstrated these benefits:the NBT models match the performance of modelswhich make use of such lexicons and vastly outper-form them when these are not available. Finally, wehave shown that the performance of NBT modelsimproves with the semantic quality of the under-lying word vectors. To the best of our knowledge,we are the first to move past intrinsic evaluationand show that semantic specialisation boosts per-formance in downstream tasks.

In future work, we intend to explore applicationsof the NBT for multi-domain dialogue systems, aswell as in languages other than English that requirehandling of complex morphological variation.

Acknowledgements

The authors would like to thank Ivan Vulic, UlrichPaquet, the Cambridge Dialogue Systems Groupand the anonymous ACL reviewers for their con-structive feedback and helpful discussions.

1785

ReferencesMartın Abadi, Ashish Agarwal, Paul Barham, Eugene

Brevdo, Zhifeng Chen, Craig Citro, Greg S. Cor-rado, Andy Davis, Jeffrey Dean, Matthieu Devin,Sanjay Ghemawat, Ian Goodfellow, Andrew Harp,Geoffrey Irving, Michael Isard, Yangqing Jia, RafalJozefowicz, Lukasz Kaiser, Manjunath Kudlur, JoshLevenberg, Dan Mane, Rajat Monga, Sherry Moore,Derek Murray, Chris Olah, Mike Schuster, JonathonShlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar,Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan,Fernanda Viegas, Oriol Vinyals, Pete Warden, Mar-tin Wattenberg, Martin Wicke, Yuan Yu, and Xiao-qiang Zheng. 2015. TensorFlow: Large-scale ma-chine learning on heterogeneous systems.

Dan Bohus and Alex Rudnicky. 2006. A “k hypothe-ses + other” belief updating model. In Proceedingsof the AAAI Workshop on Statistical and EmpiricalMethods in Spoken Dialogue Systems.

Asli Celikyilmaz and Dilek Hakkani-Tur. 2015. Con-volutional Neural Network Based Semantic Taggingwith Entity Embeddings. In Proceedings of NIPSWorkshop on Machine Learning for Spoken Lan-guage Understanding and Interaction.

Ronan Collobert, Jason Weston, Leon Bottou, MichaelKarlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011.Natural language processing (almost) from scratch.Journal of Machine Learning Research 12:2493–2537.

Franck Dernoncourt, Ji Young Lee, Trung H. Bui, andHung H. Bui. 2016. Robust dialog state tracking forlarge ontologies. In Proceedings of IWSDS.

Ondrej Dusek and Filip Jurcıcek. 2015. Training a Nat-ural Language Generator From Unaligned Data. InProceedings of ACL.

Juri Ganitkevitch, Benjamin Van Durme, and ChrisCallison-Burch. 2013. PPDB: The ParaphraseDatabase. In Proceedings of NAACL HLT .

Xavier Glorot and Yoshua Bengio. 2010. Understand-ing the difficulty of training deep feedforward neuralnetworks. In Proceedings of AISTATS.

Matthew Henderson, Milica Gasic, Blaise Thomson,Pirros Tsiakoulis, Kai Yu, and Steve Young. 2012.Discriminative Spoken Language Understanding Us-ing Word Confusion Networks. In Spoken LanguageTechnology Workshop, 2012. IEEE.

Matthew Henderson, Blaise Thomson, and Jason D.Wiliams. 2014a. The Second Dialog State TrackingChallenge. In Proceedings of SIGDIAL.

Matthew Henderson, Blaise Thomson, and Jason D.Wiliams. 2014b. The Third Dialog State TrackingChallenge. In Proceedings of IEEE SLT .

Matthew Henderson, Blaise Thomson, and SteveYoung. 2014c. Robust Dialog State Tracking using

Delexicalised Recurrent Neural Networks and Unsu-pervised Adaptation. In Proceedings of IEEE SLT .

Matthew Henderson, Blaise Thomson, and SteveYoung. 2014d. Word-Based Dialog State Trackingwith Recurrent Neural Networks. In Proceedings ofSIGDIAL.

Youngsoo Jang, Jiyeon Ham, Byung-Jun Lee, Young-jae Chang, and Kee-Eung Kim. 2016. Neural dialogstate tracker for large ontologies by attention mecha-nism. In Proceedings of IEEE SLT .

Nal Kalchbrenner, Edward Grefenstette, and Phil Blun-som. 2014. A Convolutional Neural Network forModelling Sentences. In Proceedings of ACL.

Yoon Kim. 2014. Convolutional Neural Networks forSentence Classification. In Proceedings of EMNLP.

Diederik P. Kingma and Jimmy Ba. 2015. Adam: AMethod for Stochastic Optimization. In Proceed-ings of ICLR.

Byung-Jun Lee and Kee-Eung Kim. 2016. Dialog His-tory Construction with Long-Short Term Memoryfor Robust Generative Dialog State Tracking. Dia-logue & Discourse 7(3):47–64.

Bing Liu and Ian Lane. 2016a. Attention-Based Recur-rent Neural Network Models for Joint Intent Detec-tion and Slot Filling. In Proceedings of Interspeech.

Bing Liu and Ian Lane. 2016b. Joint Online SpokenLanguage Understanding and Language Modelingwith Recurrent Neural Networks. In Proceedings ofSIGDIAL.

Fei Liu and Julien Perez. 2017. Gated End-to-EndMemory Networks. In Proceedings of EACL.

F. Mairesse, M. Gasic, F. Jurcicek, S. Keizer, B. Thom-son, K. Yu, and S. Young. 2009. Spoken LanguageUnderstanding from Unaligned Data using Discrim-inative Classification Models. In Proceedings ofICASSP.

Gregoire Mesnil, Yann Dauphin, Kaisheng Yao,Yoshua Bengio, Li Deng, Dilek Hakkani-Tur, Xi-aodong He, Larry Heck, Dong Yu, and GeoffreyZweig. 2015. Using recurrent neural networksfor slot filling in spoken language understanding.IEEE/ACM Transactions on Audio, Speech, and Lan-guage Processing 23(3):530–539.

Nikola Mrksic, Diarmuid O Seaghdha, Blaise Thom-son, Milica Gasic, Lina Rojas-Barahona, Pei-HaoSu, David Vandyke, Tsung-Hsien Wen, and SteveYoung. 2016. Counter-fitting Word Vectors to Lin-guistic Constraints. In Proceedings of HLT-NAACL.

Nikola Mrksic, Diarmuid O Seaghdha, Blaise Thom-son, Milica Gasic, Pei-Hao Su, David Vandyke,Tsung-Hsien Wen, and Steve Young. 2015. Multi-domain Dialog State Tracking using Recurrent Neu-ral Networks. In Proceedings of ACL.

1786

Vinod Nair and Geoffrey E. Hinton. 2010. Rectifiedlinear units improve restricted Boltzmann machines.In Proceedings of ICML.

Baolin Peng, Kaisheng Yao, Li Jing, and Kam-FaiWong. 2015. Recurrent Neural Networks with Ex-ternal Memory for Language Understanding. In Pro-ceedings of the National CCF Conference on Natu-ral Language Processing and Chinese Computing.

Jeffrey Pennington, Richard Socher, and ChristopherManning. 2014. Glove: Global Vectors for WordRepresentation. In Proceedings of EMNLP.

Julien Perez. 2016. Spectral decomposition method ofdialog state tracking via collective matrix factoriza-tion. Dialogue & Discourse 7(3):34–46.

Julien Perez and Fei Liu. 2017. Dialog state tracking, amachine reading approach using Memory Network.In Proceedings of EACL.

Christian Raymond and Giuseppe Ricardi. 2007. Gen-erative and discriminative algorithms for spokenlanguage understanding. In Proceedings of Inter-speech.

Tim Rocktaschel, Edward Grefenstette, Karl MoritzHermann, Tomas Kocisky, and Phil Blunsom. 2016.Reasoning about entailment with neural attention.In ICLR.

Iman Saleh, Shafiq Joty, Lluıs Marquez, AlessandroMoschitti, Preslav Nakov, Scott Cyphers, and JimGlass. 2014. A study of using syntactic and seman-tic structures for concept segmentation and labeling.In Proceedings of COLING.

Hongjie Shi, Takashi Ushio, Mitsuru Endo, KatsuyoshiYamagami, and Noriaki Horii. 2016. ConvolutionalNeural Networks for Multi-topic Dialog State Track-ing. In Proceedings of IWSDS.

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky,Ilya Sutskever, and Ruslan Salakhutdinov. 2014.Dropout: A Simple Way to Prevent Neural Networksfrom Overfitting. Journal of Machine Learning Re-search .

Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016a. Continuouslylearning neural dialogue management. In arXivpreprint: 1606.02689.

Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016b. On-line activereward learning for policy optimisation in spoken di-alogue systems. In Proceedings of ACL.

Kai Sun, Lu Chen, Su Zhu, and Kai Yu. 2014. TheSJTU System for Dialog State Tracking Challenge2. In Proceedings of SIGDIAL.

Kai Sun, Qizhe Xie, and Kai Yu. 2016. Recurrent Poly-nomial Network for Dialogue State Tracking. Dia-logue & Discourse 7(3):65–88.

Blaise Thomson and Steve Young. 2010. Bayesian up-date of dialogue state: A POMDP framework forspoken dialogue systems. Computer Speech andLanguage .

Gokhan Tur, Anoop Deoras, and Dilek Hakkani-Tur.2013. Semantic Parsing Using Word Confusion Net-works With Conditional Random Fields. In Proceed-ings of Interspeech.

Andreas Vlachos and Stephen Clark. 2014. A new cor-pus and imitation learning framework for context-dependent semantic parsing. TACL 2:547–559.

Miroslav Vodolan, Rudolf Kadlec, and Jan Kleindienst.2017. Hybrid Dialog State Tracker with ASR Fea-tures. In Proceedings of EACL.

Ngoc Thang Vu, Pankaj Gupta, Heike Adel, and Hin-rich Schutze. 2016. Bi-directional recurrent neuralnetwork with ranking loss for spoken language un-derstanding. In Proceedings of ICASSP.

Ivan Vulic, Nikola Mrksic, Roi Reichart, Diarmuid OSeaghdha, Steve Young, and Anna Korhonen. 2017.Morph-fitting: Fine-tuning word vector spaces withsimple language-specific rules. In Proceedings ofACL.

Wayne Wang. 1994. Extracting Information FromSpontaneous Speech. In Proceedings of Inter-speech.

Zhuoran Wang and Oliver Lemon. 2013. A Simple andGeneric Belief Tracking Mechanism for the DialogState Tracking Challenge: On the believability of ob-served information. In Proceedings of SIGDIAL.

Tsung-Hsien Wen, Milica Gasic, Dongho Kim, NikolaMrksic, Pei-Hao Su, David Vandyke, and SteveYoung. 2015a. Stochastic Language Generationin Dialogue using Recurrent Neural Networks withConvolutional Sentence Reranking. In Proceedingsof SIGDIAL.

Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic,Pei-Hao Su, David Vandyke, and Steve Young.2015b. Semantically Conditioned LSTM-based Nat-ural Language Generation for Spoken Dialogue Sys-tems. In Proceedings of EMNLP.

Tsung-Hsien Wen, David Vandyke, Nikola Mrksic,Milica Gasic, Lina M. Rojas-Barahona, Pei-Hao Su,Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialoguesystem. In Proceedings of EACL.

John Wieting, Mohit Bansal, Kevin Gimpel, and KarenLivescu. 2015. From paraphrase database to compo-sitional paraphrase model and back. TACL 3:345–358.

1787

Jason D. Williams. 2014. Web-style ranking and SLUcombination for dialog state tracking. In Proceed-ings of SIGDIAL.

Jason D. Williams, Antoine Raux, and Matthew Hen-derson. 2016. The Dialog State Tracking Challengeseries: A review. Dialogue & Discourse 7(3):4–33.

Jason D. Williams, Antoine Raux, Deepak Ramachan-dran, and Alan W. Black. 2013. The Dialogue StateTracking Challenge. In Proceedings of SIGDIAL.

Jason D. Williams and Steve Young. 2007. Partiallyobservable markov decision processes for spokendialog systems. Computer Speech and Language21:393–422.

Kaisheng Yao, Baolin Peng, Yu Zhang, Dong Yu, Ge-offrey Zweig, and Yangyang Shi. 2014. Spoken lan-guage understanding using long short-term memoryneural networks. In Proceedings of ASRU.

Steve Young, Milica Gasic, Simon Keizer, FrancoisMairesse, Jost Schatzmann, Blaise Thomson, andKai Yu. 2010. The hidden information state model:A practical framework for POMDP-based spoken di-alogue management. Computer Speech and Lan-guage 24:150–174.

Xiaodong Zhang and Houfeng Wang. 2016. A JointModel of Intent Determination and Slot Filling forSpoken Language Understanding. In Proceedingsof IJCAI.

Lukas Zilka and Filip Jurcicek. 2015. IncrementalLSTM-based dialog state tracker. In Proceedings ofASRU.

1788