Safe Visual Navigation via Deep Learning andNovelty Detection

Charles Richter and Nicholas RoyMassachusetts Institute of Technology

Cambridge, MA, USA

Abstract—Robots that use learned perceptual models in thereal world must be able to safely handle cases where they areforced to make decisions in scenarios that are unlike any oftheir training examples. However, state-of-the-art deep learningmethods are known to produce erratic or unsafe predictions whenfaced with novel inputs. Furthermore, recent ensemble, bootstrapand dropout methods for quantifying neural network uncertaintymay not efficiently provide accurate uncertainty estimates whenqueried with inputs that are very different from their trainingdata. Rather than unconditionally trusting the predictions of aneural network for unpredictable real-world data, we use anautoencoder to recognize when a query is novel, and revert toa safe prior behavior. With this capability, we can deploy anautonomous deep learning system in arbitrary environments,without concern for whether it has received the appropriatetraining. We demonstrate our method with a vision-guided robotthat can leverage its deep neural network to navigate 50% fasterthan a safe baseline policy in familiar types of environments,while reverting to the prior behavior in novel environments sothat it can safely collect additional training data and contin-ually improve. A video illustrating our approach is availableat: http://groups.csail.mit.edu/rrg/videos/safe visual navigation.

I. INTRODUCTION

In many ways, cameras are ideal sensors for mobile robotapplications since they provide rich contextual informationthat is required for many different types of navigation anddecision making problems. They are much smaller, lighter andcheaper than LIDAR, and work in a wide variety of indoorand outdoor lighting conditions, unlike most infrared RGBDsensors. However, cameras pose substantial computationalchallenges to extract information in usable forms. For example,the range of dense, accurate depth information from visualSLAM may be limited (like RGBD) to only a few meters.Therefore, it is reasonable to assume that mobile robots willbe able to build geometric maps from images, but that thosemaps will be very limited in range. Range limitations, in turn,restrict the speed at which a robot can travel, and impede itsability to anticipate more distant structures in the environment.

Nevertheless, camera images contain information about thecontext and structure of the world far beyond the limitedrange of accurate geometric inference. For instance, if a robotobserves a straight empty hallway ahead, it might reasonablylearn from experience and visual appearance that it can safelytravel at high speed down the hallway, even if it cannot inferthe exact geometry of the entire length of the hallway. If wecan extract this type of appearance-based information by othermeans, we can improve navigation performance.

Deep learning has been shown to be well suited to extractinguseful information for navigation from raw sensor data suchas camera images [13, 27]. However, unlike visual SLAMalgorithms that infer geometry using domain-invariant prin-ciples, neural networks adapt their representation to domain-specific training datasets. Therefore, they may make unreliablepredictions when queried with inputs far from the distributionof the training data [4]. Yet, we would still like to deployrobots using neural networks “in the wild,” where safety iscritical but real-world data will almost certainly differ fromthe training dataset, violating the common i.i.d. assumptionin machine learning. Moreover, conventional neural networksprovide no principled measure of how certain they are, or howwell-trained they are to make a given prediction.

Recognizing this limitation, some work has attempted toquantify uncertainty with Bayesian neural networks [20, 5],by training ensembles of networks on re-sampled versions ofthe training dataset [26, 17, 8, 18], or by using dropout totrain an implicit family of models [9]. While these methodsare equipped to model the variance within the training data,there has yet to emerge an established technique for reliablyproducing appropriate uncertainty estimates for regions ofinput space very far from the training data. Recent techniquesappear to be sensitive to particular choices of network archi-tecture and residual randomness from initialization [10, 25],and unlike Bayesian techniques such as Gaussian processes,these methods do not predictably converge to a prior meanand variance far from the training data. Furthermore, thesemethods require tens or hundreds of network evaluations toapproximate the output distribution, compromising runtimeefficiency. Given these limitations, we believe that determiningthe trustworthiness of a neural network prediction may bemore effectively addressed as a novelty-detection problem.

In this paper, we demonstrate safe, high-speed visual nav-igation of an autonomous mobile robot, guided by a neuralnetwork in environments that may or may not resemble thetraining environment(s). We use a conventional feedforwardneural network to predict collisions based on images observedby the robot, and we use an autoencoder to judge whetherthose images are similar enough to the training data forthe resulting neural network predictions to be trusted. Infact, Pomerleau [28] utilized a similar form of novelty detec-tion when using a neural network to control an autonomousvehicle, which we extend in several ways. First, if our noveltydetector classifies a planning scenario as novel, we revert to

a safe prior behavior, which enables our system to operateautonomously in any environment, regardless of whether ithas received appropriate training. Second, we continuouslycollect training data and label it in a self-supervised mannerso that we can periodically re-train our models to improve,and expand the set of familiar environment types, rather thanneeding human demonstrations or manually-designed visualsimulators for training in every environment type.

In the next section, we will motivate and present theproblem formulation of our robot navigation scenario, whichbuilds upon our prior work in Richter et al. [29]. Then, wewill describe our method of learning to predict collisionsfrom camera images, and our approach to novelty detection.Finally, we will present results including novelty detection andnavigation performance in several domains. Our simulationresults show that our learning method improves performancesignificantly, in multiple environment types, while remainingcollision-free in all trials throughout the learning process.Our experimental results demonstrate the effectiveness of ourapproach in a real navigation scenario on a high-speed RC car.

II. VISUAL NAVIGATION PROBLEM FORMULATION

We aim to solve the problem of navigation to a goal locationincurring minimum expected cost in an unknown map. Wedefine the cost of navigation to be equal to the time spenten route from start to goal, plus a fixed penalty on anycollisions that may occur along the way. We assume thatwe will be building a geometric map online using SLAMbecause it is helpful for practical reasons such as measuringprogress toward the goal. But, for the reasons discussed inSection I, we also assume that our ability to accurately inferdense environment geometry through our sensors is limitedto 5 m or less, which is characteristic of many monocular,stereo and RGBD methods. The Microsoft Kinect sensor, forinstance, has an advertised range of only 4 m [1]. Since a shortperceptual horizon limits the available stopping distance of therobot, the map itself is insufficient for high-speed navigation.As has been shown previously, using learned models forimage-based guidance can significantly extend the effectiveperceptual range and improve navigation performance [13, 34].

Let m denote the true, hidden, stationary map of theenvironment, and mt be a partial estimate of the map builtfrom a SLAM process. The map estimate mt is computedfrom the complete history of geometric sensor measurementsz0, . . . , zT up to time T . The geometric measurements zt maycome from a geometric sensor such as an RGBD camera orfrom processing stereo or monocular camera images (or evenfrom a simulator during training time). We refer to the rawcamera image at time t as it. We consider zt and it to beseparate measurements, although in certain types of visualSLAM systems, it could be processed to produce a geometricmeasurement zt.

Since the map is initially unknown, it is impossible tomeaningfully plan a detailed complete path from the startlocation to the goal. Therefore, we adopt a receding-horizonreplanning approach that selects a local action, at, of some

length, that avoids local obstacles that appear in the mapestimate m, avoids future collisions in expectation, and makesprogress toward the goal. The optimal action at time t ischosen according to the following optimization problem:

a∗t =argminat∈A

Ja(at) + Jc · p(c|mt, it, at) + h(at, mt) (1)

s.t. g(mt, at) = 0. (2)

In this control law, at ∈ A is a dynamically feasible action,Ja(at) returns the time duration of action at, and h(at, mt)is a heuristic function that estimates the remaining time tonavigate from the end of action at to the goal. The secondterm is an expected collision penalty in which p(c|mt, it, at)is the probability that a collision will inevitably occur beyondthe end of action at, given the current map estimate and image.

Finally, g(mt, at) is a collision checking function thatreturns 1 if action at intersects an observed obstacle in themap estimate mt, and returns 0 if the action lies entirely withinknown free space or unobserved space. Note that a conven-tional collision-checking routine would also classify actionsintersecting unknown regions of the map as being in collisionas well, in order to formally guarantee safety. However, weconsider potential collisions beyond the mapped regions ofmt to be the responsibility of the collision prediction term,not the constraint. This formulation will allow us to planactions whose stopping distance exceeds the available knownfree space in cases where we trust the collision prediction—atopic that we will describe in detail below.

As stated, this receding-horizon control formulation isstraightforward, with one exception: The collision probabilityp(c|mt, it, at) is an unknown quantity. In a conventional MDPor POMDP scenario, this probability distribution might bederived from the distribution over environments given in theproblem definition. However, since we do not know the distri-bution over the real-world environments in which we hope toplan, we will approximate the distribution p(c|mt, it, at) bylearning a function from data.

In this work, we employ a neural network to predictcollision probability. Therefore, we may be tempted to simplymodel the collision probability term in equation (1) as:

p(c|mt, it, at) = fc(c|it, at), (3)

where fc(c|it, at) is implemented as a neural network thattakes an image and action as input. Since we would like topredict collisions based on images, not geometric maps, wehave dropped the dependence on mt.

However, as we have noted, neural networks may make er-ratic predictions when queried with inputs unlike their trainingdistribution, so we must be able to detect when an input isnovel and revert to a default collision probability estimate thatis known to be safe, though perhaps conservative. To do so,we introduce a function fn(it), which returns 1 if image itis novel (i.e., unlike the distribution of training images), andreturns 0 otherwise. We then model the collision probability

distribution as follows:

p(c|mt, it, at) =

{fc(c|it, at) if fn(it) = 0

fp(c|mt, at) if fn(it) = 1.(4)

In this equation, the function fc(c|it, at) is a neural net-work trained to predict collisions, as in equation (3), andfp(c|mt, at) represents a prior estimate of collision probabilitybased on the geometric map estimate mt that can encouragesafe and sensible behavior when the robot is faced with anovel-appearing environment for which fc is untrained. Areasonable prior might be to limit speed such that the robot canstop within the known free space of map mt if need be, andsince our geometric sensing range is limited, this speed maybe rather slow. In completely novel-appearing environments,the robot will navigate according to the behavior suggested bythe prior fp, whereas in environments of familiar appearance,the robot will trust its learned collision prediction model fc.

In the next section, we will describe how to represent andtrain the function fc(c|it, at) as a feedforward neural networkand how to model the prior collision probability estimate,fp(c|mt, at). Then, we will describe the method we use forbuilding fn(it) to detect novel images.

III. LEARNING TO PREDICT COLLISIONS

As we have observed in Section I, there are several advan-tages to predicting collision from images based on trainingexperience, rather than relying solely on a geometric map.First, images contain information about the environment thatextends well beyond the range to which detailed geometry canbe reliably inferred. And second, a prediction based on trainingdata can account for characteristics of the environment that arenot explicitly visible in the image but are implied by the visualappearance, such as free space around a blind corner. Similarto our previous work in Richter et al. [29], our objective intraining a model to predict collision probabilities is to capturethese advantages implicitly through training examples. Weapproach this problem as a probabilistic binary classificationproblem: Given a camera image, and some choice of action,what is the probability that a future collision is inevitablebeyond the end of the chosen action? Therefore, we mustdevise a method for building a training dataset that provideslabeled examples of image-action pairs, with a binary responseindicating whether a collision was inevitable or not.

A. Labeling Data

To produce a labeled training dataset offline, we assumea collection of raw data of the following form: Draw ={m, (q0, i0), (q1, i1), (q2, i2), . . . , (qN , iN )}, where m is a(perhaps incomplete) geometric map of an environment as pro-duced by our SLAM system having traversed the environment,qi is a robot configuration within that map, ii is an image takenfrom configuration qi, and N is the number of data points.

Our strategy for labeling this dataset will be to cycle througheach image-configuration pair, randomly select an action aithat could have been taken from qi, and use the geometricmap m to determine whether that action would have resulted

(a) “Collision”. (b) “Collision”. (c) “Non-Collision”.

(d) Image for 1(a). (e) Image for 1(b). (f) Image for 1(c).

Fig. 1: Examples of the offline data labeling procedure, withthe associated image for each scenario.

in a collision at some point in the future. By associating iiand ai with the resulting label, the learned model will beable to predict (without access to the future measurements thatcontributed to map m) that next time it encounters a similarimage-action pair, a similar outcome will be likely.

We illustrate several example labeling scenarios in Figure 1.In Figure 1(a), the robot is in the middle of the hallway, andthe randomly selected action, illustrated in blue, steers towardthe right wall at low speed. To determine whether a collisionis inevitable after executing this action, we test a range ofpossible actions that would bring the robot to a stop undermaximum deceleration with a variety of steering strategies. InFigure 1(a), all of these so-called “emergency-stop” maneuversintersect the right wall, and are therefore colored red. Hence,there exists no feasible action that can avoid collision afterexecuting the blue action. Figure 1(d) illustrates the robot’sview from this configuration. This image, paired with the blueaction, would be assigned a label of “collision”.

Figure 1(b) illustrates another scenario in which the robotis approaching a turn at a high speed, with Figure 1(e)illustrating the view from this scenario. Again, all emergency-stop maneuvers are doomed to collide with the far wall,regardless of steering angle. Hence, in this case it is speedthat makes collision inevitable. However, Figures 1(c) and 1(f)illustrate a scenario in which the robot is approaching thecorner, this time more slowly and farther from the turn. In thiscase, an emergency-stop maneuver is feasible, and is illustratedin black. This scenario would be labeled “non-collision”.

The distribution of “collision” and “non-collision” labelsis intended to capture the relationship between the imageobserved by the robot, the geometry of the environment, andthe steering and braking capabilities of the robot with respectto that image and environment geometry. After training ona dataset constructed in this manner, our learned model ofcollision probability will be able to safely guide a robotthrough an environment like its training examples.

We also observe that we need not limit ourselves to a singleaction and label for a given image, and since real images arerelatively expensive to obtain, it is very beneficial to reuse each

image to produce numerous labeled data points. Therefore, wepair each image with 50 different actions and their associatedlabels, resulting in a 50x increase in the amount of availabledata for the collision predictor network to use for training.We assume that for every image in our dataset, we will havea sufficiently dense sampling of labeled image-action pairsassociated with that image to train our collision predictor.

B. Neural Network

We model collision probability using a fully connectedfeedforward network, with an image and action as input. Werepresent this input as a vector concatenating the grayscalepixel intensities of the image with the x, y-position of theend point of the action in the robot’s body-fixed coordinateframe, and the speed at the end of the action. Therefore, ournetwork input is: x = [i0, i1, i2, . . . , iK , ax, ay, av], whereik ∈ [0, 1] is the intensity of the kth pixel in image i, andax, ay , and av represent the terminal position and speed ofthe action respectively. While it is easy to provide a richeraction parameterization, we found no benefit from doing so.

We use three hidden layers (200 nodes per layer) withsigmoid activation functions, and a softmax output layer forprobabilistic classification of “collision” vs “non-collision”categories. We train this network using mini-batch stochasticgradient descent to minimize the negative log likelihood ofthe data labels given the network parameters. We do not usedropout or regularization. While this architecture functionedwell for this work, it will be useful to test larger networks andconvolutional architectures in future work.

C. Prior Estimate of Collision Probability

The prior is designed to encourage safety with a minimumof domain knowledge or model-specific information. Ourprior predicts that an action carries zero probability of futurecollision if there exists enough free space ahead of it to cometo a stop before intersecting an obstacle or unknown space.If there does not exist enough free space to come to a stop,we assign probability equal to zero for actions that reducespeed, and probability equal to one otherwise. While moresophisticated priors could be designed (or learned), this oneserves our purposes and illustrates that the performance of ourapproach does not hinge on carefully hand-crafted functions.

We define the function dfree(mt, at), which returns thelength of known free space in map estimate mt, along theray extending from some robot pose along action at, averagedover a number of equally-spaced poses along the action. Wealso define the function dstop(at), which returns the stoppingdistance required by the robot to come to a stop from thespeed at the end of action at. Finally, let v0(at) and vF (at)be the initial and final speeds along action at, respectively.With these functions, we define our prior as:

fp(c|mt, at) =0 if dfree(mt, at) > dstop(at)

0 if dfree(mt, at) < dstop(at) and v0(at) > vF (at)

1 if dfree(mt, at) < dstop(at) and v0(at) < vF (at).

(5)

IV. NOVELTY DETECTION

To detect novelty, we adopt an autoencoder-based approachproposed by Japkowicz et al. [15], and focus on camera imagesas the domain of interest, although this general approachis also applicable to a wide variety of complex domains,including acoustic signals [22], network server anomalies [33],data mining [14], document classification [21] and others.Classification of novel data amounts to a one-class classifi-cation problem since we assume that we only have accessto the set of images that have been observed and collectedby our system, and no examples of any other “unfamiliar”images. Therefore, we cannot pose novelty classification as aconventional binary classification problem.

To classify images as novel, we use a feedforward autoen-coder with three hidden sigmoid layers and a sigmoid outputlayer. The autoencoder is trained using mini-batch stochasticgradient descent to reconstruct images by minimizing the sumof squared pixel differences between the input and recon-structed output. Let ik ∈ [0, 1] denote the intensity value ofthe kth pixel of image i. The autoencoder loss function is:

Ln(i, i) =1

K

K∑k=1

(ik − ik)2, (6)

where i refers to the output of the autoencoder, i.e., thereconstructed image. The number of pixels in i and i is K.

This novelty detection approach assumes that the autoen-coder network will learn a compressed representation captur-ing the essential characteristics of its training dataset such thatit will reconstruct similar inputs with low reconstruction error.On the other hand, when the autoencoder is presented with anovel input—that is, an input unlike any of its training data—we expect the reconstruction error to be large. Therefore, toclassify novel images, we need only to set a threshold anddetermine whether the reconstruction error for a given imagefalls above or below that threshold.

Similar to Japkowicz et al. [15], we might consider settingthe threshold as some function of the largest reconstruction er-ror of any single training data point. However, since this valuewould depend only on a single outlier, we adopt a method thatwe have found to produce more robust and consistent results,which is to compute the empirical CDF of the distribution oflosses in the training dataset and select some high percentile,such as the 99th percentile, to be the threshold. To distinguishbetween two visually distinct categories of images, usually theautoencoder will separate their distributions of reconstructionerror clearly, so the exact value of the threshold is not critical.

A. MNIST Examples

To illustrate the basic function of the autoencoder novelty-detection method, we trained an autoencoder on the MNISTdigit recognition dataset using three hidden sigmoid layerswith 50 nodes per layer. Figures 2(a) and 2(b) show anexample test input drawn from the MNIST dataset, and itscorresponding output, respectively. The reconstruction errorfor this example is Ln = 6.9 × 10−3, which falls below

(a) Input. (b) Output. (c) Input. (d) Output.

Fig. 2: (a)-(b) Input and reconstruction (output) for an imageof a ‘5’ digit, drawn from the training dataset. (c)-(d) Inputand reconstruction for an image of a ‘J’ character, drawn fromthe notMNIST dataset, exhibiting high reconstruction error,indicating that this input image is considered novel.

3.5 4 4.5 5 5.5 6 6.5

Autoencoder Loss (Reconstruction Error)×10

-3

Density

Fig. 3: Autoencoder loss on a training dataset of simulatedhallway images with novelty threshold (vertical dashed line).

our computed threshold of Ln = 1.5 × 10−2. On the otherhand, Figures 2(c) and 2(d) illustrate an input drawn from thenotMNIST dataset [2] of letters from non-standard fonts, andits corresponding output, respectively. In this case, it is clearthat the reconstruction does not resemble the input, and that isreflected in the high reconstruction error of Ln = 3.2× 10−1,which is well above the threshold for novelty. Rather thanusing a discriminative prediction of class membership from aneural network on this input, we should recognize that we donot have the necessary training data and instead revert to aprior such as a uniform distribution over class labels.

B. Robot Navigation Examples

To evaluate this novelty detection approach on a visualnavigation domain, we trained an autoencoder on a set ofsimulated camera images from a hallway-type environmentand tested it on simulated images of hallways (the trainingdistribution) as well as forests of cylindrical obstacles (thenovel test distribution). Again, we used three hidden layerswith sigmoid activations, but since these images are morecomplex than MNIST, we increased the hidden layer size to200 nodes per layer. For computational efficiency, we use verylow resolution 60×44 images in this work, so the input andoutput layer have dimension 2640. Figure 3 shows a histogramof reconstruction errors for images in the hallway trainingdataset, along with the PDF of a normal distribution computedfrom the sample mean and variance of the reconstructionerrors. The vertical dashed line illustrates the threshold setat the 99th percentile of the CDF of this distribution.

Figure 4 illustrates two examples of novelty detection.Figure 4(a) illustrates an input image of a simulated hallwaydrawn from the distribution of images used to train theautoencoder and 4(b) shows the corresponding reconstruction

(a) Input. (b) Output. (c) Input. (d) Output.

Fig. 4: (a) A familiar input image of a simulated hallway,drawn from the training distribution, and (b) the associatedaccurate autoencoder reconstruction. (c) A novel input imageof a simulated “cylinder forest,” not drawn from the trainingdistribution, and (d) the resulting inaccurate reconstruction.

with a reconstruction error of Ln = 4.5×10−3, which is lowerthan the threshold of 6.3× 10−3, resulting in a classificationof “not novel”. On the other hand, Figure 4(c) illustratesan input image of a simulated forest of cylindrical obstaclesnot drawn from the training distribution and 4(d) shows thecorresponding reconstruction, which has a reconstruction errorof Ln = 4.4×10−2, which is much greater than the threshold,resulting in a classification of this image as “novel”. It isvisually apparent that the reconstruction in Figure 4(b) is muchmore accurate than that in Figure 4(d), indicating that the“cylinder-forest” image does not follow the same encodingthat the autoencoder learned for the hallway images.

C. Comments on Autoencoder-Based Novelty Detection

The feedforward autoencoder method works well for highlystructured data like MNIST digits and robot navigation envi-ronments, where we have many similar examples that lie ona fairly low-dimensional manifold of images. However, thismethod might not be expected to work well for highly varieddatasets of unique images like CIFAR-10. Such datasets mayrequire larger convolutional autoencoders, or other techniquesto capture the true structure of the data rather than learn-ing a trivial pixel-copy representation. As an alternative toautoencoder-based novelty detection, we might also considerkernel density estimation [4], or density estimation in a featurespace learned with a large CNN, as in the content-basedimage retrieval application proposed by Krizhevsky et al. [16].However, non-parametric methods would be very computa-tionally expensive since they would require querying a high-dimensional dataset for each prediction.

We also note that our autoencoder is not guaranteed tobecome familiar with images of a certain appearance at thesame rate as the collision predictor network. An image maybe classified as ”not novel” while the collision predictor is stillinsufficiently trained for that image. We have tried to preventthis type of failure by matching the size and architecture of thehidden layers of the two networks and have not experiencedthese types of failures. However, we could also merge the twonetworks so that they learn a single feature representation bothfor prediction and novelty classification [28].

While the autoencoder approach has been effective andefficient for our purposes, we believe that novelty detectionand uncertainty estimation for neural networks are still openproblems, and that other methods may be useful as well.

V. RESULTS

We tested our approach both in simulation and in experi-ments using a small autonomous RC car. In both cases, weinitialized the system with an empty dataset and a randomlyinitialized collision prediction network and autoencoder. Weinitialized the autoencoder threshold to zero so that all imageswould at first appear novel. We then repeatedly deployed therobot to drive to a specified goal location, collecting imagesduring each run, labeling them, and re-training the collisionpredictor and autoencoder networks with each addition of datato the growing dataset. Video of our results is available at: http://groups.csail.mit.edu/rrg/videos/safe visual navigation.

In all simulation trials, we used a 110◦ field-of-view cameramodel and a 5 m range for geometric sensing. In our experi-mental trials, we used two different sensor configurations. Inone configuration, we used monocular visual-inertial odometry(VIO) based on GTSAM for state estimation, and an XtionRGBD camera for depth estimation. In the other configuration,we used a Hokuyo UTM-30LX planar LIDAR to performscan-matching for state estimation, and for geometric percep-tion (artificially limited to a 110◦ field-of-view and 5 m range).In both experimental configurations, we used a Point GreyChameleon camera with a 110◦ horizontal field-of-view asinput to our collision-prediction and novelty-detection models.

While this work is targeted at camera-only perception, thedrift and error in VIO prevented us from achieving maximumperformance and reliability in the LIDAR-free configuration.Therefore, we used the LIDAR to test the merits of ourapproach under the idealized circumstances of highly accuratestate estimation. We emphasize that in all tests, regardless ofthe sensor used for state estimation and mapping, collisionprediction was performed only on camera images, combinedwith the action description, as described above.

A. Visualizing Neural Network Predictions

Figure 5 shows the predictions of collision probability onone real camera image from an experimental run and twosimulated images. In each case, the networks were trainedwith a large dataset of relevant images, so novelty is not afactor. The graph below each image illustrates the collisionprobability as a function of the steering direction of the action(i.e., atan2(ay, ax)), shown on the horizontal axis, and thespeed at the end of the action, indicated by color. Speeds varyfrom 1 m/s (blue) to 10 m/s (red), illustrating that faster actionscarry a greater probability of resulting in future collision.

Figure 5(a) illustrates a view roughly aligned with the axisof a real hallway, and therefore the learned model assignslow probability of collision to actions aligned with that axis.Figure 5(b) illustrates the robot in a right-hand turn in asimulated hallway, so the probability model assigns lowestprobability of collision to actions that bear 25◦ to the right.Finally, Figure 5(c) illustrates the robot in a simulated forestof obstacles. Since there is a long gap between obstacles downthe center of the image, the predicted collision probability islowest at this point, but rises sharply for turns to the right,due to an obstacle in the right foreground of the image.

-50-2502550Action Direction (degrees)

0

1

Pro

b.

Collisio

n

(a) Real hallway.

-50-2502550Action Direction (degrees)

0

1

Pro

b.

Collisio

n

(b) Simulated hallway.

-50-2502550Action Direction (degrees)

0

1

Pro

b.

Collisio

n

(c) Simulated forest.

Fig. 5: Illustration of predicted collision probabilities. Eachimage is illustrated with the learned collision probabilitymodel below it, with the horizontal axis representing actiondirection and color representing speed, from 1 m/s to 10 m/s.

B. Simulation Results

We tested our system in simulation with a run through eachof eight random hallway-type environments followed by eightrandom “cylinder-forest”-type environments, re-training aftereach run. Examples of both environment types are shownin Figure 6. Each run contributed about 1250 images tothe dataset, so that after 16 trials, the system had collected20000 images in total. For reporting performance statistics, weperformed 10 separate testing runs for each amount of trainingdata. All simulated training and testing trials were completedwithout collision, demonstrating the safety of our approach.

Figure 7 illustrates the performance of our simulated systemas a function of the number of training images observed. Thefirst eight data points (blue) represent hallway environments,while the second eight (red) represent “cylinder-forest” en-vironments. As the system gained more training data fromhallway-type environments, it began to trust the learned modelof collision probability on a greater fraction of observedimages, resulting in faster average speed of navigation. As thenovelty fraction decreased monotonically, the average speedof navigation increased. In total, the average speed increased42% from 4.90 m/s to 6.97 m/s in the hallway environments.

After completing trials in the hallway environments, wetested our system in “cylinder-forest” environments, carryingover the complete dataset and trained models from the finalhallway test. Initially, all images were again classified as novelsince the forest environment type is visually distinct from thehallway type. However, as our system accumulated imagesfrom the forest environments, the same pattern emerged:Monotonic decrease in novelty with corresponding increase inspeed. In forest environments, the speed increased 50% from5.84 m/s to 8.80 m/s. The reason forest navigation was fasterthan hallway navigation, both before and after training, is thatthere is on average more free space ahead of the robot in forestenvironments and no sharp turns are required.

C. Experimental Results using LIDAR

We performed a total of 80 experimental trials using a smallautonomous RC car in much the same way that we performedour simulation trials, except that instead of re-training after

(a) Simulated hallway map.

(b) Simulated forest map.

Fig. 6: Maps of hallway (top) and “cylinder-forest” (bottom)environments built by range-limited SLAM during navigation.

0 0.5 1 1.5 2

Number of Training Images ×104

0

0.5

1

FractionNovel

0 0.5 1 1.5 2

Number of Training Images ×104

5

6

7

8

9

Average

Speed(m

/s) Hallway

Forest

Fig. 7: Simulation results. Perceived novelty of images en-countered online, and average navigation speed, as a functionof the number of training images. Blue indicates hallway en-vironments and red indicates “cylinder-forest” environments.

each run, we instead re-trained after every 10 runs through theenvironment since the experimental environment was smallerand yielded fewer images per run. A generated map of theexperimental environment is shown in Figure 8(a) and animage from this environment is shown in Figure 10(a).

We observed essentially the same qualitative results in ourexperiments as we did in simulation. As the system collectedmore training images, the fraction of images considered noveldecreased, resulting in a corresponding increase in speed ofnavigation, as illustrated in Figure 9. We observed an averagespeed increase of 25%, from 3.38 m/s to 4.24 m/s. However,since the experimental environment was small compared to thesimulation environment, a substantial portion of the navigationtime was spent accelerating from the start and decelerating atthe goal. Therefore, we also provide the average maximumspeeds observed for each amount of training data, where therewas a much more substantial improvement of 55%, from 3.79m/s to 5.90 m/s. This improvement in maximum speed impliesthat if the experimental environment were larger, we wouldobserve a greater improvement in average speed as well.

Figure 11 illustrates the autoencoder reconstruction error forthree categories of data after training on nearly 9000 trainingimages from numerous runs through the experimental environ-ment depicted in Figure 8(a) and 10(a). The blue histogramand PDF indicates the reconstruction error of the trainingimages, from which the threshold (dashed line) was computed.The red histogram and PDF represents the reconstructionerror of approximately 1000 images collected during testingruns in the same environment, but not used for training theautoencoder. While the reconstruction error of these images is

(a) LIDAR-mapped hallway environment for training and testing.

(b) Xtion-mapped novel environment used for LIDAR-free testing.

Fig. 8: Maps of experimental environments.

0 2 4 6 8

Number of Training Images ×103

0

0.5

1

FractionNovel

0 2 4 6 8

Number of Training Images ×103

3

4

5

6

7

Speed(m

/s) Average

Maximum

Fig. 9: Experimental results for 80 experimental trials. Eachdata point represents the average over 10 trials with a givenmodel between re-training episodes.

not as low as the training images, most fall below the noveltythreshold. Finally, the black histogram and PDF illustrate thereconstruction error of a truly novel environment—a well-litlab area with obstacles (Figures 8(b) and 10(b)). Visually, thisenvironment is quite different from the training environment,and that is reflected in the much greater reconstruction error,with all images falling above the novelty threshold.

D. Experimental Results using Monocular VIO and RGBD

We performed experiments on our autonomous RC car usinga LIDAR-free sensor configuration in order to demonstratesteps toward a completely vision-based learning and naviga-tion pipeline. However, as noted previously, inaccuracies in themonocular VIO state estimate made this sensor configurationunsuitable for demonstrating the entire training and testingprocedure. Nevertheless, we were able to test the LIDAR-freesensor configuration in both environments depicted in Fig-ures 8 and 10 using the collision-prediction and autoencoder

(a) Familiar Environment. (b) Novel Environment.

Fig. 10: Images from (a) the hallway in which training tookplace, and (b) a novel environment not used in training.

0 0.01 0.02 0.03 0.04

Autoencoder Loss (Reconstruction Error)

Density

Training

Testing – Familiar Environment

Testing – Novel Environment

Threshold

Fig. 11: Histogram of autoencoder loss for experimental data.

models previously trained with the other sensor configuration.In the hallway training environment, we achieved a mean

speed of 3.26 m/s and a top speed over 5.03 m/s. Thisresult significantly exceeds the maximum speeds achievedwhen driving in this environment under the prior estimate ofcollision probability before performing any learning. On theother hand, in the novel environment, for which our model wasuntrained, the novelty detector correctly identified every imageas being unfamiliar. In the novel environment, we achieved amean speed of 2.49 m/s and a maximum speed of 3.17 m/s.

We emphasize that these results represent an improvementof 31% in mean speed and 59% in maximum speed afterlearning, for monocular vision-based navigation using a neuralnetwork. Naturally, the robot navigated more slowly in thenovel environment than it did in the familiar one, but ourmethod enables the robot to navigate in novel environmentssafely, potentially gathering data and re-training, whereas thatwould not be feasible without novelty-detection.

E. Autoencoder Validation

To characterize the performance of our autoencoder noveltydetection approach on a wider variety of real environments, wetested it on 16 visually distinct hallways at MIT. We collectedapproximately 5000 images from each hallway and trainedautoencoders on the images from between 1 and 13 differenthallways, while testing on three other hallways reserved fortesting. We then repeated this process for ten different permu-tations of which hallways were used for training and whichwere used for testing. The results are shown in Figure 12,with each colored line representing a particular permutation,and the black line indicating the average.

These results show that after training on only a singlehallway, the three test hallways were considered to be novel,indicating that a single hallway does not contain enoughvariation to capture the characteristics of hallways in general.However, after training on 13 hallways, our models classified90.6% of test hallway images as “not novel”, while correctlyclassifying 91.5% of test images from an outdoor courtyardand 94% of test images from a parking garage as “novel”.These results show that the autoencoder method of noveltydetection is able to meaningfully generalize to a family ofimages with common structure, rather than merely memorizingor overfitting to a particular training dataset. See Richter [30]for additional validation examples and discussion.

0 2 4 6 8 10 12

Number of Selected Training Hallways Included in Training Dataset

0

20

40

60

80

100

Perceived

Novelty

(%)of

Unseen

TestingHallways

Fig. 12: Perceived novelty of unseen testing hallways as afunction of the number of hallways used for training. Fourexample hallways are pictured in the upper right.

VI. RELATED WORK

There has been substantial work applying neural networklearning to image-based navigation. For instance, the DARPALAGR program featured long-range terrain classification usinga neural network, in an similar role to our collision predictionapproach [13]. Barnes et al. [3] and Oliveira et al. [24] providemore recent examples of deep learning used to segmentimage pixels into drivable routes, which can then be used forplanning. While we focus on supervised learning of collisionprobability, there are many other ways to map between imagesand actions, including end-to-end neural networks [6, 19],affordance-based representations [7], reinforcement learningusing either simulated or real images [23, 31], and classifi-cation of paths from manually collected data [11]. While itwould be impossible to survey the vast literature on visualnavigation here, we note that these examples and many othersdo not explicitly address uncertainty about the learned model.

In our prior work, we considered model uncertainty incollision prediction, but focused on features of geometricmaps, rather than images and neural networks [29]. Grimmettet al. [12] suggest that if a classification error is to be madein robotic decision making, it should be made with highreported uncertainty so that the robot can avoid consequencesof a wrong decision. Sofman et al. [32] use online noveltydetection in this way, to avoid scenarios for which their robotis untrained. In contrast, our use of novelty detection allowsthe robot to proceed safely under the prior, gather its owntraining data, and continually improve.

VII. CONCLUSION

In this work, we have demonstrated a safe, self-supervisedmethod of learning a visual collision prediction model online.We have shown that by using an autoencoder as a measureof uncertainty in our collision prediction network, we cantransition intelligently between the high performance of thelearned model and the safe, conservative performance of asimple prior, depending on whether the system has beentrained on the relevant data. In future work, it will be valuableto train and test on a much wider variety of environments,using large convolutional network architectures, to scale upour approach and examine the generalization of our models.

REFERENCES

[1] Microsoft kinect sensor. https://msdn.microsoft.com/en-us/library/hh438998.aspx. Accessed: 2016-12-29.

[2] notMNIST dataset. http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html. Accessed: 2017-01-26.

[3] D. Barnes, W. Maddern and I. Posner. Find your ownway: Weakly-supervised segmentation of path proposalsfor urban autonomy. arXiv:1610.01238, 2016.

[4] C. M. Bishop. Novelty detection and neural networkvalidation. IEE Proceedings - Vision, Image and SignalProcessing, 141(4):217–222, 1994.

[5] C. Blundell et al. Weight uncertainty in neural networks.arXiv:1505.05424, 2015.

[6] M. Bojarski et al. End to end learning for self-drivingcars. arXiv:1604.07316, 2016.

[7] C. Chen et al. DeepDriving: Learning affordance fordirect perception in autonomous driving. In Proc. Inter-national Conference on Computer Vision (ICCV), 2015.

[8] J. Franke and M. H. Neumann. Bootstrapping neuralnetworks. Neural Computation, 12(8):1929–1949, 2000.

[9] Y. Gal and Z. Ghahramani. Dropout as a Bayesianapproximation: Representing model uncertainty in deeplearning. In Proc. International Conference on MachineLearning (ICML), 2015.

[10] Y. Gal. Uncertainty in deep learning. Ph.D. Thesis,University of Cambridge, 2016.

[11] A. Giusti et al. A machine learning approach to visualperception of forest trails for mobile robots. IEEERobotics and Automation Letters, 1(2):661–667, 2016.

[12] H. Grimmett, R. Triebel, R. Paul and I. Posner. Intro-spective classification for robot perception. InternationalJournal of Robotics Research, 37(7):743–762, 2016.

[13] R. Hadsell et al. Learning long-range vision for au-tonomous off-road driving. Journal of Field Robotics,26(2):120–144, 2009.

[14] S. Hawkins et al. Outlier detection using replicator neuralnetworks. In Proc. International Conference on DataWarehousing and Knowledge Discovery, 2002.

[15] N. Japkowicz, C. Myers and M. Gluck. A novelty de-tection approach to classification. In Proc. InternationalJoint Conference on Artificial Intelligence (IJCAI), 1995.

[16] A. Krizhevsky, I. Sutskever and G. Hinton. ImageNetclassification with deep convolutional neural networks.In Advances in Neural Information Processing Systems(NIPS), 2012.

[17] A. Krogh and J. Vedelsby. Neural network ensembles,cross validation, and active learning. In Advances inNeural Information Processing Systems (NIPS), 1995.

[18] B. Lakshminarayanan, A. Pritzel and C. Blundell. Simpleand scalable predictive uncertainty estimation using deepensembles. arXiv:1612.01474, 2016.

[19] Y. LeCun et al. Off-road obstacle avoidance through

end-to-end learning. In Advances in Neural InformationProcessing Systems (NIPS), 2005.

[20] D. J. C. MacKay. Probable networks and plausiblepredictions—a review of practical Bayesian methods forsupervised neural networks. Network: Computation inNeural Systems, 6(3):469–505, 1995.

[21] L. Manevitz and M. Yousef. One-class document clas-sification via neural networks. Neurocomputing, 70(7-9):1466–1481, 2007.

[22] E. Marchi et al. A novel approach for automatic acousticnovelty detection using a denoising autoencoder withbidirectional LSTM neural networks. In Proc. IEEE In-ternational Conference on Acoustics, Speech and SignalProcessing (ICASSP), 2015.

[23] J. Michels, A. Saxena and A. Ng. High speed obstacleavoidance using monocular vision and reinforcementlearning. In Proc. International Conference on MachineLearning (ICML), 2005.

[24] G. L. Oliveira, W. Burgard and T. Brox. Efficientdeep models for monocular road segmentation. In Proc.IEEE International Conference on Intelligent Robots andSystems (IROS), 2016.

[25] I. Osband et al. Deep exploration via bootstrapped DQN.In Advances in Neural Information Processing Systems(NIPS), 2016.

[26] G. Paass. Assessing and improving neural networkpredictions by the bootstrap algorithm. In Advances inNeural Information Processing Systems (NIPS), 1993.

[27] D. A. Pomerleau. ALVINN: An autonomous land vehiclein a neural network. In Advances in Neural InformationProcessing Systems (NIPS), 1989.

[28] D. A. Pomerleau. Input reconstruction reliability esti-mation. In Advances in Neural Information ProcessingSystems (NIPS), 1993.

[29] C. Richter, W. Vega-Brown, and N. Roy. Bayesianlearning for safe high-speed navigation in unknown envi-ronments. In Proc. International Symposium on RoboticsResearch (ISRR), 2015.

[30] C. Richter. Autonomous navigation in unknown envi-ronments using machine learning. Ph.D. Thesis, Mas-sachusetts Institute of Technology, 2017.

[31] F. Sadeghi and S. Levine. (CAD)2RL: Real single-imageflight without a single real image. arXiv:1611.04201,2016.

[32] B. Sofman et al. Anytime online novelty and changedetection for mobile robots. Journal of Field Robotics,28(4):589–618, 2011.

[33] B. B. Thompson et al. Implicit learning in autoencodernovelty assessment. In Proc. International Joint Confer-ence on Neural Networks, 2002.

[34] S. Thrun et al. Stanley: The robot that won the DARPAgrand challenge. Journal of Field Robotics, 23(9):661-692, 2006.