Anomaly Detectionbrief review of my prospectus

Ziba Rostamian

CS590 – Winter 2008

What I am planning to accomplish

Study Learning Finite Automaton. Focusing of CSSR algorithm.

Choose an application of desire and test the performance of the CSSR algorithm. (Once I implement the algorithm I can try it for different application and find out where it performs better).

Study CSSR and its extensions and use it for detecting anomaly of moving object.

Apply some modification in to the algorithm (it depends on how I proceed).

Why this is academically interesting Finite automaton inference has several "real

world" applications. Electrical engineering DFA’s have been proposed as a model of players. Model the problem of robot trying to learn its

envirounment. The application of PFAs (Probabilistic Finite

automaton), of which Hidden Markov Models (HMMs) are special case, are much more extensive. Speech recognition and handwriting recognition recognizing patterns in biological sequences such a

DNA and proteins

Anomaly Detection

What are anomalies/outliers? The set of data points that are considerably different

than the remainder of the data Variants of Anomaly/Outlier Detection Problems

Given a database D, find all the data points x D with anomaly scores greater than some threshold t

Given a database D, find all the data points x D having the top-n largest anomaly scores f(x)

Applications: Credit card fraud detection, telecommunication fraud

detection, network intrusion detection, fault detection

Importance of Anomaly DetectionOzone Depletion History In 1985 three researchers (Farman, Gardinar

and Shanklin) were puzzled by data gathered by the British Antarctic Survey showing that ozone levels for Antarctica had dropped 10% below normal levels

Why did the Nimbus 7 satellite, which had instruments aboard for recording ozone levels, not record similarly low ozone concentrations?

NASA discovered that the spring-time ''ozone hole'' had been covered up by a computer-program desinged to discard sudden, large drops in ozone concentrations as ''errors''.

Anomaly detection in moving object

Example: There are a large number of massive vessels sailing near

American coasts. It’s unrealistic to manually trace such a enormous number of moving objects and identify the suspicious ones. Therefore, it’s highly desirable to develop automated tools that can evaluate the behavior of all maritime vessels and flag the suspicious ones.

This will allow human agent to focus their monitoring more efficiently and accurantely.

Mechanisms for Anomaly detection

Classification, which relies on training data set. Normal Outliers

Clustering, which performs automated grouping without using training set.

Anticipated Challenges

Tracking moving object can generate an enormous amount of complex data. Example: the time and the location of a vessel might

be recorded every few seconds, and non-spatial information such a vessel’s weight, speed, shape and color may be included in this recording

There exists substantial complexities of possible abnormal behavior.