Post on 12-Feb-2017
Survey of feature recognition Survey of feature recognition techniquestechniques
Work package 5 Work package 5 Bradford University & Meudon Bradford University & Meudon
Observatory Observatory
V V Zharkova, S S Ipson V V Zharkova, S S Ipson
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Summary of the recognition techniquesSummary of the recognition techniques
MethodMethodFeatureFeature
Histogram Histogram methodmethod
LOGLOG Region Region GrowingGrowing
Simulated Simulated AnnealingAnnealing
Baysian Baysian InferenceInference
ANNANN Hough Hough TransformTransform
Valley Valley DetectionDetection
MALMMALM
SunspotsSunspots P/IP/I P/IP/I P/AP/A P/IP/I
FilamentsFilaments P/AP/A II P/IP/I
PlagePlage P/IP/I A, P/AA, P/A P/AP/A
CMEsCMEs P/IP/I P/AP/A P/IP/I
Emerging Emerging MagnFluxMagnFlux
P/AP/A
Coronal Coronal HolesHoles
WavesWaves
FlaresFlares
Bright Bright PointsPoints
P - P - pre-processingpre-processing,, I - I - user interaction was requireduser interaction was required andand A A - automated - automated methodmethod
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
2.1 Histogram-based segmentation2.1 Histogram-based segmentation 2.2. Region-based segmentation2.2. Region-based segmentation
– 2.2.1 Region growing2.2.1 Region growing– 2.2.2 Clustering2.2.2 Clustering– 2.2.3 Multi-resolution transforms2.2.3 Multi-resolution transforms
2.3 Edge-based segmentation2.3 Edge-based segmentation– 2.3.1 Gradient operator based edge detection2.3.1 Gradient operator based edge detection– 2.3.2 Canny edge detection2.3.2 Canny edge detection– 2.3.3 Laplacian of Gaussian zero-crossing edge 2.3.3 Laplacian of Gaussian zero-crossing edge
detectiondetection 2.4 Artificial neural networks2.4 Artificial neural networks
– 2.4.1 Standard technique2.4.1 Standard technique– 2.4.2 Cascade-correlation architecture2.4.2 Cascade-correlation architecture– 2.4.3 Evolving cascade neural networks2.4.3 Evolving cascade neural networks– 2.4.4 GMDH-type neural networks2.4.4 GMDH-type neural networks– 2.4.5 Generalized regression neural networks2.4.5 Generalized regression neural networks
2.5 Explicit-model based segmentation2.5 Explicit-model based segmentation– 2.5.1 The Hough transform2.5.1 The Hough transform– 2.5.2 Ribbon detection2.5.2 Ribbon detection
2.6 Models based on functionals2.6 Models based on functionals– 2.6.1 Active contours2.6.1 Active contours
2.7 Bayesian inference2.7 Bayesian inference 2.8 Motion segmentation2.8 Motion segmentation 2.9 Shape analysis2.9 Shape analysis 2.10 Classification2.10 Classification
II. Survey of Pattern Recognition TechniquesII. Survey of Pattern Recognition Techniques3.1 3.1 Image preparationImage preparation
3.1.1 Geometrical distortion3.1.1 Geometrical distortion3.1.2 Blurring3.1.2 Blurring3.1.3 Intensity calibration3.1.3 Intensity calibration3.1.4 Miscellaneous defects3.1.4 Miscellaneous defects
3.2 Detection of sunspots3.2 Detection of sunspots3.2.1 Histogram methods3.2.1 Histogram methods3.2.2 LOG methods3.2.2 LOG methods3.2.3 Region growing methods3.2.3 Region growing methods3.2.4 Simulated annealing3.2.4 Simulated annealing
3.3 Filament detection3.3 Filament detection3.3.1 Chain linking procedure3.3.1 Chain linking procedure3.3.2 Region growing procedure3.3.2 Region growing procedure
3.4 Detection of active regions (plage)3.4 Detection of active regions (plage)3.4.1 Global intensity threshold 3.4.1 Global intensity threshold 3.4.2 Region growing methods3.4.2 Region growing methods3.4.3 Bayesian inference method3.4.3 Bayesian inference method
3.5 Detection of coronal mass ejections3.5 Detection of coronal mass ejections3.5.1 Hough transform method3.5.1 Hough transform method3.5.2 Multiple abstraction level 3.5.2 Multiple abstraction level
mining mining methodmethod
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
III.III.a. Why the pre-processing techniques?a. Why the pre-processing techniques?
Difficulties with images:Difficulties with images: ErrorsErrors in FITS header information in FITS header information
Image Image shapeshape (ellipse), (ellipse), centrecentre and and
the the pole pole coordinatescoordinates
Weather Weather transparencytransparency (clouds) (clouds)
and different and different thickness thickness of of
atmosphereatmosphere
Centre-to-limb Centre-to-limb darkeningdarkening
DefectsDefects in data (strips, lines, in data (strips, lines,
intensity)intensity)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
SUNSPOTSSUNSPOTSSynoptic ChartsSynoptic Charts
Central Meridian Synoptic Map
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Image segmentation proceduresImage segmentation procedures
Thresholding approaches (histogram-based segmentation)Thresholding approaches (histogram-based segmentation)
Edge-based methods (using the first or second derivatives of the spatio-Edge-based methods (using the first or second derivatives of the spatio-temporal functionstemporal functions
Region growing methods (intitial starting pixel + criterion for merging)Region growing methods (intitial starting pixel + criterion for merging)
Hybrid region growing and edge detection techniquesHybrid region growing and edge detection techniques
Neural networks (training without explicit criteria)Neural networks (training without explicit criteria)
Global Information methods (Bayesian, functional models, Hough Global Information methods (Bayesian, functional models, Hough transform)transform)
Miscellaneous (data clustering, simulated annealing, data mining)Miscellaneous (data clustering, simulated annealing, data mining)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
General techniquesGeneral techniques Histogram-based segmentation –Histogram-based segmentation –
– Analyse the grey-level histogramsAnalyse the grey-level histograms– Size of the segmented object varies with the thresholdSize of the segmented object varies with the threshold– Give good results on a uniform backgroundGive good results on a uniform background– Objects had a distinct intensity rangeObjects had a distinct intensity range
Region-based segmentation Region-based segmentation – Region growing (start from seeds and grow regions on specified criteria)Region growing (start from seeds and grow regions on specified criteria)– Clustering (pixels are clustered in a feature space using any discriminating Clustering (pixels are clustered in a feature space using any discriminating
feature asociated and then connecting regions are found)feature asociated and then connecting regions are found)
Edge-based segmentation Edge-based segmentation – Relies on discontinuities in the image data to locate boundariesRelies on discontinuities in the image data to locate boundaries– But edge profile is not knownBut edge profile is not known– Profile can vary with edge (shading or texture)Profile can vary with edge (shading or texture)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Edge-based segmentationEdge-based segmentation Gradient operator based edge detection –Gradient operator based edge detection –
– Vertical and horizontal components are finite difference formulae withVertical and horizontal components are finite difference formulae with– Sobel convolution masks: vertical and horizontalSobel convolution masks: vertical and horizontal
-1 -2 -1 -1 0 1-1 -2 -1 -1 0 1 0 0 0 -2 0 20 0 0 -2 0 2 1 2 1 -1 0 11 2 1 -1 0 1– Gradient magnitude - a square root of the sum of the square gradient componentsGradient magnitude - a square root of the sum of the square gradient components– Candidate edge located with gradient magnitude above thresholdCandidate edge located with gradient magnitude above threshold– Multi passes of the detected edgeMulti passes of the detected edge
Canny edge detection Canny edge detection – Smooth image with a Gaussian filterSmooth image with a Gaussian filter– Compute gradient magnitude and orientation with finite differencesCompute gradient magnitude and orientation with finite differences– Apply non-maxima suppression to thin the gradient-magnitude edge imageApply non-maxima suppression to thin the gradient-magnitude edge image– Track along edges starting from the point esceeding higher threshold with the edge point Track along edges starting from the point esceeding higher threshold with the edge point
esceeding the lower thresholdesceeding the lower threshold– Apply edge linking to fill small gapsApply edge linking to fill small gaps
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Edge-based segmentationEdge-based segmentation
Laplacian of Gaussian zero-crossing edge detection (LOG)Laplacian of Gaussian zero-crossing edge detection (LOG)
– The Laplacian - 2D isotropic measure of the second spatial derivative of an imageThe Laplacian - 2D isotropic measure of the second spatial derivative of an image– L of an image has the lagest magnitudes at peaks of intensityL of an image has the lagest magnitudes at peaks of intensity– L of an image has zero crossings at the points of inflectionL of an image has zero crossings at the points of inflection– Common convolution kernels to calculate digital Laplacian:Common convolution kernels to calculate digital Laplacian:
0 1 0 1 1 10 1 0 1 1 1 1 -4 1 1 -8 11 -4 1 1 -8 1 0 1 0 1 1 10 1 0 1 1 1
– L sensitive to noise => applied after a Gaussian smoothing filterL sensitive to noise => applied after a Gaussian smoothing filter– Hence => LOG or Marr-Hildreth operatorHence => LOG or Marr-Hildreth operator
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Explicit model-based segmentationExplicit model-based segmentation
The Hough transform (CMEs – Bergmans)The Hough transform (CMEs – Bergmans)
– Uses an accumulator array with dimension equal the number of parameters in the Uses an accumulator array with dimension equal the number of parameters in the family of curves to be detectedfamily of curves to be detected
If If y y = = ax + b, then a and b and accumulator array indices (2) correspondax + b, then a and b and accumulator array indices (2) correspond Accumulator arrayAccumulator array
Ribbon detectionRibbon detection
– Modified Hough transform which includes a directions of the intensity gradientModified Hough transform which includes a directions of the intensity gradient across the line or curveacross the line or curve
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Miscelleneous methodsMiscelleneous methods
Image cleaning (solar: shape and intensity)Image cleaning (solar: shape and intensity) Image filteringImage filtering Image enhancement (to increase a contrast)Image enhancement (to increase a contrast) Morphological operations (to complete the Morphological operations (to complete the
feature shape)feature shape) Others (reported by other speakers)Others (reported by other speakers)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Artificial Neural NetworksArtificial Neural Networks
Standard techniqueStandard technique
– Exploits a feed-forward fully connected network: input, hidden or output neuronsExploits a feed-forward fully connected network: input, hidden or output neurons connected by adjustable synaptic weightsconnected by adjustable synaptic weights– The technique implies that ANN structure is well definedThe technique implies that ANN structure is well defined– It means that one must preset the input and hidden neuronsIt means that one must preset the input and hidden neurons– Apply suitable neuron activation functionApply suitable neuron activation function– Sigmoid activation function:Sigmoid activation function:
y y = = ff((x, wx, w) = 1/(1 + exp(– w0 – ) = 1/(1 + exp(– w0 – ΣΣiimm wi x wi xi)), i)),
where m – number of variables xwhere m – number of variables x11, xm, X is the input vector, , xm, X is the input vector, w w is a synaptic weigh is a synaptic weigh
vectorvector– User must choose a suitable learning algorithmUser must choose a suitable learning algorithm– Rationally set learning rate, a number of the training epochs etc.Rationally set learning rate, a number of the training epochs etc.– If ANN includes 2 hidden neurons -> back-projection algorithm provides best resultsIf ANN includes 2 hidden neurons -> back-projection algorithm provides best results
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Filament recognition with ANNFilament recognition with ANN
…
f1
f2
f3 z1
(j) z2
(j)
zr(j)
(j)
w0(2)
w0(1)
w0(3)
uj
sj
yj = (0, 1)
1(b)
0 5000 100000
500
1000
1500
2000
Y0 5000 10000
-4
-2
0
2
4
F
1(a)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
Recognised filamentsRecognised filaments
1(a) 1(b)
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
MethodMethodFeatureFeature
Histogram Histogram methometho
dd
LOGLOG Region Region GroGrowingwing
SimulateSimulated d
AnnAnnealinealin
gg
Baysian Baysian InferInferenceence
ANNANN Hough Hough TrTranansfosformrm
Valley Valley DeteDetectionction
MALMMALM
SunspotsSunspots P/IP/I P/IP/I P/AP/A P/IP/I
FilamentsFilaments P/AP/A II P/IP/I
PlagePlage P/IP/I A, P/AA, P/A P/AP/A
CMEsCMEs P/IP/I P/AP/A P/IP/I
Magnetic Magnetic fieldfield
P/AP/A P/IP/I
Summary of the Solar Feature Recognition Methods
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
VII. ConclusionsVII. Conclusions WP5 is successfully implementing the project planWP5 is successfully implementing the project plan
Feature recognition in solar images generated a substantial Feature recognition in solar images generated a substantial interest among the IT and solar community -FR Workshopinterest among the IT and solar community -FR Workshop
A few novel techniques were developed for each feature A few novel techniques were developed for each feature (see sunspots, ARs, filaments (ANN + MO), magnetic NL)(see sunspots, ARs, filaments (ANN + MO), magnetic NL)
Ongoing collaboration with the partners from Meudon, Ongoing collaboration with the partners from Meudon, NSO, UAS, IAS and OATONSO, UAS, IAS and OATO
The current status – a detailed catalogue design stage The current status – a detailed catalogue design stage
SFR Workshop 1, BRO, Brussels, 23-SFR Workshop 1, BRO, Brussels, 23-24 Oct '0324 Oct '03
WP5 –Feature RecognitionWP5 –Feature RecognitionWork in progressWork in progress
Adjustment of the FR techniques to the specifics of each Adjustment of the FR techniques to the specifics of each catalogue with respect to the time coverage period and catalogue with respect to the time coverage period and providers for the Unified Observing Catalogues (UOC)providers for the Unified Observing Catalogues (UOC)
Created an Access database fed by the detected sunspot Created an Access database fed by the detected sunspot feature parameters and developed a preliminary query and feature parameters and developed a preliminary query and response pagesresponse pages
Preparing a Demo on the Web for your testing Preparing a Demo on the Web for your testing
http://www.cyber.brad.ac.uk/egso/http://www.cyber.brad.ac.uk/egso/