Three methods developed for these objectives Based on machine learning and supervised learning Under...

6
• Three methods developed for these objectives • Based on machine learning and supervised learning • Under the evolutionary paradigm – specifically Genetic Programming (GP) Objectives: 1. Improve precision and recall 2. Classify minority class in extreme imbalanced datasets 2. Produce a range of rules to suit user's preferences 3. Generate comprehensible solutions for user interaction. Post-processing of Decision Trees Alma Carcia Almanza Edward Tsang

Transcript of Three methods developed for these objectives Based on machine learning and supervised learning Under...

Page 1: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

• Three methods developed for these objectives• Based on machine learning and supervised learning• Under the evolutionary paradigm

– specifically Genetic Programming (GP)

Objectives:1. Improve precision and recall2. Classify minority class in extreme imbalanced datasets2. Produce a range of rules to suit user's preferences3. Generate comprehensible solutions for user interaction.

Post-processing of Decision Trees

Alma Carcia Almanza

Edward Tsang

Page 2: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

New Classification Methods

Scenario Method (Pruning

)

Repository Method

Rα…Rµ

R’k

(Rules Collection)

Evolving Decision Rules

Repository Method + Evolution=

Page 3: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

Methods overviewRepository Method

(RM)1. Classify minority class

in imbalanced data sets2. Produce a range of

classifications to suit the user's preferences

3. Provide understandable rules

Our aim is to extract and collect different patterns that classify the positive cases (rare instances) in different ways.

Evolving decision rules

(EDR)

EDR evolves a population of decision trees to form a repository of rules. The resulting rules are used to create a range of classifications

Scenario Method

(SM)1. Analyze decision tree to

detect and remove the rules that do not contribute to the classification task.

Scenario Method (SM) is a pruning procedure for decision trees created by GP. This pruning is based on the analysis of patterns in the decision tree.

Page 4: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

Repository Method

In order to mine the knowledge acquired by the evolutionary process Repository Method performs the following steps:

Evolve a GP to create a population of decision trees

R1

R2…

Rn

The rule Rk is selected by precision; Rk is simplified to R’k

1- Rule extraction 2- Rule simplification

R’k is compared to the rules in the repository by similarity (genotype)

Rα…Rµ

3- New rule detection

4- Add rule to the repositoryIf R’k is a new rule R’k is added to the rule repository

R’k

RM Presentation

Page 5: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

Evolving Decision Trees

EDR Presentation

InitialPopulation generated at random

R1

R2

…R3

. . .

Every Rule

Rk

whose precision achieve a predefined precision threshold

is simplified to remove redundant

and vacuous

conditions

Every decision tree is divided

in rules (patterns)

R’k

Is compared

to the rules in a rule

collection

RaRb…Rn

if R’k is a new rule it is added

to the rule set

R’k

generate a new

population of decision

tree by mutating and hill-climbing on the rules in

the repository

and generating

trees at random . . .

The new population is processed until the maximum number of iterations is reached

Page 6: Three methods developed for these objectives Based on machine learning and supervised learning Under the evolutionary paradigm –specifically Genetic Programming.

Scenario Method

R1= { 2, 7, 10}R2= { 2, 14, 18}R3= { 2, 14, 21}

IF 0

AND 1

> 2

Var2 3 AND 6Var3 4

OR 5

AND 13

> 7

Var2 8 .43 9

> 10

Var3 11 .67 12

> 14

Var2 15 Var5 16

OR 17

> 18

Var7 19 .68 20

> 21

Var9 22

> 24 OR 25The tree is composed by the following rules

Where the numbers represent the node of the condition

Evaluate every decision rule

Consider that the R2 is not contributing to the classification task. Thus, we analyze every condition in R2 to determine which conditions are involved in other rules

The only condition that is not involved in the other rules is 18

Remove the condition

Procedure

R1= { 2, 7, 10}R2= { 2, 14, 21}

Var3 23

SM Presentation


Unsupervised Learning. Supervised learning vs. unsupervised learning.

Unsupervised Learning. Supervised learning vs. unsupervised learning.

Transfer Learning & Semi-supervised Learning

Transfer Learning & Semi-supervised Learning

Supervised Learning - wnzhang.netwnzhang.net/teaching/ee448/slides/5-supervised-learning-2.pdf · Content of Supervised Learning •Introduction to Machine Learning •Linear Models

Supervised Learning - wnzhang.netwnzhang.net/teaching/ee448/slides/5-supervised-learning-2.pdf · Content of Supervised Learning •Introduction to Machine Learning •Linear Models

Bits of Machine Learning Part 1: Supervised Learning - …wasp-sweden.org/custom/...proutiere-supervised-learning-20161028.pdf · Bits of Machine Learning Part 1: Supervised Learning

Bits of Machine Learning Part 1: Supervised Learning - …wasp-sweden.org/custom/...proutiere-supervised-learning-20161028.pdf · Bits of Machine Learning Part 1: Supervised Learning

Basic Technical Concepts in Machine Learning Introduction Supervised learning Problems in supervised learning Bayesian decision theory.

Basic Technical Concepts in Machine Learning Introduction Supervised learning Problems in supervised learning Bayesian decision theory.

Supervised Machine Learning: Learning SVMs and Deep Learninghelper.ipam.ucla.edu › publications › mpstut › mpstut_13971.pdf · 2016-09-13 · Supervised Machine Learning: Learning

Supervised Machine Learning: Learning SVMs and Deep Learninghelper.ipam.ucla.edu › publications › mpstut › mpstut_13971.pdf · 2016-09-13 · Supervised Machine Learning: Learning

Lecture 1: Supervised Learning - ISyE Home | ISyE ...tzhao80/Lectures/Lecture_1.pdf · Lecture 1: Supervised Learning ... CS229 Lecture notes Andrew Ng Supervised learning LetÕs

Lecture 1: Supervised Learning - ISyE Home | ISyE ...tzhao80/Lectures/Lecture_1.pdf · Lecture 1: Supervised Learning ... CS229 Lecture notes Andrew Ng Supervised learning LetÕs

Nonparametric Supervised Learning - cda.psych.uiuc.educda.psych.uiuc.edu/multivariate_fall_2013/matlab_help/...learning.pdf · Supervised Learning (Machine Learning) Workflow and

Nonparametric Supervised Learning - cda.psych.uiuc.educda.psych.uiuc.edu/multivariate_fall_2013/matlab_help/...learning.pdf · Supervised Learning (Machine Learning) Workflow and

Chapter 5: Partially-Supervised Learning. CS583, Bing Liu, UIC 2 Outline Fully supervised learning (traditional classification) Partially (semi-) supervised.

Chapter 5: Partially-Supervised Learning. CS583, Bing Liu, UIC 2 Outline Fully supervised learning (traditional classification) Partially (semi-) supervised.

Graph-BasedSemi-Supervised Learningsemi-supervised learning, graph-based semi-supervised learning, manifold learn- ing, graph-based learning, transductive learning, inductive learning,

Graph-BasedSemi-Supervised Learningsemi-supervised learning, graph-based semi-supervised learning, manifold learn- ing, graph-based learning, transductive learning, inductive learning,

1 Machine Learning Unsupervised Learning. 2 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate.

1 Machine Learning Unsupervised Learning. 2 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate.

Semi-Supervised Learning for Optical Flow with Generative … › paper › 6639-semi-supervised-learning-for-opti… · Semi-Supervised Learning for Optical Flow with Generative

Semi-Supervised Learning for Optical Flow with Generative … › paper › 6639-semi-supervised-learning-for-opti… · Semi-Supervised Learning for Optical Flow with Generative

Supervised tensor learning

Supervised tensor learning

Chapter 3: Supervised Learning · 2019. 11. 24. · Supervised vs. unsupervised Learning •Supervised learning: classification is seen as supervised learning from examples. •Supervision:

Chapter 3: Supervised Learning · 2019. 11. 24. · Supervised vs. unsupervised Learning •Supervised learning: classification is seen as supervised learning from examples. •Supervision:

2. Supervised Learning

2. Supervised Learning

Machine Learning Basics: Supervised Learning Algorithmssrihari/CSE676/5.7 MLBasics-Supervised.pdfDeep Learning Srihari 1 Machine Learning Basics: Supervised Learning Algorithms Sargur

Machine Learning Basics: Supervised Learning Algorithmssrihari/CSE676/5.7 MLBasics-Supervised.pdfDeep Learning Srihari 1 Machine Learning Basics: Supervised Learning Algorithms Sargur

Bengio Machine learning semi-supervised learning

Bengio Machine learning semi-supervised learning

Poorly Supervised Learning - statisticalcyber.comstatisticalcyber.com/talks/anagnostopoulos.pdf · Poorly Supervised Learning ... Semi-supervised learning and co-training Expectation-Maximisation:

Poorly Supervised Learning - statisticalcyber.comstatisticalcyber.com/talks/anagnostopoulos.pdf · Poorly Supervised Learning ... Semi-supervised learning and co-training Expectation-Maximisation:

Local Structure Optimization in Evolutionary Generated Neural … · 3.3.1 Growing Cell Structures and Unsupervised Learning 19 3.3.2 Growing Cell Structures and Supervised Learning

Local Structure Optimization in Evolutionary Generated Neural … · 3.3.1 Growing Cell Structures and Unsupervised Learning 19 3.3.2 Growing Cell Structures and Supervised Learning

Semi-Supervised Learningzhuxj/tmp/book.pdf1.1.2 Semi-Supervised Learning Semi-supervised learning (SSL) is half way between supervised and unsupervised learning. In addition to unlabeled

Semi-Supervised Learningzhuxj/tmp/book.pdf1.1.2 Semi-Supervised Learning Semi-supervised learning (SSL) is half way between supervised and unsupervised learning. In addition to unlabeled