The Future of Machine Learning

IDAL; Intelligent Data Analysis Laboratory Universitat de València

http://idal.uv.es

José D. Martín Guerrero

3

Future …Something new every single day …

4

Machine Learning"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its

performance at tasks in T, as measured by P, improves with experience E”

5

Machine learning project

PROBLEM(Analysis)

Data extraction Data process

Feature engineering MODELS Validation

All is connected!; feedback is always necessary for the success of the project

6

Feature EngineeringPROBLEM(Analysis)

Data extraction Data process

Feature engineering MODELS Validation

Curse of dimensionalityTypical problem in bioinformatics: O(103) features & O(10) samples

The correct use of inputs is key for a successful

ML application

7

Feature EngineeringFEATURES

FeatureSelection

Feature ExtractionManifolds

Models

We can select a subset (selection); transform (extraction) or "attack" the model directly (deep

learning).

8

ModelsPROBLEM(Analysis)

Data extraction Data process

Feature engineering MODELS Validation

No free lunch!

9

Models

Any machine learning model has a certain structure and we have to choose this (for example, the architecture of a neural

network).

First we have to choose the model that we will use in a given problem.

Parameters are obtained by search procedures usually controlled by other parameters we have to choose.

Parameters

Search Algorithm

StructureMODEL

10

Example: Deep LearningPromising models without feature engineering; apparently,

they perform pretty well but …

How many layers, how many neurons per layer,

which activation function?Inputs

Outputs

Hidden Layers

The most widely used algorithm is the backpropagation after initialization using RBM (Restricted Boltzmann Machines); what adaptation constant must one use?; if we use regularization, how do we weigh that factor?; if

we use dropout (to avoid overfitting), what % must we remove?; if we inject noise what is the best value for its energy?

Hectic tuning

11

Example: Azure ML

Many elections

12

Automatic Workflows Automatic Model Selection

Automatic Tuning Automatic Representation

Automatic Prediction Strategies

It would be very nice to have a formal apparatus that gives us some ‘optimal’ way of recognizing unusual phenomena and inventing new

classes of hypotheses that are most likely to contain the true one; but this remains an art for the creative human mind.” E. T. Jaynes 1985

Future: Automatic

13

Future: Automatic

15

Future: Automatic

http://www.automaticstatistician.com/

Future: Just around the corner … Reinforcement Learning

Supervised Learning Unsupervised Learning

Reinforcement Learning

•  It does not need a teacher to learn a desired signal

•  There is a goal (objective function) to be maximized

•  The outcome is a sequence of actions rather than a static model

•  It can deal with long-term objectives, not only a certain steps ahead in the future!!

•  Similar to some stages of human learning

Reinforcement Learning

EXPERIENCE INTERACTION

ARTIFICIAL LEARNING

MAXIMIZATION OF A CERTAIN OBJECTIVE FUNCTION

POLICY

A priori knowledge Environment adaptation

Reinforcement Learning

AGENT

ENVIRONMENT

at

st+1 (after action at)

rt+1

st (before action at)

Long term reward

Action-value function

Optimal policy

st: State (at time t) at: Action (at time t) rt+1: Immediate reward

Discount rate: a reward received k time steps in the future is worth only k−1 times what it would be worth if it were received immediately

Values of the discount rate close to 1 avoids the agent to be myopic (maximization of rt+1)

Reinforcement Learning: Applicability

- Traditionally, RL has been theoretically studied but until very recently, practical applications were restricted to well-known synthetic problems and/or Robotics.

!!- Any dynamic problem that can be defined in a state-

space, in which certain actions can be taken, and an objective function has to be maximized, is susceptible to be tackled using RL.

!!

- Some practical applications on Marketing or Medicine (individualization of campaigns or treatments). !

!!

Reinforcement Learning: An example (drug prescription)

States: evaluation of the state of the patient ! Actions: possible actions that can taken by doctors wrt to drug prescription ! Reward: the action involves a change in the state. Depending on this

resulting state, a reward can be assigned !

The aim is to maximize the long-term reward

It is possible to know the dosage (actions) that should be administered to maintain patients within a given state.

! Other factors can also be included in the

computation of the reward (e.g., expenses).

21

Conclusions

Two ways have been mentioned:!1. Automatic election of the parameters in a machine learning project 2. Reinforcement Learning

Predicting the future is too challenging to talk about it but it is so exciting that one must talk about it

There’s plenty of room to come up with new ideas … already present!

1. Validation in Bayesian nets 2. Quantum Machine Learning

The Future of Machine Learning

IDAL; Intelligent Data Analysis Laboratory Universitat de València

http://idal.uv.es

José D. Martín Guerrero