Integrating Small Data, Synthetic Data in AI Strategy for ... · © 2019 ELSE Corp S.r.l. !3...

© 2019 ELSE Corp S.r.l.

Integrating Small Data, Synthetic Data in AI Strategy for Fashion Retail

[email protected]

Co-Founder & CEO ELSE Corp, a Virtual Retail

company

Andrey Golub

© 2019 ELSE Corp S.r.l. !2

BIG DATA X

FASHION

Today’s Technology Is Shaping Retail’s Future: Predicting fashion trends using machine learning

http://www.digitalistmag.com/customer-experience/2018/01/03/retail-tomorrow-how-todays-technology-is-shaping-retails-future-05682564


Marketers Need Artificial Intelligence to Reach the Segment of One

SMALL DATA X

FASHION

Big Data Economy & Challenges


(Big) Data Annotation Economy

The global market size is expected to reach US$ 1.6 billion by 2025.

Grand View Research

As companies are increasingly embracing artificial intelligence, a growing industry is emerging in which workers are employed to 'train' algorithms to recognize various types of data.



“The largest technology companies don’t want to be in the business of training data, they want to own customer relationships [and] are using partners and procurement wisely,” Le i la Janah , founder and CEO o f Samasource



Big Data challenge: Garbage In, Garbage Out

- Data labelers are sometimes referred to as “AI’s workforce” or the “Invisible workers of the AI era”

- A McKinsey report from 2018 listed data labelling as the biggest obstacle to AI adoption in the industry

- The data labelling industry spreads globally, specially in lower-income countries

- Teams are capable to annotate 10,000 images in eight days with 95% of accuracy (for a single task)

- An hour of video takes 8 hours to annotate

•Pixel - Wise Semantic Annotation•3D Semantic Annotation•Pixel - Wise Object Instance Annotation•Fine - Grained Road Segmentation•Moving Object Trajectory•High - Precision GPS / IMO Information

Managed Data Labelling Services


Why Small Data is Important for Advancing AI

Small data refers to data that humans can observe and process personally.

One individual’s CV is small data. A database with millions of CVs is big data.


Unprecedented Personalisation with Small Data AI

1

Product fit is typically elaborated through simulations. Individualised simulation cannot be achieved through big data, the simulation model needs to incorporate customer feedback on an individual basis. This is a small data job.

Big Data 2 Algorithm 3 Algorithm

6 Small Data 5 Big Data 4 Big Data

TightGood



1 Big Data



2 Algorithm



3 Algorithm



4 Big Data



5 Big Data


TightGood

!15


6 Small Data


ELSE Corp, a VIRTUAL RETAIL Company, since 2014

12 patents filled 5 already granted 16 trademarks registered


UVP: End-To-End Digital Ecosystem for Mass Customisation

3D CAD to Cloud PDM

3D Scanning

Virtual Retail Corner

else.shoes™, an end to end industry solution for the footwear sector, enabling Mass Customization, Industrial Made to Measure and On Demand Production.

Cloud & Hybrid ManufacturingVirtual Fitting

3D Photo Realism


CAD-to-Cloud Integration, Automated Cloud Rendering

Cloud Rendering: A tool to bring CAD 3D geometry in Cloud, via CloudPDM and Cloud Materials Library. Full control of scheduled rendering tasks


3D Consumer Engagement

https://vimeo.com/318865684



New PHYGITAL Customer Experience: DIGITAL CONSUMER experience

ELSE Technology Hub


Materials & Colors Personalisation in Action


Key Factors in Training Neural Nets

Data Size

Optimisation Algorithm

Implications of Small Data

Bad Generalisation

Data Imbalance

Difficult Optimisation

Loss Function

Parameter Initialisation


Fashion and Data: Brand’s DNA

Do you know your brand’s DNA?In the worlds of marketing, advertising and PR, this is how campaigns are structured on a daily basis. It’s a useful tool to help any business owner to sell their services, any entertainment artist to sell their art, and even any employee to sell themselves. Once you know the difference, it's much easier to distinguish what you do from the competition.

Forbes


Fashion and Data: Product DNA

Do you know your

product’s DNA? • Not visible material

• Non visual computations

• Designer signature

• Services

• Qualities

• Other declarative parameters by the producer


Fashion and Data: Product Anatomy

Visual Computing & Visual Parameters

•Shape

•Components

•Color

•Material

•Brand

•Logo


Artificial Intelligence Startups

Horizontal AI Vertical AI

These are companies that sell tools to help their customers implement AI

Companies solve a specific vertical problem using AI


3D Automation Tools & Technology Stack

Matlab & Simulink Blender UI Blender Scripting

Python Automation

3D to Machine Learning Framework


3D to ML

Bad Generalisation

Data Imbalance

Difficult Optimisation

Implications of Small Data

Data Generation

Data Augmentation

Regularisation

Ensembling

3D Synthetic Data

3D Design Level Annotation & Augmentation

Ridge - Last - Elastic Net

Bagging - Boosting

Balancing Data Set Design Level Scaling

Weight Assigning

Transfer Learning

Problem Reduction


3D to ML - Model Training

Generative Adversarial Networks (GAN)

Web Blender Customization

Transfer Learning: Mask-RCNN Semantic Segmentation

Transfer Learning - Classification

ELSE Edge Cluster

Supported Models

•ResNet•AlexNet•SqueezNet•Inception•Unet•DeepLab


3D to ML - Data Generation

Transfer Learning

Transfer Learning

Big Data

3D Rendered Images &

Annotations

Small Data (Target Images)

Traditional ML Transfer Learning

Isolated, single task learning: K n o w l e d g e i s n o t r e t a i n e d o r accumulated. Learning is performed w.o. considering past learned knowledge.

Learning of a new tasks relies on the previous learned tasks: Learning process can be faster, more accurate and / or need less training data.

Learning System Task 1

Dataset 1

Dataset 2


Knowledge


Dataset 1

Dataset 2Learning System Task 1

VS




Random Background

Different Annotations

Multiple Model Support

- Mask - RCNN - Semantic Segmentation - Landmark Detection - Transfer Learning Classification

Different Configurations

Different Material

Multiple View Points



Instance Segm (Classification + Semantic Segmentation) with MRCNN

How the model sees an input.


Layer Visualisation of Mask R CNN, TensorBoard

In the new activation layers it can be seen that the network sees the shoe almost in every layer.


ELSE.ai Layer Functionalities

B a c k g r o u n d Segmentation2Input1 3 Recognition 4 Use Case / 3D

Visual Search

Properly Trained Model with Conditional Random Fields

Multi Layer + CRF Model

False Positive / False Negative Bad Mask Localization

Challenges

Mask R-CNN Scene Segmentation /

Background RemovalResnet Shape Classification

Semantic Segmentation Component Detection

3D Configuration MappingInput Use Case

Recommendation Results

Closest 3D Configuration

Component Array

- Laces - Toe Cap - Tongue - Outsole

3D Model Pre-Population


Data preparation Pipeline and Training Steps

Reinforcement and Retraining

Component Level Feature Map Analysis Feature Map

Initial Future Map Analysis

Design Level Annotation + Augmentation

Component Based

Features

Code Level Augmentation +

Annotation

Initial Dataset

General AgentEvaluation Parameter Satisfied Special Agent

(Expert Reinforced)

Component + Design Level Metadata Reinforcement

Designers + Fashion Fields Experts- Customized 3D/CAD data generation

- Supervised by fashion experts and designers

- Reinforced by field knowledge

- Multi layer evaluation

- Code level and design level knowledge transfer


Synthetic Data, Data Augmentation

Forcing AI to «see» the SHAPES, not just textures/ pixels

- Smart small data to big data transformation

- Automated augmentation and Annotation

- Annotation format adoptive (Box – Mask – Polygon , …)

- Coupled with weight and Bias analysis

- Coupled with Evolutionary Algorithms for Hyper Parameter Optimization

- Adoptive towards other parameter optimization methods


Reinforcement by Field Experts, Humans to AI Knowledge Transfer

1.Expert defined style groups for clustering

2.Style and design based feature (component) analysis

3.Customized 3D / CAD data generation based on feature analysis

Designers + Fashion Field Experts

Component + Designer Level Metadata Reinforcement

Design Level Annotation + Augmentation

Component Based Features

Feature Map

Evaluation Parameter Satisfied

Component Level Feature Map

Analysis

Initial Feature Map Analysis


Input Image

Data

Featured Map

Data

Combined Feature Map

Input Image

Data

Featured Map Featured Map

Data


Train on the Cloud Deploy on the Edge

• Accelerated Computing at the Edge

• Powerful Computing for Data Centres

• Democratization, from Data Centre to the Cloud



3D rendering

Color Coded Multi - Label Annotation of Components

Landmark Annotation

3D CAD Model


3D to ML - Edge Deployment, Re-Training & Fine Tuning

Pre-Trained Model with Synthetic Data

The fine tuning is done in-store by a sales assistant. AI knowledge is not needed in this step

In-Store Fine - TuningELSE.Ai

Real Shoe Image Gathering


Re-Training/ Fine Tuning

Approach 1: ELSE Edge Data Collection App

Data Collection

Fine Tuning

Approach 2: ELSE Edge as a PC

Training Progress

Data Collection App UI Manual Directory Creation


Fine Tuning Process

Before Fine Tuning During Fine Tuning After Fine Tuning

Evaluation Parameters After Fine Tuning (5 Epochs in 1 Hour)

Entering Epochs (Between 5 - 10)

Starting the Fine Tuning after

Adding Images

Some Product Images are Added (Around

30-60 each class)

Pre-Trained Model (Accuracy Increases to 72% in less than 1 Epoch)

CASE STUDY: 3D Visual Search


Visual Style Search

• Image recognition and other A. I. technologies • It index, search, tag and find images without

using words • Gives recommendations showing visually similar

products • Generates the perfect look based on brand

style guides and available stock

• Visual search tool

• Users can use the application and upload a screenshot from Instagram, a photo taken from a magazine, or to take a picture from the app to get results of shoppable items.

• Visual recognition tool called Style Match

• It uses Google Lens to take pictures of clothing and then offers up similar shoppable items from around the Internet

http://wide-eyes.it http://www.asos.com https://www.racked.com

http://wide-eyes.it/

http://www.asos.com/

https://www.racked.com/2018/6/1/17417176/google-lens-visual-recognition-style-match


Amazon in here: StyleSnap

Amazon launches AI-powered ‘Shazam for clothes’ fashion search

- Amazon unveiled a new feature in its app where customers can upload a photograph of a fashion look that they like and Amazon will find the closest match on its site.

- Asos, Wayfair and Target have similar tools in place


3D Visual Search: Customer ScenarioC

usto

mer

Jou

rney

Gen

era

l Arc

hite

ctur

e

analysing… Detecting type…

what are we searching for today?

TYPE DETECTED

Detect ModelDetecting Model…

what are we searching for today?

CLASSIFICATION RESULT

TYPE: KED LOW

MODEL: XXX

REMOTE SEARCH

LOCAL SEARCH

CLASSIFICATION RESULT

TYPE KED

MODEL XXX

REMOTE SEARCH

LOCAL SEARCH

3D VS engine

Product gets recognized


Overall Scenario

3D Visual Search On «Small Data»

- Synthetic 2D&3D data generation - Automated Annotation and Augmentation

pipeline - Internal definition of Style in machine language - Reinforcement by field experts - Retraining by knowledge transfer - Evaluation by cross-field fashion & AI experts

(humans & AI)

A reinforcement learning agent, builds knowledge by constantly interacting with an environment which allows it to record and replay past experiences in a more efficient way than traditional supervised models.


Data Set: Train / Validation / Test Split x Supervised Learn

Training Set:

Synthet ic data, rendered images f rom 3D configurator coupled with auto annotation and augmentation pipeline

Validation Set:

Real product images, gathered from fashion brands & retailers

Test Set:

Real product photos, made from physical products


Components Detection with Mask R CNN

Original Image Tested

•The model trained was able to detect the objects giving the mask and the bounding box and to categorize the objects.

•The categories are grouped into supercategories, in our case all the categories belong to the shoe supercategory.

•The Neural Network was trained with one class only (Tennis shoes). A further step consisted in the creation of a custom dataset with several classes (ked low, ked high, slip on, tennis shoe)

•The images used were the rendered shoes on the grey background.

The masks proposed by Mask R CNN for the original image


Components Recognition and Configuration Mapping Mask R CNN

Detected & recognized components forms an array which then is mapped to a 3D configuration package.


Configuration Detected: The Results

Component array is mapped to 3D package and results are visualized for further customization


3D to Visual Search in the Lab

Ked-High Ked-Low Slip-On Tennis

Sneaker Classification


Technology Stack

- Supervised Learning

- Deep Learning

- Computational Vision

- Computational Intelligence

- Data Mining & Data Analysis

- Reinforcement Learning

Initial Dataset

Code Level Augmentation + Annotation

General Agent

Initial Feature Map Analysis

Feature Map


Component Level Feature Map Analysis

Component + Design Level Metadata ReinforcementDesign Level

Annotation + Augmentation

Designers + Fashion Field Experts

Component Based

Features

Evaluation Parameter Satisfied

Special Agent (Expert

Reinforced)


THANK YOU! СПАСИБО! GRAZIE ;)

[email protected]

Co-Founder & CEO ELSE Corp, a Virtual Retail

company

Andrey Golub

© 2019 ELSE Corp S.r.l.!58

Мастер-класс «Искусственный интеллект в индустрии моды»: РАНХиГС, 11/10/19

© 2019 Andrey Golub

VIRTUAL COUTURE FASHION «…Творческий потенциал человечества и

лучшие умы будут привлечены к машинному проектированию и автоматизации, которое станет основой для новой системы производства, --но что же в таком случае произойдет с творчеством и рукотворным мастерством?»

Ашок Сом , профессор Глобальный стратегии и со-директор бизнес-школы ESSEC-Bocconi EMiLUX Program

Integrating Small Data, Synthetic Data in AI Strategy for ... · © 2019 ELSE Corp S.r.l. !3...

Documents

Transcript of Integrating Small Data, Synthetic Data in AI Strategy for ... · © 2019 ELSE Corp S.r.l. !3...