FUTURE WORKCONCLUSIONS

SMART RECIPE MEASUREMENTS WITH LEARNED VOLUME PREDICTION

FEATURE EXTRACTION:

OVERVIEWWouldyouratherchewoffyourpinkyormeasureingredientsfortherestofyourlife?Theresponseisunanimous -

nooneneedsapinkyanyhow.Noonelikesmeasuring,butpreviousresearch(Nelson&Maticka,2017)suggeststhattheItalianGrandmaMethod(IGM)– asplashofthisandadabofthat- producesgastronomicalgrenadesforallbutthemostexperiencedchefs(e.g.IG’s).

ManyofuscasualcooksarenotabletoreproducewinningrecipeswhenapplyingtheIGM,andinsteadareleftwithaone-hit-wonderfollowedbyanonslaughtoffailedrecreationattemptsthatpaleincomparisonandlitteryourfridgewithdreadfulleftovers.Weproposeasolutiontothisdystopia.

Thebroaderpurposeofthisprojectistocreateasmartreciperecorderandinstructor. Youcaneither1)makearecipe-freedish,andaddingredientsatwillwhileasmartdevicefilmsandrecordstherecipe,or2)youcancreateasavedrecipe,andhavethedevicetellyouwhentostoppouringaspecifiedingredient.

Weappliedmachinelearningandcomputervisiontoteachphoneshowtomeasureingredientsforus.Wechosetofocusonpouredliquidsasafirststep.

DATA COLLECTIONFeature

Extraction

PouringVideos

X1:DurationX2:SpeedX3:Area

PredictedVolumes

Machine

Learning

Δy

Speed:Δy/Δt

Start

Stop

Duration:Δt

Δt

L

Area:<L2>

X1

X2

X3

EdgeDetection

FEATURES & SELECTION PROCESSFEATURE PROPERTIES:

• 3fundamentalphysically-relevantfeatures

• Inherentfeaturevariance

• 3-wayinteractiontermisaphysicalestimateofvolume

FEATURE SELECTION PROCESS:

• Forwardfeatureselection foreachmodeltype

• Fullfeatureset:3fundamentalfeatures,2nd-orderterms,2-wayinteractions,and13-wayinteraction(seeFeatureProperties)

• CVerrorwithk=10folds(split:90/10)wasusedastheselectionmetric

• ThefeaturesetthatyieldedthelowestCVerrorwaschosenfortherespectivemodel

5

10

Dur

atio

n (s

)

100

150

200

Spee

d (c

m/s

)

1

2

Area

(cm

2 )

0 0.5 1 1.5 2 2.5Actual Volume (cups)

2

4

6

Area

*Spe

ed*D

urat

ion

MODEL SELECTION/ASSESSMENT METHOD

SplitData:90/10

SelectModelWithLowestDevelopment

Error

KurtNelsonSamMaticka

{knelson3,smaticka}@stanford.edu

Checkifenoughdata

ForA

llMod

els

TestonReservedData(10%)

Tunemodelparameters

0 50 1000.015

0.02

0.025

CV

Err

or

(MSE

or %

wro

ng)

Featureselection

4 6 8 100.018

0.02

0.022

0.024

50 100 1500

0.05

0.1

0.15

0.2

TestOtherModels:• Neuralnetworks.Wewouldneedtocollectalotmoredata• PCAcombinedwithothermethods• RegressionTrees(preliminaryresultsshowpromise)

Thingsthatmayhelpcurrentmodels:• Improveexperimentalsetup- stereocamerasforbettercross-sectionalareaestimate• Improvefeatureextractionalgorithm:

o Betterestimateofflowrate(speed)- somesortofintermittentparticletrackingo Morerigorousremovaloferroneouscross-sectionalarea

Expansionofthesmartdevice’sabilities:• Generalizethemodelfordifferentingredients– dryingredients,clearliquids,

• Machinelearningdoesfarbetterthanabaselinepredictionusingtheory:oMSEontestdata: 0.016cupsvs.0.072cupsoMisclassificationError:25%vs.75%

• Non-parametricmodelsperformbetterthanparametric

• Datasamplesizewasadequateforsimpleregressionmodelstested.Thiswasconfirmedbyconvergenceoftestanddevelopmenterrors.However,thedatasamplesizelimitedtheclassificationmodelswewereabletotest.

MODEL TESTINGDevelopment SetErrors

RegressionModel MSE (cups)

1)Weighted LeastSquares 0.017

2) K-NearestNeighbors 0.018

3)OrdinaryLeastSquares 0.021

4)RidgeRegression 0.021

5)LassoRegression 0.023

ClassificationModel MisclassificationError(%)

1)Softmax 25

2)K-NearestNeighbors 26

3)Linear DiscriminantAnalysis 28

4)SupportVectorMachines 42

5)Physical Model(rounded) 78

Chosen

Mod

els

LOCALLY WEIGHTED LINEAR REGRESSION

• ℎ 𝑥# = ∑ 𝜃(𝑥(#)(*+

• 𝐽 𝜃 = ∑ 𝑤.(ℎ 𝑥. − 𝑥.)2#.*+

• 𝑤. = exp[− 7897:7897

2;<],Tuned 𝜏=2.2

• Selected7features:3fundamental features,3-wayinteraction,time2, 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 𝑎𝑟𝑒𝑎 ,(𝑠𝑝𝑒𝑒𝑑) 𝑎𝑟𝑒𝑎

• MSEerroronunseentestdatawas 0.016cups

SOFTMAX

• 𝑝(𝑦 = 𝑗|𝑥; 𝜃) = NOP(Q8:7)

∑ NOP(QR:7)S

RTU

• Maximize:ℓ 𝜃 = ∑ 𝑙𝑜𝑔∏ NOP(Q8:7)

∑ NOP(QR:7)S

RTU

+{[8*+}]^*+

#.*+

• Selected6features:3fundamentalfeatures,3-wayinteraction,𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛2,speed2

• Misclassificationerroronunseentestdatawas25%

PHYSICAL MODEL

• ℎ 𝑥. = 𝛼𝑥+.𝑥2. 𝑥`.=𝛼(𝑎𝑟𝑒𝑎)(𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛)(𝑠𝑝𝑒𝑒𝑑)

• 𝛼 = ∑ [8

7U8 7<8 7a8#.*+