Exclusive Interview with Yoshua Bengio!€¦ · Learning from Simulated and Unsupervised Images...

The magazine of the algorithm community

A publication by

Project Management:7 Tips by Ron Soferman

Spotlight News

October 2017

BEST OF MICCAI: 32 pages!!!Workshops, Presentations, Challenges, Tutorial

Upcoming Events

We Tried for You:FirstAid on Google Cloud in 30 minutes!

Women in Science: Sandrine De RibaupierreChristine Tanner

Review of Research Paper by Apple:Learning from Simulated and UnsupervisedImages through Adversarial Training

Exclusive Interview with Yoshua Bengio!

http://www.rsipvision.com/computer-vision-news/

http://www.rsipvision.com/





2Computer Vision News

Read This Month

Spotlight NewsFrom elsewhere on the WebReview of Research PaperLearning from Simulated and Unsupervised Images through Adversarial Training

Project Management7 Tips by Ron Soferman

We Tried for YouFirstAid on Google Cloud in 30’!Computer Vision EventsCalendar of August-October events

37

38

44

48

55

Best of MICCAI Daily 2017Interview:

Yoshua BengioPresentations:

Benjamin Hou, Dana Cobzas Women in Science:

Sandrine De Ribaupierre,Christine Tanner

4 Challenges7 Workshops1 Tutorial

04

Dana Cobzas

04

Yoshua Bengio

14

Women in ScienceSandrine De Ribaupierre

10

Women in ScienceChristine Tanner

22

Research of the monthReview by Assaf Spanier

38

Spotlight News

37

Upcoming Events

55

44

Project Management7 Tips by Ron Soferman

MICCAI Workshops

08


Computer Vision News

Editor:Ralph Anzarouth

Engineering Editor:Assaf Spanier

Publisher:RSIP Vision

Contact us

Give us feedback

Free subscription

Read previous magazines

Copyright: RSIP VisionAll rights reservedUnauthorized reproductionis strictly forbidden.

Dear reader,

This October issue of Computer Vision Newsopens with the exclusive interview thatProfessor Yoshua Bengio granted me atMICCAI2017: in addition to the exceptionalmerits of Prof. Bengio in the computer visioncommunity, the reasons why I recommend thatyou read this interview are the precious take-home thoughts suggested by him during ourfascinating discussion. I was myself fascinatedby his remarkable kindness and availability insetting up this candid talk, which you will readat page 4. I am grateful to MICCAI for theopportunity to add this interview to the manyothers that we had the chance to conduct withmajor scientists in our community: Jitendra Malik,Michael Black, Nassir Nawab and more.

Today I wish to thank MICCAI (in particularSimon Duchesne and Wiro Niessen) for yetanother reason: partnering with us and lettingus cover the conference with our MICCAI Dailypublication. We gained direct exposure tobrilliant technology and personalities: have aglimpse of all this in our BEST OF MICCAIsection, for the preparation of which we owe alot to Tal Arbel, who was Satellite Events Chair.

Before you ask, I will point out that this issue isnot only about MICCAI. You will also read ourregular reviews of outstanding researchpapers, as well as our regular sections: theproject management column by RSIP Vision’sCEO Ron Soferman; the list of upcomingcomputer vision events; the Spotlight News;and more…

Enjoy the reading!

Ralph AnzarouthMarketing Manager, RSIP VisionEditor, Computer Vision News


Welcome 3

mailto:[email protected]

http://www.computervision.news/feedback/

http://www.computervision.news/subscribe/


http://www.rsipvision.com/ECCV2016-Friday/#4

http://www.rsipvision.com/ECCV2016-Thursday/#4

http://www.rsipvision.com/CARS2017-Thursday/#18


http://www.rsipvision.com/MICCAI2017-Monday/#5


http://www.rsipvision.com/ComputerVisionNews-2017April/#14

http://www.rsipvision.com/MICCAI2017-Tuesday/#4



What would you say is the mostprecious lesson that you learned fromyour teachers?

Well, I would say the enthusiasm forlearning, for discovering, forunderstanding.

Understanding is your main drive?

Yes. In fact, I would say that it’s aphysical thing. It’s like a sensorypleasure, almost, when you realizesomething, understand, things areclicking. It’s the “Eureka” moment. It’sone of the foods for my happiness.

Does that mean that if you don’tunderstand, it is like a physical wound?

Yes, especially if I believe that it’ssomething that matters to me or that Ican do something about.

It bothers you to see the solutionthere and not be able to grab it?

I would say it differently: when I’mthinking a lot about a problem, andsometimes I have a little bit of arevelation of some idea, like coming inthe side behind my mind, I think of itlike some little threads that areappearing. And if I don’t pay attention,they would go away. But I can pull onthose threads, and then more comes,and then maybe nothing comes for awhile, and then something else,another thread, shows up that I canpull. And it’s like an unpredictableprocess of discovery. But it works onlyif you think about it, if you make yourmind ask yourself the questions over

and over, and let that brew in your mind.

Do you ever consult with yourstudents and see that maybe thesolution comes from somebody who isnot thinking on the problem all thetime and sees it from the outside?

Correct! All the time. In fact, theyspend more time, usually, thinkingabout a particular problem. I’m verydispersed, and I have many students,and so I would say there are two kindsof creativity that are really pillars of myresearch.

One is alone: it’s what I was talkingabout with those threads and so on.

“I think the reason I’ve been successful in my

career isn’t because I’m smarter than anyone else. It’s just that I’m able to focus a lot.”

Prof. Yoshua Bengio is Full Professor atthe Department of Computer Scienceand Operations Research, Universitéde Montréal. He granted me afascinating interview at MICCAI andhis kindness is the most valuablelesson I retain from our meeting.


Yoshua Bengio


And the other is in brainstorming withcolleagues, researchers, students, andthere are also “Eureka” moments fromthere. But it’s a very different kind ofprocess. The brainstorming, of course,is something that happens very fastand is very active, whereas the otherone is something more meditative, andso it’s a very different type.Can you remember a “wow” momentthat you had with one of yourstudents, somebody who surprisedyou with some particular spark?

Right, yeah! Recently, I worked withBenjamin Scellier, one of my Ph.D.students, on a problem that I’ve beenthinking of a lot for the last few yearsthat has to do with how the brainmight do something likebackpropagation: this is themathematical technique that is theworkhorse of deep learning, and it’sreally behind a lot of the success wesee, and we don’t know if brains aredoing something similar. I had anumber of ideas that I fed him, andthen he found something I hadn’tthought about that’s hard to explainbecause it’s very mathematical, but Iwas really impressed when he came tome with that. It’s been the basis ofmore work we’ve been doing sincethen.

What are the best friend and theworst enemy of a scientist?

Best friend? Time, having time.

You’d like to have more?Yes, yes.

And the worst enemy? Lack of time?

Lack of time, yes, yes. I would addsomething to put that into context, Ithink scientists - well, at least the wayI’ve been working - rely on ability tofocus. So focus, concentration ofthought, is crucial. And I think the

reason I’ve been successful in mycareer isn’t because I’m smarter thananyone else. It’s just that I’m able tofocus a lot. And of course, that’sconnected to the time, because youneed time to focus. You can’t beconstantly bombarded by email andmeetings and people, whatever. Youneed to have the mental space tofocus. That takes time, and then also,to actually do it.

Do you have any tips for those forwhom the focus is there, but theycan’t get it?

I don’t know. Open your horizons. Lookat other things. I get inspiration fromall kinds of horizons. I read a lot. So,just being plunged in the community -remember, I told you there’s thesemoments alone, and then there’smoments of brainstorming. Sologically, I get a lot of inspiration andideas from reading other scientists’work. A lot of what makes a goodscientist is that he or she spends a lotof time reading, talking to otherscientists who understand whatthey’re doing. That’s a huge thing.

“Science is not done alone. Science is a community effort. We build on each

other’s ideas all the time.”


Yoshua Bengio 5


Science is not done alone. Science is acommunity effort. We build on eachother’s ideas all the time.

Will science ever be able to mastertime?

I don’t know. I’m not a physicist.

I see. But you’d like that!

Sure.

Any advice for the young studentswho join this conference or anotherconference for the first time? Whatshould they do to get the maximum tomove forward in their career?

Well, at any conference, or when youread papers, you should focus on tryingto understand what is going on. It’s notalways easy because you get intosomething new, and it’s an effort. Youhave to accept that it’s an effort, thatpart of yourself will maybe want to goand have lunch or check your email ordo something else than actually do theeffort of trying to understand. Andagain, listen carefully to what theperson is saying, and really concentrateon that person’s words. The same thingwhen you read a paper. That makes ahuge difference.

We are at MICCAI2017 in Canada. Canyou add a word about Medical Imaging?

I think that what this community isdoing is incredibly important, andpersonally, I care a lot about seeing AIbeing used for good. And of course,medical implications - medical is central

to this, and medical images areprobably the golden place for currentdeep learning and in AI in general, tomake an impact on society. Right nowsome people may feel like it’s hard,mostly because there is not enoughdata, but it’s changing. It’s a socialissue. There is data. There’s going to beeven more. It’s not clear yet how it’sgoing to be shared and so on, but thesize of the data sets is greatlyincreasing. The potential impact ofthese things is going to be even morethan what people think right now.

“Listen carefully to what the person is saying, and

really concentrate on that person’s words!”

Prof. Bengio during his keynote lectureat MICCAI2017. Sitting on stage:

Maxime Descoteaux and Tal Arbel


Yoshua Bengio




The Multimodal Brain TumorSegmentation (BraTS) challenge is aninternational competition, with itsparticipants attempting to address theproblem of delineation and partitioningof the most common adult brain tumors(glioma/glioblastoma). Segmenting thevarious partitions/sub-regions of thesebrain tumors can clinically help towardspersonalizing radiation treatment inpatients via customized target volumedefinitions, allowing for refinedpersonalized dose escalation planningon radiation regimens. BraTS 2017 wentbeyond the sole task of segmentationand called for feature extraction andmachine learning approaches towardsthe prediction of patient overall survivalfrom pre-surgical multi-parametric MRI(mpMRI) scans.

With a continuously growing publiclyavailable dataset of almost 500 pre-surgical (mpMRI) scans from 16institutions, BraTS focuses on creating a

standardized dataset to benchmarksegmentation algorithms towardsidentifying the current state-of-the-art.The amount of available data in BraTS2017, with the accompanying manually-revised ground truth, enable bothgenerative and discriminative methodsto be applied towards providingsolutions to the problem ofsegmentation. This year alone, BraTShad 497 registered teams thatdownloaded the training data (including285 mpMRI scans with accompanyingground truth) and 53 teams that finallyparticipated in the challenge.BraTS 2018 will continue addressing thesame problems leading to acomprehensive post-challenge analysismanuscript towards evaluating whetherfine performance differences acrossalgorithms affect further analysis basedon extracted features, or even furtherprediction tasks such as this of overallsurvival.

by Spyridon BakasComputer Vision News

BraTS 2017 top-ranking teams on Segmentation (Seg) and Survival Prediction (SPred) tasks. From left to right: Tom Vercauteren (University College London - #2

Seg), Tsai-Ling Yang (National Taiwan University of Science and Technology -#3(tie) Seg), Fabian Isensee (German Cancer Research Center - #3(tie) Seg),

Konstantinos Kamnitsas (Imperial College London - #1 - Seg), Zeina Shboul (Old Dominion University - #1 SPred), Spyridon Bakas (University of Pennsylvania -

BraTS 2017 Lead Organizer), Alain Jungo (University of Bern - #2 Spred), Craig H. Meyer (University of Virginia - #3 - Spred).


Challenge: BraTS 7

http://braintumorsegmentation.org/



It’s the 6th year of the CVII-STENT(Computing and Visualization forIntravascular Imaging and ComputerAssisted Stenting – yes, it’s a mouthful)workshop! This MICCAI workshop hasbeen bringing together academic,clinical, and industrial researchers inthe field of endovascular interventionsto discuss the state-of-the-art in thearea. This year, there was a little moreto celebrate with the recent release ofthe CVII-STENT book in the Elsevier-MICCAI Society Book series, presentingimaging, treatment, and computedassisted technological techniques fordiagnostic and intraoperative vascularimaging and stenting. The contributorsof the chapters are all workshopregulars and we hope that the bookwill be a useful reference to those in orjust entering the field.Invited speakers Prof. Guy Cloutier(CHUM Research Centre, Canada) andKatharina Breininger (SiemensHealthcare, Germany) kicked off a lot

of discussion on IVUS (intravascularultrasound) segmentation andapplications on coiling and stenting,respectively. Katharina highlighted themain ways in which to aidendovascular applications: improvehardware, enhance image features,extract additional information fromimages, and integrate informationfrom other image modalities. Guydiscussed the new industrialcollaboration he forged after takingpart in a past CVII-STENT challenge onIVUS segmentation – “I encouragepeople to [work hard] in thesecompetitions and the results can bepositive!” We’re hoping to bring back achallenge in a future workshop – watchthis space!

The oral sessions were no lessinteresting, with topics ranging fromdeep learning for vessel segmentationto stent localisation in endovascularimaging; there were plenty ofdiscussion points. I, Su-Lin Lee, and Luc

by Su-Lin Lee

“I encourage people to [work hard] in these competitions

and the results can be

positive!”The author of this report Su-Lin Lee is a Lecturer at The Hamlyn Centre, Imperial College London.

Here she is with the night’s poutine.


Workshop: CVII-STENT

http://www.rsipvision.com/CARS2017-Wednesday/#14


Duong (École de technologiesupérieure, Canada) presented onrobotics and motion compensation,respectively, in endovascularprocedures. Unexpectedly, there was alittle overlap in our presentationswhere we both agreed that in this dayof automation and self-driving cars,we’re moving towards robotic controland the necessary improved imagingand navigation for these advancedsystems. Luc highlighted one particularissue that can be overlooked inacademic research in this field: “Howcan you improve the patient well-being?” We need to bridge the gapthat exists between us thetechnologists and the clinicians whoare performing these procedures on adaily basis.

The workshop ended with a discussionon the future of CVII and stenting.There was a general consensus thatthe field required larger datasets forvalidation and for the implementationof deep learning techniques. Therewas some general discussion on thedevelopment of a data collectionprotocol that would work worldwideand that could lead to a largecollaborative research database. It wasalso noted that there are a number of

researchers working in the field ofvascular segmentation who were notaware of the workshop. Are youworking in this field? Are youinterested in meeting otherresearchers in the area? We hope tosee you at next year’s workshop!

Finally, what is a celebration without afew drinks? We ended the night at LeBureau de Poste where much poutineand beer were had - it did seem a littleperverse though to have such a heart-clogging repast after a workshop onthe treatment of heart disease!

“How can you improve

the patient well-being?”The CVII-STENT workshop at lunchtime! There

were fantastic talks by Katharina Breininger (front row, second from left) and Guy Cloutier (front

row, fourth from right).

The CVII-STENT group at Le Bureau de Poste. Co-organisers Simone Balocco (front, centre) and Luc

Duong (back, left) seem very happy with the success of the workshop!


Workshop: CVII-STENT 9


Sandrine, tell us about your work.

Sure. I’m partly a clinician and aresearcher, so it means that 95% of mywork is clinical work where I dopediatric neurosurgery, so I operate onbrains of little kids and teenagers. Thenthe other 5% I’m trying to do research.Typically, the type of research I’minterested in is surgical simulator to tryto make it better for my trainees. It’svisualization of medical images tomake it better for all of us, and thenit’s also function imaging, looking atfMRI and DTI along lifespans, so littlebabies, we’re actually even going to tryeven fetus MRI, and then going topediatric, to young adults, older adultsto see how we think and try tounderstand a bit better about the brainbecause a lot of things are stillunknown.

Why did you choose to specialize inthe brain?

Because it’s fascinating. First, this iswhat’s driving you every day. Right? Ifyou didn’t have a brain, you wouldn’twake up in the morning to go to work.But I think it’s where we have the mostunknown things, so the unknown andthe idea of trying to find out moreabout it was what interested me.

It’s also the most difficult part,probably. You could have choseneasier tasks.Yeah, but difficulty is challenging, sofor me at least I need to do somethingthat’s new and difficult.

What is the most challenging side ofthis work?

To be able to do both research andclinical work just because of the timeconstraint. I think the most difficultpart of research, per se, in our worldright now is to try to find enoughgrants to be able to actually getstudents to help us. I think the important

Sandrine De RibaupierreSandrine De Ribaupierre is AssociateProfessor in Neurosurgery at theUniversity of Western Ontario.

“Find the right balancebetween professional and family life…”

Keynote at the DTI challenge, MICCAI 2015 in Munich.Just like during our interview,

Sandrine is holding little Maia in her arms.


Women in Science


part about having gratitude andworking on a project is the fact thatthey can learn as well. It’s to be able tointerest young people in new areas,because obviously we could also try todo research on our own where you’dneed less funding, but then theproblem is once you’re gone there isno newer generation; it’s a big gap,and therefore it’s not useful.

What aspect of working with thebrain is the most difficult?

That’s a good question. [Laughs]Thank you.

[Laughter] The most difficult thing, as asurgeon, to work with the brain is thefact that if you cut in the wrong area orif you actually don’t know… Everybodyis slightly different, so your languagearea is probably not in exactly thesame place than somebody else’slanguage area. If you operate, you goto the wrong area, then the problem isthat person is going to have deficits, sothey won’t be able to speak anymore ifyou take language, or they won’t beable to write, or they won’t be able todo math, and a lot of things that wedon’t know. So, we know about the bigfunctions; we know where the motor

area is, we know where your brain ismaking you move, but: do we knowwhy you like music, for example?Which area of the brain does that?

Therefore, as a surgeon, if you go anddisturb some of the networks, then theproblem is that person may not enjoymusic anymore and they change.Their taste in music changes?

Either taste, or altogether and theydon’t enjoy it anymore.

So, if I let you operate on my brain,you can make me enjoy Jazz?

Maybe, or maybe it makes you hate alltypes of music, which would behorrible.How can you stop your hands fromtrembling when you’re operating onan organ so vital to your patient?

Don’t drink too much coffee. [Laughs]If you don’t drink too much coffee andyou have steady hands overall…

When you do surgery you typicallyfocus on what you’re doing, you don’tthink about: “Oh, what is that persondoing?” and so on. You really have tofocus on the task you’re doing, andyou’re not thinking about the bigpicture. Before surgery and after surgery,

Sandrine in front of Mount Everest

“I need to do something that’s new

and difficult”


Women in Science 11


obviously, you think about the personand you know what they want, whatthey like and things, but once you dothe surgery you want to focus exactlyon what you’re doing, and thereforeyou still have some pressure on yourshoulders, but it’s not about the wholepicture – you just focus, and thereforeyour hands don’t tremble that way.Can you depersonalize what you’redoing?I think so, yeah.

Can you trick yourself into thinkingyou’re dealing with meat?

No, I don’t do it quite like that. It’smore you’re dealing with a specificarea in the brain, so you think: “I’mthere…” For example, you open theskull, then you open the membranearound the brain, then you go to thebrain, and then you actually dig intothe brain, but you know which area itis. At that point, you don’t say: “Iwonder what that person ate forlunch.” You’re going to say: “I need tofocus on what I’m doing right now. Iknow that just behind it’s the motorarea so I don’t have to go too muchbehind. Oh, watch out, there arevessels here”, so you’re focusing on alittle task, so you don’t have thepassion about things; you’re justfocused and concentrated on the

technical skills you’re using at the time.

What is your worst nightmare?My worst nightmare? Probably toharm a patient. My worst nightmareoverall is probably not being able to doresearch anymore, because then itmeans every day is the same, you justtreat patients, but you’re not able tothink and to learn new things.

What is your main drive?

I think my main drive is to discovernew things: to better understand howwe’re made and how we think, and tothink that maybe little kids can growup knowing more about the brain, andtherefore from a health point of viewwe’re able to treat better disease, butalso from a knowledge point of view,maybe you’re better to teach better aswell. If I hadn’t choose the brain, I wouldprobably have chosen another surgicalspecialty, because I need to dosomething with my hands.Generateabettersituationwithyourhands.

Yes, exactly!


Women in Science


Have you ever experienced anydifficulties being a woman, aneurosurgeon, or a teacher?When I trained as a youngneurosurgeon, one of the professors inSwitzerland said: “Neurosurgery is notreally for women, so are you sure youwant to do that?”, therefore I decided Iwasn’t going to train in that hospital, Iwas going to train somewhere else.That was my first challenge. After that,because the two hospitals wereassociated, I kept meeting with him andshowed him that I was perfectlycapable of being a neurosurgeon.

Did he change his mind?I’m not sure he ever changed his mind,but there are a few women inSwitzerland that are neurosurgeonsnow, therefore he probably, at least infront of people, will admit that womenare able to do it. In his mind, I’m notsure. There are some older people thatwill never change their mind… [Laughs]Was it even a drive for you to show hewas wrong or did you not really care?No, I didn’t really care.

What would you suggest to a youngwoman starting her career, hearing thesame remark?

You can either decide that you’re goingto prove them wrong, or you can decideyou can go somewhere else wherepeople won’t put obstacles in yourwork all of the day. Overall, I think youneed to be a bit better than a man to beable to go up the echelons. Otherwise,you’re not getting promoted. I think,overall, we still have to show that we’rebetter in order to get the sameposition, but that’s okay, it’s easy. I alsoadvise to find the right balancebetween professional and family life.

What do you know about the brain

that the public isn’t aware of?

People tend to imagine that every cellin the brain is useful, and that’s notquite true. If you see when we’reoperating, people tend to say: “If youcut a millimeter away, then it’s acatastrophe!” That’s true in some areas,but it’s not true everywhere. One of themyths about the brain is the fact thateverything is useful and that brainsurgery is really precise everywhere.That’s true, in some areas, but most ofthe time we have a bit of slack, a bit ofmargin of error around that.What do you think science willdiscover about the brain before theend of your career?

One thing I’m trying to work on rightnow is looking at fMRI in the babybrain, in the fetus brain. Right nowwe’re able to do it in little babies thatare just born or as a child, but what wewant to try to do is in a fetus, andtherefore try to see if we’re able todiscover things before they are evenborn. One of the things we would liketo see is: we know that some of themental illnesses, such as autism andschizophrenia, might have some groundeven before people are born, duringpregnancy; and maybe some of theenvironment factors, things that apregnant woman might do or notduring her pregnancy, could influencehow the brain of the baby is.

So you’d be able to treat them beforebirth?

Maybe. The main thing is: can weactually monitor it? We are not thereyet. The idea would be to try to monitorit, and then if we can monitor it, then itmight be possible to act on it and treatit before. That would be one of thethings I want to achieve before the endof my career.


Women in Science 13


Dana Cobzas gave a posterpresentation on her current workwhich focuses on the statistical side ofpopulation analysis. Dana’s goal is todesign a method that - given twopopulations, like the healthy and thediseased - will detect the significantanatomical difference between them,and be significant enough that it’shopefully related to the diseasedpopulation.

In the study, a new regularization isintroduced, so it’s very much on themathematical side. Along with a newregularization, a new penalty was alsointroduced. So, it’s formulated assparse classification. The usualpenalties are L1 to impose thesparseness, using what is called a SCADpenalty, which is a different type ofregularization that is better than theoriginal L1.

Dana at her poster

Dana Cobzas is an assistantprofessor at MacEwan Universityin Edmonton, Alberta.

“We also want to know how different anatomical structures differ in shapes,

rather than in volume”


Presentation: Dana Cobzas

An unbiased penalty for sparse classificationwith application to neuroimaging data


Dana works on this study with peoplefrom the department of statistics inthe mathematical sciences, and thus,coming from a computer scienceenvironment, she admits, “I think thiswas maybe the most difficult part, tobe able to understand the peculiarityof SCAD and work with them.”

From an algorithmic point of view,Dana explains that the computerstask is the optimization of theseproblems. It results in an energyminimization, which, unlike the classL1 minimization, is non-convex, andalso has discontinuity, just like L1. “So,we use it in what is called ADMM tosolve it, which is an optimizationmethod that is called the Alternate

Method of Multipliers. That isoptimizing these kind of problems, sothat was the most difficult part,doing the numerical method.” Themethod is designed such that itbrings a solution, It’s a numericalmethod that is for the optimization,and is proved to cover the solution.

If given the chance to add one morefeature to the model, Dana says shewould like to extend it to shape data,to define the same type of model forshapes, which is also very relevant,because, she says, “we also want toknow how different anatomicalstructures differ in shapes, ratherthan in volume.”

“We also want to know how different anatomical structures differ in shapes, rather than in volume”


Presentation: Dana Cobzas 15


The ever-growing ophthalmic imageanalysis community got together forthe 4th time at MICCAI 2017. This yearthe Workshop on Ophthalmic ImageAnalysis (OMIA) was organized jointlywith the Retinal OCT Fluid Challenge(RETOUCH). Thus, in addition topresenting their work the participantshad an opportunity to go head to headon the tasks of retinal fluid detectionand segmentation in optical coherencetomography (OCT) scans.Machine and deep learning methodswere prominent players in theworkshop for achieving successfulclassification, segmentation andembedding of 2D fundus or 3D OCTimages. The keynote speaker, Prof.Joseph Caroll, gave us a fascinatinginsight into the enormous potentialand also some pitfalls of adaptiveoptics for retinal imaging.

OMIA WebsiteRETOUCH Website

by Hrvoje Bogunovic

“…a fascinating insight into the

enormous potential and also

some pitfalls of adaptive optics for retinal imaging.”


OMIA / RETOUCH

Prof. Joseph Carroll from Medical College of Wisconsin giving the

keynote on adaptive optics

Donghuan Lu from Simon Fraser University presented with the 1st

place award at RETOUCH challenge by Hrvoje Bogunovic from Medical

University of Vienna

http://www.rsipvision.com/ComputerVisionNews-2017July/#18

https://sites.google.com/site/mwomia4/

https://retouch.grand-challenge.org/


Eight teams participated in theRETOUCH challenge, which wasconvincingly won by a team fromSimon Fraser University, Canada -congratulations! - who performed thebest in both the fluid detection andthe fluid segmentation tasks. Theirapproach consisted of a combinationof a graph-cut for retinal layersegmentation, a fully convolutionalneural network for subsequent fluidsegmentation and a random forestclassification for post-processing.

Interestingly, there were also severalpapers on retinal image analysis inthe main conference, which showsthat the interest for this topic isclearly growing. OMIA/RETOUCH aimto bring growing sections of theinterdisciplinary research communitytogether within MICCAI, arguably thebest technical conference in the worldfor medical image analysis. Hence

OMIA/RETOUCH offer a lot of addedvalue!

See you all in Granada, Spain forOMIA/RETOUCH 2018!

Read more about MICCAI 2018 in thisinterview and editorial preparedwith Alex Frangi and Julia Schnabel.

“…a combination of a graph-cut for retinal layer

segmentation, a fully convolutional neural

network for subsequent fluid segmentation and a

random forest classification for post-processing”

Left to right, the organizers of the event : Yanwu Xu, Emanuele Trucco, Xinjian Chen, Mona K. Garvin, Hrvoje Bogunovic, Freerk Venhuizen, Clara I. Sanchez


OMIA / RETOUCH 17

http://www.rsipvision.com/MICCAI2017-Wednesday/#18

http://www.rsipvision.com/MICCAI2017-Wednesday/#4

http://www.rsipvision.com/ComputerVisionNews-2017January/#20

http://www.rsipvision.com/ComputerVisionNews-2017July/#18


The third edition of the DeepLearning in Medical Image Analysis(DLMIA) workshop has witnessed itslargest number of paper submissions(73) and acceptances (38), withexcellent works on topics rangingfrom adversarial training tosegmentation, image normalisationand registration.

Since DLMIA’s first edition in 2015,the number of paper submissionsand acceptances has almost doubledevery year, showing a growinginterest by the MICCAI community onthis topic. In addition to the papers,DLMIA had three outstanding invitedspeakers.

Dr. Kevin Zhou presented his recentworks on recognition, segmentationand parsing based on deep learning,where one of the main questions heposed to the audience was theimportance of exploring contextualinformation in deep learningmethods. This was an important

question present in severaldiscussions, not only at DLMIA, but atMICCAI 2017 at large.

Should we invest more researchresources in data acquisition, datalabelling, and powerful deep learningmodel training? Or should we devotemore research resources to theexploration of contextual informationin order to allow for the developmentof lighter deep learning models?

Dr. Ronald Summers presented hisunique perspective on how deeplearning is influencing radiology. Infact, an important question thatyoung radiologists are currentlyasking is whether deep learning willreplace them in the future.

Although it is unlikely that this willhappen in the near future, it is quiteclear that the systems currently beingdeveloped in medical image analysiswill have an important impact inradiology.

by Gustavo Carneiro


Workshop: DLMIA

“Should we invest more research

resources indata acquisition,

data labelling, and powerful deep learning

model training?”Workshop chairs: Jacinto Nascimento,

Joao Manuel Tavares and Gustavo Carneiro


An important message in Dr. ChrisPal’s talk was the need to developdeep learning methods that can beused in the automatic normalisationof medical images – this was also atopic explored by a couple of papersin the workshop. The industry talkspromoted by DLMIA were also verywell received by the audience, withNVidia promoting their new powerfulGPUs and the Deep Learning Institute

and the Butterfly Networkshowcasing their impressiveultrasound machines. To summarise,we believe that this DLMIA was oneof the most successful satelliteevents of MICCAI 2017, with a largeaudience, high-quality papers andexcellent invited talks. Finally, we areplanning to organise DLMIA 2018 inGranada, and we hope to receivemany new paper submissions!

“Exploring contextual

information in deep learning

methods”


Workshop: DLMIA 19


Tutorial on Designing Benchmarksand Challenges for MeasuringAlgorithm Performance in BiomedicalImage Analysis.

Challenges and benchmarks are agreat way to advance the biomedicalimage analysis field. Michal Kozubekshowed why this is important andwhat design choices have been usedfor challenges and benchmarks.However, with over 150 challengesthat have now been organized in thefield, the time is ripe to review thesechallenges. Lena Maier-Hein,Matthias Eisenmann and AnnikaReinke presented some of the currentweaknesses of grand challenges thatwere found when reviewing thechallenges, and stressed theimportance of well-describedchallenges. Adriënne Mendrik andStephen Aylward presented atheoretical framework to help guidechallenge design and redefine theobjective of grand challenges, toeither gain insight (insight challenge)or solve a problem (deployment

challenge). Urging challengeorganizers to either follow aqualitative or a quantitativeexperimental design, and correctingfor leaderboard climbing using the re-usable holdout (Dwork et al) or theLadder method (Blum et al).

by Adriënne Mendrik

“There is to date no long-term solution that fully facilitates the sustainability of

grand challengesin the biomedical

image analysis field”


Tutorial: Designing Benchmarks and Challenges…

Adriënne Mendrik (Netherlands eScience Center)

OrganizerMichal Kozubek

(Masaryk University, Czech Republic)


The session was closed with adiscussion. One of the topics was therole of companies in organizingchallenges. Companies haveinteresting data and clinicallyimportant questions for thecommunity to work on, butrestrictions apply that limit datasharing. Rather than sharing all testdata, algorithm submission could beenforced, such that algorithms can beapplied to the test data by theorganizers, without having to releasethe data. Another discussion topic wasthe sustainability of grand challenges.

Most research challenges arecurrently not well maintained due tolack of funding and personnel. Long-term funding for an infrastructure tohost challenges, with data storage andcloud computing facilities is currentlylacking. Although there are multipleplatforms like Kaggle, COMIC (grand-challenges.org), COVALIC (Kitware),VISCERAL, CodaLab (Microsoft), virtualskeleton and DREAM challenges,there is to date no long-term solutionthat fully facilitates the sustainabilityof grand challenges in the biomedicalimage analysis field.


21

Lena Maier-Hein (DKFZ, Germany)

Stephen Aylward (Kitware, USA)

Tutorial: Designing Benchmarks and Challenges…


http://challenges.org/

Christine Tanner is a senior researcherat ETH. 2 days after this interview, shereceived the MICCAI IJCARS BestPaper Runner-up Award.

Christine, are you at MICCAI 2017 as asupervisor?Yes, exactly. I’m here as a supervisor ofNeerav Karani, a student of mine whois presenting a poster.

What is your work about?I’m doing research in medical imageprocessing. In particular, imageregistration. I’m supervising PhDstudents. I’m helping out professors tolook after them.Why did you choose medical imaging?

That was a chance choice. First, I wentinto industry, then I studied, and then Isaw a post about medical imageprocessing and I found it reallyinteresting. I went there and wanted todo a research position. It was onlypossible to do a PhD, so I did my PhD.

Where was it?That was at Kings College in London.

Where are you originally from?

I’m from Munich, Germany.How is it to be a future scientist as achild in Munich?I never imagined I would be one.Growing up, my parents did not haveany higher education. I was good inmath, but I was a girl and I was a bitshy. So I didn’t go to the class wherethere were all boys and just me doingmathematics. Instead I did thebusiness thing, and I couldn’t quiteimagine going to school too long. I likemathematics and I did anapprenticeship as a computer assistantfor Siemens.

How did you like it at Siemens?

It was very good. I grew into my job. Iwas doing software engineering at theend, as well as doing tests andintegration. But I felt like somethingwas missing, so I went back to study.

Let’s go one step back. How, in therelatively open and modern Germansociety, a girl who is strong in math isless supported?It was quite a while ago. I mean, on theone hand, it was kind of the mindset ofmy parents to say, “Oh, you don’t reallyneed to study!” On the other hand, itwas me, saying, “Okay, I don’t reallywant to be in school too long.” I wasn’tput under pressure and I was, as I said,a bit shy with all these boys and methe only girl in that class at 13 yearsold, or whatever I was then.

Does it ever still happen to be the onlywoman in a meeting with all men?

Yes, but I’m used to it now. This is justnormal, and I have no problems with

Christine Tanner

At Sierre-Zinal, one of the best Trail races


Women in Science

“You don’t buy confidence by running after others. You have to find it in yourself!”


that at all. But when you’re young, itcan be intimidating.

How did you overcome that?

I guess I grew up. Well, my Siemensapprenticeship in software assistancewas all women. So you grow up in thisenvironment where there are allwomen, and you’re okay working withmen. It wasn’t so threatening then.These were old men. They didn’t reallymean anything, you know? I had lots offemale environments for two years inthis apprenticeship.What is the most important thing thatyou’ve learned from your teachers?That’s a tricky question.

Thank you. I’m here for that.[laughter] My teachers gave mevaluable knowledge, but I cannot thinkof anything in particular now.

What is the best thing that you’velearned from your students? Is thereanything that impressed you?

I mean, I see how the PhD studentsgrow into becoming more confident.It’s not knowledge only, it’s also aboutthe things you need to sustain in orderto go through a PhD.

Resilience? Maturity?

Yes. I really like to be on their side. Youget different characters. Every studentcomes with different properties anddifferent abilities, and to pull out whatthey are good and to be able to turntheir projects around, so that theyreally are blooming in this project.You like to help bring out the best inpeople.Yes.

I like that. Do you see this evolution inyour students every time?

Students are quite different. For some,

from the beginning, you don’t have todo too much. They’re really great.Others are unfortunately sometimeslazy. I don’t mind students who are notso good, but they work hard and try.But if someone is lazy, I’m not veryhappy about that, because I think it’sreally terrible that people don’t makethe most out of what they are able to do.When students arrive with a lot ofpotential but not much self-confidence, is it mostly females ormales?

It might be more the females.

Can you advise a young femalestudent who does not feel veryconfident? Where can she buyconfidence?No, you don’t buy confidence byrunning after others. You have to find itin yourself. This comes by goingthrough hard times, not giving up,really sticking to the job. Of course,with some guidance. I mean, I’m here.I should be more experienced and beable to see the missing dots and getalong, so that next time, the studentknows better how to progress andwhat to do if the same problem comes.Can you give one example of a femalewho started at a low level ofconfidence, and she found in herselfthe force to progress?


Women in Science 23


I have to go through my students.

Please do.I can only say that there were sometimes where there was crisis striking,and I think I could stand by her andhelp her go through it.

Did you also see some of themquitting?

None of the female students quit. Youneed to encourage them more. Theyare not giving up, generally. I mean, bythe time they’re PhD students, theyhave gone very far in our field. Theonly ones which gave up were male.When you were a student, you sawother students giving up?When I was a bachelor student, I was amature student. Of course, I knewwhat I wanted. I had made my choiceto go there and do this. Now, youngstudents, they sometimes come out ofschool and study for no good reason. Ican see that they might give up,because it wasn’t due to a purpose,due to a real inner calling to go there.

I’ve heard you mention several timesthe support that you give to yourstudents. I guess this is a very, veryimportant drive for you.Yes, it is.

May I suspect that you’re trying togive more support to your studentsthan you received when you were intheir place?

I received support when I was a PhDstudent, but it was maybe not quite asclose a relationship. But I got goodsupport. I can’t complain, really.Aren’t you trying to give more?

I do it differently.

Because you know how important itcan be for the success of the student.

Maybe it’s just how I am. I’m more apositive person who is trying to getpeople with a positive aspect tosomething, instead of threateningthings. Of course, if someone reallyunderperforms and is lazy, I don’t likethat at all, and I can be tough on themas well.

You want to help people, but youwant them also to help themselves.

Absolutely.

What did you overcome?My main thing to overcome was inmyself, to be able to not do whateveryone was expecting from me.

What is the key to finding it?To listen to yourself, to understandwhat you’re about, to not get thrownaround by the environment which tellsyou what you should be doing. Stopwatching television, that’s all fake. Ofcourse, it’s not easy. We grow up withcertain role models.

That requires a very independentspirit.

Yes.

But not everybody has thisindependence easily inside them.

It’s not easy. It was not easy.

Studying for PhD in London


Women in Science


You are an independent spirit. How didyou become that?

Yes. It took me longer than I wanted to.

But how did you become anindependent spirit?

It came out. I mean, I had to break up. Iwas about to get married, have ahouse. I had to break up with my friend,and I had to leave home at a relativelyold age and go my way.

What did you do?I went and studied.

You didn’t marry him?

No.

He did not deserve you.No, he’s a good guy…. [laughs] He’s agood guy, but I would now be at home,having kids, being bored. No!

You have no kids?I have no kids, by choice. I’m not madefor kids. I doubt it would be easy to dothis job with kids. Yes, I have a husband.That’s enough.

Is he the kid in the family?

No, he isn’t. [Laughing]I’m joking. What if a younger femalelistens to this interview and says,“Okay, is Christine telling me not tomarry and not to have kids? Is thisincompatible with the career that Iwant?”

That’s a tough question.Sorry, that’s what I’m here for. [I smile]

Yes, I know. It depends on the manthese days, if they have grown to beable to support women.

If a man is able to share the burden,this career is feasible, even with kids?

My career: I’m not a professor. It’s notpossible with a husband who does notnecessarily support you. But as aprofessor, I think you need more support.

Sharing the burden.Yes.

Did you plan all this?

My career wasn’t planned at all. I justtook what came on me. Sometimes Ididn’t maybe do bigger steps, but I feelhappy with what I do. I like to beconfident with what I do. I like to do it.Maybe I could do more.

Are you a happy woman, Christine?

Yes.I hope you’re happy forever.

Thank you.

“I would now be at home, having kids, being bored. No!”


Women in Science 25


Stroke is a devastating disease, and aleading cause of death and seriouslong-term disability.

Whereas traditionally the treatmentfor ischemic stroke is intravenousadministration of tPA to dissolve theocclusion, several recent studies haveshown that mechanicalthrombectomy is an effectivetreatment for stroke patients. Patientselection and outcome predictionremain challenges, large studies arerunning to investigate the whichpatients may benefit most fromtreatment.

In these studies, imaging plays acrucial role. It is in this context thatthe first SWITCH workshop wasorganized at the MICCAI conference,with the main goal of bringingtogether clinicians and engineers, todiscuss challenges and opportunitiesfor the medical imaging community inthe management of stroke patients.To this end, three clinical experts

introduced the participants in theimaging for stroke patients: Dr.Roland Wiest from the Inselspital inBern introduced MR imagingprotocols for stroke patients, andtheir challenges. MR is a versatileimaging technique, that permits theassessment of various relevantimaging parameters.

Dr. Kambiz Nael similarly introducedthe standard CT imaging protocol,consisting of an NCCT, followed by acontrast enhanced CT (either CTA,multiphase CTA or CT perfusion) forstroke patients.

Thirdly, Dr. Vitor Mendes Pereiradiscussed mechanical thrombectomy.Previous studies failed todemonstrate the effectiveness ofremoving the occlusion withendovascular devices, but recentstudies have changed thisdramatically, and consistentlydemonstrate benefits of thistreatment for stroke patients.

by Theo van Walsum

“A forum for interaction between

clinicians and engineers is required for

making progress

in this field”


Workshop: SWITCH

Keynote lecture by Kambiz Nael.The author of this report is standing on the left.

http://switchmiccai.mit.edu/switch/


Next, four contributions ofresearchers from the MICCAIcommunity were presented: one onthe segmentation of ventricles instroke patients, one on the effect ofslice thickness in thrombusquantifications in CT, and two onquantifications of collaterals. Thelatter is relevant, as collaterals areassumed to play an important role inkeeping the infarct core small andprolonging the time that athrombectomy may be effective.

The workshop was finalized with anopen discussion, with as main

conclusions that: 1) a forum forinteraction between clinicians andengineers is required for makingprogress in this field; 2) this workshopshould thus be a recurrent event; and3) the workshop organizers willdiscuss how to shape a future event,which may include a stroke-relatedchallenge.

The SWITCH workshop was followedby the ISLES workshop; thecombination of the morning andafternoon session proved verypositive towards the design of futuretechnical challenges.


27Workshop: SWITCH

Keynote lecture by Vitor Mendes Pereira

Keynote lecture by Roland Wiest

“This workshop should be

a recurrent event!”

“The combination of the morning and afternoon session proved very positive…”


The main goal of the MICCAIworkshop on Bio-Imaging andVisualization for Patient-CustomizedSimulations (BIVPCS), initiated inMICCAI 2013, is to provide a platformfor communications amongspecialists from complementaryfields such as signal and imageprocessing, mechanics,computational vision, mathematics,physics, informatics, computergraphics, bio-medical-practice,psychology and industry.

In this 2017 edition of BIVPCS, 12highly motivating works were orallypresented, which promotedinteresting discussions concerningdifferent advanced techniques,methods and applications, and theexploring of the translationalpotential of the related technologicalfields; particularly of Signal

Processing, Imaging, Visualization,Biomechanics and Simulation.

Hence, the workshop was anexcellent opportunity for theparticipants to refine ideas for futurework and to establish constructivecooperation for new and improvedsolutions of imaging and visualizationtechniques and modeling methodstowards more realistic and efficientcomputer simulations.

by João Manuel R. S. Tavares

“New and improved solutions of imaging and visualization techniques and modeling methods towards more realistic and efficient computer

simulations”


Workshop: BIVPCS

BIVPCS was an excellent discussion forum concerningMedical Image Analysis, Biomechanics and Computer Simulation.


Based on the review results and onthe presentations given, theworkshop organizers awarded theBest Paper prize to the work “RapidPrediction of Personalised MuscleMechanics: Integration withDiffusion Tensor Imaging”, by J.Fernandez, K. Mithraratne, M.Alipour, G. Handsfield, T. Besier andJ. Zhang.

The workshop organizers would liketo take this opportunity toacknowledgment the MICCAI society,the MICCAI 2017 conferenceorganizers and the members of the

workshop program committee for allthe support given, and the authorsand the participants for theworkshop success.

As to the future of BIVPCS, theworkshop organizers are preparing aspecial issue of the Taylor & Francis“Computer Methods in Biomechanicsand Biomedical Engineering: Imaging& Visualization”, journal devoted tothe workshop.

They also intend to promote itsedition again at MICCAI 2018 inGranada.

“An excellent opportunity for the participants to refine ideas for future work and to establish constructive cooperation…”

The CVII-STENT workshop at lunchtime! There were fantastic talks by Katharina Breininger (front

row, second from left) and Guy Cloutier (front row, fourth from right).


Workshop: BIVPCS 29


BrainLes(s) MICCAI workshop washeld at MICCAI 2017 in Quebec City,offering an overview of medical imageanalysis advances in glioma, multiplesclerosis (MS), stroke and traumabrain injuries (TBI). It was the thirdedition, and as usual we hadresearchers from the medical imageanalysis domain, radiologists andneurologists, discussing the mostcommon brain diseases and traumas.

The event was held in conjunction withthe challenges on Brain TumorSegmentation (BraTS), and WhiteMatter Segmentation (WMH) tocomplement the program as the focuson segmentation of those lesions inmedical imaging.

The keynote speakers of this yearwere:

Tal Arbel, Professor at McGillUniversity, who gave a talk aboutsegmentation of variousneurodegenerative diseases,including MS and brain cancer.

Michel Bilello, Professor at theUniversity of Pennsylvania, whogave a clinical perspective ofcomputational neuro-oncology.

Rivka Colen, Professor ofNeuroradiology at the UT MDAnderson Cancer Center, who gavea talk focusing on how to mergegenetics and medical imaging tounderstand better brain tumors.

Jerry L. Prince, Professor at JohnHopkins University, discussing thenew frontiers of brain lesionsegmentation from medicalimaging.

by Alessandro Crimi


Workshop: BrainLes

Spyros Bakas, Lead Organizer of theBrain Tumor Segmentation Challenge (BraTS)

http://www.med.upenn.edu/sbia/brats2017.html

http://wmh.isi.uu.nl/



Among the methods presentedduring the workshop and challenges,many techniques were using deeplearning. This is a further confirmationof the potential of deep learning insegmentation of lesions. Allparticipants joined the event with theattempt of comparing theirtechniques applied to one disease anddiscussing whether they can beapplied them to other domains anddiseases. Extended papers of thepresented works - including keynote

speeches - will be published in avolume edited by Springer.

We hope to see you at the nextedition, and keep following us at theBrainLes website.

If you want to learn more on theauthor of this report AlessandroCrimi, read what he told us last yearat MICCAI 2016 about his work onbrain connectivity.

“We had researchers from

the medical image analysis domain, radiologists and

neurologists, discussing

the most common brain diseasesand traumas”


Workshop: BrainLes 31

Michel Bilello giving a clinical perspectiveof computational neuro-oncology

http://www.brainlesion-workshop.org/

http://www.rsipvision.com/MICCAI2016-Thursday/#22


The WMH Segmentation Challengesuccessfully compared twentyparticipating methods for theautomatic segmentation of whitematter hyperintensities (WMH) ofpresumed vascular origin. WMH arewell visible on brain MR images andone of the main consequences ofcerebral small vessel disease, playinga crucial role in stroke, dementia, andageing. Quantification of WMHvolume, location, and shape is of keyimportance in clinical researchstudies, but visual rating hasimportant limitations.

Sixty 3T brain MR scans (T1 andFLAIR) with manual segmentations ofWMH were provided for training ofautomatic methods, originating fromthree sites (UMC Utrecht, NUHSingapore, and VU Amsterdam) andthree vendors (Philips, Siemens, GE).The secret test data consistent of 110scans, originating from five differentscanners (the three from the training;and additionally a 3T PET-MR and a1.5T scanner). All non-WMHpathology was segmented as welland ignored during evaluation.

by Hugo Kuijf

“20 participating methods for the automatic segmentation of white matter hyperintensities

(WMH) of presumed vascular origin”


Challenge: WMH

The challenge was organized by the author of this report Hugo Kuijf of the UMC Utrecht, the

Netherlands

Team sysu_media receives the 1st prize in the WMH Segmentation Challenge for their U-Net approach. The team had the overall best score

and the highest score on 3 of the 5 metrics.


Unlike other challenges, participantshad to containerize their method withDocker and submit that for anobjective evaluation by the organizers.This guaranteed that the test setremained secret and the evaluationresults were not shared until thechallenge session at MICCAI.

During the well-attended session atMICCAI, each participating team brieflypresented their method. After that,the evaluation results were revealed tothe participants and the audience.

First place went to team sysu_media,

second place to cian, third place tonlp_logix. The final ranking was donewith relative metrics, highlighting thesmall performance differencesbetween the top-ranking teams.

The session closed with an interactiveposter session. Since the final resultswere unknown to the participants untilthe very last moment; the organizersprinted and pinned the results on theindividual team posters.

Results are posted here. The challengeremains open for new and updatedsubmissions.

“Key features of this challenge included the containerized approach where participants had to send in their method

for evaluation; the absolute secrecy on the test results until after the session at MICCAI; and the large dataset

with high quality images and manual

annotations provided.”


Challenge: WMH 33

Team cian receives the second prizein the WMH Segmentation Challenge

for their MD-GRU approach.

Team nlp_logix receives the third prize in the WMH Segmentation Challenge.

This team had the highest scorefor two out of the five metrics.

“1st place went to team sysu_media, 2nd place to cian,

3rd place to nlp_logix”

http://wmh.isi.uu.nl/


SASHIMI: Simulation and Synthesis inMedical Imaging, organized bySotirios A Tsaftaris, Ali Gooya,Alejandro F Frangi and Jerry L Prince.

Most common methods presented arebased on Generative AdversarialNetwork.

The workshop started off with anoverview of Adversarial DomainAdaptation given by Hugo Larochelle,a research scientist at Google Brain.One of the interesting works that werementioned in his talk was the DomainSeparation Networks. In this method,they explicitly learn to extract imagerepresentations that are partitionedinto two subspaces: one componentwhich is private to each domain andone which is shared across domains.The model is trained to not onlyperform the task we care about in thesource domain, but also to use thepartitioned representation to

reconstruct the images from bothdomains.

Two works used Cycle-GAN to mapbetween different modalities. The firsttitled Adversarial Image Synthesis forUnpaired Multi-Modal Cardiac Databy Agisilaos Chartsias et al. and thesecond titled Deep MR to CT Synthesisusing Unpaired Data by Jelmer M.Wolterink et al.

Another work that used GAN relatedapproach was titled Virtual PET Imagesfrom CT Data Using DeepConvolutional Networks: InitialResults by Avi Ben Cohen et al., in thiscase it used a Conditional GANapproach combined with a FullyConvolutional Network to achieve thevirtual PET images from CT images.

Additional interesting works that werepresented at the SASHIMI workshopcan be found in the workshop's webpage.

“Most common methods presented are based on GAN - Generative Adversarial Network”

Alejandro Frangi (standing at the right) during AviBen Cohen’s presentation about Virtual PET images from CT Data using Deep Convolutional Networks


Workshop: SASHIMI

http://www.cistib.org/sashimi/


This year the third edition of theIschemic Stroke Lesion Segmentation(ISLES) gathered much interest fromthe medical image computingcommunity, with more than 150 pre-conference data access requests, and afinal set of 16 highly competitive teamsparticipating on the on-site challenge.

Originally conceived as a segmentationchallenge for the analysis of acute andsub-acute stroke lesions from multi-sequence MRI, the ISLES challengeevolved in the last years to host amuch challenging task: forecast strokelesion outcome.

This holds much promise for clinicians,as it can leverage and assist theinterventionalist in the difficultdecision-making process needed todecide whether a mechanicalthrombectomy is pertinent for apatient. Participating teams receivedthen a combination of multisequenceMRI and clinical information taken atthe acute stroke state and were askedto predict the stroke lesion outcome atthree-month follow-up.

This year’s edition of ISLES featured a

richer training and testing dataset thanfor previous editions, which wascurated by an expert radiology team atInselspital, University Hospital in Bern.

The competition this year resulted in ahighly competitive setup. All sixteenparticipating teams presentedvariations of deep learning networks.In terms of performance, the scoresimproved from last year, indicatingachieved progress, but are stillsuboptimal for clinical exploitation.

Indeed, the ISLES challenge has beenmentioned by several competingteams (also participating in otherchallenges) as one of the toughestchallenges they have ever participatedin. The complexity of the task relates inturn to the high complexity of strokerecovery and brain blood circulation.

In this regard, the discussions that tookplace during ISLES and the SWITCHworkshop were very fruitful, leadingthe organizing team to pinpoint futuretechnical challenges focusing onleveraging the modeling of collateralflow for a better characterization ofstroke lesion recovery.

by Mauricio Reyes

The ISLES 2017 organizing team. From left to right: Mauricio Reyes, Roland

Wiest, Arsany Hakim, Stefan Winzeck

The ISLES 2017 Challenge: forecasting stroke

lesion outcome from multisequence MRI and

clinical information


Challenge: ISLES 35

“…one of thetoughest challenges…”


http://www.isles-challenge.org/


Trained experts, like medical imagingscientists and radiologists, canmentally estimate the approximateorientation of a randomly oriented 2Dslice through the body. But getting theentire picture of the full 3D anatomygoes beyond human abilities.

With the recent advent of DeepLearning, we have developed a fullyautomatic CNN-based method to learnthis expert-intuition about slicetransformations. Our approach canestimate the full 3D transformation ofrandomly oriented 2D slices purelyfrom the learned features in theimage. We can do this withoutinitialization from, e.g. scannercoordinates.

Transformation predictions aregenerated relative to a canonical atlascoordinate space, which facilitates, forexample, direct application of 3D atlas-based segmentation. The nature ofGPU accelerated Deep Learning allowsto make estimates within a fewmilliseconds per slice.

In practice, there are many problemswhich can benefit from suchinitialization-free 2D to 3D registration.Two applications, that are featured inthis work, are motion correction forfetal brain imaging and 2D to 3Dregistration with projective C-Armimages.

“Getting the entire pictureof the full 3D anatomy

goes beyond human abilities”

Benjamin Hou is a PhD Studentat Imperial College London.

Fetal MR brain image(in-plane view). Many

overlapping stacks of slices are acquired while the

fetus is moving.

Fetal MR brain image(out-of-plane view). Heavy

motion corruption between individual slices

and stack acquisitions.

State-of-the-art 3D motion compensation and

reconstruction from several overlapping, heavily

motion corrupted, stacks of slices very often fails.

3D reconstruction after our CNN-based approach re-aligned each slice individually in canonical atlas space

Further registration refinement using iterative slice-to-volume optimization

“make estimates within a few milliseconds per slice”

“…estimate the full 3D transformation of randomly oriented 2D slices purely from

the learned features in the image”


Presentation: Benjamin Hou


Computer Vision News lists some of the great stories that we have just foundsomewhere else. We share them with you, adding a short comment. Enjoy!

A Smart Recycling Bin Could Sort Your Waste for YouComputer vision can remove any confusion whendisposing of different types of plastic. The algorithm learnsto recognize images to identify the material held in frontof its cameras and then tells the consumer exactly wherewaste should be placed. Read More

Toronto's early lead in artificial intelligence: UofT expertsDon’t suspect this to be a self-promotional article by theUniversity of Toronto! UofT is really being instrumental intransforming Canada into an early leader of the ArtificialIntelligence revolution. Find out who did it and how, fromProfessor Emeritus Geoffrey Hinton to two dear friends ofComputer Vision News: we are not a little proud to seeassociate professor Raquel Urtasun and assistant professorSanja Fidler be recognized for their work. Read Now...

Quick and reliable 3D imaging of curvilinear nanostructuresNano-sized objects can be observed by transmissionelectron microscopy (TEM), generally limited to 2Dimages. Researchers from Ecole Polytechnique Federalede Lausanne (EPFL) are now able to overcome the needfor hundreds of tilted views and sophisticated imageprocessing to reconstruct their 3D shape. Their electronmicroscopy method obtains 3D images of complexcurvilinear structures without tilting the sample. Read More

This AI program can make 3D face models from a selfieAI experts from Kingston University and the Universityof Nottingham have trained a Convolutional NeuralNetwork to convert two-dimensional images of facesinto 3D. They fed the CNN tons of data on people’s facesand from there it figured out by itself how to guess whata new face looks like from a previously unseen pic, includingparts that it can’t see in the photograph. Read More

Another Microsoft news, as they launch new machinelearning tools: Azure Machine Learning Experimentationservice, Azure Machine Learning Workbench and AzureMachine Learning Model Management service. Read More

Microsoft’s AI chiefwantsto launcha chatbotineverybig countryRemember Harry Shum? I interviewed him at CVPR. Nowthat Microsoft’s CEO Satya Nadella wants to “democratize”AI, Shum wants to create machines that aren’t just smartbut are also able to connect emotionally with us. Read More


Spotlight News 37

http://www.smithsonianmag.com/innovation/smart-recycling-bin-could-sort-your-waste-you-180964848/


https://www.utoronto.ca/news/it-s-time-capitalize-toronto-s-early-lead-artificial-intelligence-u-t-experts

http://www.rsipvision.com/CVPR2017-Monday/#16

http://www.rsipvision.com/CVPR2017-Tuesday/#12


https://phys.org/news/2017-09-microscopy-method-quick-reliable-d.html


https://thenextweb.com/artificial-intelligence/2017/09/15/1076934/#.tnw_je0kjP1D


https://techcrunch.com/2017/09/25/microsoft-launches-new-machine-learning-tools/







https://www.geekwire.com/2017/microsofts-ai-chief-wants-launch-chatbot-every-country-100m-people/

http://www.rsipvision.com/CVPR2017-Tuesday/







Learning from Simulated and Unsupervised Images through Adversarial Training

Every month, Computer Vision News reviews a research paperfrom our field. This month we have chosen to review Learningfrom Simulated and Unsupervised Images through AdversarialTraining. We are indebted to the authors from Apple (AshishShrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, WendaWang, Russ Webb) for allowing us to use their images to illustratethis review. Their work is here.

Aim:The need for large labeled training datasets is constantly growing, as ever-highercapacity deep neural networks continue to appear. Since labeling large datasetsis time-consuming, expensive and uncertain, researchers consider usingsynthetic rather than real images for training, since they come alreadyannotated. However, training a network on synthetic images is problematic,because of the difference in distributions between real image and the syntheticimage, causes the network to learn qualities unique to synthetic images, leadingto poor performance when dealing with real images.

Motivation:SimGAN is a network using Simulated+Unsupervised (S+U) learning, whose goalis learning to improve the realism of a simulator’s synthetic images usingunlabeled real data. Improved realism will make it possible to train bettermachine learning models on large datasets without new data collection or theneed for human annotation.

Novelty:To overcome this difference, the authors propose simGAN, whose goal islearning a model to improve (refine) the realism of a simulator’s output usingunlabeled real data, while preserving the annotation relevance of the images.This idea is demonstrated in the image below:


Research

Re

search

“Learning to improve the realism of a simulator’s synthetic images using unlabeled real data”

by Assaf Spanier

https://arxiv.org/abs/1612.07828



Background:Traditional GAN Networks train a Generator and a Discriminator network. Duringtraining the two networks train with competing losses the Generator’s goal is tomap a random vector to a realistic image, and at the same time theDiscriminator tunes its loss function to distinguish between the generated fromthe real images. Since the first GAN framework by Goodfellow et al. introducedin 2014, many improvements and applications have been presented the researchcommunity. Wang and Gupta use a Style GAN to generate natural indoor scenes.iGAN enables users to interactively produce photo-realistic image. CoGAN usescoupled GANs to learn a joint distribution of images from multiple modalitiesand many more.

SimGAN is a method for S+U learning that uses an adversarial network similar toGenerative Adversarial Networks (GANs), but with synthetic images as inputsinstead of random vectors.

Past research, such as Gaidon et al, shows that pre-training a deep neuralnetwork on synthetic data leads to improved performance. SimGANcomplements these efforts, as images refined for improved realism will makepre-training on synthetic data that much more effective. The SimGAN approachis to make several key modifications to the standard GAN algorithm to preserveannotations, avoid artifacts, and stabilize training: (i) a ‘self-regularization’ term;(ii) a local adversarial loss; and (iii) updating the discriminator using a history ofrefined images.

Method:The SimGAN has two components: the Refiner and the Discriminator. TheRefiner minimizes the combination of an adversarial loss and a ‘regularization’term whose aim is ‘fooling’ the Discriminator network. The Discriminator goal isto classify an image as real or refined. Next, we will look more closely at each ofthe two components.

Next, let’s look at SimGAN’s pseudocode, followed by more detailed descriptionof the steps:


Research 39

Re

sear

ch


The Refiner (𝑅𝜃) is a ConvNet the aim of which is to make the synthetic imagesmore realistic by bridging the difference between the distributions of syntheticand real images. Ideally, it should make it impossible to classify a given image asreal or refined with high confidence. Let’s dive into the Refiner’s optimizationfunction to see how each of its components is minimized as we near our goal:


Research

Re

search

Input:

● 𝑥𝑖 ∈ 𝑋 Set of synthetic images ,

● 𝑦𝑖 ∈ 𝑌Set of real images

● 𝑇max number of update steps

● 𝐾𝑔 number of generative network update steps

● 𝐾𝑑 number of discriminator network update steps

Output:

● A Refiner ConvNet model, denoted by 𝑅𝜃

Initialize the Refiner 𝜙 and the Discriminator 𝜃 parameters.

For t = 1… T do

// Refiner

For𝑘 = 1 . . . 𝐾𝑔 do

1. Sample a mini-batch of synthetic images 𝑥𝑖2. keep 𝜙 fixed and update 𝜃 by taking a SGD step on mini-batch loss

𝜁𝑅 𝜃 = − 𝑖(𝑙𝑜𝑔(1 − 𝐷𝜙(𝑅𝜃(𝑥𝑖 ))) + 𝜆 𝜓(𝑅𝜃(𝑥𝑖 )) − 𝜓(𝑥𝑖 ) 1End

// Discriminator

For 𝑘 = 1 . . . 𝐾𝑑 do

1. Sample a mini-batch of synthetic images 𝑥𝑖2. Compute 𝑥 = 𝑅𝜃(𝑥𝑖 )with current 𝜃

3. Keep 𝜃 fixed and update 𝜙by taking a SGD step on mini-batch loss

𝜁𝐷(𝜙) = − 𝑖(𝑙𝑜𝑔(𝐷𝜙(𝑥𝑖)) − 𝑖(𝑙𝑜𝑔(1 − 𝐷𝜙(𝑦𝑖 ))

𝜁𝑅(𝜃) = −

𝑖

(𝑙𝑜𝑔(1 − 𝐷𝜙(𝑅𝜃(𝑥𝑖 ))) 𝜆 𝜓(𝑅𝜃(𝑥𝑖 )) − 𝜓(𝑥𝑖 ) 1

𝐷𝜙 is the probability of the image being

synthetic, with 1 meaning the function is

certain the image is synthetic. As

𝐷𝜙approaches 1, 1-𝐷𝜙 approaches 0 and

minimizes the entire expression.

The method uses𝜓, a feature-extraction function.

To preserve relevance of the annotation, self-

regularization loss is used to minimize the

difference in image features between the refined

and original synthetic image


The Discriminator (𝐷𝜙) is a ConvNet whose goal is to correctly classify an imageas real or refined synthetic, trying to overcome the Refiner’s attempts to fool it -to score every real image with probability 0, and every refined synthetic imagewith 1. Let’s dive into the Discriminator’s optimization function to see how eachof its components is minimized as we near our goal:

Evaluation and Results:Let’s look at results on two datasets:

(a) The NYU hand pose dataset:

The dataset is composed of 72,757 training frames and 8,251 testing framescaptured by 3 Kinect cameras – one frontal and 2 side views, each depth frame islabeled with hand pose information.


Research 41

Re

sear

ch

𝜁𝐷(𝜙)

= −

𝑖

(𝑙𝑜𝑔(𝐷𝜙(𝑥𝑖)) −

𝑖

(𝑙𝑜𝑔(1 − 𝐷𝜙(𝑦𝑖 ))

𝐷𝜙is the probability of the input

being a synthetic image

1 − 𝐷𝜙 is the probability of the

input being a real image.

● 𝐷𝜙 is a ConvNet whose last layer outputs the probability of the

sample being a refined image

● The target labels for the cross entropy loss layer are 0 for every

𝑦𝑖 , and 1 for every 𝑥𝑖 .

● When the network successfully discriminated between the real

and the refined synthetic - both terms of 𝜁𝐷(𝜙) are minimized.


The qualitative results are illustrated in the figure above. The edge depthdiscontinuity is the main source of noise in real depth images, the refinedsimulated images show a good facsimile of this discontinuity.

Quantitative results for hand pose estimation: the plot shows cumulative curvesas a function of distance from ground truth keypoint locations. SimGANoutperforms the model trained on real or synthetic images by 8.8%.

Implementation details of the NYU hand pose dataset: The input image size is 224× 224, filter size is 7 × 7, and 10 ResNet blocks are used. The discriminative net Dφis: (1) Conv7x7, stride=4, feature maps=96, (2) Conv5x5, stride=2, feature maps=64,(3) MaxPool3x3, stride=2, (4) Conv3x3, stride=2, feature maps=32, (5) Conv1x1,stride=1, feature maps=32, (6) Conv1x1, stride=1, feature maps=2

(b) The UnityEyes Gaze estimation:

Gaze estimation is a key to many human computer interactions. The figure aboveshows examples of real, synthetic and refined images from the eye gaze dataset.SimGAN achieves a significant qualitative improvement of the synthetic images, byrecreating real skin texture, iris qualities and sensor noise.


Research

Re

search


Conclusion:The key idea of SimGAN is to refine synthetic images so that they look like realimages, and at the same time preserve their annotation relevance. SimGAN S+Ulearning method, using an adversarial network demonstrated state-of-the-artresults without any labeled real data. In future, authors propose to researchnoise distribution to be able to create more than one refined image from eachsynthetic image, and to research refining videos as well as images.

Source code:All source code of the SimGAN method can be found here:Packages required are:Python 2.7TensorFlow 0.12.1SciPypillowtqdmAfter installing and downloading, refining a synthetic image with a pre-trainedmodel is done by typing:$ python main.py --is_train=False --synthetic_image_dir= "./data/gaze/UnityEyes/"

As for training, all images must be located in the samples directory. Then all youneed is typing:$ python main.py$ tensorboard --logdir=logs --host=0.0.0.0

For training with different optimizer and regularization term, you can use thefollowing command:$ python main.py --reg_scale=1.0 --optimizer=sgd


Research 43

Re

sear

ch

Quantitative results forappearance-based gazeestimation on the MPIIGazedataset with real eye images.The plot shows cumulativecurves as a function of degreeerror. 22.3% absolutepercentage improvement wasgained from training with theSimGAN output.

https://github.com/carpedm20/simulated-unsupervised-tensorflow

https://www.tensorflow.org/

http://www.scipy.org/install.html

https://github.com/python-pillow/Pillow

https://github.com/tqdm/tqdm


http://main.py/

http://main.py/

http://main.py/

RSIP Vision’s CEO Ron Soferman has launcheda series of lectures to provide a robust yetsimple overview of how to ensure thatcomputer vision projects respect goals, budgetand deadlines. This month we learn 7 Tips forProject Management in Computer Vision.

When we run a project in software, it isalways possible to control thedevelopment process and make it workin the right direction. Agile and otherskills help us keep on track. Today I will giveyou 7 additional tips, the specific target ofwhich are computer vision projects.

1. Review 100 images: it’s all aboutvisual results; thus, as a projectmanager, you should not stopviewing the presentation after afew satisfactory results. Ask thedeveloper for 100 images, so thatyou can get acquainted with thereal problem, the current resultsand the challenges still to besolved. Diligent review is essentialfor any computer vision project.

2. Team for new ideas: sincecomputer vision is an eclecticscience, borrowing frommathematics, physics, statistics,signal processing, deep learningand more, I recommend that youinvolve a large team (with differentsources of ideas) during thebrainstorming and at each crucial

step in the algorithmic decisions.

3. Matlab to C++: it is quite typical tostart with a Proof of Concept inMatlab. On the other hand,migration to C++ might prove to bea difficult task, because theinfrastructure of the functions inthe libraries are different, makingthe fine-tuning of the algorithmproblematic. I recommend to startporting some of the functions andsome of the infrastructure to C++environment as soon as it ispossible.

4. Deep learning in different ways:there are many ways to implementdeep learning. When you want toachieve a definite progress in yourproject, you might decide to splitthe project in a few stages, takingadvantage of those stages in whichdeep learning gives you goodresults (even if it doesn’t solve thewhole problem in one net); andthen take advantage of the stableand robust results to step into thenext stages of the problem solving.

7 Tips for Project Management in Computer Vision

Involve a large team during the brainstorming and at each crucial step in the algorithmic decisions

Man

ageme

nt


Project Management Tip


5. Limit you effort to the data setsize: in many cases, datasets forthe first stages of the projects arelimited, until we get more data. Iadvise not to spend excessive R&Dresources as long as the dataset issmall, lest we incur in overfittingand waste of time solvingirrelevant problems.

6. Algorithmic Design Document(ADD): this detailed document,describing all the algorithms, withtheir assumptions, parameters,results and limitations, is essentialto keep the development groupfocused on the task and to ensure

efficient communication. Youshould keep it concise, relevantand updated! This will enable youto solve the problems and get theright help at any stage of theproject.

7. Do not correct algorithms withmore and more algorithmic levels:sometimes, especially when youare working with young teams, youneed to make sure that eachalgorithmic addition has beenvalidated and debugged correctlybefore you proceed to the nextstage. This is necessary becauseyoung programmers tend to addadditional stages without testingand correcting the work alreadydone. This is very risky, since itmight make the algorithmic partexcessively complex and devoid ofany mathematical background.

Keep the Algorithmic Design Document concise, relevant

and updated!

Man

agem

en

t


45Project Management Tip

Take advantage of the stable and robust results to step into the next stages of the problem solving

On September 11, Ron has lectured at the Boston Imaging and Vision (BIV) meetup.


The problem of medical imagesregistration is a common problem inmany medical applications where apatient moves during a procedure,scans are taken at different times,different modalities should becombined etc. Specifically, at RSIPVision we concentrate on the use ofdeep learning for registration inmedical operations and surgeries.Usually, deep learning can be used todirectly predict the transformationparameters for registration usingregression networks. Alternatively, itcan be used to estimate anappropriate similarity measurebetween two images.

When considering runtime, running adeep network for registrationpurposes can be time consuming andclassic approaches might be faster.However, a less complex network canbe used to help decide what parts ofthe image are important. As deeplearning showed great success indetection, segmentation, andclassification, we make use of deepconvolutional networks to localize the

regions of interest in an image.

For example, cataract is the leadingcause of blindness in the world andcataract surgery is the mostcommonly performed operationworldwide. A surgery can improve theeyesight for patients suffering fromcataract by removing the “clouding” ofthe eye lens and inserting an implantinstead. In cataract surgery there is aneed to follow the eyeball’smovements (which include rotations)as well as the physician’s tools. In thiscase, there are two objects: 1) theretina which is used for theregistration; and 2) the physician’stool which needs to be determinedbut not used for registration. Thus,deep convolutional networks aretrained to separate the region ofinterest for registration and otherregions, as well as to identify therelevant tool during the surgery.

Another similar example is retinalsurgeries which also include the retinaas the region of interest and thephysician’s tools as can be seen in thefollowing figure:


Project

Deep Learning for Medical Images Registration

Every month, Computer Vision News reviews a successful project. Ourmain purpose is to show how diverse image processing applications canbe and how the different techniques contribute to solving technicalchallenges and physical difficulties. This month we review RSIP Vision’smethod in Deep Learning for Medical Images Registration, based onadvanced image processing algorithms designed to support surgeons intheir task. RSIP Vision’s engineers can assist you in countless applicationfields. Contact our consultants now!

by RSIP Vision

“Deep Learning can be used to directly predict the transformation parameters for registration using

regression networks”

Pro

ject


http://www.rsipvision.com/contact-us/


Project 47

The network can be quite simple if it istrained to classify patches and bydown-sampling or using high stridevalues the runtime is improvedsignificantly. This approach helped usimprove the registration results indifferent Retinal related projects. Itcan be used as a pre-processing stepfor classic Registration/Homographymethods which may use correlation,mutual information or other similaritymeasure according to the application.It can also be combined with a deeplearning based registration, making iteasier on the registration network to

concentrate on the regions of interest.

By creating an easy-to-use, generic UIwe extract annotated images fast andtrain the relevant convolutional neuralnetworks to get the regions of interestand separate the different objects.Then for each frame we find therelevant regions and mask out non-relevant regions for the registrationprocess.

You can read on our website aboutother Deep Learning projectsconducted by RSIP Vision in severalfields of application.

Pro

ject


http://www.rsipvision.com/deep-learning/

The guide is divided in 2 parts: first, we’ll see how to install and configure a VMinstance on Google Cloud. Then, we’ll install the FirstAid software package,which will make running and training the latest deep learning methods formedical imaging segmentation and classification nice and easy. Of course, youcould install and run FirstAid on your own PC, but unless it has very strongcomputing capabilities and a GPU, you’re better off setting it up on GoogleCloud, which you can easily do, as we’ll explain right away.

A. Setting up a virtual machine on Google Cloud.

Google Cloud is a virtual machine service, which allows anyone to get a virtualmachine for remote execution. When you set up your account you’re given $300for initial experimentation. We can use a little of this gift to check out FirstAid’scapabilities.

We will guide you step-by-step how to set up your own virtual machines (calledVM instances on Google Cloud):

1. Go to cloud.google.com and open up a basic account (or upgradedaccount if you want to use a GPU).

2. Once you’ve set up your account, in the menu select compute engine andin the menu that pops up select VM instance.

3. In the VM instances screen, at the top, select the ‘create instance’button. In the form that opens:

a. Give a name to your VM instance.

b. In the machine type select 4 vCPUs.

c. Optionally, for better performance, if you set up an upgradedaccount: Press the ‘customize’ button on the right, and in themenu that opens, under GPUs select 1 GPU.


We Tried for You

Too

l

FirstAid on Google Cloud in 30 minutes!

We Tried For You once again! This time, we willdemonstrate how in under half an hour, you can setup a virtual machine on Google Cloud that will runand train the latest deep learning models forsegmentation and classification specialized forhandling medical images.

by Assaf Spanier


http://cloud.google.com/

d. In the Boot disk click on ‘change’ and in the menu that opensselect Ubuntu 16.04.

e. Under Firewall, select ‘allow http’ and ‘allow https’.

f. Finally, press create.

4. Coming back to the VM-instance screen, pick the line of your just-createdVM-instance. Press ‘SSH’ and in the menu that pops up, select ‘Open inbrowser window’.

5. In the console, install the following packages to run Python andTensorFlow:

i. $ sudo apt-get update

ii. $ sudo apt-get install python-pip

iii. $ sudo pip install tensorflow

iv. $ sudo apt-get install ipython

v. $ sudo pip install --upgrade pip

6. Now we want to set up a VNC server, which will enable us to use a GUIinterface with the VM instance.

a. First, we’ll Install:

i. sudo apt-get install gnome-core

ii. sudo apt-get install gnome-core

iii. sudo apt-get install vnc4server

b. Now we want to configure the VNC server, running the vncservercommand creates the configuration files we want to makechanges to.

c. Run vncserver (please note that you will need to select apassword here). And kill it with a command killall Xvnc.

Too

l


We Tried for You 49



We Tried for You

Too

l

d. Now config the script by typing vi .vnc/xstartup, and change thefile to the following:

e. Now run the VNC server again by typing vncserver.

f. One last thing to do before you can use your VM instance, in theFirewall settings you need to open the port for the VNC client toconnect to your VM instance:

i. In the VM instance window select the machine you created.

ii. In Network interfaces click on ‘default’ under Network.

iii. And in the menu that opens, click on Add firewall rule

iv. You need to fill out 2 fields: Source filters, under ‘IP range’enter 0.0.0.0/0 Protocols and Ports, enter tcp: 5901

#!/bin/sh

# Uncomment the following two lines for normal desktop:

# unset SESSION_MANAGER

# exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup

[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

xsetroot -solid grey

vncconfig -iconic &

x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

x-window-manager &

gnome-panel &

gnome-settings-daemon &

metacity &

nautilus &



We Tried for You 51

d. Now config the script by typing vi .vnc/xstartup, and change thefile to the following:

7. On your local machine download and install the VNC client from this URLa. In the window that opens enter the IP address of your VM

instance. A window asking for a password will open, you need touse the password you selected when you first ran the VNC server.

B. Now you’re all set to start with the FirstAid package:

The FirstAid package was created for work with deep neural network, especiallytuned for medical imaging. First, we will guide through the setup process. Then,we’ll go through the process of using FirstAid’s demo.

For FirstAid Installation, please type the following in the command windows:

a. cd ~

b. sudo pip install h5py

c. git clone https://github.com/yidarvin/FirstAid.git

d. sudo pip install matplotlib

e. sudo pip install scipy

f. sudo pip install sklearn

g. sudo apt-get install python-tk

The package comes with the following 6 common deep learning segmentationand classification networks: Le_Net: 5 layers (4 for segmentation) Alex_Net: 8layers (6 for segmentation) VGG_Net: 11, 13, 16, and 19 layer versions (9, 11, 14,and 17 for segmentation, respectively) GoogLe_Net: 22 layers (22 forsegmentation) InceptionV3_Net: 48 layers (48 for segmentation) Res_Net: 152layers (152 for segmentation)

FirstAid supplies the following 2 main scripts, the first for training classificationnetworks, the second - segmentation networks. The python scripts to call are:train_CNNclassification.py, train_CNNsegmentation.py respectively.

Too

l


https://www.tightvnc.com/download.php

https://github.com/yidarvin/FirstAid.git

http://cnnclassification.py/

http://cnnsegmentation.py/

Medical images has a special structure which is the centrality of the patient.Various images all supply partial information which we want to aggregate toarrive at a single diagnosis for the patient. The patient-centric structure has thefollowing format:. The main image stored under the key "data", thesegmentation under "seg," please note that the input image must be a square.FirstAid supplies a script (img3h5.py) which converts the images into patient-centric structure (h5 format).

Now, let’s look at FirstAid’s Demo:

The demo of FirstAid it a toy example classifies faces into “with glasses” and “noglasses” categories. To download the demo type the following command in theconsole:

git clone https://github.com/yidarvin/glasses.git

For running the demo please update the glasses.sh script according to your locallibraries, as you can see below:


We Tried for You

Too

l

folder_data_0 (e.g. training folder)

● folder_patient_0

○ h5_img_0

■ data: (3d.array of type np.float32) 2d image (of any channels) of size $n \times n$

■ label: (int of type np.int64) target label

■ seg: (2d.array of type np.int64) target n-ary image for segmentation

■ name: (string) some identifier

■ height: (int) original height of the image

■ width: (int) original width of the image

■ depth: (int) original depth (number of channels) of the image

○ h5_img_1

■ etc...

○ etc...

● folder_patient_1

○ h5_img_0

● Etc…


http://img3h5.py/

https://github.com/yidarvin/glasses.git

http://glasses.sh/


We Tried for You 53

The last line in the script calling from the CNN classification. There are four mainfilepaths of importance:

● --pTrain training: used for training the model

● --pVal validation: used to help dictate when to save the model during training

● --pTest: testing: held out test set with associated ground truth data for grading

● --pInf: inference: images without ground truth to use the model on

● --pModel: model savepath for saving and loading

● --pLog: log filepath

● --pVis: figure saving filepath

Network Specific Definitions

● --name: name of experiment (will be used as root name for all the saving)default: 'noname'

● --net: name of network default: 'GoogLe'

○ valid: Le, Alex, VGG11, VGG13, VGG16, VGG19, GoogLe, Inception, Res

● --nClass: number of classes to predict default: 2

● --nGPU: number of gpu's to spread training over (testing will only use 1 gpu)default: 1

Too

l

declare -r path_FirstAid=~/FirstAid/train_CNNclassification.py

declare -r path_train=$PWD/h5_data/training

declare -r path_val=$PWD/h5_data/testing

declare -r name=glasses

declare -r path_model=$PWD/model_state/$name.ckpt

declare -r path_log=$PWD/logs/$name.txt

declare -r path_vis=$PWD/graphs

python $path_FirstAid --pTrain $path_train --net Alex --pVal $path_val --name

$name --pModel $path_model --pLog $path_log --pVis $path_vis --nGPU 1 --bs 8

--ep 50 --nClass 2 --lr 0.001 --do 0.5


http://cnnclassification.py/

Hyperparameters

● --lr: learning rate default: 0.001

● --do: keep probability for dropout default: 0.5

● --l2: L2 Regularization default 0.0000001

● --l1: L1 Regularization default 0.0

● --bs: Batch Size (1/nGPU of this will be sent to each gpu) default: 12

● --ep: Maximum number of epochs to run default: 10

The output of this run should look on your screen as the following figure:


We Tried for You

Too

l

This is how, in under half an hour, you can set up a virtual machine on Google Cloud that will run and train the latest deep learning models for segmentation and classification specialized for handling medical images


SIPAIM - Int. Symp. on Medical InformationProcessing and AnalysisSan Andres - Colombia Oct 5-7

RE•WORK Deep Learning SummitMontreal, Canada Oct 10-11

Vipimage on Computational Vision and Medical Image Proc.Porto, Portugal Oct 18-20

ICCV 2017Venezia, Italy Oct 22-29

EMMCVPR 2017 - Energy Minimization in Computer VisionVenezia, Italy Oct 30-Nov 1

AAO 2017 - American Academy of OphthalmologyNew Orleans, LA Nov 11-14

AI Expo North America - Delivering AI for a Smarter FutureSanta Clara, CA Nov 29-30

PReMI 2017 - Pattern Recognition and Machine IntelligenceKolkata, India Dec. 5-8

International Congress on OCT Angiography and Advances in OCTRoma, Italy Dec 15-16

Did we miss an event?Tell us: [email protected]

Dear reader,

Do you enjoy readingComputer Vision News?Would you like to receive itfor free in your mailboxevery month?

You will fill the SubscriptionForm in less than 1 minute.Join many others computervision professionals andreceive all issues ofComputer Vision News assoon as we publish them.You can also read ComputerVision News on our websiteand find in our archive newand old issues as well.

We hate SPAM andpromise to keep your emailaddress safe, always.

FREESUBSCRIPTION

Subscription Form(click here, it’s free)

Website and Registration



Dear reader,

How do you like Computer Vision News? Did youenjoy reading it? Give us feedback here:

It will take you only 2 minutes to fill and it will helpus give the computer vision community the greatmagazine it deserves!

Give us feedback, please (click here)

FEEDBACK








Upcoming Events


Next month your will read a preview of AI Expo North America

in Santa Clara, Nov 29-30

mailto:[email protected]

http://www.rsipvision.com/ComputerVisionNews-2017October



http://iccv2017.thecvf.com/

http://www.dsi.unive.it/emmcvpr17/




http://www.computervision.news/feedback/

https://www.ai-expo.net/northamerica/

http://www.apmeetings.com/singolo_evento.php?id=94

https://paginas.fe.up.pt/~vipimage/nav/aims.html

https://www.re-work.co/events

http://sipaim.org/home

http://www.isical.ac.in/~premi17/index.php


https://www.aao.org/annual-meeting

Improve your vision with

The only magazine covering all the fields of the computer vision and image processing industry

A publication by

Subscribe(click here, it’s free)

Map

illary

RE•W

OR

KG

auss Su

rgical






Exclusive Interview with Yoshua Bengio!€¦ · Learning from Simulated and Unsupervised Images...

Documents

Transcript of Exclusive Interview with Yoshua Bengio!€¦ · Learning from Simulated and Unsupervised Images...