October 10, 2019T R A N S C R I P T

FDA’S USE OF AND GUIDANCE ON REAL-WORLD DATA

Sam Roosz Good morning everybody. I am so excited to be here today with you all to introduce Dr. Janet Woodcock. Dr. Woodcock has had a long career of public service starting as a clinician where she worked in the Veterans Affairs Administration and then progressing in 1986 to the FDA where she has been a tireless advocate for patient safety.

More recently, she’s taken on a leading role in integrating real-world data into the practices of the FDA to both develop new regulatory pathways to bring drugs to the market, as well as to bring tools to bear to improve patient safety through surveillance. So we’re quite excited to have her here today to both share some perspectives on how FDA is thinking about these data, as well as the recent rollout of 21st Century Cures. Janet, thank you so much for being here with us today.

KEYNOTE

MODERATOR

Janet Woodock Director, Center for Drug Evaluation and Research, FDA

Sam Roosz Co-founder & General Manager, Life Sciences, Datavant

Janet Woodcock Happy to be here.

Sam Roosz So my fi rst question for you is, that we have heard quite a bit in recent months and seen much momentum from FDA in the use of real-world data and real-world evidence. Can you tell us a little bit more about why real-world data is so important to FDA?

2

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Janet Woodcock Well, the traditional development programs, as probably most of you know, have given us enough information to approve a drug and get it on the market. But there’s always still so much we don’t know. So that’s one thing: can we really learn from all the records that are being collected in healthcare, and can we understand more about what drugs to use, when, what are the side effects and in who, particularly. Because traditional drug development programs usually are looking at population mean results. And so we don’t really get much insight into who to use a drug in and who not to use a drug in, which drug works best for different people and so forth.

And then the second issue is of course, drug development is still a very fraught enterprise and nine out of 10 drugs fail in development. This drives up, of course, the cost. [The cost] of development is extremely high. So are there ways that we can capture and utilize all the data that’s usually collected in healthcare to make the clinical evidence generation, that whole enterprise, the clinical trial enterprise more effective and more effi cient – get the cost down, but actually learn more to help our patients?

Sam Roosz

Janet Woodcock Well, the traditional development programs, as probably most of you know, have given us enough information to approve a drug and get it on the market. But there’s always still so much we don’t know. So that’s one thing: can we really learn from all the records that are being collected in healthcare, and can we understand more about what drugs to use, when, what are the side effects and in who, particularly. Because traditional drug development programs usually are looking at population mean results. And so we don’t really get much insight into who to use a drug in and who not to use a drug in, which drug works best for different people and so forth.

And then the second issue is of course, drug development is still a very fraught enterprise and nine out of 10 drugs fail in development. This drives up, of course, the cost. [The cost] of development is extremely high. So are there ways that we can capture and utilize all the data that’s usually collected in healthcare to make the clinical evidence generation, that whole enterprise, the clinical trial enterprise more effective and moreefficient – get the cost down, but actually learn more to help our patients?

Excellent. Thank you. Can you tell a little bit more about how we came to this place? What are the factors that led FDA’s current focus on real-world data? And what has had to happen in terms of the technology, the policy, the clinical practice to bring us to where we are today?

3

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Sam Roosz

Janet Woodcock Well, I hate to tell this audience, but I think we’re just starting. We’re just stepping onto the road, or maybe we’re on the path that leads to the road. I see Rob Califf here in the audience, and he might disagree with me here. He’s trying to get us onto the super highway, right? But right now, we’re kind of on the trucking path of this. But that doesn’t mean we should despair. One thing about this type of revolution that we’re undergoing is it can move very quickly and technologies and so forth can move it along. So I think the fact that we’re taking the fi rst steps FDA has approved, numerous supplements

Janet Woodcock

Sam Roosz Excellent. Thank you. Can you tell a little bit more about how we came to this place? What are the factors that led FDA’s current focus on real-world data? And what has had to happen in terms of the technology, the policy, the clinical practice to bring us to where we are today?

Well, I think one of the major events was a transition in healthcare itself into electronic record keeping digital data, when it was all in charts, in doctor’s offi ces, paper charts. We had to do archeological expeditions, literally, to fi nd out what was going on. And the same is true at FDA. Don’t forget, we used to get tractor trailers’ worth of paper and that’s what we would have to trawl through to do a review of a drug. Now, we get digital, we get data and we get datasets that are standardized. Now, [in] the healthcare system, unfortunately the data are much more chaotic even in digital format, and that’s a problem. But it’s out there and it can be utilized.

So I think that transformation, we have the lab data, we have data in different areas, we can utilize those data. So that was one thing that happened. And then the other things are just the digital revolution in general, people are more used to that. So we are having wearables, a lot of people have iPhones, they have all sorts of technology, we have telehealth that’s developing… so all these things are coming together to provide rich evidence, data sources that we can turn into evidence hopefully. And that’s going to be the task to take all this data out there and turn it into something that’s actionable that we can use, make decisions on, improve the health of the country.

Where are we on this journey? If you think about, from the very beginning of initial uses of real-world data to an imagined future where we’re putting it to practice in all of the ways that you imagine… Where are we on that road and what does it look like from here?

4

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Sam Roosz

Janet Woodcock Well, the long-term vision which was articulated quite some time ago by then, Institute of Medicine, now NASM, is the learning healthcare system: that we are able to make all the information we collect actionable and make us have quick knowledge turns and be able to assimilate that learning from treating patients and rapidly transform it back into the best practices that then everybody will have access to hear about and understand, and it will be quickly actionable.

I can tell you we’re very, very far from that, but that’s the long-term vision. I think in the shorter term, what we’re going to see is both in the drug development and understanding of use of pharmaceuticals and other intervention space and also in the healthcare system practices, right? The best practices and for delivering healthcare where we’re already generating a lot of insights about what to do, what’s the best way to do this, or that, or the other thing or which drug works better, in whom, and so forth. And I think that will continue to grow.

At the same time out in the periphery, we’re going to see the growth of telehealth. We’re going to see the use of remote access, wearables, all these things and we’re going to begin to learn. We are beginning to learn the performance characteristics of using these different technologies. Then we can begin to incorporate that both in healthcare delivery as well as in evaluating new technologies.

Sam Roosz Excellent. So as we think about the barriers to overcome to truly bring real-world evidence to its full potential, what are the barriers that need to be

and many drugs based on real-world evidence already. These are usually for rare conditions or most recently, say, male breast cancer for example, which doesn’t happen very often, but as just as devastating as any other cancer. And so we were able to use evidence generated from use to add that indication to the label. So we’re seeing this starting. And each year, I think the growth will be not just linear, it will probably be exponential growth.

And if you’re to imagine the end of that road or rather, something that is on the near horizon of how real-world evidence can be used at scale, and once we have the various practices in place to make effective use of it, what does that world look like?

5

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

overcome by industry, and what are the barriers that need to be overcome by regulators in working with that data in the context of clinical development?

Janet Woodcock Well, I think industry and regulators probably share the same set of problems.Number one, the data out there are not necessarily fi t-for-purpose right now. So we’re doing work with Laura Esserman at UCSF called OneSource, and with the ASCO to try and ... for example, for cancer. What does it say in the chart about your cancer? Is it right? Do you have the right cancer written down in your chart? Sometimes, the answer is no, or the wrong stage or... are the biomarkers in there? Are they entered correctly? Sometimes no. Well, that’s not actionable. It has to be right.

And of course, that’s very important information for the patients. So there’s an attempt to develop a core dataset of critical information for somebody who has cancer, make sure that’s in the person’s data fi le and that it’s in there correctly. Because the fi rst time, you want to get the information in. What we see now, we’re collaborating say, with Flatiron and other groups, you know – they have to curate it to get it right. Well, that’s not going to fl y over the long term I think. It ought to be right the fi rst time. We need a healthcare system that can get the information in there correctly.

Okay, a second major problem, which I think we can overcome, is the reason... people often don’t understand, the reason we do randomized controlled trials is because there are sources of confounding that get into observational data that we can’t correct for with all kind of fancy math or algorithms. We just can’t, because we don’t know what they are. But what some of the things we’re exploring, in some certain trials that are ongoing now, can you randomize people in practice easily, and then collect the real-world outcomes data? And can you do all that very easily, without all this very elaborate clinical trial apparatus? And can you still get a valid result that is actionable?

And so that’s a barrier that has to be overcome. A lot of folks, and I imagine a lot of people in this room, feel that we can rely upon observational studies with lots of corrections, right? And that may be true if the treatment effect is very large, if the difference is very extreme, and so forth. But in many cases, we’ve seen two observational studies – we see this all the time – that contradict each other. Even in using the same dataset with slightly different methods, we get different answers. Do you really want us to make a decision based on that? We need information we can rely upon to make decisions because our regulatory decisions really have big consequences, and that’s true for healthcare system and healthcare delivery as well, it has big

6

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

consequences.

I’m not saying that observational or real-world data can’t be made reliable and can’t be utilized, but I think we have to recognize that’s one of the big elephants in the room is… we’re making causal inferences. We still have a way to go.

Sam Roosz We hear that narrative quite a bit in news articles and in industry discus-sions around deployment of real-world data and clinical developments, that the fi delity of data entered into an EMR is quite different from the fi delity of data entered into in an EDC. So as you think about the various folks that are working with real-world datasets, coming from claims, from EMR, from lab that are used in routine clinical care – what are the pieces that they can put into place in making these data useful to FDA when it is submitted to you as a part of a package under 21st Century Cures?

Janet Woodcock We’re collaborating with a lot of groups trying to sort this out, what can be done? As I said, we’re working on OneSource, we’re working with different groups that are trying to raise the reliability of that evidence to the point. And it may require some type of curation. My favorite approach is the sort of network approach that you get in standing networks of investigators. And they work on that reliability so that it can be raised to the level that you can believe it.

Of course, we have extremely long experience with using observational datasets because of Sentinel and the other data sources that we use, and we have made many regulatory decisions based on those types of data. But generally they’ve generally been in the safety area. We do have a lot of experience in the reliability of those data and what the pitfalls might be.

But we have embarked upon an extensive set of collaborations based on 21st Century Cures and what fl owed out of that. We’re collaborating with a large number of people where randomized controlled trials are ongoing. We’re doing a companion observational study, and we’ll see what the results are. We hope to have 20 retrospective ones done by next year and then we’re probably going to have seven prospective ones. In which case, of course, the real-world evidence study will get done a lot quicker than the randomized controlled trial, but at the end of the day, we can compare the performance of both of those. And we aren’t going to say one is right and one is wrong frankly... But it will be very interesting to see how they compare, what kind of answers they give.

7

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Sam Roosz You referenced Sentinel and some of the post-market work that FDA is doing to look at patient safety for on-market assets. With the award of the new fi ve-year Sentinel contracts, what can we expect in terms of evolutions of Sentinel in these current practices, of FDA’s use of real-world data to guard patient safety in the United States?

Janet Woodcock Well, I think there’s only so far you can get with claims. So with the new Sentinel award, we hope to really get heavily into the electronic health record. We always had access to electronic health records within Sentinel, but here, we hope to use it more as a resource rather than the archeologi-cal expedition type where you have to go and do chart review and have an expert determine, did the person have an MRI or not, and all these sorts of things. Without incorporating electronic health record, you really have a very incomplete idea of what happened.

Another one of the big problems with the current real-world evidence that we have is that death is usually not recorded in the electronic health re-cord, unless it happened in a hospital. So we have very incomplete records, just using those sources, of when people died. They just drop out of the claims, they drop off their health record, and we don’t know what hap-pened. So Sentinel is going to really be aggressively exploring linkage with other data sources including registries, different death record approaches, and so forth. Also, in the new contract, we are aggressively going to be pursuing additional methodologies for utilizing data. So I think it’s very ex-citing. It’s a game-changer. We’ve declared victory on the current Sentinel and its methods and we’re going to try to take it to a next level.

Sam Roosz Thank you. I want to turn to some questions from the audience. Question from Daniel: “Regarding the high cost of drug development and the con-vergence of explanatory and pragmatic trials, what is the future of synthet-ic placebo groups or external comparators and registrational randomized controlled trials?”

Janet Woodcock We use external control groups now. It’s usually for rare diseases where it’s really hard to get a concurrent randomized control group. So we have multiple approvals over the years of drugs for rare diseases where we’ve used historical data basically, an external control group constructed from either registries or patient experience. And we have even approved drugs on patient experience that would usually be from during the IND, but it’s basically a treatment use where we look at the patient’s course before and the patient’s course afterward and we’re able to say that this drug was

8

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Sam Roosz For industry stakeholders who are interested in collaborating with FDA on leveraging these novel approaches of integrating real-world evidence either into their own studies or supporting sponsors and integrating real- world evidence into their lifecycle planning for their assets, what are the things that industry can be doing today to partner with FDA?

Janet Woodcock Well, we have partnered. We have multiple partnerships going on with in-dustry. Often, they’re doing randomized trial. They want to put a real-world evidence, some study onto it. We’re part of that, we’re participating, giving advice, learning from what’s going on. We’ve also given advice on about 21 instances where industry wants to incorporate real-world evidence as part of the development program, but that’s within the formal regulatory process we’re giving that advice.

So we’re very interested in partnering with industry, collaborating on different studies and looking at everything from wearables, their performance to use of real-world evidence to recruit patients and how you can do that easily, use of the electronic health record for some of the case report, rather than having a separate electronic data capture that people have to enter the data in. Or even a separate study, a parallel study that looks at using an external control group. So any of those, all of those we’re interested in and we’re having a randomized group comparison. And then using an external control group will really give us some idea – especially, this should all be defi ned prospectively and we have developed the rules of the road for doing this, so that everything is prospectively laid out.

And you talk about how much variance you expect between a control arm, the randomized control arm and the historical control arm. So it really is an experiment that we’re doing and I think only by getting these hard data in hand and having a lot of instances like this, that will make us be able to move forward the fastest.

effective. So that was for rare diseases.

Now, a lot of people have noticed a lot of disease subsets are becoming morerare, right? For example in cancer, because as we understand the molecular pathogenesis, we understand that, sure, this person has lung cancer, but there are like 50 different biomarkers and it subsets into 50 different groups. So there is interest and there is work going on in looking at how we can construct external control groups in some of those circumstances for the very rare subsets.

9

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Sam Roosz Turning to another question from the audience. So one of the pieces that you mentioned is for current EMRs, the variance that can be seen in the inputs to those, and the challenges that that can create for FDA in querying those data—

Janet Woodcock And probably the patients too. I would want my record to be correct, but…

Sam Roosz Mine as well. So the question from the audience is, “There’s much frustration around EMR silos. What is the FDA’s take on a consistent structured data output for EMR records?”

Janet Woodcock And probably some type of standardization which as we were talking about before this, everyone resists ... I deal with this every day in my job. Everyone resists standardization, and doctors especially, right? They want to do things their own way and they want to put it down their own way. I’ll never forget, Peter Bach once tried to do something when he was with Medicare and he tried to have lymphoma standardized for Medicare so that they could see the outcomes for the patients with lymphoma. He’s an oncologist and he said, some doctors just seemed completely different than what they wrote down and he asked them, and they said, “Well, I don’t agree with the consensus defi nition of stages of lymphoma. I have my own stages.”

So to some extent, that’s what we’re dealing with. People in healthcare are ... it is very, “It’s my patient, my way of dealing with things,” and … I think we have to sell people on the value of the return of that evidence. If you put it down a standard way, you can get something back that will be very valuable to you. And the same pertains in regulatory work. We need to standard-ize what we do so that then we understand what’s going on better. It’s an uphill battle. There’s much discussion of course in healthcare circles about what the electronic record is doing to clinicians and doing to the doctor-pa-tient relationship. And it doesn’t have to be that way. That can be fi xed. But we’re really going to have to focus on that.

Sam Roosz Thank you. And I see that our time has fl own by. So I want to leave you with one fi nal question and perhaps the most important of this discussion. As we think about real-world data, what is the benefi t that accrues to patients from our use of these datasets?

10

T R A N S C R I P T

F U T U R E O F H E A LT H DATA .C O M

Janet Woodcock Better care. And as I said, when we approve a drug, often, there’s a large amount of information we don’t know about that. And in particular, because we’re talking about population means, we don’t know in whom the drug works best and in whom it doesn’t work very well. We know there’s a spread, but we don’t know why. And despite all the money that’s spent on drug development, huge, $1 billion say, to develop a drug or something like that, those kind of basic questions aren’t answered. Our only hope for getting that answered and really individualizing therapy and having better care delivery is to utilize information from healthcare and continue to learn, and learn after drugs are approved and are actually in use.

And that will help patients because many people of course are exposed. We know people don’t respond to the drugs, but we don’t even know why or who they are. So we try the drugs empirically and then we may try another drug and so forth. And this degrades the quality of care and the outcomes for patients.

Same on the safety side. If we could fi gure out, which were beginning to do, and often, this is from real-world evidence today: who really shouldn’t take a drug, right? Who shouldn’t be administered a drug? That would help tremendously too and improve that benefi t risk and the outcomes for patients. So the prospect for improving healthcare and improving outcomes in health-care is just tremendous, but the barriers are also high and we all are going to have to work together to fi x all that.

Sam Roosz Excellent. On that note, thank you so much for your time today.

Janet Woodcock Happy to be here.

Sam Roosz It’s been a pleasure chatting with you.

Janet Woodcock Thank you.