EP550 Lecture/Demonstration 16 and Homework 10, Due Friday, March 25

The purpose of this exercise is to teach you how to create Markov models using the TreeAge

2011 software program and to use what you’ve created to illustrate additional features of cost-

effectiveness analysis. The exercise uses the lupus example from class.

Create the model

1. Open TreeAge, clear any material from the center panel where decision trees are created and

modified, and maximize this panel. Create a decision node by clicking “File/New.../Blank Tree

Diagram/OK.” A square decision node should appear. If not, in the “Tree Nodes” box to the right, double

click “Decision.” Label the decision node “Lupus.” Right click on the decision node, and add two chance

nodes. Label the upper chance node “Usual Care.” Label the lower chance node “Intervention.”

2. Change the Usual Care chance node to a Markov node: right click the node and click on “Change

type/Markov.” (Note the difference between selecting the node and selecting the subtree attached to the

node.) Make the display simpler: click on “Edit/Tree Preferences/Display/Variables/Markov info,” select

“Hide Definitions,” and click “OK.”

3. Add four branches to the Markov node. (“Node/Add branch” or “right click/Add branch.” The Lupus

node must be selected each time you repeat this process.)

4. Click on the Markov node, select its subtree (“Subtree/Select Subtree”), copy the subtree (“Edit/Copy”),

select the first of the four branches of the Markov node, and paste the selected subtree to this branch

(“Edit/Paste”).

5. Label the four branches of the Markov node (above the line): “Remission, Active, Flair, Death.”

6. Select the first branch of the “Remission” node. Change this node to a terminal node (right-click the

node, click “Change Type” and select “Terminal”), when the box pops up select the “Remission” state for

this node to jump to, and click “OK.” Change the other three nodes to terminal nodes and select the other

three states for them to jump to.

7. Click on the “Remission” chance node that immediately follows the Markov node, select its subtree

(“Subtree/Select Subtree”), copy the subtree (“Edit/Copy”), and paste the subtree to the second and third

branches of the Markov node (select the terminal node and then select “Edit/Paste Subtree”).

8. Change the fourth branch of the Markov node (“Death”) to a terminal node.

Your tree should look like the tree on the following page.

2

Add information about the transition probabilities using the following table from the lecture notes:

9. Start by clicking immediately underneath the”Active” terminal branch of the “Remission” branch of the

Markov node, typing in “0.41,” and then “Enter.” Type in “0.00" for the transition probabilities for the

terminal branches “Flare” and “Death.” Type in “#” for the transition probability for the terminal node

“Remission.” The reason for this way of indicating probabilities will become obvious later in the exercise.

Add the other transition probabilities to the other terminal nodes. Use the pound sign (#) for the transition

probabilities for “Death” in the other terminal nodes.

Add information about the initial distribution of patients

The following table is from The lecture notes:

10. To start, click underneath the “Remission” branch of the Markov node and type in “0.10.” Add

numbers that correspond to the “Active” and “Flare”initial states, and use “#’ to indicate the value for

“Dead.”

Add values for the outcome variables

11. In this example, we will use two outcome variables. The first outcome variable will be a cost variable.

That variable is created using the probability of hospitalization, which is described in the following table

from the lecture notes:

3

To create the outcome variable, multiply the mean number of hospitalizations times the average cost of

hospitalization (cHosp = $10,000). To add the dollar cost of transitioning from remission to remission,

select the “Remission” terminal node of the “Remission” subtree coming out of the Markov node. Select

Node/Values List/Markov Info, or go to the top of your screen, click on the symbol that represents Markov

information (includes an “M” with a circle around it). Maximize the box that opens, and click on the row

that corresponds to Transition Reward 1, which is the row labeled “Trans Rwd” that follows the row labeled

“All Sets (Unused+Active).” In the column labeled “Value” type in (cHosp*.05)/((1+r)^ stage). Note that

dividing by (1+r)^_stage discounts the outcome. Push “Enter,” and a box will pop up. Select “General.”

W hen asked for descriptions, cHosp is the “Cost of a hospitalization, and r is the “Discount rate” and

stage is “Cycle number.” For each variable make sure the button for “Show definitions in tree” is

selected.

The second outcome variable measures the years of life. One year in any state except death will equal

one, and one year in the state of death will equal 0.5, because we assume death at midyear.

To recognize that years of life have different qualities, use the hypothetical quality data from the following

table in the lecture notes:

4

To add the number of QALYs associated with transitioning from remission to remission, select the

“Remission” terminal node of the “Remission” subtree coming out of the Markov node. Select

Node/Values List/Markov Info, or go to the top of your screen, click on the symbol that represents Markov

information (includes an “M” with a circle around it). In the box that opens, click on the row that

corresponds to Transition Reward 2, which is the row labeled “Trans 2” that follows the row labeled “All

Sets (Unused+Active).” In the column labeled “Value” type in 0.9. To discount this value, divide by (1+r)^_

stage, so the whole expression is: 0.9/((1+r)^ stage). Push “Enter.”

To display outcome information in the tree, go to “Edit/TreePreferences/Display/Variables/Markov Info and

select “Show Definitions.”

To prepare the tree for a cost-effectiveness analysis, go to “Edit/TreePreferences/Calculation/Calculation

Method/Cost-Effectiveness, and make sure “Cost payoff” is number 1 and “Effectiveness payoff” is

number 2. If the program says “C-E calculation method is not active. Do you want to activate it now?”

click “Yes.” Click on “Numeric Formatting,” and three boxes should appear. In all three, select 2 “Decimal

places,” “Add trailing zeros,” “Use thousands separator,” and “Show numbers” exactly. In the top most

box select “Units : Currency.” In the middle box, select “Units: Custom suffix” and “Prefix/suffix: QALYs.”

In the lowest box select “Units: None.”

Save your partially completed Markov model in *.trex format (“File/

save As ...”) in a file labeled with your last name followed by the number 8, for example, Williams8.

Repeat these steps to add discounted values for cost as Transition Reward 1 and discounted values for

Quality Adjusted Life Years as Transition Reward 2 to other terminal nodes, including the nodes for

“Death,” even when those values are zero.

If you want to avoid the tedium of adding all these values, download “Lupus Markov model 1" from the

assignments section of Blackboard, which has these values already added.

Even though you are not going to use stage rewards in this exercise, the program requires that there be

stage rewards. Check the stage rewards in the tree (they are located underneath the four branches of the

Markov node) to make sure all of them have values of zero. If any do not have the value of zero, go to the

top of your screen, click on the symbol that represents Markov information (includes an “M” with a circle

around it), and put zeros in the locations for stage rewards.

Tell the program how many cycles it should run

12. Select the Markov node, go to the top of your screen, select Node/Values List/Markov Info, or click on

the symbol that represents Markov information (includes an “M” with a circle around it). A panel should

appear at the bottom of your screen with the name “Termination conditions.” In the row labeled “Term” in

the column labeled “Value” type in STAGE>1999.

Define values for the variables you have created

Select the “Lupus” node, go to the top of your screen, select Node/Define Variable, or click on the symbol

at the top (and the bottom right) of your screen that includes “V=” twice, one over the top of the other. A

panel should appear with the names of the variables you have created. Select “cHosp” and assign it the

value of 10000 in the box that pops up. Repeat the process to assign the value of 0.03 to “r.” Repeat the

process and select stage. In the box that pops up there is a section labeled “Groups:” which is

immediately below the section labeled “_stage =”. In this section, select “Keywords.” Doing this activates

a list of words in the section labeled “Element:,” which is to the right. In this section, double click “_stage.”

Check to make sure that at the top of the box you have “ stage `= stage.”

Your tree should look like the following tree:

5

6

Analyze the model to make sure it works

13. To determine the average number of QALYs that would be expected for a population of lupus patients

who started with this distribution of states, go to “Edit/Tree Preferences .../Calculation Method/Active

Method/Simple/Active Payoff: 2/OK.” Select the Markov node and then go to “Analysis/Markov

Cohort/Markov Cohort (Full)/Decimal probability/ Don’t collapse subtrees/Show both stage and cumulative

rewards/Stages to include: 0 to 100/OK.”

Use the fourth icon of the nine icons at the upper right of this panel to save the text report as a

.pdf file. Label the file with your last name followed by 9, for example, Williams9.

Conduct a cost-effectiveness analysis comparing the Usual Care of SLE with a hypothetical

medical intervention.

14. Change the “Intervention” chance node into a Markov node, select the “Usual Care” subtree, copy it,

and paste it onto the “Intervention” Markov node.

Change the transition probability of the “Active” terminal node of the “Remission” branch of the

“Intervention” Markov node from 0.41 to rr*0.41.

Select the Lupus decision node, right click, select “Define variables” and assign the value 0.8537 to rr.

Assign values to the other variables at this node (cHosp = 10000, r = 0.03, and _stage = _stage).

Because the treatment costs one dollar a day, you should change the cost of the outcomes to reflect that

additional cost. For example, for the transition from Remission to Remission in the Intervention Markov

node change the value of Transition Reward 1 from “(cHosp*.05)/(1+r)^ stage” to

“((cHosp*.05)+cInterv)/(1+r)^ stage,” where cInterv is a variable representing the cost of the intervention

and has the value $365 (select the terminal node, click Node/Value Lists/Markov Info, and type in the new

information when the transition reward box pops up). Select the Lupus decision node, right click, select

“Define variables” and assign the value 365 to cInterv.

Save your partially completed Markov model in *trex format (“File/save As ...”) in a file labeled with

your last name followed by the number 10, for example, Williams10.

Make similar changes to the other transition rewards for cost (except those with zero values, because no

one made these transitions).

If you want to avoid the tedium of adding all these values, download “Lupus Markov model 2" from the

assignments section of Blackboard, which has these values already added.

Select the “Intervention” Markov node and go to “Node/Value Lists/Markov Info.” Type in “_stage > 1999”

for the termination rule. Check to make sure you have the same values for Termination in the boxes under

the two Markov nodes.

Select the “Lupus” node and use “Node/Value Lists/Variable Definitions to insert 0.03 to define the value

of r and 10000 to indicate the value of cHosp. To avoid potential conflicts between the definitions for

these variables at the “Lupus” node versus the definitions for these variables at the 2 Markov nodes, get

rid of definitions for these variables at the 2 Markov nodes by selecting each node and using

“Node/Values List/Variable Definition to bring up a list of variables. Highlight each variable you want to

delete and click on the red X to delete each one. Do not delete the variable definitions for the “Lupus”

node, which also are listed here.

Click on the “Lupus” node, and prepare for a cost-effectiveness analysis by clicking on “Edit/Tree

7

Preferences/Calculation/Calculation Method/Cost Effectiveness/ Cost Payoff 1, Effectiveness Payoff

2/OK.”

15. Perform a cost-effectiveness analysis. Click on the node “Lupus,” and click “Analysis/Cost-

Effectiveness...” Select “Yes” to the question “Draw chart with cost on the vertical axis?” W ait – this could

take a few seconds. On the right-hand side, select “Text Report.” If “Text Report” is not visible, minimize

the material in the extreme right side of this panel.

Use the fourth icon of the nine icons at the upper right of this panel to save the text report as a

.pdf file. Label the file with your last name followed by 11, for example, Williams11.

Email your saved files (file 8 through file 11) to sankey@wharton.upenn.edu.

Prepare the tree for a probabilistic sensitivity analysis.

The material that follows is optional, in part because it refers to last year’s information, which

included a model that was based on TreeAge 2009. If you decide to use this material anyway, when

you’re finished, you will have a model that will allow you to do probabilistic sensitivity analyses. If

you want to do probabilistic sensitivity analyses without doing all the things described in the

following material, download “Lupus Markov Model 3” from the assignments section of

Blackboard, and go to item 21 “Conduct a probabilistic sensitivity analysis.”

16. Start by creating Dirichlet distributions for the transition probabilities and the initial distribution. For the

transition probabilities of the Active subtree in the Usual Care Markov node, click

“Values/Distributions/New/Dirichlet,” type in the box labeled Alphas list the following material

“List(66;806;56;9),” which comes from the slide below, click OK, name the distribution “ddA,” describe it as

“Dirichlet distribution of the Active subtree,” click “Close.”

Go to the Remission terminal node of the Active subtree, and click underneath the branch next to 0.07 in

the grey box. Click “Insert Distribution,” click on “ddA,” click “Use,” and make sure the material in the white

box reads “Dist(1;1).” If it doesn’t, type the material in the white box. Click “OK.” Use similar steps to label

the Active terminal node “Dist(1;2),” the Flare terminal node “Dist(1;3),” and the Death terminal node

“Dist(1:4)” (or, alternatively, leave the pound sign in place).

8

Use similar steps to create an additional Dirichlet distribution for the transition probabilities of the flare

subtree, and put the transition probabilities in the tree, except in the terminal node Remission because we

have no observations that this transition takes place. Use similar steps to create an additional Dirichlet

distribution for the Remission subtree. Put zeros as the transition probabilities into the Flare and Death

terminal nodes of the Remission subtree (because we have no observations that these transitions take

place). Put a pound sign for the transitional probability of the Remission terminal node and “Dist(3;2)” for

the transitional probability for the Active terminal node. Create an additional Dirichlet distribution for the

initial distribution and put the probabilities into the tree below the branches leading from the Markov node.

Create distributions for cost and QALYs using the following information.

Table. Hypothetical mean number of hospitalizations for those who begin in state i and end in state j.

Remis. Active Flare Death

Remission 0.05 0.25 0.00 0.00

Active 0.10 0.20 1.00 0.50

Flare 0.00 0.25 1.25 0.75

9

QALY Distributions

Assume normal distribution (mean, SE)

Assume SD = 0.1

Assume N for calculating SE is same as the number of

individuals observed making the transitions reported in

the Table on slide 24

RtoR: 0.9, 0.0130; RtoA: 0.8, 0.0156

AtoR: 0.8, 0.0123; AtoA: 0.7, 0.0035

AtoF: 0.6, 0.0134; AtoD: 0.35, 0.0333

FtoA: 0.6, 0.0213; FtoF: 0.5, 0.0236

FtoD: 0.25, 0.0158

17. For example, to create outcomes for the Remission terminal node of the Remission subtree, first

create the distributions for the outcomes: click “Values/Distributions/new/Poisson”(Poisson because the

slide above says to use Poisson distributions for the number of hospitalizations), type “0.05" in the white

box labeled lambda (from the hypothetical table above of the number of hospitalizations for going from

Remission to Remission), click “OK,” name the distribution “dhRR,” describe the distribution as

“Distribution of number of hospitalizations, Remission to Remission,” and click “OK.” For the next

distribution click “Values/Distributions/new/Normal,” type “10,000" in the white box labeled Mean and 100

in the white box labeled Std Dev (from the slide above describing the cost of hospitalization), click “OK,”

name the distribution “dcHosp,” describe the distribution as “Distribution of the cost of hospitalization,” and

click “OK.” Use similar steps and the information in the slide immediately above to create a distribution for

the number of QALYs gained for going from Remission to Remission and name this distribution dqRR. To

put these distributions in the Markov model, “select the first terminal node, right click the terminal node,

select “Markov Transition Rewards,” type in the box for Transition Rwd 1 (which is labeled “Cost:”)

“(dhRR*dcHosp)/(1+r)^t,” type in Transition Rwd 2 (which is labeled “Effect;) (dqRR/(1+r)^t,” and click

“OK.”

Create and insert other distributions for the transition rewards for the remaining terminal nodes in the

Usual Care Markov node.

Transfer information from the Usual Care Markov node to the Intervention Markov node, and

modify it.

18. Start by deleting the subtree following the existing Intervention Markov node: right click the

Intervention Markov node, click “Select Subtree,” right click again, and click “Cut Subtree.” Right click the

Usual Care Markov node, click “Select Subtree,” right click again, click “Copy Subtree,” right click the

Intervention Markov node, and click “Paste nodes.”

19. There are two differences between the Usual Care and the Intervention Markov nodes. The transition

probabilities in the Remission subtree are different and the cost of the intervention must be added to the

outcomes in the Intervention model. To make these changes, start by creating a distribution for the

10

relative risk of fewer transitions from Remission to Active in the Intervention model (see slide above).To

create this distribution, click “Values/Distributions/New/LogNormal,” type “-.1582" in the white box labeled

mu (mean of logs) and .1816 in the white box labeled sigma (std dev of logs), click “OK,” name the

distribution “drr,” descibe the distribution as “Distribution of the relative risk of fewer transitions with

intervention,” click “OK, and click “Close.” Insert this distribution in the tree: click the Remission to Active

branch of the Intervention Markov node, change the transition probability from “Dist(3;2)” to “drr*Dist(3;2).”

Do not insert drr in any other transition nodes.

20. To create the distribution for the cost of the intervention, click “Values/Distributions/New/Normal,” type

“365" in the white box labeled mean and “50" in the white box labeled Std Dev, click “OK,” name the

distribution “dcDrug,” descibe the distribution as “Distribution of the cost of the intervention,” click “OK, and

click “Close.” Insert this distribution into the tree: for each terminal node in the Intervention model, right

click the node, and add dcDrug to the cost outcome. For example, change the cost outcome of the first

terminal node from “(dhRR*dcHosp)/(1+r)^t” to “((dhRR*dcHosp)+dcDrug)/(1+r)^t.”

Conduct a probabilistic sensitivity analysis.

21. Highlight the Lupus decision node. Click “Analysis/Monte Carlo Simulation/Sampling (Probabilistic

Sensitivity). . ./Sample all distributions.” Type 1000 in the white box labeled “Number of samples. . . .”

Click “More options/Seed random number generator,” make sure 1 is in the white box, and click

“OK/Begin.” W hen the program has finished running, click “Stats Format/Effectiveness,” put 4 in the white

box labeled Decimal places, click “OK/Close.” Compare your output with the output in the following 3

slides. To compare your output with the output in the first slide, click the rectangle at the bottom of the

page labeled “Stats shown for” to toggle between output for Usual Care and Intervention. To compare

your output with the output in the second slide, click “Graph/Distribution of Incrementals/Incremental

Cost/OK” and then “Graph/Distribution of Incrementals/Incremental Effectiveness/OK.” To compare your

output with the output in the third slide, repeat what you did to display the graphs and click on

11

“Actions/Distribution Statistics.”

12

13