How to Interpret Regression Analysis Results_ P-values and Coefficients _ Minitab.pdf

5/18/2015 HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab

http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients 1/22

Regression analysis generates an equation to describe the statistical relationship betweenone or more predictor variables and the response variable. After you use Minitab StatisticalSoftware http://www.minitab.com/enus/products/minitab/ to fit a regression model, andverify the fit by checking the residual plots http://blog.minitab.com/blog/adventuresinstatistics/whyyouneedtocheckyourresidualplotsforregressionanalysis, youll want tointerpret the results. In this post, Ill show you how to interpret the pvalues and coefficientsthat appear in the output for linear regression analysis.

How Do I Interpret the PValues in Linear RegressionAnalysis?The pvalue for each term tests the null hypothesis that the coefficient is equal to zero noeffect. A low pvalue < 0.05 indicates that you can reject the null hypothesis. In otherwords, a predictor that has a low pvalue is likely to be a meaningful addition to your modelbecause changes in the predictor's value are related to changes in the response variable.

Conversely, a larger insignificant pvalue suggests that changes in the predictor are notassociated with changes in the response.

In the output below, we can see that the predictor variables of South and North aresignificant because both of their pvalues are 0.000. However, the pvalue for East 0.092 isgreater than the common alpha level of 0.05, which indicates that it is not statisticallysignificant.

Typically, you use the coefficient pvalues to determine which terms to keep in theregression model. In the model above, we should consider removing East.

59 http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients

How to Interpret Regression Analysis Results: Pvaluesand CoefficientsJim Frost http://blog.minitab.com/blog/adventuresinstatistics 1 July, 2013.

98 778 153

Master StatisticsAnytime,AnywhereQuality Trainer teachesyou how to analyzeyour data anytime youare online.

Take the Tour! http://www.minitab.com/products/quality

trainer/?WT.ac=BlogQT

The Minitab Blog http://blog.minitab.com

Data Analysis http://blog.minitab.com/blog/dataanalysis2

Quality Improvement http://blog.minitab.com/blog/qualityimprovement2

Project Tools http://blog.minitab.com/blog/projecttools2 Minitab.com http://www.minitab.com



How Do I Interpret the Regression Coefficients for LinearRelationships?Regression coefficients represent the mean change in the response variable for one unit ofchange in the predictor variable while holding other predictors in the model constant. Thisstatistical control http://blog.minitab.com/blog/adventuresinstatistics/atributetoregressionanalysis that regression provides is important because it isolates the role of onevariable from all of the others in the model.

The key to understanding the coefficients is to think of them as slopes, and theyre oftencalled slope coefficients. Ill illustrate this in the fitted line plot below, where Ill use apersons height to model their weight. First, Minitabs session window output:

The fitted line plot shows the same regression results graphically.

The equation shows that the coefficient for height in meters is 106.5 kilograms. Thecoefficient indicates that for every additional meter in height you can expect weight toincrease by an average of 106.5 kilograms.

The blue fitted line graphically shows the same information. If you move left or right alongthe xaxis by an amount that represents a one meter change in height, the fitted line rises orfalls by 106.5 kilograms. However, these heights are from middleschool aged girls andrange from 1.3 m to 1.7 m. The relationship is only valid within this data range, so we wouldnot actually shift up or down the line by a full meter in this case.



If the fitted line was flat a slope coefficient of zero, the expected value for weight wouldnot change no matter how far up and down the line you go. So, a low pvalue suggests thatthe slope is not zero, which in turn suggests that changes in the predictor variable areassociated with changes in the response variable.

I used a fitted line plot because it really brings the math to life. However, fitted line plots canonly display the results from simple regression, which is one predictor variable and theresponse. The concepts hold true for multiple linear regression, but I would need an extraspatial dimension for each additional predictor to plot the results. That's hard to show withtoday's technology!

How Do I Interpret the Regression Coefficients forCurvilinear Relationships and Interaction Terms?In the above example, height is a linear effect; the slope is constant, which indicates that theeffect is also constant along the entire fitted line. However, if your model requirespolynomial or interaction terms, the interpretation is a bit less intuitive.

As a refresher, polynomial terms model curvature in the datahttp://blog.minitab.com/blog/adventuresinstatistics/curvefittingwithlinearandnonlinearregression, while interaction terms indicate that the effect of one predictordepends on the value of another predictor.

The next example uses a data set that requires a quadratic squared term to model thecurvature. In the output below, we see that the pvalues for both the linear and quadraticterms are significant.

The residual plots not shown indicate a good fit, so we can proceed with the interpretation.But, how do we interpret these coefficients? It really helps to graph it in a fitted line plot.



You can see how the relationship between the machine setting and energy consumptionvaries depending on where you start on the fitted line. For example, if you start at a machinesetting of 12 and increase the setting by 1, youd expect energy consumption to decrease.However, if you start at 25, an increase of 1 should increase energy consumption. And ifyoure around 20, energy consumption shouldnt change much at all.

A significant polynomial term can make the interpretation less intuitive because the effect ofchanging the predictor varies depending on the value of that predictor. Similarly, asignificant interaction term indicates that the effect of the predictor varies depending on thevalue of a different predictor.

Take extra care when you interpret a regression model that contains these types of terms.You cant just look at the main effect linear term and understand what is happening!Unfortunately, if you are performing multiple regression analysis, you won't be able to use afitted line plot to graphically interpret the results. This is where subject area knowledge isextra valuable!

Particularly attentive readers may have noticed that I didnt tell you how to interpret theconstant http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowtointerprettheconstantyintercept. Ill cover that in my next post!

Be sure to:

Check your residual plots so you can trust the resultshttp://blog.minitab.com/blog/adventuresinstatistics/whyyouneedtocheckyourresidualplotsforregressionanalysisAssess the goodnessoffit and Rsquared http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowdoiinterpretrsquaredandassessthegoodnessoffit



If you're learning about regression, read my regression tutorialhttp://blog.minitab.com/blog/adventuresinstatistics/regressionanalysistutorialandexamples!

CommentsName: Lovemore Friday, January 24, 2014

That's sounds great but for me I am finding difficult how do I instigate a six sigma project in a medical laboratory using soof the Minitab tools

Name: Henry Mwangi Thursday, February 20, 2014

Thank you for an elaborate explanation on the interpreting reg coefficients and mostly the pvalue.

Name: Deeps Dee Thursday, March 27, 2014

It has been useful for my thesis whereby I've been struggling to interpret my results :sThank you for the explanation.

Name: taiwo lucas Wednesday, April 2, 2014

Thank you very much the explanation really help me in my thesis.God bless you.

Name: O.Jobi Saturday, May 10, 2014

This is very helpful information for my dissertation page 4&5.

Name: yashika Tuesday, May 13, 2014

really i was confused and you clear this concept of regression coefficient. very good explanation.can you do this with ttest explanation also?

You Might Also Like:Regression Analysis: How to Interpret the Constant Y Intercept http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowtointerprettheconstantyinterceptRegression Analysis: How Do I Interpret Rsquared and Assess the GoodnessofFit?http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowdoiinterpretrsquaredandassessthegoodnessoffitWhy Are There No P Values for the Variables in Nonlinear Regression? http://blog.minitab.com/blog/adventuresinstatistics/whyaretherenopvaluesforthevariablesinnonlinearregressionRegression Analysis Tutorial and Examples http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysistutorialandexamples



Name: omid Saturday, June 7, 2014

hi dear,I am doing a censored least absolute deviation model using STATA, when I got output there was a column indicated with"Bias" , does it mean Pvalue ?

Name: Jim Frost Monday, June 9, 2014

Hi Omid,

Thanks for your question. I can't really offer guidance about using Stata. However, bias and Pvalue are not synonymous, sothat's probably not what the output means.

I suspect it has to do with the censoring in your data. Regression with censored data can cause biased estimates becauseyou may be less likely to observe the response value for certain classes of observations. In other words, the model that fitsthe observed responses may not provide an unbiased fit for the censored observations.

Minitab can perform regression with censored data and can assume different distributions. In Minitab: Stat >Reliability/Survival > Regression with Life Data.

You can try a free 30 day trial of Minitab 17 here:http://it.minitab.com/enus/products/minitab/freetrial.aspx http://it.minitab.com/enus/products/minitab/freetrial.aspx

Thanks for writing!Jim

Name: Mrv Yrd Wednesday, August 27, 2014

Hi Jim,First of all Thank you for the useful information! I am little confused about p value and significance for regression. If our pvalue is 0.02 for SLR can we say that regression analysis is statistically significant at 95% confidence level ? Or should we sayit is significant at 98%? My second question is that if we are not given the p value for the variable and the constant for SLR, but the regression pvalue is smaller than 0.05 , can we conclude the factor significantly affects the response ? Thank you in advance.

Name: Jim Frost Thursday, August 28, 2014

Hi,

Typically you choose the significance level before the study, and that's the level you cite after the analysis. For example, youcan state that the SLR is statistically significant at the the 0.05 level. Or for multiple regression, identify the variables thatare significant at that level e.g. 0.05. You typically don't change the significance level to match your pvalues.

However, I'd also report the exact pvalues as well. The exact pvalue is important in terms of understanding the liklihoodthat your test drew the correct conclusions. I cover that in this post:http://blog.minitab.com/blog/adventuresinstatistics/fiveguidelinesforusingpvalueshttp://blog.minitab.com/blog/adventuresinstatistics/fiveguidelinesforusingpvalues

For your second question. Yes, in a simple linear regression model Y = a + bX, the regression pvalue in the ANOVA is fora test of the hypothesis that the linear coefficient is zero.



Thanks for reading!Jim

49Comments TheMinitabBlog Login1

Share SortbyOldest

Jointhediscussion

Reply

Joel 6monthsagoHello,

Ifittedthemodely=a+bX1+cX2+dX1.X2+e(X1)^2+f(X2)^2onadatasetbutIhavesomeproblemsininterpretingthepvaluesofthecoefficients.IfIusenormalizedvaluesforX1andX2(smallestvalue:1,largestvalue:+1)andIperformaregressionIgetdifferentpvaluesforthecoefficientsa,bandc(notd,eandf)comparedtottherealvalues.Infactformydatasetp0.05fortherealvalues.SoIguessnormalizationistobedonealwaystoanalyzedata?

Thanksinadvance.

Jol1

Reply

inez 6monthsagoInmylinearregressionresults,whatdothetvaluesmean?caniputthemintableofresults?

Reply

JimFrostAtMinitab 6monthsagoMod >inezHiInez!Thanksforwritingwiththeexcellentquestion!

Thetvalueisastatisticthatmeasurestheratiobetweenthecoefficientanditsstandarderror.

Minitabusesthetvaluetocalculatethepvalue,whichyouusetomakeadecisionaboutthestatisticalsignificanceofthetermsandmodel.

Asufficientlylargeratioindicatesthatthecoefficientestimateisbothlargeandpreciseenoughtobesignificantlydifferentfromzero.Conversely,asmallratioindicatesthatthecoefficientestimateistoosmallortooimprecisetobecertainthatthetermhasaneffectontheresponse.

Youcanusethetvaluetodeterminewhethertorejectthenullhypothesis.However,thepvalueisusedmoreoftenbecauseitiseasiertointerpret.

Unlessyouhaveaspecialneedtoincludeit,Iwouldnotincludeitinyourresults.

Jim

Cain 5monthsagoHowcanItellthelevelofsignificancefromanoutput?IhaveanexamusingminitabandI'mnotsure

Recommend

Share

Share

Share



Reply

Reply

JimFrostAtMinitab 5monthsagoMod >CainHi,thatsoundslikeatrickquestiontome.Thesignificancelevel(alpha)issomethingthatyoushouldchoosebeforeyouperformyourstudy.Afteryouperformtheanalysis,youcomparethepvaluesintheoutputtoyoursignificancelevel.

Jim

Reply

WDC123 5monthsagoHiJim,IfIreducethemodelbytakingouttermswithpvalueslessthank0.05andthennoticethatRsquaredhasalsoreducedhowdoIexplain.ShouldIconsiderleavinginsometerms?

Reply

JimFrostAtMinitab 4monthsagoMod >WDC123Hi,typicallyyouconsiderremovingpredictorsfromthemodelifthepvalueisgreaterthanyoursignificancelevel.I'llassumethatiswhatyoumeanttotype!:)

It'sfairlytypicalfortheRsquaredtodeclineasyouremovepredictors,evenwhenthosepredictorsarenotsignificant.

Hereareacoupleofsuggestions:*UseadjustedRsquaredtocomparemodelswithdifferentnumbersofterms.*Don'tchoosethemodelbasedsolelyonthehighestRsquaredbecausethatcanleadyouastray.*Useyourexpertise,theory,andcommonsenseratherthanrelyingsolelyonsimplisticmodelselectionrules.

Foryourcase,don'tfeellikeyoushouldincludethoseinsignificantpredictorsjusttogetthehigherRsquared.However,youcanconsiderincludingthemiftheorysuggeststhattheybelonginthemodel.Ingeneral,youshouldalreadyhaveanideaofwhattheimportantvariablesarealongwiththeirrelationships,coefficientsigns,andeffectmagnitudesbasedonpreviousresearch.

There'snotalwaysaclearansweronwhichpredictorsyoushouldincludeinyourmodel.Useboththestatisticaloutputandtheoretical/subjectareaconsiderationstohelpyoudecide.

Thanksforwritingwiththegreatquestion!Selectingthecorrectmodelhasalwaysbeenaveryinterestingsubjectforme!

Jim

Ronja 4monthsagoHello,myquestionisquitesimilartotheothers:inordertodevelopaforecastIwanttousemultipleregression.Itriedvariousindependentvariablesthatwouldallmakesense(meaningtheyallmayhaveanimpactontheforecast)togainthebestsuitedequationfortheforecast,butIfinditdifficulttochoosetherightsetofvariables.Withtheonesetofindependentvariables,mypvaluesarehigherthan0,05(theyare0,12)howevertheRsquaredishighestwith0,9904.Takingouttermswithpvalueshigherthan0.05won'tworksincethentherewon'tbeanyleft.Withtheotherset,myRsquaredisjust0.8473howeverthepvaluesarelessthan0,05.Howdoyouselecttherightset?DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformy

Share

Share

Share

Share



Reply

DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformyselection?

Thankyouverymuchinadvance!!!

Ronja

Reply

JimFrostAtMinitab 4monthsago

seemore

Mod >Ronja

HiRonja,

Selectingthecorrectmodelcanbeaverydifficultprocessinsomecases.Readmyresponsetothecommentdirectlyaboveyours(toWDC123)becauseitappliestoyourcaseaswell.

Specifically,don'tfeellikeyoumustgetthehigherRsquaredbecauseit'spossibletohaveanRsquaredthatistoohighandcauseproblems.YourRsquaredof0.99maybetoohighandcouldindicatethatyou'reoverfittingthemodel.Also,youshoulduseadjustedRsquaredtocomparemodelswithdifferentnumbersofpredictorsratherthanRsquared.

IsuggestthatyoureadmyblogpostaboutadjustedRsquared,whichcoversalloftheabovepoints.

AsforpvaluesversusadjustedRsquaredvalues,researchhasshownthatusingpvaluesinastepwisemannergenerallyworksbetterthanusingadjustedRsquaredtopickthecorrectmodel.However,usinganysimplemodelselectionprocedurelikethatgenerallydoesnotpickthecorrectmodel.I'vewrittenanotherpostaboutthisissuewhereIcomparestepwisetobestsubsetsregression.

Theimplicationsofthesefindingsareprofoundevenifyou'renotusingeitheroftheseautomatedmethods.Thefindingsshowthatchoosingthecorrectmodelisasmuchascienceasitisanart.The

Reply

Fiachra 4monthsagoHi,AfterrunningmyregressionIendedupwithpvalueslike6.9345E05.WhatdoesthisEmeanandhowdoIworkoutthePvaluethanks.

Reply

JimFrostAtMinitab 4monthsagoMod >FiachraThatiscalledscientificnotationandisusedtowriteverylargeandverysmallnumbers.ItworksbyshiftingthedecimalpointleftorrightbythenumberofplacesindicatedaftertheE,whichstandsforexponent.

The05indicatesthatyouneedtotakethe6.9345andshiftthedecimalpointtotheleftby5places.So,yourpvalueis0.000069345.That'saverylowvaluesoitisverysignificant!

Jim

Fiachra 4monthsago>JimFrostAtMinitabThanksamillion!myheadwaswreckedthinkingitwassomethingmuchmorecomplex.IdohaveoneotherquestionhoweverinarecentmcqIwasgivenaregressionoutputbasedonsalary=b1+b2(Rank).(Rankbeingthequalityoftheindividualsuniversity,thebestwasawardedarankof1andtheworstarankof142).Thecoefficientstheregressionproducedfortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowp

Share

Share

Share

Share



Reply

fortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowpvaluessotheyweresignificant.Thequestionwaswhatisthetrueeffectofaoneplaceincreaseinuniversityrankingsonsalaries.TheanswerIgavewas206.731butthecorrectansweristhatitcannotbedeterminedfromthesefigures.(Figuresbeingapictureofaregressionoutputinexcel).Whyisthisthecorrectanswer?Ithoughtthiswouldhavebeenexactlywhatthecoefficientintheregressionindicates.Thanks.

1

Reply

JimFrostAtMinitab 4monthsagoMod >FiachraIfthequestionaskedyouspecifically,whatwasthe"true"effect,youhavetorememberthatregression,andotherstatisticaltechniques,canonlyprovideanestimateofthetrueeffect.It'sgenerallyimpossibletoeverknowthetrueeffectitselfbecauseyou'reworkingwithasampleofthepopulationratherthantheentirepopulation.

Instead,inferentialstatisticscanonlyprovideanestimateofthetrueeffectandgiveyouaconfidenceintervalforarangeofvaluesthatislikelytocontainthetrueeffect.Inregressionanalysis,thecoefficientsaretheparameterestimates.

Reply

sewnsew 4monthsagoihavearegressionmodelhowdoIcalculatethechangeinpwhenItakeoutvariablesoraddvariablebackintoamodeltoseewhichhasthemostpredictivevalue?InthedataIhave,Ihaveachangeinp,butinSPSS,Idon'tseeanythingthatshowsorrelatestothechangeinp,sowhenIrerunthedata,Idon'tknowwhattolookfororwhattointerpretasachangeinp.Thanks.

Reply

JimFrostAtMinitab 4monthsagoMod >sewnsewHi,Ican'tspeaktowhatyouseeinothersoftwarepackages.Also,I'mnotsurewhichpyouarereferringto.

Youmaywanttolookattheadjustedsumsofsquaresintheoutput.Thisindicatestheuniqueportionofthetotalsumsofsquaresthateachtermexplainsregardlessoftheordertheywereenteredinthemodel.Ifyouwanttofindouthowmuchvariationeachpredictorvariableaccountsforinamodel,thisiswhatyouneed.

Jim

Reply

Scott 4monthsagoIsthereanywaytoset/holdaparticularregressionequationcoefficientataparticularvalue,andthenperformtheregressionanalysis?

Inmyexample,Iamanalyzingpsioutvaluebasedonanumberofinputs,IwanttoholdPsiIncoefficientat1,andlettheothervariablesbeapartoftheregression.Hopethismakessense,:/

Thanks!

JimFrostAtMinitab 4monthsagoMod >ScottHiScott,

Share

Share

Share

Share

Share



Reply

HiScott,

That'saninterestingquestion.

Typically,you'refittingamodellikethis:Y=B0+BX1+BX2+BX3...whereyouestimatetheBsfromthedata.

Youwanttofitthis:Y=B0+1X1+BX2+BX3...wherethefirstcoefficientis1.

WhatyoucantrydoingismovingthetermwiththefixedcoefficientovertotheYsideoftheequation:Y1X1=B0+BX2+BX3

You'dhavetocreateanewcolumnofresponsedatawhereyoutaketheoriginalmeasureandsubtractoutthe1X1.Inyourcase,you'dtaketheoutputPSIandsubtracttheinputPSIforeachobservationandusethenewlycalculatedvaluesastheresponse.Then,includetherestofthepredictorsinthemodel.

You'dessentiallybelookingathowthepredictorsarerelatedtothechangeinPSIratherthantheabsolutePSI,whichsoundspromisingifIunderstandyourscenariocorrectly.

Theestimatesfortheotherpredictorswouldbethevaluesifforcedthefirstpredictortoequal1.You'dhavetobecarefulhowyouinterpretthemodelfitvalues.Forexample,Rsquaredindicateshowmuchvariationyouaccountforwiththenewresponsevariable.

Jim

Reply

sewnsew 4monthsagoInmyhomogeneoussubsetstheNisdifferentthantheNthatIgotwhenIranfrequencies.Why?Isthisnormal?

Reply

SharonEdgeWilkie 4monthsagoThisisaPostHocquestion.WhyaretheNinmyhomogeneoussubsetsnotthesameastheNinmyfrequencycharts?

Reply

PatrickKajubili 4monthsagoHi,Iamstilljuniorinthefield.iwanttoknowifihaveF714andSig761inmyANOVAtablewhatdoesthismean?Havingsiglikedoesshowmodelfit?

JimFrostAtMinitab 4monthsagoMod >PatrickKajubiliHiPatrick,theFstatisticisatestoftheoverallsignificanceoftheregressionmodel.WhileRsquaredandadjustedRsquaredtellyoutheoveralldegreeofthefitforaregressionmodel,theydon'tprovideaformalhypothesistestfortheoverallfit.

That'swheretheFtestanditsassociatedpvaluecomesin.

ThenullhypothesisfortheFtestisthatallofthecoefficientsintheregressionmodelequalzero.Ifallthecoefficientsequalzero,thisisequivalenttosayingthatthefittedvaluessimplyequalthemeanof

Share

Share

Share

Share



Reply

theresponsevariable.Inotherwords,yourmodelpredictstheresponsenobetterthanusingtheresponsemean.

Thealternativehypothesisisthattheydon'tallequalzero.Or,thatyourmodeldoesprovidebetterpredictionsthanjustusingthemean.

Alowpvaluemeansthatyoucanrejectthenullandconcludethatyourmodelisbetterthanjustusingthemeanandthatatleastonecoefficientdoesn'tequalzero.

You'llstillneedtochecktheresidualplotsbecausethistestwon'ttellyouwhetherthemodelprovidesanadequate,unbiasedfit.InthebulletsneartheendofthispostIprovidealinktoablogpostIwroteaboutcheckingtheresidualplots.

Thanksforwriting!Jim

Reply

Marija 3monthsagoHello,Ineedyurhelpaboutmyexamquestion:

(i)EstimatethefollowingregressionsPRICE=b1+allindependentvariables+utLnPRICE=b1+allindependentvariables+utAccordingtotherelevantcriteria,judgewhichoneisbetter.Continueworkingwiththebetterfromthetwo.Fullyinterpret(statisticalandeconomicsignificance)theresultsofhedonichousepriceestimation.Myquestion:Whicharethecriteriatodecidewhichisabetterregression?

Ihavecalcualtedthembothandherearetheresults(valuesonlyfromthevariableswithsig.MarijaHiMarija,

InadditiontothefactthatIreallyshouldnotansweryourexamquestionforyou,Ireallycan'tanswerthequestionwiththeinformationthatyouprovided.Thereisinsufficientinformationtobeabletochoose.But,Icangiveyousomegeneralguidelinesonhowtochoose.

Youshouldchecktheresidualplotsforbothmodels.Iftheplotslookgoodforonemodelbutnottheother,thatwillhelpyouchoose.

Youshouldalsolookatthecoefficientsforthepredictorsanddeterminewhethertheymatchtheory.Forexample,ifonemodelsuggeststhatagoodcharacteristiclowerstheprice(negativecoefficient),youshouldseriouslyquestionthatmodel.

Thosearethetypesofthingsyouneedtoassesstodeterminewhichmodelisbetter.Irecentlywroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpful

Share

Share



Reply

wroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpfulinformationforyou!

Goodluckwithyourtest!Jim

Reply

wuyr 3monthsagoHelloJim,Thanksalotforyourposting.Itisveryhelpful.IhaveanofftopicMinitabquestion,andhopingthatyoucouldhelpmeout.DoesminitabhasafunctionlikeVlookupinexcel?Thanksalot.Yan

Reply

JimFrostAtMinitab 3monthsagoMod >wuyrHi,thankyouforthenicecomment!

Unfortunately,Minitabdoesn'thaveanexactlyequivalentfunction.However,inMinitab,youcanuseControlFtousetheFindinDataWindowfunction.Thiswillsearchwithinacolumnforaspecificvalue,eitherexactmatchornot.Whenitfindsamatchinacell,youcanlookattheassociatedinformationinthethatrowasawaytomimicthefunctionalityofVLOOKUP.

Jim

Reply

JackWotton 3monthsagoHijim,

I'mabletoexplainmyresultsthroughthepvalue,s=,rsq,andthegraphs.butiamunsureonothervaluesthathaveshownupe.g.,DF,SS,MF,F,(howtointerprettheresidualerrortomyresults?whatdoesDF20,SS235.57MS11.78allmean)ithinkthismostlyrelatestotheanalysisofvarience.hopeyourabletohelpasihaveadissertationhandinnextmonth)cheers

Jack

JimFrostAtMinitab 3monthsagoMod >JackWottonHiJack,

Alotofthesestatisticsarethe"behindthescenes"typeofnumbersthatMinitabneedstocalculateinordertocomputethemorecommonstatisticsthatpeopleneed,likethepvalues,Rsquared,adjustedRsquared,andS.Unlessyouhaveaspecialneed,youoftendon'tneedthestatisticsthatyoulist.

I'llrunthroughthemingeneralforyou.Ifyouneedmoredetailedinformationabouthowthey'recalculated,youcanalwayslookattheMethodsandFormulaHelpinMinitab:Help>MethodsandFormulas.TheMinitabGlossary(Help>Glossary)alsohasdefinitionsoftheseterms.

DF:Thedegreesoffreedom(DF)describetheamountofinformationyourdataprovidethatyoucan"spend"toestimatethevaluesofunknownpopulationparameters,andcalculatethevariabilityoftheseestimates.Degreesoffreedomareaffectedbythesamplesizeandthenumberofparametersinyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,and

Share

Share

Share

Share



Reply

seemore

inyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,andconsequentlyincreasesthedegreesoffreedompresentinyourdata.Addingparameterstoyourmodel(byincreasingthenumberoftermsinaregressionequation,forexample)"spends"informationfromyourdata,andlowersthedegreesoffreedomavailabletoestimatethevariabilityoftheparameterestimates.

Reply

JackWotton 3monthsago>JimFrostAtMinitabThankyousomuchforyourhelp:)

Reply

Fardeen 3monthsagoHiMrJim.Imhavinggreatproblemsindoingmydissertation.Idontknowhowtomakeuseofregression.Iwouldbegratefulifyoucouldhelpme.Isthereasitewhereitshowsclearlytouseregression?Thanks

Reply

JimFrostAtMinitab 3monthsagoMod >FardeenHiFardeen,

Irecommendthatyoureadmyregressiontutorialwithexamples.Ithinkthiswillansweralotofyourquestions.

Bestofluckwithyourdissertation!Jim

Reply

becbec 2monthsago>JimFrostAtMinitabHiJim,thankyousomuchfortheinformativediscussionshere.Iammakingmythesishowever,Iamfindingdifficultiesininterpretingmydata.Whatdoesthisresultmeanifmyconstanttvalueis7.114,pvalue=.000,LIFCAStvalue=10.228pvalue.000,LERIANStvalueis2.971pvalue.003,andEFCOStvalue,2.186andpvalue.029.iwouldappreciateyourhelp.thanks.

JimFrostAtMinitab 2monthsagoMod >becbecHi,

Withtheinformationyouprovide,Ican'tbesurethatyourmodelmakessensetheoreticallyorwhetherthemodelprovidesanadequate,unbiasedfittothedata.Onethingyoushoulddoisdefintelycheckyourresidualplots.

Assumingthemodelisgood,here'swhatyou'vegot.

Youhaveaconstanttermthatissignificantlydifferentfromzero.However,theconstanttermusuallyhasnomeaningfulinterpretation.There'salinktoablogpostIwroteaboutwhythisistrueneartheendofthisblogpost(beforethecommentssection).

Share

Share

Share

Share

Share



Reply

Youhave3significantpredictors.Thissuggeststhatchangesineachpredictorarerelatedtochangesintheresponse.Forexample,aoneunitincreaseinLIFCASisrelatedtoanincreaseinthemeanresponsevalueequaltotheLIFCAScoefficient.SameforLERIANS.ForEFCOS,everyoneunitincreaseisrelatedtoadecreaseinthemeanresponse(youdidn'tincludethecoefficientsbutfromthetvalueIknowthattheEFCOScoefficientisnegative).

Typically,youdon'tneedtoworryaboutthetvaluesandinsteadfocusonthepvaluesandcoefficients.

Youmightwanttoreadmyblogpostaboutchoosingthebestregressionmodeltohelpyoubesurethatyoudohavethebestmodel!

Bestofluckwithyourthesis!Jim

Reply

dunmao 2monthsago

seemore

>JimFrostAtMinitab

HiJim,

Couldyoupleasegivemeadirectionforthefollowingquestion?

Myquestions:IamdoingridershipmodelingusingmultiplelinearregressionmethodinExcelsoftware.Mydependentvariableisboardings,threeindependentvariablesarepopulation,feederbusservices,andemploymentdata.Eventhoughtheconstantismeaninglessdiscussedfromyourdiscussgroup.Inmycase,thepvalueforYinterceptis0.6(greatthan5%),howevertheYinterceptcanminimizetheresidual(observeddatapredictedvalue).Seetheregressionresult:

RSquare=0.943573,

PvalueforYintercept=0.6,Pvaluesforthethreeindependentvariablesarelessthan5%

AccuracyValidationwithoutYintercept(ObservedPredicted):

Reply

JimFrostAtMinitab 2monthsagoMod >dunmaoHi,Irepliedtoyourquestionintheotherpostwhereyousharedyourcomment.Youcanfindithere.

Theshortansweris,yes,youshouldalmostalwaysincludetheconstantregardlessofthepvalue!

Jim

dunmao 2monthsago>JimFrostAtMinitabHiJim,

Share

Share

Share



Reply

HiJim,

Thankyousomuchforyourquickresponse!

Iwanttoincludetheconstanteventhoughthepvalueoftheconstantisgreatthan5%.Theconstantcanbeexplainedasanadjustedfactorinmypredictionmodeltominimizetheerror.

Youranswerconfirmsmytestresults.

Thanksagain,Hope

Reply

JimFrostAtMinitab 2monthsagoMod >dunmaoHi,you'reverywelcome!

Justtoclarifyonepoint.Yougenerallyshouldincludetheconstantregardlessofthepvalue.Youdon'tneedajustificationtoincludetheconstant.Instead,youneedaverystrongjustificationtoevenconsidernotincludingtheconstant.

Infact,I'veneverpersonallyworkedwitharegressionmodelwhereIfeltjustifiedtonotincludetheconstant.Aregressionmodelwithouttheconstantisveryrarebecausethepotentialforintroducingbiasisveryhigh.

Jim

Reply

dunmao 2monthsago

seemore

>JimFrostAtMinitab

HiJim,

Icomeback.IhaveanotherpredictionmodelwithYinterceptpositive.Seethefollowings:

AccuracyValidationwithoutYintercept(ObservedPredicted):

Predictedmodel:DV_37pm=0.441*IV2+0.179*IV3Error=Observed(3559)predicted(3961)=402(overestimated1678)

AccuracyValidationwithYintercept

Predictedmodel:DV_37pm=0.441*IV2+0.179*IV3+0.714Error=Observed(3559)predicted(3971)=412(overestimated412)

JimFrostAtMinitab 2monthsagoMod >dunmaoHi,

Share

Share

Share



Reply

Hi,

YoushouldalmostalwaysincludetheYinterceptinthemodel.Irecommendthatyoudoleaveitinthemodel.

ThisistrueregardlessofthePvalue.Ihopeyou'vereadmypostabouttheregressionconstant?Ishowthereasonswhyyoushouldalwaysincludeitinthemodel.

Ifyou'reboundanddeterminedtoconsiderremovingit,thereareimportantconsiderationsyoumustevaluatefirst.

1)Checkthestandarderroroftheregression.TheErrorisinyouroutputisnotthestandarderrorbecauseSisalwayspositive.Yourerrorreductionisnotsubstantialanywayonlyfrom412to402.Theminisculereductioninerrorsuggestsyoumightaswellleavetheconstantinthemodel.

2)Checkyourresidualplots.Inparticular,besurethattherearenononrandompatternsforeithermodel.Thisisespeciallyimportantinthemodelwithouttheconstantbecauseoftenremovingtheconstantintroducesabiasthatyou'llseeintheresidualplots.Ifyouremovetheconstantandyouseeapatterninresiduals,puttheconstantbackinyourmodel.

But,really,youshouldincludetheconstantevenwiththehighpvalue.It'snothurtinganythinganditislikelyhelpingreducebiasinyourmodel.

JIm

dunmao 2monthsago>JimFrostAtMinitabThankyouJim!

Iwanttolearnmore,soIcomparethetwocasesWiththeconstantinmypredictionmodelANDwithouttheconstantinmypredictionmodel.WITHtheconstantinmypredictionmodel:Standarderror:82Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.WITHOUTtheconstantinmypredictionmodel:Standarderror:78Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.Therearenononrandompatternsforeithermodel.

Frommyunderstanding,theconstantissmall,sothereisnopatternintheresidualsdistributions.

Lastquestion:iftheconstantisbig,itcausestheerrorreductionsubstantial,doIstillneedtokeeptheconstant?(sorry,Idon'thavetheregressionresults,butIwanttoknowifthecaseexists.)

Thankyou,Hope

Share



Reply Hope

Reply

JimFrostAtMinitab amonthagoMod >dunmaoHi,

Givenwhatyousay,theredoesn'tseemtobeanynumericreasontonotremovetheconstant.

However,beforeyoudothat,askyourselfifit'stheoreticallyjustifiedthatifyousetallofthepredictorstozero,you'dexpecttheresponsetoequalzeroaswell.Preferably,youwouldalsohavemeasuredvaluesnear/atthisallzeroregiontoconfirmthattheregressionlinetrulygoesthroughtheorigin.

It'sonlywhentheconstantissmallthatyouhaveachance(smallchance)toremoveitfromthemodel.Ifitislarge,removingitfromthemodelwillalmostcertainlybiasyourmodel!Iwouldneverremovealargeconstant.

Jim

Reply

dunmao amonthago>JimFrostAtMinitabHiJim,

Thankyousomuchforyourexplanation!Icompletelyunderstandtheconstant(regardlessofpvalue)now.

NowIhaveanewregressionresult:

R=99.35%,

AdjustR=99.06%

DV=20+0.129*IV1+0.178*IV2+0.078*IV3

Errors=observed(4088)predicted(4052)=36

Averageerrors=5.75%

Questions:WhyistheRsobigat99.35%?Maybesomeonewouldaskmeaboutthequestion.However,thisistrueregressionresult.Howwouldyouexplaintheresult?

Thankyouagain,

Hope

JimFrostAtMinitab amonthagoMod >dunmaoHi,

Withoutknowingthespecificsofthemodelandthestudyarea,it'simpossibletosayforsure.IfIremembercorrectly,youaremodelingridershipovertime.Iftherearetrendsinthedatathataffectbothsidesoftheequation,thisisaproblemandcanoftenproduceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeif

Share

Share

Share



Reply

produceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeiftheyarestationary(constantmeanandvarianceovertime)ornonstationary(upwardordownwardtrendornonconstancevariance).

Ifyouhavenonstationarydata,youmustmakeitstationarybydifferencingthedatasothateachdatapointisthechangeinvaluebetweenconsecutivepoints.Usingregressionanalysiswithtimeseriesdatainvolvesadditionalconsiderationslikethis.Unfortunately,Idon'thaveahandyreferencetoreferyoutoobutyoushouldperformsomeadditionalresearchtoensurethatyouendupwithavalidmodel.

Jim

Reply

dunmao amonthago

seemore

>JimFrostAtMinitab

HiJim,

Inoticedanewquestion:

AsItoldyouIhavedonethetestingasthefollows:=====================================================Whenletintercept=0,theregressionresult:

Rsquared=0.96

AdjustedRsquared=0.88

StandardError=78

Observations=14

ANOVA:dfRegression:2Residual:12

Reply

Em 2monthsagoHi,thankyouforyourextremelyhelpfulblogs!Iwaswondering,ifyoucanhelpmeoutwithmymultipleregressionanalysis.ForthePearsoncorrelation,Ifoundthatonlyoneofmypredictorsissignificant(p=0.037).However,Idon'tquiteunderstandwhyinthettestsection,noneofmyindependentvariablesmakeasignificantcontributiontothemodel.Howisitpossible?Icouldn'tfigureoutthelinkbetweenthetwo.Canyouexplainthis?Thanksinadvance!

JimFrostAtMinitab 2monthsagoMod >EmHi,

ThePearsoncorrelationpvaluesandregressionpvaluestestdifferentthingssotheanswersmaynotagree.Thecorrelationpvalueonlytestsonepairofvariablesatatimewithoutconsideringtheothervariables.Theregressionpvaluesfactorinalltheotherpredictorvariablesthatareincludedinthemodel.

Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthe

Share

Share

Share



Reply

Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthepredictorsandtheresponsevariable.Tryaregressionmodelwithjustthatonepredictor.Itshouldbesignificantinaregressionmodelbyitself.Then,addintheotherpredictors.Ifthesignificancegoesaway,itindicatesthattheotherpredictor(s)areaccountingforsomeofthesamevarianceintheresponse.Bysplittingupthevariancethatisaccountedforbetweenthevariables,itmaybethatnonearesignificantwhenthereismorethanoneinthemodel.

Also,checkyourVIFsinthefullmodel.It'spossiblethatmulicollinearity(correlationbetweenthepredictors)issappingthesignificanceofthepredictors.Theproblemsassociatedwithmulticollinearitydonotoccuronlywhenthereisastrongcorrelationbetweenindividualpairsofpredictors.Theseproblemscanoccurwhenthereisamoderatecorrelationbetweenanumberofpredictors.ThismoderatecorrelationmaynotbesignificantwhenyoulookatthePearsoncorrelationbetweenpairsbutcanbedetectedwithVIFs.ReadmoreaboutthisinmypostaboutmulticollinearityandVIFs!

Ihopethishelpsandthanksforwriting!Jim

Reply

Sayeed 2monthsagoHeyJim,howdoyouinterpretanadjustedRSquareresult.Foreg,Ihadtofindthecorelationbetweenexchangerateandstockprice,ItgavemeananswersayingtheadjustedRSquaretobe0.3925.Isthereacorelationandifthereisthanhowdoyouwritethat?Thanksinadvance

Reply

JimFrostAtMinitab 2monthsagoMod >SayeedHiSayeed,

That'sagreatquestion!

I'vewrittenabouthowweoftenuseadjustedRsquaredtohelpincludethecorrectnumberofpredictorsinthemodel.

However,thereisaspecificinterpretationforadjustedRsquare.AdjustedRsquaredprovidesanunbiasedestimatedofthestrengthoftherelationshipbetweenthepredictorsandresponse.

RegularRsquaredisthestrengthofrelationshipinyoursamplebutitisabiasedestimateofthepopulationbecauseittendstobetoohigh.AdjustedRsquaredis"shrunken"soitisnotbiased.

Foryourresults,themodelaccountsforanestimated39.25%ofthevariabilityintheresponseinthepopulation.WhatevervaluetheregularRsquaredis,itonlyappliestoyoursample.

IwroteanentirepostaboutthisthatIrecommendyouread:Rsquaredshrinkage.

Thanksforwriting!Jim

Javaid 2monthsago>JimFrostAtMinitabIhaveaquestion:

Share

Share

Share



RegressionEquation

MR=0.00349+0.003154A+0.16467B+0.000595C

andthatgivesmeModelSummary

SRsqRsq(adj)Rsq(pred)

0.001568899.56%99.30%98.23%

AmIcorrectinassumingthatthevalueofRsqis0.9956?

Visit Us at Minitab.comBlog Map http://blog.minitab.com/sitemap.html | Legalhttp://www.minitab.com/legal/ | Privacy Policyhttp://www.minitab.com/legal/#privacypolicy | Trademarkshttp://www.minitab.com/legal/trademarks/

Copyright 2015 Minitab Inc. All rights Reserved.

Who We AreMinitab is the leading provider of software and services for qualityimprovement and statistics education. More than 90% of Fortune 100companies use Minitab Statistical Software, our flagship product, andmore students worldwide have used Minitab to learn statistics than anyother package.

Minitab Inc. is a privately owned company headquartered in State College,Pennsylvania, with subsidiaries in the United Kingdom, France, andAustralia. Our global network of representatives serves more than 40countries around the world.

AuthorsCarly Barryhttp://blog.minitab.com/blog/realworldqualityimprovementPatrick Runkelhttp://blog.minitab.com/blog/statisticsandqualitydataanalysisJoel Smithhttp://blog.minitab.com/blog/funwithstatisticsKevin Rudyhttp://blog.minitab.com/blog/thestatisticsgameJim Frosthttp://blog.minitab.com/blog/adventuresinstatisticsGreg Foxhttp://blog.minitab.com/blog/dataanalysisandqualityimprovementandstuffEric Heckmanhttp://blog.minitab.com/blog/startingoutwithstatisticalsoftwareDawn Kellerhttp://blog.minitab.com/blog/adventuresinsoftwaredevelopmentEston Martzhttp://blog.minitab.com/blog/understanding



statisticsKaren Meldrumhttp://blog.minitab.com/blog/statisticstipsfromatechnicaltrainerBruno Scibiliahttp://blog.minitab.com/blog/applyingstatisticsinqualityprojectsEduardo Santiagohttp://blog.minitab.com/blog/understandingstatisticsanditsapplicationCody Steelehttp://blog.minitab.com/blog/statisticsandqualityimprovement

How to Interpret Regression Analysis Results_ P-values and Coefficients _ Minitab.pdf

Documents

Transcript of How to Interpret Regression Analysis Results_ P-values and Coefficients _ Minitab.pdf