Beruflich Dokumente
Kultur Dokumente
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com
How
tohttp://blog.minitab.com/blog/projecttools2
Interpret Regression Analysis Results:
Pvalues
Project Tools
Minitab.com http://www.minitab.com
and Coefficients
Jim Frost http://blog.minitab.com/blog/adventuresinstatistics . 1 July, 2013
98
778
153
59 http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
Master Statistics
Anytime,
Anywhere
Quality Trainer teaches
you how to analyze
your data anytime you
are online.
The pvalue for each term tests the null hypothesis that the coefficient is equal to zero no
effect. A low pvalue < 0.05 indicates that you can reject the null hypothesis. In other
Take the Tour!
http://www.minitab.com/products/quality
words, a predictor that has a low pvalue is likely to be a meaningful addition to your model
trainer/?
because changes in the predictor's value are related to changes in the response variable.
WT.ac=BlogQT
Conversely, a larger insignificant pvalue suggests that changes in the predictor are not
associated with changes in the response.
In the output below, we can see that the predictor variables of South and North are
significant because both of their pvalues are 0.000. However, the pvalue for East 0.092 is
greater than the common alpha level of 0.05, which indicates that it is not statistically
significant.
Typically, you use the coefficient pvalues to determine which terms to keep in the
regression model. In the model above, we should consider removing East.
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
1/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
The fitted line plot shows the same regression results graphically.
The equation shows that the coefficient for height in meters is 106.5 kilograms. The
coefficient indicates that for every additional meter in height you can expect weight to
increase by an average of 106.5 kilograms.
The blue fitted line graphically shows the same information. If you move left or right along
the xaxis by an amount that represents a one meter change in height, the fitted line rises or
falls by 106.5 kilograms. However, these heights are from middleschool aged girls and
range from 1.3 m to 1.7 m. The relationship is only valid within this data range, so we would
not actually shift up or down the line by a full meter in this case.
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
2/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
If the fitted line was flat a slope coefficient of zero, the expected value for weight would
not change no matter how far up and down the line you go. So, a low pvalue suggests that
the slope is not zero, which in turn suggests that changes in the predictor variable are
associated with changes in the response variable.
I used a fitted line plot because it really brings the math to life. However, fitted line plots can
only display the results from simple regression, which is one predictor variable and the
response. The concepts hold true for multiple linear regression, but I would need an extra
spatial dimension for each additional predictor to plot the results. That's hard to show with
today's technology!
The residual plots not shown indicate a good fit, so we can proceed with the interpretation.
But, how do we interpret these coefficients? It really helps to graph it in a fitted line plot.
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
3/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
You can see how the relationship between the machine setting and energy consumption
varies depending on where you start on the fitted line. For example, if you start at a machine
setting of 12 and increase the setting by 1, youd expect energy consumption to decrease.
However, if you start at 25, an increase of 1 should increase energy consumption. And if
youre around 20, energy consumption shouldnt change much at all.
A significant polynomial term can make the interpretation less intuitive because the effect of
changing the predictor varies depending on the value of that predictor. Similarly, a
significant interaction term indicates that the effect of the predictor varies depending on the
value of a different predictor.
Take extra care when you interpret a regression model that contains these types of terms.
You cant just look at the main effect linear term and understand what is happening!
Unfortunately, if you are performing multiple regression analysis, you won't be able to use a
fitted line plot to graphically interpret the results. This is where subject area knowledge is
extra valuable!
Particularly attentive readers may have noticed that I didnt tell you how to interpret the
constant http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowto
interprettheconstantyintercept. Ill cover that in my next post!
Be sure to:
Check your residual plots so you can trust the results
http://blog.minitab.com/blog/adventuresinstatistics/whyyouneedtocheckyour
residualplotsforregressionanalysis
Assess the goodnessoffit and Rsquared http://blog.minitab.com/blog/adventures
instatistics/regressionanalysishowdoiinterpretrsquaredandassessthe
goodnessoffit
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
4/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Comments
Name: Lovemore Friday, January 24, 2014
That's sounds great but for me I am finding difficult how do I instigate a six sigma project in a medical laboratory using so
of the Minitab tools
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
5/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
6/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
49Comments
TheMinitabBlog
Recommend
Share
Login
SortbyOldest
Jointhediscussion
Joel 6monthsago
Hello,
Ifittedthemodely=a+bX1+cX2+dX1.X2+e(X1)^2+f(X2)^2onadatasetbutIhavesomeproblemsin
interpretingthepvaluesofthecoefficients.IfIusenormalizedvaluesforX1andX2(smallestvalue:1,
largestvalue:+1)andIperformaregressionIgetdifferentpvaluesforthecoefficientsa,bandc(notd,e
andf)comparedtottherealvalues.Infactformydatasetp<0.05forX2ifInormalizemydatabut>0.05for
therealvalues.SoIguessnormalizationistobedonealwaystoanalyzedata?
Thanksinadvance.
Jol
1
Reply Share
inez 6monthsago
Inmylinearregressionresults,whatdothetvaluesmean?caniputthemintableofresults?
Reply Share
JimFrostAtMinitab
HiInez!Thanksforwritingwiththeexcellentquestion!
Thetvalueisastatisticthatmeasurestheratiobetweenthecoefficientanditsstandarderror.
Minitabusesthetvaluetocalculatethepvalue,whichyouusetomakeadecisionaboutthe
statisticalsignificanceofthetermsandmodel.
Asufficientlylargeratioindicatesthatthecoefficientestimateisbothlargeandpreciseenoughtobe
significantlydifferentfromzero.Conversely,asmallratioindicatesthatthecoefficientestimateistoo
smallortooimprecisetobecertainthatthetermhasaneffectontheresponse.
Youcanusethetvaluetodeterminewhethertorejectthenullhypothesis.However,thepvalueis
usedmoreoftenbecauseitiseasiertointerpret.
Unlessyouhaveaspecialneedtoincludeit,Iwouldnotincludeitinyourresults.
Jim
Reply Share
Cain 5monthsago
HowcanItellthelevelofsignificancefromanoutput?IhaveanexamusingminitabandI'mnotsure
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
7/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Reply Share
JimFrostAtMinitab
Hi,thatsoundslikeatrickquestiontome.Thesignificancelevel(alpha)issomethingthatyoushould
choosebeforeyouperformyourstudy.Afteryouperformtheanalysis,youcomparethepvaluesin
theoutputtoyoursignificancelevel.
Jim
Reply Share
WDC123 5monthsago
HiJim,IfIreducethemodelbytakingouttermswithpvalueslessthank0.05andthennoticethatRsquared
hasalsoreducedhowdoIexplain.ShouldIconsiderleavinginsometerms?
Reply Share
JimFrostAtMinitab
Hi,typicallyyouconsiderremovingpredictorsfromthemodelifthepvalueisgreaterthanyour
significancelevel.I'llassumethatiswhatyoumeanttotype!:)
It'sfairlytypicalfortheRsquaredtodeclineasyouremovepredictors,evenwhenthosepredictors
arenotsignificant.
Hereareacoupleofsuggestions:
*UseadjustedRsquaredtocomparemodelswithdifferentnumbersofterms.
*Don'tchoosethemodelbasedsolelyonthehighestRsquaredbecausethatcanleadyouastray.
*Useyourexpertise,theory,andcommonsenseratherthanrelyingsolelyonsimplisticmodel
selectionrules.
Foryourcase,don'tfeellikeyoushouldincludethoseinsignificantpredictorsjusttogetthehigherR
squared.However,youcanconsiderincludingthemiftheorysuggeststhattheybelonginthemodel.
Ingeneral,youshouldalreadyhaveanideaofwhattheimportantvariablesarealongwiththeir
relationships,coefficientsigns,andeffectmagnitudesbasedonpreviousresearch.
There'snotalwaysaclearansweronwhichpredictorsyoushouldincludeinyourmodel.Useboth
thestatisticaloutputandtheoretical/subjectareaconsiderationstohelpyoudecide.
Thanksforwritingwiththegreatquestion!Selectingthecorrectmodelhasalwaysbeenavery
interestingsubjectforme!
Jim
Reply Share
Ronja 4monthsago
Hello,
myquestionisquitesimilartotheothers:
inordertodevelopaforecastIwanttousemultipleregression.Itriedvariousindependentvariablesthat
wouldallmakesense(meaningtheyallmayhaveanimpactontheforecast)togainthebestsuitedequation
fortheforecast,butIfinditdifficulttochoosetherightsetofvariables.Withtheonesetofindependent
variables,mypvaluesarehigherthan0,05(theyare0,12)howevertheRsquaredishighestwith0,9904.
Takingouttermswithpvalueshigherthan0.05won'tworksincethentherewon'tbeanyleft.Withtheother
set,myRsquaredisjust0.8473howeverthepvaluesarelessthan0,05.Howdoyouselecttherightset?
DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformy
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
8/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformy
selection?
Thankyouverymuchinadvance!!!
Ronja
Reply Share
JimFrostAtMinitab
HiRonja,
Selectingthecorrectmodelcanbeaverydifficultprocessinsomecases.Readmyresponsetothe
commentdirectlyaboveyours(toWDC123)becauseitappliestoyourcaseaswell.
Specifically,don'tfeellikeyoumustgetthehigherRsquaredbecauseit'spossibletohaveanR
squaredthatistoohighandcauseproblems.YourRsquaredof0.99maybetoohighandcould
indicatethatyou'reoverfittingthemodel.Also,youshoulduseadjustedRsquaredtocompare
modelswithdifferentnumbersofpredictorsratherthanRsquared.
IsuggestthatyoureadmyblogpostaboutadjustedRsquared,whichcoversalloftheabovepoints.
AsforpvaluesversusadjustedRsquaredvalues,researchhasshownthatusingpvaluesina
stepwisemannergenerallyworksbetterthanusingadjustedRsquaredtopickthecorrectmodel.
However,usinganysimplemodelselectionprocedurelikethatgenerallydoesnotpickthecorrect
model.I'vewrittenanotherpostaboutthisissuewhereIcomparestepwisetobestsubsets
regression.
Theimplicationsofthesefindingsareprofoundevenifyou'renotusingeitheroftheseautomated
methods.Thefindingsshowthatchoosingthecorrectmodelisasmuchascienceasitisanart.The
seemore
Reply Share
Fiachra 4monthsago
Hi,AfterrunningmyregressionIendedupwithpvalueslike6.9345E05.WhatdoesthisEmeanandhow
doIworkoutthePvaluethanks.
Reply Share
JimFrostAtMinitab
Thatiscalledscientificnotationandisusedtowriteverylargeandverysmallnumbers.Itworksby
shiftingthedecimalpointleftorrightbythenumberofplacesindicatedaftertheE,whichstandsfor
exponent.
The05indicatesthatyouneedtotakethe6.9345andshiftthedecimalpointtotheleftby5places.
So,yourpvalueis0.000069345.That'saverylowvaluesoitisverysignificant!
Jim
Reply Share
Fiachra>JimFrostAtMinitab 4monthsago
Thanksamillion!myheadwaswreckedthinkingitwassomethingmuchmorecomplex.Ido
haveoneotherquestionhoweverinarecentmcqIwasgivenaregressionoutputbasedon
salary=b1+b2(Rank).(Rankbeingthequalityoftheindividualsuniversity,thebestwas
awardedarankof1andtheworstarankof142).Thecoefficientstheregressionproduced
fortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowp
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
9/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
fortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowp
valuessotheyweresignificant.Thequestionwaswhatisthetrueeffectofaoneplace
increaseinuniversityrankingsonsalaries.TheanswerIgavewas206.731butthecorrect
answeristhatitcannotbedeterminedfromthesefigures.(Figuresbeingapictureofa
regressionoutputinexcel).Whyisthisthecorrectanswer?Ithoughtthiswouldhavebeen
exactlywhatthecoefficientintheregressionindicates.
Thanks.
1
Reply Share
JimFrostAtMinitab
Ifthequestionaskedyouspecifically,whatwasthe"true"effect,youhaveto
rememberthatregression,andotherstatisticaltechniques,canonlyprovidean
estimateofthetrueeffect.It'sgenerallyimpossibletoeverknowthetrueeffectitself
becauseyou'reworkingwithasampleofthepopulationratherthantheentire
population.
Instead,inferentialstatisticscanonlyprovideanestimateofthetrueeffectandgive
youaconfidenceintervalforarangeofvaluesthatislikelytocontainthetrueeffect.
Inregressionanalysis,thecoefficientsaretheparameterestimates.
Reply Share
sewnsew 4monthsago
ihavearegressionmodelhowdoIcalculatethechangeinpwhenItakeoutvariablesoraddvariableback
intoamodeltoseewhichhasthemostpredictivevalue?InthedataIhave,Ihaveachangeinp,butin
SPSS,Idon'tseeanythingthatshowsorrelatestothechangeinp,sowhenIrerunthedata,Idon'tknow
whattolookfororwhattointerpretasachangeinp.Thanks.
Reply Share
JimFrostAtMinitab
Hi,Ican'tspeaktowhatyouseeinothersoftwarepackages.Also,I'mnotsurewhichpyouare
referringto.
Youmaywanttolookattheadjustedsumsofsquaresintheoutput.Thisindicatestheuniqueportion
ofthetotalsumsofsquaresthateachtermexplainsregardlessoftheordertheywereenteredinthe
model.Ifyouwanttofindouthowmuchvariationeachpredictorvariableaccountsforinamodel,this
iswhatyouneed.
Jim
Reply Share
Scott 4monthsago
Isthereanywaytoset/holdaparticularregressionequationcoefficientataparticularvalue,andthen
performtheregressionanalysis?
Inmyexample,Iamanalyzingpsioutvaluebasedonanumberofinputs,IwanttoholdPsiIncoefficientat
1,andlettheothervariablesbeapartoftheregression.Hopethismakessense,:/
Thanks!
Reply Share
JimFrostAtMinitab
HiScott,
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
10/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
HiScott,
That'saninterestingquestion.
Typically,you'refittingamodellikethis:
Y=B0+BX1+BX2+BX3...whereyouestimatetheBsfromthedata.
Youwanttofitthis:
Y=B0+1X1+BX2+BX3...wherethefirstcoefficientis1.
WhatyoucantrydoingismovingthetermwiththefixedcoefficientovertotheYsideoftheequation:
Y1X1=B0+BX2+BX3
You'dhavetocreateanewcolumnofresponsedatawhereyoutaketheoriginalmeasureand
subtractoutthe1X1.Inyourcase,you'dtaketheoutputPSIandsubtracttheinputPSIforeach
observationandusethenewlycalculatedvaluesastheresponse.Then,includetherestofthe
predictorsinthemodel.
You'dessentiallybelookingathowthepredictorsarerelatedtothechangeinPSIratherthanthe
absolutePSI,whichsoundspromisingifIunderstandyourscenariocorrectly.
Theestimatesfortheotherpredictorswouldbethevaluesifforcedthefirstpredictortoequal1.
You'dhavetobecarefulhowyouinterpretthemodelfitvalues.Forexample,Rsquaredindicates
howmuchvariationyouaccountforwiththenewresponsevariable.
Jim
Reply Share
sewnsew 4monthsago
InmyhomogeneoussubsetstheNisdifferentthantheNthatIgotwhenIranfrequencies.Why?Isthis
normal?
Reply Share
SharonEdgeWilkie 4monthsago
ThisisaPostHocquestion.WhyaretheNinmyhomogeneoussubsetsnotthesameastheNinmy
frequencycharts?
Reply Share
PatrickKajubili 4monthsago
Hi,
Iamstilljuniorinthefield.iwanttoknowifihaveF714andSig761inmyANOVA
tablewhatdoesthismean?Havingsiglikedoesshowmodelfit?
Reply Share
JimFrostAtMinitab
HiPatrick,theFstatisticisatestoftheoverallsignificanceoftheregressionmodel.WhileR
squaredandadjustedRsquaredtellyoutheoveralldegreeofthefitforaregressionmodel,they
don'tprovideaformalhypothesistestfortheoverallfit.
That'swheretheFtestanditsassociatedpvaluecomesin.
ThenullhypothesisfortheFtestisthatallofthecoefficientsintheregressionmodelequalzero.Ifall
thecoefficientsequalzero,thisisequivalenttosayingthatthefittedvaluessimplyequalthemeanof
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
11/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
theresponsevariable.Inotherwords,yourmodelpredictstheresponsenobetterthanusingthe
responsemean.
Thealternativehypothesisisthattheydon'tallequalzero.Or,thatyourmodeldoesprovidebetter
predictionsthanjustusingthemean.
Alowpvaluemeansthatyoucanrejectthenullandconcludethatyourmodelisbetterthanjust
usingthemeanandthatatleastonecoefficientdoesn'tequalzero.
You'llstillneedtochecktheresidualplotsbecausethistestwon'ttellyouwhetherthemodel
providesanadequate,unbiasedfit.InthebulletsneartheendofthispostIprovidealinktoablog
postIwroteaboutcheckingtheresidualplots.
Thanksforwriting!
Jim
Reply Share
Marija 3monthsago
Hello,Ineedyurhelpaboutmyexamquestion:
(i)
Estimatethefollowingregressions
PRICE=b1+allindependentvariables+ut
LnPRICE=b1+allindependentvariables+ut
Accordingtotherelevantcriteria,judgewhichoneisbetter.Continueworkingwiththebetterfromthetwo.
Fullyinterpret(statisticalandeconomicsignificance)theresultsofhedonichousepriceestimation.
Myquestion:Whicharethecriteriatodecidewhichisabetterregression?
Ihavecalcualtedthembothandherearetheresults(valuesonlyfromthevariableswithsig.<0.05:
PRICE=27978,841+(140,661)+372,079+(12080,847)+11032,510+
(5154,908)+7478,822=29485,84
LnPRICE=10,5030,003+0,0060,247+0,1240,075+0,1126=10,42026
Ineedaninfofromyouinordertocontinueinterpretingtheresultsbasedonthebetterregression
Thankyouverymuchinadvance
Reply Share
JimFrostAtMinitab
HiMarija,
InadditiontothefactthatIreallyshouldnotansweryourexamquestionforyou,Ireallycan'tanswer
thequestionwiththeinformationthatyouprovided.Thereisinsufficientinformationtobeableto
choose.But,Icangiveyousomegeneralguidelinesonhowtochoose.
Youshouldchecktheresidualplotsforbothmodels.Iftheplotslookgoodforonemodelbutnotthe
other,thatwillhelpyouchoose.
Youshouldalsolookatthecoefficientsforthepredictorsanddeterminewhethertheymatchtheory.
Forexample,ifonemodelsuggeststhatagoodcharacteristiclowerstheprice(negativecoefficient),
youshouldseriouslyquestionthatmodel.
Thosearethetypesofthingsyouneedtoassesstodeterminewhichmodelisbetter.Irecently
wroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpful
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
12/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
wroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpful
informationforyou!
Goodluckwithyourtest!
Jim
Reply Share
wuyr 3monthsago
HelloJim,
Thanksalotforyourposting.Itisveryhelpful.IhaveanofftopicMinitabquestion,andhopingthatyoucould
helpmeout.DoesminitabhasafunctionlikeVlookupinexcel?
Thanksalot.
Yan
Reply Share
JimFrostAtMinitab
Hi,thankyouforthenicecomment!
Unfortunately,Minitabdoesn'thaveanexactlyequivalentfunction.However,inMinitab,youcanuse
ControlFtousetheFindinDataWindowfunction.Thiswillsearchwithinacolumnforaspecific
value,eitherexactmatchornot.Whenitfindsamatchinacell,youcanlookattheassociated
informationinthethatrowasawaytomimicthefunctionalityofVLOOKUP.
Jim
Reply Share
JackWotton 3monthsago
Hijim,
I'mabletoexplainmyresultsthroughthepvalue,s=,rsq,andthegraphs.butiamunsureonothervalues
thathaveshownupe.g.,DF,SS,MF,F,(howtointerprettheresidualerrortomyresults?whatdoesDF20,
SS235.57MS11.78allmean)ithinkthismostlyrelatestotheanalysisofvarience.hopeyourabletohelp
asihaveadissertationhandinnextmonth)cheers
Jack
Reply Share
JimFrostAtMinitab
HiJack,
Alotofthesestatisticsarethe"behindthescenes"typeofnumbersthatMinitabneedstocalculatein
ordertocomputethemorecommonstatisticsthatpeopleneed,likethepvalues,Rsquared,
adjustedRsquared,andS.Unlessyouhaveaspecialneed,youoftendon'tneedthestatisticsthat
youlist.
I'llrunthroughthemingeneralforyou.Ifyouneedmoredetailedinformationabouthowthey're
calculated,youcanalwayslookattheMethodsandFormulaHelpinMinitab:Help>Methodsand
Formulas.TheMinitabGlossary(Help>Glossary)alsohasdefinitionsoftheseterms.
DF:Thedegreesoffreedom(DF)describetheamountofinformationyourdataprovidethatyoucan
"spend"toestimatethevaluesofunknownpopulationparameters,andcalculatethevariabilityof
theseestimates.Degreesoffreedomareaffectedbythesamplesizeandthenumberofparameters
inyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,and
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
13/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
inyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,and
consequentlyincreasesthedegreesoffreedompresentinyourdata.Addingparameterstoyour
model(byincreasingthenumberoftermsinaregressionequation,forexample)"spends"
informationfromyourdata,andlowersthedegreesoffreedomavailabletoestimatethevariabilityof
theparameterestimates.
seemore
Reply Share
JackWotton>JimFrostAtMinitab 3monthsago
Thankyousomuchforyourhelp:)
Reply Share
Fardeen 3monthsago
HiMrJim.
Imhavinggreatproblemsindoingmydissertation.Idontknowhowtomakeuseofregression.Iwouldbe
gratefulifyoucouldhelpme.
Isthereasitewhereitshowsclearlytouseregression?
Thanks
Reply Share
JimFrostAtMinitab
HiFardeen,
Irecommendthatyoureadmyregressiontutorialwithexamples.Ithinkthiswillansweralotofyour
questions.
Bestofluckwithyourdissertation!
Jim
Reply Share
becbec>JimFrostAtMinitab 2monthsago
HiJim,thankyousomuchfortheinformativediscussionshere.Iammakingmythesis
however,Iamfindingdifficultiesininterpretingmydata.Whatdoesthisresultmeanifmy
constanttvalueis7.114,pvalue=.000,LIFCAStvalue=10.228pvalue.000,LERIANSt
valueis2.971pvalue.003,andEFCOStvalue,2.186andpvalue.029.iwouldappreciate
yourhelp.thanks.
Reply Share
JimFrostAtMinitab
Hi,
Withtheinformationyouprovide,Ican'tbesurethatyourmodelmakessense
theoreticallyorwhetherthemodelprovidesanadequate,unbiasedfittothedata.One
thingyoushoulddoisdefintelycheckyourresidualplots.
Assumingthemodelisgood,here'swhatyou'vegot.
Youhaveaconstanttermthatissignificantlydifferentfromzero.However,the
constanttermusuallyhasnomeaningfulinterpretation.There'salinktoablogpostI
wroteaboutwhythisistrueneartheendofthisblogpost(beforethecomments
section).
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
14/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Youhave3significantpredictors.Thissuggeststhatchangesineachpredictorare
relatedtochangesintheresponse.Forexample,aoneunitincreaseinLIFCASis
relatedtoanincreaseinthemeanresponsevalueequaltotheLIFCAScoefficient.
SameforLERIANS.ForEFCOS,everyoneunitincreaseisrelatedtoadecreasein
themeanresponse(youdidn'tincludethecoefficientsbutfromthetvalueIknowthat
theEFCOScoefficientisnegative).
Typically,youdon'tneedtoworryaboutthetvaluesandinsteadfocusonthep
valuesandcoefficients.
Youmightwanttoreadmyblogpostaboutchoosingthebestregressionmodelto
helpyoubesurethatyoudohavethebestmodel!
Bestofluckwithyourthesis!
Jim
Reply Share
dunmao>JimFrostAtMinitab 2monthsago
HiJim,
Couldyoupleasegivemeadirectionforthefollowingquestion?
Myquestions:Iamdoingridershipmodelingusingmultiple
linearregressionmethodinExcelsoftware.Mydependentvariableisboardings,
threeindependentvariablesarepopulation,feederbusservices,andemployment
data.Eventhoughtheconstantismeaninglessdiscussedfromyourdiscuss
group.Inmycase,thepvalueforYinterceptis0.6(greatthan5%),however
theYinterceptcanminimizetheresidual(observeddatapredictedvalue).
Seetheregressionresult:
RSquare=0.943573,
PvalueforYintercept=0.6,Pvaluesforthethreeindependentvariablesare
lessthan5%
AccuracyValidationwithoutYintercept(ObservedPredicted):
seemore
Reply Share
JimFrostAtMinitab
Hi,Irepliedtoyourquestionintheotherpostwhereyousharedyourcomment.You
canfindithere.
Theshortansweris,yes,youshouldalmostalwaysincludetheconstantregardless
ofthepvalue!
Jim
Reply Share
dunmao>JimFrostAtMinitab 2monthsago
HiJim,
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
15/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
HiJim,
Thankyousomuchforyourquickresponse!
Iwanttoincludetheconstanteventhoughthepvalueoftheconstantisgreatthan
5%.Theconstantcanbeexplainedasanadjustedfactorinmypredictionmodelto
minimizetheerror.
Youranswerconfirmsmytestresults.
Thanksagain,
Hope
Reply Share
JimFrostAtMinitab
Hi,you'reverywelcome!
Justtoclarifyonepoint.Yougenerallyshouldincludetheconstantregardlessofthe
pvalue.Youdon'tneedajustificationtoincludetheconstant.Instead,youneeda
verystrongjustificationtoevenconsidernotincludingtheconstant.
Infact,I'veneverpersonallyworkedwitharegressionmodelwhereIfeltjustifiedto
notincludetheconstant.Aregressionmodelwithouttheconstantisveryrare
becausethepotentialforintroducingbiasisveryhigh.
Jim
Reply Share
dunmao>JimFrostAtMinitab 2monthsago
HiJim,
Icomeback.IhaveanotherpredictionmodelwithYinterceptpositive.Seethe
followings:
AccuracyValidationwithoutYintercept(ObservedPredicted):
Predictedmodel:
DV_37pm=0.441*IV2+0.179*IV3
Error=Observed(3559)predicted(3961)=402(overestimated1678)
AccuracyValidationwithYintercept
Predictedmodel:DV_37pm=0.441*IV2+0.179*IV3+0.714
Error=Observed(3559)predicted(3971)=412(overestimated412)
seemore
Reply Share
JimFrostAtMinitab
Hi,
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
16/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Hi,
YoushouldalmostalwaysincludetheYinterceptinthemodel.Irecommendthatyou
doleaveitinthemodel.
ThisistrueregardlessofthePvalue.Ihopeyou'vereadmypostaboutthe
regressionconstant?Ishowthereasonswhyyoushouldalwaysincludeitinthe
model.
Ifyou'reboundanddeterminedtoconsiderremovingit,thereareimportant
considerationsyoumustevaluatefirst.
1)Checkthestandarderroroftheregression.TheErrorisinyouroutputisnotthe
standarderrorbecauseSisalwayspositive.Yourerrorreductionisnotsubstantial
anywayonlyfrom412to402.Theminisculereductioninerrorsuggestsyoumight
aswellleavetheconstantinthemodel.
2)Checkyourresidualplots.Inparticular,besurethattherearenononrandom
patternsforeithermodel.Thisisespeciallyimportantinthemodelwithouttheconstant
becauseoftenremovingtheconstantintroducesabiasthatyou'llseeintheresidual
plots.Ifyouremovetheconstantandyouseeapatterninresiduals,puttheconstant
backinyourmodel.
But,really,youshouldincludetheconstantevenwiththehighpvalue.It'snothurting
anythinganditislikelyhelpingreducebiasinyourmodel.
JIm
Reply Share
dunmao>JimFrostAtMinitab 2monthsago
ThankyouJim!
Iwanttolearnmore,soIcomparethetwocasesWiththeconstantinmyprediction
modelANDwithouttheconstantinmypredictionmodel.
WITHtheconstantinmypredictionmodel:
Standarderror:82
Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.
WITHOUTtheconstantinmypredictionmodel:
Standarderror:78
Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.
Therearenononrandompatternsforeithermodel.
Frommyunderstanding,theconstantissmall,sothereisnopatternintheresiduals
distributions.
Lastquestion:iftheconstantisbig,itcausestheerrorreductionsubstantial,doIstill
needtokeeptheconstant?(sorry,Idon'thavetheregressionresults,butIwantto
knowifthecaseexists.)
Thankyou,
Hope
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
17/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Hope
Reply Share
JimFrostAtMinitab
Hi,
Givenwhatyousay,theredoesn'tseemtobeanynumericreasontonotremovethe
constant.
However,beforeyoudothat,askyourselfifit'stheoreticallyjustifiedthatifyousetall
ofthepredictorstozero,you'dexpecttheresponsetoequalzeroaswell.Preferably,
youwouldalsohavemeasuredvaluesnear/atthisallzeroregiontoconfirmthatthe
regressionlinetrulygoesthroughtheorigin.
It'sonlywhentheconstantissmallthatyouhaveachance(smallchance)toremove
itfromthemodel.Ifitislarge,removingitfromthemodelwillalmostcertainlybias
yourmodel!Iwouldneverremovealargeconstant.
Jim
Reply Share
dunmao>JimFrostAtMinitab amonthago
HiJim,
Thankyousomuchforyourexplanation!Icompletely
understandtheconstant(regardlessofpvalue)now.
NowIhaveanewregressionresult:
R=99.35%,
AdjustR=99.06%
DV=20+0.129*IV1+0.178*IV2+0.078*IV3
Errors=observed(4088)predicted(4052)=36
Averageerrors=5.75%
Questions:WhyistheRsobigat99.35%?Maybesomeonewouldaskmeabout
thequestion.However,thisistrueregression
result.Howwouldyouexplaintheresult?
Thankyouagain,
Hope
Reply Share
JimFrostAtMinitab
Hi,
Withoutknowingthespecificsofthemodelandthestudyarea,it'simpossibletosay
forsure.IfIremembercorrectly,youaremodelingridershipovertime.Ifthereare
trendsinthedatathataffectbothsidesoftheequation,thisisaproblemandcanoften
produceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeif
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
18/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
produceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeif
theyarestationary(constantmeanandvarianceovertime)ornonstationary(upward
ordownwardtrendornonconstancevariance).
Ifyouhavenonstationarydata,youmustmakeitstationarybydifferencingthedata
sothateachdatapointisthechangeinvaluebetweenconsecutivepoints.Using
regressionanalysiswithtimeseriesdatainvolvesadditionalconsiderationslikethis.
Unfortunately,Idon'thaveahandyreferencetoreferyoutoobutyoushouldperform
someadditionalresearchtoensurethatyouendupwithavalidmodel.
Jim
Reply Share
dunmao>JimFrostAtMinitab amonthago
HiJim,
Inoticedanewquestion:
AsItoldyouIhavedonethetestingasthefollows:
=====================================================
Whenletintercept=0,theregressionresult:
Rsquared=0.96
AdjustedRsquared=0.88
StandardError=78
Observations=14
ANOVA:
df
Regression:2
Residual:12
seemore
Reply Share
Em 2monthsago
Hi,thankyouforyourextremelyhelpfulblogs!Iwaswondering,ifyoucanhelpmeoutwithmymultiple
regressionanalysis.ForthePearsoncorrelation,Ifoundthatonlyoneofmypredictorsissignificant
(p=0.037).However,Idon'tquiteunderstandwhyinthettestsection,noneofmyindependentvariables
makeasignificantcontributiontothemodel.Howisitpossible?Icouldn'tfigureoutthelinkbetweenthetwo.
Canyouexplainthis?Thanksinadvance!
Reply Share
JimFrostAtMinitab
Hi,
ThePearsoncorrelationpvaluesandregressionpvaluestestdifferentthingssotheanswersmay
notagree.Thecorrelationpvalueonlytestsonepairofvariablesatatimewithoutconsideringthe
othervariables.Theregressionpvaluesfactorinalltheotherpredictorvariablesthatareincludedin
themodel.
Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthe
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
19/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthe
predictorsandtheresponsevariable.Tryaregressionmodelwithjustthatonepredictor.Itshouldbe
significantinaregressionmodelbyitself.Then,addintheotherpredictors.Ifthesignificancegoes
away,itindicatesthattheotherpredictor(s)areaccountingforsomeofthesamevarianceinthe
response.Bysplittingupthevariancethatisaccountedforbetweenthevariables,itmaybethat
nonearesignificantwhenthereismorethanoneinthemodel.
Also,checkyourVIFsinthefullmodel.It'spossiblethatmulicollinearity(correlationbetweenthe
predictors)issappingthesignificanceofthepredictors.Theproblemsassociatedwith
multicollinearitydonotoccuronlywhenthereisastrongcorrelationbetweenindividualpairsof
predictors.Theseproblemscanoccurwhenthereisamoderatecorrelationbetweenanumberof
predictors.ThismoderatecorrelationmaynotbesignificantwhenyoulookatthePearsoncorrelation
betweenpairsbutcanbedetectedwithVIFs.Readmoreaboutthisinmypostaboutmulticollinearity
andVIFs!
Ihopethishelpsandthanksforwriting!
Jim
Reply Share
Sayeed 2monthsago
HeyJim,
howdoyouinterpretanadjustedRSquareresult.Foreg,Ihadtofindthecorelationbetweenexchange
rateandstockprice,ItgavemeananswersayingtheadjustedRSquaretobe0.3925.Isthereacorelation
andifthereisthanhowdoyouwritethat?
Thanksinadvance
Reply Share
JimFrostAtMinitab
HiSayeed,
That'sagreatquestion!
I'vewrittenabouthowweoftenuseadjustedRsquaredtohelpincludethecorrectnumberof
predictorsinthemodel.
However,thereisaspecificinterpretationforadjustedRsquare.AdjustedRsquaredprovidesan
unbiasedestimatedofthestrengthoftherelationshipbetweenthepredictorsandresponse.
RegularRsquaredisthestrengthofrelationshipinyoursamplebutitisabiasedestimateofthe
populationbecauseittendstobetoohigh.AdjustedRsquaredis"shrunken"soitisnotbiased.
Foryourresults,themodelaccountsforanestimated39.25%ofthevariabilityintheresponseinthe
population.WhatevervaluetheregularRsquaredis,itonlyappliestoyoursample.
IwroteanentirepostaboutthisthatIrecommendyouread:Rsquaredshrinkage.
Thanksforwriting!
Jim
Reply Share
Javaid>JimFrostAtMinitab 2monthsago
Ihaveaquestion:
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
20/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
RegressionEquation
MR=0.00349+0.003154A+0.16467B+0.000595C
andthatgivesmeModelSummary
SRsqRsq(adj)Rsq(pred)
0.001568899.56%99.30%98.23%
AmIcorrectinassumingthatthevalueofRsqis0.9956?
Who We Are
Authors
Carly Barry
http://blog.minitab.com/blog/real
worldquality
improvement
Patrick Runkel
http://blog.minitab.com/blog/statistics
andqualitydata
analysis
Joel Smith
http://blog.minitab.com/blog/fun
withstatistics
Kevin Rudy
http://blog.minitab.com/blog/the
statisticsgame
Jim Frost
http://blog.minitab.com/blog/adventures
instatistics
Greg Fox
http://blog.minitab.com/blog/data
analysisand
quality
improvementand
stuff
Eric Heckman
http://blog.minitab.com/blog/starting
outwithstatistical
software
Dawn Keller
http://blog.minitab.com/blog/adventures
insoftware
development
Eston Martz
http://blog.minitab.com/blog/understand
Visit Us at Minitab.com
Blog Map http://blog.minitab.com/sitemap.html | Legal
http://www.minitab.com/legal/ | Privacy Policy
http://www.minitab.com/legal/#privacypolicy | Trademarks
http://www.minitab.com/legal/trademarks/
Copyright 2015 Minitab Inc. All rights Reserved.
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
21/22
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
statistics
Karen Meldrum
http://blog.minitab.com/blog/statistics
tipsfroma
technicaltrainer
Bruno Scibilia
http://blog.minitab.com/blog/applying
statisticsinquality
projects
Eduardo Santiago
http://blog.minitab.com/blog/understand
statisticsandits
application
Cody Steele
http://blog.minitab.com/blog/statistics
andquality
improvement
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
22/22