ClearlyandSimply

IntelligentDataAnalysis,Modeling,SimulationandVisualization
Home
Archives
Subscribe
Subscribebyemail
About
EmailMe
Thursday,May13,2010

OptimizationModelsandAlgorithmsusingMicrosoftExcelandVBA
IsMicrosoftExcel(includingVBA)aviableplatformforthedevelopmentofcomplexmathematical
optimizationmodelsandalgorithms?
Wheneveritcomestoimplementbusinesscalculationmodels,probably
everyoneimmediatelyconsidersusingMicrosoftExcel.Exceliswidelyspread
incorporateenvironmentsandmostpeoplearefamiliarwithusingittoa
greaterorlesserextent.ThemostexcitingthingaboutExcel,however,isits
apparentlyunlimitedflexibility.Exceldoesthemath,imports,storesand
exportsdata,createsvisualizations,providesapredefinedtabularstructure,
includesabuiltinprogramminglanguage,etc.
Sometimesthisflexibilityisboonandbane.Thefactthatitseemsasifyou
coulddoanythingwithMicrosoftExceldoesnotmeanyoushould.Ihaveseen
peoplemisusingMicrosoftExcelasawordprocessor,adatabase,aproject
planningsoftwareoragraphicalapplication.Mostofthoseworkbooks
remindedmeofthefollowingquotebyAbrahamMaslow(ThePsychologyof
Science,1966):
Iftheonlytoolyouhaveisahammer,
youtendtoseeeveryproblemasanail.
Havingsaidthis,itisdefinitelyappropriatetoalwaysquestioninadvancewhetherthetoolyouareintendingtouse
istherightoneforthetaskyouarefacing.
Besidesfinancialplanning,simulationmodels,visualizationandprojectmanagement,optimizationmodelsand
algorithmsalwaysplayedanimportantroleinmyprofessionallife.Thus,thequestionwhetherMicrosoftExcelisan
appropriateoptionfordevelopingoptimizationmodelsandalgorithmsforcomplexcombinatorialproblemssuggests
itself.Todayspostdiscussesthisquestion,includingtheprosandconsaswellasthepossibleusecases.
Thebackground
EveryversionofMicrosoftExcelcomeswithafeatureprovidingalgorithmstosolvelinearandnonlinearproblems
(theSimplexalgorithmandtheGeneralizedReducedGradientMethod):TheSolveraddin.
Butwhatifyouwanttosolveothercombinatorialproblemsusingotheralgorithmsandheuristics?IsMicrosoftExcel
aviableplatformforthisaswell?
Herearetheprosandcons:
Thepros
Areadytouseandfamiliarenvironment
YouhaveMicrosoftExcelinstalledonyourcomputer,dontyou?Andyouarefamiliarwithusingit,arentyou?

Well,thenyouarereadytogo.Youdonthavetoselect,installandlearnotherapplicationsorprogramming
environmentsandlanguages.SimplyopenMicrosoftExcelandstartdeveloping.
Anintegratedprogramminglanguage
VBAisaseasy(orhard)tolearnasanyotherprogramminglanguages.Therearetonsofwebsites,blogsand
forumsontheinternetwhereyoucanfindanswerstoalmosteveryquestionyoumayhave.MoreoverExcels
macrorecordercangiveyouasmoothstartintocreatingandunderstandingVBAcode.Althoughtherearesome
thingsonecouldwishfor,theVisualBasicEditor(VBE)isalreadyaratherconvenientenvironmentfor
developingcode.
Theallinoneapproach
MicrosoftExcelprovidesanallinonesolution.Youhaveyourinputdata,thecode,allcalculationresultsand
thevisualizations/thedashboardinonesinglefileandapplication.YourExcelworkbookisallyouhavetotake
careof:noexternaldatafiles,noadditionallibraries,etc.Ifyouwanttotransferyourworktoanothercomputer,
allyouhavetodoistocopyyourExcelworkbook.
Builtindataimportandexportfeatures
Excelprovidesdifferentwaysofimportingandexportingdata.Therangeisfromsimplecutandpastetousing
MicrosoftQueryforbuildingmoresophisticatedconnectionstoexternaldatabases.Whereveryourdatacome
from:MicrosoftExcelwillassistyouandminimizetheeffortandtimeneeded.
Builtindataanalysisfeatures
Youneedfurtheranalysisofyourinputdataortheresultsofyourmodel?Thereisnoneedforadditionalcoding.
Excelprovidesavarietyofbuiltinfeaturesandfunctionsthatwilldothejob.ThefunctionFREQUENCY,for
one,willbeofgreathelpifyouwanttocalculateandvisualizethedistributionofdata.ThefunctionsRANDand
RANDBETWEENorExcelsGoalSeekwillassistyouwithcreatingdummydata,etc.
Builtinvisualizationfeatures
Youdonothavetotakecareofvisualizingtheinputdata,youralgorithmandtheresults.MicrosoftExcelsbuilt
inchartengineandtheoptiontocombinecharts,tabulardataandinputcellsononesingledashboardshould
meeteveryrequirementyoumighthaveforvisualizingyouralgorithm.
Comparability
Developingalgorithmsrequirescomparisonsallover:comparetheresultsofthealgorithmfordifferentproblems,
comparetheresultsofdifferentparameterconstellations,comparetheresultsofyouralgorithmwiththeresults
ofanotheralgorithm,etc.Whenitcomestocomparedata,MicrosoftExcelisunbeatable:writetheresultsto
adjacentcellrangesandaddsimpleExcelformulastocalculatethedeviationasabsolutenumbers,as
percentages,etc.
Awidelyspreadapplication
Ifyouwanttodistributeyouroptimizationmodeltoothers,aMicrosoftExcelworkbookisprobablytheeasiest
wayofdoingthis.Othersdonthavetoinstallanewapplicationorgetusedtoanewuserinterface.Itsgoodold
Excel.Itisinstalledonalmosteverycomputerandmostpeoplearefamiliarwithit.Enablingmacroswhen
openingtheworkbookistheonlythingyourusershavetotakecareof.
Thebottomlineofthepros:
MicrosoftExcelandVBAprovideaflexibleandeasytouseenvironmentincludingcountlessoptionsofcreating
meaningfulvisualizations,convenientuserinterfacesandperformingadditionalanalyseswithintheworkbook.The
resultcaneasilybedistributedandalmosteveryoneisabletousethemodelwithouttheneedtoinstallnewsoftware.
Thecons

Limitedscalability
OptimizationmodelsandalgorithmsimplementedinMicrosoftExcelandVBAhavelimitationsregardingthe
scalability.Firstlythereisthewellknownmaximumsizeofworksheetranges(e.g.65,536rowsinExcel2003
andearlier).Secondlychangingthedimensionsoftheunderlyingproblem(likeadoptingtheobjectivefunction,
addingnewdecisionvariablesoraddingnewconstraints)usuallyrequireschangesintheworksheetstructures
andtheVBAcode.AdoptingdeclarationstatementswouldtakeafewsecondsinC#orC++sourcecode.
ChangingtheaccordingstructureofaMicrosoftExcelworkbookandtheaccordingVBAcode,however,could
turnintolaboriouswork,ishighlyerrorproneandrequiresadditionaleffortsforretesting.
Poorperformance
VBAmacrosincludedinMicrosoftExcelworkbooksareinterpreted,i.e.VBAisalinebylineapproach.Eachline
isread,checkedforsyntaxerrors,compiledandthenexecuted.Althoughsomeofthisworkisdoneinadvance,
thisapproachleadstoconsiderablylongerruntimesthanusinganapplicationdevelopedinacompilerbased
programminglanguagelikeC#.EspeciallytheinteractionoftheVBAcodewiththeMicrosoftExcelworkbook,
likereadingandwritingdatafromandtoworksheetcellrangesorupdatingthescreen(i.e.
Application.ScreenUpdating=True),significantlyincreasestheruntime.
SomeperformanceimprovementcanbeachievedbytransformingtheVBAmacrosintoExceladdins.However,
creatingaddinsslowsdowntheprocessofdevelopmentandtakesawaypartsoftheflexibilityofusingExcelas
adevelopmentenvironment.Andevenworseattheendoftheday,aVBAbasedaddinwillstillbeslower
thanletssayC#compiledapplications.Ifyouhavetosolvereallylargescaleproblems,youprobablyhaveto
transferyourcodeintoacompilerprogramminglanguage.
Lackofstabilityandhigherrorproneness
Aslongasyoudonotprotecttheworksheetsortheentireworkbook,aspreadsheetisalwaysaveryfragile
construct.Changestotheworksheets(intendedorunintended)caneithermakethewholemodelunusableor
evenworseleadtowrongresults.Insertingordeletingrowsorcolumns,deletingorrenamingcellrangenames,
changingformulas,etc.canandusuallywillhaveunpredictableconsequences.
Myfavoriteexample:arrayformulas.Letsassumesomeoneenterstheformulabarofacellincludinganarray
formulaandleavesthebarusingEnter(insteadofCtrlShiftEnter).Ifheslucky,thiswillleadtoanerrorandhe
willatleastnoticethatsomethingiswrong.Ifnot,themodelwillstillwork,buttheresultswillbeplainwrong.
Suboptimaloptionsfordocumentationandmaintenance
Letscallaspadeaspade:alotofspreadsheetmodelsandapplicationsarenotorinsufficientlydocumentedand
thereforehardtomaintain.Themorecomplexaspreadsheetis,theharderitistounderstandwhatitdoesand
howitisworking.Forsure,MicrosoftExcelisntthecreamofthecropwithregardstoitsdocumentation
capabilities.
However,therearesomewaysofhowtomitigatethisweakness.Itisaquestionofdisciplineforthedeveloper
andaquestionofknowhowfortheonetryingtounderstandthemodel.Thedevelopercanimprovethe
documentationbye.g.commentingtheVBAcode,usingrangenames,describingcellsandcellrangesonthe
worksheet,insertingcellcomments,etc.Ifyouaretheonewhotriestounderstandsomeoneelsesmodel,you
cantakeadvantageofalotofExcelsbuiltinfunctionality,likethetracingfeatures,theformulaevaluator,the
formulaview,theGoToSpecialfeature,etc.
Thebottomlineofthecons:
Exceloptimizationmodelsdontscalewell.Slowperformance,lackingstabilityandhigherrorpronenessarethe
pricesyoupayforaconvenientandflexibledevelopmentenvironment.Lastbutnotleast,youneedalotofdiscipline
duringthedevelopmentinordertocometoawelldocumentedandmaintainablemodel.

Theusecases
Developandtestyourownalgorithm
Youhaveanewideaofhowtosolveacomplexmathematicaloptimizationproblem(eitheranexactalgorithmor
aheuristic)?Duringthedevelopmentofanalgorithm,theperformancecomessecond.Aconvenient,flexibleand
easytousedevelopmentenvironmentismoreimportantthantheruntimeofthealgorithm.
AndthisiswhereMicrosoftExcelcomesintoplay:ExcelsworksheetsincombinationwithVBAandtheVBEcan
servebothasyourdeveloperenvironmentandyouruserinterface.
Basicallyyouneedthreedifferenttypesofworksheetsinyourworkbook:
1. Thedataworksheet:storetheexampleproblem(s),eithercopiedfromothersourcesorgeneratedbyusing
MicrosoftExcelsfunctionsRANDorRANDBETWEEN
2. Thedashboardworksheet:aonepageoverview,includingmeaningfulvisualizationsofparametersand
results,inputcellsforchangingthealgorithmsparameters,outputcellsformonitoringtheresultsand
someinteractivefeaturestoe.g.switchbetweendifferentproblems
3. Calculation/controlworksheet(s):oneorseveralworksheetstostoretemporaryresultsofthealgorithmand
tosetupthechartsonthedashboard
AftersettinguptheworkbookanddevelopingyouralgorithminVBA,youcanconductseveralrunsfordifferent
problemsanddifferentparameterconstellations,watchthealgorithmworkandfindoutitsstrengthsand
weaknessesaswellastheopportunitiesforimprovements.
Understandhowanexistingalgorithmworks
RebuildanalreadyexistingalgorithmwithMicrosoftExcelandVBA.Implementinganalgorithmfromscratchis
quiteaneffort,butitdefinitelypaysoff.Itguaranteesthatyoufullyunderstandtheprinciple,thestrengthsand
theweaknessesofthealgorithm.
Comparedifferentalgorithms
UsingMicrosoftExcelasyourintegrateddevelopmentenvironmentmakesiteasytoimplementdifferent
algorithmsforthesametypeofproblemandtodirectlycomparethemregardingthewaytheyareworking,their
resultsandtheirperformance.
Optimizeexistingalgorithmsbyrebuildingandtuningtheoriginal
Thisoneisacombinationofthethreeusecasesmentionedabove:implementtheexitingalgorithm,buildyour
ownversionwithinthesameworkbookandcomparethem.
Performscenarioanalyses
Awelldesigneddashboardincludingallrelevantparametersofyouralgorithmshelpsyoutoconductvarious
whatifanalyses,toidentifythemostrelevantdriversandtofindtheoptimalparameterconstellationforagiven
problem.
Presentation
Buildauserfriendlydashboardandpresenttheprinciples,thefunction,theresultsandthesensitivitiesofthe
algorithmtoyourmanagementorclient.
Teaching/Training
UseMicrosoftExcelinyourOperationsResearchtrainingsandlectures.Meaningfulstepbystepvisualizations
combinedwiththeflexibilityofdoingfurtheranalysisandownmodificationswillhelpyourstudentstobetter
understandthealgorithms.
EnhanceexistingMicrosoftExceldecisionsupportmodelswithnewalgorithms

AddintelligentoptimizationtechnologiestoyourexistingMicrosoftExcelbaseddecisionsupportsystemsand
improveyourresults.
Thislistisnonexhaustive.IamsuretherearesomemorefieldsofapplicationwhereusingMicrosoftExcelandVBA
isapossibleoptionforimplementingandusingoptimizationalgorithms.
Myconclusion
MicrosoftExcelandVBAcanbeaviableoptionfordeveloping,evaluating,comparing,presentingandteaching
optimizationmodelsandalgorithmsaslongasthecomplexityoftheproblem(s)islimitedandaslongasminimizing
theruntimeisnottheprimaryobjective.
References
Theuseofspreadsheetbasedapproachesformathematicaloptimizationhasalreadybeendiscussedinthe
OperationsResearchcommunityforalongtime.Hereareacoupleoflinkstoresourcesyoumayfindinterestingfor
furtherreadingonthistopic:
OperationsResearchModelsandMethods
Solver
WeighingtheProsandConsofDecisionTechnologyinSpreadsheets
ASpreadsheetBasedApproachforOperationsResearchTeaching
ImplementingLargeScaleOptimizationModelsinExcelusingVBA
Whatsnext?
ThenextpostwillpresentanexampleofsuchaMicrosoftExcelbasedoptimizationmodel:aselforganizingmapto
solveTravellingSalesmanProblems,includingtheexampleworkbookforfreedownload.
Staytuned.
PostedbyRobertat04:00PMinMicrosoftExcel,Optimization|Permalink
Reblog(0)|

Tweet

Like

Comments

DanMurraydijo...
ItendtothinkofExcelasafinancialmodelingandforecastingtoolbecausethatisthewayI'veuseditmost.
However,Ihavetriedtouseitforinventoryplanning.
Whilelimitedit'sbetterthanthealternativeofdoingnothing.Plus,thepricepointofstatisticalmodelingtoolsisa
bithigherbuttherearesomedecentalternativesfrom:
http://www.statsoft.com/Statistica
Forcomplexfinancialmodelingusingaspreadsheetbuiltfroma"database"contextIthinkQuantrixModeler
http://www.quantrix.com/index.phpisafantastictool.Formulasarecreatedusingdimensionsversuscells.
ThisprovidesameansforveryelaboratemodelingwithfarfewerformulasthanacellbasedsolutionlikeExcel.
I'mnotaffiliatedwitheitherStatSoftorQuantrixbutIdobelievethesesolutionsofferreasonableaffordabilityand
betterperformance/capabilityinthisrealmthanExcelinmanyusecases.
Robert,I'dbecurioustohearwhatyouthinkaboutQuantrixinparticular.Itwouldseemtobetheperfectcompanion
foratoollikeTableau.
ResponderSaturday,May15,2010at01:45PM

RobertdijoenrespuestaaDanMurray...
Dan,
manythanksforyourideasandtheinterestinglinks.
Ifullyagreewithyou:mostofmyExcelworkbooksaresimulationmodels(financials,pricing,revenuemanagement,
loyaltyprogramsandothers)andforecastingtoolstoo.
True,Excel'scapabilitiesaresomehowlimitedoryouneedalotofveryspecialknowhowtoovercomeitsweaknesses.
NeverthelessItrulyloveExcelsflexibility.That'swhyIwasaskingmyselfwhetheritcouldbeusedforoptimization
modelsaswell.Theupcomingarticleherewillhopefullyshowthatitispossibletoacertainextent.Yourthoughts
andcommentsonthenextpostwouldbehighlyappreciated.
IwilldefinitelyhavealookonStatsoftandQuantrixandcomebackwithmy2cents.Thanksforthehints.
ResponderSaturday,May15,2010at02:56PM
Commentbeloworsigninwith Typepad Facebook

Twitter

Google+andmore...

(ParadarformatoaltextopuedesusaretiquetasHTMLcomo<b><i>y<ul>.URLs
enlazadasautomticamente.)

Nosemostrarladireccindecorreoenelcomentario.

Nombre
Correoelectrnico
SitioWeb
Comentar Vistaprevia

ClearlyandSimply
PoweredbyTypePad