ReportoftheMidTermEvaluation

ExpandingMaternalandNewbornSurvival
(EMAS)Program

USAID/Indonesia

April6May7,2014

Dr.BrotoWasisto

Dr.MeiwitaBudiharsana
Dr.MarjorieKoblinsky
Dr.AlfredBartlett

ExecutiveSummary

DuringAprilMay,2014,anindependentevaluationteamcommissionedbyUSAID/Jakartacarriedouta
midtermevaluationoftheExpandingMaternalandNewbornSurvival(EMAS)Program.TheUSAID
supportedEMASprogramisafiveyearcooperativeagreementwithJHPIEGOaimedatcontributingto
thereductionofmaternalandnewborn(MN)deathsinIndonesia.MaternalmortalityinIndonesia
remainssubstantiallyhigherthanexpectedincomparisonwithothersouthAsiancountrieshaving
similareconomiesandstatesofdevelopment.Also,despiteprogressinreducingmortalityamongolder
infantsandchildren,duringthepastdecadeIndonesiahasnotsubstantiallyreducedmortalityamong
newborns(thefirstmonthoflife).Asaresult,thecountryislosingmomentuminitsinchildsurvival.

TheEMASprogrambuildsuponanalysisofpreviousIndonesianmaternalchildhealthprogramming
approachesandUSAIDsexperienceinsupportingthoseapproaches.Inthe1990s,Indonesiaand
USAIDssupportfocusedontraininganddeploymentoflargenumbersofcommunitymidwives.More
recently,thisapproachexpandedtoimprovemanagementofroutinedeliveries.During20052010,
withdecentralizationoftheIndonesianhealthsystem,USAIDsupportedstrengtheningofdecentralized
MNhealthservicesandengaginglocalgovernment,whichhasbudgetandmanagementauthorityover
localhealthservices.During20102011,USAIDsupporteddevelopmentofapproachestoimprove
qualityandaccesstoMNcarethroughincreasinguseofevidencebasedlifesavinginterventions,
improvingthereferralsystem,andpromotingdistrictlevelproblemsolving.

Theseexperiencesledtorecognitionoftheneedtoimproveeffectivemanagementoftheillnessesand
complicationsthatresultinmaternalandnewborndeath.Therefore,USAIDandtheGovernmentof
IndonesiaagreedthattheEMASprogramshouldfocuson:
- Improvingdetectionandmanagementofcomplicationsatthepuskesmas(primaryhealthcare
center),wherecomplicatedMNcasesareexpectedtoenterthehealthsystem;
- StrengtheningtheeffectivenessandtimelinessofreferralofcomplicatedcasesfromPuskesmasto
thehospitallevelwheredefinitivemanagementissupposedtobeavailable;and,
- ImprovingqualityofcareandorganizationofservicesforcomplicatedMNcasesatreferralhospitals
EmergencyDepartment,MaternityUnit,OperatingRoom,andNeonatalUnit.
Withinthisfocus,EMAShasseveraloverarchingobjectives:
- Contributingto25percentnationalreductionsinmaternalandnewbornmortality;
- ImprovingqualityofemergencyMNcareinatleast150hospitals;and,
- Improvinglifesavingclinicalinterventionsandeffectivenessofreferralsinatleast300puskesmas.

TheEMASagreementwasawardedinSeptember2011;programimplementationeffectivelystartedin
2012,meaningthattheprogramhasbeencarryingoutitsprogramapproachesforjustunder2years,
withroughly2.5yearsremaining.TheEMASapproachhasthreemaincomponents,eachwithasetof
specificactivities:
- ImprovingqualityofemergencyMNcareatpuskesmasandreferralhospitallevels;
- ImprovingeffectivenessofMNemergencyreferrals;and,
- GeneratingpoliticalandcivilsocietydemandandsupportforimprovedMNservicesandimproved
outcomesofMNcomplications.

Useofinformation/communicationtechnologyinsupportoftheseapproachesisacrosscutting
elementofEMASsapproach.

i
EMASoperatesin6provinces(allareamongthe9provincesidentifiedbytheMinistryofHealthashigh
burdenMNpriorityprovinces).Byendofagreement,itiscurrentlyplannedthatEMASwillhave
operatedin30districts(ofwhich28areamongthe64identifiedbytheMOHashighburdenMN
prioritydistricts).

EMASproposeda3Phasestrategy.InPhase1,EMASimplementingpartnersplayastrongrolein
introducingthecomponentslistedabovethroughamultistagementoringapproach.Thefacilities,
Pokjas(oversightcommittees),andCivicForumsreceivingthisPhase1supportaredesignatedas
Vanguardorganizationswhentheyreachahighlevelofcompliancewiththekeycomponentsofthe
approach.TheseVanguardfacilitiesandorganizationsaretobethesourceofmentoringsupportto
Phase2facilitiesandorganizations.However,organizationwideimprovementhasnotbeenuniform,
althoughindividualunitsandindividualstaffhavereachedthelevelofcapabilityrequiredformentoring.
Forthisreason,EMAShasbegunPhase2byusingacombinationofitsownimplementingpartnersand
selectedmentorsfromPhase1facilities.EMAShasalsobegundevelopingexpertsfromprovincialand
staterunteaching(vertical)hospitalsasadditionalmentors,consistentwiththeroleofthesehigh
levelfacilities.

InPhase1(May2012September2013),EMASprovidedmentoringandassistanceto23hospitals,93
puskesmas,andassociatedstakeholderorganizationsin10districts.InPhase2(throughSeptember
2014),theprogramintendstoreach69additionalhospitalsand116puskesmasin13additionaldistricts
and6cities.Phase3(throughSeptember2016)proposestoreachadditionalservicesandorganizations
inanadditional7districts.

Thepurposesofthemidtermevaluationwereto:
- AssessEMASprogressinachievingthegoal,objectivesandplannedoutputsasstatedinthe
agreementsprojectdescriptionandinapprovedworkplans;
- ProviderecommendationstoimproveEMASprogrameffectivenessovertheremaining2+yearlife
ofproject;and,
- ProviderecommendationsforUSAIDtoconsiderinthedesignoffutureprojectsaimedatimproving
maternalandneonatalhealthinIndonesia.

TheevaluationwascarriedoutbyateamoftwoseniorIndonesianhealthexpertsandtwoUSbased
experts,allwithsubstantialexperienceinmaternal,child,andreproductivehealthandhealthsystems.
TheevaluationincludedreviewofUSAIDandEMASprogramrelateddocumentsaswellasasubstantial
numberofdocumentsrelatedtoMNhealthandhealthpolicyandsystemsinIndonesia;reviewand
analysisofprogramreports,tools,frameworks,data,assessments,clinicalstandardsandguidelines,and
monitoring/reportinginstruments;meetingsinJakartawithUSAID,EMAS,MinistryofHealthandother
governmentofficials,representativesofprofessionalassociations,multilateralorganizations,academics,
andotherstakeholders;andtraveltofieldsitesin4districtsacross3provinces,includingmeetingswith
localgovernmentandhealthauthorities,directors,clinicians,andclinicalstaffinpuskesmasand(public
andprivate)hospitals,membersoflocalnongovernmentorganizationsandhealthadvocacygroups,
andpatients.Duringthecourseoftheevaluation,theteaminterviewedatotalofover200informants
(Appendix4).

MajorfindingsoftheevaluationintermsofRelationshipswithGovernmentofIndonesiaandGOI
MNHstrategiesandprogramsinclude:
- TheEMASapproachisinlinewithGOIstrategiesandprogramapproaches.

ii
- Atprovincialanddistrictlevel,politicalengagementbyEMASishigh,contributingtoincreased
awarenessofmaternalandnewbornmortalityandtouptake,support,andexpansionofEMAS
approaches.
- AtcentralGOIlevel,bothUSAIDandEMASdonothaveadequateengagementandcommunication
withthepoliticallevel,withtheresultthatEMASsworkandlearningarenotcurrentlyperceivedas
connectedwithnationalstrategiesandprogramapproaches.

IntermsofResultsofEMASimplementation,keyfindingsinclude:
- EMAScontentisnotnewhowever,theapproachthroughwhichthiscontentissupportedby
EMASappearstoencourageuptakeandpracticeimprovement.
- Atprovincial,district,andfacilitylevels,EMASappearstobecontributingtopositivechangesin
quality,organization,andmanagementofMNHservices.
- OneofEMASsmostimportantresultshasbeenturningafragmentednonsystemforreferralintoa
functionalnetwork,andthedevelopmentofrelationships,connection,andcommunicationwithin
thatnetwork.
- Inthepastyear,EMAShasundertakenastrategicapproachtoimprovementofdataavailability,
quality,anduse,andhasinstitutedsomesolidandpotentiallyusefulapproaches.
- Overall,EMAShasgeneratedsomeimportantandinnovativeengagementofprivatesectorpartners
inorganizationandqualityimprovementofMNHservices;however,thisislimited.

InrelationtoAchievingimpactandsustainabilityatscale,evaluationfindingsinclude:
- AmajorissueisthatdataavailablefromEMASandfromhealthserviceswhereitisworkingdonot
allowconnectionoftheobservedandreportedchangesinprocesseswithchangesinhealthworker
orsystemperformance,norwithMNmortality.
- Becauseofthislackofcertaintyaboutperformanceandmortality,oneofthemostimportantthings
todetermineistheactualoperationalandclinicalcausesofcontinuingmaternalandnewborn
deaths.
- ThelimitationsofEMAScapacitytodirectlyengageinexpansionmaylimitachievementof
implementationatscale.
- Overall,EMAShaselementsthatcanpotentiallybeimplementedatscale,butdoesnotactually
haveastrategyforleveragingsuchimplementationatscale.
- Akeyelementofbeinganeffectivethinking,learning,andcommunicatingorganizationandof
contributingtoimpactatscaleistosystematicallygenerateandsharelearningfromEMASs
engagementattheoperationallevel.
- TheMTEprovidesanopportunityforreconsiderationanddiscussionofthequantitativetargetsthat
havebeensetforEMAS,inthelightofbothpoliticalrealityandrealityontheground.
- ThereareimportantchangeshappeninginIndonesiathatcouldoverrideefforts(including,butnot
limitedtoEMASs)toincreaseeffectivenessofMNservices.

IntermsofEMASsownManagement,theevaluationteamfoundthat:
- Someimportantmanagementissues,includingactinguponremainingfindingsand
recommendationsoftherecentManagementAssessment,needtobeactedupon.

Basedonthesefindings,theevaluationteamconcludesthattheapproachesdevelopedand
implementedbyEMAShaveimportantpotentialtoimprovetheperformanceofIndonesiashealth
servicesinmanagingthecomplicationsthatarethemajorcausesofmaternalandnewbornmorbidity
andmortality.Torealizethispotentialatscale,however,EMASneedstoaddresscriticalchallenges.
OneistoidentifytheconnectionbetweentheprocesschangesthatEMAShassucceededingenerating

iii
withimprovementinpatientoutcomes.Equallyimportantisstrengtheningcommunicationand
connectionwiththeGOIespeciallytheMOHatthecentrallevel.EMASalsoneedstoworkwith
USAID,theGOI,andotherstakeholderstodevelopastrategyforimplementationatscalethatgoes
beyonditsdirectengagement,tolinkitsresultswithnationalscaleinitiativesandprograms.Internally,
EMASneedstodealwithmanagementandorganizationissuesidentifiedbytheearlierManagement
Assessmentandtheevaluationitself.Morebroadly,keyareasinwhichEMASisastakeholder(butnot
theleadplayer)includeindepthstudyoftheclinicalandoperationalcausesofhighmortalityin
Indonesia,aswellasexaminationoftheeffectsoftransitiontothenewJKNinsuranceprogramon
effectivemanagementofMNcomplications.

Basedonthesefindingsandconclusions,theevaluationteammakesseveralkeyrecommendations,
including:
ForEMAS
- DrawupontheirProvincialTeamLeadersaspartofregularcommunicationandexperiencesharing
withthecentralMOH.
- Documentandsharetheprocessbywhichthispoliticalengagementisgenerated,andtheresultsof
thisengagement.
- Urgentlyseektoengageinsystematic,regular,closeandongoingstrategictechnicalandpolicylevel
communicationandconsultationonMNpolicyandprogramdirectiontobuildabridgewithsenior
decisionmakersinthecentralMOH.
- Withinthenext23months,completeEMASsLearningAgendathroughacollaborativeprocess,
engagingcentralandoperationallevelpartners.
- Systematicallydocumentthedevelopmentofreferralnetworksandtheimprovedcommunication
withinthemthatoccursasaresult,andbringthisdocumentationtopartnersandstakeholdersas
soonaspossible.
- Continueandincreaseitseffortstoconnectitsdatagenerationandindicatorswithfacility
management,referralsystemstrengthening,andservicedeliveryimprovementwithlocal
governmentandadvocatestoinformdecisionmaking,andwithotherongoingorpotential
approachestoimprovedataavailabilityandusebyfacilitiesandbyhealthauthorities.
- Defineandmonitortheconnectionoftheprogramsinputswithintendedchangesinprocessand
intermediateoutcomes,andofthosewithreductionofMNmortality.
- RelatechangesassociatedwithEMASsdistrictlevelworktothebroaderdistrictcontexti.e.,
numbersofannualbirthsinthedistrict,MNcoverageatfacilitylevel(whetherEMASsupportedor
not),anddistrictwideMNmortality.
- DefineandsharewithUSAIDandpartnerstheapproachbywhichitwillmonitorandevaluatethe
effectivenessofPhase1facilitiesandorganizationsintransferringtheEMASapproachesthey
themselveshavetakenon.
- Engagethecentralandoperationallevelsofprofessionalassociationsasmuchandaseffectivelyas
possibletoenlisttheminthementoringapproach.
- BeawareoftheJKNparametersandensurefacilitiesatalllevelsproviderapidreceptivitytowomen
inlabor(whetherreferredornot),andworkwithprovincialanddistrictlevelstaffofEMASandthe
MOHtosocializetheJKNrequirements,reviewthereferraloptionsforwomen,andensurethat
womenhavethepaperworknecessarytobeadmittedtofacilitiesforthemselvesand/ortheir
newborns.
- CarefullydocumentandbringtothecentralleveltheeffectstheyencounterofJKNimplementation
oncareandreferralreceivedbywomenandnewbornsinthedistrictswhereEMASisworking.

iv
- ImplementtherecommendationsoftherecentManagementAssessment,includinghiringa
seasonedDeputyDirectorwithsubstantialmanagementandprogramexperience,andorganizing
HQstaff,relations,andcommunicationclearlyandeffectively.
- Makeallpossibleeffortstofillstaffvacanciesatprovinceanddistrictteamlevels,sincethese
vacanciesareaffectingaspectsofprogramsupportandthebackupcapacityatthenextlevelis
limited.

ForUSAIDandEMAS
- CommunicateclearlyhowEMASsfocusofworkalignsandconnectswiththebroadercontextof
andsystemrequirementsforMNmortalityreduction,andwithnationalandsubnationalstrategies
forMNH.
- Developandsupportastudytodefinetheprobableclinicaland/oroperationalcausesofsuch
deaths,inEMASareas/facilitiesandmorebroadlyinEMASdistricts.
- AdvocatewiththeGOItomakefundsavailableformentoringactivitiesbyprovincialandvertical
hospitals,tosupporttheirparticipationinmentoring.
- Developaplanforachievingeffectandimpactatscalebyconnectingkeylessonsandcomponents
ofEMASsapproachwithotherforcesandinitiativesthatcanbringtheseintothemainstreamof
MNhealthpolicyandprogramsinIndonesia.
- Usethismidpointevaluationtodiscussamongthemselves,andwithGOIandotherstakeholders,
themostrelevantimpactgoalanddistrict/facilitytargetstomaximizeEMASseffectivecontribution
andlearning.

ForUSAID
- RequestandencourageMOHleadershiptoparticipateinexperiencesharingactivitieswith
provincialanddistrictrepresentativesandEMASstaff.
- Engageinsystematic,regular,closeandongoingtechnicalandpolicylevelcommunicationand
consultationonMNpolicyandprogramdirectionwithseniordecisionmakersinthecentralMOHby
drawingonitsseniorlevelhealthexperts.
- Considerfundingastudyregardingtheimplicationsofthefertilityandfamilyplanningplateauand
itsimpactonmaternalmortality,toinformdiscussionsofhowtoensurehealthyfertilityrates.
- IFANDONLYIFUSAIDdeterminesthatEMAShasdevelopedthecapacityandtakentheactions
requiredtorespondtotherecommendationsofthisevaluation,THENUSAIDshouldconsider
identifyingadditionalfundsandamechanismtoextendEMASsworkbytwoadditionalyears,
withoutwaitinguntilyear4or5;extendingtheprogramsworkwillsubstantiallyincreasethe
probabilityofhavingtheimportantinvestmentUSAIDismakingthroughEMASachievescalableand
sustainableresults.

v
TableofContents

Introduction.1

Thepurposeoftheevaluation...4

EvaluationQuestions.5

BriefdescriptionoftheEMASprogram....6

MajorFindingsandRecommendations

- RelationshipswithGovernmentofIndonesiaandGOIMNHstrategiesandprograms..8
- ResultsofEMASimplementation.12
- Achievingimpactandsustainabilityatscale15
- Management.23

AdditionalFindings.25

SummaryofConclusionsandRecommendations.27

[Areasforconsideration:Followonmaternalnewbornhealthprogramming]..28
[INUSAIDVERSIONONLY]

APPENDICES

vi
Abbreviations

Bappenas BadanPerencanaanPembangunanNasional
BEmONC BasicEmergencyObstetricandNewbornCare
BPJS BadanPenyelenggaraJaminanSosial
BPS BadanPusatStatistik
CEmONC ComprehensiveEmergencyObstetricandNewbornCare
CPAP ContinuousPositiveAirwayPressure(breathingsupport)
DHO DistrictHealthOffice
GoI GovernmentofIndonesia
IBI IkatanBidanIndonesia
IDAI IkatanDokterAnakIndonesia
IDI IkatanDokterIndonesia
Jamkesda JaminanKesehatanDaerah(LocalGovernmentHealthInsurance)
Jamkesmas JaminanKesehatanMasyarakat(CommunityHealthInsurance)
Jampersal JaminanPersalinan(Communityinsuranceforantenatal,childbirth,and
postnatalcare)
JNPK JaringanNasionalPelatihanKlinis(NationalClinicalTrainingNetwork)
JPKM JaminanPemeliharaanKesehatanMasyarakat(PublicHealthCare
Insurance;managedcaremodel)
JKN JaminanKesehatanNasional
JNPK JaringanNasionalPelatihanKlinis
LKBK LembagaKesehatanBudiKemuliaan
MDGs MillenniumDevelopmentGoals
MenKoKesra MinistryofPeoplesWelfare
MN MaternalandNewborn
MoH MinistryofHealth
MOU MemorandumofUnderstanding
MTE MidTermEvaluation
PE/E PreEclampsia/Eclampsia
PerDa PeraturanDaerah(District/municipalityregulation)
PKK PemberdayaandanKesejahteraanKeluarga
POGI PerkumpulanObstetriGinekologiIndonesia
PolindesPondokBersalinDesa(VillageMaternityHut)
PosyanduPosPelayananTerpadu(IntegratedHealthPost)
PUSDATIN PusatDataIndonesia(CenterforData,MoHIndonesia)
PWSKIA PemantauanWilayahSetempatKesehatanIbudanAnak(LocalArea
MonitoringforMaternityandChildHealth)
Puskesmas PusatKesehatanMasyarakat(CommunityHealthCenter)
Risfaskes RisetFasilitasKesehatan
Riskesdas RisetKesehatanDasar
SOP StandardOperatingProcedures
TN2PK TimNasionalPercepatanPenanggulanganKemiskinan
(NationalTeamforAcceleratingPovertyReduction)

vii
IntroductionMaternalNewbornhealthcontextinIndonesia(4,610,000birthsexpectedin2015)and
USAIDsresponse

Indonesia,thefourthmostpopulouscountryintheworld,continuestofacemajorhealthcare
challengessincetheeconomiccollapseof1997thatresultedinasharpincreaseinthepopulationinand
nearpoverty.Inresponse,thegovernmentmovedquicklytoreducesocioeconomicinequityinhealth
careaccessthroughaseriesofhealthinsuranceplans,resultingrecentlyinthe2014launchofuniversal
healthcoverageby2019throughJaminanKesehatanNasional(JKN).In2001,decentralizationand
devolutionofauthoritytodistrictswasinitiatedtoincreaseresponsivenesstolocalconditions.Evenso,
thehealthstatusofthecountrycontinuestolagbehindneighboringcountries,especiallyinmaternal
andnewbornhealth.ThisisespeciallyprominentinIndonesiassmaternalmortalityratio,whichis
substantiallyhigherthanthatofothersouthAsiancountrieshavingsimilarlevelsofeconomic
development.Whileithasmadesubstantialprogressinreducingmortalityratesamongolderinfants
andchildrenunderagefive,Indonesiahasfailedtomakeprogressinreducingtherateofnewborn
mortality,whichhasbeenstagnantforthepastdecade.Contributingtothislackofprogressisoverall
lowgovernmentspendingonhealth.AndalthoughIndonesianwomensstatusimprovedbetween1990
and2010,withgenderparityineducationattheprimary,secondaryandtertiarylevels(WorldBank
2012),betterwomensrights(SatriyoHA.2008),moreparticipationingovernment(BacheletM.2012),
andprogressinwomensparticipationindecisionmakingathouseholdlevel,theimportantexceptionis
womensdecisionmakingfortheirownhealthcare(IDHS2003,2013).

Outcomes
ReducingmaternalmortalityisnowandhasbeenanationalpriorityinIndonesiasincetheSafe
MotherhoodInitiativewaslaunchedgloballyin1987(AbouZahr2003).Whileestimatesoftheabsolute
numbersofmaternaldeathshavedecreasedbynearlytwothirdsbetween1990and2010tolessthan
10,000,reductionofthematernalmortalityratio(MMR)appearsslowandwithvariableprogress
dependingontheestimationmethodused(IDHSetal2013;IDHS2003;NASandAIPI2013).The2013
estimatefortheMMRis190/100,000livebirths,accordingtoWHO(2013).Inequitiesremain:mothers
whodiearetypicallybetween2034yearsold,rural,andpoorlyeducated(NASandAIPA2013).For
neonatalmortality,thepoorestexperiencethreetimesmoredeathsthantherichest(IDHSetal2013;
IDHSetal1991).

Facilitybirthinghastripled,from21%to63%between1991and2012(IDHSetal2013;IDHS1991),with
thepoorestmakingthesmallestgains:30%facilitybirthingversus88%intherichestquintile(2012)
(IDHS2013).Progressinfacilitybirthingneedstobequalified:ofthe46%womenwhowereusing
healthfacilitiesinthemid2000s,onlyoneoffourgavebirthinahospital(IDHS2008).65%offacility
birthswereinprivatemidwiferyclinicsandvillagebirthingpostsoftenthehomeofavillagemidwife
where,forexample,90%lackedasterilizerorresuscitationequipmentand80%lackedmagnesium
sulphate(AIPMNHNTT2008).Afurther6%ofwomengavebirthinhealthcenters,puskesmas.Only
15%ofthepublicpuskesmasarefunctioningasPONEDcenters(havestafftrainedinBEmONC)(World
Bank2014).

Caesareansectionrateshaveincreasedfrom0.8%(198691)to12.3%(20072011)(IDHS2013;IDHS
1992).Althoughwomenwithsevereobstetriccomplicationstypicallyrelyonpublichospitals,most
caesareansectionsareprovidedinprivatefacilities,withalargegapbetweenthepoorandrich:only
3.7%amongthoseinthepoorestquintilehadacaesareanforbirthversus23%amongtherichestin
2012(IDHS2013).
Accesstoservicesandqualityofcare
WhentheGovernmentofIndonesia(GoI)launchedtheirSafeMotherhoodInitiativeinthelate1980s,
themainfocuswasonarapidscaleupofaccesstoprofessionalcare,thecenterpieceofwhichwasthe
Bidandidesa,thevillagemidwifeprogram.By1997,over54,000midwiveshadbeendeployed,andby
2012,thenumberofmidwiveshadrisentoover200,000(NASandAIPA2013).Evensoonly40%ofthe
villagesarereportedcovered,withmanymidwivesmovingtourbanareastoincreasetheirpatientload.

Between1991and2012midwifeassistedbirthsincreasedby53percentagepoints,from31%to84%
(IDHS2013;IDHS1991),butpersistentpoorqualityofcarehasbeenwelldocumented(WorldBank
2010;Ensoretal2008;Rokxetal2010).ConfidentialinquiriesinwesternJavafoundvillagemidwives
emergencydiagnosticskillstobeaccurate,butclinicalmanagementofcomplicationswanting
(DAmbruosoLetal2009).

Reasonsbehindthepoorperformanceofmidwivesarepartlyrelatedtodeficienciesinthebasictraining
consequenttothepaceofscaleup,andpartlytothedeploymentstrategy.Midwiferyacademieshave
proliferatedoverthispastdecadeandwithover750nowexisting,midwiferystudentsdonothave
enoughclientsduringtrainingtobecomeproficient.Whenposted,amidwifemaybeasoleproviderat
villagelevel,workingunderdifferentemploymentmeans(civilservant,shorttermcontractstaff,or
privatepractitioners)withvaryinglevelsofsupervisionandreferralsupport.Thelowvolumeofwork
permidwifecompoundsthelackoftrainingandexperiencewithobstetricemergenciesandreferral
possibilitiesformanymidwives:villagemidwivesmayaverage30birthsorlowerperyear(IBI2014pers
comm).Andwhiletheissueofindividualcapacitiesandpreparationofmidwivesledtoathreeyear
trainingprogrambythemid1990s,andcertificationofgraduatesiscurrentlyindevelopment,
performanceproblemshavebeenexacerbatedbypoorcommunicationbetweenmidwivesandreferral
support.Suboptimalsupportfromreferralsites,includingthelackof24houraccessibility,thelackof
communicationbetweenthelevelsofcare,andtheunintendedconsequencesof
incentives/disincentivesinthesystem,havecontinuedtohinderqualityimprovement.

Thelackofcoordinationbetweenmidwivesandtheirreferralsystemhasbeenknownforsometime
butlittleefforthasbeenmadetoovercometheproblems.Forexample,deploymentofmidwiveswas
poorlycoordinatedwiththeparallelexpansionofthehospitalnetwork(a22%increaseinthenumberof
hospitalsbetween1998and2008,withmostoftheincreaseinlargersizehospitals[Hortetal2011])
andcontinuedexpansionofthepuskesmassincethe1980s.Equipmentandsupplysystemsfor
maternalandnewborncarealsolaggedbehind.In2011,anationalfacilitysurveyshowedthatofthe
nearly9000healthcentersonly45%metthepersonnelrequirementtoprovideBEmONC,12%hadthe
requiredequipment,and28%couldprovide24hourservices(Riskfaskes2011).While83%ofpublic
hospitalshadatleastoneobstetrician(notnecessarilyfulltime),only21%metthenineCEmONC
criteria,includinga24houroperatingroom,blood,laboratoryandradiologyservices,andateam
available24hoursaday.Lessthanhalfcouldprovidecomprehensivematernityservicesduetolackof
qualifiedhumanresources,equipmentandblood.Thereisalsoregionalandgeographicimbalancein
healthcaredeliverygiventhe15,000islandsofIndonesia,thisisnotsurprising,butpresentsproblems
intermsofensuringallhaveaccesstothecareneeded(NASandAIPI2013).TheGoIhasrecently
launchedpoliciesandregulationstoimprovehospitalandhealthcenterservicesincludingappropriate
recruitmentanddistributionofhumanresources,accreditationofhospitalsandpuskesmas,
introductionofqualityimprovementcycles,maternalandperinatalaudits,andincreasedfinancial
supportfromcentralaswellaslocalgovernmenttoaddressthegapsininfrastructure,equipmentand
supplies.

2
Poorqualityofcareatbothmidwiferyandhospitallevelshasinfluencedthewayfamiliesrecognise
problemsandmakedecisionstomovewomentocare.Thishasbeencompoundedbyverysubstantial
transportandinpatientcosts:typicallyUS$111foranormalbirthandUS$423foraCaesareansection
(Pujiyanto.2009).A2005financialsafetynetforhealthhassinceevolvedintonationalanddistrictlevel
insuranceprogramsforthepoorandnearpoor,withtheambitiousgoalofuniversalcoverageby2019
(WorldBank2010).Theseinsuranceprogramshavereducedtheequitygapinaccessingservices,but
notyeteliminatedit.Theyalsocovertransportcosts,butonlypartiallyandnottothefirstlevelofcare,
costsofwhicharebornebyfamilies.

Steeringandgovernancesupport
Giventhesizeandcomplexityofthecountry,withover500districtsandmunicipalities,andtheheavy
relianceonaprivatesectorthatrepresentsachallengeaswellasanasset,effectivegovernanceand
integrationofthehealthcaredeliverysystemhasbeenapersistentproblem.In1999,Indonesia
decentralizedhealthpolicyandprogrammanagementtodistrictlevelwiththeintentionofimproving
accessandqualityofhealthservices.Giventhevariablecapacitytodesignpoliciesandtofundand
manageprogramsacrossthedistrictsandmunicipalities,theresultshavebeenuneven.Persistentlack
ofcoordinationofthedifferentlevelsofgovernmentinstitutions,especiallyatdistrictlevel,hasresulted
inunevenprogressandachievementamongdistricts,amultiplicationofapproachesandorganizational
setups,withlittlecapitalizationonlessonslearned.Midwiferycareatprimaryhealthfacilitiesand
hospitalcareforemergencieshavebeenmanagedandfundedseparately,withresultingcommunication
andaccountabilityproblems(Heywood,andHarahap2009).Theabsenceofintegrationandcontinuity
inthesystemhasseverelyconstrainedtheeffectivenessofthematernalandnewbornprograms.

Nationalregulationssetminimumstandardsfordistrictsfor18healthindicatorsin2008;fiveofthese
healthindicatorsrelatetomaternal,newbornandchildhealth.Absenceofareliablehealthinformation
systemtoenableefficientandeffectivemanagementofhealthandinsuranceprogramsiswell
recognized;however,effectivesolutionshaveremainedattheplanningstage.

TheGoIiscommittedtoidentifyingandaddressingongoingchallengesastheyarise:forexample,the
GoIrecentlyincludedprivatesectorprovidersintheNationalInsuranceProgram,theJKN.Muchhope
hasbeenputintheflexibilitydecentralizationwouldallow.Thelessonlearned,however,isthat
decentralizationdoesnotalwaysleadtoimprovedmaternalandnewbornservices.Theissuesofequity
andqualityofcarealsorequireattention.Decentralizationanddevolutionofauthoritytodistrictsgives
themayorofeachdistricttheauthoritytoselectprogrammaticdirectionforthedistrict.Thepolitical
commitmentshownatnationalMinistrylevelhasnotnecessarilybeentakenupatdistrictlevel.

USAIDsresponsetothecurrentsituationofmaternalandnewbornhealthinIndonesiaisitssupportfor
theExpandingMaternalandNewbornSurvival(EMAS)program.EMASbuildsuponanalysisofprevious
IndonesianprogrammingapproachesandUSAIDsexperienceinsupportingthoseapproaches.Asnoted
above,IndonesiaandUSAIDhavepreviouslyfocusedonincreasingavailabilityofanddemandfor
skilledbirthattendantsthroughtraininganddeploymentoflargenumbersofcommunitymidwives,
andonimprovedmanagementofroutineandemergencydeliveries.During20052010,inthefaceof
decentralizedmanagementofhealthservices,USAIDsupportedtheHealthServicesProgram,which
workedonstrengtheningdecentralizedMNHservicesandengagementoflocalgovernment,aswellas
seekingapproachestoimprovequalityandaccesstoperinatalcare.From20102012,abridge
programimplementedthroughUSAID/WashingtonsMaternalChildHealthIntegratedProgram
(MCHIP)focusedonincreasinguseofevidencebasedlifesavinginterventions,improvingthereferral
system,anddistrictproblemsolving.

3

Asillustratedbelow,fromtheseexperiences,andtheemergingrecognitionthatimprovedmaternaland
newbornsurvivalinIndonesiarequiresimprovedmanagementoftheillnessesandcomplicationsthat
resultinmaternalandnewborndeath,USAIDandtheGovernmentofIndonesiaagreedthattheEMAS
programshouldfocuson:
- Improvingdetectionandmanagementofcomplicationsatthepuskesmas,wherecomplicated
maternalandnewborncasesareexpectedtoenterthehealthsystem;
- Strengtheningtheeffectivenessandtimelinessofreferralofcomplicatedcasestothehospitallevel
wheredefinitivemanagementissupposedtobeavailable;and,
- ImprovingqualityofcareandorganizationofservicesforcomplicatedMNcasesatreferralhospitals
(EmergencyDepartment,MaternityUnit,OperatingRoom,andNeonatalUnit).

Figure1Strategicframeworkformaternalhealthinterventions,
indicatingareas(redcircles)selectedbyUSAIDandGoIasfocusof
theEMASprogram(basedonTheLancet)

EMASbeganinSeptember2011;actualprogramimplementationeffectivelystartedin2012,meaning
thattheprogramhasbeencarryingoutitsprogramapproachesforlessthan2years.Theagreement
hasroughly2.5yearsremaining.Atthispoint,inaccordancewiththetimetableintheprogramdesign,
USAIDdeterminedthatanindepthmidtermevaluationwasappropriate.

I. Thepurposeoftheevaluation
Thismidtermevaluationisintendedto:
AssessEMASprogressinachievingthegoal,objectivesandplannedoutputsasstatedinthe
agreementsprojectdescriptionandinapprovedworkplans;
ProviderecommendationstoimproveEMASprogrameffectivenessovertheremaining2+yearlifeof
project;and,
ProviderecommendationsforUSAIDtoconsiderinthedesignoffutureprojectsaimedatimproving
maternalandneonatalhealthinIndonesia.

4
EvaluationQuestions
1. WhatarethemajorEMASaccomplishmentstodate?IdentifykeystrengthsintheEMASprogram
approach.
2. What evidence is there to validate the overall development hypotheses and programmatic
approach?Acompleteresponsewilladdressataminimum:
a. EffectivenessoftechnicalcontentofEMAS.
b. StrengthsandweaknessesoftheEMASvanguard model,mentoringapproach,engagementof
partnersthroughPOKJAs,andengagementofprovincialhospitals.
c. Effectiveness of ICT and governance interventions, judged by contribution to achieving health
objectives?
d. What success has been achieved in engaging the private sector service providers? What
opportunities,strengthsandweaknessescanbeidentifiedtoguideadditionalactions?
e. Have there been any unanticipated changes in the host country or donor environment that
suggest the need for changes in emphasis in the EMAS project to minimize implementation
problemsorunintendedconsequencesand/ormaximizeimpactintheremainingtimeavailable?
3. To what extent have monitoring information and lessons learned during project implementation
been used to inform project management decisions? A complete response will address at a
minimum:
a. Whethersystemsforprogrammonitoringareprovidingtimelyandrelevantinformationtothe
appropriateindividualswithresponsibilityandauthoritytoact.
b. Adjustmentstoprogramapproachesthathavebeenmadebasedonsuchinformation.
c. Whether such adjustments are likely to improve prospects for program impact, sustainability
andscaleability.
d. Recommend specific new approaches and decision support tools to improve feedback for
informeddecisionmaking.
4. WhataretheprospectsforEMASachievingimpactatscale?Acompleteresponsewilladdressata
minimum:
a. Theextenttowhichtheapproachtoachievingsustainabilityandimpactatscalearearticulated
inprojectdocuments.
b. Whether EMAS approaches and materials are sufficiently inline with existing standards and
systemstobeintegratedintostandardpracticeinsystemsoperatingatscale.
c. The extent to which the EMAS learning agenda addresses main policy and program questions
and evidence requirements to support sustainability and spread of EMAS innovations and
approaches.
d. The effect of partnerships with U.S. hospitals, commodity donation charities, or the private
sector(Laerdal,GE,Chevron)onprogrammaticresultsorprospectsforsustainability.Whatare
the strengths, weaknesses, lessons learned, unintended outcomes, and cost effectiveness of
theseendeavors?
e. Opportunities, strengths, and weaknesses of EMAS engagement of Indonesian partners both
withintheprojectandexternalincludinggovernmentandprivatesectorentitiesatthecentral,
provincialanddistrictlevels,leadershipofpublicandprivatefacilities,professionalassociations,
academics,andcivilsociety.
5. Are all expected results likely to be achieved by the completion of the project and, if not, what
changesintargetedresultsand/orimplementationapproachesshouldUSAID/Indonesiaconsider?
a. Areworkplanmilestonesandresultsbeingachieved?
b. Are EMAS project implementation priorities sufficiently focused for the best application of
limited resources? Are there low yield (or likely low yield) project elements that should be

5
 reduced or eliminated? Are there elements that should receive increased attention and
resources?
c. Istheprojectreachingthedesiredbeneficiaries?Ifnotwhynot?

[NoteQuestion6(financialmanagement)istobeansweredthroughadifferentmechanism,notby
thisEvaluationteam]

II. BriefdescriptionoftheEMASprogram
TheUSAIDsupportedEMASprogramisafiveyearcooperativeagreementwithJhpiegoaimedat
contributingtothereductionofmaternalandnewborn(MN)deathsinIndonesia.Subgranteesinclude:
- theBudiKemuliaanHealthInstitute(formentoringtoimprovequalityandmanagementoffacility
basedMNcare);
- Muhammadiyah(forMNservicedeliveryimprovementinitsownandotherprivatefacilities,andfor
organizationofcivilsocietysupportforMNserviceimprovement);
- SavetheChildren(fortechnicalsupportinimprovingnewborncare);and,
- ResearchTriangleInstituteRTI(forengagementoflocalgovernmentanddevelopmentof
information/communicationtechnologyapproachestohelptheprogramachieveitsgoals).

EMAShasseveraloverarchingobjectives.Theseinclude:
- Contributingto25percentnationalreductionsofmaternalandnewbornmortality;
- ImprovingqualityofemergencyMNcareinatleast150hospitals;and,
- Improvinglifesavingclinicalinterventionsandeffectivenessofreferralsinatleast300puskesmas
(healthcenters).
USAIDrecognizesthatimprovementofcare,eveninthisrelativelyambitiousnumberoffacilities,cannot
bethesoleapproachrequiredtoachievetheatscalemortalityreductionsproposedunderthis
agreement.TheseatscalemortalityreductionobjectivesareUSAIDsprimaryobjectiveforEMAS.

TheEMASapproachisnowconsideredtohavethreemaincomponents,eachwithasetofspecific
activities.Theseare:
1. ImprovingqualityofemergencyMNcareatpuskesmasandreferralhospitallevels,by
- Engagingfacilityleadership
- Modellingandmentoringfromfacilitieswithhighqualityservices
- Carryingoutsharedassessmentoffacilitycapacityandservices
- Establishinguseofperformancestandards
- Introducingprocessesandtoolstosupportimprovedproviderpractice(e.g.,emergencydrills,
organizationofservices,jobaidsandotherdecisionsupporttools,assuranceofstockedand
accessiblematernalandnewbornemergencytrolleys)
- Establishingdeathandnearmissaudits
- Establishinguseofclinicaldashboardforservicemonitoring
- Developingandpromulgatingservicecharters(agreementsbetweenfacilitiesandstakeholderson
servicesprovidedoperationalizingthe2009PublicServicesLaw)
- Improvingfeedbackandcommunicationwithinfacilities
- Promotingrotationsofpuskesmasstaffinreferralhospitals
- Strengtheningfacilitybaseddatarecordingandusefordecisionmaking.
2. ImprovingeffectivenessofMNemergencyreferrals,by
- Engaginglocalgovernmentandhealthauthorities,professionalassociations,hospitals(publicand
private),healthcenters,andotherstakeholdersindevelopingagreementonreferralpathways

6
 (MOUs);alsoinsomecases,developingjointStandardOperatingProcedures(SOPs)todefineroles
andresponsibilitiesinthereferralchain
- Introducingreferralperformancestandards
- Establishingcommunicationchannelstosupportbetterinformationexchangeduringemergency
MNreferrals,includingdevelopingansmsbasedsystemtofacilitatereferrals(SijariEMAS)
- PromotingMaternalPerinatalAudits(MPAs)
- PromotingeffectiveuseofavailableinsuranceprogramsthatsupportMNservices
- Developingcitizenfeedbackmechanisms
3. GeneratingpoliticalandcivilsocietydemandandsupportforimprovedMNservicesandimproved
outcomesofMNcomplications,by
- SupportingformationofmultistakeholderPokjas(oversightcommittees,convenedbyProvincial
andDistrictHealthOfficers)tomonitorandpromoteeffectivenessofMNservices,andestablish
legalandbudgetarysupportforthoseservices
- SupportingformationofgroupsofrelevantcivilsocietyorganizationsinaCivicForumtoincrease
awarenessofMNcomplicationsandappropriatepreparationandcareseeking,channelcommunity
concernstothepoliticallevel,andparticipateindevelopmentofservicecharters.
Useofinformation/communicationtechnologyinsupportoftheseapproachesisacrosscutting
elementofEMASsapproach.

EMASoperatesin6provinces(allofwhichareamongthe9provincesidentifiedbytheMinistryof
HealthashighburdenMNpriorityprovinces).Byendofagreement,EMASintendstohaveoperatedin
30districts(ofwhich28areamongthe64identifiedbytheMinistryofHealthashighburdenMN
prioritydistricts).Toimplementitsapproaches,EMASproposeda3Phasestrategy.InPhase1,EMAS
implementingpartnersplayastrongroleinintroducingthecomponentslistedabove.Thisworkisdone
throughasystematicmultistagementoringapproach,whichdiffersfromconventionaltraininginbeing
asidebysideprocessofassessment,problemidentification,problemsolving,andskillbuilding
approach.Thisapproachisintendedtodevelopconsciousnessoftheneedforandvalueaddedofthe
elementsEMASprograminimprovingMNservices.

Thefacilities,Pokjas,andCivicForumsreceivingthisPhase1supportareintendedtobecome
Vanguardorganizationswhentheyreachahighlevelofcompliancewithindicatorsofimplementation
ofkeycomponentsoftheapproach.TheseVanguardorganizationsandfacilitiesareintendedtobethe
sourceofmentoringsupporttoPhase2organizationsandfacilities.However,organizationwide
improvementhasnotbeenuniform,althoughindividualunitsandindividualshavereachedthelevelof
capabilityrequiredformentoring.Forthisreason,EMAShasbegunmentoringinPhase2byusinga
combinationofitsownimplementingpartnersandselectedmentorsfromPhase1facilities.EMAShas
alsobegundevelopingexpertsfromprovincialandverticalhospitalsasadditionalmentors,consistent
withtheroleofthesehighlevelfacilities.

InPhase1(May2012September2013),EMASprovidedmentoringandassistanceto23hospitals,93
puskesmas,andassociatedstakeholderorganizationsin10districts.InPhase2(throughSeptember
2014),theprogramintendstoreach69additionalhospitalsand116puskesmasin13additionaldistricts
and6cities;itwillalsoprovidelimitedsupporttoanadditional474lowmaternityvolumepuskesmasin
thosesamedistricts/cities.Phase3(throughSeptember2016)proposestoreachadditionalservices
andorganizationsinanadditional7districts.

7
III. MajorFindingsandRecommendations

A. RelationshipswithGovernmentofIndonesiaandGOIMNHstrategiesandprograms

- TheEMASapproachisinlinewithandaimstostrengthenGOIstrategiesandprogram
approachessuchasPONEDandPONEK.
o EMASaddressespart(notall)ofwhatIndonesiahopestodotoreduceMNmortality.
o ItworkswithinandstrengthenseffortsinMOHpriorityprovincesanddistricts.
o Itworkswithandstrengthenstheteaching/provincialhospitalsoftheseareastobecome
mentorsofdistricthospitalsandpuskesmasintheseareas,andensurescoordinatedreferral.

Figures2a&2bWhereEMASfitswithinGoIstrategiesformaternal
newbornhealth

8
 o Withminorexceptions(e.g.,completinginitialsetupofemergencytrolleys)EMASworks
entirelywithintheresourceenvelopebudget,staff,facilities,equipment,drugsand
commoditiesofhealthservicesintheprovincesanddistrictswhereitworks.
o ThetoolsEMAShasdevelopedandpromotestoimproveperformanceandMNhealthservice
organizationarebasedonandhelpimplementnationalstandardsandguidelines.
o Therefore,ifEMASapproachesaredocumentedtoimproveeffectiveimplementationof
nationalandlocalMNpoliciesandservices,scaleupandsustainabilitywouldnotrequire
extraordinaryinputsbythehealthsystemitself.

Recommendation(forUSAIDandEMAS)EMASandUSAIDneedtocommunicateclearlyhow
EMASsfocusofworkalignsandconnectswiththebroadercontextofandsystemrequirements
forMNmortalityreduction,andwithnationalandsubnationalstrategiesforMNH.

- Atprovincialanddistrictlevel,politicalengagementbyEMASishigh,contributingto
increasedawarenessofmaternalandnewbornmortalityandtouptake,support,and
expansionofEMASapproaches.
o EMASsProvincelevelTeamLeadersarehighlyexperienced,networked,credible,and
politicallyeffective;theyaretypicallyhaveheldhighpositionsintheirgeographicareasand
arewellconnected.
o EMASengagementatProvinceandDistrictlevelshasfrequentlyledtoengagementbyand
supportfromBupatis,DHOs,andhospital/puskesmasleadership.
o Insomecases,Bupatishaveissueddecreesmandatingimplementationandevenexpansionof
approachesinitiatedbyEMAS(e.g.,referralMOUs);insomecases,thispoliticalsupporthas
beenmatchedbybudgetsupportforimprovementofequipmentandfacilitiesandevenfor
programexpansion(e.g.,JombangDistrict)orthepromiseofbudgetsupportinthecoming
financialcycle(e.g.,PinrangDistrict).
RecentdatafromEMASindicateatotalof31.5billionrupiahofadditionalfundsmobilized
fromProvincial,District,andSubDistrictsourcesforlocalreplicationandexpansionof
EMASswork.
o ThisgovernancedimensionisacriticalcomponentofEMASsworkinrelationtoachieving
bothscaleandsustainability;itisseparatefromwhatEMAScallsclinicalgovernance(which
isreallyfacilitymanagement).
o Inadditiontodirectliaisonwithlocalgovernmenttoexertinfluence,thereisroomfor
strengtheningadvocacy(throughPokjas),moreeffectiveuseofdata(forbothadvocacyand
planning),andidentificationandgroomingoflocalchampionstosolidifylocalsupport.
o Theseniorityandrelationshipswithlocalgovernmentleadersofprovincialanddistrictteam
leadersareimportantcomplementstotheoperationalsupporttogovernanceactivities(such
asMOU,SOP,andServiceCharterdevelopment)beingcarriedoutbyEMASsprovincial
governancespecialists.

Recommendations(forEMAS)EMASshoulddrawupontheirProvincialTeamLeadersaspart
ofregularcommunicationandexperiencesharingwiththecentralMoHforexample,having
theseTeamLeaderscometoJakartaeveryquartertomeetwithMoHstakeholdersandthe
centralPokja.

EMASshoulddocumentandsharetheprocessbywhichthispoliticalengagementis
generated,andtheresultsofthisengagementincludingwhereattemptstodeveloppolitical

9
 supportdidnotwork,whythisappearstohavehappened,andwhetherandhowtheyovercame
thosedifficulties.

EMASshouldalsoidentifyadditionalapproachesandeffortsrequiredtobroadenandsustain
politicalcommitmenttoMNsurvivalandhealthinthefaceofchangesovertimeinindividual
electedofficialsandlocalchampions.

Recommendation(forUSAIDandEMAS)USAIDandEMASshouldreviewtheprogressand
effectsofworkintheareaofgovernance(thatis,engaginglocalgovernment,versusclinical
governanceoffacilities)includingimplementationofoperationalcomponentssuchas
developmentofreferralMOUsorServiceCharters,aswellastheeffectofprovincialanddistrict
teamleaderspersonaleffortstoidentifyessentialelementsrequiredforeffectingchangeat
scale.

Recommendation(forUSAID)USAIDshouldrequestandencourageMoHleadershipto
participateinexperiencesharingactivitieswithprovincialanddistrictrepresentativesandEMAS
staff,sincetheseactivitiesrepresenttherealitythatMoHistryingtosupport,butMoH
participationinsuchinteractioninthepasthasbeenlimited.

AtcentralGoIlevel,bothUSAIDandEMASdonothaveadequateengagementand
communicationwiththepoliticallevel,withtheresultthatEMASsworkandlearningarenot
currentlyperceivedasconnectedwithnationalstrategiesandprogramapproachesandmay
notbeacceptedandsupportedforbroaderapplication.
o SinceMNmortalityreductionisamongUSAIDsandtheGoIshighesthealthpriorities,and
sinceEMASisacooperativeagreementthatis,apartnershipbetweenJhpiego(anditssub
grantees)andUSAID/IndonesiabothUSAIDandEMAShaveimportant,butdifferentand
complementary,rolestoplayinengagingtheGoIatseniorlevels.
o USAIDhadsubstantiveinteractionandagreementwithseniorGoI(MoH)counterpartsduring
programdesign,agreeingontheprogramfocus,design,targets,andawardeeselection,as
wellasduringthefirstyearofimplementation.
o However,duringmorerecentEMASimplementationtherehasbeenturnoverofsenior
leadershipinmostofthecomponentsofthecentralMoHwithwhichEMASneedstowork.
o Atthesametime,USAIDsapproachtoseniorlevelcommunicationwiththeMoHalso
changedafteryearone,sothatitnowappearsthatUSAIDhasnotbeenaseffectively
engagedattheseniorpolicylevel.
o TheexistingUSAIDMoHliaisonmechanism(fundedthroughEMAS)appearstobeusefulfor
intermediatelevelcommunication,butdoesnotprovidethetechnicalandpoliticalseniority
requiredtoestablishandmaintainimprovedMOHsupport.
o Similarly,whileEMASitselfhasongoingtechnicalinteractionwithMoHcounterpartsin
severalareas,itdoesnothaveadequaterepresentationnoreffectiverelationshipsatthe
policyleadershiplevel.
o Insomecases,decisionsormiscalculationsbyEMASpartnershavecontributedto
misunderstandingwiththecentralMoH.
TheselectionofandheavyrelianceonLKBKasleadmentoringpartner,ratherthanGOI
facilitiesorotherexpertcapacities,hasledtosomeresentments.
ThespecificquestionwasraisedofwhytheformerlyUSAIDsupportedNationalClinical
TrainingNetwork(JaringanNasionalPelatihanKlinik,JNPK),inwhichthenationaland

10
 localchaptersoftheIndonesianObstetricsandGynecologyAssociation(POGI)playa
leadingrole,wasnotengaged.
Attheoperational(district)level,thevisionaryandtransformationalapproachtakenby
LKBKhasproventobeakeyelementintheacceptanceanduptakeofEMASsinputs.
However,atthecentrallevel,thisapproachanditsrepresentationhavesometimes
generatedmisunderstandingswithsomeEchelon2officialsoftheMoH,withresulting
difficultiesinrelationshipsandsupport.
o TheresultofthisapparentpoliticalisolationofUSAIDsprogramandEMASsassistancefrom
MoHseniorleadershiphasinsomecasesledtotheperceptionofEMASasbeingparalleltoor
competitivewithGOIstrategies,andinsomecasestomisunderstandingofUSAIDsassistance
(e.g.,anapparentmistakenperceptionbysomeIndonesiangovernmentofficialsthatthe$55
millionbudgetforUSAIDsupportoftechnicalassistancethroughEMASmightrepresentan
increaseintheMoHsownMNbudgetandmightthereforejustifyareductionintheGoIs
ownMNfunding).
o Thereispositivesupporttobuildon,includingtheexistenceofthecentralPokja(established
bydecreeoftheMoHandchairedbyheadofChildHealth),therecentMinisterialDecree
establishingcontinuedMoHleadershipforthatPokja,andtheveryrecentdesignationthe
DirectorGeneralofNutrition&MaternalChildHealthasChairofthecentralPokjabythe
SecretaryGeneralofHealth.
o However,seniorMoHmanagersatDGlevelwhohavepositiveperceptionsofEMASs
approachesstillreportnotseeingEMASasbeingclearlyconnectedtotheMoHsown
strategicapproaches;theywouldliketoseeitbecloselylinkedtothegovernmentsongoing
policyandprogramdevelopment.
TheMoHMaternalHealthdepartmenthasstatedthatthereneedstobebetterconnection
andcommunicationofEMASworkwiththegovernmentsownMNHstrategy.
o IntheabsenceofproactiveestablishmentofbetterconnectionandrelationsatMoH
leadershiplevel,negativefeelingsandconcernsaboutEMASappeartobespreading.
o WhilecentralMoHpoliticalsupportbyitselfcannotassuretheachievementofscaleand
sustainability,lackofthatsupportwillcertainlyimpedethatachievement.
o EMASalsobelievesthatconditionsrequiredforachievingscaleandsustainabilitywillrequire
engagementofadditionalelementsofcentralgovernment,includingtheMinistriesofHome
Affairs(decentralization)andWomensEmpowermentaswellasJNPK(NationalTraining
Network)andBPJS(managersoftheJKNinsuranceprogram),andothersoutsidegovernment
(e.g.,theprofessionalassociationsIBI,POGI,IDAI,IDI).

Recommendations(forUSAID)USAIDshouldcloselyexaminethehistoryandstatusoftheir
ownpolicylevelrelationswiththecentralMoH,toidentifywhererelationsmayhavegoneoff
trackandwhatstepsneedtobetakentorevitalizethoserelationships.

- Basedonthatanalysis,USAIDshouldengageinsystematic,regular,closeandongoing
technicalandpolicylevelcommunicationandconsultationonMNpolicyandprogramdirection
withseniordecisionmakersinthecentralMOHbydrawingonitsseniorlevelhealthexperts,
specificallyDr.BatemanandMs.Koek(andhersuccessor);thisseniorlevelofcommunication
cannotbeachievedwithlessseniorUSAIDstaff.

Recommendation(forEMAS)EMASmusturgentlyseektoengageinsystematic,regular,close
andongoingstrategictechnicalandpolicylevelcommunicationandconsultationonMNpolicy

11
 andprogramdirectiontobuildabridgewithseniordecisionmakersinthecentralMoHandto
repairrelationshipswheremisunderstandingsexistandhavenotbeendealtwitheffectively.
- Todothis,EMASwillneedtoaddtoitsstaffahighlyexperienced,wellregardedand
politicallysavvyandconnected,diplomaticseniorpolicyadvisor;theEvaluationTeam
stronglyrecommendsthatthisbeanewseniorstaffposition,sincewedonotseethis
capabilityamongexistingEMASJakartastaff(theroleandprofileofsuchanadvisorat
centrallevelwouldbesimilartothoseofEMASsProvincialTeamLeaders).

NOTE:TheEvaluationTeamidentifiesthisseniorpolicyadvisorpositionasdistinctand
separatefromthecompetent,wellseasonedDeputyDirectorfromoutsidetheEMAS
structuretoleadoperationsidentifiedinRecommendationsoftherecentEMASProgram
ManagementAssessment.AsnotedinourFindingsandRecommendationsonprogram
management(below),theEvaluationTeambelievesthatthisrecommendedDeputyPosition
postisalsoessentialtoEMSAfunctioning,andshouldalsohaveindepthunderstandingof
theprogramandpolicyenvironmentinwhichEMASisoperatingatcentralandoperational
level,aswellastheabilityandcredibilityneededtoestablishandmaintainexcellentworking
relationswithGoIcounterparts.

B. ResultsofEMASimplementation

- EMAScontentisnotnewhowever,thependampinganapproachthroughwhichthis
contentissupportedbyEMASappearstoencourageuptakeandpracticeimprovement.
o Mentoring&Assisting,notTraining.
o Healthworker(includingprofessionals)perceivedselfimprovement:
Doingourjobsbetter;
Notbeingjudgedortalkedat;
Ifwearegoingtomentorothers,weneedtobeasgoodaspossibleourselves;
Practice,drills,selfcriticismforimprovement.
o Referralcoordination.
MOUdevelopmentandendorsement,andcollaborationinMOUdevelopmentamong
careprovidersatthedifferentlevelsofcare.
Teamdevelopment:Emergencyteamsatbothlevelsthatcontinuetopracticedrillsas
partoftheirjob.
Referralstandardsthatprovideinformationonhowtodiagnoseandstabilizepatients
priortoreferral,andhowtorespondattherecipientend.

Recommendation(forEMAS)AsthementoringresponsibilityisspreadamongPhase1
facilitiesandorganizations,andamongothertrainers(e.g.,verticalandprovincialhospitalstaff,
professionalassociations)EMASmustensurethatthispendampingandimensionis
understoodandappliedeffectivelybythoseadditionalmentors,sinceitappearstobeakeyto
effectivechangeofpractice.

- Atprovincial,district,andfacilitylevels,EMASappearstobecontributingtopositivechanges
inquality,organization,andmanagementofMNHservices.
Observedchangesinclude:
HospitalsandpuskesmasmakingrenovationsandchangesinfacilitiesformanagingMN
emergencies(e.g.,establishingmaternalandnewbornemergencysubareasinemergency

12
 rooms,relocatingmaternityornewborncareunitstobemoreaccessible,increasing
privacy);
Increasingandupdatingkeyequipment(CPAP,incubators);
Assuringavailabilityandorganizationofemergencydrugsandequipmentthrough
emergencytrolleys;
Carryingoutregularemergencydrillstoestablishandmaintaineffectiveemergencycare);
Organizationofmaternalandnewbornemergencyteamswithdefinedrolesforspecific
teammembers;
Postingemergencyrecognitionandmanagementguidelinesinrelevantunits;
Usingdashboards(thoughnotuniformly)totrackmanagementindicatorsinMNunits;
Stabilizationofreferralpatientsandtimelyreferralsandresponse.

- OneofEMASsmostimportantresultshasbeenturningafragmentednonsystemforreferral
intoafunctionalnetwork,andthedevelopmentofrelationships,connection,and
communicationwithinthatnetwork.
o EnsuringregulationsfromtheBupatiorlocalparliamentforthecaresystem/approachisan
importantfirststepinestablishingthecommitmentandcoordinationoftheprovidersat
differentlevels.
o ThereferralMOUdevelopmentprocessnotonlyspecifiesreferralpathways,butalsobuilds
relationships:hospitalbidantopuskemasbidan,bidanstospecialists,etc.
o SOPdevelopment,whencombinedwithanMOUestablishesasysteminplaceofwhatisnow
fragmentation.
o TheSijariEMASsmsbasedsystemforbidan,puskesmas,andhospitalreferralcoordination
hassubstantialacceptanceandappearstobefeasibleformostdistrictsandfacilities.
SijariEMASalsohassubstantialappealitmayactuallyturnouttobeanimportantand
effectivedriverofattentiontoimprovedreferral.
SijariEMAScanalsobecomeanimportantsourceofdataandanalysisregardingreferral
processes(andforaccountability).
o Bypromotingteamapproaches,EMASsassistancealsofostersandsupportsleadership
withinfacilitystaff.
o Thenetworking/teamapproachhasengagedandestablishedsomedegreeofcommunication
betweenpublicandprivatefacilitiesinaninnovativemanner.
o ThisisprobablyoneofthemostimportantproductsofEMASsworktoimprovemanagement
ofMNcomplicationshowever,EMASitselfdoesnotyetseemtorecognizeitassuch,andat
centralandfieldleveldoesnottalkaboutthissuccessfulformingofanetworkanymorethan
ittalksaboutother(probablylesssignificant)piecesofwhatitdoes.

Recommendations(forEMAS)EMASshouldsystematicallydocument(bothnarrativelyand
withrelevantindicators)thedevelopmentofthisreferralnetworkandtheimproved
communicationwithinitthatoccursasaresult,andbringthisdocumentationtopartnersand
stakeholdersassoonaspossiblethisfunctionalnetworkresultmayturnouttobeoneofthe
programsmostimportantcontributionstoimprovedmanagementofMNcomplicationsin
nationalandlocalhealthsystems.

SijariEMASappearstobethemosteffectiveapplicationofEMASsmandatetouse
communicationtechnologytoimproveMNservices.EMASneedstoconsiderthemarketing
aswellastechnicalvalueoftheSijariEMASitstechnologicalandphysicalaspectsappearto
capturetheattentionoffacilitystaffandmanagers,aswellaspolicyleveldecisionmakers,

13
 whilepresentationofthedetailsofimprovedreferralnetworksmightbelesseffectivein
engagingthoseindividuals.Thisattractivenesscanbebuiltuponasanentrypointtodevelop
understandingofandcommitmenttoimprovedreferralnetworksasawholenotjustthe
technology.However,touseSijariEMASeffectivelyasthisentrypointinsellingthelarger
processrequiredforimprovedreferral,EMASmayrequireexpertiseinmarketingtotakefull
advantageofthisattractivetechnologyanduseittobringreferralimprovementtoscale.

- Inthepastyear,EMAShasundertakenastrategicapproachtoimprovementofdata
availability,quality,anduse,andhasinstitutedsomesolidandpotentiallyusefulapproaches.
o EMAShasdevelopedsystematicdatacollectionapproachesforbothpuskesmasandreferral
facilities,beginningwithstandardizedregistersatpuskesmasandhospitalleveltogather
requiredinformationonmaternitycases.
Theseregistershavebeengenerallywellaccepted;theyappeartobeseenasabetterand
moreusefulwaytocollectpatientrelatedinformation.
Theseregistersallowpatientdatatobecompiledforbothfacilitymanagementandfor
reportingtodistrictlevel(andabove).
However,sofarthesedataaremostlybeingusedinternallybyEMASitselfdataarenot
yetbeingaggregatedbyservices,andarenotbeinganalyzedorusedtomanageormodify
services,toidentifyandrespondtoclinicalorsystemproblemareas,orinadvocacy.
o EMAShasbegunworkingwithpartnerfacilitiestostrengthencapacityfordatageneration
andanalysis,inwaysthatcanimprovebothfacilitymanagementandservicedelivery.
o ThedatageneratedthroughtheseapproachescanbelinkedwithDHOandotherdistrictdata
collectionandmanagementprocesses,andpotentiallywithPUSDATINnationaldata
collection.
o Dataondistrictwidematernityandnewborncasemanagementandmortalitynotjustin
publicfacilitiesisalsoneeded.

Recommendation(forEMAS)WhileacknowledgingEMASscontentionthatitisnotdesigned
tobeanHMISdevelopmentprogram,giventhepervasivelackofdataandtheuncertainty
abouteffectiveremediesthatresult,EMASshouldcontinueandincreaseitsefforttoconnectits
datagenerationandindicatorswithfacilitymanagementandservicedeliveryimprovement,
withlocalgovernmentandadvocatestoinformdecisionmaking,andwithanyotherongoingor
potentialapproachestoimproveddataavailabilityandusebyfacilitiesandlocal,provincial,and
nationalhealthauthorities.Theeffortsonbuildingtheawarenessandcapacityofimproved
dataavailabilityanduseaimedathospitalsandpuskesmasaregoodandshouldbecontinued.

- Overall,EMAShasgeneratedsomeimportantandinnovativeengagementofprivatesector
partnersinorganizationandqualityimprovementofMNHservices;however,thisislimited.
o Thereissomeinvolvementoftrueprivatesector(forprofit)hospitalsinreferralnetworks
thisisinnovativeandpromising,thoughnotyetamajorcomponent.
o Motivationofthesehospitalsisvariablesomeareorientedtowardincreasingpatientand
revenuenumbers,assumingthatqualityofcareisnottheirissue;otherswelcomequality
improvementandservicemanagementassistance.
o Muhammadiyahengagementinclinicalservicesimprovementhasfocusedlargelyon
Muhammadiyahfacilities,whichinitselfisasubstantialsystem.
o BroaderengagementbyMuhammadiyahwithotherfaithbasednetworks(InterfaithAlliance,
NU)isbeginningandispromising.

14
 o Muhammadiyahisalsoengagedincivilsociety(CivicForum)organization(seenotebelow
onthis).
o Privatebidans(thosewhoarenotalsoworkinginthepublicsector)appeartobeoutside
EMASreach;insomedistricts,theyprovidesubstantialcoverageofmaternal/newborncare
howcantheircapacitiesandpatientcoveragebecaptured?

- ThecorecomponentsofEMASsworkrepresentasystematicapproachtoengagingboth
healthservicesandstakeholderstosupportstrengthenedmanagementofMNcomplications;
however,theremaybetoomanypiecestothecurrentapproachtoallowfocusonsuccessof
themostimportantparts(andotherkeypiecesthatmayneedtobedeveloped).
o Thebasiccomponentshospitals,puskemas,referral,politicalsupport(Pokja),civilsociety
involvement(CivicForum)areallvalid,andtheymayberequiredforexpansionand
institutionalization.
o However,theyarenotallequallystrong
Pokjasarevariableintheirorientationandeffectiveness,andinsomecasesarefocusing
moreongettingsupportfortheirownfunctionsthanonprogressinMNoutcomes.
CivicForumsappeartoincludemanyenthusiasticoftenyoungpeople,butitappeared
thattheymaynothaveasolidsenseofhowtheyandtheirorganizationscanmeaningfully
supportimprovedMNservices.InthelimitedinteractiontheMTEwasabletohave,many
CivicForummembersseemedunclearabouttheirrolesasindividualsversusas
organizationalrepresentatives(intwoinstances,MTEmembersofferedsuggestionsabout
howtostrengthentheengagementoftheindividualmembersownorganizations).
CivicForummembersknowledgeofthesubjectgenerallyappearslow,andsomevoiceda
requestformorehelp.However,somemembersarewellconnectedandinfluential(e.g.,
PKKinPinrang).
o SomepiecesofEMASsworkforexample,someoftheclientfeedbackmechanismslike
SIGAPKUandcitizenreportcardsmaynotyieldsubstantialpayoffandmaydilutethe
limitedcapacityofEMAS,especiallyforimplementingatscale.
o Ontheotherhand,governanceatdistrictlevelmaynotbepursuedenoughe.g.datafrom
theEMASdatasystemcanbesharedandbecomeameansforthedistrictstatisticalofficesto
bestrengthenedtoprovideneededdataonMNcoverageanddeath,aswellasservingas
inputformoreeffectiveadvocacyandplanning.
o WhilethefieldofficesareawareofJKNanditspotentialimpactoncoverageandreferral
pathways,thereislittletonoobviousefforttoinfluenceJKNimplementationandeffects.

Recommendations(toEMAS)EMASshouldapplytheTheoryofChangecausalpathway
analysistocriticallyexaminethemanymovingpartsoftheapproachthatithasdeveloped,to
identifythosethatarecriticaltosupportimplementationandsustainabilityatscaleandfocusits
energiesandresourcesonmakingthosework.Strongconsiderationshouldbegivento
droppingothercomponentsthatarelesscriticalorlesseffective.

C. Achievingimpactandsustainabilityatscale

- AmajorissueisthatdataavailablefromEMASandfromhealthserviceswhereitisworking
donotallowconnectionoftheobservedandreportedchangesinprocesses(e.g.,adoptionof
standardsandprocedures)withchangesinhealthworkerorsystemperformance,norwith
MNmortality.

15
 o ThemajorityofdatareportedregardingfacilitiesworkingwithEMASareprocessdata
achievementofstandards,percentageofdeaths/nearmissreviewed.
o ManyoftheseprocesseshaveshownsubstantialimprovementduringEMASassistance.
o WhileintendedtobestandardizedandbasedontheregistersrecentlyintroducedbyEMAS,
thecompletenessofdataonoutcomes(e.g.,percentofPE/EcasestreatedwithMgSO4)and
impact(MNdeathsandcasefatalityrates)remainsuncertain.
o Virtuallyalldatawithinthesystemareimperfectanddonotallowinferenceofsolid
conclusionsaboutpatientmanagementormortality.
o Correlationofprocessimprovements(e.g.,achievementofstandards)withtheavailable
dataonuseofkeyinterventionsisvariableandinconclusive.

16


Figures3a,b,c,dEMASresults:(a)Processindicators;(b,c)correlationofprocesschanges
withinterventiondelivery;(d)mortality/casefatalityratesinEMASfacilities(Baselineand
Year2only,withdifferentdatacollectionmethods)

o ReportedmortalityratesandcasefatalityratesinEMASfacilitieshavenotcomedown(but
interpretationislimitedbydifferencesindataavailabilityandcollectionmethodsatbaseline
andsubsequently).
o Becauseoftheincompletenessofdistrictwidebirthandmortalitydata,evenwhereEMAS
isharvestingthosedistrictdataitsimpactwithinthedistrictswhereitisworkingalso
remainsunknown.
o Thereisreasontobelievethatthechangesinfacilityandreferralperformanceonwhich
EMASisfocusedarelikelytobenecessarybutnotsufficienttoaddressMNmortalityin

17
 thecomplexIndonesiancontext(e.g.,isthereatimingissuewomenarrivetoolate?
Responseistoosloworincomplete?Referralsaredelayed?Orissomethingmoreneededre
treatmentassuggestedinSouzaetal,2013).
o Thusfar,EMASappearstohavefocusedmoreonimplementationofthevarious
componentsofitsprocessimprovementapproaches,andhavenotfocusedonthis
disconnectofprocesseswithimprovementsinoutcomeandimpactasapriorityissue.
o TheTheoryofChangepathwaysconnectingprocesseswithperformanceandoutcomes,
andcarefulmonitoringofindicatorsalongthosepathways,mayhelpunderstandthe
potentialconnectionofEMASsapproachwithMNoutcomes.

Recommendation(forEMAS)EMASshouldfirstapplytheTheoryofChangecausalpathway
analysisapproachtodefinetheintendedconnectionoftheprogramsinputswithchangesin
processandintermediateoutcomes,andofthosewithreductionofMNmortality.EMASthen
needstosharethisthinkingwiththeirfieldstaffandMoHstaffatalllevels,carryoutjoint
analysisofitsindividualinterventionpathways,andinstituteameaningfulwaytomonitorthe
stepsalongthekeypathways.

Ifadisconnectintheassumptionsofthepathwaysisdetectede.g.,actualhealthworker
performanceisdocumentedtoimprove,butMNoutcomesexpectedtobeaffectedbythatbetter
performancedonotimproveEMASshouldworkwithpartnerstoseekunderstandingofthe
forcesthatarenegatingtheprogramslogic.

EMASshouldreflectthechangesassociatedwiththeirdistrictlevelworkwithinthebroader
districtcontexti.e.,numbersofannualbirthsinthedistrict,coverageatfacilitylevel(bytype
offacility,andbywhetherEMASsupportedornot).Theyshouldalsorelatethechanges
associatedwithEMASsworkwithcasemanagementoutcomesandmortality,againatdistrict
level.Whileeventhisexercisewillnotbeperfectduetoquestionabledatavalidityandinand
outmigrationofwomenatthetimeofbirth,itmayhelptodeterminethebroaderbasisfor
EMASseffortstoachieveimpactatscaleandforotherneededinterventions.

- Therefore,oneofthemostimportantandurgentthingstodetermineistheactual
operationalandclinicalcausesofcontinuingmaternalandnewborndeaths.
o Theincompletenatureofbothfacilityandpopulationbaseddatacontributetoourinabilityto
identifythekeyfailuresthatneedtobeaddressed.
o However,evenwheredataexist,theyarenotenoughtodistinguishoperationalorclinical
causesofdeath.
Forexample,dataonrecentmaternaldeathsinEMASfacilitiesinSulselindicatethat
almostallthosematernaldeathsinPinrangDistrictoccurredinthedistricthospital,the
majorcausewaspreeclampsia/eclampsia,thewomenwereneitherprimagravidasnor
uneducated,andtheydiedwithinfortyeighthoursofreachingfacilitieshowever,these
datadonotallowustodeterminewhetherthesewerefailuresofearlyentryintothe
system,failuresofreferral,failuresofhospitalemergencycare,ormanifestationsof
especiallyseverePE/E.
Similarly,consolidatedJanuaryMarch2014dataindicatelow(reported)ratesoffresh
stillbirths,alongwithlownumbersofnewborndeathsonday1,but3/4thsofdeathsin
thefirstweekoflife(40percentofwhichareinbabiesweighingover2,000grams).
WehaveincompleteinformationonbirthsanddeathsoutsideEMASassistedfacilities.

18
 o Thesedatafindingsindicatepotentiallyimportantissuesregardingclinicalandoperational
causesofdeathbuttheydonotclearlydefinethoseissues.Weneedtounderstandof
what,andwhy,mothersandbabiesaredying.
o Clinicalrecordkeepingisgenerallyoflowqualityanddoesnotcontaintheinformation
neededtoanswertheseclinicalandoperationalquestions.
o Deathauditsaregenerallynotperformedandwhendonearereportedlynotinsufficient
depthtoanswerthesequestionseither;weneedtolookindetailatthepotentialsystemand
clinicalfailuresmothersandbabieswhodidNOTsurvivetounderstandthethingsthat
needtobedonetoincreasesurvival.
Nearmissauditscananswersomeimportantquestionsespeciallytodeterminethe
timingofdecisionmakingandmovement,plusresponsewheninfacilities;however,by
definitiontheyarestudiesofsuccesses(orgoodfortune),i.e.,motherswhosurvived.

Recommendation(forUSAIDandEMAS)USAIDshouldseekpartners(includingtheGoI),
resourcesandamechanismtosupportastudytopermitsystematic,expertexaminationofa
largeandrepresentativesampleofmaternalandnewborndeaths,withtheobjectiveofdefining
theprobableclinicaland/oroperationalcausesofsuchdeaths,bothinEMASareas/facilitiesand
throughoutEMASdistricts.USAIDshouldconsiderdrawinguponexistingexperiencedRMNH
researchorganizationsinIndonesiatodevelopandimplementsuchastudy.

- Whiledevelopmentandexecutionofsuchastudyisnotexpectedtobewithintheexisting
capabilityofEMAS,EMASmustbeapartytothisprocess.EMASsroleshouldinclude
promotingandgeneratinglocalsupportforthisinvestigation,aswellashelpingtobestpossible
identifylocaldataandinformants.EMASshouldalsoseektodevelopapproachesthatincrease
theacceptabilityandactualimplementationofmaternalandnewborndeathauditsinfacilities,
astheyhavebeguntodoformaternalperinataldeathauditsbyusingstandardized
approachesandavoidingthetendencytoassignblame.Successindoingthis,ifitcanbeshared
morewidelyacrossthehealthsystem,wouldgoalongwaytowardclarifyingtheclinicaland
operationalfailuresthatneedtobeovercometoimproveMNsurvival.

- ThelimitationsofEMAScapacitytodirectlyengageinexpansionmaylimitachievementof
effectiveimplementationatscale.
o Phase2mentoringbyPhase1facilitieshasnothadsufficienttimeforitseffectivenesstobe
evaluated(thisisthesecondhypothesisoftheEMASdesign).
o Theinvolvementofprovincialandverticalhospitalsandotherresources(e.g.,JNPK)isan
importantstepinrespondingtotheGOIsconcernaboutEMASslimitedengagementofthe
governmentsownmandatedresourcesinmentoringandimprovementofMNservices.
EngagementoftheseadditionalresourcesmayhelpovercomethelimitationofEMASs
owncapacity.
Engagementoftheseadditionalresources(e.g.,verticalhospitals)mayalsolessenthe
perceivednegativeattitudeoftheMOHtoEMASanditshouldbeusedtodoso.
ThereareGOIfundspotentiallyavailablefromtheregularbudgetsofthesefacilities,their
ownrevenuestreams,andsourcessuchastheprovincialtobaccotaxthatcouldsupport
thisrolebyprovincialandverticalhospitals.
However,thesefundsneedtobemobilized.
o Therearelimitationsinthenumberofspecialistsavailableatdistrictandprovinciallevelsto
providementoring;forexample,inPinrangDistrictandSulSelitisestimatedthatitmaytake

19
 asmanyasthreemoreyearstogetasufficientnumberofspecialistsformentoring
(accordingtoanObGynattheprovincialhospital[RSWS]).
o Furtherengagementofprofessionalsocieties(POGI,IDAI,IBI)mayalsoexpandthepoolof
effectivementors;thisishappeningtosomedegreeattheoperationallevel.
o However,thereisalwaysthedangerthatEMASSapproachbasedonpendampingan,
empowerment,networkandrelationshipbuilding,motivationmaygetreducedtojust
trainingasthecircleofmentorsexpandsappropriateselectionandorientationof
mentorsforPhase2andbeyondmaymitigatethisdanger;however,thisprocessneedstobe
monitoredcarefullyintermsofqualityandeffectivenessasitproceeds.

Recommendations(forEMAS)EMASshoulddefineandsharewithUSAIDandpartnersthe
approachbywhichitwillmonitorandevaluatethissecondhypothesisoftheEMASdesign
thatis,theeffectivenessofPhase1facilitiesandorganizationsintransferringtheEMAS
approachestheythemselveshavetakenon.

- EMASshouldengagethecentralandoperationallevelsofprofessionalassociationsasmuch
andaseffectivelyaspossibletoenlisttheminthismentoringapproach.

Recommendation(forEMASandUSAID)EMASandUSAIDshouldadvocatewiththeGOIto
makefundsavailableformentoringactivitiesbyprovincialandverticalhospitals,tosupport
theirparticipationinmentoring.

- Overall,EMAShaselementsthatcanpotentiallybeimplementedatscale,butdoesnot
actuallyhaveastrategyforleveragingsuchimplementationatscale.
o Summingallthecommentsabove,itisclearthatEMAShas:
HadsomesuccessinaffectingpotentiallyimportantserviceswithintheIndonesianhealth
system,workingwithintheexistingresourceenvelope(staff,budget,equipment,supplies);
Succeededinhelpingservicesfillsomegapsbymobilizinglocalresources(ratherthan
providingthoseresourcesthemselves);
Shownthattheengagementoflocalpoliticalandhealthsectorleadershipcanbe
mobilized,andthatthisisacrucialdeterminantofsupportforimprovedMNservices;
Createdanapproachthatgoesbeyondclinicalservicestoincludetechnicalpolitical(Pokja)
andcivilsocietyelements,whichifsuccessfularelikelytobeimportantcontributorsto
sustainingandextendinganyimprovementsinMNservicesandawareness;
Insomesettingshasraisedimportantresourcesanddevelopedinfluentialchampions;
Atoperationallevels(ProvinceandDistrict)hasinvolvedcriticalcomponentsofthehealth
system,includingseveralprovincialandverticalhospitals.
o Despitehavingtheseelementsinplace,EMASdoesnothaveanactualstrategytoleverage
implementationatscale.
o Suchaplanwillneedtoincludedefinitionofthepolitical,resource,advocacy,and
operationalcomponentsthatwillbeneeded;itwillalsorequireascalablestrategic
approachtolocalgovernance,derivedfromtheprogramsexperience.
o TherearemultipleinitiativesinIndonesiathatarelikelytohaveeffectatscaleandofferthe
possibilityoftakingupEMASsprogramapproachesincludingtheMOHsnewMNH
strategy,thepushto2015MDGsandthepost2015agenda,theIndonesiaNewbornAction
Plan(andthenewmaternalmortalityreductiongoal),theJKN,possiblyregionalization.
o ThesupportiveandproactivelypositivepositionstakenbyhighlevelGoIofficialsregarding
connectingEMASwithsuchinitiativesandwithothergovernmentandpoliticalprocesses,in

20
 responsetobriefingsonEMASsworkandtheMTEfindings,areindicativeofhowastrategic
approachtoconnectionatthepolicyandpoliticallevelscanallowEMAStocontributeto
impactatscale.

Recommendation(forUSAIDandEMAS)USAIDandEMASshoulddiscussanddefinethe
approachthatEMASshouldtake,thestakeholderswhoneedtobeinvolved,andthepotential
assistanceandanalyticworkthatarerequiredtodevelopanEMASplanforachievingeffectand
impactatscale.ThisshouldbeastructuredplanforconnectingEMASseffortswithother
forcesandinitiativesthatcanbringkeylessonsandcomponentsofEMASsapproachintothe
mainstreamofMNhealthpolicyandprogramsinIndonesia.Inthiscontext,itshouldclearly
definehowEMASwillnotjustimplementthecomponentsofitsapproach,butwillalsofunction
asathinking,learning,andcommunicatingorganizationtobringitsexperiencesandresultsinto
developmentstreamsthatarelikelytoleadtoscale.

- Akeyelementofbeingsuchathinking,learning,andcommunicatingorganizationandof
contributingtoimpactatscaleistosystematicallygenerateandsharelearningfromits
engagementattheoperationallevel.
o EMASenjoysauniquepositionbyhavingoperationalconnectiontoMNservicesandpolitics
attheoperational(facilityanddistrict)level,aswellashavingpresenceandengagementat
provincialandcentrallevels.
o ThecontributionofEMAStoachievementofimpactatscalewillabsolutelyrequirethe
distillationandeffectivetransferofsuccessfulEMAStoolsandapproaches,andalsokey
experiencegainedthroughitsoperationalengagement.
o ThedevelopmentofaLearningAgendapresentsanmajoropportunitytoworkwithcentral
andlocalMoHandotherstakeholderstoidentifyandhelpanswercriticalquestionsaboutkey
determinantsofMNhealthandservices.
o Inthisprocess,EMAScanreviewwithstakeholdersthemostimportantcomponentsofits
experiencetodate,andassurethattheLearningAgendareflectsthatexperience.
Inadditiontoprovidingquantitativeanalysisofprogrameffects,onepotentiallyuseful
waytopackagethisexperiencewouldbethedevelopmentofwellconstructedcasestudies
reflectingpracticeinkeyareas(e.g.,examplesofeffectivereferralcontributingto
successfulMNoutcome,successfulmanagementofacomplicationresultingfrom
improvedorganizationforemergenciesinapuskesmasorfacility,useofdatatoimprove
MNservicemanagement,etc.).
EMASsownexperienceasitfinalizesandmonitorsmovementalongitsTheoryof
Changecausalpathwayswillalsobeinstructivetothebroadersystem,sincesucha
systematicapproachisoftenlackinginprogramandserviceimplementation.
o ThepresentsetofquestionsthatEMAShasdrafteddoaddressimportantquestionsabout
policyandprogramissuesrelatedtoreductionofMNmortality,includingtherelationship
betweenEMASsapproachandtheoutcomesithopestoaffect.
o However,thisAgendahassofarbeendevelopedonlyinternallyitisimportanttorecognize
thatdecisionmakerswhowillbeimportantinmovingEMASexperiencetoscalemayneedto
answerdifferentquestionsthantheonesEMASitselfmightidentifyforexample,regarding
humanresourcerequirements.
o Therefore,realizingthevalueanduptakeofEMASslearninginrelationtoachievingimpact
atscalerequiresopeningdevelopmentandimplementationofthislearningprocessto
engagementwithcentralandlocalMoHandotherstakeholders.

21
 Recommendation(forEMAS)Giventhelatestageoftheprogram,EMASneedstofinishthis
LearningAgendawithinthenext23months,butmustdosocollaboratively.ThedraftLearning
Agendashouldnowbetakentocentralandoperationallevelpartnersforrefinementandfor
partnershipindesignandimplementationofastructuredlearningapproach.Sucha
collaborativelearningprocessismorelikelytoleadtoprogramandpolicychangesbykey
decisionmakers,andcanalsobuildpartnersorientationandcapacitytobelearning
organizations.

- TheMTEprovidesanopportunityforreconsiderationanddiscussionofthequantitative
targetsthathavebeensetforEMAS,inthelightofbothpoliticalrealityandrealityonthe
ground.
o SubstantialexperienceindesignandmanagementofUSAIDhealthsectorprogramsindicates
thathavingaspecifiedimpactlevelgoal(i.e.,percentmortalityreduction)towhichits
programsmustcontributeisessentialtokeepafocusondeliverablesfromdisplacingthe
focusonimpact.
o Thecontributingtoanational25percentmortalityreductiongoalwassetinagreement
withtheGOI(whichwasatthetimecontemplatinganevenlargerreduction).
o TherecentDHSMaternalMortalityRatioestimatehascausedconfusionaboutratesof
progressinmaternalandnewbornmortalityreduction,andtheappropriatetargetatthis
point.
o Intermsofnumbersofdistricts(30)andfacilities(150hospitals/300puskesmas)specifiedfor
EMAStoreachduringlifeofprogram,thismidpointevaluationisanappropriatepointfor
USAIDandEMAStoreconsiderwhatlevelofnumberofdistrictsandfacilitiesismost
appropriatetomaximizetheeffectivenessandlearningfromEMASswork.
Toensureusefulcoordinatedresponsivereferral,EMAShasfoundtheyneedtoexpand
theirreachtoincludeallpuskesmasofthetargetdistricts,andallhospitals;thisrealityhas
alreadyledtoEMASengagementwithroughly700puskesmasand70hospitalsinjust
Phase2.
EachadditionaldistrictrequiressettingupadditionalPokjasandCivicForums,aswellas
orientingandengagingwithhospitalsandprovidingsomelevelofsupport(whetherFull
SupportorLimitedSupporttoallpuskesmasinthecatchmentareas).
o Severalstakeholders,includingwithintheMoH,haveaskedwhetherEMASmightnotwork
acrossallfacilitiesandinmoredepthinasmallernumberofdistricts,tomaximize
effectivenessandlearning(butnotactasapilot).

Recommendation(forUSAIDandEMAS)USAIDandEMASshouldusethismidpoint
evaluationtodiscussamongthemselves,andwithGoIandotherstakeholders,themost
relevantimpactgoalanddistrict/facilitytargetstomaximizeEMASseffectivecontributionand
learninginthecontextofexperiencetodate,realitiesontheground,andlatestGOIstrategic
planningforMNhealth.

- ThereareimportantchangeshappeninginIndonesiathatcouldoverrideefforts(including,
butnotlimitedtoEMASs)toincreaseeffectivenessofMNservices.
o JKN(universalhealthcoverage)hasrecentlybeenimplemented(effectiveJanuary1,2014);
atthesametimeJampersal,thehealthinsurancethatprovidedmaternitycoverageforall
pregnantwomen,ended.

22
 ThisshifttoJKNcomeswithhigherbenefitsforcoverageofnormalbirthsat
puskesmas/polindes/puskesdeslevelsandforcomplicatedbirthsatpublichospitallevels
(forcesareanaswellasforothercomplicateddeliveries).
Ithowevercreatesdifferentincentivesandchangesthereferralpathway(specifically
statingthatwomenmustdeliveratprimarycarelevelifnormal,andproceedfrom
PuskesmastolevelC,B,andthenAhospitalsifcomplicated);thismayaffectthereferral
networksthatEMAShashelpedtodevelop.
SocializationofthesepoliciesspecificallyforMNHhasbeenlefttotheMOH.
ThoseeligibleforJKNfundsarepublicfacilitiesandprivatehospitalsthatsignaMOU;
privatebidansmustworktogetherwithadoctorinordertosubmitclaims.
- Socialinsuranceprogramshavealreadystimulatedalliancesthatareunethical(e.g.,
bidansbeingpaidbyprivatehospitalstobringpatientstothemforcaesareansection;
orprovidingcaesareansectionsfornonmedicalreasons)
Womenmaygodirectlytoanyhospitaliftheyhaveanemergency;whetherwomenwith
riskfactors(e.g.,twins,anemia)canattendanyhospitalisnotclear,andifthereceiving
facilitydoesnotagreethatthesituationisanemergency,thewomanmaybedenied
care.
Womenmusthavecomplicatedpaperworkavailablewhentheyattendanyhospital(e.g.,
Jamkesmascard,IDcard,SKfromvillagehead;andifreferred,needreferralpapersfrom
theprimarylevelfacility).
TherelationshipofJKNtoJamkesmas(locallyfundedhealthsupportforthepoor)is
unclear,evolving,andvariablebecauseJamkesmasisanonstandardizedlocalprogram.
TodatenofundsforJKNhavebeenpaidasthereisnoPresidentialsignatureatthistime
o Asecondmajorissueisthatfertilityhasplateauedat2.6andcontraceptiveprevalenceis
essentiallystagnant(in2012,58percentmodernmethodsamongcurrentlymarriedwomen
ofreproductiveage).
Thesestagnatingpatternsoffertilityandfamilyplanningmaymeanthatwomenat
higherrisk(e.g.olderwomen,higherparity;veryyoungwomen;thosewithcoinfections
suchasHIVandAIDS,TBormalaria;thoselivinginremoteareas,orwithlesseducation,
andthepoorest)aremorelikelytobecomingpregnant,andalsoaremorelikelytosuffer
complicationsandtodie.

Recommendations(forEMAS)EMASshouldbeawareoftheJKNparametersandworkto
ensurethatfacilitiesatalllevelsproviderapidreceptivitytowomeninlabor(whetherreferred
ornot).Inthelocalitieswheretheyareworking,EMASshouldalsoseektoclarifythe
relationshipofJKNwithlocalhealthinsurance(Jamkesda).ItwouldbeusefulforEMAStowork
withitsprovincialanddistrictlevelstaffandtheMoHtosocializetheJKNrequirements,review
thereferraloptionsforwomen,andensurethatwomenhavethepaperworknecessarytobe
admittedtofacilitiesforthemselvesand/ortheirnewborns.Districtandprovinciallevelpokjas
andcivicforumscanpotentiallyplayimportantrolesinmonitoringtheeffectsofJKNoncare
andreferralofwomenandadvocatingtolocalgovernmentandhealthauthoritiestoassure
maintenanceofefficientcareandreferral.

AspartoftheirstrengthenedcommunicationandcollaborativelearningwiththecentralMoH
andGoI(includingBPJS),EMASshouldalsocarefullydocumentandbringtothecentrallevelthe
effectstheyencounterofthetransitiontoJKNanditsimplementationoncareandreferral
receivedbywomenandnewbornsinthedistrictswhereEMASisworking.Earlyexperience
suggeststhattheMoHmayhavetheabilitytoinfluenceJKNimplementation;EMASmighthelp

23
 identifywaystoapplythisinfluenceconstructivelyforthebenefitofwomenandnewbornsas
thenewinsurancesystemrollsout.

D. Management

- Someimportantmanagementissues,includingactinguponremainingfindingsand
recommendationsoftherecentManagementAssessment,needtobeactedupon.
o InadditiontopointsidentifiedaboveregardingrelationshipswithMoH,etc.
Therehasbeenimportantstaffturnoverinprovinceanddistrictteams;somepositions
remainvacantdespitethosevacanciesaffectingprogramwork(e.g.,QualityImprovement
OfficerinDeliSerdang,ICTOfficerinSurabaya).
SomeEMASfieldstaffexpressedfrustrationaboutcommunicationandbeingintheloop:
Noneofourideashavebeenaccepted,wewanttobeheardandWhereisfollowup
actionaftertheManagementReview?
TheManagementAssessmentrecommendationsabouthiringawellseasonedDeputyis
important;thiscriticalpositionwillrequiresubstantialexperienceinbothprogram
managementandMNCHprogramming,aswellastheabilitytoformexcellentworking
relationswithcounterparts(sincethispositionwillrepresenttheprogramtotheGoI
whentheDirectorisinthefieldoroutofthecountry).
Recommendationsforimprovingorganization,reportinglines,andclearcommunicationin
theJakartaofficeshouldalsobeoperationalized.
Thecurrentorganogramhas6differentprogramcomponentsreportingdirectlytothe
Directorsuchanarrangementismanageriallyineffective.

Recommendations(toEMAS)EMASshouldcontinueimplementingtherecommendationsof
therecentManagementAssessment,includinghiringaseasonedDeputyDirectorwith
substantialmanagementandprogramexperience,andorganizingHQstaff,relations,and
communicationclearlyandeffectively.

- EMASshouldalsomakeallpossibleeffortstofillstaffvacanciesatprovinceanddistrictteam
levels,sincethesevacanciesareaffectingaspectsofprogramsupportandthebackupcapacity
atthenextlevelislimited.

- EMASshouldreviewwithUSAIDtheprogramseffortsandaccomplishmentsintheextremely
importantgovernancearea,sincetheMTEwasunabletoevaluatethisareabeyondtheeffects
ofMOUsandotheroutputsatoperationallevel,andbecauseinteractionwithlocalgovernment
appearedtobesostronglycorrelatedwiththeinfluenceofprovincialanddistrictteamleaders
themselves(thesamedimensionthatneedsstrengtheningatcentrallevel).

OverallRecommendationtoUSAIDEMASisdemonstratingpotentiallyimportantchangesinthe
wayemergencyMNcareisprovidedinIndonesia.Theprocessisslowerthanoriginallyproposed,
butmaystillprovetodevelopimportantcontributionstoMNmortalityreductioninIndonesia.The
EvaluationTeambelievesthatclosingtheprogramafterYear5maycutshortthetimerequiredto
fullydevelopascalableandscaledapproachandderiveandinstitutionalizekeycomponentsofwhat
EMASislearningtodo.Theentropygeneratedbyendingoneprogramandstartinganotherwill
forcethecurrentprogramtohaveeffectivelyjustoveronemoreyeartorespondtomidterm
evaluationrecommendationsandalsoachieveexpansion,scale,andlearning.Experiencewith
previousprogramtransitions(e.g.,theendoftheUSAID/IndonesiasupportedHealthServices

24
 Project)teachesusthatimportantexperiencesandprogressarelikelytobelostinsuchatransition
after5years.

Therefore,theEvaluationTeamstronglysuggeststhatIFANDONLYIFUSAIDdeterminesthat
EMAShasdevelopedthecapacityandtakentheactionsrequiredtorespondtothe
recommendationsaboveregarding:
strengtheningEMASspolicylevelcommunicationandrelationswithkeystakeholders(including
hiringaneffectiveseniorpolicyadvisortoserveasbridgebuilderwiththeMOH,otherGoI
agencies,andseniorleadershipofprofessionalorganizationsandotherstakeholders);
ClearlydefiningtheTheoryofChange(causalpathway)connectionsbetweentheprocess
changesitpromotes,healthsystemperformanceintermsofdeliveringevidencebased
interventions,andMNsurvival,andthensystematicallymonitoring,evaluating,anddocumenting
theprogress(orlacktherefof)inmakingthoseconnections;
implementingacollaborativeandcontinuousapproachtodevelopingandcarryingoutahighly
relevantLearningAgenda;
respondingtokeyManagementAssessmentrecommendations,includinghiringaseasoned
DeputyDirectorwithsubstantialmanagementandprogramexperience,andorganizingHQstaff,
relations,andcommunicationclearlyandeffectively;and,
appropriatedevelopmentofameaningfulstrategytocontributetoimpactatscale;
THENUSAIDshouldconsideridentifyingadditionalfundsandamechanismtoextendEMASswork
bytwoadditionalyears,withoutwaitinguntilyear4or5,sinceactivitiesandkeystaffbegin
standingdownintheendstagesofaprogram.

Extendingtheprogramsworkwillsubstantiallyincreasetheprobabilityofhavingtheimportant
investmentUSAIDismakingthroughEMASachievescalableandsustainableresults.The
conditionalitiesnotedabovelinktheawardofanextensiontomeaningfulcompliancewithMTE
recommendationsactionsnecessarytosupportEMASsuccessandimpactatscale.

IV. AdditionalFindings

Managementrelated

- BurnratesLimitedfinancialreportsmadeavailabletotheMTEteamindicatethatasof
February,2014,theleadpartner(Jhpiego)hasexpended49%ofplannedavailablefundsforits
workintheprogram,whilesubagreementpartnershaveexpendedconsiderablyless(Savethe
Children37%;RTI30%;Muhammadiyah27%;LKBK20%).Theexplanationfortheselow
expenditureratesbysubgranteesmaybeactuallowburnrates,slowbilling(withsubstantial
accruals),orotherfundingpatterndiversionsoracombinationofthese.EMASandUSAID
programmanagersshouldtrackthiscloselyastheprogramentersitssecondhalf.

- NeedforAdvocacy/PublicRelationscapabilityInadditiontourgentlyneedingsenior
presenceatthecentrallevel,asdetailedextensivelyabove,theMTETeambelievesthatEMAS
hasimportantstoriestotell,bothwiththeGOIandalsotoawiderpolicyandpublicaudience.
Thiswilldemandexpertiseinthepublicrelations/advocacyarea.Thatexpertisecanalsohelp
thestrugglingCivicForumstoidentifyappropriateandeffectiveapproachestotheirworkatthe
districtlevel.

25
 Technical

- PreEclampsia/EclampsiaNeedforaspecialfocus(EMAS,MoH,USAID)PE/Ehasemerged
astheleadingcauseofmaternaldeathsinareasthattheMTEwasabletolookat.Accordingto
localspecialistsandlimiteddataonactualnumbersandoutcomesofcases,thisincreased
prominenceofPE/EisNOTsimplytheresultoftheapparentincreasingsuccessinprevention
andmanagementofpostpartumhemorrhagethroughappropriate(3rdstageoflabor)useof
uterotonics.SomespecialistsreportasubstantialincreaseinabsolutenumbersofPE/Ecases,
andalsointheirseverity(withgreateroccurrenceoforgandamageandhighercasefatality
rates).Again,therearenogooddatatoevaluatethesereports.However,theimportanceof
PE/EasacauseofmaternalmorbidityandmortalityinIndonesiacannotbecontested.

OneelementthatEMAShasnotdealtwith,becausetheyarefocusedonmanagementof
complications,isthepotentialimportanceandfeasibilityofimprovingbetterearlydetection
andexpectantmanagementofPE.However,datafromboththeMoHitselfandtherecent
WorldBankstudyonServiceAvailabilityandReadinessindicatethatroughlyhalfof
puskesmasdoNOThaveurinedipsticksforidentifyingproteinuria.Thus,manycasesofPEmay
begoingundetected(orundermanaged).

Againstthisbackground,theMTEidentifiestwopossibilitiesforEMAS,theMoH,andpossibly
alsoUSAID:
o ThroughEMASsengagementwithdistrictlevelMNservices,includingatpuskesmaslevel,
EMAScouldexplorewaystoimprovethesupplies,capability,andawarenessofstaffand
associatedcommunitybidansregardingdetectionandexpectantmanagementofPE/E;and,
o DrawingonbestavailabledatafromEMASswork,plusadditionalfocusedstudiesbypublic
healthresearchers,partnerscouldexploretheincidenceandoperationalfactorsassociated
withPE/Eandassociatedmortality.

- Asimilarneedtolearnmoreaboutprevalenceandmanagementoflowbirthweight/
prematurityAgain,someanecdotalsourcessaythisincidenceisincreasing.Ifso,thismaybe
associatedwiththehighratesofcaesareansectionsbeingperformedinreferrallevelfacilities
(seebelow).Inanycase,managementofthesesmallandpretermbabiesisunevenatbestand
inmostfacilitiescanbenefitfromimprovement.Careduringtransportofthesesmall/preterm
babiesduringreferralisanunknown.Somemovementhasbeennotedinintroductionof
antenatalcorticosteroidsforpretermlaborandKangarooMotherCaremoreworkonthese
willberequired.AnadditionalareawhereinputisneededisContinuousPositiveAirway
Pressure(CPAP)tosupportlungfunctioninpreterminfants.Manyspecialistsinbothpublicand
privatefacilitiesexpressinterestinandexperiencewithuseofCPAPapproachesrangefrom
veryexpensiveCPAPequipment(>$10,000US)to$10bubbleCPAPadaptations.Thereis
muchtolearnfromthisexperience,includingthepossibilityofprovidingCPAPforsmallbabies
duringtransport.Withtheincreasedattentionexpectedtobepaidtonewbornsurvivalasa
resultoftheIndonesiaNewbornActionPlan,andthefocuswithinthatplanonthehigh
proportionofneonataldeathsrelatedtoLBW/prematurity,EMASmightcontributemeaningful
learningandexperiencetomanagementofthesebabiesinthecontextswhereitisworking.
(ThiswouldbeanexcellentresearchtopicforanMD/MPHcandidateorothergraduatestudent
fromadepartmentwithrelevantexpertiseandsupervision.)SavetheChildren/USisinvolvedin
evaluationofCPAPinlowresourcedevelopingcountrysettingsthroughitsSavingNewborn

26
 LivesprogramtheexistingconnectionofSavetheChildrenneonatalexpertswithEMAScould
transferthisexperiencetoIndonesiaandhelpEMASsupportrelevantactioninthisarea.

- MortalitybelowpuskesmasisthereanyEMASrole?Thequalityofbidancareischallenged
bytheincreasingnumberofbidansandmidwiferyacademies.EMASitselfisnotdesignedto
dealwithbidansoutsidethepuskemaslevel.However,someEMASassistedfacilitiesandEMAS
consortiumpartnersareworkingtoimprove/sustainqualityofcarebybidandidesaandprivate
bidans.EMASmaywanttoconsiderworkingwiththosepuskemasandpartnerstoprovide
simpletoolsthatwillhelpthemintheseefforts.

- InfectioncontrolappearsweakatbestWhileEMASdoeshaveprocessindicatorsoninfection
control,theMTETeamdidnotcomeawayhighlyimpressedbytherealitiesofinfectioncontrol
inmanyfacilities.Inmorethanonecase,weaccompaniedseniorneonatologistswhoentered
highrisknurserieswithoutwashinghands,andthenwhilenotdirectlytouchingbabies(while
wewerethere)touchedobjectsthatotherstaffwouldthenlatertouchinprovidingcare.
Handwashingwasnotcommonlyobserved,andinonefacilitythealcoholdispensersoutsidethe
neonatalunitwereallempty.EMASshouldlookmorecloselyattherealitiesofinfection
control,andeffectiveinterventionstostrengthenit.

- HighratesofcsectionandinductionCsectionratesinmanyofthereferralfacilitieswe
visitedwereveryhigh,evenconsideringthefactthattheseareplacesreceivingcomplicated
casesandcsectionisthenecessaryinterventioninmanycases.Ratesreachedashighas90per
centinoneprivatefacility.Insuranceschemesandotherfinancialincentives,possiblyaswellas
demand,providesomeperverseincentivesforperformingcsections.Thereisadocumented
riskofcomplicationsassociatedwithcsectionforthemotherandbabyincludinginmany
casesthefactthatbabiesdeliveredbycsectionareatleastsomewhatpreterm.Thequestions
forEMASare,whatistheappropriatelevelofcsectionsinthefacilitieswithwhichtheyare
working,andwhatisEMASsroleifthepresentrateisclearlytoohigh?EMASshouldalsoseek
waystopreventeffectivereferralfromincreasinginappropriateratesofdeliveryby
cesareansection.

Inaddition,theuseofoxytocintoinducelaboralsoseemedpotentiallyexcessive(inoneRSUD
hospital,almosthalfofdeliveries).Oxytocinoverusecancauseharmtothemotherand
especiallythefoetus,possiblyincreasingthedangerofbirthasphyxia.Again,whatisEMASs
roleinlookingatthesepractices,andinterveningifinappropriatepracticesareprevalent?

- ValueaddedofexpatriatespecialistvisitsTheteamreceivedlimitedfeedbackonthis
practice,butwhatitreceivedwasgenerallypositive.Accordingtooneprivatefacility,the
practiceimprovementinputsprovidedbyanoutsidergotmuchmoreattentionthanwouldthe
sameinputsfromanIndonesianexpert.Forprivatefacilities,therealsoappearedtobea
prestigefactor.TheMTETeamisunabletojudgewhetherthisvalueaddedjustifiesthis
componentoftheprogram.

V. SummaryofConclusionsandRecommendations

Basedontheinformationitreviewedandreceivedanditsobservations,theevaluationteam
concludesthattheapproachesdevelopedandimplementedbyEMAShaveimportantpotentialto
improvetheperformanceofIndonesiashealthservicesinmanagingthecomplicationsthatarethe

27
 majorcausesofmaternalandnewbornmorbidityandmortality.Reportedly,whenUSAIDwas
workingwiththeGoIondesignofEMAS,seniorhealthofficialsstatedthatWehavegoodpoliciesand
guidelines;whatweneedhelpwithnowisimplementation.Basedonthechangesinservicedelivery
reportedandobserved,theMTEteambelievesthatEMASisprovidingthathelp,inwaysthatappear
tobeacceptable,feasible,andthereforescalableinthedecentralizedIndonesianhealthsystem.
ThesechangesinqualityofMNcareandinthepoliticalandresourcesupportthatEMAShasalso
developedwouldbeexpectedtopositivelyaffectoutcomesformothersandnewbornsexperiencing
lifethreateningcomplications.

However,torealizethispotentialatscale,EMASneedstoaddresscriticalchallenges.Oneisto
defineandmonitortheconnectionbetweentheprocesschangesthatEMAShassucceededin
generating,withsuchimprovementinpatientoutcomesandultimatelymortality.Equallyimportant
isstrengtheningpolicyandpoliticallevelcommunicationandconnectionwiththeGoIespeciallythe
MoHatthecentrallevel.PartofthatimprovedcommunicationwillrequireEMASbecomingthe
thinking,learning,andcommunicatingorganizationthatitsengagementacrosslevelsofthehealth
systemuniquelypositionsittobe.EMASalsourgentlyneedstoworkwithUSAID,theGoI,andother
stakeholderstodevelopastrategyforleveragingimpactatscalethatgoesbeyonditsdirect
engagement,tolinkitsresultsandlearningwithnationalscaleinitiativesandprograms.Internally,
EMASneedstodealwithmanagementandorganizationissuesidentifiedbytheearlierManagement
Assessmentandtheevaluationitself.

Morebroadly,theMTEhasidentifiedseveralkeyareasidentifyingactualoperationaland/orclinical
causesofongoingmortality,identifyingeffectsofthenewJKNinsuranceprogramonmanagementof
maternalandnewborncomplications,astrongerfocusonlearningaboutandrespondingtohighrates
ofPE/Eandprematurity/lowbirthweightinwhichbothEMASandUSAID,alongwithother
stakeholders,canpotentiallymakeimportantcontributions.

Appendices[Attachedseparately]

28
Appendix1

MIDTERM EVALUATION SOW

I. BACKGROUND OF PROGRAM
EMAS, USAID/Indonesias flagship MCH program, aims to substantially contribute to a rapid
reduction in maternal and new born mortality - by 25% over the next five years. With an
investment of $55 million, EMAS is the largest bilateral agreement in the USAID Health office
portfolio.
EMAS differs from prior USAID maternal and child health programs in its strong strategic focus
on health facilities with the highest burden of mortality of mothers and newborns. EMAS works
in six priority provinces in Indonesia: South Sulawesi, North Sumatra, East Java, Central Java,
West Java, and Banten. Almost 70% of all maternal deaths and 75% of newborn deaths occur in
Java and Sumatera alone, mostly from preventable causes.
In order to accelerate progress in reducing maternal and neonatal mortality, EMAS is focused on
two main results:
1) Improve the quality of maternal and newborn health services at health facilities in the
event of complications; and
2) Increase efficiency and effectiveness of referral systems for maternal and newborn health
complications to save the lives of mothers and babies by assuring that they get to the
right place, at the right time, in order to receive the right services.

These are supported by major cross-cutting themes:

Work in the province and district level to increase accountability and transparency in the
health sector;
Provide technical assistance on policy and decision-making related to maternal and
newborn survival; and
Use technology, such as cell phones and social media, to increase information flow
between Ministry of Health, health facilities, and patients.

EMAS is implemented by a consortium led by JHPIEGO with Save the Children, RTI, Budi
Kemuliaan Foundation and Muhammadiyah. It is coordinated with other assistance programs
funded by USAID including UNICEF, WHO maternal assessment, AIPI/NAS Maternal
Mortality Study, the WHO accreditation project, and the Indonesia DHS Survey.
A strategy to reach at least 150 hospitals and 300 health centers is aimed to be achieved through
a vanguard referral network approach with influence on hospitals and districts outside of the
facilities and districts where EMAS works. The vanguard network functions as a mentoring
network. This mentoring network will be paired with additional referral networks over the life of
the program to promote peer learning and reinforcement of best practices. Each referral network
will typically include one public hospital, two to three private hospitals and approximately 10

health centers. In the first year and second year, the EMAS program provided EmONC technical
support to 10 districts working with district hospitals and health centers as well as private
hospitals and health centers. The initial strategy is to develop these 10 networks to function as
mentoring hospitals to other district hospitals and classify them as vanguard networks. A
network of hospital and health centers who are able to provide mentoring support to other
districts. These 10 networks were expected to function as high performing district hospitals to
support other district hospitals in the third year. Additional 29 districts and cities will receive
technical support to improve the quality of EmONC services and improve referral network
through the course of the project.
EMAS has completed its second year marking the first full year of implementation of EMAS.
The second year annual report revealed steady progress across all program interventions. The
vanguard network approach is focused on a staggered approach roll out series: Phase 1
(Program Years 1-2), Phase 2 (Program Year 3) and Phase 3 (Program Years 4-5)
Initially EMAS first year and second year approach included only district hospitals and relied
heavily on their participation to roll out the activities to other district. The approach was revised
in the middle of year 2 implementation to include hospitals in the cities and at the province level.
Province level hospitals were defined to function as regional experts to provide technical support
to district and city hospitals. In addition provincial hospitals were also assessed for their ability
to function as the referral network umbrella to support the emergency referral network. This
effort is in line with Ministry of Healths effort to improve the referral network at the central,
province and district level.
USAID commissioned a management review of EMAS to begin in January to identify
management gaps, constraints and opportunities in EMAS management and staffing. In addition
to the management review initiated by USAID mission, JHPIEGO central office included in their
2014 workplan technical support to Chief of Party and the Deputy Provincial Officer to improve
management systems and structure within EMAS. Lastly an assessment/ study on the use of
Ante-natal corticosteroids is currently on going and being implemented together with MCHIP in
EMAS selected district. The results of these reports will be available to the evaluation team.
II. PURPOSE AND UTILIZATION OF THE EVALUATION
The purpose of the evaluation is to:
1. Assess EMAS progress in achieving its goal, objectives and planned outputs as stated in
the agreements project description and in approved workplans;
2. Provide recommendations to improve EMAS program effectiveness over the remaining
2+ year life of project.
3. Provide recommendations for USAID to consider in the design of future projects aimed
at improving maternal and neonatal health in Indonesia.

The timing of this evaluation is propitious for both making mid-term changes in EMAS
implementation as well as for providing input toward future program design. Therefore the
evaluation should produce two sets of recommendations for USAID. The first set should provide
specific recommendations for mid-course corrections to the EMAS project. The second set of
2

recommendations should provide USAID with recommendations to take into consideration for
future project design in maternal and neonatal health. There will be two versions of the final
report an internal USAID only version which contains the recommendations for future
programs and an external version that is available to the public and does not contain this section.
III. EVALUATION QUESTIONS
The Evaluation Team will answer the following questions. Question 6 will be addressed by a
USAID financial specialist and provided to the Evaluation Team for review and inclusion in the
report as appropriate. The term governance when used here is the common term regarding
such issues as local government policy and budget support, public participation in decision-
making, public oversight of the quality of services, etc. This term does not include or refer to
clinical governance which indicates aspects of leadership, management and administration of
clinical services.
1. What are the major EMAS accomplishments to date? Identify key strengths in the EMAS
program approach.

2. What evidence is there to validate the overall development hypotheses and programmatic
approach? A complete response will address at a minimum:
a. Effectiveness of technical content of EMAS.
b. Strengths and weaknesses of the EMAS vanguard model, mentoring approach,
engagement of partners through POKJAs, and engagement of provincial hospitals.
c. Effectiveness of ICT and governance interventions, judged by contribution to achieving
health objectives?
d. What success has been achieved in engaging the private sector service providers? What
opportunities, strengths and weaknesses can be identified to guide additional actions?
e. Have there been any unanticipated changes in the host country or donor environment that
suggest the need for changes in emphasis in the EMAS project to minimize
implementation problems or unintended consequences and/or maximize impact in the
remaining time available?

3. To what extent have monitoring information and lessons learned during project
implementation been used to inform project management decisions? A complete response
will address at a minimum:
a. Whether systems for program monitoring are providing timely and relevant information
to the appropriate individuals with responsibility and authority to act.
b. Adjustments to program approaches that have been made based on such information.
c. Whether such adjustments are likely to improve prospects for program impact,
sustainability and scale-ability.
d. Recommend specific new approaches and decision support tools to improve feedback for
informed decision making.

4. What are the prospects for EMAS achieving impact at scale? A complete response will
address at a minimum:

 a. The extent to which the approach to achieving sustainability and impact at scale are
articulated in project documents.
b. Whether EMAS approaches and materials are sufficiently in-line with existing standards
and systems to be integrated into standard practice in systems operating at scale.
c. The extent to which the EMAS learning agenda addresses main policy and program
questions and evidence requirements to support sustainability and spread of EMAS
innovations and approaches.
d. The effect of partnerships with U.S. hospitals, commodity donation charities, or the
private sector (Laerdal, GE, Chevron) on programmatic results or prospects for
sustainability. What are the strengths, weaknesses, lessons learned, unintended outcomes,
and cost effectiveness of these endeavors?
e. Opportunities, strengths, and weaknesses of EMAS engagement of Indonesian partners
both within the project and external including government and private sector entities at
the central, provincial and district levels, leadership of public and private facilities,
professional associations, academics, and civil society.

5. Are all expected results likely to be achieved by the completion of the project and, if not,
what changes in targeted results and/or implementation approaches should USAID/Indonesia
consider?
a. Are work plan milestones and results being achieved?
b. Are EMAS project implementation priorities sufficiently focused for the best application
of limited resources? Are there low yield (or likely low yield) project elements that
should be reduced or eliminated? Are there elements that should receive increased
attention and resources?
c. Is the project reaching the desired beneficiaries? If not- why not?

6. Is there a clear financial system of the prime and the sub-awardee that includes internal
mechanism to ensure a clear financial reporting and cash flow?
a. How effective has cash flow been managed in the project? Have there been any significant
delays in cash flow either from the prime awardee to the major partners? What was the
cause of the delays? What changes were made in managing cash flow?
b. Have there been annual audits conducted for the prime as well as the sub-awardees? What
have been the audit findings?
c. Have there been problems with financial reporting from the sub-awardee to the prime and
how are they resolved?
d. How is the cost-share commitment being met?

IV. TASKS
1. Review background documents
2. Participate in a team planning meeting in Jakarta review and refine SOW and
evaluation framework, develop outline of report, finalize roles and responsibilities of
team members, and develop detailed work plan with USAID and key stakeholders.
3. Review and further analyze further program information PMP, Assessments, etc.
4. Conduct interview with stakeholders and key informants (list)
5. Conduct field visits to xx districts
4

 6. Analyze information collected and draft main findings and recommendations

7. Conduct interim briefings with USAID, EMAS team, others as needed
8. Draft full report
9. Develop and present power point summary in exit briefing for . . .
10. Finalize report incorporating feedback from USAID and other reviewers (TBD)

V. COMPOSITION OF EVALUATION TEAM

The Evaluation Team shall consist of four professionals with 10+ years of experience in MCH
programming in low-income countries. The team leader will be independent of USAID. The
team shall also include a translator/interpreter.
The required areas of subject matter expertise that should be represented on the team correspond
roughly to the technical foci and implementation context of the project being evaluated:
1) Maternal Health Expert
2) Neonatal Health Expert
3) MNH Policy and Health Systems Expert
4) MNH Evaluation Expert

All team members must be fluent in English, have proven ability to interact with people from
many different social and economic backgrounds, and possess excellent writing and presentation
skills. The team will have combined skills and experience in rapid appraisal methodologies
(interviews, focus groups, etc.), institutional analysis, and strong knowledge of Indonesias
public sector functioning and Indonesian political processes. All team members must be willing
and able to travel to remote zones.
The Team composition is suggested as follows:
1. Team Leader
The Evaluation team leader will lead the evaluation team to carry out the SOW listed below.
The team leader will be specifically responsible for ensuring evaluation questions are answered,
the report is complete, and deliverables are met on time. S/he is the lead on clinical and global
best practices for programming related to best practices in neonatal health, from both clinical
and policy perspectives. As such the team leader will take the lead in developing all components
related to neonatal health within the questionnaire, indicators and analyis. S/he will collaborate
with the team to answer all the evaluation questions, analyze inputs to make conclusions, and
provide written recommendations.
2. USAID/Washington Maternal Expertise:
The Maternal Health Senior Advisor will work with the other members of the evaluation team to
carry out the SOW listed below. She is the lead on clinical and global best practices for
programming related bto best practices in maternal mortality prevention and response, from a
clincial and policy perspective. The Maternal Health Senior Advisor will take the lead in
developing all components related to maternal health within the questionnaire, indicators and

analyis. She will collaborate with the team to answer all the evaluation questions, analyze inputs
to make conclusions, and provide written recommendations.
3. Indonesian MNH Policy and Health System Advisor:
The Senior Maternal and Newborn Health Policy and Health System Advisor will work with the
other members of the evaluation team to carry out the SOW listed below. The Senior Maternal
and Newborn Health Policy and Health System Advisor will collaborate with the evaluation
team lead to engage government stakeholders at the central, provincial and district offices. The
MNH Policy and Health System Advisor will be the lead to ensure all relevant and updated
standards and policies related Maternal and Newborn Health are gathered and included in the
evaluation process and contribute to answering the evaluation questions, report, and expected
results.

4. Indonesian MNH Evaluation Specialist:

The Maternal and Newborn Health Evaluation Specialist will work with the Evaluation Team
Leader and other members of the evaluation team to carry out the SOW listed below. The MNH
Evaluation Specialist will provide general support to the evaluation and will work closely with
the team leader in supporting evaluation design, development of all evaluation documents and
completion of deliverables. The MNH Evaluation Specialist will have specific responsibility for
evaluation of EMAS M&E functions and products, and will contribute to evaluation of progress
on the learning agenda. The MNH Evaluation Specialist will also take the lead to evaluate the
contribution of the ICT and governance elements of the program to EMAS main results.
VI. USAID MANAGEMENT OF EVALUATION
The USAID/Indonesia point of contact for the evaluation will be Ria Wardani. An Evaluation
Committee comprised of the EMAS AOR, a representative of the Mission Program Office and
MCH team members from the Health Office will be formed to respond to questions from the
team, resolve administrative or logistical obstacles, and review Evaluation Team deliverables.
DRAFT: Logistics & Time Frame
The following provides a notional presentation of a prospective allocation of level of effort for
the Evaluation assuming a six-day work week:

Estimated Activity Working Location

Start Date Days

April 1-2 Preparation Selection of site visit locations and preliminary 2 days Anywhere
specification of planned interviews. Finalization of
evaluation methodological approach(es) and field schedule.
Document review. Development of questionnaires and/or
other tools to be used in conducting surveys and fieldwork.

April 7-9 Team Planning Meeting and In-brief with USAID/Indonesia 3 days Indonesia

 staff, EMAS implementing partners and stakeholders.

April 10- Field Work and Data Analysis Interviews, site visits, and 12 days Indonesia
23 analysis of comparative performance data. The team may
split into two groups for interviews at different stages of
field work.

April 24- Initial synthesis In-country team work culminating in 5 days Indonesia
29 delivery of Detailed Evaluation Report Outline and draft
PowerPoint presentation for review by Evaluation
Committee. Additional meetings and interviews may also be
scheduled to validate findings.

April 30 Revision and refinement In response to comments from 5 days Indonesia

May 5 Evaluation Committee, team will incorporate feedback and
other input into finalized PowerPoint presentation and initial
full report draft. Presentation to USAID/Indonesia and other
stakeholders.

May 9 Final report production Completion and delivery of final 3 days Anywhere
evaluation report based on Mission feedback.

Total:30 days
VII. DELIVERABLES
The Evaluation Team will be responsible for producing the following deliverables:

Revised evaluation approach and draft schedule of field activities (prior to field work)
Draft and final questionnaire(s) to be used during interviews/stakeholder meetings (prior
to field work)
Draft Report Outline (prior to field work)
Detailed Evaluation Report Outline with bulleted response to evaluation questions and
Draft PowerPoint Briefing (at the end of the synthesis phase)
Finalized PowerPoint De-briefing and initial full report draft (before evaluation team
departs Indonesia)
Final Evaluation Report following standard reporting format and branding guidelines
(within 2 weeks of receiving Mission comments on draft report).
An illustrative outline of the Evaluation Report is provided below:
Executive Summary
The Executive Summary will state the EMAS objectives; purpose of the evaluation; study
method; findings; conclusions, lessons learned and recommendations for remaining
EMAS implementation, future USAID programming priorities.
Table of Contents

 Introduction
The context of what is evaluated including the relevant history demography
socioeconomic and basic political arrangements.
Body of the Paper
1. The purpose and study questions of the evaluation. A brief description of the project.
2. Evidence, findings and analysis of the study questions.
3. Conclusions drawn from the analysis of findings stated succinctly.
4. Recommendations for EMAS mid-course corrections
5. Recommendations for USAID future directions

Appendices shall include:

1. Evaluation scope of work
2. List of relevant targets and results
3. List of documents consulted
4. List of individuals and agencies contacted
5. Schedule of activities in an Excel format
6. Evaluation Team composition
7. Details on evaluation methodology if necessary

All reports are to be submitted in English in both electronic and hard copies. The Team will
provide 5 printed copies of the Draft and Final Evaluation Reports and 5 printed copies of the
PowerPoint presentation.
The Final Evaluation Report should not exceed 30 pages in length in its body, not including title
page; Table of Contents; List of Acronyms; usage of space for tables, graphs, charts, or pictures;
and/ or any material deemed important and included as Annexes. The executive summary with
brief evaluation findings, conclusions and recommendations will be translated into Bahasa
Indonesia and included in the final report.
The Final Evaluation Report and PowerPoint addressing the Mission's comments should be
submitted in both Word and PDF formats. Once the PDF format has been approved by the
Mission, the Team will submit the Final Evaluation Report to the Development Experience
Clearinghouse for archiving.

Appendix2ListofRelevantTargetandResults

TheEMASvanguardnetworkapproachoriginallywouldberolledoutinthreephases:Phase1
(ProgramYears12),Phase2(ProgramYear3)andPhase3(ProgramYears45).InPhase1,
activitiescovered23hospitalsand93puskesmas(healthcenters)in10districtsoutofthe
targeted30districtsandcitiesinsixprovincesNorthSumatra,Banten,WestJava,Central
Java,EastJava,andSouthSulawesi.BetweenPhase1andthefirstquarterofPhase2
(September2011andDecember2013),theresultsofEMASEmONCtechnicalsupportshowed
thatonlyonehospital(RSMargonoinEastJava)hasachievedatleast80%ofallfourclinical
performancestandardsattheendofYear2.EMASthenchangeditsstrategytomeetthe
mentoringdemandastheprogramenteredPhase2(Oct2013Sep2014).Phase2plannedto
coveranadditional55hospitalsand100morehealthcentersin13districtsandcities.Thenin
Phase3,7moredistrictsandcitieswillbeincluded.EMASincludedcitieswiththeexpectation
toincreaseprogramimpactbywideningitsgeographicareastocovermajorreferralhospitals
andvertical,provincialorinfluentialMuhammadiyah

EMAShastoassignindividualvanguardhospital/puskesmastomentoronmuchnarrowertasks,
mentoringonlytheelementsofComponent1and2wheretheythemselveshadsuccessfully
achievedstandards.But,thisstrategyhastodealwiththefactthatsomehealthfacilities
(publicorprivatehospitalsandpuskesmasorclinics)wouldnotbeabletosendtheirstaffto
traveltoPhase2districtstoconductmentoring.Despitebeingverysupportive,thehospital
directorsinthevisitedSidoardjoandJombanghospitals)expressedtheirconcernsonhowtheir
ObGyns,midwivesandPediatricianswillmanagetheirtimetoPhase2districtstodo
mentoringwhiletheymustmeetalltheirroutineresponsibilitytothehospitalincluding
teachingandcoachingtheresidentspecialists,medicalschoolinterns,andstudentsfrom
midwiferyandnursingacademies(becausemostlargedistricthospitalsserveasteaching
hospitals).Withinthissameperiod(2012/2013),theMinistryofHealthissuedanewpolicyon
appointingaregionalhospitalatprovincethathasspecialistswhoarecapabletoprovide
technicalsupporttodistrictandcityhospitals,andaregionaldistricthospitalthathas
specialiststoaccommodateandsupportpuskesmasandprivatepracticemidwives(Bidan
PraktekSwastaorBPS).

Atthepuskesmaslevel,themodifiedmentoringassignmentreceivedhigheracceptance.
DuringfieldvisitstoPuskesmasTalungKenasinNorthSumatraandPuskesmasBarenginEast
Java,midwivesandnursesconfirmedthatreceivingthementoringassignmenthasboosted
theirconfident,disciplinetolearnmoreandbecomeagoodrolemodelforthementees.Also,
theyaffirmedthatmentoringapproachisfeasibleanddoesnotrequireadditionalhugecosts.

AdeskreviewofEMASaccomplishmentswasdonebycomparingresultsofeachEMAS
indicators(Component1andComponent2)inYear1and2withthePMPtargetsofYear3end,
andfurtherlinkingwithresultsfromQuarter1ofYear3(uptoDecember2013)toconfirm
achievementsofYear2.Achievementswererankedaccordingtothreecategories:
outstandingresults(targetachievedfastandwaybeyondthePMPtargets);slowprogress
9

(reachedbarelyjustthetarget)andfluctuatedresults(notmuchprogressrecorded).The
outstandingresultsreflectnotonlythecompetencelevelbutalsowhichstandardcarehas
beenenforcedbytheMOH(government)inthepast.Theslowprogressachievementindicate
newhabitsandpracticesareinneedtobemonitoredlongertobecomepermanent
behavior/practices.Thefluctuatingresultsandnoresultcategoriesneedmoreattentionand
actionscomingdirectlyfromEMAScentrallevel.

Component1:ImprovedqualityofEmONCinhospitals(privateandpublic)andcommunity
healthcenters(puskesmasandBPS).
Successfuloroutstandingresultswereseenintwoclinicalinterventions:activemanagement
ofthethirdstageoflabor(AMTSL)oruseofoxytocin(uterotonicagent)withinoneminute
followingthedeliveryofthebaby(94%)andearlyinitiationofbreastfeeding(60%)[passingthe
year3endPMPtargetsof90%and50%].ThisindicatesthattheMOHhasenforcedAMTSLas
afeasibleandinexpensiveinterventiontobepracticedbyallskilledattendants(includingin
homebaseddeliveries)becausethe2002to2003DemographicandHealthSurvey(IDHS)anda
2002mortalitystudybytheIndonesianNationalInstituteofHealthandDevelopment(NIHRD)
reportedthat77percentofmaternaldeathswereduetodirectcauses.Ofthesedirectcauses,
themaincauseswere:postpartumhemorrhage(33percent);preeclampsia(25percent);
infection(12percent);unsafeabortion(5percent);andprolongedlabor(5percent)(reported
inPOPPHIforUSAID.ManagementoftheThirdStageofLabor:Dataobtainedfromhome
deliveriesintheCirebondistrict,AugustSeptember2006).

Successinnonclinicalinterventionswasseenintheintroductionofnearmissauditsthat
achieved43%,waybeyondthe15%target.Inpublichospitals,thisauditincreasedsignificantly
from27%to64%;whileprivatehospitalswhichneverhadthisbefore(0%)begantoperformup
to25%afterbeingsupportedbyEMAS.However,the43%achievementsfellbackto26%in
Quarter1ofYear3,indicatingthatbehavioralandhabitualchangeisstillnotpermanently
planted.

Anotherfouroutcomeindicatorsachievedyear3endtargetsbyDecember2013,although
achievementsattheendofYear2hadnotcomeclosetotheyear3endtargets.Theseare:
- Percentageofnewbornswhosemothersreceivedantenatalsteroids(PMP#6);
- PercentageofEMASsupportedfacilitiesthatconductdeathauditsonallfreshstillbirths
>2000grams(PMP#7);withpublichospitalsshowedsignificantprogressbutnodata
frompuskesmasbecauseofthereferralstohospitals;
- PercentageofEMASsupportedfacilitiesthatconductdeathauditsonallneonatal
deaths>2000grams(PMP#8);
- PercentageofEMASsupportedfacilitiesthatconductdeathauditsonallmaternal
deathswithin24hoursofoccurrence(PMP#9);bothprivateandpublichospitals
showedsignificantincreases,butnodatafrompuskesmasbecauseofthereferralsto
hospitals.

SlowProgressResults:thefollowingPMPoutcomeindicatorwasnotachieved:
10

 - PercentageofEMASsupportedfacilitiesthatachieve80%ofEmONCperformance
standards(PMP#2),whilethisistheprerequisiteforfacilitiesreadinesstofunctionas
DistrictVanguards(achieving80100%complianceinallfourkeycategoriesofmaternal,
newborn,infectionpreventionandclinicalgovernance).Theoverallachievementwas
17%inSeptember2013andonlyincreasedupto51%byDecember2013,despite
promisingaverageperformanceformaternal(65%inSeptember2013and84%in
December2013),newborn(65%inSeptember2013and84%inDecember2013)and
infectionprevention(78%inSeptember2013and84%inDecember2013).Onlyone
hospital(RSUDMargono)passedthecompliancewithatleast80%ofallfourclinical
performancestandardsattheendofYear2.EMAShastochangeitsmentoringstrategy
becausetheoriginaltargetwastohavesixVanguardhospitalsreadytobeginmentoring
inPhase2.
Thepuskesmasperformancewereactuallyunexpectedlygood,becausetheywereable
toachieve42%(n=93),beyondthe40%Year3endtargetofachieving80%ofBEmONC
standards.Puskesmasperformanceforinfectionpreventionstandardsreached61%(n=
87),alsobeyondthe40%Year3endtargetofPhase1.
Becauseoftheabovelowachievement,thereadinesstofunctionasVanguardfacilitieswas
redefinedtoallowmorethanonePhase1hospitalbeganmentoringinPhase2.Andsix
hospitalshavebeenassignedtobeEMASmentorfacilities:Asahan,Banyumas,Margono,
Sidoardjo,MajalayaandKanjuruhanMalang.

FluctuatingResultwasseeninaveryimportantclinicalintervention:percentageofcasesof
severepreeclampsia/eclampsiamanagedwithmagnesiumsulfate(MgSO4)accordingtoglobal
standardsatEMASfacilities(PMP#3).Percentagesfluctuatedfromhighbaselinedataofover
80%to92%inYear1backto80%attheendofYear2,andwasstillaslowas79%inQuarter1
ofYear3(December2013)withprivatehospitalsperformedworsethanpublichospitals.
Surprisingly,Puskesmasperformanceincreasedfromnobaselinetoashighas88%attheend
ofYear2(September2013).Thequarter1ofYear3didnotreportonthehospitaland
puskesmasdifferences.Aninvestigationisneededtoobtainreasonsoflowyear2
achievementscomparedtoyear1.

EMASdevelopedadditionalindicatorstomonitorclinicalperformanceaspartofthementoring
inhospitalsinQuarter1ofYear3(Sep.December2013),withtheintroductionofEMAS
decisionsupporttools(DST)asetoftoolstoimproveadherencetoevidencebasedprotocols
onmajorcomplicationscontributingtomaternalandneonatalmortality..Thetoolsare
supposedtobepilotedinMuhammadiyahCempakaPutihhospitalinfirstquarterofPhase2,
togetherwithintroducingthesetoolsin8facilities:Majalaya,ParePare,Lasinrang(Pinrang),
Kardinah(KotaTegal),Soesilo(Tegal),PKUMuhammadiyahTegal,DeliSerdangandRSUP
Bantenhospitals.MentoringhasbeenprovidedbyEMASstaffandUSpediatricians.Five
facilitieshavereadilyadoptedthesetoolsandinstructedtheirnursestousethem,butthree
facilitieshavebeenlessreceptive(aspediatricspecialistshavenotyetacceptedthetools).

EMAScollectsadditionalindicatorsthatarenotrequiredinPMP(USAID)suchas:
11

1. #offacilities(RSUD)usingdecisionsupporttools;
2. #ofhospitalsusingdashboardsw/minimumsetofindicators;
3. #ofPhase1hospitalsservingasmentorsforPhase2hospitals;
4. #offacilitieswithsignedservicechartersinplace;
5. #ofhospitalswithcitizenfeedbackmechanisminplace;
withthefirstthreeachievedmuchlowerthantheYear3endtargetof15hospitalsandthelast
twohavemettheYear3endtargetof100%.

Noresultsarefoundforthreeindicators:(notrequiredinPMP):
NumberofcallsmadefromprovidersinEMASsupportedfacilitiestoanemergencyobstetric
andneonatalcarehotline;
PercentageofcorrectresponsestoSMSprovidersupportquizzessenttoproviders;
PercentageofSMSrecipientswhorespondtoquizzes.
Somefieldstaffinformedthatwheredatawereavailable,thevaliditywasquestionable.

Component2:IncreasedEfficiencyandEffectivenessofReferralSystemsbetweenCommunity
HealthCentersandHospitals
Successfuloroutstandingresultswereseenontwooutcomeindicators:PercentageofEMAS
referralnetworksachieving100%ofreferralperformancestandards(VanguardNetwork
ReadinessSummary)(PMP#14)andPercentageofreferralcaseswithahospitalresponse
occurringwithin10minutesuponreceiptofSijariEMASnotification(PMP#16).
ResultsshowedthatSijariEMASreferralhaveachieveditsperformancetargets.

Fiveotheroutstandinginput,process,outputindicatorsthathaveachievedresultsbeyond
year3endtargetsare:
- PercentageofEMAStargetfacilitiesthatsignaservicecharterwithcommunity.This
indicatorreachedanoveralof100%(n=116)withall3typesoffacilities(Private
Hospital,PublicHospital,Puskemas)bySeptember2013;
- Numberofdistricts/citieswherereferralsystemstandardsareintroduced(Phase1and
2):10districtshaveachieved100%ofthetargetof10districtsofPhase1;
- NumberofdistrictsusingSijariEMAStofacilitatereferrals(Phase1and2):10districts
haveachievedthetargetof100%outof10districtsofPhase1;
- Percentageofwomenwithseverepreeclampsia/eclampsia(PE/E)whoarereferredto
EMAShospitalsfrompuskesmas/clinicsandwhoreceiveatleastonecorrectdoseof
magnesiumsulfate(MgSO4)beforereferral.WhileattheendofYear2thisindicator
wasstillfarfromthetargetof40%beforereferralbytheendofQuarter1ofYear3,this
indicatorhasachieved73%.
- PercentageofnewbornswithsuspectedsevereinfectionwhoarereferredtoEMAS
hospitalsfrompuskesmas/clinicsandwhoreceiveatleastonedoseofantibioticsper
nationalguidelinesbeforereferral.Achievementsamongprivatehospitalsareveryhigh,
morethandoubledthe30%targetofendofYear3.

12

SlowProgressresults:DocumentingresultsweredifficultonNumberofobstetricornewborn
casesreferredtoEMASsupportedhospitalusingSijariEMAS(PMP#15)althoughtheuseof
SijariEMAShasbeenhighlyaccepted.Hospitalsarereluctanttohireaspecialstafftorecordthe
numberobstetricandnewbornreferalcasesreceivedusingSijariEMAS.AlsotheMaternal
PerinatalAudit(MPA)conductedbythedistrictreviewteamisanactivitythatneedsnotonly
districtPokjaleadershipbutalsohealthresourcepersonsthatisnotthereatthemoment(PMP
#19).Morethoughtsshouldbedirectedonwhetherdatafromthesetwoindicatorsareworth
tocollect.

Thefollowingisanindicatorwithnoresultbutactuallyhaveapromisingfutureifactionsto
solvetheproblemscomedirectlyfromEMAScentrallevel.
Referralstandards(performancemonitoringtools)developedwithEMASassistanceare
adoptedbyMOH.TheMOHhasnotadoptedanyofEMASreferralmonitoringtools.A
discussionisongoingwithDrDiarIndriatitheHeadofSubDirectorateforPublicHospital
(underDirectorateofHealthReferral)(11April2014),andpositiveresponsewasgiven(byDr
DiartotheEMASteamofevaluatorsafterthemeeting)thatherSubDirectorateintendedto
adoptmostofEMASreferralindicators.Butsheadmittedthatthisneedstobeendorsedby
manyupperlayersintheDirectorateofHealthReferral.EMASCOPshouldapproachMOH
centraltoacceleratetheprocess.

AdditionalObjective3:StrengthenedAccountabilityamongstGovernment,thecommunityand
healthsystemhasbeenaddedinPhase2.Andtwooutofthethreeoutcomesindicatorsin
PMPhavebeenachieved:
- PercentageofEMASsupporteddistrictswithVanguardPokjas(WorkingGroups)(PMP
#21):Pokjashavebeenestablishedinall10PhaseIdistrictsbytheendofYear2,and
havebeenabletoadvocateforincreasedfundingallocationsformaternalandnewborn
activities(includingforMPAsandmentoringactivitieswithindistricts)inthe2014
revenueandspendingbudget.ThesePokjasofPhase1(Malang,Sidoarjo,Bandung,
Cirebon,Banyumas,Tegal,andAsahan)haveallbeenhelpingtomentorandestablish
PokjasofPhase2;
- PercentageofEMASsupporteddistrictswithVanguardCivicForums(PMP#22):
CivicforumshavebeenestablishedinallPhase1districts(n=10).CivicForumactivities
includecollaborationwithDHO,communitygroupsandprofessionalorganizationsto
helpmonitorservices,organizebloodbanksandhelpcoordinateMCHMotivator
activities.

Twoindicators(SIGAPKU)andCitizenFeedback(complaints/suggestionsreceivedthrough
citizenfeedbackmechanisms,documentedandresolvedbylocalgovernmentsorpublicservice
deliveryunits)aretoosoontobeevaluated.Thesemechanismsneedtobereconsideredand
discussedfurtherinexpertsoftheuseofsocialmedia.

13

Appendix3ReferencesandDocumentsReviewed

References(Introduction)

AbouZahrC,2003:SafeMotherhood:abriefhistoryoftheglobalmovement19472002.Br
MedBull67:1325.

AustraliaIndonesiaPartnershipforMaternalandNeonatalHealth(AIPMNH),2008:Surveyof
MidwivesinThreeDistrictsinNusaTenggaraTimur,2007:Number,Characteristicsand
WorkPatterns.

BacheletM,2012:WomensLeadership.
http://www.unwomen.org/en/news/stories/2012/12/womenareintegralpartof
indonesiansuccess/#sthash.PPjQ2WpD.JHKWcpQj.dpuf.

BadanPusatStatistik(BPS),NationalFamilyPlanningCoordinatingBoard,MinistryofHealth
Indonesia,MacroInternational:IndonesiaDemographicandHealthSurvey1991.Jakarta,
Indonesia,Jakarta,Indonesia,CentralBureauofStatistics,1992Oct.,1992.

BadanPusatStatistik(BPS)andORCMacro.IndonesiaDemographicandHealthSurvey2002
2003.Jakarta,Indonesia,2003http://www.measuredhs.com/pubs/pdf/FR147/FR147.pdf.

BadanPusatStatistik(BPS)andMacroInternational:IndonesiaDemographicandHealthSurvey
2007.Jakarta,Indonesia,
2008http://www.measuredhs.com/pubs/pdf/FR218/FR218[27August2010].pdf.

BadanPusatStatistik(BPS),NationalPopulationandFamilyPlanningBoard(BKKBN),
InternationalKementerianKesehatan(KemenkesMOH)ICF:IndonesiaDemographicand
HealthSurvey2012.Jakarta,Indonesia,
2013http://www.measuredhs.com/pubs/pdf/FR275/FR275.pdf.

DAmbruosoL,AchadiE,AdisasmitaA,IzatiY,MakowieckaK,HusseinJ.2009:Assessingquality
ofcareprovidedbyIndonesianvillagemidwiveswithaconfidentialenquiry.Midwifery25,
528539

EnsorT,NadjibM,QuayyumZ,MegrainiA.2008:Publicfundingforcommunitybasedskilled
deliverycareinIndonesia:towhatextentarethepoorbenefiting?EurJHealEcon9:385
92.

HeywoodPandHarahapN.2009:HumanresourcesforhealthatthedistrictlevelinIndonesia:
thesmokeandmirrorsofdecentralization.HumanResourcesforHealth,7(6).

14

HortK,AkhtarA,TrisnantoroL,DewiS,MelialaA.2011:TheGrowthofNonStateHospitalsin
Indonesia:Implicationsforpolicyandregulatoryoptions.Heal.PolicyHeal.Financ.Knowl.
HubWork.Pap.Ser.NossalInst.Glob.Heal.;12.

IBIIndonesianMidwivesAssociation.2012:InfoKegiantanIBI.MonArchOct2012.

KoblinskyM,QomariyahSN.2014:Increasingaccessforthepoortofacilitybasedbirthin
Indonesia.USAIDDraftReport.Washington,DC

MinistryofNationalDevelopmentPlanning/NationalDevelopmentPlanningAgency
(BAPPENAS),2010:TheRoadmaptoAccelerateAchievementoftheMDGsinIndonesia.
Jakarta,Indonesia.

NASandAIPA,2013:ReducingMaternalandNeonatalMortalityinIndonesia:SavingLives,Savingthe
Future.WashingtonDC:TheNationalAcademyofSciences

Pujiyanto.Sakit,PemiskinandanMDGs.In:ThabranyH,ed.Jakarta,Indonesia,Kompas,2009.

Risfaskes.2011:MoH,Healthfacilitysurvey.Jakarta,Indonesia.

RokxC,SchieberG,HarimurtiP,TandonA,SomanathanA.2010:HealthFinancinginIndonesia:
AReformRoadMap.WashingtonD.C.,WorldBank,.

SatriyoHA.2008.10yearsofReformasi:TowardswomensequalstatusinIndonesia.Asia2008;
AsiaFoundation.http://asiafoundation.org/inasia/2008/05/28/10yearsofreformasi
towardswomen%E2%80%99sequalstatusinindonesia/.

WorldHealthOrganization(WHO),2006:MakingPregnancySaferTechnicalUpdateon
PreventionofPostpartumHaemorrhage,viewed20April2014,
<http://www.who.int/making_pregnancy_safer/publications/PPH_TechUpdate2.pdf>.

WorldHealthOrganization,2014:Trendsinmaternalmortality:1990to2013.EstimatesbyWHO,
UNICEF,UNFPA,TheWorldBank,andtheUNPopulationDivision.Geneva:WHO

WorldBank,2010:...andthenshedied"Indonesiamaternalhealthassessment.Washington,
DChttp://documents.worldbank.org/curated/en/2010/02/12023273/diedindonesia
maternalhealthassessment

WorldBank,2012:Worlddevelopmentreport2012:Genderequalityanddevelopment.
WashingtonDChttp://documents.worldbank.org/curated/en/2011/01/15156082/world
developmentreport2012genderequalitydevelopment

WorldBank,2014:UniversalMaternalHealthCoverage?AssessingReadinessofPublicHealth
FacilitiestoProvideMaternalHealthCareinIndonesia.WashingtonDC.WorldBank

15

OtherDocumentsReviewed

BadanPusatStatistik(BPS)andORCMacro,2003,IndonesiaDemographicandHealthSurvey
20022003,Calverton,Maryland,USA:BPSandORCMacro.

BadanPusatStatistik(BPS)andORCMacro,2012,IndonesiaDemographicandHealthSurvey
20022003,Calverton,Maryland,USA:BPSandORCMacro.

ESDforUSAID2010,CreatingaModelforEmergencyObstetricandNewbornCare:ScalingUp
BestPracticesinIndonesia,ExtendingServiceDeliveryproject,WashingtonDC,USA.

ESDforUSAID,2010,ReducingMaternalandNeonatalMortalityintheDistrictHospitalthrough
theBestPracticesImplementationPackage(ComprehensiveEmergencyObstetricsandNeonatal
Care):TheAssociationofIndonesianHealthProfessionalTrainingUnits,Jakarta.

Hermiyanti,Srietal.,2006,Indonesia&ItsMotivations:AMTSLfor70,000Midwives,Jakarta.

Kosen,S,2011,Jampersal(maternityinsurance)asasteptowardsuniversalcoverageandhealth
equity:experienceofIndonesia,presentedatthe4thTechnicalReviewandPlanningMeeting
fortheHealthPolicyandHealthFinanceKnowledgeHub,1011October,Melbourne,Australia

IndonesiaDemographicandHealthSurvey,2012,MaternalandNeonatalCare,Ministryof
Health,Jakarta.

EMASandUSAID,2012,AnnualWorkplanforExpandingMaternalandNeonatalSurvival
(EMAS)FiscalYear2012,ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2012,ProjectYear2012AnnualReport,ExpandingMaternalandNeonatal
Survivalproject,Jakarta.

EMASandUSAID,2012,ReportBaselineAssessmentReferralSystemsinEMASProgram
Districts,ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2012,SituationalAnalysisofSocialInsuranceAvailability,Regulations,and
Utilizationin6Provinces,ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2013,DocumentationForEMASPerformanceManagementPlanRevision,
revised6December2013,ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2013,EMASMonitoringandEvaluationUpdateYear3Quarter1,Expanding
MaternalandNeonatalSurvivalproject,Jakarta.

16

EMASandUSAID,2013,EMASMonitoringandEvaluationUpdateYear2Quarter4,Expanding
MaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2013,EMASPerformanceManagement,revised6December2013,Expanding
MaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2013,ProjectYearTwo2013AnnualReport,ExpandingMaternaland
NeonatalSurvivalproject,Jakarta

EMASandUSAID,2013,SOPPengumpulanDataIndikatorProgramEMAS,ExpandingMaternal
andNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2013,Year2WorkplanforExpandingMaternalandNeonatalSurvival
(EMAS),ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2014,EMASProgramLearning,ExpandingMaternalandNeonatalSurvival
project,Jakarta.

EMASandUSAID,2014,ExecutiveSummary:EMASYearTwoResultsandAchievementsReport,
ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

EMASandUSAID,2014,EMASProgramLearningMatrix,ExpandingMaternalandNeonatal
Survivalproject,Jakarta

EMASandUSAID,2014,PerformanceManagementPlanIndicatorReferenceSheets,Expanding
MaternalandNeonatalSurvivalproject,Jakarta

EMASandUSAID,2014,Year3WorkplanforExpandingMaternalandNeonatalSurvival
(EMAS),ExpandingMaternalandNeonatalSurvivalproject,Jakarta.

Qomariyah,S.N.,E.S.Pambudi,R.Zahara,I.Sahputra,T.Anggondowati,A.Achadi,M.Levy,P.
Deviany,andN.Parmawaty,2008,ReviewofDistrictHealthInformationSystems(DHIS)inDeli
Serdang(NorthSumatra)andSumedang(WestJava)Districts.Jakarta:CenterforFamily
Welfare,UniversityofIndonesiathroughtheHealthServicesProgram(USAID).

POPPHIforUSAID,2006,ManagementoftheThirdStageofLabor:Dataobtainedfromhome
deliveriesintheCirebondistrict,AugustSeptember2006,WashingtonDC,USA.
[http://www.path.org/publications/files/MCHN_popphi_amtsl_rpt_cirebon.pdf]

Williams,B,2014,USAID/IndonesiaHealthOfficeProgramManagementAssessmentfor
USAID/IndonesiasassistancetoJHPIEGOandConsurtiumforExpandingMaternalandNeonatal
Program(EMAS),DalarInternationalConsultancy,NorthCarolina,USA

17

WorldHealthOrganization(WHO),2006,MakingPregnancySaferTechnicalUpdateon
PreventionofPostpartumHaemorrhage,viewed20April2014,
<http://www.who.int/making_pregnancy_safer/publications/PPH_TechUpdate2.pdf>.

WorldBank,2013,Theproduction,distribution,andperformanceofphysicians,nursesand
midwivesinIndonesia:anupdate.Jakarta:WorldBank,December.

18

Appendix4ListofIndividualsandAgenciesContacted,7April14May2014

# Name Institution Position/Title

JAKARTA
1 Dr.Oscar MinistryofHealth SubDirectorateInformationand
CommunicationTechnology
(PUSDATIN)
2 Dr.AmnurKayo MinistryofHealth SubDirectorateInformationand
CommunicationTechnology
(PUSDATIN)
3 DrRudy MinistryofHealth SubDirectorateInformationand
CommunicationTechnology
(PUSDATIN)
4 DrBoga MinistryofHealth HeadofSubDirectorate
InformationandCommunication
Technology(PUSDATIN)
5 Dr.DeddyTedjasukmana MinistryofHealth HeadofSubDirectorateSupport
ServicesandHealthFacility(BUK
Penunjang)
6 Sodikin MinistryofHealth SubDirectorateSupportServices
andHealthFacility(BUKPenunjang)
7 Dr.AnungSugihantono MinistryofHealth DirectorateGeneralofG/KIA
8 Dr.GitaMaya MinistryofHealth DirectorofMaternalHealth
9 Dr.JaneSupardi MinistryofHealth DirectorofChildHealth
10 Dr.KartiniRustandi MinistryofHealth DirectorBasicHealthServices(BUK
Dasar)
11 Dr.RinaSpA IDAI IkatanDokterAnakIndonesia(IDAI)
12 Dr.Nurdadi POGIJakarta PerkumpulanObsteridan
GinekologiIndonesia
13 Dr.Muhamad BudiKemuliaan Director,LKBK
BaharuddinSpOGMARS
14 Dr.Retno BudiKemuliaan Staff
15 Dr.Fatchiati BudiKemuliaan Staff
16 Dr.FahrulArbi,SpA BudiKemuliaan Staff
17 Dr.DwiraniAmelia, BudiKemuliaan TeamLeader,Quality
SpOG Improvement,Maternal
18 Dr.EkaNursiati,MARS BudiKemuliaan Staff
19 Dr.IrmaSapriani,SpA BudiKemuliaan Staff
20 AgusRahmanto,SKM, BudiKemuliaan Staff
MARS
21 Dr.CutVirollina BudiKemuliaan Staff
22 WildanSaleh,SE BudiKemuliaan Staff
19

 23 ARahmanMuttaqin Muhammadiyah ProgramDevelopmentOfficer

24 SitiMasyitahRahma Muhammadiyah SeniorProgramManager
25 Dr.SudibyoMarkus Muhammadiyah GovernanceAdvisor
26 DwiDwiyuliantinaSari Muhammadiyah FinanceOfficer
27 AnneHyre EMAS(JakartaOffice) ChiefofPartyofEMAS
28 KristinaGrear EMAS(JakartaOffice) SeniorOperationsand
CommunicationsAdvisor
29 MiaPesik EMAS(JakartaOffice) SeniorProgramManager
30 Dr.EvodiaA.Iswandi EMAS(JakartaOffice) SeniorImpactatScaleStrategic
Advisor
31 MayaTholandi EMAS(JakartaOffice) SeniorStrategicInformation
Advisor
32 ItaYusdarita EMAS(JakartaOffice) ProgramManager
33 HartonoRakiman EMAS(JakartaOffice) CommunicationsManager
34 CindyRakhmawati EMAS(JakartaOffice) CommunicationsOfficer
35 CutSofaKumala EMAS(JakartaOffice) ProgramCoordinator
36 Dr.Trisnawati EMAS(JakartaOffice) TeamLeaderReferralquality
Gandawijaya
37 AliZazri EMAS(JakartaOffice) M&EDirector
38 Dr.WilsonWang EMAS(Savethe TeamLeader,Quality
Children) Improvement,Newborn
39 PanchoHekageriAK EMAS(Savethe NewbornAdvisor
Children)
40 BambangWijayanto RTI ICTAdvisor
41 Nuwirman EMAS(JakartaOffice) GovernmentAdvisor
42 EkoPrasetyo Jhpiego ICT4Dofficer
43 Dr.MasseeBateman USAID HealthUnit
44 IreneKoek USAID HealthUnit
45 RachelCintron USAID HealthUnit
46 MildredPantouw(Millie) USAID HealthUnit,AOTR
47 RiaWardhani USAID HealthUnit
48 AmeliaGinting USAID HealthUnit
49 Dr.RatnaKurniawati USAID HealthUnit
50 Dr.EdieRachmat USAID HealthUnit

EASTJAVA
51 Dr.WasisNupiksoSpOG RSUDSidoarjo Head,Sie.MedicalServices
Inpatient
52 Dr.SetyoBudi RSUDSidoarjo Coordinator,MentoringTeam
PamungkasSpOG
53 Dr.PramudyoDwiputro RSUDSidoarjo Head,PeristiRSUD
20

 SpOG
54 Dr.TjuputPurwastono RSUDSidoarjo Head,PONEKRSUD
SpOG
55 EkoHariWidarto RSUDSidoarjo Head,ERNursing
56 MamikSetyoIndrayani RSUDSidoarjo Head,BirthDeliveryRoom
57 SitiYunaria RSUDSidoarjo Head,Perinatology
58 UmyNurjayah RSUDSidoarjo Midwife,MNEMatenal&Neonatal
Emergency
59 VivinAuliawati RSSitiKhodijah Head,BirthDeliveryRoom
60 SutikWinarsih RSSitiKhodijah Head,Perinatology
61 Dr.HetyPuspitaningrum PuskesmasTaman Doctor
62 NurulTri PuskesmasTaman CoordinatorMidwife(Bidan
Koordinator)
63 EndahRetno DHOKab.Sidoarjo Staff,FamilyHealth
64 Dr.EndangDamayanti DHOProv.Jatim Sekretary,DHO
65 Dr.HerlinFerliana,M. DHOProv.Jatim Head,HealthServices
Kes
66 Dr.NunikDhamayanti DHOProv.Jatim Head,Sie,ReferralandSpecial
67 Avianto DHOProv.Jatim Staff,FamilyHealth
68 Dr.SriUtami DHOProv.Jatim Staff,ReferralandSpecial
69 Dr.HeraPrasetia RSUDJombang Head,MedicalServices
70 Dr.Kuspardani DHOKab.Blitar Head,DHO
71 Dr.EndahWoro RSUDNgudiWaluyo DeputyDirector,HealthServices
Blitar
72 SitiAfridaS.Kep,Ners RSUDKanjuruhan Head,Nursing
Malang
73 drg.LoembiniPedjati DHOKab.Pasuruan Head,DHO
Lajoeng
74 Dr.ArmaRosalina RSUDBangilPasuruan Head,MedicalServices
75 Dr.SetyoBudi RSUDSidoarjo Coordinator,MentoringTeam,
PamungkasSpOG Sidoarjo
76 Dr.TjuputPurwastono RSUDSidoarjo Head,PONEKRSUDSidoarjo
SpOG
77 Dr.HeriWibowo,M.Kes DHOKab.Jombang Head,DHO
78 IdaNikmatulUlfa DHOKab.Jombang Head,Sie,FamilyHealth
79 Dr.Iskandar DHOKab.Jombang Head,PSDK
80 LulukNurKholisah DHOKab.Jombang Staff,MCH
81 Dr.Asnan PuskesmasCukir Head,Puskesmas
82 Dr.Widi RSUDPlosoJombang Director,RSUDPlosoJombang
83 Dr.M.Darusalam RSMuhammadiyah Director,RSMuhammadiyah
Jombang Jombang
21

 84 Dr.Andri PuskesmasBareng Head,Puskesmas

85 Dr.Hexawan PuskesmasTapen Head,Puskesmas
86 Dr.Nanik PuskesmasBandar Head,Puskesmas
KedungMulyo
87 Dr.Ismindari PuskesmasTembelang Head,Puskesmas
88 Dr.PudjiUmbaran,MKP RSUDJombang Director,RSUDJombang
89 Dr.M.Mahfudz,Sp.PD RSUDJombang DeputyDirector,HealthServices
90 Dr.HeraPrasetia,Sp.BS RSUDJombang Head,MedicalServicesandNursing
91 Dr.UlfahKhannatul RSUDJombang Head,MedicalQualityControl
Izzah,MKP
92 DenokEkoY, RSUDJombang Head,Sie,NursingQualityControl
Amd.Keb.,S.Sos.,MM.Kes
93 Dr.PudjiUmbaran,MKP RSUDJombang Director,RSUDJombang
94 Dr.M.Mahfudz,Sp.PD RSUDJombang DeputyDirector,HealthServices
95 Dr.Subur,Sp.OG RSUDJombang Dr.Obgyn
96 Dr.AdiNugroho,Sp.OG RSUDJombang Dr.Obgyn
97 Dr.RizalFitni,Sp.OG RSUDJombang Dr.Obgyn
98 Dr.RahadiHamsya, RSUDJombang Head,ERCentral
Sp.An
99 Dr.HusnuRaji'in,Sp.An RSUDJombang Staff,MedicalAnaesthesia
10 Dr.RetnoWulandari, RSUDJombang Pediatrician
0 SpA
10 Dr.TriPutriY,Sp.PK RSUDJombang Head,ClinicalPathology
1
10 NenyNurmiwati, RSUDJombang Head,PharmaInstallation
2 S.Si.,MSc.,Apt
10 SlametDjoko,S.Kep.,Ns RSUDJombang Head,ERCentral
3
10 RahayuA,Amd.Keb RSUDJombang Head,PONEK
4
10 AniKuncoro RSUDJombang DeputyHead,NeonatalRoom
5
10 AnikMasrifah RSUDJombang Head,ER
6
10 Rahmi,Amd.Kep RSUDJombang Head,IBS
7
10 Suhariyono RSUDJombang NurseAnesthesia,ORER
8
10 AdiFatkhurR,Amd.PK RSUDJombang Staff,MedicalRecord
9
11 Wahyu,Amd.Keb RSUDJombang Midwife
0
22

 11 ErikaDianRatri,SH RSUDJombang Staff,MedicalServicesandNursing

1
11 ErniaRosita,S.Kep.Ns RSUDJombang Staff,MedicalServicesandNursing
2
11 Tamam,SH RSUDJombang Staff,PublicRelations
3
11 IndahHeraD,Amd.Kep RSUDJombang Staff,Neonates
4
11 Arifin RSUDJombang Staff,PONEK
5
11 Dr.Puspitasari RSUDJombang Head,KSMGeneralPractitioners
6 (GPs)
11 Dr.DewiNugrahini RSUDJombang ERDoctor
7
11 TommyS FMMPeduliKIA MemberCivicusForum
8
11 Armie FMMPeduliKIA TreasurerCivicusForum
9
12 WahyuningAsri FMMPeduliKIA Secretary,CivicusForum
0
12 MitraOktafistaA FMMPeduliKIA MemberCivicusForum
1
12 TophanTohary FMMPeduliKIA MemberCivicusForum
2
12 Yati FMMPeduliKIA MemberCivicusForum
3
12 Dr.Suparman RSMuslimat Director
4
12 Dr.Teja RSMuslimat SpOG
5
12 Dr.H. RSMuslimat Pediatrician
6
12 Dr.Bagus RSMuslimat Doctor
7
12 DewiErma RSMuslimat Nurse
8
12 Solichan RSMuslimat Nurse
9
13 Sukma RSMuslimat Nurse
0
13 AkmalFitri RSMuslimat Nurse
1
23

 13 UmiR RSMuslimat Nurse

2
13 Suryono RSMuslimat Head,OperatingRoom
3
13 TriA RSMuslimat Nurse
4
13 Soelistiyorini RSMuslimat Nurse
5
13 Titik RSMuslimat Midwife
6
13 Dr.AndriSuharyono PuskesmasBareng Head,Puskesmas
7
13 Dr.SriRahayu PuskesmasBareng Doctor
8
13 Dr.SonniEkoW PuskesmasBareng Doctor
9
14 Sutami PuskesmasBareng MidwifePONED
0
14 Putuyah PuskesmasBareng MidwifeCoordinatorPONEK
1
14 UlfaIda PuskesmasBareng MidwifeKoordinator
2
14 UmiMaslihah PuskesmasBareng VillageMidwifeTebel
3
14 WiviSetyaningrum PuskesmasBareng VillageMidwifeB.Agung
4
14 DinaYunitap.w PuskesmasBareng VillageMidwifePulosari
5
14 EkaN PuskesmasBareng VillageMidwifeNgampungan
6
14 ElisSusanti PuskesmasBareng VillageMidwifeNgrimbi
7
14 S.TitikParwati PuskesmasBareng CoordinatorER
8
14 NinikIsmiatinNingsih PuskesmasBareng NurseER
9
15 Syamsiah PuskesmasBareng MidwifeMCH
0
15 KiswiyatiWahyuni PuskesmasBareng MidwifeMCH
1
15 IrinSuprihatin PuskesmasBareng MidwifeMCH
2
24

 15 MungkyN.R PuskesmasBareng MidwifePONED

3
15 Dini PuskesmasBareng MidwifePONED
4
15 Vivin PuskesmasBareng MidwifePONED
5
15 IzaTriRahmawati PuskesmasBareng VillageMidwifePakel
6

NOTHSUMATERA
15 Dr.JeniusL.Tobing, RSUdr.Pringadi Head,HealthCommittee
7 SpOG Medan
15 Dr.Syamsul,SpOG RSUdr.Pringadi Head,ObGynDept.
8 Medan
15 Dr.Syamsul,SpAn RSUdr.Pringadi Head,AnesthesiaDept.
9 Medan
16 Dr.BerlianaHasibuan, RSUdr.Pringadi Head,PediatricDept.
0 SpA Medan
16 BidanFauziah RSUdr.Pringadi Head,BirthDeliveryRoom
1 Medan
16 BidanRusmawati RSUdr.Pringadi Head,BirthDeliveryRoom
2 Medan
16 BidanElizatuti RSUdr.Pringadi Head,ER
3 Medan
16 RisdaNadeak RSUdr.Pringadi StaffPerinatology
4 Medan
16 Endang RSUdr.Pringadi Staff,Preeclampsia
5 Medan
16 SitiAisyah RSUdr.Pringadi Head,ObstetricGyn.
6 Medan
16 Dr.Risma RSUdr.Pringadi MedicalCommittee
7 Medan
16 Dr.Riza RSUdr.Pringadi Secretariat,ObGyn
8 Medan
16 SyamsudinAngkat RSUPH.AdamMalik Director,RSUPAdamMalik
9
17 Dr.Fahdy,SpOG RSUPH.AdamMalik Head,QualityCommittee
0
17 Dr.Hanudse,SpOG RSUPH.AdamMalik SMFObGyn
1
17 Dr.Yudha,SpOG RSUPH.AdamMalik SMFObGyn
2
25

 17 Dr.Ferawahyuni,SpA RSUPH.AdamMalik SMFPediatric

3
17 Dr.Andriamuri,SpAN RSUPH.AdamMalik SMFAnesthesia
4
17 Bd.Sumiariani RSUPH.AdamMalik Head,ObstetricDept.
5
17 BdRosmahasa RSUPH.AdamMalik Head,ER
6
17 Rehulina RSUPH.AdamMalik Head,Perinatology
7
17 Masrida RSUPH.AdamMalik Head,OperatingRoom
8
17 AsniAngkat RSUPH.AdamMalik StaffPPI
9
18 Dr.Rizkiharahap RSUPH.AdamMalik Head,Nursing
0
18 Dr.MarasiSibarani,SE PERSI ViceChairmanPERSINorthSumatra
1 Province
18 Drs.AfwanApt PHONorthSumatra Secretary,PHO
2
18 Dr.AzwanNasution PHONorthSumatra Coordinator,MaternalandChild
3 Services
18 RosidahBerutu,M.Kes PHONorthSumatra Head,PrimaryHealthCare
4
18 Dr.IrmaNasution PHONorthSumatra Staff,PrimaryHealthCareReferral
5
18 Dr.MasroelSiregar, PHONorthSumatra VicechairmanofJointHealth
6 MPH Council
18 CholiqulKamal,SKM PHONorthSumatra MemberofJointHealthCouncil
7
18 Dra.EliSuhaeriyah PHONorthSumatra HeadofAPTELDiskominfo
8
18 DediIrawan PHONorthSumatra StaffDISKOMINDO
9
19 MarioKasduri MuhammadiyahNorth PWMSUMUT
0 Sumatera
19 Elynita MuhammadiyahNorth PWASUMUT
1 Sumatera
19 Azwinar MuhammadiyahNorth KomiteEMAS/Majelis
2 Sumatera Kesehatab'Aisyiyah
19 PartaonanHarahap MuhammadiyahNorth ERStaff,RSMuhammadiyah
3 Sumatera
26

 19 Dr.HerlinaSembiring PKMTalunKenas Head,PuskesmasTalunKenas

4 KabupatenDeli
Serdang
19 Dr.EvaR.Pinem PKMTalunKenas Doctor,PuskesmasTalunKenas
5 KabupatenDeli
Serdang
19 RugunSianipar PKMTalunKenas Coord.Midwife(BidanKoordinator)
6 KabupatenDeli PKMTalunKenas
Serdang
19 HieronimusMeliala RSSembiring Head,Adm.,RSSembiring
7
19 Sarmana RSSembiring Head,NursingRSSembiring
8
19 Dr.HartatyAgnes DHOKab.DeliSerdang Head,HealthServices
9 Saragih
20 ElmiHaryuniSKM,MKes DHOKab.DeliSerdang Head,BidangKesehatanKeluarga
0
20 SriRejekiSKM DHOKab.DeliSerdang TeamICTEMASDeliSerdang
1
20 drg.ReshkiJonian RSUDLubukPakam Director,RSUDLubukPakam
2
20 Rosmawaty RSUDLubukPakam Head,Nursing
3
20 Dr.Evi RSUDLubukPakam DeputyDirector,MedicalServices
4
20 MarthaBarus RSUDLubukPakam Head,Perinatology
5
20 Heni RSUDLubukPakam Head,Obstetric
6
20 Rasmi RSUDLubukPakam Head,ER
7
20 Dr.RatnaTanjung PKMArasKabu Head,PuskesmasArasKabu
8
20 Dr.HennyAndrianie PKMArasKabu Doctor,PuskesmasArasKabu
9
21 RohanaSimarmata PKMArasKabu BidanKoordinatorPKMArasKabu
0
21 NensiHerlina PKMArasKabu CoordinatorVKPKMArasKabu
1
21 Deliana PKMArasKabu MCHMotivator,CivicForumDeli
2 Serdang

27

 SOUTHSULAWESI
21 Dr.Nurdin EMAS TeamLeader,SouthSulawesi
3 Province
21 Dr.Mapatoba, PHOSouthSulawesi Head,HealthServices
4
21 Dr.Syamsurizal RSWahidin Head,MedicalServices
5 Sudirohusodo
21 Dr.EffendiSpOG RSWahidin Head,ObstetricCare
6 Sudirohusodo
21 Dr.HadiyahSpA IDAIS.Sulawesi Secretary
7
21 Dr.BobWahyudiSpA IDAIS.Sulawesi Member
8
21 Profdr.DazrilDaudSpA IDAIS.Sulawesi Head,PediatricRSWS
9
22 AriefSetiadi BPJSMakasar Head,InformationandTechnology
0 Dept
22 Ali BPJSMakasar Head,BPJSMakassar
1
22 Burhanuddin RSSitiKhadijah Coordinator
2
22 Hedijusumah RSSitiKhadijah Head/OwnerofRSIA
3
22 Yulanti RSSitiKhadijah DeputyDirector
4
22 JhIsmanDahlan RSSitiKhadijah DeputyDirector
5
22 Asawait CivicForum Head,PKKWatansawito
6
22 Fatimah CivicForumAisyiyah SecretaryofForum
7
22 Amrullah CivicForum Muhammadiyah
8
22 NurfoyriAliah BPJSRSLansinrang Staff
9
23 Dr.Asma DHOPinrang
0
23 DrNuryanti DHOPinrang Head,HealthServices
1
23 Dr.NurhidiFauzi DHOPinrang FundingandPharmacyDiv
2
23 Dr.Aswar DHOPinrang CommunityServices
28

 3
23 Dr.RamliYunus LampaPuskesmas Head,Puskesmas
4
23 AugustinaAmKeb LampaPuskesmas Midwife,Puskesmas
5
23 Khadijah LampaPuskesmas VillageMidwife
6
23 Dr.Sriyanti RSLasinrang DeputyDirector,PublicServices
7

29

Appendix6EvaluationTeamcomposition

1.TeamLeader:AlfredBartlett,MD,FAAP(Captain,USPublicHealthService,retired;formerSenior
AdvisorforChildSurvival,USAID/Washington,andformerDirectorSavingNewbornLivesProgram,Save
theChildren/US)

2.USAID/WashingtonMaternalExpertise:MarjorieKoblinsky,PhD(SeniorMaternalHealthAdvisor,
USAID/Washington;formerSeniorAdvisorforWomensHealth,JohnSnowInc.,andformerDirector,
PublicHealthSciencesDivision,InternationalCentreforDiarrhoealDiseaseResearch,Bangladesh
(ICDDRB)

3.IndonesianMNHPolicyandHealthSystemAdvisor:BrotoWasisto,MD(MemberandChairmanof
theOversightCommittee,CountryCoordinatingMechanism(CCM)oftheGlobalFundIndonesia,and
ViceChair,ConsortiumofHealthCareServices,MOH)

4.IndonesianMNHEvaluationSpecialist:MeiwitaBudiharsana,PhD(TeachingFacultyDepartment
ofBiostatisticsandPopulation,FacultyofPublicHealth,UniversityofIndonesia,Jakarta,Indonesia;
formerCountryDirectorandSeniorAssociate,PopulationCouncilVietNam(Hanoi),andformer
CountryRepresentative,TheFordFoundation,Jakarta,Indonesia)

30

Appendix 5

USAID EMAS Mid Term Evaluation

Schedule of Activities
April 7 May 9, 2014

Week 1
CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

Preparation: reading documents
56 April 2014 Anywhere
etc
USAID Meeting
Room 810, 17/F, Gedung
Monday, Amelia Ginting,
Jakarta 08.30 17.00 Team Planning Meeting Sarana Jaya USAID and MTE team
April 7, 2014 Telp 02134359485
Jl. Budi Kemulian I/1
Jakarta 10110
MTE team
Irene Koek
Rachel Cintron
USAID Meeting Amelia Ginting,
09.0012.00 Team planning meeting Massee Bateman
Room 1710 Telp 02134359485
Mildred Pantouw
Ratna Kurniawati

Dr Anung
Sugihantono (Head of
Tuesday, April 8, GKIA)
Jakarta
2014 Dr Gita Maya
(Director Maternal
JH Luwansa Hotel
Health) Ratna Kurniawati
Jl. HR Rasuna Said
12.3014.30 Meeting with Ministry of Health Dr Kartini Rustandi MOH
Kav.C.22
(Dir. Basic Health
Jakarta 12940
Services)
Dr Dedi Teja Sukmana
(Dir. Medical Services
for Human Resources
and Equipment)
Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

Dr Erna Muladi (Head
Div. Child Health)
Dr Nida, staff medical
services
USAID

EMAS Secretariat office,

14.30 17.00 Team planning meeting continue MTE Team MTE Team
MOH fl.9th

Agenda Dev. by team

Wednesday, April
Jakarta 9.00 13.00 Team planning meeting leader and the team MTE Team MTE Team
9, 2014

Borobudur Hotel MTE Team
13.00 16.00 Meeting with EMAS Anne Hyre
EMAS Leadership Team
EMAS Office,
16/F,Tempo Scan
Thursday MTE Team
Jakarta 09.0017.00 Meeting with EMAS Building Anne Hyre
April 10, 2014 EMAS Team
Jl. Rasuna Said Kav.10 11
Kuningan, Jakarta Selatan
MTE Team
Dr Jane Supardi
(Director of Child
Health)
Dr Eni , Sub
Friday Meeting with Ministry of Health directorate neonatal
Ratna Kurniawati
April 11, 2014 Jakarta 08.00 12.00 (Child and Maternal department ) MOH Office health

Dr Laveli Desi (Head
of Div.
standardization
neonatal health)
Dr Meli (staff)

Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

EMAS Office,
Meeting with EMAS 16/F,Tempo Scan
MTE Team
13.0017.00 (Monitoring and Data Building Anne Hyre
EMAS Team
Management; Program Learning) Jl. Rasuna Said Kav.10 11
Kuningan, Jakarta Selatan
MTE Team
RSB Budi Kemuliaan
Dr Muhammad
Sat April 12, 2014 Jakarta 09.0012.00 Visit Budi Kemuliaan Hospital Jl. Budi Kemuliaan No.25 Budi Kemuliaan
Baharuddin and Budi
Jakarta Pusat
Kemuliaan staff
Sunday April 13,

2014
MTE Team
EMAS Secretariat office,
Monday Jakarta 09.00 12.00 Interviews POGI; other (TBD) Dr. Nurdadi Saleh Anne Hyre
April MOH fl.9th
April (head of POGI)
14
14, Field Work and Data Analysis
17,2014 Field visit to MTE Team
2014 15.0517.30 Interviews, site visits, and analysis Medan EMAS Province
Medan EMAS province
of comparative performance data
Irene Koek (Health
10.00 13.00 Director)
Friday April 18, MTE Team
Jakarta USAID Leadership meeting Hotel Borobudur
2014 Rachel Cintron
13.3016.00 (Deputy Director)
MTE Team
Saturday, April Possible

19, 2014 meetings

Sunday April 20,

2014
Ministry of Health MTE Team
Meeting with Director of Dr. M. Adhyatma Building, Dr Oscar Primadi
Monday
Jakarta 08.30 Information Centre (PUSDATIN) 2/F, A Building, R. 207, Jl. Dr Amnur Kayo Ibu Ratna Kurniawati
April 21, 2014
HR. Rasuna Said, Blok X5, Dr Rudy
Kav. 49, Jakarta Dr Boga (Head of Sub
Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

ROOM 614 Directorate
Information and
Communication
Technology)
Ministry of Health
Dr. M. Adhyatma Building, MTE Team
Meeting with Director of Support
2/F, A Building, R. 207, Jl. Dr Deddy
13.00 Services and Health facility (BUK Ibu Ratna Kurniawati
HR. Rasuna Said, Blok X5, Tedjasukmana Basuni
Penunjang)
Kav. 49, Jakarta Bapak Sodikin
ROOM 517
MTE Team
Sekai Restaurant
Dr Anung
Working Dinner with Dirjen Jl. Suryo No. 30, Blok S
19.00 Sugihantoro, Ibu Ratna Kurniawati
G/KIA Senopati Jakarta
Dr Gita Maya (Dir.
Selatan
Maternal health)
Ministry of Health
Dr. M. Adhyatma Building,
MTE Team
Meeting with Director Basic 2/F, A Building, R. 207, Jl.
07.3008.30 Dr Kartini Rustandi Ibu Ratna Kurniawati
Health Services (BUK Dasar) HR. Rasuna Said, Blok X5,
and staff
Kav. 49, Jakarta
ROOM 509
RSCM
Gedung PJT (Pelayanan
Tuesday Jakarta Jantung Terpadu)
MTE Team
April 22, 2014 09.30 Meeting with IDAI Perinatologi Div. 3th
Dr Rina Roshsiswanto
floor
April 2225, 2014 Jl. Diponegoro No.71
Salemba Jakarta
Gedung Dakwah
Bapak Sudibyo
Muhammadiyah
13.0015.00 Meeting with Muhammadiah Markus
Jalan Menteng Raya No.
MTE Team
62 Jakarta
MTE Team
Field Work and Data Analysis
Field visit Makassar Ibu Marge and Bapak
15.00 Interviews, site visits, and analysis EMAS Province
Team 1 (Pinrang District) Broto
of comparative performance data
EMAS Province
Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

MTE Team
Field Work and Data Analysis
Field visit Bapak Alfred and
15.00 Interviews, site visits, and analysis East Java EMAS Province
Team 2 Ibu Meiwita
of comparative performance data
EMAS Province
Saturday

April 26, 2014
Initial synthesis Incountry team
work culminating in delivery of
Sunday
Detailed Evaluation Report
April 27, 2014
Outline and draft PowerPoint
Jakarta
presentation for review by
Monday
Evaluation Committee. Additional
April 28, 2014
meetings and interviews may also
be scheduled to validate findings.
Tuesday

April 29, 2014

Revision and refinement In

response to comments from
Evaluation Committee, team will
Wednesday incorporate feedback and other
April 30May 5, Jakarta input into finalized PowerPoint
2014 presentation and initial full report
draft. Presentation to
USAID/Indonesia and other
stakeholders.
USAID Meeting
Room 1610,
Discuss Prefinding with USAID USAID MCH Team and
April 30 Jakarta 09.0012.00 16/F, Gedung Sarana Jaya
and MTE team only Health office Director
Jl. Budi Kemulian I/1
Jakarta 10110

MTE Team
Discuss prefinding with MTE
13.0014.00 Front Office USAID USAID
Team, USAID, MCH,
MCH
Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

USAID
Discussion on preliminary finding MTE Team
14.3016.00 USAID Office
and recommendation EMAS (Kristina and
Maya)

USAID Meeting
Room 1410, USAID Team
Amelia Ginting,
9.00 10.30 Further discussion 14/F, Gedung Sarana Jaya MTE team
Telp 02134359485
Jl. Budi Kemulian I/1
Jakarta 10110

May 2, 2014 Jakarta

Discussion on preliminary finding USAID Amelia Ginting,
10.3012.00 Front Office
and recommendation MTE team Telp 02134359485

Discussion on preliminary
USAID
findings and recommendation,
13.0016.00 MTE Team
and preparation of Ministry of
EMAS
Health debrief
MTE presentation for USAID and
EMAS Meeting with EMAS and
EMAS Office,
Consortium members (Save the
16/F,Tempo Scan EMAS Team
Children, LKBK, Muhammadiyah,
May 5, 2014 Jakarta 08.3011.30 Building MTE Team Anne Hyre
EMAS staff, USAID John Rogosh<
Jl. Rasuna Said Kav.10 11 USAID
Irene K, Massee Bateman: Finding
Kuningan, Jakarta Selatan
and recommendation

Min.ofSocial Welfare MTE Team
Meeting with Coord. Ministry of
Jl. Medan Merdeka Ministry of welfare,
May 7, 2014 Jakarta 09.00 Social Welfare (Health and FP Ratna Kurniawati
Barat No.3 Jakarta Deputy staff
MOH and BKKBN)
Pusat USAID
Appendix 5

CONTACT
VENUE
DATE TIME ACTIVITIES ATTENDEES PERSON
(ADDRESS)

Ministry of Health
MTE Team
Meeting with the General Dr. M. Adhyatma Building,
EMAS team
May 14, 2014 Jakarta 10.0011.00 Secretary of the Ministry of 2/F, A Building, R. 207, Jl. Ratna Kurniawati
USAID
Health, and . HR. Rasuna Said, Blok X5,
Kav. 49, Jakarta Secretary Jendral