8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

SOLUTIONS
SolutionsOverview
UnstructuredDataGrowth
MultiVendorHybridCloud

SearchVeritas

Healthcare

Government

HowtotroubleshootNDMPBackups
failureswhenstatuscode99(orother

NDMPbackupfailure)isreported.Includes
English
Site:Veritas

logginginstructions.

PRODUCTS
SERVICES
CUSTOMERCENTER
ABOUT

Article: 000081335
Publish: October22,2015
ArticleURL: http://www.veritas.com/docs/000081335

SIGNIN
Username
Password
Rememberme

ForgotPassword?

SUBMIT

Don'thaveaVeritasAccount?
CreateaVeritasAccountnow!

PrintArticle
Products
https://www.veritas.com/support/en_US/article.TECH56492

1/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

RelatedArticles
ArticleLanguages
SubscribetothisArticle
ManageyourSubscriptions

Problem
HowtotroubleshootNetBackupforNDMPBackupfailureswhenstatuscode:99(NDMPbackup
failure)isreported.Includeslogginginstructions.
ThisarticlecanbeusedformostNDMPstatuscodes.

ErrorMessage
STATUSCODE:99

Cause
Status99inNDMPbackupsisgeneric.Furthertroubleshootingisrequired.

Solution
Duringregular(standard)NDMPbackups,avoidpotentialNDMPcommunicationsfailuresbetween
NetBackupmediaserversandtheNetworkAttachedStorage(NAS)host.
ThefollowingtroubleshootingstepsmayhelpisolatetherootcauseofNDMPbackupissues.If
theydonot,proceedwithlogginginstructionsinStep9.
NOTE:TerminologyforthetypesofNDMPbackupscanbeconfusing:

https://www.veritas.com/support/en_US/article.TECH56492

2/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

RemoteNDMP:Thebackupisperformedonthemediaserver,usingthedevicepaths
definedtothemediaserver.
Local:Thebackupisperformedusingthedevicepathsattached/zonedtotheNAShost.
3Way:ThebackupofaNAShostisperformedusingthedevicepathsattached/zonedto
anotherNAShost

ToisolatetherootcauseofaNDMPbackupfailure:
1.Onthemediaserver,testtheconnectiontotheNASdeviceontheNDMPport(10000)bytyping
thetelnetcommand.TryboththehostnameanditsIPaddress.Forexample:telnetndmp_host
10000
2.TotestconnectivityandcredentialsoftheNDMPhostanditsdevices:
Typetpautoconfverify[NAShostname]
Typetpautoconfprobe[NAShostname]toprobethescsibusfordevices(localor3Way
only).
Onanothermasterormediaserver,typetheabovetwocommandstocheckforanother
routeornetworkpath.
ComparetheproberesultswiththenativeNAScommands:
NetApp:sysconfigt
EMCCelerra:server_devconfig[datamovername]lsn(list,scsi,non
disk)
Other:Seesystemdocumentation
3.OntheNAS,andtypethecommandndmpdstatustoverifythattheNDMPdaemonisrunning.
Ifitisnot,enablethendmpdaemonandverifyagainbytypingthecommandndmpdstatus.For
informationonstartingthendmpdaemon,seetheNASsystemdocumentation.
4.ForEMCCelerradevices,ensurethattheSnapSurefeatureisenabled.SeeTECH170310for
informationonSnapSureandotherconsiderations.
5.Onthemediaormasterserver,typethetelnetandtpautoconfcommandtestsagainfromStep
1totestconnectivity,credentials,anddevices.
6.IfLocalor3Waybackup,dothefollowingtoverifythatthevolumecanbeaccessedand
backedup:
First,mountascratchtapeinthedriveusingrobtestorsomeothermeansandperforma
backup.
OntheNASdevice,typetheappropriatenativecommand:

https://www.veritas.com/support/en_US/article.TECH56492

3/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

NetworkAppliance(NetApp):dump0ufnrst0a/<filesystem>(Substitutethepropernon
rewinddevicepathifitisnotnrst0aasinthisexample).
Note:ForClustermodeNetApp,refertotheNetAppDocument'ClusteredDataONTAP8.2
CommandMapfor7ModeAdministrators'asthedumpcommanddoesnotworkforCMode
volumes.
Excerpt:
UnderstandingthedifferentclusteredDataONTAPshellsforCLIcommands
AclusterhasthreedifferentshellsforCLIcommands:
Theclustershellisthenativeshell,startedautomaticallywhenyoulogintothecluster.
Itprovidesallthecommandsyouneedtoconfigureandmanagethecluster.
Thenodeshellisaspecialshellthatletsyourunasubsetof7Modecommands.
Thesecommandstakeeffectonlyatthenodelevel.Youcanswitchfromtheclustershelltoa
nodeshellsessiontorunnodeshellcommandsinteractively,oryoucanrunasinglenodeshell
commandfromtheclustershell.Youwillknowacommandisanodeshellcommandifithasthe
(long)form:
systemnoderunnode{nodename|local}commandname
ThiscommandwillwritedatafromthespecifiedfilesystemontheNetworkApplianceserver
directlytotape.Ifthisdumpdoesnotcompletesuccessfully,thentheproblemismostlikelya
NetWorkApplianceproblem.Toresolvetheissue,openacallwithNetworkAppliancetechnical
support.Ifthedumpdoescompletesuccessfully,openacallwithNetBackupsupport.
ForEMCCelerra,refertotheserver_archivecommandinthesystemdocumentation.
NOTE:TheBackupPathspecifiedintheNetBackuppolicymustbetheexactcasesensitivepath
asthefilerseesit.
7.Forperformanceissues,trybackingupthepathinthepolicytoanulldeviceandnotethe
timing:
NetApp:dump0fnull/vol/volname
EMCCelerra:server_archive<datamovername>wf/dev/null/J<file
path>
8.Basedonyourfindings,proceedwithanyrequiredconfigurationchangesor,ifthereareno
changes,beginbackupsorrestores.
9.Ifnoneoftheabovetroublesootingstepsresolvetheissue,enabledebugloggingonthe
NetBackupmediaserverandontheNASdevice.

Toenabledebuglogging:

https://www.veritas.com/support/en_US/article.TECH56492

4/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

1.Dependingonyourmediaserverplatform,dothefollowing:
ForWindowsmediaserver:
OntheNetBackupconsole,gotoHostProperties>MediaServeranddoubleclickthe
mediaserverthatservicesthisbackuptoopenthePropertiesdialogbox.Ifthismedia
serveristhesameasthemasterserver,gotoMasterServerinstead.
GototheLoggingsectionandchangethegloballogginglevelto5(maximum).
ClickOKtoexit.
ForUNIXmediaserver:
AddVERBOSEtothe/usr/openv/netbackup/bp.conffileinthefollowingformat:VERBOSE=
5
2.Makesurethatthefollowinglegacylogdirectoriesexistonthemediaserverthatservicesthe
backup:
ForWindowsmediaserver:
\NetBackup\logs\bpbrm
\NetBackup\logs\bptm
\NetBackup\logs\ndmpagent
ForUnixmediaserver:
/usr/openv/netbackup/logs/bpbrm
/usr/openv/netbackup/logs/bptm
/usr/openv/logs/ndmpagent
3.Runthefollowingcommandsonthemediaserverthatservicesthisbackuptoincreasethe
unifiedlogginglevels:
ForWindowsmediaserver:
<installpath>\NetBackup\bin\vxlogcfgap51216o134sDebugLevel=6
sDiagnosticLevel=6
<installpath>\NetBackup\bin\vxlogcfgap51216o151sDebugLevel=6
sDiagnosticLevel=6
ForUnixmediaserver:
/usr/openv/netbackup/bin/vxlogcfgap51216o134sDebugLevel=6s
DiagnosticLevel=6
/usr/openv/netbackup/bin/vxlogcfgap51216o151sDebugLevel=6s
DiagnosticLevel=6
4.EnablendmpddebugloggingontheNASdevice:
https://www.veritas.com/support/en_US/article.TECH56492

5/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

ForNetworkApplianceNAS,typethefollowingtoenablendmpddebuglogging:ndmpd
debug70
NOTE:ForOnTap8.1andabove,thecommandsaredifferent.RefertoTECH178502for7
ModeorTECH218460forClusterMode(CMode).
OSversionofanyfilercanbedeterminedbyviewingtheoutputoftpautoconfverify[NAS
hostname]
Ifnecessary,furtherndmpdeventloggingcanbeenabled.Refertothe
ndmpd.backup.log.enablecommandinthisNetworkApplianceSupportarticle.
EMCCelerra:EnabledebugloggingonthedatamoverasdescribedinArticleTECH150646
ForotherNASdevices,seethevendordocumentation.
5.Reproducetheerror.
6.Disablelogging:
ForNetworkApplianceNAS,typethefollowingtodisablendmpddebuglogging:ndmpd
debug0
RefertotheNetApparticleslistedinStep4forOnTap8.1andabove.
EMCCelerra,seethedisablecommandsinTECH150646.
ForotherNASdevices,seethevendordocumentation.
7.Retrievethebpbrm,bptm,ndmpagentlogsfromtheNetBackupmediaserverfortoday'sdate.
8.DooneofthefollowingtoretrievelogsfromtheNASdevice:
Pullthedatamoverserver_logfromtheEMCCelerra:server_logserver_2as>
/output.file(assumesserver_2isthedatamovername)
Pullthesystemlog,backuplog,andthendmpdlogfromtheNetworkApplianceNAS:
/vol/etc/messages
/vol/etc/logs/backup
<volume>/etc/log/ndmpdlog(/vol/etc/log/mloginDataOnTap8.x)
Forremoteaccesstothefiles,trywindowsbrowser(\\Filer_Name\C$\etc\File_Name)
9.Typethecommandsonthemediaservertodisablelogging:
ForWindowsmediaserver:
<installpath>\NetBackup\bin\vxlogcfgrp51216o134sDebugLevel=1
sDiagnosticLevel=6
<installpath>\NetBackup\bin\vxlogcfgrp51216o151sDebugLevel=1
sDiagnosticLevel=6
ForUnixmediaserver:

https://www.veritas.com/support/en_US/article.TECH56492

6/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

/usr/openv/netbackup/bin/vxlogcfgrp51216o134sDebugLevel=1s
DiagnosticLevel=6
/usr/openv/netbackup/bin/vxlogcfgrp51216o151sDebugLevel=1s
DiagnosticLevel=6
10.Createatempdirectory.
Forexample:C:\temp\vxulor/tmp/vxul
11.Runthefollowingcommandtoretrieveallunifiedlogsfortoday'sdate:
ForWindowsmediaserver:<installpath>\NetBackup\bin\vxlogmgrct1f
c:\temp\vxul
ForUnixmedaserver:/usr/openv/netbackup/bin/vxlogmgrct1f/tmp/vxul
Wherethe't1'valuewillretrievetheunifiedlogsfromthelasthour.Adjustthevalueas
appropriatetogobackpastthebeginningofthejob.
12.Compressthetempdirectory(ZipforWindows,tarforUnix)andsendtoSymantecNetBackup
SupportalongwiththelegacylogsandtheNASlogs.
13.Tocreatealonglistingofthepolicy,runthefollowingcomandontheNetBackupmaster
server:
Windowsmasterserver:<installpath>\NetBackup\bin\admincmd\bppllist<name
ofpolicy>L
Unixmasterserver:/usr/openv/netbackup/bin/admincmd/bppllist<nameof
policy>L
14.Onthemediaserver,onthecommandline,runthenbsutoolasfollowstocreatetheoutputfile:
Windowsmediaserver:
Inacommandwindow,changethedirectoryto<installpath>\NetBackup\bin\support
Typethefollowingcommand:nbsuctnozip
Unixmediaserver:
Typethefollowingcommand:/usr/openv/netbackup/bin/support/nbsuct
Thescriptwillcreateanoutputfileandtellyouwhereitwasstored.
15.CutandpastetheJobDetailsforthejobintheActivityMonitorduringthebackupattempt.
SendthattoSymantecSupportalongwiththenbsuoutputfileandthebppllistoutput.
ErrorwhenaddingcredentialsforNDMPhost:
https://www.veritas.com/support/en_US/article.TECH56492

7/8

8/19/2016

HowtotroubleshootNDMPBackupsfailureswhenstatuscode99(orotherNDMPbackupfailure)isreported.Includeslogginginstructions.

Unabletovalidatethefilerwidecredentials.NDMPfailedtoverifyhost
(58).
EnsurethendmpuseridiscreatedwithMD5encryption.ForNetworkAppliance,itcanbe
switchedbetweenplaintextandMD5.Refertothisarticle.
ForEMC,useMD5.Searchforthedocument'AddinganNDMPuserintheNetBackupAdmin
Console'ontheEMCSupportwebsite(requiresanaccount).

TermsofuseforthisinformationarefoundinLegalNotices.
SEARCH

AskaQuestion

SURVEY
Didthisarticleansweryourquestionorresolveyourissue?
No

Yes

SUBMIT

Create/ManageCase

19952016VeritasTechnologiesLLC
Sitemap

LegalTerms

Privacy

Contact

https://www.veritas.com/support/en_US/article.TECH56492

8/8