Beruflich Dokumente
Kultur Dokumente
Maintenance Guide
Product Document
Document Approvals: Version # 1.0 Date 02/Sep/2011 Name Leon Visa D.P.M
The following table lists the contents of this chapter. Section 2.1 Equipment Room Environment 2.2 System Environment 2.3 System Configuration 2.4System Running Describes The indexes of environment The round inspection standards of the system environment. The round inspection standards of the running platform. The round inspection standards of the system running.
Round inspection of the equipment room environment: Content Power/Status air-conditioners Temperature Humidity Cleanness Antistatic environment Space Power supply Acceptance Standard The air-conditioners in the equipment room run normally. The temperature ranges from 15 to 30 C C. The humidity range is 40%65%. The concentration of dusts with diameter 4 3 more than 5mm is less than 310 /m . The equipment room uses the antistatic raised floor and the floor is grounded. The space is enough and the equipment room is well ventilated. At least two channels of independent power supplies provide power for equipment. Check Method After turning on the air-conditioners in the equipment room, check whether they can run continuously and normally. Check the temperature with a thermometer. Check the humidity with a hygrometer. Check whether the raised floor is connected with the ground cable. Check whether the ventilation equipment runs normally. Check whether two channels of power supplies are connected to the rack or subrack.
Round inspection of the system environment: Content Running status of system fans CPU and memory status Hard disk status and file system Acceptance Standard All fans of the host and disk chassis run normally. Air exhaust is normal. The temperature on the equipment surface is room temperature and is not hot. The number of CPUs, Memory,hard disks are same as configured. All components run normally. Check Method Observe the air exhaust status of equipment air outlets and touch equipment surfaces. Running the following commands on the servers as the root user: # free -o # df h # cat /proc/cpuinfo Running the following command to check the versions of the OS # cat /etc/redhat-release
Versions of the OS
OS version
Round inspection standards of the system configuration: Content Software version Acceptance Standard SWEngine V3.X Check Method Running the following command # /etc/init.d/swengine version Floating IP One floating IP addresses must be allocated: Running the following command : # vi /etc/hosts # ifconfig /etc/hosts file 127.0.0.1 localhost Adjust the addresses according to the current system & network architecture. Check the /etc/hosts file. :
Round inspection standards of the system running : Content File system Space Acceptance Standard The usage of each file system does not exceed 75%. The usage of the memory swap area must be less than 90%. No abnormity prompts such as warning, error, fail and panic, and no prompts of system process restart in the log. No abnormity prompts such as warning, error, fail and panic in MAIL. Check Method Log in to each sever, and running the following command: # df -h Memory Swap Usage System Log Running the following command as the root user: # top | grep Swap Running the following command as the root user: # vi /var/log/messages Check the mails of the last week, running the following command as the root user: # mail Times of the active and standby systems The time difference of the two-node cluster does not exceed 1 minute. Running the commands to view the time of the active and standby systems: # date
Unable to Login to Web Interface Unable to access via Telephone Interface (TUI) remotely Messaging System not working, customers hear greeting but no message can be left. SMS Notifications not working Unable to receive Fax Calls are not routing to Messaging System Call Messaging System but no prompt heard or blank deposit message Cannot navigate into Messaging System Menus Cannot enter secret code Call notification dont work Callback dont work Call screening dont work
No calls going to Messaging System at all Call Notification dont work Callback dont work CSV statistics not received
L1
No User try by short Number at home? User try by his own Number out of home?
No
Yes
Yes
END
Yes Check files: /var/log/swmessaging/ums_va.log And /var/log/swengine/sigserver.log On Voice Servers, for checking if the call arrive on voice servers
Yes
Check if RTP work between voice server and user. Make a tcpdump on voice server.
3 1 L2 2 L2
5 L2 4 L2
L2
Yes
4
Escalation to STREAMWIDE
END
No User can navigate in Vmail? No User can enter secret code?
Yes
Check if DTMF correctly receive by voice servers. Check files: /var/log/swmessaging/ ums_va.log And /var/log/swengine/sigserver.log On Voice Servers
Yes
END
END
No
Yes
Check /etc/swengine/ sigserver.cf to know if voice server is DTMF INBAND and RFC2833 or RFC2833 only.
Escalation to STREAMWIDE
END
6 L3
No
Yes
No MEDIABOXRECORDSTATE is set to Y?
Yes
Escalation to STREAMWIDE
Yes
END
END
Look on voice server in /var/log/swmessaging/ums_va.log for the mediabox if you have a line with: Sending sms notification
No
Check the network between voice server and Kannel server and check Kannel server No This parameter is set to Y Network and Kannel server work? No Yes
Yes
END
Check in DB. In MEDIABOX table: SMSNOTIFICATIONCOUNTMONTH needs to be lower than PROFILE table ALLOWEDSMSNBMONTH
Yes
Escalation to STREAMWIDE
END
END
END
L1
Look in /var/log/swengine/sigserver.log on voice server if you have for this call an invite with m=image 11386 udptl t38
Voice Gateway dont make Invite with T38. Check Voice Gateway
Look in /var/log/swmessaging/ums_va.log on voice server if you have a line with: Fax Not Allowed For MediaBox
END
No Do you have this line? Look in /var/log/swmessaging/ums_va.log on voice server if you have a line with: pages received Escalation to STREAMWIDE No Do you have this line? Yes
Yes
END
1 L2
2 L2
L2
No FAXSTATE set to Y
Yes No Look PROFILE table of this Mediabox in DB if ALLOWEDFAXSTATE is set to Y Look PROFILE table of this Mediabox in DB if ALLOWEDFAXSTATE is set to N Do you have this file?
Yes
Escalation to STREAMWIDE
END
No ALLOWEDFAXSTATE set to Y? ALLOWEDFAXSTATE set to N? No
END
Yes
Yes
Escalation to STREAMWIDE
This Profile allow receive FAX but FAX is deactivated on this Mediabox. Try to change FAXSTATE to Y.
END
END
END
L1
Check in /var/log/swmessaging/ums_va.log if you have line looks like Key pressed: X for you call
Check in /var/log/swmessaging/ums_va.log if you have line looks like AddToQueue /home/swmessaging/Sounds/en/pr ompts/ for you call
No
Check in /var/log/swengine/sigserver.log if you have a line looks like received dtmf X from mediaserver for your call
Yes
Escalation to STREAMWIDE
1 L2
2 L3
END
L2
Make a new call and make in the same time a tcpdump -i ethx -s 1500 -w debug.cap (where x is RTP interface number)
RFC2833
InBand
Yes
No
END
Escalation to STREAMWIDE
END
END
L3
Check in folder seen in /var/log/ swmessaging/ums_va.log the sound file size with command: ls -l filename
Yes
Escalation to STREAMWIDE
END
END
Beware Message need to be in status new, if email notification with attachment activated with archive you never have call notification.
L1
Check in /var/log/swmessaging/ ums_va.log if you have a line looks like Call postponed for 2010-12-08 15:14:00 for your call
Check in /var/log/swmessaging/ ums_va.log if you have a line looks like BEGIN of CALL NOTIFICATION for PhpPid = 9264 : Checking msg where PhpPid = previous incoming call identifying
No
1 L2
2 L3
END
L2
No
Change Allow Call Notification to Yes Check in DB for this mediabox if CALLNOTIFICATIONCOUNTMONTH <= ALLOWEDCALLNOTIFICATIONNBMONTH defined in Profile
3 L1
Yes
Escalation to STREAMWIDE
This Mediabox have deplete Call Notification quota for this month.
END
END
L3
Check in /var/log/swengine/sigserver.log if you have a line looks like STA Outgoing SIP call from: MediaboxNum to CallNotifNum callid:493D2BAA00000038@192.168.1.6:5060 ip:outputgatewayIP:5060
Escalation to STREAMWIDE
Yes
END
END
L1
No
Yes
END
No
No
2 L2
Check /var/log/swengine/ sigserver.log on Voice server. Search Invite message for this Call.
No Do you have Invite message? Yes Check /var/log/swmessaging/ ums_va.log on Voice server. Search line like The Mediabox nnnnnnnn does not exist in VMS for this call. Check Voice Gateway.
END
1 L3
L2
No
END
Check /var/log/swmessaging/ ums_va.log on Voice server. Search line like: AddToQueue /storage/ MediaBox/nn/nn/nn/nn/ inviteprimary.al for this call
Yes
Check /var/log/swengine/ sigserver.log on Voice server. Search Invite message for this call.
Check in MEDIABOX folder if you have this file? No No Do you have this file? Yes Invite customer to put a secondary greeting or go back to default primary. Retry a test call. Do you have invite message?
Escalation to STREAMWIDE
Check if RTP work between voice server and user. Make a tcpdump on voice server.
3 L4
No
Escalation to STREAMWIDE
END
L4
3
No
Escalation to STREAMWIDE
END
L1
Check in /var/log/swmessaging/ ums_va.log if you have a line looks like Call postponed for 2010-12-08 15:14:00 for your call
Check in /var/log/swmessaging/ ums_va.log if you have a line looks like BEGIN of CALL NOTIFICATION for PhpPid = 9264 : Checking msg where PhpPid = previous incoming call identifying
No
1 L2
2 L3
END
L2
No
Change Allow Call Notification to Yes Check in DB for this mediabox if CALLNOTIFICATIONCOUNTMONTH <= ALLOWEDCALLNOTIFICATIONNBMONTH defined in Profile
3 L1
Yes
Escalation to STREAMWIDE
This Mediabox have deplete Call Notification quota for this month.
END
END
L3
Check in /var/log/swengine/sigserver.log if you have a line looks like STA Outgoing SIP call from: MediaboxNum to CallNotifNum callid:493D2BAA00000038@192.168.1.6:5060 ip:outputgatewayIP:5060
Escalation to STREAMWIDE
Yes
END
END
This chapter describe how to handle Emergency faults for Streamwide Messaging System. Definition of Emergency faults Requirements for Emergency Maintenance Personnel General Emergency Maintenance Process
Contact the Streamwide Support Centre for technical support at the earliest when an emergency fault occurs. (see Section 3 page 16)
4.2.1.
Emergency Faults that might occurs on the Streamwide Messaging System are : - Rack Electrical Power Problem : Messaging system is located in dedicated Racks that are connected to 2 independent power sources. And all Messaging servers are connected to theses 2 independent power sources. If an electrical fault occurs, Messaging System might be affected and become no more available - Network Problem causing Messaging System unreachable : Messaging system is connected to the Customer IP network through Ethernet switches. If theses Ethernet switches fails down, Messaging system might be unreachable. - Swengine service is interrupted : Swengine service is the core software of the Messaging system. If Swengine process is interrupted or down, Messaging System will not be able to provide messaging services. - Database service is interrupted : Database service is used to store all subscribers settings and all call informations. If the Database process is interrupted or down, Messaging System will not be able to provide messaging services. - Web service is interrupted : Web services are used for provisioning the Messaging system but also to provide a Web interfaces. If the web process is interrupted or down, Messaging system will not be able to be provisioned and subscribers will not be able to access their web user interface. - SMS service is interrupted : SMS service is used to send SMS notifications to subscribers when a message is left in their mailbox. If SMS service is interrupted, no SMS notifications will be sent. - Email service is interrupted : Email service is used to send EMAIL notifications to subscribers when a message is left in their mailbox. If EMAIL service is interrupted, no EMAIL notifications will be sent. - MWI service is interrupted : MWI service is used to send Message Waiting Indicator to subscribers when a message is left in their mailbox. If MWI service is interrupted, no MWI notifications will be sent.
The Emergency maintenance personnel should have the Streamwide Messaging System Training courses : Messaging System : Introduction & Operation Messaging System : Configuration, Use & Maintenance
The Emergency maintenance personnel should know how to perform Remote Acccess to the Messaging System.
The following flow chart describes the process of handling fault related to Electrical Power problem :
No Electrical Power on Messaging System Contact Streamwide Support Center
Each Power Cord of Messaging System is connected to the Rack PDUs ? Yes
No
No
Verify that ALL components of Messaging System are UP and reachable (ping test) ? No
Yes
No
Verify that Ethernet switches are UP & Running? Yes Swengine or Database services interrupted (see 25.3 / 2.5.4)
Yes
Connect all Ethernet to the appropriate port of Ethernet switches. (Follow the network cabling & design of Messaging System)
No
Send Swengine logs (sigserver.log, mediaserver.log) to Streamwide for Analysing the cause
No
Connect to database and run a test request : SQL> select count(*) from PROFILE
Send Voice Application logs (vmsva.log) to Streamwide for Analysing the cause
Check if Apache process is alive and running ? Yes Check if Network Pb Connect to Web interface
No
No
Restart EM client daemon Process. Run the command : service sweventmanagerclient-daemon restart
No
No
No
Send Mail Application logs (maillog.log) to Streamwide for Analysing the cause
Check if Network Pb
Yes
No
Send MWI Application logs (ums_mwi.log) to Streamwide for Analysing the cause
Email Phone
support@Streamwide.com : +33 6 33 60 33 60
The support center may be reached during standard business days and standard business hours (Monday to Friday, 9:30 am to 5:30 pm GMT+1:00) NOTE: Outside standard business hours (nights, week ends, national holidays), an emergency support team is still available. Customer should use the same email address and phone number as previously mentioned.
4.5.1.
Trouble ticket
A support technician creates a trouble ticket and launches simultaneously a Support Task for the technical team responsible for handling issues. To ensure the proper creation of the trouble ticket, the Customer should provide the following information: Customers Corporate Name Detailed description of the encountered issue When issue has been experienced (Date and Time) Name, phone and e-mail for the primary point of contact
4.5.2.
Once this information has been provided, a Support Technician issues a Trouble Ticket Number to the Customer. This number will be used to track a customer issue. NOTE: This number ensures that the issue is correctly tracked and correctly addressed. It is reserved for the Customer. Under no circumstances, it should be given to any end-user. Customer should provide this ticket number every time it would contact the support center. Equally, it is very important to NOT distribute the emergency contact information to end-users as well in order to not disturb the standard process of issue resolving.
4.5.4.
Issue closure
When an issue is closed and require no further action, the technical team informs the customer that its product or service is up and running. When the Customer acknowledges this information, the technical team closes the support task. The technical team sends an email including the causes of the impairment, the encountered downtime as well as the repair duration.
4.5.5.
A priority-based system has been set up. It is intended to classify support calls in order to resolve Customer issues in a timely and efficient manner. The table shown below is an example of how issues are classified, depending of the type of problem encountered.
Example of issues * Unable to leave a message into a significant number of mailboxes (>10% of the total number of mailboxes, or more than 20000 mailboxes) * Unable to leave a message into a significant number of mailboxes (<10% of the total number of mailboxes and not less than 500 mailboxes) * Failure of a redundant module (except hardware failure) * Usability issues
Major
Minor
This issue classification might change based on a contract between StreamWIDE and the Customer. If there is a disagreement about an issue initial classification, it may be modified according to the Customers will. However, it will be asked to the Customer to provide the precise reasons for making this change once the issue has been solved. In addition, an issue classification may as well be modified while it is being addressed. If an issue is wrongly classified in a frequent manner, the Customer and StreamWIDE can meet and define a new way of classifying issues. If, despite such modifications, wrong classifications still occur, Customer and StreamWIDE can define a penalty process.
4.5.7.
Escalation levels
Levels of escalation are defined according to the duration of the repair process and the classification of the issue. Times and dates are recorded and start when the Customer contacts the support team.
Type of Issue >>> Critical issue >> Major issue > Minor issue
Escalation
Service
Phone Number
During Business hours:
Level 1
Support team
+33 1 70 08 52 64 or +33 1 70 08 52 62
Outside Business hours:
support@streamwide.com
+33 6 33 60 33 60 Level 2 Level 3 Technical Manager Zakaria Nadhir General Manager Lilian Gaichies +33 6 82 99 47 53 +33 6 82 99 47 90 znadhir@streamwide.com lgaichies@streamwide.com
4.5.8.
Continual Improvement
To reduce number of troubles and to increase quality of the technical support service, StreamWide sets up a Quality Improvement Process. It implies to plan quality meetings on a regular basis. These meetings are useful to discuss about dysfunctions which have been reported and to suggest preventive actions to avoid the recurrence, with help of StreamWide Quality Manager. Besides, to evaluate technical support effectiveness and to maintain improvement of the Technical Support Process, indicators and quality objectives are established and reviewed each year during the management reviews meetings.
StreamWIDE All rights reserved Author: Wenbo CAI SW_UMS302_MaintenanceGuide10 Date: 30/Aug/2011 50/51
5. Glossary
API DNS Application Program Interface Domain Name Server
DTMF Dual Tone Multi-Frequency HTTP Hyper Text Transfer Protocol IAD IVR LAN LED MWI NAS NFS NTP PHP Integrated Access Device Interactive Voice Response Local Area Network Light Emitting Diode Message Waiting Indicator Network Attached Network Network File system Network Time Protocol Hypertext Preprocessor
PSTN Public switched Telephone Network RPM RTP SAN SIP SMDI SMS originally Red Hat Package Manager Real Time Protocol Storage Area Network Session Initiation Protocol SCSI Musical Data Interchange Short Message Service
SNMP Simple Network Management Protocol UMS VMS Voip XML Unified Messaging System Voicemail System Voice Over Internet Protocol Extensible Markup Language