“EMAN is our enterprise monitoring tool that monitors all devices on the network,” said Anthony Garcia, IT AVVID Engineer. “It checks for heartbeat and every 15 seconds, it pings the device to make to sure it's still online. If one of the systems fails to respond within a pre-set number of pings, the Tier 2 Support staff will be automatically paged.”
EMAN collects availability statistics from both the Cisco CallManager and from any IP addressable adjunct equipment, including the TFTP server, the DHCP server, and the Digital gate-way or Analog gategate-way devices.
However, EMAN does more than just monitor the IP telephony infrastructure. The Call Detail Record (CDR) provides informa-tion on who is calling whom, from what extension the call is being made, whether the call went through, and the voice quality of the call.
“My team is the development organization that works closely with the Business Unit to influence product strategy and develop tools that will enhance Cisco’s product solutions,” said Patricia Justusson, EMAN IT Infrastructure Senior Manager.
“Monitoring, availability, and alert services are just one of the functional areas we are responsible for. We also develop products
that help lower infrastructure costs and improve operational sys-tems support—including Change Management, Disaster Recovery tools, SLA tools, and a variety of others that help manage avail-ability and improve efficiency.”
Justusson’s team is looking ahead to enhancing the support process. “What’s really exciting is how we can auto provision these solutions,” Justusson said. Currently, CallManagers are sep-arate clusters located regionally around the world. Every Cisco site around the globe has its own set of people responsible for managing those clusters. “We’re looking at how to scale Operating Support System (OSS) cost over clusters and we’re working on an appliance-size model where we can lock down the servers—going down the stack—and standardize on one box.”
The new tool will also host an application called the Telephony Member Management (TMM) that will track and manage all of Cisco’s phone numbers enterprise-wide.” This will enable every-one to use every-one tool to manage all of their phevery-one numbers,”
Justusson said. “For example, the tool will track phone number availability and when people move or leave, it will automatically relinquish that number to the pool of re-usable extensions. And it is a granular entitlement—it gives people control only over their own set of phone numbers.”
4–11 Monitoring Tools
Microsoft Performance Monitor®
Performance Monitor is a Windows 2000 application that displays the activities and status of Cisco CallManager system and reports both general and specific information in real time. “It takes certain values on the server, such as the heartbeat, system uptime, the number of phones and gateways, memory and processor load on the machine, etc.,” Garcia said. “It also monitors the number of calls in progress at any time and the number of calls currently passing through a specific gateway. We want to make sure that if there's any problem with the IP phone service, we know about it as soon as the first user is aware of it.”
Performance Monitor collects and displays system and device statistics for any Cisco CallManager installation and administrative tool and enables a full understanding of a system without studying the operation of each of its components. After adding the Cisco CallManager parameters, the support team defined the terms under which Cisco CallManager displays statistics generated by the system. Performance Monitor shows both general and Cisco CallManager-specific status information in real-time.
Microsoft Event Viewer®
Microsoft Event Viewer is a Windows NT Server application that displays system, security, and application events (including Cisco CallManager) for the Windows NT Server. “Most of the time, we use that tool after the fact to see what the root cause of the problem was,” Garcia said. “When there's a problem, it is listed in the Event Viewer. For example, if you tried to log into our server and you didn't have authorization, it would give you three tries and then kick you out. I could look in Event Viewer to see what happened in the last hour and it would show me that your user name tried to log in unsuccessfully three times.”
CallManager®Trace
CallManager traces are local log files. The IP address, TCP handle, device name, or the time stamp can be used when reviewing the CallManager trace to monitor the occurrence or the disposition of a request. “Within the CallManager is a database that stores traces of everything that happens in the system,” Garcia said. “Whenever there is a problem, we look through these traces that include CCM traces, SDL traces, CTI traces, etc.”
Sniffer®Trace
A sniffer is a software application that monitors IP traffic on a network and provides information—such as quantity and type of network traffic—in the form of a trace. Sniffer traces can also help identify high levels of broadcast traffic that could result in voice audio problems or dropped calls. “For example, say we're having problems with our PacBell circuit number one,” said Dennis Silva, Voice Services Senior Network Design and Engineer. “We’ll get a sniffer on it and see what's happening and do some test calls over it. It analyzes at the packet level and we get the results immediately.”
Troubleshooting
Cisco’s manual, “Operating the IP Telephony Network”, addresses some common problem categories that may occur with Cisco CallManager and related devices. Each problem category suggests troubleshooting tools that can be used to help isolate and troubleshoot problems. Although not an exhaustive list, some of the more com-mon problem categories are presented below:
T h e C i s c o I P Te l e p h o n y C a s e S t u d y
Problem Category Description
Voice Quality Lost or distorted audio during phone calls.
Phone Resets Phones will power cycle or reset due to one of the following:
• TCP failure connecting to Cisco CallManager, or
• Failure to receive an acknowledgement to the phone's KeepAlive messages.
Dropped Calls Occur when a call is prematurely terminated.
Cisco CM Feature Issues Caused by configuration errors or a lack of resources: these issues may include Conference Bridge or Media Termination Point.
Slow Server Response Mismatched duples, screen savers (that consume all of the CPU when active), and third party software.
Reorder Tone Through Occurs when users try to make a restricted call, call a number that Gateways has been blocked, call a number that is out of service, or if the
PSTN has an equipment or service problem.
Gatekeeper Registration Problems Sometimes occurs when IP connectivity is lost within the network
“When the GTRC encounters a problem they cannot resolve using the troubleshooting tools and utilities, they call us for assistance,”
said Mike Telang, Manager, Network Operations. “They provide us with CallManager Administration Details, plus any diagnostic infor-mation that they were able to gather, and we take it from there.”