HK1107461B - End-to-end test and diagnostic management system - Google Patents

Description

End-to-end test and diagnostic management system

Technical Field

The present invention relates to an end-to-end test and diagnostic manager for testing components and diagnosing service problems for services delivered via a layered multi-domain network environment.

Background

In the past, various telecommunications services, such as voice communications, data services, video services, etc., were delivered via independent vertical networks. For example, fig. 1 shows separate and distinct networks for delivering voice communication services and applications 10, data services and applications 12, and video services and applications 14. Each individual network includes a service layer 16, a control and switching layer 18, a transport hardware layer 20, and a physical network layer 22. In each case, the service is tightly connected to the physical network and network elements provided for distributing the service. For example, in the Public Switched Telephone Network (PSTN), the control and switching protocols and the physical switches themselves are geared towards creating end-to-end circuits for providing point-to-point voice communications. Similar private networks, such as ethernet, token ring, etc., with their own switching and control protocols and physical transport structures have been developed for providing data transfer services. Video services, broadcast or delivered via broadband cable, have likewise developed their own suite of control and switching protocols, transport hardware, and networks. In each case, all aspects of the various integrated networks are configured specifically for a particular service, where the network is designed to deliver the particular service.

Troubleshooting service problems in the past vertically distributed proprietary networks (prepractic networks) is a fairly straightforward proposition. Typically, the entire network is owned and controlled by a single entity familiar with the end-to-end service delivery chain. This fact, together with the fact that the service is tightly connected to the network elements provided for delivering the service, makes it quite simple to identify components in the service delivery chain that might affect the delivery of the service to a particular customer. Knowledge of the likely source of the service problem makes it fairly simple to test the appropriate components and isolate the source of the problem.

Today, the delivery of more and more telecommunication and broadcast services is focusing on IP as the preferred transport layer. Convergence over IP (convergence) separates the service from the underlying access and transport networks. The result is a multi-service, multi-domain network where the services themselves are substantially independent of the physical transport layer. Fig. 2 shows the convergence on IP as the preferred transport layer. In contrast to the proprietary vertically integrated networks of the past, the new service model defines a wide range of horizontal layers. Open protocols and APIs provide an operational interface between layers where access, transport, and services are clearly separated. Thus, service layer 30 represents any and all services that may be delivered digitally over a network, including, but not limited to, voice 24, data 26, and video 28. The transport and distribution layer (distribution layer)32 includes an IP layer 34 and a physical distribution layer 36. By using open protocols and APIs, all services can be encapsulated for transport over IP. IP encapsulated data can be carried over virtually any and all physical networking technologies.

In the convergence on IP as a common transport layer for delivering multiple services to customers, significant complexity is added to the service delivery chain. Today, Value Added Services (VAS) are based on complex network architectures and operating platforms. Part of the service delivery chain may be and likely is outside the service provider's own control. In these cases, service providers must rely on networks and hardware provided and maintained by other providers to deliver their services. The separation of services from the underlying access and transport system makes diagnosing and correcting service problems a much more difficult task than in the past stand-alone service delivery platforms.

To illustrate the complexity of today's service delivery platforms, consider the basic DSL service architecture 40 shown in fig. 3. On the left, an Asymmetric Digital Subscriber Line (ADSL)52 or a symmetric high-speed digital subscriber line (SHDL)54 provides digital connectivity to a customer's residence or business. The DSL connection provides access to a broader IP network or the internet via a Network Application Server (NAS) 42. The customer's DSL line (ADSL or SHDL) is connected to a first local DSL access multiplexer (DSLAM) 50. The local DSLAM 50 serves a limited number of customers in a smaller geographical area. The local DSLAM 50 is connected to a remote DSLAM 48 that serves a plurality of local DSLAMs. The remote DSLAM 48 is connected to the ATM network 45 via a first BPX switch 46. The NAS42 is connected to a second BPX switch 44 elsewhere in the ATM network 45. There is likely to be additional BPX switches in the ATM network 45. However, from the perspective of delivering internet access to customers at the end of the ADSL line 52 via NAS42, BPX switches 44 and 46 serve as ingress and egress points to ATM network 45, and any other BPX switches within the ATM network are not associated with a service delivery chain. DSL technology is currently evolving to introduce the possibility of: current and new services are delivered through different architectures using IP DSLAMs connected to GBE (gigabit ethernet) or MAN (metropolitan area network) via ethernet interfaces.

When Customer Premise Equipment (CPE) is considered, the service delivery chain becomes more complex. At the customer end, the DSL service depicted in fig. 3 requires at least a DSL modem. Local area networks, WI-FI routers and other CPEs can further complicate service delivery images (pictures). Fig. 4 shows a client configuration 60 that is not atypical. The ADSL line 62 enters the client via a network interface device 64. Splitter 66 separates the voice and data to connect the voice signal to a standard telephone 72 via a first wall jack 68 and the data signal to a DSL modem 74 via a second wall jack 70. A local area network hub 76 connects a plurality of computers 78, 80, 82 to the DSL modem 74. In this way, each computer can access the DSL line via the LAN. In addition to the network components described with respect to fig. 3, the DSL modem 74 and LAN HUB 76 must also be operational and appropriately configured in order for users at computers 78, 80, 82 to receive service over the DSL line 62. Thus, if these components do not work properly or are not configured properly, the CPE domain may provide another source of service problems.

As the complexity of the delivered services increases, the service delivery chain becomes even more complex. For example, fig. 5 illustrates a typical architecture for voice over internet (VoN) services. Tracking service issues to individual customers in such an environment would be a very complex task. It can be seen that VoN service requires interaction of multiple network domains (network domains) that include multiple network elements and service components. In addition, services are provided to multiple customers located at different locations, having different connections, and having different CPEs.

The VoN architecture 100 is divided between a VoN or service layer 102, a transport layer 104, and a CPE layer 106. Starting with the CPE layer 106, a number of different customer configurations are shown. Different client configurations will depend primarily on the type of access to the transport layer, available bandwidth, and other factors.

The client 108 includes a local area network 114, one or more VoN telephone devices 116, and a plurality of user computers 118. Customer Edge (Customer Edge)120 interfaces with a Provider Edge (Provider Edge) in transport layer 104 to provide access to Virtual Private Network (VPN) 140. VPN 140 is based on multi-protocol label switching (MPLS). VPN 140 accesses a broader IP network via NAS 154.

The client arrangement 110 includes a local area network 126 and a plurality of VoN telephone devices 122. Customer edge 124 provides an interface to IP edge network 142. IP edge network 142 provides access to a broader IP network 148.

The customer 112 includes a plurality of conventional telephony devices 128, an access gateway 130, and a customer edge 132. The access gateway 130 and the customer edge 132 interface with the DSLAM158 via digital subscriber lines.

Also shown in the customer layer 106 are a conventional PBX 134 and conventional telephone equipment 136. In the illustrated system, the PBX 134 and the conventional telephone device 136 may be engaged in a telephone call with a customer employing VoN service. However, the PBX 134 and telephone device 136 operate on a conventional public switched telephone network 146 and are not themselves affected by VoN services.

The transport layer 104 includes a VPN 140, an IP edge network 142, an ADSL network 144, a PSTN 146, an IP network 148, and a backbone network transit level (transit level)150 for traditional voice services and interface components between them. For example, VPN 140 interfaces with IP network 148 via NAS 154. The ADSL network interfaces with IP network 148 via NAS 156. The PSTN interfaces with the backbone network of a backbone transit level layer (BB-TL) 150. The IP network interfaces with the BB-TL network via a Media Gateway (MG) 162.

The VoN service layer 102 includes a user profile (user profile) database 172, a packet in application server 174 including service logic and user profile data, one or more SIP servers for interfacing with the PSTN 146 via the BB-TL 150, and the softswitch 166.

In this example, the service delivery chain includes all network elements involved in delivering VoN service to the end customer. For example, to deliver VoN service to customer 112, the service delivery chain includes Sip server and softswitch 170, switch 166, media gateway 162, NAS156, DSLAM158, and access gateway 130. Additional components within the various network domains, such as ATM switches, IP routers, etc., may also comprise part of the service delivery chain. All of these elements must function properly and be configured properly to provide VoN service to the customer 112. Of course, different networks and different network elements will be part of the service delivery chain that delivers VoN services to other customers. Identifying all network elements involved in delivering a service to a particular customer is an important task for troubleshooting service delivery problems. Performing tests on all different equipment including service delivery chains, and checking configuration data across all service delivery domains, is completely nearly impossible.

WO 00/72183 a2 discloses a method and apparatus for service level management in which a business process is composed of a plurality of services. The state of a service is defined by one or more service parameters, and these service parameters depend on the performance of the network components supporting the service, e.g., component parameters. The status of a service may depend, for example, on the collection of service parameter values for availability, reliability, security, integrity, and response time.

To date, there has been no system or program for employing a structured end-to-end approach to providing comprehensive end-to-end testing and diagnostics for services delivered via the IP transport layer. The lack of such systems and procedures significantly increases the time, effort, and expense of troubleshooting service delivery problems.

Disclosure of Invention

The present invention relates to an end-to-end test and diagnostic manager for troubleshooting service problems of services delivered via a layered multi-domain service delivery network across different technology platforms. In particular, one embodiment of the present invention provides an end-to-end test and diagnostic manager for services delivered using the Internet Protocol (IP) as a common transport layer. The end-to-end test and diagnostic manager provides a comprehensive solution for troubleshooting services delivered via the IP transport layer, such as VoN, IP-TV, video on demand, unified messaging, etc.

According to one embodiment, the end-to-end test and diagnostic manager includes a software application that allows front-end operators (e.g., call center customer service representatives) and back-end operators (e.g., more experienced operators that may be used to troubleshoot more complex service issues that the front-end operators cannot solve) to troubleshoot the entire chain of services that extends from the service provider across multiple network domains to the end customer. The end-to-end test and diagnostic manager provides a customer-centric end-to-end operational view of services, thereby concentrating more on managing the services than on managing the underlying transport mechanism or network.

One embodiment of the present invention provides a system for troubleshooting service problems for services delivered over a multi-domain service delivery network. A plurality of test tools are provided for testing the operational status and configuration of network elements in the various domains involved. The test and diagnostic manager has access to network and service inventory data. In response to service outages or other customer complaints, the test and diagnostic manager accesses network and service inventory data to learn which services are delivered to the customer and to identify the network elements needed to deliver those services to the customer. The test and diagnostic manager then selectively invokes various test tools to determine the operational status and configuration of the selected network element. The graphical user interface displays the results of the tests that are selectively invoked so that the operator can determine the source of the service problem and take steps to resolve it.

In another embodiment, the end-to-end test and diagnostic manager includes a graphical user interface for displaying status and configuration data regarding the network, network components, and services to the operator, as well as receiving commands from the operator. The application server has access to network and service inventory data. Upon receiving the report of the service issue, the application server determines the networks and network components needed to deliver the service to the client. The application server is adapted to execute a workflow to selectively invoke one or more test tools to determine an operational state or configuration of a network element involved in delivering a service. The results may be displayed by a graphical user interface so that an operator may determine the source of the problem and take steps to resolve it.

Finally, a method of diagnosing service delivery problems in services delivered over a multi-domain service delivery network is provided. The method includes collecting information for a set of service issues by meeting a customer. Upon receiving a report of a problem using the customer account, it is possible to gather information about all network elements needed to deliver the service to the customer. Once all network elements in the service delivery chain are known, the network elements are tested remotely to determine if they are working properly and are configured correctly to identify the cause of the problem. Once the source of the problem is determined, steps are taken to resolve it. Finally, when the problem is resolved, remote end-to-end testing is performed to ensure that the service has been restored.

Alternatively, the diagnostic process may be driven by the reporting of network device problems. Upon receiving a report of a network problem, it is possible to select a particular network resource (network element or application component) and collect information about the status of the particular network resource to see if it is operating properly and if it is configured correctly.

Other systems, methods, features and advantages of the invention will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the accompanying claims.

Drawings

FIG. 1 is a diagram illustrating an application specific vertical integration service delivery network;

FIG. 2 is a diagram showing convergence on IP as a preferred service delivery transport layer;

figure 3 is a block diagram of a DSL network;

fig. 4 is a block diagram illustrating a typical client device.

FIG. 5 is a schematic diagram illustrating a typical Voice over Internet (VoN) architecture;

FIG. 6 is a flowchart showing the steps involved in troubleshooting service delivery problems;

FIG. 7 is a simplified block diagram of an end-to-end test and diagnostic manager;

FIG. 8 is a more detailed block diagram of an end-to-end test and diagnostic manager and service delivery environment;

FIG. 9 is a diagram of an end-to-end test and diagnostic manager GUI page.

Detailed Description

Diagnosing service delivery problems in a multi-domain service delivery environment requires access and related information from different sources and presents the resulting data in a structured and unified manner to allow operators to view the data to quickly identify the nature and cause of the service delivery problem and to allow them to take corrective action immediately to resolve the problem.

Figure 6 shows an overview of the steps required to identify and solve the service delivery problem. First, at 250, a problem is reported. At 252, information about the issue is collected. The collected information may include information about the customer experiencing the problem, such as the services to which the customer subscribes and the services affected by the problem, information about the service delivery equipment at the customer, and information about the network and network elements that comprise the service delivery chain used to provide services to the customer. The collected information may also include information about the service that is experiencing the problem. Additional collected data may include alarm repositories, existing trouble tickets (trombone tickets), planned engineering/maintenance activities, performance management reports, predictive systems reports, and the like.

At 254, the root cause of the service delivery problem is identified. This may involve analyzing network and service listings, performing intrusive/non-intrusive tests on network elements, analyzing historical test results, and so forth. Once the root cause of the problem has been identified, steps may be taken to resolve the problem at 256. This may involve dispatching support personnel to repair client equipment or other hardware, accessing network element managers and test systems, modifying configuration data, and so forth. Finally, once the problem has been resolved, the problem may be closed at 258. Closing the problem may include performing a final end-to-end verification test, obtaining test results, and so on.

A simplified view of the end-to-end test and diagnostic manager 300 and the environment in which it operates is shown in fig. 7. The end-to-end test and diagnostic manager 300 has access to network and service inventory data 302 and other Operational Support Systems (OSS) 304. The end-to-end test and diagnostic manager 300 may retrieve data from the network and service inventory 302 and OSS 304 as needed to generate a comprehensive image of the service delivery chain including the network platforms, systems and components needed to deliver a particular service to a particular customer.

According to one embodiment, the network and service inventory data is stored in an Oracle database or other commercially available database and is accessible via SQL queries. The information exchange between the network and service inventory and the end-to-end test and diagnostic manager includes "business information" and "technical information". The business information relates to the customer, their service provider, their service plan, service ID, service date, etc. The technical information gathered from the network and service inventory includes all the information needed to rebuild the circuit used to deliver the service to the customer in order to prepare an accurate representation of the circuit, and to allow the end-to-end test and diagnostic manager to query all the primary devices involved. The technical information must include information about all services subscribed to by the customer and all components required to deliver those services.

According to one embodiment, three separate database tables are accessed to obtain all technical relevant data. The key input parameter is the customer ID (account number) or some other reliable identifier. The first table accessed is an a-Z circuit table that identifies all of the major components needed to build a circuit from a service provider to a customer for delivering a particular service to a particular customer. In this table, point "a" is represented by DSLAM and point "Z" is represented by NAS. The necessary components include, inter alia, point "a" of the ATM domain (service provider oriented ATM access device) and point "Z" of the ATM domain (customer oriented ATM access device). An example A-Z circuit table is shown in Table 1. The A-Z circuit table includes a plurality of attributes that define the types of components needed to build the circuit for delivering the service to the customer. The corresponding values identify the real-world components that satisfy the roles. The customer ID value (account number) identifies the customer for whom the service delivery chain is being constructed. The DSLAM name identifies the DSLAM that serves the identified customer. The next few attributes relate to the connection and configuration data for the identified DSLAM, the identified port, the subscriber line, etc. used to deliver the service to the identified customer. The NAS name identifies the network application server that provides services to the customer via the ATM network serving the customer. The identified NAS provides access to services and allows services to be delivered over an IP domain. The remaining attributes relate to the connection and configuration data for the NAS.

Table 1: (A & Z Circuit)

Attribute name	Value of
		Customer ID	0065940342
Name of DSLAM	MILANUSER/DSL/005
		Port client side (ATU-C)	ADSL2-2-12-19
DSL line	2:2:12
		Port client side (ATU-C)	ADSL2-2-12-19

VCI	2-2-12-19:8.35
		Cross-connect	LT＝8.35/NT＝34.65
Port ATM side	MRL08C
		VCI	34.65
NAS name	AT-BO256
		Port ATM side	ATM4:3:7
VCI	23.65

The value from the "port ATM side" of the DSLAM portion of table 1 is used to query a second database table (representing the logical link between the MUX and the ATM domain) to determine point "a" of the ATM domain network. The query identifies the appropriate ATM device, the appropriate port on the ATM, etc. These data are used to identify the corresponding IP address in the IP table and then to check the operational status and configuration of the ATM device and the designated port. An example of a second database table (MUX logical link) is shown in table 2.

Table 2: (MUX logical Link)

Attribute name	Value of
		ATM device name	MGX9063
Port(s)	MGX 9063:3:5
		VPI	MGX 9063:3:5:34

The value of the "port ATM side" of the NAS portion of table 1 is then used to perform a similar query against a third database table to see which device represents point "Z" at the opposite end of the ATM domain (from the standpoint of service delivery). Again, the query identifies the appropriate ATM device, the appropriate port, etc. so that the operational status and configuration of the ATM device and the designated port can be verified. An example of a third table (NAS logical link) is shown in fig. 3.

Table 3: (NAS logical Link)

Attribute name	Value of
		ATM device name	MGX9064
Port ATM side	MGX 9064:5:5
		VPI	MGX 9064:5:5:27

The final step is to determine the IP addresses of the above identified devices, i.e., the DSLAM, "a" and "Z" ATM devices, the NAS, and any other relevant devices that may be needed to deliver a particular service to an identified customer. This is done by separately querying a fourth database table (IP address device). An example of an IP address device table is shown in fig. 4.

Table 4: IP address equipment table

Attribute name	Value of	IP address
			Name of DSLAM	MILANUSER/DSL/005	10.11.134.178
ATM device name	MGX9063	10.11.156.170
			NAS name	AT-BO256	10.11.124.138

The business information is obtained by querying another table. Also, the customer ID (account number) is a primary key. An exemplary business information table is shown in fig. 5.

Table 5: (Business information)

Attribute name	Value of
		Customer ID	0065940342
Client name	Kate Moss
		ISP	AOL
Plan for	VoN quick access
		Service ID	986241585
Scheduled start date	07/13/1977

The technical information and the business information may also be retrieved from another operation support system 304. For example, the IP address of the device at the client may be obtained from a Service Delivery Platform (SDP) 338. The IP address may be used. For example, to determine whether a client device is on and running, a simple "Ping IP" test is used. Other information may also be retrieved from SDP 338. The exchange of information between the SDP and the end-to-end test and diagnostic manager may be accomplished by a web service. Parameters from the SDP may include:

technical key word (TK)Which is the true identifier of the circuit; which comprises

(name NAS, card, port, VPI, VCI) ═ AT-BO256, 3, 7, 23, 65)

Managing sessions：

The date of the first and last connection (11/07/200522: 34; 11/07/200523:18)

Data sessions：

The date of the first and last connection (11/07/200522: 35; 11/07/200523:17)

Main Account (Master Account)：

User name and creation date ═ c (sirio.baldini@aol.us，08/15/2005)

Data service：

Plan, business offer ═ (VoN quick access, VoN-3numbers)

Added services：

Parental control state, state of a Portal (Captive Portal) ═ off

IP address：

IP address of management session ═ (10.10.148.170)

Once all component links of the service delivery chain have been identified, an operator 306 employing the end-to-end test and diagnostic manager 300 may invoke a test program for testing various aspects of the service delivery process. For example, an operator may initiate tests that analyze the status of the hardware components and network elements that make up a service delivery chain in order to identify the root cause of a service delivery problem.

According to one embodiment, the end-to-end test and diagnostic manager 300 may access a full complement of test tools (fulllcompliance) for testing all functional domains involved in delivering services to customers. The various functional domains may include a CPE domain 308, a DSL/ATM domain 310, an IP network domain 312, and a VoN (or service) domain 314.

Examples of useful diagnostic tools include:

-CPEMS Ping test: this is a simple ping test to verify that all client devices (all devices of the client) are reachable by the end-to-end test and diagnostic manager.

-Sync scene mode (Profile): this is a tool to collect the configuration parameters (attenuation, noise margin, etc.) of the customers on the DSLAM.

-Port authentication: the tool checks the operational status of the DSLAM port to which the customer device is connected.

-Port locking/unlocking: this is to turn off the destination device (DSLAM, ATM, IP routing)Device … …) and restart the port's system wide management command.

-No Sync test: the test returns boolean results on the synchronization between the modem and DSLAM (Siemens, Alcatel, etc.) and the customer equipment (modem or CPE).

-Port reset: this test is used to reset the configuration for the DSLAM (customer side port).

-IP Ping test: this test is used to verify the reachability of IP devices (NAS: Cisco 10k, Cisco6004, Juniper ERX, etc.).

-Test call: this test simulates an IP call to a customer phone through the element manager of the switch, but does not cause the customer phone to ring.

-Call history: this is a tool that provides information about IP calls made by the client (date, duration, time of day, etc. of the first and last calls).

-End-to-end VoN service: this is a tool that examines and reports information about IP calls (bandwidth, communication quality, inbound/outbound, etc.).

-End-to-end BB service: this is a tool that gathers information about packets sent, lost and received for a particular NE.

-MLT(mechanical loop tests), these are a set of tools used to determine the condition of the customer's PSTN line:

the test line is OK;

potential problems with customer equipment, such as defective phones or listeners going off-hook;

call on line.

The end-to-end test and diagnostic manager 300 may be configured to access and invoke the various tests in many different ways. For example, the end-to-end test and diagnostic manager may include a test repository 316 that includes workflows defining the operational states and steps to be taken to test the various components in different domains. The workflow defined in the test repository may include steps for directly interrogating and testing hardware devices and network elements. Alternatively, the workflow may call an external test tool and business package (business package)318 developed for testing specific components in various domains. In another alternative, the end-to-end test and diagnostic manager 300 may invoke the layer 2, 3 domain manager 320 to test the functionality of one or more domains. Finally, the end-to-end test and diagnostic manager 300 may invoke an Element Manager (EM)322 configured to test various components within various domains. The workflow within the test repository 316 may be quite complex. The end-to-end test and diagnostic manager may invoke the test and interrogation components in a combination of different ways depending on whether internal/external test tools are available, whether a layer 2, 3 domain manager is present, or whether any element manager is available. For example, the end-to-end test and diagnostic manager may test some components directly via an API executing on an API server within the end-to-end test and diagnostic manager. Other tests may be invoked by external test tools, other tests may be invoked by the layer 2, 3 domain manager 320, and still other tests may be invoked by a separate element manager 322.

The end-to-end test and diagnostic manager 300 presents a Graphical User Interface (GUI) to the front-end and back-end operators 306. The front-end operator may use an end-to-end test and diagnostic manager to access service complaints and run simple initial diagnostics to determine whether a service delivery problem can be quickly resolved by standard, easily executed procedures, or whether it is necessary to perform more complex analysis to resolve the problem. If necessary, back-end operators may use the system to perform more complex tests and investigate some of the more troublesome service issues more deeply. Various tools accessible by the front-end and back-end operators may be defined in the user profile database. Back-end operators may be given access to more complex test and diagnostic tools than their front-end test and diagnostic tools. For example, the interface may also enable the back-end operator to access historical test results, alarm and error logs, maintenance and service schedules, performance management systems, predictive systems, etc., in order to assist the back-end operator in determining the cause of difficult service delivery problems. The end-to-end test and diagnostic manager interface may also provide back-end operators with means for taking steps to address service issues that are not appropriate for the front-end operator, such as dispatching the appropriate support team, accessing the element manager and configuration management system, and so forth. Once the problem is resolved, the back-end operator may perform validation of the service test and obtain various test results and actions to be taken to resolve the problem.

Fig. 8 shows a more detailed view of the end-to-end test and diagnostic manager 300 and the surrounding multi-domain service delivery environment. Many of the components depicted in fig. 7 have been repeated in fig. 8 and have been given similar reference numerals. As shown, the end-to-end test and diagnostic manager 300 interfaces with a network and service inventory 302 and other Operations Support Systems (OSS) 304. The OSS 304 may include a field access system (field access system)328, an LDAP 330, an integrated command management system 332, a network failure management system 334, an error management system (failure management system)336, a service delivery platform 338, and a performance management system 340.

The end-to-end test and diagnostic manager 330 has three possible types of interaction with other OSS systems 304 and network resources.

1) The end-to-end test and diagnostic manager may take information from:

a network and services inventory 302, which provides data relating to:

services offered by service providers;

customers of the service provider

Services offered to specific customers

The network components involved in delivering a particular service to a particular customer.

Service delivery platform 338 (service and user profile information).

Specific tools 318, domain manager 320, EM322 and network elements.

Performance management 340 in order to check the degradation (degradation) of the devices involved in the service provided to the customer.

Integrated command management 332 to collect information about the status of work requests on the purported customer.

2) The end-to-end test and diagnostic manager provides information to:

network fault management 334, in order to manage customer faults related to service problems on the network and enrich the relevant fault tickets.

Error management 336 to turn on alarms when problems are found and to provide information about affected customers.

A field access system 328 to perform a final check of service recovery from the technician's mobile terminal at the customer premises accessing the test manager.

3) End-to-end testing and diagnostics may also perform information exchange with LDAP (Lightweight Directory Access Protocol) servers to authorize user Access.

A plurality of external test tools 318 are shown in fig. 8, including CPE test tool 342, ATM test tool 344, DSL test tool 346, IP test tool 348, and VoN test tool 350. Similarly, a plurality of element managers 322 are also shown. The element manager 322 includes a CPE element manager 354, an ATM element manager 356, an IP element manager 358, and a VoN element manager 360. Multiple element managers are shown for each domain, as each domain may include multiple network element types. Also shown in fig. 8 is a layer 2, 3 domain manager 320. It should be noted that in various end-to-end test diagnostic manager implementations, it is not necessary that all test tools, element managers or layer 2, 3 domain managers be present. In most cases, various combinations of test tools, element managers, and layer 2, 3 domain managers will be combined within the end-to-end test and diagnostic manager 300, along with additional direct access test protocols. Preferably, the end-to-end test and diagnostic manager 300, the test tool 318, the layer 2, 3 domain manager, and the network element manager 322 present in certain embodiments may be combined to provide a complete complement of diagnostic tests for performing a comprehensive analysis of the service chain for all services provided by the service provider.

The end-to-end test and diagnostic manager 300 itself includes an internal database 320, a user profile database 322, a web server 324, and an application server 326. The user profile database 322 stores user profiles of front-end and back-end operators and other administrative personnel accessing the end-to-end test and diagnostic manager 300 for troubleshooting service problems. The user profile will include access rights and privileges for various users that define test and diagnostic tools and configuration data available to different types of users.

Internal database 320 stores data about customers and services delivered to them. According to one embodiment, internal database 320 does not store a comprehensive data set of all customers, services, and service delivery components. Rather, the internal database 320 takes the necessary data from the network and service inventory 302 and other OSSs 304 as needed to address the particular service delivery issue. The data is then stored in an internal database 320 where they are accessed by a web server 324 to populate the display of various data fields and GUIs, or by an application server 326 to perform the various workflows required to complete end-to-end testing of the service delivered to the particular customer making the complaint.

Fig. 9 shows an example of an end-to-end test and diagnostic manager Graphical User Interface (GUI) page 400. The GUI page 400 includes many different areas in which different types of information are displayed, and different actions may be initiated by an operator interacting with the interface. These include a customer display area 402, a diagnostic type area 404, a tools area 406, a provisioning and configuration information area 408, a network display area 410, and a customer test history area 412.

According to one embodiment, the customer account number is used as a customer ID and drives the end-to-end test and diagnostic manager. Thus, when a customer calls to report a service disruption or some other service delivery issue, the operator enters the customer's account number into the customer ID field 414 in the customer display area 402. Upon receiving the client ID, the application server 326 retrieves relevant data about the client, and the services subscribed to by the client, from the network and services inventory 302 and other OSS 304. The retrieved data is stored in the internal database 320 and is used to populate pages of the GUI served to the operator by the web server 324. Thus, in the example shown in FIG. 9, the operator has entered customer ID124578909 in customer ID field 414. Information about the client and the services subscribed to by the client is displayed in the client display area 402. In this case, the customer is Jim Green. He subscribes to VoN Plus service and DSL Plus service. Additional data such as the customer's phone number, service ID number, schedule start date, DSL ID number, etc. may also be displayed. The operator may select a customer failure or network failure view in failure view area 402. The selection of a network failure view may allow an operator to perform diagnostics through a network resource (network element or application component involved in a service) by using an identifier of the network resource or selecting a particular application component. This may be the most efficient way to locate the source of the problem in the case of a large area wide service outage.

The diagnostic type area 404 lists the types of services that the end-to-end test and diagnostic manager is configured to diagnose. The diagnostic type area may list all services offered by the service provider or the displayed list may be limited to specific services subscribed to by the identified customer. In either case, the operator may select an appropriate diagnostic type from the diagnostic type area 404 based on the nature of the customer complaint. In the illustrated example, the operator has selected the customer's VoN service to perform diagnostics.

The tools area 406 displays test and diagnostic tools that may be used to troubleshoot various services. The tools displayed in the tools area 406 may be limited to tools that are applicable to the service selected in the diagnostic type area 404, or may display a complete complement of available tools. In either case, the operator selects various tools from the tools area 406 for testing components and diagnosing service delivery problems.

The network display area 400 displays a simplified view of the service delivery chain. Data regarding the service delivery chain is retrieved from the network and service inventory 302 and includes all network elements involved in delivering the selected service to the customer. The simplified graphical representation of the service delivery network helps front-end operators to easily understand where the problem is located and to dispatch them to the appropriate service department. For example, the very complex VoN service delivery architecture shown in fig. 5 is reduced to a simplified network shown in network display area 410 of GUI page 400. Although simplified, the network shown in fig. 9 includes all critical components needed to deliver VoN service to the identified customer jimsreen.

Provisioning and configuration area 408 displays equipment and equipment configuration data for various domains of the service delivery chain. For example, the provisioning and configuration information area 408 includes a label (tab) for displaying provisioning and configuration data for the CPE domain, transport domain, IP domain, and VoN domain. In the example shown, the CPE tag is selected and the provisioning and configuration information area 408 displays information about the modem within the customer premises, including the modem type, IP address and MAC address.

Finally, customer test history area 412 displays data regarding previous tests that have been performed for the selected customer to troubleshoot earlier problems. The operator may enter a date range in the "From" and "To" fields 416, 418, and the customer test history area 412 will display information about all tests performed for that customer during the specified period. This information includes the date of the test, the test ID, the type of test, the bug manifest number associated with the test, and the operator who created the test. The test history may provide insight into current service delivery issues.

The information displayed on the GUI page 400 provides a complete end-to-end view of the service delivery chain for delivering various services to the customer. The GUI also displays various testing and diagnostic tools that may be used by the operator for troubleshooting service delivery problems.

Returning to FIG. 8, the application server 326 includes a workflow that is executed when some test and diagnostic tools are invoked. For example, when a particular test is to be performed on a particular network element, the application server determines whether the test is to be performed by the network element manager, a separate stand-alone test tool, a layer 2, 3 domain manager, or the application server itself. In the case where the test must be performed by the application server 326 itself, the application server directly accesses the appropriate device and executes the commands needed to complete the selected test. The application server receives the test results and stores them in a database 320, where they are accessible by the web server for presentation to the operator via the GUI. Otherwise, the application server sends instructions to the appropriate test tool, layer 2, 3 domain manager, or appropriate network element manager, according to the workflow, and invokes the required tests. The test tool, layer 2, 3 domain manager, or network element manager, as the case may be, accesses the appropriate network element and performs the requested test. In either case, the test results are returned to the application server and incorporated into a GUI page served to the operator via the web server 324. Since the application server can access all network elements in the service delivery chain either directly or through a layer 2, 3 domain manager or network element manager as an intermediate testing tool, the end-to-end test and diagnostic manager is able to provide full end-to-end coverage of the service delivery chain. The workflow may also define a sequence of basic test tools to be executed in order to perform more complex diagnostics. The single test result is used by the test manager as a trigger to follow a particular workflow.

While various embodiments of the invention have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.

Claims (29)

1. A system for troubleshooting service delivery problems for services delivered over a multi-domain network, wherein the multi-domain network includes a service delivery chain of network components for delivering the services, the system comprising:

a server adapted to:

receiving a report of the service delivery issue;

in response, retrieving data from a network and service inventory database and storing the retrieved data in an internal database, wherein the retrieved data specifies a service delivery chain of a network component that delivers the service;

determining applicable test and diagnostic tools that are available for troubleshooting network components of the service delivery chain; and

executing a workflow for invoking selected ones of the applicable test and diagnostic tools; and

a graphical user interface GUI in communication with the internal database and adapted to:

displaying a service delivery chain of the network component in a network display area of a GUI; and

displaying the test and diagnostic tool in a tool area of a GUI;

the server is further adapted to:

operator input is obtained specifying the selected test and diagnostic tool and invoking a workflow of the selected test and diagnostic tool.

2. The system of claim 1, wherein the data comprises a service plan subscribed to by a customer.

3. The system of claim 1, wherein the data comprises customer data describing service equipment at the customer and under customer control.

4. The system of claim 1, wherein the data comprises information retrieved by the server from an operations support system.

5. The system of claim 1, further comprising a mobile terminal having a field access system, wherein the server is adapted to fetch data from the field access system to perform a final check of service recovery.

6. A system as described in claim 1, wherein the server is adapted to fetch data from a lightweight directory access protocol LDAP to perform an exchange of information to authorize user access.

7. The system of claim 1, wherein the server is adapted to fetch data from an integrated command management system to collect information about the status of work requests related to the service delivery issue.

8. The system of claim 1, wherein the server is adapted to provide information to a network fault management system.

9. The system of claim 1, wherein the server is adapted to fetch data from an error management system to turn on alerts and provide information about affected clients when problems are found.

10. The system of claim 1, wherein the server is adapted to retrieve data from a service delivery platform to collect service and user profile information.

11. The system of claim 1, wherein the server is adapted to retrieve data from a performance management system to check for degradation in the network component.

12. The system of claim 1, wherein the server is adapted to provide data collected from the selectively invoked diagnostic tool to an external operations support system.

13. The system of claim 1, wherein the test and diagnostic tool comprises a stand-alone test kit.

14. The system of claim 1, wherein the functionality of the test tool is provided by a network element manager, the network element manager having access to configuration data for a particular network element, and the test tool being capable of testing the operational status of the particular network element.

15. The system of claim 1, wherein the functionality of the test tool is provided by a layer 2, layer 3 domain manager.

16. The system of claim 1, wherein the functionality of the test tool is included as an internal test module.

17. The system of claim 1, wherein the test tool comprises a combination of a stand-alone test kit, a network element manager, a layer 2, layer 3 domain manager, and an internal test module.

18. The system of claim 1, further comprising a user profile database for storing user profiles and access privileges.

19. The system of claim 1, further comprising a web server for serving GUI pages.

20. An end-to-end test and diagnostic manager for diagnosing service delivery problems for delivering services to customers over a multi-domain network via a service delivery chain of network components, the end-to-end test and diagnostic manager comprising:

a graphical user interface GUI for displaying status and configuration data of the network components comprising the service delivery chain, and for displaying test tools available for troubleshooting the service delivery problem, whereby a user can select one or more of the test tools; and

a server to:

receiving a report of a service delivery issue, and in response thereto, accessing network and service inventory data to determine a comprehensive view, comprising:

the network services provided to the customer are provided,

the network component, and

the status and configuration data; and

a workflow is executed to invoke at least one of the test tools selected by the user.

21. The end-to-end test and diagnostic manager of claim 20, wherein the test tools comprise any combination of tools taken from a list, the tools comprising: performing a CPEMS Ping test; a Sync profile; verifying a port; port locking/unlocking; no Sync test; resetting the port; IP Ping test; testing the call; a call history; an end-to-end VoN service; and end-to-end BB services; mechanical loop test MLT.

22. The end-to-end test and diagnostic manager of claim 20, wherein the server is adapted to directly execute one or more of the test tools.

23. The end-to-end test and diagnostic manager of claim 20, wherein the server is adapted to invoke an external test tool to test network elements within the multi-domain network domain.

24. The end-to-end test and diagnostic manager of claim 20, wherein the server invokes at least one of the test tools via a layer 2, layer 3 domain manager.

25. The end-to-end test and diagnostic manager of claim 20, wherein the server invokes at least one of the test tools via an element manager associated with a network element.

26. The end-to-end test and diagnostic manager of claim 20, further comprising a web server, the graphical user interface comprising a web-based graphical user interface having interface pages served to a client terminal associated with an operator.

27. The end-to-end test and diagnostic manager of claim 26, further comprising a user profile database for storing user profiles defining access rights test privileges for operators allowed to use the end-to-end test and diagnostic manager.

28. The end-to-end test and diagnostic manager of claim 20, wherein the server is configured to invoke a plurality of test tools to test the status of each network component involved in delivering a service.

29. A method for diagnosing service delivery problems for delivering services to customers over a multi-domain network via a service delivery chain of network components, the method comprising:

displaying status and configuration data of the network components comprising the service delivery chain, and displaying test tools available for troubleshooting the service delivery problem, such that a user can select one or more of the test tools;

receiving a report of a service delivery issue;

accessing network and service inventory data and determining a comprehensive view, comprising:

the network services provided to the customer are provided,

the network component, and

the status and configuration data; and

a workflow is executed to invoke at least one of the test tools selected by the user for identifying a root cause of the service delivery issue.