Chapter 7

Man's mind, stretched to a new idea, never goes back to its original dimensions. -Oliver Wendell Holmes

This chapter describes how services are provided using MPLS. Included in this description is a high-level overview of the pieces used to provide these services, including specific protocol support functions. Specifics of the protocols and components of MPLS used in providing these services are discussed in greater detail in earlier sections of this book.

7.1 Basic Services

Basic services in MPLS are effectively enabled using hop-by-hop LSPs established using LDP in the intranet (IGP) case and MPLS-BGP in the Internet (EGP) case.

Using LDP

In the IGP/LDP case, and assuming that the goal is to establish a best-effort LSP for each route table entry at each LSR, the process is as follows:
  1. Route advertisements build up the local LSR's picture of the network.
  2. While the routing function builds a table of routes, the LDP function establishes sessions with peers as it discovers their adjacency.
  3. As the routing function within the LSR converges on a routing top-ology, the LDP function begins distributing labels for each route to each peer with which it has established a session and for which the peer is not the next hop.1

Note that this process assumes that label distribution is downstream unsolicited. In the event that the label distribution mode is actually downstream on-demand, labels are distributed as described except that the local LSR requests labels from the next-hop LSR peer for each route table entry.

If conservative label retention is used, the LSR retains those labels that it will use and releases those that it will not use. Otherwise, any number of labels may be retained, up to and including all labels received. Using the labels retained specifically for each given FEC's next hop, an LSR constructs an NHLFE and an FEC-to-NHLFE map (FTN) if the LSR is an ingress for the LSP, an Incoming Label Map (ILM) if the LSR is not an ingress for the LSP and will receive only labeled packets for the corresponding FEC, or both if the LSR expects to receive both labeled and unlabeled packets from upstream routers that will be merged onto the same LSP.

If the LSR has constructed an FTN, it may act as the ingress for any unlabeled packets it receives by using a matching NHLFE (if a valid one exists).2 An LSR may act as an egress for labeled packets it receives having an ILM and no matching NHLFE. An LSR is expected to perform some label operation (push, pop, or swap) if it receives a labeled packet for which it has both an ILM and a matching NHLFE. An LSR may be expected to forward unlabeled packets as unlabeled packets if it has no FTN or no matching NHLFE.

As route table entries are added, removed, or changed, the LSR takes corresponding label distribution actions (advertising, requesting a new label or withdrawing the request, releasing the existing invalid label). If an LSR loses a peer (because the peer session is terminated, perhaps because the adjacency is lost), it invalidates all corresponding labels.

  1. It is possible to distribute labels in this case as well. However, there is little point because anything received with that label should be dropped rather than returned.
  2. An NHLFE is only "matching" if it is a valid and active NHLFE.

Using BGP

With BGP, labels are distributed piggyback on BGP route advertisements (using BGP Update messages). The label stack distributed as a part of a BGP route remains valid until that route is explicitly replaced or invalidated based on BGP routing. In practice, the behavior is much like the behavior for LDP because the two protocols have a great deal in common.

7.2 Quality of Service: Premium Services

Using the Integrated Services Model

Both RSVP-TE and CR-LDP support the Integrated Services QoS model and can support this model when used in tandem. RSVP-TE is a more natural fit than CR-LDP for this usage because RSVP was designed with Integrated Services model support in mind. However, the same objects-with relatively minor alterations, at most-are used in CR-LDP, thus providing a seamless adaptation between the two approaches.

The Path message (in RSVP-TE) or the Label Request message (in CR-LDP) carries the resource requirements for an LSP intended to support the Integrated Services QoS model. In each case, when this information arrives at an egress, a corresponding response is generated (Resv for RSVP-TE, and Label Mapping for CR-LDP) and LSR resources are committed during the process of propagating the response back to the originator. Clearly, in both cases, the downstream on-demand label distribution and ordered control modes are used. Also, in either case, implementations are free to commit resources during the request phase (the portion of the signaling process during which label requests are being propagated downstream).

Using the Differentiated Services Model

Support for the Differentiated Services QoS model can be achieved via establishment of specific L-LSPs,3 each of which is administratively associated with some defined per-hop behavior (PHB). It can also be achieved via establishment of a single E-LSP (see footnote 3) for each Ordered Aggregate of PHBs, allowing the use of configured values of the Experimental (EXP) bits to determine which specific PHB is to be used for any labeled packet.

An E-LSP is distinguished from an L-LSP by the former's use of the EXP bits in the generic label format. The Cell Loss Priority (CLP) field in ATM and the Discard Eligibility (DE) field in Frame Relay are used similarly, though with less effect, for E-LSPs in their respective technologies. With L-LSPs, the label itself implies the behavior that applies to packets within the given LSP. Either RSVP-TE or LDP may be used to establish LSP support for the Differentiated Services model. In the RSVP-TE case, new Class of Service (COS) objects are provided to allow for setup of an LSP for this purpose, effectively establishing an LSP for classes of best-effort service.

  1. L-LSP and E-LSP stand for Label-only-inferred-PSC LSP and EXP-inferred-PSC LSP, respectively. EXP refers to the Experimental bits field in the generic label, the Cell Loss Priority (CLP) field in the ATM header, or the Discard Eligibility (DE) field in the Frame Relay header. PSC stands for PHB Scheduling Class, and PHB stands for per-hop behavior. See Le Faucheur et al. (w.i.p.) for details on how Differentiated Services is supported in MPLS. I suspect that many proponents of Differentiated Services may have migrated into the field out of frustration at the lack of acronyms being used in other standards organizations and bodies.

7.3 Traffic Engineering

Traffic engineering (TE) is the process of optimizing the performance of operational networks. In doing this, TE skirts the ragged edge of computational intractability in an effort to extend the utilization of network resources. TE uses both computation and heuristics to achieve a good-enough utilization factor for a given set of network resources and traffic conditions. Because heuristics produce an inexact solution to the TE problem, the survival of service providers in the competition to offer low-cost, high- quality services will depend in a large measure on the long-term effectiveness of the heuristics chosen by each service provider.

The key goals in TE are to maximize network efficiency and total data "good put." Priorities within the "good put" category are as follows (approximately in order):

Specific goals for network efficiency revolve around ensuring that the average utilization of network resources is as close to 100% as possible while minimum and maximum utilization of individual resources are as close to the average as possible.

Both goals are affected by congestion; thus, avoidance of congestion is of paramount importance. Problems associated with congestion are made worse by inefficient use of network resources. These problems are directly addressable using TE.

The TE model consists of a connected network, performance monitoring feedback, and a management and control system. The traffic engineer determines the current state of the network (via performance monitoring), analyzes the traffic characteristics and trends, and attempts to control the network in such a way as to alter the current state to one that maximizes the desired characteristics of the network and accommodates existing traffic characteristics and trends. This process is a continuously ongoing effort.

To minimize the amount of operator involvement in the TE model, it is desirable to minimize the operator's involvement in modifying traffic management and routing parameters and in modifying the way in which the system's use of resources is artificially constrained. A desirable solution is one that is both scalable and resilient.

Interior gateway routing protocol capabilities are not up to the task. In fact, prevalent IGPs contribute to congestion because they are effectively designed to develop a consistent view of the topology that results in traffic being forwarded dominantly along shortest paths. As a result, shortest-path routes are likely to be highly congested while similar routes are likely to be underutilized.

An important factor in the long-term effectiveness of a TE solution is system responsiveness to changes in traffic conditions and corresponding measurement of resource utilization.

The Role of MPLS

MPLS is useful for TE in the specific aspects of measurement and dynamic control of Internet traffic. Because of the high cost of networking resources and the competitive environment that each service provider faces, dynamic performance optimization in service provider networks is a critical factor in determining a service provider's ability to survive in the industry.

One important aspect of TE is the introduction of simple load-balancing techniques. However, traffic engineering also needs to take into account other factors affecting total income production from use of the service provider's network. This requires mechanisms for supporting more complicated policies than a simple load-balancing scheme. MPLS provides a means for effecting more complex TE solutions at a potentially lower cost than alternative technologies.

MPLS offers dynamic mechanisms for establishing explicitly routed LSPs that can operate independently of IGP-determined routes. This reduces the impact that limitations in routing protocol behavior have on congestion in the network. Because MPLS mechanisms are dynamic, LSPs can be established with desirable resiliency and can be reoptimized as needed.

Specific attractive features of MPLS are as follows:

How Traffic Engineering Works

A TE solution needs to provide a means for directing traffic along paths that would not be taken using the routing infrastructure alone. ATM and Frame Relay virtual circuits (VCs) have been successfully used as TE solutions to date. Use of virtual circuits in an overlay topology allows VC-based routing, explicit administrative configuration of VC paths, path compression, admission control, traffic shaping and policing, and VC survivability.

Using virtual circuits thus allows many TE functions to be accomplished with today's networks.

Equipping MPLS with a similar virtual circuit capability is important for future network TE needs. MPLS offers the ability to provide an integrated overlay model at a lower cost than existing ATM and Frame Relay equipment. MPLS also offers the opportunity to automate some of the traffic engineering functions.

The difficulty in realizing a TE solution using MPLS is the hierarchical nature of the mapping of traffic onto LSPs in a TE model. Ultimately, the traffic engineer wants to create traffic trunks to shunt traffic in the network in such a way as to produce efficient utilization. These traffic trunks would be realized using explicitly routed LSPs in order to achieve independence from the underlying routing infrastructure. However, traffic is mapped onto LSPs using forwarding equivalence classes (FECs). Thus, it is necessary to determine the FEC-to-traffic-trunk mapping that will produce the most efficient mapping of traffic to traffic trunks and then onto the overlay of explicitly routed LSPs.

To do this, TE over MPLS requires

LSP-based traffic trunks are inherently unidirectional; however, bidirectional traffic trunks may exist as well. A traffic trunk is considered bidirectional if the LSPs used to create the traffic trunk include the same ingress and egress LSRs (obviously, in reversed roles) and are created, maintained, and destroyed together. Bidirectional traffic trunks may be symmetrical or asymmetrical in the sense that they are not required to use the same set of LSRs (in reverse order) as long as they have the same termination points.

RFC 2702 (Awduche et al. 1999) defines actions with respect to traffic trunks, which are shown in Table 7.1. In addition to billing, capacity planning, and related functions, measurement of traffic trunk statistics is important in determining more immediate traffic characteristics and trends for use in optimizing network performance. From a TE perspective, the ability to collect this information is essential.

Traffic Trunk Attributes

The attributes of a traffic trunk are values that can be computed or administratively configured to control the behavior of traffic within the trunk. Attributes suggested by RFC 2702 are defined in Table 7.2.

Traffic Parameter Attribute

Statistical approaches have been defined (see, for example, Chang and Thomas 1995) for determining approximately how much real bandwidth is required to support traffic based on well-understood traffic parameters and the queuing behavior of network equipment. These approaches have been used to determine service admissibility in, for example, ATM virtual circuit establishment.

Traffic engineering can turn this around somewhat by using the observed traffic parameters of existing flows to determine the size of traffic trunks needed to carry these flows. Alternatively, the sizing of traffic trunks may be determined from measurements of congestion at various points in the network. An admission control function is then used to select specific flows to apply to each traffic trunk.

Policing Attribute

The policing attribute determines specific activity with respect to out-of-compliance traffic associated with a TE traffic trunk. Possible activities include rate limiting (dropping excess traffic), marking or coloring (associating packets with a Cell Loss Priority, Drop Precedence, or a similar marking), or forwarding without action.

Some type of policing action must occur somewhere in a traffic trunk unless all traffic in the trunk is best-effort traffic (implying that no compliance agreement exists). Otherwise, all traffic is treated the same, regardless of its in-compliance status. However, it is not generally desirable to perform policing at every node in the network. Policing for an LSP is generally done only at the ingress for that LSP.

From a TE perspective, however, policing for a traffic trunk is either done or not done. A traffic trunk may start at some point within a service pro-vider's network or may otherwise have been subject to policing (or traffic shaping) already. In this case, it is necessary to be able to disable policing for the traffic trunk.

Priority Attribute

Priority is used to determine the order of setup for TE traffic trunks when more than one traffic trunk is pending (for example, during system initialization or fault recovery). A TE solution may need to recompute paths after each successful traffic trunk establishment, particularly when a traffic trunk consumes resources that affect the path selection process for subsequent traffic trunks. Because available resources are consumed with each traffic trunk established, it is likely that each successive traffic trunk will be more constrained than similar traffic trunks established previously.

Priority should take into account the resources each traffic trunk will consume. This is analogous to the problem of fitting as many rocks into a bottle as possible, given a fixed set of rocks of various sizes. Putting larger rocks in first can be the best strategy for getting the largest volume of rock into the bottle. In the TE case, setting up those traffic trunks that consume the most resources later in the setup process increases the likelihood that trunk establishment will fail, even if all of the existing trunks would have succeeded using a different order.

Priority should also take into account the preemption levels of various traffic trunks. Each traffic trunk that is preempted may need to be reestablished. In this case, the system will take longer to establish the full set of traffic trunks if trunks that will be subsequently preempted are established prior to those that might preempt them. As defined in RFC 2702, this occurs automatically because priority and exemption levels are dependent.

Preemption Attribute

Preemption is useful in ensuring that high-priority traffic trunks will be routed using a favorable path and in implementing a prioritized restoration process following a network fault. Preemption is defined in two dimensions: the ability of a traffic trunk to preempt other traffic trunks and the ability of a traffic trunk to be preempted by other traffic trunks.

RFC 2702 defines preemption as binary along these two dimensions. That is, a trunk either can or cannot preempt another trunk, and a trunk either can or cannot be preempted by another trunk. If a trunk being established can preempt other trunks and cannot otherwise be established, it will preempt another trunk (that may be preempted) if that other traffic trunk is of a lower priority. In general, a network element processing the setup in this case will preempt existing LSPs-starting with the LSP having the lowest priority-until either there are sufficient resources to satisfy the requirements of the new LSP setup or there are no remaining lower-priority LSPs. Note that LSPs should not actually be preempted if there will not be sufficient resources to establish the new LSP when all lower-priority LSPs have been preempted.

Many implementations handle preemption using a two-level priority:

Because a circuit that has been preempted may be reestablished, it is essential that the holding priority never be lower than the setup priority. Distinct set up and hold priorities may be useful when it is desirable to set up a low-priority circuit that must have a high-priority survivability if it is successfully established. This might be the case for large numbers of short-duration circuits. It would also be the case if disruption of services is intended to be implemented as a breadth-first search for lower-priority circuits to preempt. It is relatively simple to implement the behavior defined in RFC 2702 by always setting setup and hold priorities to the same value.

Path Attributes

Paths used by traffic trunks may be determined in two general ways: using loose or strict explicit routes. The traffic engineer may select key points to include in the explicit path while leaving the actual path used by the traffic trunk between these points to be determined by the interior routing protocol. This approach deforms existing traffic flows by effectively creating new ingress points for trunk traffic, thus affecting the utilization of network resources along routed paths. Alternatively, the traffic engineer may select each hop to be traversed by a traffic trunk based on a priori knowledge of node and link capacity along a strict path. Each approach has advantages and disadvantages.

The traffic engineer may be either a TE automaton or an operator administratively configuring LSPs for use with traffic trunks. In a generalized TE solution, it is possible for variant traffic engineers to each determine and attempt to establish traffic trunks for the same purpose. For instance, a TE automaton may determine one path while an operator has configured another. In general, it is necessary to provide a means to resolve which path will be used to establish a traffic trunk in this case. Specifically, it should be possible to force the system to accept the traffic trunk configured by the network operator. Ideally, the system will report inconsistencies of this type, especially in the event that the configured path is not feasible (or is suboptimal by some threshold value). Alternatively, the path selected by one method (for example, manual configuration) can be treated as the preferred path and will be used as long as this path is not infeasible or seriously suboptimal.

RFC 2702 defines the behavior of arbitrating between a manually configured path and a dynamically computed path by describing manually configured paths as either mandatory or nonmandatory. A mandatory configured path is used regardless of the computed path.

Path maintenance criteria affect whether or not a traffic trunk will be moved in response to specific changes in network topology. In general, a traffic trunk may be established such that the path will not change unless the current path optimization is exceeded by an alternative path optimization by some threshold. In the event that the threshold is exceeded, however, the LSP for the traffic trunk will be reoptimized. If it is the intent that the LSP not be reoptimized, the threshold value would be effectively infinity. Path maintenance criteria may also include other values, such as a delay value (to avoid transient reoptimization). Adaptivity and resilience are subattributes, or aspects, of the path attribute and are discussed in detail in a subsequent section.

In summary, a path attribute includes the strictness of the explicit route, arbitration (mandatory or nonmandatory in RFC 2702), adaptivity, and resilience.

Resource Class Affinity Attribute

Resource classes are also used to constrain the path selection process. Specific resources are either necessarily included or excluded from the path selection process, depending on the affinity value associated with the specific resource. For example, RFC 2702 suggests an affinity value of 1 for explicit inclusion and 2 for exclusion. Using these values, if a resource is assigned an affinity value of 1, then the path selected must include only network elements having this resource. A resource, in this context, may be a certain type of queuing behavior or bounded delay characteristic, or it may be a specific set of network elements. The default for unspecified resource class affinities is that associated resources are not considered in selecting a path.

Resource Attributes

This section describes the resource attributes allocation (or subscription) factor and resource class.

Allocation or Subscription Factor

Because of the statistical nature of the distribution of traffic, it is possible to oversubscribe network resources in an effort to achieve better overall utilization. This is most useful when traffic distributions of multiple sources sharing the same resources do not have coincident peaks and troughs. People who are familiar with overbooking of airline reservations are aware that there is usually some association of a lower grade of service with overbooking of resources. This is likely to be true in networks using oversubscription of network resources as well.

Where a very high degree of traffic delivery assurance is desired, undersubscription of network resources may be used. When this is done, a subscription (or allocation) factor is applied to the bandwidth determination for the applicable traffic trunk.

Because network resources typically do not natively support the concept of oversubscribing and undersubscribing their resources, the traffic engineer applies a subscription/allocation factor prior to establishing the traffic trunk. For example, if oversubscribing by 25% corresponds to an allocation factor of 1.25, the traffic engineer would multiply the bandwidth requirement otherwise determined for the traffic trunk by 1.25 prior to requesting the corresponding LSP setup. Note that this is, in effect, an effort to fine tune an effective-bandwidth calculation4 as might have been required to determine the bandwidth requirements in the first place.

Resource Classes

A resource class is a characteristic that may be arbitrarily assigned to a resource. Resources belonging to the same resource class are treated similarly by path selection and other policies. The resource class abstraction can be used to determine the set of policies that apply to this resource irrespective of other factors (such as topological location of the resource). These policies include the following:

Resource classes may also be used simply to identify resources.

  1. Effective bandwidth is a theoretical approach to determination of a single bandwidth value based on traffic parameters such as peak and average propagation rates, burst duration, and so forth. This concept first arose as early as 1991 and is referred to in RFC 2702 (Awduche et al. 1999) and discussed in Chang and Thomas (1995) and (by a similar name) in Guerin, Ahmadi, and Naghshineh (1991) and is-in concept-not unlike the calculations that airlines apply in determining how much to overbook flights.

Constraint-Based Routing

Constraint-based routing is based on the idea of describing a set of link characteristics that are either desirable or undesirable for a particular route and then trying to find a route that has all desirable characteristics and no undesirable characteristics. Using a combination of the metrics defined for traffic engineering and the capabilities of routers, constraint-based routing can substantially reduce the requirements for operator activity necessary to implement TE.

An ingress LSR does constraint-based route computations in order to automatically compute explicit routes used for traffic trunks originating at that LSR. For TE, the traffic engineer would initiate this process.

The traffic engineer specifies the ingress and egress of a traffic trunk and assigns a set of characteristics for a desirable route. These characteristics define constraints in terms of performance needs and other aspects. Constraint-based routing then finds an explicit route that will satisfy these constraints among the set of available routes. Note that selecting an optimal route requires determining all possible routes for N + 1 TE trunks (assuming N existing TE trunks) and selecting the optimal set of routes in reestablishing the full set of TE trunks plus the new one requested. This is a task that is easily recognizable as NP-complete.

An example of use of constraint-based routing to satisfy a traffic engineering need is the attempt to move a portion of the traffic on a congested link to another link. Assigning the congested link to a resource class that would be treated as an undesirable characteristic of the desired route is a simple and direct way to represent the desired constraint. The traffic engineer defines a portion of the traffic that would normally traverse the congested link (possibly in terms of a set of destination addresses) and initiates the constraint-based routing process. The traffic engineer causes a set of ingress LSRs to each seek a new path that satisfies the constraint that it not use any link that is in the resource class associated with the undesirable (congested) link.

Although finding the optimal route using constraint-based routing is known to be computationally difficult for almost any realistic constraint-limited routing problem, a simple heuristic can be used to find a route satisfying a set of constraints-if one exists. The traffic engineer may simply prune resources that do not match the traffic trunk attributes and run a shortest-path route computation on the residual graph. Other approaches may be used as well.

Continuing the previous example, ingress LSRs prune the set of available links known to them (for example, as a result of using a link-state routing protocol) of all links belonging to the resource class of the congested link (possibly a "congested" resource class is defined for all such links). These ingress LSRs can then run a route computation (using the pruned link-state information) and establish explicit routes on the basis of their results. The ingress LSRs then use this explicit route solely for routing the portion of traffic defined. Because the ingress LSRs no longer route this traffic via the congested link, the congestion on that link would be reduced by an amount that may be as much as the amount of traffic associated with that defined portion of the traffic now being forwarded on the new explicit route.

These procedures, being heuristic in nature, will not necessarily find the optimal solution. In addition, successive applications of these approaches may lead to failure to find a route for one or more traffic trunks when all such traffic trunks could have been accommodated with an optimal solution. This implies that it will be necessary to tear down TE trunks at some point to avoid increasingly suboptimal constraint-based route determinations.

To perform the automated constraint-based routing computation in the example, the information provided by the link-state routing protocol must include information about the links that would allow ingress LSRs to determine what links satisfy which constraints. For example, when the congested link was assigned to a resource class, this assignment would have to be advertised in the link-state routing protocol in common use by LSRs in the TE domain. Support for constraint-based routing computations is currently being developed in the IGP routing protocols IS-IS (Intelligent Scheduling and Information System) and OSPF (Open Shortest Path First).

Path Establishment and Maintenance

The path used by a traffic trunk may be determined automatically by using traffic trunk attributes to either explicitly include or exclude network resources and then performing a path computation. This is referred to as constraint-based routing in RFC 2702 (Awduche et al. 1999). Once a path is determined, the path is established and maintained using the adaptivity and resiliency aspects of the path attribute as in the following subsections.

Use of Strictness of the Explicit Route

The traffic engineer computes an explicit route for use in establishing the traffic trunk. If strict explicit routing of the traffic trunk is not required, the traffic engineer can perform this task in the absence of perfect knowledge of the network. If strict routing is required, determination of the entire strict route is part of the computation process. If the traffic engineer starts with imperfect knowledge of the network topology, the LSP signaling process may be used as an aid in computing the explicit path. For example, the Record Route object may be used in RSVP-TE signaling for explicit route setup. Signaling of the explicit route is accomplished using either CR-LDP or RSVP-TE and including the Explicit Route object.

Use of the Adaptivity Aspect of the Path Attribute

Path reoptimization is controlled by the adaptivity subattribute of the path control and maintenance attributes. This aspect determines whether or not the LSP associated with a traffic trunk will be reoptimized as a result of changes to network resources. Control of this behavior is highly desirable because reoptimization itself is not always desirable. To understand why this is so, consider that there must be some reason why a new path is considered to be more optimal. Maybe there are more resources, or less congestion, associated with a new path. Consequently, it is reasonable to expect that packets may be delivered more quickly along the new path; however, this can cause trouble for specific applications that are sensitive to either delay variation or to ordering of packet delivery. For these applications, reoptimization is undesirable.

Adaptivity is preventable in signaling if (in CR-LDP, for example) it is possible to pin a route explicitly. An explicit route may also be pinned by being strictly routed at all hops. As described in the Piggyback Label Distribution Using RSVP section in Chapter 6, it is possible to use the Record Route object to determine the exact route currently being used by an LSP and then use this information to pin the LSP. Maintenance of a pinned explicit route is simpler because it is unnecessary to retain information required to reroute the LSP at every network element that might otherwise be required to do so.

Use of the Resilience Aspect of the Path Attribute

Reoptimization is distinct from resilience. A traffic trunk that is not subject to reoptimization can be required to be resilient to link and node failures along its established path. Resiliency is implicit for LSPs that are adaptive to reoptimization.

The resilience aspect can be broken into two parts: basic and extended resiliency. Basic resiliency determines whether a traffic trunk is subject to automatic rerouting as a result of a partial path failure (one or more segments fail). Extended resiliency determines the specific actions taken in response to a fault-for example, the order in which specified alternative paths are considered. Support for resilient behavior depends on interactions with underlying routing technology, both in detecting a fault and in selecting a new path.

Resilience at the local level is only possible if the original path was a loosely specified portion of an explicit route or if the fault is part of a segment where there is more than one strictly specified explicit route provided for this purpose.

Load Distribution Using TE Traffic Trunks

Being able to distribute traffic across multiple similar-cost LSPs between an ingress and egress pair is an important TE capability. For example, the aggregate traffic between the two nodes may be more than can be supported using any single path between the two nodes. Distributing the traffic as multiple substreams allows the system to provide forwarding that exceeds the limitations of single links in paths between the two nodes.

This distribution can be done using MPLS by establishing multiple LSPs (effectively as a single combined traffic trunk), each of which will carry a portion of the traffic for the combined traffic. To do this, however, the ingress LSR must be capable of assigning packets to each of the multiple LSPs in an intelligent fashion.

For example, assume two LSPs are established to carry the traffic from an ingress LSR to an egress LSR for the same aggregate traffic. One is expected to carry two-thirds of the traffic, whereas the other carries one-third. In this scenario, the ingress LSR must map corresponding portions of the incoming traffic aggregate to each LSP. It is desirable that this mapping be done in such a way as to ensure that packets that are part of the same source- destination flow follow the same LSP as a safeguard against out-of-order delivery.

Fault Handling

In general, four functions are associated with fault handling:

These functions are not necessarily performed in the order listed. For example, notification may need to occur before isolation can begin, and restoration may have begun before a fault was detected (for example, establishing a redundant circuit in anticipation of failure) and may in any case begin before notification takes place. In some technologies (for exampe, IP routing), detection and isolation are not separable functions.

Because TE uses explicitly routed LSPs, mechanisms intrinsic to the underlying routing infrastructure will not necessarily be sufficient for recovering from a fault, particularly in strictly routed (or pinned) portions of the LSP. Because (by default) routing is blind to the paths taken by an explicitly routed LSP, MPLS needs to provide separate mechanisms for detecting a fault in an LSP, notifying the ingress (especially if the fault is not locally repairable), and initiating restoration of service.

Because it is possible that MPLS is using technology that may provide some alternative fault recovery mechanisms, fault recovery mechanisms defined specifically for MPLS must be able to be disabled.

Fault recovery must also take into account the priority and precedence attributes of the traffic trunk.


This section describes signaling approaches for support of traffic engineering.


LDP may be used in a simple TE application in which a TE traffic trunk is desirable from one LSR to another, and a degenerate explicit route (in which only the egress is specified) is sufficient to satisfy TE requirements. In this case, the two LSRs may be assumed not to be directly connected, and an LSP tunnel is constructed between them. The specific mechanism used to accomplish this is that the ingress LSR establishes an LSP associated with an address prefix FEC and that matches the prefix length of an address of the egress LSR. The ingress LSR now maps traffic onto this LSP using the specific FEC defined for the corresponding traffic trunk.

This process may be extended to include additional LSPs in tandem. In this case, either the egress LSR is also the ingress to one or more further LSPs, the ingress LSR is egress to one or more LSPs, or both LSRs are both ingress and egress to LSPs. The LSPs in this discussion are LSPs for which there is a similar mapping of TE forwarding classes corresponding to a traffic trunk that uses two or more LSPs in tandem.

Because it is not possible to pin an LSP routed from an ingress to an egress LSR using LDP alone, a traffic trunk established using this approach is both adaptive and resilient by nature. Therefore, this approach cannot be used to establish traffic trunks for which either of these properties is undesirable.

In addition, it is not possible to explicitly assign resources from the path used for this approach via the LDP signaling protocol. If it is necessary to assign resources to a traffic trunk explicitly via the signaling protocol, some other approach must be used.

CR-LDP Explicit Routes

The difference between LDP and CR-LDP is that CR-LDP supports explicit routes and allocation of resources. CR-LDP support of TE traffic trunks is thus very similar to that provided by LDP, but without the restrictions that apply when LDP is used by itself.

Because CR-LDP has the Explicit Route object (and procedures to support its use), a traffic trunk LSP can be fully specified as a set of strict explicit hops. CR-LDP supports explicit pinning of an explicit route as well. CR-LDP also includes extensions to provide RSVP-like resource allocation in setting up explicitly routed LSPs.

RSVP Tunnels

RSVP-TE (Awduche et al. w.i.p.) defines procedures for use in establishing explicitly routed LSPs using standard RSVP messages with extensions. Extensions to the base RSVP protocol are defined as objects. These objects are opaque to RSVP speakers that are not MPLS enabled; however, support for piggyback label distribution using RSVP requires all participants to be MPLS enabled.

An explicit-route LSP is constructed using procedures defined in RSVP-TE and including an Explicit Route object in a Path message. Support for route pinning is provided by including the Record Route object in both Path and Resv messages and then including the Explicit Route object with a fully specified strict explicit route in all subsequent Path messages.

7.4 Virtual Private Networks

In general, the distinction between a virtual private network (VPN) and an actual private network (APN) is that in the VPN case, network resources are shared among multiple VPN instances while maintaining the notion of privacy between VPN instances. In an APN, privacy is a result of not sharing network resources with other private networks.

To effectively provide the illusion of a private network using shared resources, it is necessary to support private address spaces and to provide for separation of traffic by preventing leakage of traffic from one VPN to either another VPN or to the Internet and by providing some level of isolation of VPN traffic from the effects of traffic in other networks sharing the same resources.

Methods of isolating traffic from sharing effects (among VPN alternatives discussed to date) fall into one of four categories:


Numerous proposals for supporting VPNs using MPLS have been discussed within the MPLS working group. The section entitled Virtual Private Networks, Traffic Engineering, and Optimized Multipathing Draft Development in Chapter 2 provides a brief overview of specific drafts on the subject. No single approach has become a standard; however, there are two major contenders: VPNs using BGP and MPLS, and explicitly routed VPNs. It is most likely at present that BGP-MPLS VPNs will emerge as the commonly used approach, in part because it is the approach advocated by the current market leader.


RFC 2547 (Rosen and Rekhter 1999) is an informational RFC that describes how BGP and MPLS would be used to provide a VPN service. Unfortunately, this procedure is dependent on BGP-MPLS (Rekhter and Rosen 2000), which is not yet completely defined.

The essence of the procedure is that BGP is used to propagate VPN-specific routes to populate separate forwarding tables in the VPN service provider's network. RFC 2547 defines a provider edge (PE) router that must determine which forwarding table to use based on which customer edge (CE) router it was received from.

Some measure of scalability is achieved in this approach by limiting the distribution of VPN-specific routes to those PE routers that attach CE routers within the given VPN. In this way, each PE router only needs to maintain routes for CE routers to which it is directly attached.

Route distribution for VPN support using BGP is accomplished using BGP multiprotocol extensions (defined in RFC 2283 [Bates et al. 1998]) and a new Address Family Identifier (AFI) and Subsequent Address Family Identifier (SAFI)-1 and 128, respectively-that identify the VPN-IPv4 address family. Addresses from this address family are 12 bytes long and include an 8-byte route distinguisher (RD) and an IPv4 address (prefix). The mapping between RDs and specific VPNs is not guaranteed because an RD need only be unique to the PE set participating in a VPN and will vary across service provider domains. A PE determines which routes to distribute for a given VPN based on target VPN attributes that are associated with per-site VPN-specific forwarding tables. Association of target VPN attributes with specific sites is determined by configuration.

BGP-MPLS VPN routes are distributed using peer-to-peer BGP direct connections or connections via a route reflector. The BGP Update messages used to distribute these routes include MPLS labels corresponding to each route (using appropriate AFI/SAFI and address length values). Procedures and formats for carrying labels in a BGP Update message are defined in Rekhter and Rosen (w.i.p.) and described in Piggyback Label Distribution Using BGP, in Chapter 6. Setup and maintenance of an LSP between two PE routers that are not directly connected is accomplished using LDP, CR-LDP, or RSVP-TE, with or without explicit routes.

Explicitly Routed VPNs

Several approaches exist and have been proposed for supporting a VPN service using explicitly routed LSPs. This approach is essentially similar to creation of multiple instances of TE traffic trunk overlays.

Although most proposals are currently either entirely proprietary or based on proprietary extensions to a TE-based solution, there are several common distinctions between this general approach and BGP-MPLS VPNs. Some of the ways in which explicitly routed VPNs may differ from BGP-MPLS VPNs include the following:


Awduche, Daniel O., Lou Berger, Der-Hwa Gan, Tony Li, Vijay Srinivasan, and
George Swallow. RSVP-TE: Extensions to RSVP for LSP tunnels; a work in progress.
Awduche, Daniel O., Joe Malcolm, Johnson Agogbua, Mike O'Dell, and Jim McManus.
1999 (September). Requirements for traffic engineering over MPLS. RFC 2702. Available at
Bates, Tony, Ravi Chandra, Dave Katz, and Yakov Rekhter. 1998 (February).
Multiprotocol extensions for BGP-4. RFC 2283. Available at http://
Chang, C. S., and J. A. Thomas. 1995. Effective bandwidth in high-speed
networks. IEEE Journal of Selected Areas in Communication 13(6):1091-1100.
Fox, Barbara A., and Bryan Gleeson. 1999 (September). Virtual private
networks identifier. RFC 2685. Available at
Gleeson, Bryan, Arthur Lin, Juha Heinanen, Grenville Armitage, and Andrew
Malis. 2000 (February). A framework for IP based virtual private networks. RFC 2764. Available at
Guerin, R., H. Ahmadi, and M. Naghshineh. 1991. Equivalent capacity and
its application to bandwidth allocation in high-speed networks. IEEE Journal of Selected Areas in Communication 9(7):968-981.
Jamoussi, Bilel, ed. Constraint-based LSP setup using LDP; a work in
Le Faucheur, Francois, Liwen Wu, Bruce Davie, Shahram Davari, Pasi Vaananen,
Ram Krishnan, Pierrick Cheval, and Juha Heinanen. MPLS support of Differentiated Services; a work in progress.
Rekhter, Yakov, and Eric C. Rosen. 2000 (January). Carrying label
information in BGP-4, version 4. Internet Draft (draft-ietf-mpls-bgp4-mpls-04). Available at
Rosen, Eric C., and Yakov Rekhter. 1999 (March). BGP/MPLS VPNs. RFC
2547. Available at