Tuesday, 26 August 2014

Cisco Express Forwarding

  Cisco Express Forwarding (CEF) is a packet-switching technique that is the default for many of Cisco’s router over the last ten years. It provides the ability to switch packets through a device in a very quick efficient way while also keeping the load on the router’s processor low. This way the route process can be tasked with dealing with other duties that require larger amounts of processor time (Quality of Service, Encryption, etc.). This article takes a brief look at the different structures that are used by CEF and how they are built and interrelate.

Benefits


CEF offers the following benefits:

Improved performance—CEF is less CPU-intensive than fast switching route caching. More CPU processing power can be dedicated to Layer 3 services such as quality of service (QoS) and encryption.

Scalability—CEF offers full switching capacity at each line card when dCEF mode is active.

Resilience—CEF offers an unprecedented level of switching consistency and stability in large dynamic networks. In dynamic networks, fast-switched cache entries are frequently invalidated due to routing changes. These changes can cause traffic to be process switched using the routing table, rather than fast switched using the route cache. Because the Forwarding Information Base (FIB) lookup table contains all known routes that exist in the routing table, it eliminates route cache maintenance and the fast-switch or process-switch forwarding scenario. CEF can switch traffic more efficiently than typical demand caching schemes.


 Cisco Express Forwarding Concepts


Cisco Express Forwarding components:

Information conventionally stored in a route cache is stored in several data structures for CEF switching. The data structures provide optimized lookup for efficient packet forwarding. The two main components of CEF operation are described in the following sections:



CEF Operation Modes


CEF can be enabled in one of two modes described in the following sections:



Central CEF Mode


When CEF mode is enabled, the CEF FIB and adjacency tables reside on the RP, and the RP performs the express forwarding. You can use CEF mode when line cards are not available for CEF switching or when you need to use features not compatible with dCEF switching. 

Distributed CEF Mode


When dCEF is enabled, line cards, such as VIP line cards or GSR line cards, maintain an identical copy of the FIB and adjacency tables. The line cards perform the express forwarding between port adapters, relieving the RSP of involvement in the switching operation.

dCEF uses an Inter Process Communication (IPC) mechanism to ensure synchronization of FIB tables and adjacency tables on the RP and line cards.

 RIB, FIB, LFIB, Adjacency table
Control Plane - maintains routing information:
  • Routing Information Base(RIB/Routing table) operates in software - sh ip route
    • Directly connected
    • Static Routes
    • Dynamic routing information
  • ARP Table  sh arp
Data/Forwarding Plane - responsible for moving data in and out(ingress/egress):
  • Forwarding Information Base(FIB) - sh ip cef
    • Built from RIB best routes and the ARP table
    • RIB changes are reflected in FIB
  • Adjacency Table - sh adjacency
    • Contains L2 next hop information for all entries in FIB
  • Label Forwarding Information Base(LFIB)
    • Used for labeled packets(MPLS)
Adjacency Types:
  • Glean - FIB maintains a subnet prefix and needs additional ARP information for specific hosts
  • Null - Packets destined for Null0 are dropped(Bit Bucket/Black Hole/Discard/etc)
  • Drop - Device drops packets that can't be forwarded normally(by default generates ICMP unreachables)
    • Encapsulation Failure
    • Unresolved Address
    • Unsupported Protocol
    • No Valid Route
    • No Valid Adjacency
    • Checksum Error
  • Discard - Device discards packets based on policy(by default does not generate and ICMP unreachables)
    • Unassigned Loopback IP addresses that belong to the interface subnet but are unassigned are also discarded; for example lo0 is assigned 1.1.1.1 255.255.255.0, 1.1.1.2-1.1.1.254 will be discard adjacencies
  • Punt - CEF cannot forward the traffic, so packets are sent to the Control Plane(L3) for processing
CEF Load Balancing Hash

 CEF supports TCP/IP load balancing over parallel links, if there are multiple paths to the same  destination, the device will create a 16 row hash table with individual path pointers; sh ip cef <prefix> internal
  • per-destination mode(default) - All packets to a given destination follow the same path, preserving packet order.  Can cause unequal usage of the links if the majority of traffic is               destined for one host.
    • source and destination IP address is hashed and pointed to a specific entry in the adjacency table
  • per-packet mode(ip load-sharing per-packet) - Guarantees equal load across links, but may cause an out of order packet situation 
    • packets are distributed round robin across all paths
Polarization Concept and Avoidance

Polarization occurs when all devices use the same hash to reach the same destination.  Per-packet load sharing could be used as a solution, but due to the negative effect of out of order packets, it is not the preferred solution.
The preferred solution is to alternate between hashing inputs at each layer of the network.  The default load sharing is Source IP, Dest IP and unequal weights of the links
  • mls ip cef load-sharing simple - Source IP and Dest IP equal weights
  • mls ip cef load-sharing full - Source IP, Dest IP, and L4 port number unequal weights
  • mls ip cef load-sharing full simple - Source IP, Dest IP, and L4 port number equal weight

Differences between IOS and IOS XE





 Differences between IOS and IOS XE


Cisco IOS:

  • IOS is monolithic, completely adherent to the hardware, and does not provide any kind of isolation between “processes”, neither from a CPU nor memory point of view.
  • Virtual memory is shared by all IOS processes: nothing prevents buffer overflows.
  • Scheduler is non-preemptive: if SNMP decides it should keep CPU busy, it can, and other processes (BGP…) will be prevented from running.
  • You cannot upgrade IOS (or parts of it) without disruption unless you are running expensive dual-supervisor hardware.

Cisco IOS-XE:

IOS XE retains the exact same look and feel of IOS, changing in some commands due to his ability to be used in multi-core CPU providing enhanced features and improved functionality, high availability, distributed software architecture and  modularity.

Cisco IOS XE Software is a modular operating system built on a Linux kernel. The Linux kernel is designed to meet greater requirements for security and high availability. IOS XE includes a software module derived from the Cisco IOS Software. In IOS XE, IOS 15.0 runs as a single daemon within a modern Linux operating system.

  • Additional system functions now run as additional, separate processes in the host OS environment.
  • The actual IOS XE software comes in seven individual sub-packages (files) which are combined into a complete consolidated package (file). See below.

Allowing these routers to use Cisco IOS software redundancy, Cisco high-availability features, Nonstop Forwarding (NSF), and In Service Software Upgrades (ISSUs). This option requires for example the Cisco ASR 1000 Series Route Processor to have 4 GB of DRAM memory.



Cisco IOS XE Software and introduce a distributed software architecture that moves many operating system responsibilities out of the IOS process. In this architecture, Cisco IOS, which previously was responsible for almost all of the internal software processes, now runs as one of many Cisco IOS XE processes while allowing other Cisco IOS XE processes to share responsibility for running the router.

Software Packaging

A Cisco IOS XE Software image consists of seven individual modules (also referred to as subpackages). Individual modules or subpackages of a Cisco IOS XE Software image cannot be downloaded from Cisco.com individually. Example Individual subpackages can be extracted from the consolidated package from the Cisco ASR 1000 Series router command-line interface (CLI). Figure 1 illustrates how individual software subpackages are bundled in Cisco IOS XE Software.



 Cisco IOS-XE software use a new software packaging model consisting of:
 Consolidate package

 Individual software sub=packages within a consolidated package

Optional software sub=packages outside of consolidated packages

Each Cisco IOS XE consolidated package contains a collection of individual software sub-packages.

Each individual software sub-package is an individual software file that controls a different element or elements of the Cisco ASR 1000 Series Router. Some individual sub-packages may be installed per element (for example, per SPA).


Note: the sub-package functionality is intended for both upgrade and field support, and not all combinations of sub-packages are supported.


Each individual software sub-package can be upgraded individually, or all individual software

Sub-packages for a specific Cisco IOS XE consolidated package can be upgraded as part of a complete

Cisco IOS XE consolidated package upgrade.


Importantly, IOS (the RPIOS individual software sub-package) is considered one of the individual

software sub-packages that makes up the complete Cisco IOS XE consolidated package.




The following are the individual software sub-packages within a consolidated package:

Route Processor
RPBase: Provides the operating system software for the route processor
RPControl: Provides the control-plane processes that interface between Cisco IOS Software and the rest of the platform.

          RPIOS: Provides the Cisco IOS Software kernel, which is where Cisco IOS Software features

are stored and run; each consolidated image variant has a different RPIOS sub-package:

RPIOS-ipbase, RPIOS-ipbasek9, RPIOS-advipservices, RPIOS-advipservicesk9,

RPIOS-adventservices, and RPIOS-adventservicesk9.



-==============================================================

Note: The RPIOS-advipservices and RPIOS-adventservices sub-packages are only available beginning

with Cisco IOS XE Release 2.2.1 and later releases. These two sub-packages are not available

with Cisco IOS XE Release 2.1.2 and earlier releases.

-============================///-==================================

RPAccess: Provides components to manage enhanced router access functionality.(ssh)


ESPBase: Provides the ESP operating system and control processes and the ESP software.

 

SIPBase: Share interface processor (SIP) carrier card operating system and control processes.
SIPSPA: Provides the shared port adaptor (SPA) driver and associated field-programmable device (FPD) images



A Cisco IOS XE consolidated package allows users to upgrade all individual software sub-packages on

the router with a single Cisco IOS XE image download. The Cisco IOS XE consolidated packages available vary based on the Route Processor (RP1 or RP2) installed in the system and the Cisco IOS XE Release.





Note

  • Normally, the router boots from the single consolidated package which automatically loads each of the seven sub-packages into memory.
  • However, you can extract individual sub-packages yourself and specify which sub-packages you want loaded (maybe 5 instead of all 7).
  • When individual sub-packages are loaded “content from the RP is copied into memory on an as-needed basis only” which conserves memory.
  • The router can run at highest peak traffic load when configured to run using individual sub-packages.
  • IOS XE Software architecture



Why IOS XE?

The IOS feature set for routing and switching is unmatched in the industry, delivering functionality required for business critical applications. Preserving these advantages of IOS to our customers is critical for Cisco.

IOS XE retains the exact same look and feel of IOS, while providing enhanced future-proofing and improved functionality. In IOS XE, IOS 15.0 runs as a single daemon within a modern Linux operating system. Additional system functions now run as additional, separate processes in the host OS environment. The operation, support and management of IOS XE does not require re-training from classic IOS.

Multi-Core CPUs and SMP

Running IOS and other applications as separate processes also enables load balancing the multi-core CPU, allowing each process to use a different core. IOSd within the IOS XE environment supports multiple threads and multi-core CPUs.

Control Plane and Data Plane Separation

IOS XE introduces an opportunity to enable teams to now build drivers for new Data Plane ASICs outside the IOS instance and have them program to a set of standard APIs which in turn enforces Control Plane and Data Plane processing separation.

IOS XE accomplishes Control Plane / Data Plane separation through the introduction of the Forwarding and Feature Manager (FFM) and its standard interface to the Forwarding Engine Driver (FED). FFM provides a set of APIs to Control Plane processes. In turn, the FFM programs the Data Plane via the FED and maintains forwarding state for the system. The FED is the instantiation of the hardware driver for the Data Plane and is provided by the platform.

Platform Abstraction

Since, historically, IOS has served as an Operating System as well as providing the key Routing Infrastructure, there has always been an aspect of Platform Dependent (PD) and Platform Independent (PI) code within IOS. IOS XE allows the platform dependent code to be abstracted from a single monolithic image. By moving drivers outside of IOS, IOS XE enables a more purely PI-focused IOS process. This provides a more efficient software delivery model for both the core IOS team, as well as platform developers, since the software can be developed, packaged and released independently.

Application Integration

Prior to IOS XE, the only way to integrate functionality into an IOS product was to either port the functionality into the IOS operating system or run the functionality on a service blade outside of IOS. This model has fundamentally constrained how quickly Cisco can integrate homegrown features, products through acquisition, or third party applications.

IOS XE permits the integration of non-IOS applications through the following mechanisms:

• Standard Linux-based environment for hosting applications;

• Extending IOS functionality into peripheral applications through well-defined APIs exported via Linux-shared client libraries;

• Provide a robust management infrastructure called Common Management Enabling Technology (COMET) that allows for CLI, XML, SNMP, and HTTP-based management of integrated applications.

IOS XE Software Architecture

The IOS XE foundation is a POSIX environment along with Free and Open Source Software (FOSS) for the common drivers, tools and utilities needed to manage the system. In addition to the standard set of off-the-shelf drivers, IOS XE is comprised of a set of Cisco specific drivers and associated chassis/platform management modules.

On top of the base operating system and drivers, IOS XE provides a set of infrastructure modules which define how software is installed, how processes are started and sequenced, how high-availability and software upgrades are performed and, lastly, how the applications are managed from an operational perspective. The core application that runs on top of this new infrastructure is the IOS feature set. Cisco products immediately reap the benefits of an extensive feature set for routing and switching platforms that has been built into IOS over the years. Customers can expect the same features to be available and for them to perform and be managed in the exact same way as previous products.

Finally, the evolved IOS architecture is specifically designed to accommodate other applications outside of IOS. These applications can either be tightly integrated with IOS or they could run side-by-side with IOS with very little or no interactions. These applications can be upgraded or restarted independently of IOS. If an application does require services from IOS, it integrates with IOS through a set of client libraries called "service points". These service points generically extend IOS information and services to outside applications such that these services are not replicated or managed separately.

IOS XE CLI Differences

IOS XE looks and feels the same as the IOS that we all know. There is almost no change in the different feature configurations, making the migration and user experience consistent with IOS. If you know how to operate IOS today, you already know how to operate IOS XE. The only minor difference in the CLI, and some outputs, is due to the customization that reflects the process-oriented approach of IOS XE, and the ability to use a multi-core CPU.


-================FAQ-==================


Is Cisco IOS. XE another operating system at Cisco?

A. Absolutely not! Cisco IOS XE is a representation of the continuing evolution of Cisco IOS Software to support our next-generation platforms. Cisco IOS XE itself has been shipping on the ASR-1000 since 2008 and Cisco IOS XE 3 SG has been shipping on the new Catalyst. 4500-E Series since October 2010. It provides an improved software architectural strategy, while maintaining all the benefits and familiar manageability interface of the long IOS legacy.

Q. What are the benefits of IOS XE over IOS?

A. There are multiple benefits of the transition from IOS to IOS XE that the end-users will enjoy. IOS XE will help lower the Total Cost of Ownership of many Cisco solutions by offering enhanced services integration for enhanced functionality within the network. In addition, it supports multiple CPU cores, control plane and data plane separation, and platform abstraction.

Q. Do I need to train my staff or change anything in my management platform?

A. No, Cisco IOS XE looks like and is managed the same way as traditional Cisco IOS Software. Only a handful of commands have changed, such as "show processor" and "show memory," which had to be extended to account for the multicore CPUs that Cisco IOS XE now supports. Otherwise, if you know how to manage Cisco IOS Software, you know how to manage Cisco IOS XE

Q. What, if anything, does Cisco IOS XE share with Cisco IOS Software Release 15?
A. Cisco IOS XE contains Cisco IOS Release 15 within itself. Cisco IOS Software runs as a process within Cisco IOS XE in what is called the IOS daemon, or IOSd. While many of the infrastructure components have migrated from Cisco IOS itself to Cisco IOS XE-such as High Availability- the features of Cisco IOS Software are exactly the same running within Cisco IOS XE as they would be in a traditional Cisco IOS release. You may determine exactly which Cisco IOS Release 15 image that the IOS daemon is using by issuing a "show version running" on the console or by simply looking at the filename of the image.
Q. What is Cisco's long-term commitment to Cisco IOS XE?
A. Most next-generation platforms will be migrating to Cisco IOS XE over the coming months and years.
Q. Will my current switches and routers be upgraded to Cisco IOS XE?
A. No, in an effort to simplify the transition, Cisco IOS XE will only be introduced as new generations of hardware platforms are released. No in-service upgrade of an existing platform will be provided. Similarly, any platform that runs Cisco IOS XE will not support running Cisco IOS.
Q. How will features be shared between IOS and IOS XE?
A. Since Cisco IOS XE contains Cisco IOS within itself as IOSd, all features created within IOS will also appear in IOS XE and vice versa. Only new integrated services and functionality created outside of IOSd will not be shared with a Cisco IOS release. However, these integrated services may be introduced on a Cisco IOS platform through the use of Integrated Services daughter cards which will be available on a platform by platform basis.
Q. What services will be provided on Cisco IOS XE, and how open will this platform be to integrated third-party services and applications?
A. Services that were traditionally managed by standalone appliances or servers will now be integrated into the Cisco IOS XE environment. Examples today include Cisco Unified Border Element (CUBE) and Session Border Controller (SBC), but this will evolve over time.


           Cisco IOS XE Software Package Compatibility for ISSU


When upgrading the Cisco IOS XE operating system software using the In Service Software Upgrade

(ISSU) process, it is important to determine the compatibility of the upgraded software to your current software and hardware. The ISSU process allows software to be updated or otherwise modified while packet forwarding continues with minimal interruption.

Cisco IOS XE release compatibility using the ISSU process utilizes the SSO functionality to preserve
state while software versions on the router differ, as during an upgrade. Most SSO-capable features in

each Cisco IOS XE release are ISSU capable. ISSU is only supported if SSO is enabled in the configuration and the system is in a steady state (SSO ready state has been achieved). ISSU compatibility depends on the set of specific feature clients that are in use and whether they support ISSU. All ISSU upgrades include at least one IOS switchover operation. It is important to understand which features are in use and whether these features are ISSU compatible.

The Cisco ASR1006 Series Router is a hardware-redundant chassis. The hardware-redundant chassis hastwo ESP line cards and two RPs which exchange state using hardware links. The Cisco ASR1002 andASR1004 Series Routers are not hardware redundant, but are software-redundancy capable. The
non-redundant chassis has a single RP and a single ESP, but allows the operation of up to two IOS

processes on the single RP to exchange states locally.


Non-hardware-redundant chassis models (such as the Cisco ASR 1002 Router and Cisco ASR 1004

Router)—Supports ISSU only if the router is running in subpackage mode.

Hardware-redundant chassis models (such as the Cisco ASR 1006 Router)—Supports ISSU when

the router is running in sub-package mode or in consolidated package mode.


For a complete discussion about the ISSU upgrade process on the Cisco ASR 1000 Series Routers,including prerequisites and restrictions, see the “In Service Software Upgrade (ISSU)” chapter of the Cisco ASR 1000 Series Aggregation Services Software Configuration Guide.





Reference: Cisco