Internet Engineering Task Force Jan Novak Internet-Draft Benoit Claise Intended status: Informational Cisco Systems, Inc. Expires: 9 April, 2010 9 October, 2009 IP Flow Information Accounting and Export Benchmarking Methodology draft-novak-bmwg-ipflow-meth-04.txt Abstract This document provides methodology and framework for quantifying performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired and exported as the performance metric. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Novak Expires April, 2010 [Page 1] Internet-Draft Flow Monitoring Benchmarking October 2009 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Table of Contents 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Existing Terminology. . . . . . . . . . . . . . . . . . . 4 2.2 New Terminology . . . . . . . . . . . . . . . . . . . . . 4 3. Flow Monitoring Performance Metric. . . . . . . . . . . . . . 6 3.1 The Definition . . . . . . . . . . . . . . . . . . . . . .6 3.2 Device Applicability . . . . . . . . . . . . . . . . . . .6 3.3 Test Concept . . . . . . . . . . . . . . . . . . . . . . .6 3.4 The Measurement Procedure Overview . . . . . . . . . . . .8 3.5 Software Platforms . . . . . . . . . . . . . . . . . . . .9 3.6 Hardware Platforms . . . . . . . . . . . . . . . . . . . .9 4. Test Set Up. . . . . . . . . . . . . . . . . . . . . . . . . 10 4.1 Test Topology. . . . . . . . . . . . . . . . . . . . . . 10 4.2 Base DUT Set Up . . . . . . . . . . . . . . . . . . . . .12 4.3 Flow Monitoring Configuration . . . . . . . . . . . . . .13 4.4 Collector . . . . . . . . . . . . . . . . . . . . . . . .16 4.5 Packet Sampling . . . . . . . . . . . . . . . . . . . . .16 4.6 Frame Formats . . . . . . . . . . . . . . . . . . . . . .17 4.7 Frame Sizes . . . . . . . . . . . . . . . . . . . . . . .17 4.8 Illustrative Test Set-up Examples . . . . . . . . . . . .18 5. Flow Monitoring Throughput Measurement Methodology. . . . . .20 5.1 Normal Cache Mode . . . . . . . . . . . . . . . . . . . .20 5.2 Cache Overflow Mode . . . . . . . . . . . . . . . . . . .22 6. RFC2544 Measurements. . . . . . . . . . . . . . . . . . . . .24 6.1 Flow Monitoring Configuration . . . . . . . . . . . . . .24 6.2 Tests With Single Traffic Component . . . . . . . . . . .24 6.3 Tests With Two Traffic Components . . . . . . . . . . . .25 7. Flow Monitoring Accuracy . . . . . . . . . . . . . . . . . . 25 8. Evaluating Flow Monitoring Applicability . . . . . . . . . . 26 9. Miscellaneous Tests. . . . . . . . . . . . . . . . . . . . . 27 9.1 DUT Under Traffic Load . . . . . . . . . . . . . . . . . 27 9.2 In-band Flow Export. . . . . . . . . . . . . . . . . . . 27 9.3 Variable Packet Rate . . . . . . . . . . . . . . . . . . 28 9.4 Bursty Traffic . . . . . . . . . . . . . . . . . . . . . 28 9.5 Various Flow Monitoring Configurations . . . . . . . . . 28 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . .29 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . 29 12. Security Considerations . . . . . . . . . . . . . . . . . . 29 13. References. . . . . . . . . . . . . . . . . . . . . . . . . 30 13.1 Normative References . . . . . . . . . . . . . . . . . .30 13.2 Informative References . . . . . . . . . . . . . . . . .30 Appendix A: Report Format. . . . . . . . . . . . . . . . . . . .32 Novak Expires April, 2010 [Page 2] Internet-Draft Flow Monitoring Benchmarking October 2009 1. Introduction Monitoring of IP flows (Flow monitoring) on network devices is a widely used application that has numerous uses in both service provider and enterprise segments as detailed in the Requirements for IP Flow Information Export [RFC3917]. This document intends to provide a methodology for measuring Flow monitoring performance and provide network operators a framework for considering its impact to the network and network equipment. Flow monitoring is defined in the Architecture for IP Flow Information Export [RFC5470] and related IPFIX documents. What is the cost of enabling the IP Flow monitoring and export to a collector is a basic question that this document tries to answer. This document goal is a series of methodology specifications for the monitoring of Flow monitoring performance, in a way that is comparable amongst various implementations, various platforms, and vendors. Since Flow monitoring will in most cases run on network devices forwarding packets, methodology for RFC2544 measurements (with IPv6 and MPLS specifics defined in [RFC5180] and [MPLS] respectively) in the presence of Flow monitoring is also proposed here. The most significant parameter in terms of performance, is the rate at which IP flows are created and expired in the network devices memory and exported to a collector. Therefore, this document focuses on a methodology on how to measure the maximum IP flow rate that a network device can sustain without impacting the forwarding plane, without losing any IP flow information, and without compromising the IP flow accuracy. [RFC2544], [RFC5180] and [MPLS] specify benchmarking of network devices forwarding IPv4, IPv6 and MPLS [RFC3031] traffic, respectively. Even if this document specifies the Flow monitoring methodology for network devices forwarding IPv4, IPv6, and MPLS, the methodology stays the same for any traffic type. The only restriction is the actual Flow monitoring support. A variety of different network architectures exist that are capable of IPFIX support. As such, this document does not attempt to list the various white box variables (CPU load, memory utilization, TCAM utilization etc) that could be gathered as they do always help in comparison evaluations. A better understanding of the stress points of a particular device can be attained by this deeper information gathering and a tester may choose to gather additional information during the test iterations. Novak Expires April, 2010 [Page 3] Internet-Draft Flow Monitoring Benchmarking October 2009 2. Terminology The terminology used in this document is mostly based on [RFC5470], [RFC2285] and [RFC1242] as summarised in the section 2.1. The only new terms needed by this document are defined in the following section 2.2. 2.1 Existing Terminology Device Under Test (DUT) [RFC2285, section 3.1.1] Flow [RFC5470, section 2] Flow Key [RFC5470, section 2] Flow Record [RFC5470, section 2] Observation Point [RFC5470, section 2] Metering Process [RFC5470, section 2] Exporting Process [RFC5470, section 2] Exporter [RFC5470, section 2] Collector [RFC5470, section 2] Control Information [RFC5470, section 2] Data Stream [RFC5470, section 2] Flow Expiration [RFC5470, section 5.1.1] Flow Export [RFC5470, section 5.1.2] Throughput [RFC1242, section 3.17] Packet Sampling [RFC5476, section 2] 2.2 New Terminology 2.2.1 Cache Definition: Memory area held and dedicated by the DUT to store Flow Record information prior Flow Expiration 2.2.2 Cache Size Definition: The size of the Cache in terms of how many entries of Flow Records the Cache can hold Novak Expires April, 2010 [Page 4] Internet-Draft Flow Monitoring Benchmarking October 2009 Discussion: This term is typically represented as a configurable option in the particular Flow monitoring implementation. It MUST be known in order to define the tests circumstances properly. Its highest value will depend on the memory available in the network device. Measurement units: Number of Flow Records 2.2.3 Active Timeout Definition: For long-running Flows, the time interval after which the Metering Process expires a Flow Record from the Cache so that regular Flow updates are exported. Discussion: This term is typically represented as a configurable option in the particular Flow monitoring implementation. See section 5.1.1 of [RFC5470] for more detailed discussion. It MUST be known in order to define the tests circumstances properly. Measurement units: Seconds 2.2.4 Inactive Timeout Definition: The time interval after which the Metering Process expires a Flow Record from the Cache if no more packets belonging to that specific Flow are seen. Discussion: This term is typically represented as a configurable option in the particular Flow monitoring implementation. See section 5.1.1 of [RFC5470] for more detailed discussion. It MUST be known in order to define the tests circumstances properly. Measurement units: Seconds 2.2.5 Flow Export Rate Definition: Number of Flow Records that expire from the Cache (as defined by the Flow Expiration term) within a time interval and are exported to the Collector. Discussion: The Flow Export Rate is measured using Flow Export data collected at the Collector by counting the exported Flow Records. The Novak Expires April, 2010 [Page 5] Internet-Draft Flow Monitoring Benchmarking October 2009 obtained value is an average of the instant export rates over the measurement interval (see sections 5.1.3 and 5.2.3). The lowest possible measurement interval (if the requirement is to measure rather instant DUT load than average load) is limited by the export capabilities of the particular Flow monitoring implementation. Measurement units: Number of Flow Records per second 2.2.6 Cache Overflow Definition: Cache Overflow is a state of the Cache when all the available Flow Record entries in the Cache are already used/occupied e.g. there is no space in the Cache for new incoming Flow Records. 3. Flow Monitoring Performance Metric 3.1 The Definition Flow Monitoring Throughput Definition: The maximum Flow Export Rate the DUT can sustain without losing a single Flow Record expired from the Cache and without dropping any forwarded packets. The measured Flow Export Rate MUST include the Control Information. Measurement units: Number of Flow Records per second 3.2 Device Applicability The Flow monitoring performance metric is applicable to network devices that implement RFC 5470 [RFC5470] architecture. These devices can be network packet forwarding devices or appliances which analyse the traffic but do not forward traffic (probes, sniffers, replicators). The Flow monitoring performance metric is not applicable to the Collector since it does not implement the RFC 5470 architecture. 3.3 Test Concept Traffic in the Figure 1 represents the test traffic sent to the DUT and forwarded by the DUT. When testing devices which do not act as network devices (appliances - probes, sniffers, replicators) the forwarding plane is missing. Novak Expires April, 2010 [Page 6] Internet-Draft Flow Monitoring Benchmarking October 2009 The Flow monitoring enabled (see section 4.3) on the DUT (and represented in the Figure 1 by the Flow Monitoring Plane) uses the traffic information provided by the Forwarding Plane and configured Flow Keys to create the Flow Records representing the traffic forwarded by the DUT. The Flow Records are stored in the Flow monitoring Cache and expired from there depending on the Cache configuration (Active and Inactive Timeouts, number of Flow Records and the Cache Size) and the traffic pattern. The expired Flow Records are exported from the DUT to the Collector (see Figure 2 and Figure 3 in section 4). +-------------------------+ | IPFIX | Netflow | Others| +-------------------------+ | ^ | | ^ | | Flow Export | | ^ | | ^ | | +-------------+ | | | Flow | | | | Monitoring | | | | Plane | | | +-------------+ | | ^ | | ^ | | traffic information | | ^ | | ^ | | +-------------+ | | | | | traffic ---|---->| Forwarding |-----|----> | | Plane | | | +-------------+ | | | | DUT | +-------------------------+ Figure 1. The functional block diagram of the DUT The Forwarding Plane and Flow Monitoring Plane represent two separate functional blocks, each with it?s own performance capability. The Forwarding Plane handles user data packets and is fully characterised by the metrics defined by [RFC2544]. The Flow Monitoring Plane handles Flow Records which reflect the forwarded traffic. The metric that measures the Flow monitoring activity is Flow Export Rate. Novak Expires April, 2010 [Page 7] Internet-Draft Flow Monitoring Benchmarking October 2009 3.4 The Measurement Procedure Overview The measurement procedure is fully specified in sections 4, 5 and 6. This section provides an overview of principles for the measurements. The basic measurement procedure of performance characteristics of a DUT with Flow monitoring enabled is a conventional non drop rate measurement with gradually increasing traffic rates until losses are seen as fully specified by [RFC2544]. DUT with Flow monitoring enabled contains two functional blocks which need to be measured using characteristics applicable to one or the other block (see Figure 1). See sections 3.4.1 and 3.4.2 for further discussion. On one hand the Flow Monitoring Plane and Forwarding Plane (see Figure 1) need to be looked at as two independent blocks (and the performance of each of them measured independently) but on the other hand when measuring the performance of one of them the status and conditions of the other one must be known and monitored. 3.4.1 Flow Monitoring Plane Performance Measurement The Flow Monitoring Throughput MUST be (and can only be) measured with one packet per Flow as specified in the section 5. This traffic type represents the most aggressive traffic from the Flow monitoring point of view and will exercise the Flow Monitoring Plane (see Figure 1) of the DUT most. The exit criteria for the Flow Monitoring Throughput measurement are one of the following (e.g. if any of the conditions is reached): a. The Flow Export Rate at which the DUT starts to drop Flow Records or the Flow information gets corrupted b. The Flow Export Rate at which the Forwarding Plane starts to drop or corrupt packets 3.4.2 Forwarding Plane Performance Measurement The Forwarding Plane (see Figure 1) performance metrics are fully specified by [RFC2544] and MUST be measured accordingly. A detailed traffic analysis (see the bullets below) with relation to Flow monitoring must be performed prior of any [RFC2544] measurement. Mmainly the Flow Export Rate caused by the test traffic during an [RFC2544] measurement MUST be known and noted. The required traffic analysis mainly involves the following: a. Which packet header parameters are incremented or changed during traffic generation b. Which Flow Keys the Flow monitoring configuration uses to generate Flow Records Novak Expires April, 2010 [Page 8] Internet-Draft Flow Monitoring Benchmarking October 2009 The RFC2544 performance metrics can be measured in one of the two modes: a. At certain level of Flow monitoring activity specified by a Flow Expiration Rate lower than Flow Monitoring Throughput b. At the maximum of Flow monitoring performance ? e.g. using traffic conditions representing a measurement of Flow Monitoring Throughput The details how to setup the above mentioned measurement modes are in the section 6. 3.5 Software Platforms On purely software based DUTs with no hardware assisted functionalities, the measured Flow Monitoring Throughput will be numerically equal to the RFC2544 Throughput. This is due to the fact that the DUT resources are fully shared between the two functional blocks (see Figure 1). At the maximum point of the performance measurement the DUT will become short of resources to process packets and since every packet represents in the Flow Monitoring Throughput measurement also one Flow, at the moment one packet is lost, one Flow is lost. On a software platform the Flow Monitoring Plane and Forwarding Plane are functionally independent but their performance is coupled together due to the shared resources for packets and Flow Record processing. 3.6 Hardware Platforms On a hardware based DUT, where packet forwarding and possibly other functions are assisted by specialised hardware, the Flow Monitoring Plane and Forwarding Plane may not only be functionally but also performance wise independent (if the two functional blocks do not share any resources). The possible architectures of hardware based DUTs can be so diverse which makes it impossible to provide any advice on expected DUT behaviour. The Flow Monitoring Plane and Forwarding Plane must be treated as two independent blocks and measured independently. The most typical outcome of a measurement here will be totally independent values of Flow Monitoring Throughput and RFC2544 Throughput depending on which part of the functionality is implemented in hardware and which in software. The difference and need for Flow Monitoring Throughput on top of the RFC2544 Throughput is crucial here. Novak Expires April, 2010 [Page 9] Internet-Draft Flow Monitoring Benchmarking October 2009 4. Test Set Up This section concentrates on the set-up of all components necessary to perform Flow monitoring performance testing. 4.1 Test Topology The test topology described in this section is applicable only to the tests with packet forwarding network devices. The possible architectures and implementation of the traffic monitoring appliances (see section 3.2) are too various to be covered in this document. Generally, those appliances instead of the Forwarding Plane will have some kind of feed (an optical splitter, an interface sniffing traffic on a shared media or an internal channel on the DUT providing a copy of the traffic) providing the information about the traffic necessary for Flow monitoring analysis. The test topology then needs to be adjusted to the appliance architecture. The test set-up is identical to the one used by [RFC2544], with the addition of a Collector to analyse the Flow Export: +-----------+ | | | Collector | | | |Flow Record| | analysis | | | +-----------+ ^ | Flow Export | | Export Interface +--------+ +-------------+ +----------+ | | | | | | | | (*)| | | receiver | | sender |-------->| DUT |--------->| | | | | | | traffic | | | | | | analysis | +--------+ +-------------+ +----------+ Figure 2 Test topology with unidirectional traffic In the test topology with unidirectional traffic, the traffic is generated from the sender to the receiver, where the received traffic is analyzed to check it is identical to the generated traffic. The ideal way to implement the test is using one traffic generator (device providing the sender and receiver capabilities) with a sending port and a receiving port. This allows for an easy check if all the traffic sent by the sender was transmitted by the DUT and received at the receiver. Novak Expires April, 2010 [Page 10] Internet-Draft Flow Monitoring Benchmarking October 2009 In the test topology with bidirectional traffic (Figure 3), the sender and receiver are basically on the same device, which generates the traffic to be sent through the DUT, and received back once it reaches the ?END?. In the default configuration, the DUT interface marked with (*) MUST be configured with Metering Process analysing the ingress traffic and the DUT interface marked with (**) MUST be configured with Metering Process analysing the egress traffic. +-----------+ | | | Collector | | | |Flow Record| | analysis | | | +-----------+ ^ | Flow Export | | Export Interface +----------+ +-------------+ +----------+ | | (*)| | | | | sender |------->| |-------->| receiver | | | | DUT | | END | | | (**)| | | | | receiver |<-------| |<--------| sender | | END | | | | | +----------+ +-------------+ +----------+ Figure 3 Test topology with bidirectional traffic Novak Expires April, 2010 [Page 11] Internet-Draft Flow Monitoring Benchmarking October 2009 The bidirectional traffic test can also be performed with the use of one ingress and egress interface in the full duplex operational mode: +-----------+ | | | Collector | | | |Flow Record| | analysis | | | +-----------+ ^ | Flow Export | +----------+ +-------------+ +----------+ | | (*)| | | | | sender |<------>| |<------->| receiver | | | (**)| DUT | | END | | | | | | | | receiver | | | | sender | | | | | | | +----------+ +-------------+ +----------+ Figure 4 Test Topology with Full Duplex Interfaces In both the test topology with unidirectional traffic and bidirectional traffic, the export interface (connecting the Collector) MUST NOT be used for forwarding the test traffic but only for the Flow Export data containing the Flow Records. In all tests, the export interface MUST have enough bandwidth to transmit Flow Export data without congestion. In other words, the export interface MUST NOT be a bottleneck during the test. Note that more complex topologies might be required. For example, if the effects of enabling Flow monitoring on several interfaces are of concern or the media maximum speed is less that the DUT throughput, the topology can be expanded with several input and output ports. However, the topology MUST be clearly written in the test report. 4.2 Base DUT Set Up The base DUT set-up and the way the set-up is reported in the test results is fully specified in Section 7 of [RFC2544]. The base DUT configuration might include other features like packet filters or quality of service on the input and/or output interfaces if there is the need to study Flow monitoring in the presence of those features. The Flow monitoring measurement procedures do not change in this case. Consideration needs to be made when evaluating measurements results to take into account the possible change of packets rates offered to the DUT and Flow monitoring after Novak Expires April, 2010 [Page 12] Internet-Draft Flow Monitoring Benchmarking October 2009 application of the features to the configuration. Any such feature configuration MUST be part of the test report. 4.3 Flow Monitoring Configuration This section covers all the aspects of the Flow monitoring configuration necessary on the DUT in order to perform Flow monitoring performance testing. The necessary configuration has number of components (see [RFC5470]), namely Observation Points, Metering Process and Exporting Process as detailed in the sections below. The DUT MUST support Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] for easier results comparison. The DUT configuration and any existing Cache MUST be erased before application of any new configuration for the currently executed test. 4.3.1 Observation Points The Observation Points specify the interfaces and direction where the Flow monitoring traffic analysis is performed. The (*) and (**) in figures 2, 3 and 4 designate the Observation Points in the default configuration. Other DUT Observation Points might be configured depending on the specific testing needs as follows: a. ingress port/ports(s) only b. egress port(s) /ports only c. both ingress and egress Generally, the placement of Observation Points depends upon the position of the DUT in the deployed network and the purpose of Flow monitoring deployment. See [RFC3917] for detailed discussion. The testing procedures are otherwise the same for all these possible configurations. The specific Observation Points and associated monitoring direction MUST be included as part of the report of the results. 4.3.2 Metering Process Metering Process MUST be enabled in order to create the Cache in the DUT and configure the Cache related parameters. Cache Size available to the DUT operation MUST be known and taken into account when designing the test as specified in the section 5. Novak Expires April, 2010 [Page 13] Internet-Draft Flow Monitoring Benchmarking October 2009 Inactive and Active Timeouts MUST be known and taken into account when designing the test as specified in the section 5. The Cache Size, the Inactive and Active Timeouts, and if present, the specific Packet Sampling techniques and associated parameters MUST be included as part of the results report. 4.3.3 Exporting Process Exporting Process MUST be configured in order to export the Flow Record data to the Collector. Exporting Process MUST be configured in such a way that all Flow Records from all configured Observation Points are exported towards the Collector, after the expiration policy composed of the Inactive and Active Timeouts and Cache Size. The Exporting Process SHOULD be configured with IPFIX [RFC5101] as the protocol to use to format the Flow Export data. If the Flow monitoring implementation does not supported, proprietary protocols MAY be used. The measurement time to be used to capture Flow Export data and to calculate Flow Export Rate is defined in the sections 5.1 and 5.2. Various Flow monitoring implementations might use different default values regarding the export of Control Information. The Flow Export corresponding to Control Information SHOULD be analysed and reported as a separate item on the test report. IPFIX documents [RFC5101] in section 10 and [RFC5470] in section 8.1 discuss the possibility to deploy various transport layer protocols to deliver Flow Export data from the DUT to the Collector. The selected protocol MUST be included in the test report. Only benchmarks with same transport layer protocol SHOULD be compared. If the Flow monitoring implementation allows to use all of UDP, TCP and SCTP as the transport layer protocols, each of the protocols SHOULD be tested in a separate test run. 4.3.4 Flow Records Flow Record defines the traffic parameters which Flow monitoring uses to analyse the traffic and MUST be configured in order to perform the analysis. The Flow Record definition is implementation specific. A Flow monitoring implementation might allow for only fixed Flow Record definition, based on the most common IP parameters in the IPv4 or IPv6 headers - like source and destination IP addresses, IP protocol numbers or transport level port numbers. Another implementation might allow the user to actually define his own Novak Expires April, 2010 [Page 14] Internet-Draft Flow Monitoring Benchmarking October 2009 completely arbitrary Flow Record to monitor the traffic. The requirement for the tests defined in this document is only the need for a large number of Flow Records in the Cache. The Flow Keys needed to achieve that will typically be source and destinations IP addresses and transport level port numbers. Recommended full IPv4, IPv6 or MPLS Flow Record: Flow Keys Source IP address Destination IP address MPLS label (for MPLS traffic type only) Transport layer source port Transport layer destination port IP protocol number (IPv6 next header) IP type of service (IPv6 traffic class) Other fields Packet counter Byte counter If the Flow monitoring allows for user defined Flow Records the minimal Flow Record configurations allowing to achieve large numbers of Cache entries for example are: Flow Keys Source IP address Destination IP address Other fields Packet counter or: Flow Key fields Transport layer source port Transport layer destination port Other fields Packet counter The Flow records MUST be clearly noted in the test report. The Flow Monitoring Throughput measurements on different DUTs or different Flow monitoring implementations can and MUST be compared only for exactly same Flow Record configuration. 4.3.5 MPLS Test Specifics The Flow Record configuration for measurements with MPLS encapsulated traffic SHOULD contain MPLS label or any other field which is part of the MPLS header. Novak Expires April, 2010 [Page 15] Internet-Draft Flow Monitoring Benchmarking October 2009 The DUT Cache SHOULD be checked prior the performance test to contain the correct MPLS related information. The captured export data at the Collector SHOULD be checked for the presence of MPLS labels or the monitored MPLS parameters. MPLS forwarding performance document [MPLS] specifies number of possible MPLS label operations to test. The Observation Points SHOULD be placed on all the DUT test interfaces where the particular MPLS label operation takes place. The performance tests SHOULD be performed with only one MPLS label operation at the time. The DUT SHOULD be configured in such a way, that all the traffic is subject of the tested MPLS label operation. 4.4 Collector The Collector is needed in order to capture the Flow Export data which allow the Flow Monitoring Throughput to be measured. The Collector can be used as exclusively capture device providing just hexadecimal format of the Flow Export data. In such a case it does not need to have any additional Flow Export decoding capabilities. However if the Collector is also used to decode the Flow Export data then it SHOULD support IPFIX [RFC5101] for easier results analysis. If proprietary Flow Export is deployed, the Collector MUST support it otherwise the Flow Export data analysis is not possible. The Collector MUST be capable to capture at the full rate the export packets sent from the DUT without losing any of them. During the analysis, the Flow Export data needs to be decoded and the received Flow Records counted. The Collector SHOULD support Ethernet type of interface to connect to the DUT but any media which allows data capturing and analysis can be used. The capture buffer MUST be cleared at the beginning of each measurement. 4.5 Packet Sampling A Flow monitoring implementation might provide the capability to analyse the Flows after Packet Sampling is performed. The possible procedures and ways of Packet Sampling are described in [RFC5476] and [RFC5475] and only those SHOULD be used for measurements. If the DUT is configured with one of the sampling techniques as specified in [RFC5475] the test report MUST include this sampling Novak Expires April, 2010 [Page 16] Internet-Draft Flow Monitoring Benchmarking October 2009 technique along with its parameters. The configured sampling technique on the DUT and its parameters SHOULD be verified in the Flow Export data as received on the Collector. Packet Sampling will affect the measured Flow Export Rate. If systematic sampling (see section 6.5 of [RFC5476]) is in use, the Flow Export Rate can be derived from the packet rates (see section 5 of this document) using the configured sampling parameters. If random sampling is in use the Flow Export Rate can be derived from the traffic rates as obtained on the receiver side of the traffic generator, provided that packet losses can be excluded by monitoring the DUT forwarding statistics. If tests are performed with Flows containing more than one packet per Flow (see section 6.2 of this document) the sampling ratio SHOULD always be higher than the number of packets in the Flows (for small number of packets per Flow). This significantly decreases the probability of erasing a whole Flow to a minimum and the measured Flow Expiration Rate stays unaffected by sampling. If Flow accuracy analysis (see section 7) is performed, the results will be always affected by Packet Sampling and the complete check of data cannot be performed. This document does not intend to study the effects of Packet Sampling itself on the network devices but Packet Sampling can simply be applied as part of the Flow monitoring configuration on the DUT and perform the measurements as specified in the later sections. Consideration needs to be made when evaluating measurements results to take into account the change of packet rates offered to the DUT and especially to Flow monitoring after Packet Sampling is applied. 4.6 Frame Formats Flow monitoring itself is not dependent in any way on the media used on the input and output ports. Any media can be used as supported by the DUT and the test equipment. The most common transmission media and corresponding frame formats (Ethernet, Packet over Sonet) for IPv4, IPv6 and MPLS traffic are specified within [RFC2544], [RFC5180] and [MPLS]. 4.7 Frame Sizes Frame sizes to use are specified in [RFC2544] section 9 for Ethernet type interfaces (64, 128, 256, 1024, 1280, 1518 bytes) and in [RFC5180] section 5 for Packet over Sonet interfaces (47, 64, 128, 256, 1024, 1280, 1518, 2048, 4096 bytes). Novak Expires April, 2010 [Page 17] Internet-Draft Flow Monitoring Benchmarking October 2009 When testing with large frame sizes care needs to be taken to avoid any packet fragmentation on the DUT interfaces which could negatively affect measured performance values. 4.8 Illustrative Test Set-up Examples The below examples represent only hypothetical test set-up to clarify the use of Flow monitoring parameters and configuration together with traffic parameters to test Flow monitoring. The actual benchmarking specifications are in the sections 5 and 6. 4.8.1 Example 1 - Inactive Timeout Flow Expiration The traffic generator sends 1000 packets per second in 10000 defined streams, each stream identified by an unique destination IP address. Each stream has then packet rate 0.1 packets per second. The packets are sent in a round robin fashion (stream 1 to 10000) while incrementing the destination IP address with each sent packet. The configured Cache Size is 20000 Flow Records so that Flow Expiration does not happen due to the Cache Overflow. The configured Active Timeout is 100 seconds, the Inactive Timeout is 5 seconds. Flow monitoring on the DUT uses the destination IP address as Flow Key. A packet with destination IP address equal to A is sent every 10 seconds, so it means that the Flow Record is refreshed in the Cache every 10 seconds, while the Inactive Timeout is 5 seconds. In this case the Flow Records will expire from the Cache due to the Inactive Timeout and when a new packet is sent with the same IP address A it will create a new Flow Record in the Cache. The measured Flow Export Rate in this case will be 1000 Flow Records per second since every single sent packet will always create a new Flow Record and we send 1000 packets per second. The expected number of Flow Record entries in the Cache during the whole measurement is around 5000 ? it corresponds to the Inactive Timeout being 5 seconds and during those five seconds 5000 entries are created. This particular set-up would allow for the benchmark test as specified in the section 5.1. 4.8.2 Example 2 - Active Timeout Flow Expiration The traffic generator sends 1000 packets per second in 100 defined streams, each stream identified by an unique destination IP address. Each stream has then packet rate 10 packets per second. The packets are sent in a round robin fashion while incrementing (stream 1 to 100) the destination IP address with each sent packet. Novak Expires April, 2010 [Page 18] Internet-Draft Flow Monitoring Benchmarking October 2009 The configured Cache Size is 1000 Flow Records so that Flow Expiration does not happen due to the Cache Overflow. The configured Active Timeout is 100 seconds, the Inactive Timeout is 10 seconds. Flow monitoring on the DUT uses as Flow Key the destination IP address. After first 100 packets sent, 100 Flow Records are created and placed in the Flow monitoring Cache. The subsequent packets will be counted against the already created Flow Records since the destination IP address (Flow Key) has already been seen by the DUT (provided the Flow Record did not expire yet ? see below). A packet with destination IP address equal to A is sent every 0.1 second, so it means that the Flow Record is refreshed in the Cache every 0.1 second, while the Inactive Timeout is 10 seconds. In this case the Flow Records will not expire from the Cache until the Active Timeout, e.g. they will expire every 100 seconds and then the Flow Records will be created again. If the test measurement time is 50 seconds from the start of the traffic generator the measured Flow Export Rate is 0 since during this period no Flow Records expired from the Cache. If the test measurement time is 100 seconds from the start of the traffic generator measured Flow Export Rate is 1 Flow Record per second. If the test measurement time is 290 seconds from the start of the traffic generator measured Flow Export Rate is 2/3 of Flow Record per second since during the 290 seconds period we expired 2 times the same 100 of Flows. 4.8.3 Example 3 - Cache Overflow Flow Expiration The traffic generator sends 10000 packets per second in 10000 defined streams, each stream identified by an unique destination IP address. Each stream has then packet rate 1 packet per second. The packets are sent in a round robin fashion while incrementing the destination IP address with each sent packet. Flow monitoring on the DUT uses as Flow Key the destination IP address. The configured Cache Size is 1000 Flow Records. The number of Cache entries exceeds the configured Cache Size after first 1000 packets sent (provided the Flow Records do not expire due to timeouts ? see below) ? this condition is called Cache Overflow. The configured Active Timeout is 100 seconds, the Inactive Timeout is 10 seconds. Novak Expires April, 2010 [Page 19] Internet-Draft Flow Monitoring Benchmarking October 2009 The Cache will be full after the traffic generator sends first 1000 packets which will happen in 0.1 second so the Flow Records can never expire from the Cache due to any of Active or Inactive Timeouts. Each packet sent after this initial period of 0.1 second will create one new Flow Record. The measured Flow Export Rate in this case will be 10000 Flow Records per second since every single sent packet will always create a new Flow Record and cause expiration of another Flow Record. To perform this kind of measurement the Active and Inactive Timeouts must always be larger than the time needed to fill up the Cache. 5. Flow Monitoring Throughput Measurement Methodology Objective: To measure the Flow monitoring performance which can be used to compare different implementations. Metric definition: Flow Monitoring Throughput ? see section 3. Flow Export Rate calculation: The Flow Export Rate is obtained from the data captured at the Collector (section 4.4) as the number of Flows seen divided by the measurement time as specified in the section 5.1 and 5.2 The Flow Export Rate the DUT can achieve without losing any Flow information depends on the combination of a number of Flow monitoring parameters and traffic patterns offered to the DUT. This makes it a complex task to define and measure a single metric which would characterise the Flow monitoring performance. There are two Flow monitoring operational modes which need to be distinguished and measured separately while using the same metric. 5.1 Normal Cache Mode In the normal cache mode the DUT MUST hold less Flow Records in the Cache than the available Cache Size during the whole test. Test Feasibility: The DUT Cache status MUST be monitored during this test. If the number of Flow Records starts to grow during the test and approach the Cache Size testing MUST be stopped. The reported Flow Monitoring Throughput for normal cache mode is the highest possible Flow Export Rate at which the number of Flow Records in the Cache stays constant or marginally increases during the measurement. Novak Expires April, 2010 [Page 20] Internet-Draft Flow Monitoring Benchmarking October 2009 The reason for this test mode is, that the Flow monitoring implementation might choose to use different Flow Expiration mechanism as opposed to the test mode specified in section 5.2 and the results for the two modes can be quite different. 5.1.1 Flow Monitoring Configuration Parameters Cache Size: Maximum available value on the network device Inactive Timeout Inactive Timeout is set to the minimum possible value on the network device Active Timeout Active Timeout is set to higher or equal value than the Inactive Timeout Flow Keys Definition: Needs to allow for large numbers of unique Flow Records to be created in the Cache by incrementing values of one or several Flow Keys. The number of unique combinations of Flow Keys MUST be larger than the DUT Cache Size or larger than the product of the offered packet rate and the Inactive Timeout so that the Flow Records in the Cache never get updated before Flow Expiration and Flow Export. 5.1.2 Traffic Configuration Parameters Traffic Generation The traffic generator (sender), when sending the packets, needs to increment the packet header values defined as Flow Keys values in the Flow Record with each sent packet. Each packet then represents one Flow Record in the DUT Cache and the packet rate equals to the Flow Export Rate. Maximum Packet Rate for Normal Cache Measurement The maximum packet rate which can be used for the normal cache mode measurement is the Cache Size divided by the Inactive Timeout. If the Flow monitoring implementation allows to configure Inactive Timeout 0 then any packet rate is possible to be used. This makes sure the Flow Records expire and get exported from the Cache before it becomes full. Test Duration The test duration MUST be longer than Inactive Timeout and SHOULD comply with section 24 of [RFC2544]. Novak Expires April, 2010 [Page 21] Internet-Draft Flow Monitoring Benchmarking October 2009 5.1.3 Measurement Time The measurement time to be used to calculate Flow Export Rate is equal to the time interval when traffic is offered to the DUT. The Collector MUST continue capturing export data at least for the Inactive Timeout period after the traffic generation is stopped. This makes sure that all the Flow Records created in the DUT Cache will be exported and analysed. 5.1.4 Procedure The measurement procedure is similar to the Throughput measurement in the section 26.1 of [RFC2544] for the traffic sending side. However, the output analysis is not done on the DUT receiving side but on the collector as specified in section 4.4. The analysed quantity is the Flow Record count. If the Maximum Packet Rate (as specified in the section 5.1.2) does not cause the DUT to drop any Flow Record then the reported Flow Monitoring Throughput SHOULD be equal to the Maximum Packet Rate. The limitation here is that the configurable values of Cache Size and Inactive Timeout do not allow to test with higher Flow Expiration Rates in normal Cache mode. 5.2 Cache Overflow Mode In the Cache Overflow mode the DUT MUST have the Cache fully occupied by Flow Records at any time during the measurement. Test Feasibility: The Flow monitoring implementation might choose to react to the state of Cache Overflow in two possible manners: a. Forcibly Export Flow Records from the Cache to create free space for new records. Such an implementation can be fully tested using the procedures in this section b. the Flow Records are dropped when they no longer fit into the Cache. Such an implementation cannot be tested in the Cache Overflow mode. 5.2.1 Flow Monitoring Configuration Parameters Cache Size Cache Size configuration is dictated by the expected position in the network of the DUT and the unique Flow Keys of the flow record. The number of unique Flow Keys sets that the traffic generator (sender) provides should be multiple times larger than the Cache Size. This way the Flow Records in the Cache never get updated before Flow Expiration and Flow Export. Novak Expires April, 2010 [Page 22] Internet-Draft Flow Monitoring Benchmarking October 2009 Inactive Timeout Inactive Timeout is set to the maximum possible value on the network device. The value MUST be larger than the Cache Size divided by the test packet rate offered to the DUT to make sure the Flow Records do not expire from the Cache before the number of Flow Records in the Cache reaches the Cache Size. Active Timeout Active Timeout is set to higher or equal value than the Inactive Timeout Flow Keys Definition: Needs to allow for large numbers of unique Flow Records to be created in the Cache by incrementing values of one or several Flow Keys. The number of unique combinations of Flow Keys values SHOULD be at least two times larger than the DUT Cache Size. This makes sure that any incoming packet will never refresh any already existing Flow Record in the Cache. 5.2.2 Traffic Configuration Parameters Traffic Generation The traffic generator needs to increment the Flow Keys values with each sent packet, this way each packet represents one Flow in the DUT Cache and the packet rate equals to the Flow Export Rate. Test Duration The test duration MUST be longer than the time needed to fully fill up the Cache with Flow Records. 5.2.3 Measurement Time The measurement time to be used to calculate Flow Export Rate equals to the time interval when traffic is offered to the DUT minus the time needed to fully fill up the Cache. The time to fill up the Cache can be calculated as Cache Size divided by the test traffic packet rate during the measurement. The Collector MUST stop capturing export data immediately when the traffic generation is stopped. This makes sure that the Flow Records exported later from the fully occupied Cache do not poison the data used to measure Flow Monitoring Throughput. 5.2.4 Procedure The measurement procedure is same as the Throughput measurement in the section 26.1 of [RFC2544] for the traffic sending side. The output analysis is not done on the DUT receiving side but the analysed quantity for the measurement of the maximum rate value is the Flow Record count as derived from the Flow Export data captured at the Collector as specified in the section 4.4. Novak Expires April, 2010 [Page 23] Internet-Draft Flow Monitoring Benchmarking October 2009 6. RFC2544 Measurements Objective: Provide RFC2544 network device characteristics in the presence of Flow monitoring on the DUT. The RFC2544 studies numerous characteristics of network devices. The purely forwarding characteristics without Flow monitoring present on the DUT can significantly vary when Flow monitoring starts to be deployed on the network device. The objective of the section is to define controlled environment for Flow monitoring during all the tests with all the parameters as specified below included in the test report. Metric definition: Metric as specified in [RFC2544]. 6.1 Flow Monitoring Configuration Flow monitoring configuration (as detailed in the section 4.3) needs to be applied the same way as discussed in the section 5 depending on the desired Cache test mode. RFC2544 measurements can all be performed in two modes as already discussed in the section 3.4 ? at chosen Flow Export Rate (lower than Flow Monitoring Throughput) or using the Flow Monitoring Throughput set-up. The ways how to achieve both these objectives are specified in the sections 6.2 and 6.3. 6.2 Tests With Single Traffic Component Section 12 of [RFC2544] discusses the use of protocol source and destination addresses for defined tests. To perform all the RFC2544 type measurements with Flow monitoring enabled the defined Flow Keys SHOULD contain IP source and destination address. The RFC2544 type measurements can be executed under these additional conditions: a. the test traffic is not limited to single unique pair of source and destination address b. the traffic generator defines test traffic as follows 1) define the test exactly as specified in the section 5.1 or 5.2 2) allow for a parameter to say send N (where N is an integer number starting at 1 and incremented in small steps) packets with IP addresses A and B before changing both IP addresses to the next value Novak Expires April, 2010 [Page 24] Internet-Draft Flow Monitoring Benchmarking October 2009 This test traffic definition allows execution of the above defined IP Flow monitoring tests with one defined Flow Export Rate while measuring the DUT RFC2544 characteristics. This set-up is the better option since it best simulates the live network traffic scenario with Flows containing more than just one packet. The initial packet rate at N equal to 1 defines the Flow Expiration Rate for the whole measurement procedure. The consequent increases of N will not change Flow Expiration Rate as the time and Cache characteristics of the test traffic stay the same. This set-up is suitable for measurements with Flow Export Rates below the Flow Monitoring Throughput. 6.3 Tests With Two Traffic Components The test traffic set-up in the section 6.2 might be difficult to achieve with commercial traffic generators. An alternate mechanism might be to define two traffic components in the test traffic ? one to populate Flow monitoring Cache and the second one to execute the RFC2544 tests. With this set-up both test modes as discussed in the section 3.4 can easily be achieved. Flow monitoring test traffic component - the exact traffic definition as specified in the sections 5.1 and 5.2. RFC2544 Test Traffic Component - test traffic as specified by [RFC2544] MUST create just one Flow Record in the DUT Cache. In the particular set-up discussed here this would mean a traffic stream with just one pair of unique source and destination IP addresses (but could be avoided if Flow Keys were for example UDP/TCP source and destination ports and Flow Keys did not contain the addresses). The Flow monitoring traffic component will exercise the DUT in terms of Flow activity while the second traffic component will measure the RFC2544 characteristics. The traffic rates to be reported as Throughput are the sum of rates of both components. The RFC2544 metrics do not need any change. 7. Flow Monitoring Accuracy The pure Flow monitoring tests in section 5 provide the capability to verify the Flow monitoring accuracy in terms of the exported Flow Record data. Since every Flow Record created in the Cache is populated by just one packet, the full set of captured data on the Collector can be parsed (e.g. providing the exported Flow Record contents not only the Flow Record count) and each set of parameters from each Flow Record can be checked against the parameters as configured on the traffic generator and set in packets sent to the DUT. The exported Flow Record is considered accurate if: Novak Expires April, 2010 [Page 25] Internet-Draft Flow Monitoring Benchmarking October 2009 a. all the Flow Record fields are present in each exported Flow Record b. all the Flow Record fields values match the value ranges as set by the traffic generator (for example an IP address falls within the range of the IP addresses increments on the traffic generator) c. all the possible Flow Record fields values as defined at the traffic generator have been found in the captured export data on the Collector. This check needs to be offset to potential detected packet losses at the DUT during the test If Packet Sampling is deployed then only verifications in point 1 and 2 can be performed. 8. Evaluating Flow Monitoring Applicability The test results as discussed in this document and obtained for certain DUTs allow for a preliminary analysis of a Flow monitoring deployment based on the traffic analysis data from the providers network. An example of such traffic analysis in the Internet is provided by [CAIDA] and the way it can be used is discussed below. The data needed to make an estimate if a certain network device can manage the particular amount of live traffic with Flow monitoring enabled is: Average packet size: 350 bytes Number of packets per IP Flow: 20 Expected data rate on the network device: 1 Gbit/s This results in: Expected packet rate: 357 000 pps being (1 Gbit/s divided by 350 bytes/packet) Flows per second: 18 000 being (packet rate 357 000 pps divided by 20 packets per IP Flow) Under constant load with the parameters above the network device will always run in the Cache Overflow mode (if the network is large enough the Flows will hardly ever repeat within the few seconds needed to fill up the Cache provided the Flow Keys contain parameters like IP addresses or transport level protocols and ports). Novak Expires April, 2010 [Page 26] Internet-Draft Flow Monitoring Benchmarking October 2009 It needs to be kept in mind that the above is a very rough and averaged Flow activity estimate which cannot account for traffic anomalies like large number of for example DNS request packets which are typically small packets coming from many different sources and represent mostly just one packet Flow. 9. Miscellaneous Tests This section lists the tests which could be useful to asses a proper Flow monitoring operation under various operational or stress conditions. These tests are not deemed suitable for any benchmarking for various reasons. 9.1 DUT Under Traffic Load The Flow Monitoring Throughput SHOULD be measured under different levels of static traffic load through the DUT. This can be achieved only by using two traffic components as discussed in the section 6.3, where one traffic component exercises the Flow Monitoring Plane and the second traffic component loads only Forwarding Plane without affecting Flow monitoring. The variance in Flow Monitoring Throughput as function of the traffic load should be noted for comparison purposes between two DUTs of similar architecture and capability. 9.2 In-band Flow Export The test topology in section 4.1 mandates the use of separate Flow Export interface to avoid the Flow Export data generated by the DUT to mix with the test traffic from the traffic generator. This is necessary in order to create clear and reproducible test conditions for the benchmark measurement. The real network deployment of Flow monitoring might not allow for such a luxury ? for example on a very geographically large network. In such a case, Flow Export will use an ordinary traffic forwarding interface e.g. in-band Flow Export. The Flow monitoring operation should be verified with in-band Flow Export configuration while following these test steps: a. Perform benchmark test as specified in section 5 b. One of the results will be how much bandwidth Flow Export used on the dedicated Flow Export interface c. Change Flow Export configuration to use the test interface d. Repeat the benchmark test while the receiver filters out the Flow Export data from analysis The expected result is that the RFC2544 Throughput achieved in step a. is same as the Throughput achieved in step d. provided that the bandwidth of the output DUT interface is not the Novak Expires April, 2010 [Page 27] Internet-Draft Flow Monitoring Benchmarking October 2009 bottleneck (in other words it must have enough capacity to forward both test and Flow Export traffic). 9.3 Variable Packet Size The Flow monitoring measurements specified in this document would be interesting to repeat with variable packet sizes within one particular test (e.g. test traffic containing mix of packet sizes). The packet forwarding tests specified mainly in [RFC2544] do not recommend and perform such tests. Flow monitoring is not dependent on packet sizes so such a test could be performed during the Flow Monitoring Throughput measurement and verify its value does not depend on the offered traffic packet sizes. The tests must be carefully designed in order to avoid measurement errors due to physical bandwidth limitations and changes of base forwarding performance with packet size. 9.4 Bursty Traffic RFC2544 section 21 discusses and defines the use of bursty traffic. It can be used for Flow monitoring testing as well to gauge some short term overload DUT capabilities in terms of Flow monitoring. The tests benchmark here would not be the Flow Expiration Rate the DUT can sustain but the absolute number of Flow Records the DUT can process without dropping any single Flow Record. The traffic set-up to be used for this test is as follows: a. each sent packet creates a new Flow Record b. the packet rate is set to the maximum transmission speed of the DUT interface used for the test 9.5 Various Flow Monitoring Configurations This section translates the terminology used in the IPFIX documents [RFC5470], [RFC5101] and others into the terminology used in this document. Section 9.5.2 proposes another measurement which is not possible to verify in a black box test manner. 9.5.1 RFC2544 Throughput without Metering Process If Metering Process is not defined on the DUT it means no Flow Monitoring Cache exists and no Flow analysis occurs. The performance measurement of the DUT in such a case is just pure [RFC2544] measurement. 9.5.2 RFC2544 Throughput with Metering Process If only Metering Process is enabled it means that Flow analysis on the DUT is enabled and operational but no Flow Export happens. The performance measurement of a DUT in such a configuration represents an useful test of the DUT capabilities (this corresponds to the case when the network operator uses Flow Novak Expires April, 2010 [Page 28] Internet-Draft Flow Monitoring Benchmarking October 2009 Monitoring for example for manual denial of service attacks detection and does not wish to use Flow Export). The performance testing on this DUT can be performed as discussed in this document but it is not possible to verify the operation and results without interrogating the DUT. 9.5.3 RFC2544 Throughput with Metering and Exporting Process This test represents the performance testing as discussed in section 6. 10. Acknowledgements This work could have been performed thanks to the patience and support of Cisco System Netflow development team, namely Paul Aitken, Paul Atkins and Andrew Johnson. Thanks belong also to Aamer Akhter for initiating this work. 11. IANA Considerations This document requires no IANA considerations. 12. Security Considerations Documents of this type do not directly affect the security of the Internet or corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks. Benchmarking activities as described in this memo are limited to technology characterization using controlled stimuli in a laboratory environment, with dedicated address space and the constraints specified in the sections above. The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network. Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the DUT. Special capabilities SHOULD NOT exist in the DUT specifically for benchmarking purposes. Any implications for network security arising from the DUT SHOULD be identical in the lab and in production networks. Novak Expires April, 2010 [Page 29] Internet-Draft Flow Monitoring Benchmarking October 2009 13. References 13.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, April 1997 [RFC2544] Bradner, S., "Benchmarking Methodology for Network Interconnect Devices", Informational, RFC 2544, April 1999 [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, "Architecture Model for IP Flow Information Export", RFC 5470, October 2009 13.2. Informative References [RFC1242] Bradner, S., "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991 [RFC2285] Mandeville R., "Benchmarking Terminology for LAN Switching Devices", Informational, RFC 2285, November 1998 [RFC3031] E. Rosen, A. Viswanathan, R. Callon, ?Multiprotocol Label Switching Architecture?, Standards Track, RFC 3031, January 2001 [RFC3917] Quittek j., "Requirements for IP Flow Information Export (IPFIX)", Informational, RFC 3917, October 2004. [RFC5101] Claise B., "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information", Standards Track, RFC 5101, January 2008 [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. Meyer, "Information Model for IP Flow Information Export", RFC 5102, January 2008 [RFC5180] C. Popoviciu, A. Hamza, D. Dugatkin, G. Van de Velde, "IPv6 Benchmarking Methodology for Network Interconnect Devices", Informational, RFC 5180, May 2008 [RFC5472] Zseby, T., Boschi, E., Brownlee, N., Claise, B., "IP Flow Information Export (IPFIX) Applicability", RFC 5472, October 2009 [RFC5474] D. Chiou, B. Claise, N. Duffield, A. Greenberg, M. Grossglauser, P. Marimuthu, J. Rexford, G. Sadasivan, "A Framework for Passive Packet Measurement" RFC 5474, October 2009 Novak Expires April, 2010 [Page 30] Internet-Draft Flow Monitoring Benchmarking October 2009 [RFC5475] T. Zseby, M. Molina, N. Duffield, F. Raspall, ?Sampling and Filtering Techniques for IP Packet Selection? RFC 5475, October 2009 [RFC5476] Claise, B., Quittek, J., and A. Johnson, "Packet Sampling (PSAMP) Protocol Specifications", RFC 5476, October 2009 [RFC5477] T. Dietz, F. Dressler, G. Carle, B. Claise, "Information Model for Packet Sampling Exports", RFC 5477, October 2009 [PSAMP-MIB] Dietz, T., Claise, B. "Definitions of Managed Objects for Packet Sampling", Internet-Draft work in progress, June 2006 [MPLS] Akhter A. "MPLS Forwarding Benchmarking Methodology", Internet-Draft work in progress, November 2008 [CAIDA] Claffy, K., "The nature of the beast: recent traffic measurements from an Internet backbone", http://www.caida.org/publications/papers/1998/Inet98/ Inet98.html Author's Addresses Jan Novak (editor) Cisco System Edinburgh, United Kingdom Phone: +44 7740 925889 Email: janovak@cisco.com Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 EMail: bclaise@cisco.com Novak Expires April, 2010 [Page 31] Internet-Draft Flow Monitoring Benchmarking October 2009 Appendix A: Report Format Parameter Units ----------------------------------- ------------------------------------ Test Case test case name (section 5.1, 5.2, 6) Test Topology Figure 2, 3, 4, other Traffic Type IPv4, IPV6, MPLS, other Test Results Flow Monitoring Throughput Flow Records per second or Not Applicable Flow Export Rate Flow Records per second or Not Applicable Control Information Export Rate Flow Records per second RFC2544 Throughput packets per second (Other RFC2544 Metrics) (as appropriate) General Parameters Traffic Direction unidirectional, bidirectional DUT Interface Type Ethernet, POS, ATM, other DUT Interface Bandwidth MegaBits per second Traffic Specifications Number of Traffic Components (see section 6.2 and 6.3) For each traffic component: Packet Size bytes Traffic Packet Rate packets per second Traffic Bit Rate MegaBits per second Number of Packets Sent number of entries Incremented Packet Header Fields list of fields Number of Unique Header Values number of entries Number of Packets per Flow number of entries Flow monitoring Specifications Direction ingress, egress, both Observation Points DUT interface names Cache Size number of entries Cache Mode normal or overflow Active Timeout seconds Inactive Timeout seconds Flow Keys list of fields Flow Record Fields total number of fields Number of Flows Created number of entries Flow Export Transport Protocol UDP, TCP, SCTP, other Flow Export Protocol IPFIX, Netflow, other Packet Sampling Specifications Sampling Method [RFC5475] systematic, random or none Sampling Interval milliseconds or not applicable Sampling Rate number of packets or not applicable MPLS Specifications (for traffic type MPLS only) Tested Label Operation imposition, swap, disposition or none Novak Expires April, 2010 [Page 32] Internet-Draft Flow Monitoring Benchmarking October 2009