"The methodologies and assumptions underlying the measurements described in this Report are reviewed at meetings that are open to all interested parties and documented in public ex parte letters filed in the GN Docket No. 12-264. Policy decisions regarding the MBA program were discussed at these meetings prior to adoption, and involved issues such as inclusion of tiers, test periods, mitigation of operational issues affecting the measurement infrastructure, and terms-of-use notifications to panelists. Participation in the MBA program is open and voluntary. Participants include members of academia, consumer equipment vendors, telecommunications vendors, network service providers, consumer policy groups as well as our contractor for this project, SamKnows." [Eight MBA Report at 20]
"In September 2012, the FCC announced it was expanding the program to include information on mobile broadband service performance in the United States. The program uses the FCC Speed Test app for Android and iPhone devices to test the performance of volunteers' smartphone mobile broadband services. Broadband performance data is being collected that includes upload and download speed, latency and packet loss, as well as the wireless performance characteristics of the broadband connection and the kind of handsets and versions of operating systems tested." [FCC Measuring Mobile Broadband]
"The measurement clients collected data throughout the year. However, only data collected from September 1 through 6 and September 28 through October 21, 2017, referred to throughout this report as the "September 2017" reporting period, were used to generate the charts in this Report. [This data is uploaded from the end user white boxes to samKnows, who then provides the data to the FCC]
"However, the service performance that a consumer experiences could differ from our measured values for several reasons:
Broadband measurement methodology can distort results. Aggregating can mask degraded performance underneath. This problem can emerge topologically (aggregating different networks' traffic together masking issues occurring as a result of negotiations between two networks), chronologically (aggregating different times' traffic together masking peak hour issues with non-peak hour traffic), or geographically (aggregating traffic from diverse locations on the network masking issues at a specific POP).
Broadband
Buffer / Router Cue:
Capacity
Advertised Speed
"Available bandwidth is a measure of how much capacity is unused in a link or along a path. The available bandwidth along a path is the minimum available bandwidth of the set of links along a path. … One can look at a link utilization graph and observe that the total capacity was 100 Mbps but the peak usage was 45 Mbps and conclude that the available bandwidth, or spare capacity, was 55 Mbps." [Bauer, Understanding Broadband Speed Measurements, p. 8.]
Bandwidth Caps
bandwidth of a link for packets of size k "The capacity, in bits/second, where only those bits of the IP packet are counted, for packets of size k bytes." [RFC 2330 Sec. 7]
"Bulk transfer capacity "is a measure of a network's ability to transfer significant quantities of data with a single congestion-aware transport connection (e.g., TCP)." [IETF RFC 3148] [Bauer, Understanding Broadband Speed Measurements, p. 8.]
"Capacity is a measure of the total traffic carrying capability of a link or path in a network. The end-to-end capacity is the minimum link capacity along a path. … The capacity is expressed as the amount of traffic the link can carry over a particular time interval (e.g., megabits per second)." [Bauer, Understanding Broadband Speed Measurements, p. 8.]
Experienced Speed
IP-layer link capacity, "C(L,T,I), to be the maximum number of IP-layer bits that can be transmitted from the source S and correctly received by the destination D over the link L during the interval [T, T+I], divided by I." [RFC 5136 Sec. 2.3.2]
IP-layer path capacity "simply becomes that of the link with the smallest capacity along that path." [RFC 5136 Sec. 2.3.3]
Nominal Physical Link Capacity, "NomCap(L), is the theoretical maximum amount of data that the link L can support. For example, an OC-3 link would be capable of 155.520 Mbit/s. We stress that this is a measurement at the physical layer and not the network IP layer, which we will define separately."" [RFC 5136 Sec. 2.1]
Throttling
Traffic Capacity: "The maximum traffic per unit of time that a given telecommunications system, subsystem, or device can carry under specified conditions." [FS 1037c]
Throughput (see also Traffic Capacity) "Throughput is the rate at which the traffic can be transported by a specific protocol." [TR-304 Sec. 8.1]. :: "The number of bits, characters, or blocks passing through a data communication system, or portion of that system" [ATIS Telecom Glossary] :: "The maximum rate at which none of the offered frames are dropped by the device." [RFC 1242 Sec. 3.17]
Congestion: A situation where demand for utilization of network resources exceeds capacity of the network, resulting in a degradation of service.
Delay " One-way packet (or frame) delay for a service is defined as the interval between the time that the first bit of a packet ingresses the service at the sending interface and the time that the last bit of the corresponding packet egresses the service at the receiving interface" [TR-304 Sec. 8.2] [ATIS Telecom Glossary] :: "Delay is the time for packets to traverse the network from source to destination. " [Ciavattone]
hop count of a route "The value 'n' of the route path." [RFC 2330 Sec. 7]
Host "A computer capable of communicating using the Internet protocols; includes "routers"." [RFC 2330 Sec. 5] :: "Almost any kind of computer, including a centralized mainframe that is a host to its terminals, a
server that is host to its clients, or a desktop personal computer (PC) that is host to its peripherals. In
network architectures, a client station (users machine) is also considered a host because it is a source of
information to the network, in contrast to a device, such as a router or switch, that directs traffic. " - Miles Tracy, Wayne Jansen, Karen Scarfone, Theodore Winograd, Guidelines on Securing Public Web Servers, NIST Special Publication 800-44 ver. 2 at B-1 (Sept. 2007)
Jitter:
- NIST, Security Considerations for VoIP Systems 800-58 p. 17 (April 2004)
Jitter refers to non-uniform packet delays. It is often caused by low bandwidth situations in VOIP and can be exceptionally detrimental to the overall QoS. Variations in delays can be more detrimental to QoS than the actual delays themselves [8]. Jitter can cause packets to arrive and be processed out of sequence. RTP, the protocol used to transport voice media, is based on UDP so packets out of order cannot be reassembled at the protocol level. However, RTP allows applications to do the reordering using the sequence number and timestamp fields. The overhead in reassembling these packets is non-trivial, especially when dealing with the tight time constraints of VOIP.
When jitter is high, packets arrive at their destination in spurts. This situation is analogous to uniform road traffic coming to a stoplight. As soon as the stoplight turns green (bandwidth opens up), traffic races through in a clump. The general prescription to control jitter at VOIP endpoints is the use of a buffer, but such a buffer has to release its voice packets at least every 150 ms (usually a lot sooner given the transport delay) so the variations in delay must be bounded. The buffer implementation issue is compounded by the uncertainty of whether a missing packet is simply delayed an anomalously long amount of time, or is actually lost. If jitter is particularly erratic, then the system cannot use past delay times as an indicator for the status of a missing packet. This leaves the system open to implementation specific behavior regarding such a packet.
Jitter can also be controlled at the nexuses of the VOIP network by using routers, firewalls, and other network elements that support QoS. These elements process and pass along time urgent traffic like VOIP packets sooner than less urgent data packets. Unfortunately, not all network components were designed with QoS in mind. An example of a network element that does not implement this QoS demand is a crypto-engine, which ignores Type of Service (ToS) bits in an IP header and other indicators of packet urgency (see 8.7). Another method for reducing delay variation is to pattern network traffic to diminish jitter by making as efficient use of the bandwidth as possible. Unfortunately, this constraint is at odds with some security measures in VOIP. Chief among these is IPsec, whose processing requirements may increase latency, thus limiting effective bandwidth and contributing to jitter. Effective bandwidth is compromised when packets are expanded with new headers. In normal IP traffic, this problem is negligible since the change in the size of the packet is very small compared with the packet size. Because VOIP uses very small packets, even a minimal increase is important because the increase accrues across all the packets, and VOIP sends a very high volume of these small packets.
The window of delivery for a VOIP packet is very small, so it follows that the acceptable variation in packet delay is even smaller. Thus, although we are concerned with security, the utmost care must be given to assuring that delays in packet deliveries caused by security devices are kept uniform throughout the traffic stream. Implementing devices that support QoS and improving the efficiency of bandwidth with header compression allows for more uniform packet delay in a secured VOIP network.
- "The absolute value of the difference between the Forwarding Delay of two consecutive received packets belonging to the same stream." [RFC 4689]
- RFC 3393, Sec. 1.1 ""Jitter" commonly has two meanings: The first meaning is the variation of a signal with respect to some clock signal, where the arrival time of the signal is expected to coincide with the arrival of the clock signal. This meaning is used with reference to synchronous signals and might be used to measure the quality of circuit emulation, for example. There is also a metric called "wander" used in this context.
The second meaning has to do with the variation of a metric (e.g., delay) with respect to some reference metric (e.g., average delay or minimum delay). This meaning is frequently used by computer scientists and frequently (but not always) refers to variation in delay."
Latency:
- See ITU-T G. 114 General Recommendations on the transmission quality for an entire international telephone connection (05/2003)
- NIST, Security Considerations for VoIP Systems 800-58 p. 16 (April 2004) "Latency in VOIP refers to the time it takes for a voice transmission to go from its source to its destination. Ideally, we would like to keep latency as low as possible but there are practical lower bounds on the delay of VOIP. The ITU-T Recommendation G.114 [2] set forth a number of time constraints on one-way latency. The upper bound is150 ms. for one-way traffic. This corresponds to the current latency bound experienced in domestic calls across PSTN lines in the continental United States [3]. For international calls, a delay of up to 400 ms. was deemed tolerable [4], but since most of the added time is spent routing and moving the data over long distances, we consider here only the domestic case and assume our solutions are upwards compatible in the international realm."
- Latency " Forstore and forward devices: The time interval starting when the last bit of the input frame reaches the input port and ending when the first bit of the output frame is seen on the output port. For bit forwarding devices: The time interval starting when the end of the first bit of the input frame reaches the input port and ending when the start of the first bit of the output frame is seen on the output port." [RFC 1242 Sec. 3.8]
- Joseph Coffey, Latency of Optical Fiber Systems, Commscope ("Latency is a time delay between a stimulation and its response. It is caused by velocity limitations in a physical system. In simplest terms, latency is the time it takes for a signal to travel (or propagate) from point A to point B. In telecommunications, latency describes the time delay of a packet traveling through a network or the delay imposed on a signal traveling in a transmission medium such as a copper cable, optical fiber waveguide, or even free space; in radio transmissions it is the time it takes a radio signal to propagate through free space from the transmitter to the receiver; for electrical transmissions, it is the time the electrical signal takes to propagate through a metallic or conductive medium; and, in optical transmission, latency describes the time required for the optical signal to propagate through free space or in the core of an optical fiber")
- A. Flickinger, C. Klatsky, A. Ledesma, J. Livingood, and S. Ozer, “Improving Latency with Active
Queue Management (AQM) During COVID-19”. Available: https://arxiv.org/abs/2107.13968
- M. Ford, “Workshop report: reducing internet latency, 2013,” ACM Sigcomm CCR. https://dl.acm.org/doi/10.1145/2602204.2602218
- Latency Explained, BITAG (2022) [Latency BITAG 2022]
Link "a connection between two of these network devices or nodes." [RFC 5136 Sec. 2.1] :: "A single link-level connection between two (or more) hosts;
includes leased lines, ethernets, frame relay clouds, etc." [RFC 2330 Sec. 5]
Loss "The packet (or frame) loss ratio is the ratio of total lost packets (or frames) to total transmitted packets (or frames) in a population of interest" [TR-304 Sec. 8.4]
Frame Loss Rate " Percentage of frames that should have been forwarded by a network device under steady state (constant) load that were not forwarded due to lack of resources." [RFC 1242 Sec. 3.6]
Packet Loss:
- NIST, Security Considerations for VoIP Systems 800-58 p. 18 (April 2004)
VOIP is exceptionally intolerant of packet loss. Packet loss can result from excess latency, where a group of packets arrives late and must be discarded in favor of newer ones. It can also be the result of jitter, that is, when a packet arrives after its surrounding packets have been flushed from the buffer, it is useless. VOIP-specific packet loss issues exist in addition to the packet loss issues already associated with data networks; these are the cases where a packet is not delivered at all. Compounding the packet loss problem is VOIP’s reliance on RTP, which is based on the unreliable UDP, and thus does not guarantee packet delivery. Unfortunately, the time constraints do not allow for a reliable protocol such as TCP to be used to deliver media. By the time a packet could be reported missing, retransmitted, and received, the time constraints for QoS would be well exceeded. The good news is that VOIP packets are very small, containing a payload of only 10-50 bytes [5], which is approximately 12.5-62.5 ms, with most implementations tending toward the shorter range. The loss of such a minuscule amount of speech is not discernable or at least not worthy of complaint for a human VOIP user. The bad news is these packets are usually not lost in isolation. Bandwidth congestion and other such causes of packet loss tend to affect all the packets being delivered around the same time. So although the loss of one packet is fairly inconsequential, probabilistically the loss of one packet means the loss of several packets, which severely degrades the quality of service in a VOIP network.
propagation time of a link "The time, in seconds, required by a single bit to travel from the output port on one Internet host across a single link to another Internet host." [RFC 2330 Sec. 7]
Node "hosts, routers, Ethernet switches, or any other device where the input and output links can have different characteristics" [RFC 5136 Sec. 2.1]
Non Peak Hours: Traffic measurement period outside peak utilization.
Path "a path P of length n as a series of links (L1, L2, ..., Ln) connecting a sequence of nodes (N1, N2, ..., Nn+1)." [RFC 5136 Sec. 2.1] :: "A sequence of the form < h0, l1, h1, ..., ln, hn >, where n >=
0, each hi is a host, each li is a link between hi-1 and hi,
each h1...hn-1 is a router. A pair is termed a 'hop'." [RFC 2330 Sec. 5]
Peak Hours: Daily measurement period where network experiences peak utilization. For a residential network, for example, MBA determined that peak hours were 7 pm to 11 pm.
Router "A host which facilitates network-level communication between
hosts by forwarding IP packets." [RFC 2330 Sec. 5] :: "The path, as defined in Section 5, from A to B at a given time." [RFC 2330 Sec. 7]
Speed: See Traffic Capacity
Utilization: Amount of traffic on a network. May be shown as a percentage of capacity or as Mbps. Measured in samples and may be averaged and shown as peak or non-peak utilization, or utilization over certain amounts of time (i.e., hour, day, week, month). :: "The average usage of a link L, Used(L,T,I), is the actual number of IP-layer bits from any source, correctly received over link L during the interval [T, T+I], divided by I." [RFC 5136 Sec. 2.3.4] :: "We express usage as a fraction of the overall IP-layer link capacity.
Util(L,T,I) = ( Used(L,T,I) / C(L,T,I) )
Thus, the utilization now represents the fraction of the capacity
that is being used and is a value between zero (meaning nothing is
used) and one (meaning the link is fully saturated). Multiplying the
utilization by 100 yields the percent utilization of the link." [RFC 5136 Sec. 2.3.5]
95th Percentile Measurement Methodology
Augmentation
Zero Rating
