Cybertelecom
Cybertelecom
Federal Internet Law & Policy
An Educational Project

Industry: Comcast
Dont be a FOOL; The Law is Not DIY
- IPv6
- FCC Net Neutrality BitTorrent Proceeding
- DataCaps
- Cable
- Open Access
- Comcast / Level3 Intercon Dispute
- Comcast / Cogent Intercon dispute
- Comcast / Netflix Intercon Dispute
- Mergers
- - TWC
- - NBCU
- - Adelphia
- - AT&T Broadband
- Data Caps


Physical Network

Broadband Measurement

Network Management

Data Caps

  • "In May 2012, Comcast announced that it is experimenting with different approaches to data caps and usage pricing through trials in some markets but for all markets raised the existing data cap from 250 GB to 300 GB per month.932 In the markets where Comcast is not experimenting with usage based pricing, it has suspended enforcement of its data caps." FCC Annual Video Competition Report 2013
  • Comcast: 250 Gb per month cap. Articulated network management plans in disclosure to FCC pursuant to order. [Bode 082808]
  • Comcast / NBCU: Para 94: "To address these transaction-related concerns, the Applicants have offered a number of voluntary commitments.  The Applicants have agreed that, in their provision of broadband Internet access services, neither Comcast nor Comcast-NBCU shall prioritize affiliated Internet content over unaffiliated Internet content.[211]  In addition, any Comcast or Comcast-NBCU broadband Internet access service offering that involves caps, tiers, metering, or other usage-based pricing shall not treat affiliated network traffic differently from unaffiliated network traffic.  Comcast and Comcast-NBCU shall also comply with all relevant FCC rules, including the rules adopted by the Commission in GN Docket No. 09-191,[212] and, in the event of any judicial challenge affecting the latter, Comcast-NBCU’s voluntary commitments concerning adherence to those rules will be in effect.[213]"

Congestion Management

IETF RFC 6057, Comcast's Protocol-Agnostic Congestion Management System (Dec. 2010) ("This document describes the congestion management system of Comcast Cable, a large cable broadband Internet Service Provider (ISP) in the U.S. Comcast completed deployment of this congestion management system on December 31, 2008.") Following an August 2008 FCC document [FCC_Memo_Opinion] regarding how Comcast managed congestion on its High-Speed Internet ("HSI") network, Comcast disclosed to the FCC [FCC_Net_Mgmt_Response] and the public additional technical details of the congestion management system that it intended to and did implement by the end of 2008 [FCC_Congest_Mgmt_Ltr], including the thresholds involved in this new System

"1. Software installed in the Comcast network continuously examines aggregate traffic usage data for individual segments of Comcast's HSI network. If overall upstream or downstream usage on a particular segment of Comcast's HSI network reaches a pre-determined level, the software moves on to step two.

2. At step two, the software examines bandwidth usage data for subscribers in the affected network segment to determine which subscribers are using a disproportionate share of the bandwidth. If the software determines that a particular subscriber or subscribers have been the source of high volumes of network traffic during a recent period of minutes, traffic originating from that subscriber or those subscribers temporarily will be assigned a lower priority status.

3. During the time that a subscriber's traffic is assigned the lower priority status, their packets will not be delayed or dropped so long as the network segment is not actually congested. If, however, the network segment becomes congested, their packets could be intermittently delayed or dropped.

4. The subscriber's traffic returns to normal priority status once his or her bandwidth usage drops below a set threshold over a particular time interval.

Broadband Internet, Interconnection & Backbone

In 2001, Comcast broadband Internet service was a loose confederation of local and regional networks, with backbone transit service provided by third parties such as AT&T, Savvis, and Level 3. [CAIDA] [See Comcast AtHome Network (noting 1996 Comcast agreement to join @Home; @Home backbone provided service to Comcast)]

In 2002, AT&T divided its business operations into AT&T Wireless, AT&T Business and AT&T Broadband, intending to spin off the AT&T Broadband internet access service. AT&T and Comcast announced that Comcast (the third largest cable Internet company) would acquired AT&T broadband (the largest cable Internet company). In 2006, Comcast would expand its broadband Internet service further through the acquisition of Adelphia.

Comcast argued in its AT&T Broadband merger application that the unified Comcast broadband internet service could negotiate discounted wholesale transit service from vendors. [Comcast / AT&T Application at 34 ("the increased level of Internet traffic resulting from combining the two companies’ broadband operations will allow AT&T Comcast to take advantage of volume discounts in buying Internet backbone services to transport customer Internet traffic.")] AT&T Business, which included the AT&T Internet backbone, would provide the new Comcast company with backbone transit service. [Comcast / AT&T Application at 17 n. 24 (stating that AT&T broadband has entered into an agreement with AT&T for the provision of Internet backbone service. "AT&T Broadband has entered into four principal network services agreements with AT&T: (1) a Master Carrier Agreement that reflects the market-based rates, terms and conditions on which AT&T Business Services will, inter alia, sell long distance services to AT&T Broadband for resale, terminate traffic outside of AT&T Broadband’s service area, and provide “administrative services” for internal AT&T Broadband usage,… In addition, AT&T Broadband and AT&T have entered into a High Speed Internet Services Binding Term Sheet that reflects the rates, terms and conditions on which AT&T will provide managed IP layer services, support and maintenance, and specified subscriber functionalities (such as Internet search and navigation tools) to support AT&T Broadband in its provision of high-speed Internet services to its subscribers.")] [Brodkin 2014 ("As part of [the 2002 purchase of AT&T Broadband], Comcast purchased transit from AT&T for a few years after the transaction. But with the costly contract expiring in the mid-2000s “and because no one company could support the infrastructure needs of our business,” Comcast decided to build its own backbone, Tishgart said.")][CAIDA] Typical of end-user access networks, Comcast's traffic ratio balanced in 2007 favored downloaded traffic over uploaded traffic. [Labowitz 20]

After the merger with AT&T, Comcast was significantly larger but still be reliant on third party transit service. Comcast, therefore, had an incentive to build its own backbone network in order to avoid paying a third party (and potential competititor) for transit service and in order to unify its network. The backbone could also be leveraged for MVPD video delivery and telephony service. [Labowitz (2007 lacked a nationwide network backbone; highly dependent on upstream transit)] [Werbach (predicting that Comcast would need to merge with a backbone network to compete with other access networks that had backbones)] [Saxena 10 ("The backbone interconnects regional networks to create a unified national network" listing benefit as reducing "transport payments to transit providers")] [Schanz Slide 10] [Vik]

Work on the Comcast backbone (ASN 7922) was initiated in 2003. [Vik] Comcast acquired dark fiber from Level 3 and equipment from CISCO. [Vik at 6 (equipment from Nortel)] [Comcast Press Release 2004] [CISCO] Comcast's backbone went online in 2006, interconnecting a few networks. [CAIDA] [Schanz Slide 10] [Vik] In 2007, Comcast's backbone tied together its local networks, linking 23 Comcast ASNs. Comcast published a peering policy [Comcast Settlement-Free Interconnection (SFI) Policy (first archived by Archive.org on June 12, 2008)] and advertised "Wholesale Dedicated IP Transit Service." [Comcast Dedicated IP Transit (first archived by Archive.org on March 12, 2010)]

In 2008, the Comcast tested its 100G backbone. [Vik Evolutin at 11] In 2008, CAIDA's data shows that Comcast had __ transit customers. By 2009, "Comcast is significantly different." It has become a "net contributor of Internet traffic" and the "6th largest origin / transit ASN by volume." [Labowitz 19]

Comcast's backbone network generally had four different types of interconnection arrangements with other networks:

  • A transit vendor to enterprises, local networks, academic networks, and end users;
  • A transit consumer of Level 3 and then Tata for end points that it was not directly connected with, especially international traffic;
  • A settlement free peer with other backbone networks; and
  • An Access Paid Peering vendor for content networks.

DIAGRAM

Backbone Transit Vendors

Comcast no longer purchased transit service for its local networks from AT&T. [CAIDA] [Netflix] Transit service is now provided to the Comcast local networks by Comcast's backbone. Comcast's backbone was not Tier 1 and did not have full Internet reach, especially internationally. In order to reach destinations on networks that it did not peer with, Comcast's backbone initially acquired upstream transit service from Level 3 and then from Tata. [CAIDA] [Brodkin 2014 ("Comcast has built a large enough network of its own that it now peers with the Tier 1 companies. It does buy transit, but for less than one percent of its overall traffic load. That mostly addresses the small amount of traffic coming to and from overseas, Tishgart said.")]. As Comcast arranges additional peering relationships with other networks, it can continue to reduce its need for transit service and therefore transit costs.

Backbone Peers

With transit service from Level 3 and Tata, Comcast did not need additional transit service from other backbones; these interconnection relationships gave Comcast full Internet reach. If, however, Comcast negotiated peering arrangements with other backbone networks, it could improve its connectivity and quality of service, and reduce its transit costs.

The third party backbone services were in the business of selling transit. They would like to sell transit to Comcast, but Comcast already had transit vendors and was reducing its need for transit. They also sold transit to other networks. Comcast had become so large that directly connecting to Comcast could become a business advantage. Directly connecting with Comcast would reduce the number of hops required for traffic to go through, would improve connectivity, and would improve quality of service. Comcast had become such a large destination on the Internet that if ACME network directly interconnected with Comcast, but BETA network did not, ACME would be at a significant advantage over BETA.

In short order, backbone networks began to enter into peering arrangements with Comcast. These settlement free peering arrangements reportedly contained balanced traffic ratios; the backbone network could only download as much traffic as it uploaded. This in theory prevented the backbone networks from taking on heavy content customers attempting to reach Comcast, essentially acting as CDNs. Comcast interconnected with domestic backbone networks as follows [CAIDA]:

  • 2006: Level 3 (Transit provider, 2006-2012; peering, 2013-present) [Norton]
  • 2007:
    • GTT (peering)
    • Seabone (peering)
  • 2008: Cogent (peering)
  • 2009:
    • AT&T (peering)
    • Tata (transit provider) [Norton]
  • 2010:
    • Sprint (peering)
    • Zayo (peering)
  • 2013:
    • Centurylink (peering)
    • Time Warner Cable (peering)
    • Verizon (transit provider for three years, then peering)
  • 2014:
    • NTT (peering) [Brodkin 2014]
    • Hurricane Electric (peering)
    • XO (peering)

Comcast 7922 also became an attractive peering opportunity for other networks such as Internet2 (academic), ISC (infrastructure), MERIT (academic, research) and ___. COMCAST CITATION

CONTENT ACCESS PAID PEERING

Content providers are in the business of providing content to access network's end users. The larger an access network, the greater percentage of a content provider's business plan an access network represents. For example, if an access network provides access to 24% of end users in a market, than that access network provides access to 24% of a content provider's potential business plan.

A Content Delivery Network is in the business of delivering content from the content provider to the access network. The larger an access network, the stronger the incentive to have quality connectivity to that network. If an access network represents 1% of the market, it may not be justifiable for the CDN to build out infrastructure to meet that access network. But if an access network is one of the largest access networks with 24% of the market, the CDN has an incentive to arrange quality connectivity that it can turn arount and market to its customers.

Comcast invested in and built its backbone. All access networks have the incentive to be paid to receive traffic destined for their end users. Comcast was now so large that the value proposition of access to Comcast's end users was significant. Furthermore, with Comcast's backbone, Comcast now had the ability to control the supply of access capacity. In order for Comcast to be paid by CDNs for access to Comcast's end users, Comcast had to make sure that access to end users was not something that they could satisfactorily receive for free otherwise.

Comcast purchased transit capacity from Level 3 and Tata. However, when Level 3 attempted to sell access to Comcast to Netflix, a signficant interconnection dispute unfolded. Comcast then acquired transit service from Tata (with the obvious advantage of providing access to international markets that Comcast's domestic network lacked connectivity to). Reports indicated that the connectivity with Tata lacked sufficient spare capacity to handle the traffic demands of a large content provider such as Netflix.

Comcast entered into settlement free peering arrangements with backbone providers, with a balanced traffic ratio requirement (a provision not found in access networks' acquisition of transit service from backbone proviers where the traffic was traditionally balanced in the download direction). Since backbones could only download as much as they uploaded, and since end users generally downloaded more than they uploaded, these backbones could not use this interconnection capacity to market service as CDNs to content providers. If they did - if they sold service to, for instance, Netflix - the traffic ratio would be out of balance, utilization would exceed capacity, Comcast would refuse to augment capacity, and congestion would occur. This situation would persist until the backbone had either eliminated the content traffic or agreed to enter into a CDN access paid peering agreement with Comcast.

With capacity through transit vendors and peers controlled, Comcast could negotiate with CDN quality connectivity directly with Comcast's access networks. CDNs like Limelight, reported, made clear that they would not pay for access if they did not have to. But a CDN competing with Akamai which has superior connectivity to Comcast cannot afford to have congested connectivity. The competitive disadvantage would put the CDN out of the market. Therefore the CDN pays in order to have a quality of connectivity that it can then market to its customers.

 

Comcast's new network also became attractive to content delivery networks attemtpting to connect with end users. Connecting with Comcast 7922 meant interconnecting with Comcast relatively close to subscribers and outside the cacophony of commodity backbones. Content could be delivered efficiently and with a higher quality of service, attributes that CDNs could market to potential content customers. CDNs generally engaged in cold potato routing, delivering content at the gateway of the local Comcast network, as close to end users as possible, and thereby not using the Comcast backbone for long hault transport. Comcast 7922 interconnected with Akamai in a transit relationship starting in 2008. This relationship has also been described as a paid peering arrangement. [CAIDA] [Norton] [See Faratin] Comcast 7922 interconnected with Limelight 2010. It is reported that this was a paid peering relationship. [Norton] [CAIDA]

EDGECAST NETFLIX (DOES THAT CHART REFLECT OTHER PAID RELATIONSHIPS)

For content networks to prefer the paid interconnection routes to Comcast, there could not be an alternative substitute supply of settlement free interconnection capacity from interconnection with backbones. [Norton] [Netflix] As noted above, most backbones interconnection with Comcast on a settlement free basis with a balanced traffic ratio. They could only send as much traffic as they received, and thus they could not act as a route to Comcast for large content providers. If a backbone network did take on a large content provider as a customer attempting to send traffic to Comcast, utilization over interconnection links between the networks would increase and more capacity would be required. Because the traffic ratios were out of balance, Comcast would refuse to augment interconnection capacity. Without additional capacity but with growing demand, the interconnection links would become contested. This situation would persist until either the backbone network resolved its traffic imbalance (stopped acting like a CDN) or the backbone network entered into a CDN type arrangment with Comcast, paying to access Comcast's subscribers. As noted above, a few backbones (Level 3 and Tata) provided Comcast with transit service in order to ensure full Internet reach; as the customer, Comcast would order sufficient capacity in order to maintain global connectivity but not sufficient capacity to act as a substitute route to Comcast for large content providers. [See 2010 Interconnection Dispute between Netflix and Comcast].

Comcast

Industry

  • Ciena Press Release, Comcast Upgrades to Ciena's Newest 100G Technology, January 17, 2013.
  • Reply of Netflix, Inc., Application of Comcast Corporation and Time Warner Cable, Inc., MB Docket No. 14-57, p. 41 (Dec. 23, 2014).
  • CISCO Press Release, With CISCO Services, Comcast Scales Its Commercial Metro Services and Company Reach, (2004 ) ("The company’s network spans more than 90,000 fiber-optic miles to deliver services in 41 states and 22 of the top 25 U.S.-metropolitan areas.")
  • Tata
    • Ex Parte, Adam Rothchild, Vice President, Network Architecture, Voxel dot net, Comcast and NBC Universal Merger (MB Docket No. 10-56); Open Internet Broadband Industry Practices (GN Docket No. 09-191) (Jan. 11, 2011) ("Delivering traffic to Comcast over standard “best effort” paths, such as the transit circuits it purchases from Tata Communications, we have observed extreme packet loss for the majority of the day, dating back over six months. In these conditions, it is simply not possible for competing external providers to deliver VoIP, gaming, or streaming video services to Comcast’s broadband subscribers.");
    • Peering disputes: Comcast, Level 3 and You, Internap Dec. 2, 2010, http://www.internap.com/2010/12/02/peering-disputes-comcast-level-3-and-you/ (Unattributed statements from Tata officials reported “[our] San Jose and New York links with Comcast are running full.”);
    • Comcast Answers Today's Traffic Question, Comcast Voices Dec. 14, 2010, http://corporate.comcast.com/comcast-voices/comcast-answers-todays-traffic-question ("Trying to base a theory upon a snapshot of TATA ports in one market during a particular time period (assuming the graph is legitimate) paints a narrow, inaccurate picture. It hardly represents how the Internet functions or the capacity and varied paths between Comcast and many networks on the Internet.")
    • Ex Parte of Cogent Communications Group, Dkt. 14-57 (Feb. 2, 2015) ("Comcast continues to purchase transit from Tata, but does not allow Tata to announce Cogent's routes. Cogent already peers with Tata. Accordingly, if Comcast allowed Cogent's routes to be announced by Tata, congestion between Cogent and Comcast would be relieved, since the relevant traffic would travel from Cogent to Tata to Comcast." )
  • Netflix
    • Netflix Petition, Comcast /TWC Merger, at 56. ("In 2013, congestion on Cogent's and Level 3's routes into Comcast's network steadily increased, reaching a level where it began to affect the performance of Netflix streaming for Comcast's customers.

Government

FCC Inquiry 2014

Papers and Research

  • CAIDA (data captured for December of each respective year)
  • OTI, Beyond Frustrated, The Sweeping Consumer Harms as a Result of ISP Disputes, Nov. 2014
  • Kevin Werbach, The Centripetal Network: How the Internet Holds Itself Together, and Forces Tearing It Apart, 42 U.C. Davis L. Rev. 343 (2008)
  • William Norton, The Emerging 21st Century Access Power Peering, COMMUNICATIONS & STRATEGIES, 84, 4th Q. 2011, p. 55.
  • Peyman Faratin, David Clark, Staven Bauer, and William Lehr, Patrick Gilmore, Arthur Berger, The Growing Complexities of Internet Interconnection, No. 72, 4th Q. 2008, p. 60, Communications & Strategies (senior Akamai engineers supporting access paid peering)
  • Craig Labovitz, Scott Lekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian, Internet Inter-Domain Traffic, p. 29-80  2010 (“Discussions with analysts and ISPs provide some insight into Comcast’s transformation. Over the last five years, Comcast executed on a number of technology and business strategies, including consolidation of several disparate regional backbones into a single nationwide network and rollout of a “triple play” (voice, video, data) consumer product. Most significantly, Comcast began offering wholesale transit (GigE and 10GigE IP), cellular backhaul and IP video distribution (though Comcast Media Center subsidiary)”).

News

Mergers

Comcast / Time Warner Cable Merger 2015

Derived From: Remarks of Jon Sallet, Federal Communications Commission General Counsel, As Prepared for Delivery, Telecommunications Policy Research Conference: “The Federal Communications Commission and Lessons of Recent Mergers & Acquisitions Reviews” (Sept. 25, 2015)

Hearings

News

Comcast / NBC Merger

Outages

"On March 22, 2011, a Comcast outage in 19 New Hampshire communities beginning around 3:30 p.m. left many Comcast customers in those communities unable to make any calls, including 9-1-1 calls. The problem lasted through the evening. "

"In March 2010, Comcast Internet and Digital Voice service was disrupted to customers in Nashville, Tennessee, and Atlanta, Georgia. Comcast customers experienced severely degraded service for at least two hours. During the outage, local, state, and Federal government department and agency customers of Comcast in the affected areas were unable to make or receive telephone calls. Residential and business subscribers to Comcast Internet and Digital Voice services also were affected by the outage significantly impairing their ability to engage in 9-1-1 and other communications."

Financials

Applications

Online Video Delivery

VoIP

Timeline

News

© Cybertelecom ::