When networks interconnect, they agree to announce routes to each other utilizing the Border Gateway Protocol (BGP). This is known as "interdomain interconnection." (This is not to be confused with intradomain routing. Routing within a domain or AS network can have its own set of curiosities, it is generally conducted by a different protocol than BGP such as OSPF (open shortest path first) or RIP (resource information protocol) and can involve MPLS creating virtual circuits across domains)
There are two parts to BGP: (a) route announcements by the traffic receiving network and (b) route selection by the traffic sending network.
Announcements
A receiving network announces which destinations (which ASNs) it provides a route to, and how many hops (a.k.a. "AS path length") it takes to get there. [GAO, 2006, p. 7] . If it does not announce routes, then there is no path through that network to that particular destination. The route announcement information does not relay information about capacity or quality of service. The route announcement may include localization information (i.e., MEDS, that the network would prefer to receive traffic destined for New York City at the interconnection point closes to New York City).
Routes that are within the receiving network's domain are OnNet and generally fall under peering. A receiving network can also announce routes to destinations that can be reached through the provider interconnecting with third party networks; these are OffNet and fall under transit.
Route Selection
The sending network listens to announcements and compiles a routing table. The routing table will contain list of known routes, blocks of IP addresses associated with each route, and cost metrics associated with each route. Some information comes from BGP announces; some the sending network adds to the table.
Based on the information in the routing table, the sending network will decide which route to use when sending traffic. The sending network looks in its routing table to see which networks provide a route to, for example, the destination address 192.104.54.5 and how many networks the packets have to go through. Based on that information, the router will select a route to send the packets off to, sending them off to the next hop, which will them do the same look up and make similar decisions, until the packets reach their destination.
- Network BAR (ASN 5) announces that it has routes to ASN 5 (itself) and ASN 7, ASN 8, and ASN 9 through ASN 8.
- Network FOO (ASN 4) hears BAR. FOO wants to send traffic to ASN 9. FOO hears that BAR provides a route to ASN 9 through ASN 8. FOO sends the traffic to BAR, the next hop.
- BAR now does exactly the same routing look up, to see what the best route would be to deliver the traffic to ASN 9*
* Note that a "Route Flap" can occur when FOO and BAR keep sending the traffic back and forth because their routing tables tell them that the other is the "best route" to the destination ASN 9.
Alternative Routes
If there is a choice of routes (if different networks are announcing routes to a destination), how does a sending network decide which route to utilize? A sending network will select which route to send traffic to based on the following criteria in the following order:
- Highest Local Preference
- Longer prefix [CSRIC Sec. 5.2 ("Longer prefixes represent more-specific routing information, so a longer prefix is always preferred over a shorter one. For instance, in this case the attacker might advertise 192.0.2.0/25, rather than 192.0.2.0/24, to make the false route to the destination appear more desirable than the real one.")]
[CISCO, BGP Best Path Selection "Local paths that are sourced by the network or redistribute commands are preferred over local aggregates that are sourced by the aggregate-address command."]
[NIST 800-189 Sec. 2.1 ("The rules for IP route selection on the internet always prefer the most specific (i.e., longest) matching entry in a router’s FIB. When an offending AS falsely announces a more-specific prefix (than a prefix announced by an authorized AS), the longer, unauthorized prefix will be widely accepted and used to route data.")]
- Lowest AS Path Length
- Lowest Origin Type [CISCO, BGP Best Path Selection ("IGP (interior gateway protocol) is lower than Exterior Gateway Protocol (EGP), and EGP is lower than INCOMPLETE.")]
- Lowest MED See [Danny McPherson, BGP: Good MEDS gone bad, NANOG 29]
- eBGP learned over iBGP learned
- Lowest IGP cost to border router
- Closest egress point (hot potato routing)
- Lowest Router ID (tie breaker)
[CISCO, BGP Best Path Selection]
Filtering
The sending network will engage in a certain degree of filtering of possible routes, removing prefixes that for instance your customer does not actually own, configuration mistakes, or routes involved in attacks. Almost every peering policy calls on a peering partner to filter routes. An announcing network will also filter out ASNs that it does not want to announce. An AS receiving route announcements should filter out AS paths that include their own AS number, in order to avoid a route loop. [NIST 800-189 Sec. 2.2]
Local Preference
Where there are alternative paths, there might be good business reasons for selecting one route over another. The sending network might select a customer's route over a free route (after all the customer is paying). The sending network might select a settlement free route over a route where it is the transit-customer. The sending network can assign "local preferences" to different routes so that route selection is made based on this criteria. For instance, the sending network assign values as follows:
- 90-99 customers
- 80-89 peers
- 70-79 transit providers
Route with the highest score takes the prize. [CISCO, BGP Best Path Selection] [CSRIC Sec. 5.2 ]
AS Path Length
When BAR announces that it has a route to ASN 9 through ASN 8, it is announcing a route and a path length. In this case the path length is 2 (two AS hops). If FOO was directly interconnected with ASN 9, ASN 9 would also be announcing a route to ASN 9 with a path length of 1. Under normal circumstances, FOO will listen to both BGP announcements, compare the path lengths, and send the traffic along the route with the shortest path length. In this case, FOOS would select to send the traffic directly to ASN 9 instead of sending it through BAR.
An announcing network can manipulate AS Path Length by making it appear that a route is longer than it is. An announcing network can prepend ASNs to its announcements to extend the AS Path Length. For example, in the example above, BAR made the announcement "ASN 8 ASN 9" - that it is a two hop route to ASN 9. If it makes the announcement "ASN 8 ASN 8 ASN 9," it now makes it seem like ASN 9 is three hops away, and influences the routing decisions of the sending network. BGP Best Path Selection and Manipulation, CISCO (2014)
NOTE: With the evolution of the Internet ecosystem and CDNs directly connecting to large BIAS providers at IXPs, one would anticipate that AS Path lengths would be shortening. An AS Path would include the large BIAS provider and the CDN if directly connected, or it could be the BIAS provider, an intermediary transit provider, and a CDN if indirectly connected.
Multiple Exit Discriminator (MEDs)
BAR can also announce MEDs. Basically BAR is announcing a localization preference that BAR wants traffic destined for a destination to be delivered near that destination (a.k.a. cold potato routing).
Simply because a receiving network announces MEDs does not mean that the sending network has to honor it. Generally the sending network will honor MEDs when the two networks have an interconnection contract with terms that specify provisions concerning MEDs.
References
"NSFNET introduced a complexity into the Internet, which the existing network protocols could not handle. Up to the NSFNET, the Internet consisted basically of the ARPAnet, with client networks stubbed off the ARPAnet backbone. I.e., the hierarchy between so-called Autonomous Systems (AS) was linear, with no loops/meshes, with the Exterior Gateway Protocol (EGP) used for for inter-AS routing carrying the AS Number of the routing neighbor. This made it impossible to detect loops in an environment where two or more separate national backbones with multiple interconnections exist, specifically the ARPAnet and the NSFNET. I defined that I needed an additional "previous" AS Number for the inter-AS routing to allow supporting a meshed Internet with many administrations for its components. Meetings with various constituents did not get us anywhere, and I needed it quickly, rather then creating a multi-year research project. In the end, Yakov Rekhter (IBM/NSFNET) and Kirk Lougheed (Cisco) designed a superset of what I needed on three napkins alongside an IETF meeting that included not just the "previous" AS Number but all previous AS numbers that an IP network number route had encountered since its origin. This protocol was called the Border Gateway Protocol (BGP) and versions of it are in use to this day to hold the Internet together. BGP used the Transmission Control Protocol (TCP) to make itself reliable. Use of TCP as well as general "not invented here" caused great problems with the rest of the Internet community, which we somewhat ignored as we had a pressing need, and soon with NSFNET, Cisco and gated implementations at hand, the Internet community did not have much of a choice. Eventually and after long arguments, BGP got adopted by the IETF." [Braun]
Autonomous System
"An AS is a connected group of one or more IP prefixes run by one or more network operators which has a SINGLE and CLEARLY DEFINED routing policy." IETF RFC 1930.
"An Autonomous System (AS) is a group of one or more IP prefixes run by one or more network operators that maintains a single, clearly defined routing policy. An IP prefix is a list of IP addresses that can be reached from that ISP’s network. The network operators must have an ASN to control routing within their networks and to exchange routing information with other ISPs." ARIN
Autonomous System: "A group of routers under a single administration." Service Provider Interconnection for Internet Protocol Best Effort Service, Network Reliability and Interoperability Council V, Focus Group 4: Interoperability, Sec. 1.2.2
Autonomous System Number
"An ASN is a globally unique number used to identify an Autonomous System. An ASN enables an AS to exchange exterior routing information with neighboring ASes." ARIN
"An AS has a globally unique number (sometimes referred to as an ASN, or Autonomous System Number) associated with it; this number is used in both the exchange of exterior routing information (between neighboring ASes), and as an identifier of the AS itself." IETF RFC 1930.
Prefix
In the current classless Internet (see [CIDR]), a block of class A, B, or C networks may be referred to by merely a prefix and a mask, so long as such a block of networks begins and ends on a power-of-two boundary. For example, the networks:
192.168.0.0/24
192.168.1.0/24
192.168.2.0/24
192.168.3.0/24
can be simply referred to as:
192.168.0.0/22
The term"prefix" as it is used here is equivalent to "CIDR block", and in simple terms may be thought of as a group of one or more networks. We use the term "network" to mean classful network, or "A, B, C network". [J. Hawkinson, T. Bates, Guidelines for Creation, Selection, and Registration of an Autonomous System, IETF RFC 1930, Sec. 3 Definitions (March 1996)]
Stub Network
"those networks that only provide connectivity to their end systems." [Kotikalapudi Sriram, Doug Montgomery, Resilient Interdomain Traffic Exchange: BGP Security and DDOS Mitigation, NIST SP 800-189 (Dec. 2019)] A stub network has no customer AS networks.
Hot / Cold Potato Routing
"Hot Potato Routing" is an interconnection policy between peers where one network hands off traffic to another network at the closest exchange point. If both networks follow Hot Potato Routing and if traffic levels are relatively balanced, then each network will relatively equally bare the cost of carrying the traffic. [NRIC Sec. 1.2.2 ("A form of inter-domain routing in which a packet destined
for a neighboring ISP is sent via the nearest interconnect
to that ISP.
")] [AT&T Ex Parte with Commission Pai, The Internet Interconnection Ecosystem, Slide 13, June 26, 2014. ("A form of inter-domain routing in which a packet destined for a neighboring ISP is sent via the nearest interconnect to that ISP.")]
The history of "Hot Potato Routing" has its routes back to Paul Baran. "Hot Potato Routing" for Baran was not so much a part of an interconnection / settlement scheme as much as a protocol to ensure reliability and resiliency. [Roberts, Computer Science Museum p. 14 1988]
Content Delivery Networks generally engage in "Cold Potato Routing," (a.k.a. "Best Exit Routing") holding onto traffic for as long as possible and handing it off as close to the eyeballs as possible, seeking to manage quality of service and defray the transit costs of the receiving networks.
[AT&T Ex Parte with Commission Pai, The Internet Interconnection Ecosystem, Slide 14, June 26, 2014 (diagraming cold potato routing).]
Internal Routing
In order to route traffic internally, networks use
- Internal Gateway Protocol
- Intermediate System to Intermediate System (IS-IS)
- Large BIASs tend to use IS-IS
- Open Shortest Path First (OSPF)
- MPLS
BGP Security
Derived From NIST SP 800-189 at 3
"A BGP prefix hijack occurs when an autonomous system (AS) accidentally or maliciously originates a prefix that it is not authorized (by the prefix owner) to originate. This is also known as false origination (or announcement). In contrast, if an AS is authorized to originate/announce a prefix by the prefix owner, then such a route origination/announcement is called legitimate. In the example illustrated in Figure 1, prefix 192.0.2.0/24 is legitimately originated by AS64500, but AS64510 falsely originates it. The path to the prefix via the false origin AS will be shorter for a subset of the ASes on the internet, and this subset of ASes will install the false route in their routing table or forwarding information base (FIB). That is, ASes for which AS64510 is closer (i.e., shorter AS path length) would choose the false announcement, and thus data traffic from clients in those ASes destined for the network 192.0.2/24 will be misrouted to AS64510.A BGP prefix hijack occurs when an autonomous system (AS) accidentally or maliciously originates a prefix that it is not authorized (by the prefix owner) to originate. This is also known as false origination (or announcement). In contrast, if an AS is authorized to originate/announce a prefix by the prefix owner, then such a route origination/announcement is called legitimate. In the example illustrated in Figure 1, prefix 192.0.2.0/24 is legitimately originated by AS64500, but AS64510 falsely originates it. The path to the prefix via the false origin AS will be shorter for a subset of the ASes on the internet, and this subset of ASes will install the false route in their routing table or forwarding information base (FIB). That is, ASes for which AS64510 is closer (i.e., shorter AS path length) would choose the false announcement, and thus data traffic from clients in those ASes destined for the network 192.0.2/24 will be misrouted to AS64510."
"The rules for IP route selection on the internet always prefer the most specific (i.e., longest) matching entry in a router’s FIB. When an offending AS falsely announces a more-specific prefix (than a prefix announced by an authorized AS), the longer, unauthorized prefix will be widely accepted and used to route data. Figure 1 also illustrates an example of unauthorized origination of unallocated (reserved) address space 240.18.0.0/20. Currently, 240.0.0.0/8 is reserved for future use [IANA-v4-r]. Similarly, an AS may also falsely originate allocated but currently unused address space. This is referred to as prefix squatting, where someone else’s unused prefix is temporarily announced and used to send spam or for some other malicious purpose."
"The various types of unauthorized prefix originations described above are called prefix hijacks or false origin announcements. The unauthorized announcement of a prefix longer than the legitimate announcement is called a sub-prefix hijack. The consequences of such adverse actions can be serious and include denial-of-service, eavesdropping, misdirection to imposter servers (to steal login credentials or inject malware), or defeat of IP reputation systems to launch spam email. There have been numerous incidents involving prefix hijacks in recent years. There are several commercial services and research projects that track and log anomalies in the global BGP routing system [BGPmon] [ThousandEyes] [BGPStream] [ARTEMIS]. Many of these sites provide detailed forensic analyses of observed attack scenarios."
Routing anomalies (aka, "hijacks," "false origin announcements") involve incorrect BGP route announcements
- "A BGP prefix hijack occurs when an autonomous system (AS) accidentally or maliciously originates a prefix that it is not authorized (by the prefix owner) to originate." [NIST 800-189 Sec. 2.1]
- Session hijacking: Attacker takes over BGP session between two border routers and injects new routing information. [CSRIC WG4 2013 Sec. 5.1.1 ("Despite these vulnerabilities having been widely known for a decade or more, they have not been implicated in any notable number of incidents.")]
- Compromised backbone routers
Misconfigured announcements can be of
- AS networks that the announcing network is not a route to (aka Route Hijack)
- AS networks that the announcing network is a route to but (a) the announcing network is falsely making itself look like the preferred or non-preferred route or (b) the announcing network is interconnected and has a path to the destination, but does not have a transit relationship to deliver traffic to the destination network
- "Route Leak." A route leak is defined as follows:
A route leak is the propagation of routing announcement(s) beyond their intended scope. That is, an announcement from an Autonomous System (AS) of a learned BGP route to another AS is in violation of the intended policies of the receiver, the sender, and/or one of the ASes along the preceding AS path. The intended scope is usually defined by a set of local redistribution/filtering policies distributed among the ASes involved. Often, these intended policies are defined in terms of the pair-wise peering business relationship between ASes (e.g., customer, transit provider, peer). [K. Sriram, D. Montgomery, D. McPherson, E. Osterwell, B. Dickson, Problem of Definition and Classification of BGP Route Leaks, IETF RFC 7908 (June 2016)]
[CSRIC Sec. 5.2] For example a stub network connected to two backbone networks, that announces to one backbone that it is a route to the other backbone - and starts transiting the backbone traffic, resulting in the stub networks infrastructure becoming overloaded with traffic [NIST 800-189 Sec. 2.3 ] A Route Leak, where a small stub network announces that it is a route between two larger upstream networks, typically overwhelms the infrastructure of the smaller network.
- Aftab Siddiqui, A Major BGP Hijack by AS55410-Vodafone Idea Ltd, MANRs Blog April 17, 2021
- Dan Goodin, BGP event sends European mobile traffic through China Telecom for 2 hours, Ars Technica June 8, 2019 ("The incident started around 9:43am UTC on Thursday (2:43am California time). That's when AS21217, the autonomous system belonging to Switzerland-based data center colocation company Safe Host, improperly updated its routers to advertise it was the proper path to reach what eventually would become more than 70,000 Internet routes comprising an estimated 368 million IP addresses. China Telecom's AS4134, which struck a network peering arrangement with Safe Host in 2017, almost immediately echoed those routes rather than dropping them,"); [Doug Madory, Large European Routing Leak Sends Traffic Through China Telecom, Oracle Blog June 6, 2019 archive.org ("China Telecom, a major international carrier, has still implemented neither the basic routing safeguards necessary both to prevent propagation of routing leaks nor the processes and procedures necessary to detect and remediate them in a timely manner when they inevitably occur. Two hours is a long time for a routing leak of this magnitude to stay in circulation, degrading global communications.")]
- A. Naik, “Internet Vulnerability Takes Down Google,” ThousandEyes report, November 2018 (Route Leak: "MainOne (a Nigerian ISP) leak of Google prefixes, which caused an outage of Google services for over an hourMainOne (a Nigerian ISP) leak of Google prefixes, which caused an outage of Google services for over an hour"); Dan Goodin, Google goes down after major BGP mishap routes traffic through China, Ars Technica Nov. 13, 2018 ("While Google said it had no reason to believe the mishap was a malicious hijacking attempt, the leak appeared suspicious to many, in part because it misdirected traffic to China Telecom, the Chinese government-owned provider that was recently caught improperly routing traffic belonging to a raft of Western carriers though mainland China.")
- Toonk, A., "Massive route leak causes Internet slowdown", BGPMON Blog, June 2015 ("massive Telekom Malaysia route leaks, which Level3, in turn, accepted and propagated")
- G. Huston, “Leaking Routes,” Asia Pacific Network Information Centre (APNIC) Blog, March 2012 (Route Leak: "the Dodo-Telstra incident in March 2012, which caused an outage of internet services nationwide in Australia")
- Alin C. Popescu, Brian J. Premore, and Todd Underwood, Anatomy of a Leak: AS9121. NANOG 34, May 16, 2005 ("24 Dec 2004: 100K+ routes leaked from AS9121 (TTnet), globally propagated")
The announced routing address space may be
Routing anomalies may be:
- unintentional (fat fingers, errors in traffic engineering)
- intentional (malicious)
- Spoof: Hijack, impersonate, phishing [CSRIC WG4 2013 Sec. 5.2 ("A server with an address in this range accepts logins from customers or users, such as a financial web site, or a site that hosts other sensitive information, or information of value")]
- Monitor, Espionage, Information Gathering
- Degrade, Block, DOS [CSRIC WG4 2013 Sec. 5.2 ("A server with an address in this range processes information crucial to the operation of a business the owner of AS65100 would like to damage in some way, such as a competitor, or a political entity under attack")]
[Andree Toonk, BGPMON, A Look at Recent BGP Routing Incidents, NANOG 63 ("Hijacking for Censorship Turkey Hijacking IP addresses for popular Global DNS providers")]
-
By persons
- authorized to have access to the routing infrastructure
- without authorization to have access to the routing infrastructure
- State actors (China, Pakistan (Youtube), Turkey, Syria). [Andree Toonk, BGPMON, A Look at Recent BGP Routing Incidents, NANOG 63 ] [Dan Goodin, Strange snafu misroutes domestic US Internet traffic through China Telecom, Ars Technica Nov. 6, 2018 ("traffic originating in Los Angeles first passed through a China Telecom facility in Hangzhou, China, before reaching its final stop in Washington, DC. The problematic route, which is visualized in the graphic above, was the result of China Telecom inserting itself into the inbound path of Verizon Asian Pacific.")]
Routing anomalies may result in [NIST 800-189 Sec. 2.1]
- DOS / Blackhole - preventing traffic to a valid destination from reaching that destination
- Rerouting of traffic in order to
- engage in surveillance, eavesdropping, traffic analysis, theft of login credentials
- Inject data such as malware, launch content such as spam
- drive traffic to the announcing network in order to increase transit revenue [NIST 800-189 Sec. 2.2]
Routing Security:
Prevention of routing anomalies
- Seek to verify address space allocation, vaild origin AS, valid AS path
- MANRs
- Action 1: "Prevent propagation of incorrect routing information: Network operator must implement a system whereby they only announce to adjacent networks the AS numbers and IP prefixes they or their customers are legitimately authorised to originate. Network operator must check whether the announcements of their customers are correct; specifically, that each customer legitimately holds the AS numbers and IP address space they announce." Compulsory [MANRs for Network Operators]
- Action 2: "A network operator should implement a system that enables source address validation for their own infrastructure and end users, and for any Single-Homed Stub Customer Networks. This should include anti-spoofing filtering to prevent packets with an incorrect source IP address from entering or leaving the network." Recommended See CAIDA Spoofer Software
- Action 3: "ensure that up-to-date contact information is entered and maintained in the appropriate RIR (or NIR) database and/or in PeeringDB." Compulsory
- Action 4: "Network operators must publicly document their intended routing announcements in the appropriate RIR routing registry, RADB or an RADB-mirrored IRR. " or RPKI. Compulsory
- MANRs Community report 2020 ("The number of reported routing incidents has been decreasing from more than 5,000 in 2017 to below 4,000 at the end of 2020, while the number of MANRS participants has been increasing ")
- NIST (RPKI, BGP origin validation, and prefix filtering)
- Registration of internet numbering resources with RIR and IANA [NIST 800-189 Sec. 4.1]
- Internet number resource holders should obtain RPKI certificates for their resources [NIST 800-189 Sec. 4.2]
- Internet number resource holders should digitally sign a route origin authorization. [NIST 800-189 Sec. 4.3]
"A BGP router can use the ROA information retrieved from an RPKI cache server to mitigate the risk of prefix hijacks and some forms of route leaks in advertised routes. A BGP router would typically receive a validated list of {prefix, maxlength, origin AS} tuples (derived from valid ROAs) from one or more RPKI cache servers. This list may be called a white list. The router makes use of this list with the BGP origin validation (BGP-OV) process depicted in Figure 7 to determine the validation state of an advertised route [RFC6811]. A BGP route is deemed to have a “Valid” origin if the {prefix, origin AS} pair in the advertised route can be corroborated with the list (i.e., the pair is permissible in accordance with at least one ROA; see Figure 7 for the details). A route is considered “Invalid” if there is a mismatch with the list (i.e., AS number does not match, or the prefix length exceeds maxlength; see Figure 7 for additional details). Further, a route is deemed “NotFound” if the prefix announced is not covered by any prefix in the white list (i.e., there is no ROA that contains a prefix that equals or subsumes the announced prefix)." [NIST 800-189 Sec. 4.3]
- CSRIC
- Cryptographic validation of exchange of routing information between peers in order to prevent session level man-in-the-middle routing information injection [CSRIC WG4 2013 Sec. 5.1.1.1]
- Prefix filtering. [CSRIC WG4 2013 Sec. 5.2.1.1 ("the customer communicating a list of prefixes and downstream ASes which they expect to be reachable through the connection to the ISP. The ISP will then craft a filter applied to the BGP session which explicitly enumerates this list of expected prefixes (a “prefix-list”), perhaps allowing for announcement of some more-specific prefixes within the ranges such as might be needed by the customer to achieve some goals in adjusting the load of the customer’s inbound traffic across their various connections. This configuration forms a “white-list”, in security parlance, of possible downstream destinations but does not validate the overall semantic correctness of the resulting routing table." Information is typically checked against registration records such as RIR records)]
- Internet Routing Registry: "IRRs allow providers to register information about their policies towards customers and other providers, and also allow network operators to register which address space they intend to originate. Some providers require their customers to register their address space in an IRR before accepting the customer’s routes, oftentimes the provider will “proxy register” information on the customer’s behalf since most customers are not versed in IRR details." [CSRIC WG4 2013 Sec. 5.2.1.2]
- AS Path Filtering [CSRIC WG4 2013 Sec. 5.2.1.3]
- Maximum Prefix cut off threshold [CSRIC WG4 2013 Sec. 5.2.1.4]
- Monitoring: network operators "monitor the prefixes for which they have authority to monitor for competing announcements which may have entered the BGP system." [CSRIC WG4 2013 Sec. 5.2.1.5]
Routing anomalies:
- Route Leaks
- Dan Goodin, Russian-controlled telecom hijacks financial services’ Internet traffic, Ars Technica April 27, 2017("Normally, the network traffic bound for MasterCard, Visa, and the other affected companies passes through services providers that the companies hire and authorize. Using BGP routing tables, the authorized providers "announce" their ownership of the large blocks of IP addresses belonging to the client companies. On Wednesday afternoon at around 3:36pm Pacific time, however, Rostelecom suddenly announced its control of the blocks. As a result, traffic flowing into the affected networks started passing through Rostelecom's routers.")
- China's 18-Minute Mystery, Renesys Blog Nov 18, 2010 archive.org ("Did China's government really divert 15% of the Internet's traffic for eighteen minutes in April, effortlessly intercepting sensitive traffic in flight, and generally creating a massively embarrassing man-in-the-middle attack on vulnerable global communications? Well, yes and no. Mostly no.")
- US-China Economic and Security Review Commission, Annual Report 2010 p. 243 ("For about 18 minutes on April 8, 2010, China Telecom advertised erroneous network traffic routes that instructed U.S. and other foreign Internet traffic to travel through Chinese servers. Other servers around the world quickly adopted these paths, routing all traffic to about 15 percent of the Internet’s destinations through servers located in China. This incident affected traffic to and from U.S. government (‘‘.gov’’) and military (‘‘.mil’’) sites, including those for the Senate, the army, the navy, the marine corps, the air force, the office of secretary of Defense, the National Aeronautics and Space Administration, the Department of Commerce, the National Oceanic and Atmospheric Administration, and many others. Certain commercial websites were also affected, such as those for Dell, Yahoo!, Microsoft, and IBM.")
YouTube | Pakistan
- Con-Ed Steals the 'Net Rensys Blog By Todd Underwood on January 22, 2006 11:06 PM archive.org
- House of Cards Rensys Blog By Earl Zmijewski on August 27, 2010 4:42 PM archive.org
- Reckless Driving on the Internet Renesys Blog By Earl Zmijewski on February 16, 2009 10:40 PM archive.org
- AfNOG Takes Byte Out of Internet Renesys blog By Earl Zmijewski on May 5, 2009 3:47 PM archive.org
- Longer is not always better Rensys Blog By Earl Zmijewski on February 21, 2009 2:30 PM archive.org ("This post is a follow-up to our blog last week about a small Czech provider briefly causing global Internet mayhem via a single errant routing announcement. In this incident, SuproNet (AS 47868) announced its one prefix, 94.125.216.0/21, to its backup provider, Sloane Park Property Trust (AS 29113), with an extremely long AS path.")
- late 1990s when a guy in a garage announced that he was the best route to UUNET, and suddenly all of UUNET's traffic was attempting to get through this poor guys garage.
Barriers to BGP Security
- Collective action problem. Problems of consensus on solutions [Andrei Robachevsky, ISOC, Two Years of good MANRS, Oct. 2016, Apricot 2017 ("From the routing perspective securing one’s own network does not make it more secure. The network security is in someone else’s hands")]
- Typical security investment problems (no perceived short term ROI). BGP Security measures can be expensive
- Limited deployment of RPKI / crypto solutions
[Fred Baker, Internet Routing with MANRS, MANRS (n.d.), (”Customers trust that their ISPs and IXPs will connect them to those entities with whom they want to communicate. Routing incidents, such as accepting or propagating a false prefix, are a fundamental service failure in that they connect their customers to someone else.”).]
Government Activity
- NIST
DHS
- Science and Technology Directorate :: Cybersecurity Projects :: Application of Network Measurement Science :: Predict, Assess Risk, Identify (and Mitigate) Disruptive Internet-scale Network Events Deepening Our Understanding of Internet Outages, DHS Science & Technology Blog Sept. 4, 2018 " One example of a NIDE we are studying is Border Gateway Protocol (BGP) hijacking. BGP routes traffic across the internet, and all networks connected to the internet rely on BGP to reach other networks. Researchers will measure BGP and examine connectivity issues caused by BGP hijacking. BGP hijacking occurs when a malicious attacker uses false network routing information to distort the internet’s common routing system. Incidents of these hijackings have blocked or derailed internet access for millions of people at a time."
- Secure Protocols for the Routing Infrastructure (SPRI) Sparta, Inc. (2006, Sept) Secure Protocols for the Routing Infrastructure (SPRI) Initiative: A Road Map (First Draft)
- Internet Infrastructure: DHS Faces Challenges in Developing a Joint Public/Private Recovery Plan, GAO Report 06-672, p 7 (June 2006) p. 7 "The Border Gateway Protocol—a protocol for routing packets between autonomous systems. This protocol is used by routers located at network nodes to direct traffic across the Internet. Typically, routers that use this protocol maintain a routing table that lists all feasible paths to a particular network. They also determine metrics associated with each path (such as cost, stability, and speed), so that the best available path can be chosen. This protocol is important because if a certain path becomes unavailable, the system will send data over the next best path."
FCC
- FCC CSRIC Reports:
- WG4 – BGP Security Best Practices (pdf) March 2013 [CSRIC WG4 2013 Sec. ]
- WG6 – Secure BGP Deployment (pdf)
- Service Provider Interconnection for Internet Protocol Best Effort Service, Network Reliability and Interoperability Council V, Focus Group 4: Interoperability, Sec. 1.2.2
- FCC CSRIC Best Practices
- 9-5-0524: Network Operators and Service Providers Should Operate a Route Database
- 9-7-0409: Routing Resiliency
- 9-7-0437: Route Aggregation
- 9-7-0438: CIDR Use
- 9-7-0520: Route Policy
- 9-7-8526: Recover from Interior Routing Table Corruption
- 9-7-8042: BGP Validation
- 9-7-8043: Prevent BGP Poisoning
- 9-7-8045: Protect Interior Routing Tables
- 9-7-8050: MPLS Configuration Security
- 9-8-8525: Recovery from BGP Poisoning
- 9-8-8531: Recovery from MPLS misconfiguration
- 9-8-8654-8658: Routing Integrity
- National Science Foundation
Network interconnection arrangements are announced through BGP. Organizations that listen to these announcements can develop a relatively accurate picture of who interconnects with whom and whether the arrangement is transit or peering. Because these are routing announcements, the organizations can detect what routes are announced, but not the financial terms of the arrangements. See also Internet eXchange Points, Backbones.
- CAIDA AS Rank
- DRAGON: Distributed Route Aggregation on the Global Network
- Cyclops at UCLA ("Cyclops is a network audit tool for service providers and enterprise networks, providing a mechanism to compare the observed behavior of the network and its intended behavior. Cyclops is able to detect several forms of route hijack attacks, i.e. when Internet routes are maliciously diverted from their original state. Recent incidents such as the Youtube hijack in Feb'08 show that route hijacking is currently a real threat in the Internet. ")
- BGPlay (route visualization tool)
- Hurricane Electric BGP Toolkit
- BGP Inspect MERIT
- Geoff Huston - BGP Table Growth
- Oracle Dyn (Renesys) Bakers Dozen
- Peeringdb
- Hurricane Electric BGP Toolkit
- CIDR Report (stats on AS networks, BGP)
- Route Views
- NIST RPKI Monitor "This NIST RPKI monitor is a test and measurement tool designed to monitor the dynamics of the global RPKI and the impact of RPKI-ROV on Internet routing. Its purpose is to provide measurement data and analyses to the research, standardization, and operations communities necessary to improve the trust and confidence in the underlying technologies."
- Traceroute
- Richard A Steenbergen, A Practical Guide to (Correctly) Troubleshooting with Traceroute, NANOG Tutorial
- Y. Zhang et al., "A Framework to Quantify the Pitfalls of Using Traceroute in AS-Level Topology Measurement," in IEEE Journal on Selected Areas in Communications, vol. 29, no. 9, pp. 1822-1836, October 2011, doi: 10.1109/JSAC.2011.111007
- CISCO BGPMON: "BGPmon monitors the routing of your prefixes and alerts you in case of an 'interesting' path change. "
- CISCO Thousandeyes "The company produces software that analyzes the performance of local and wide area networks"
- CAIDA BGP Stream "An open-source software framework for live and historical BGP data analysis, supporting scientific research, operational monitoring, and post-event analysis."
- BGP Artemis: "An open-source tool to monitor, detect, and mitigate BGP hijacks."
Organizations
- IETF
- Internet draft “Route Leak Attacks Against BGPSEC”, D. McPherson and S. Amante, Nov 2011.
- Internet draft “Threat Model for BGP Path Security”, S. Kent and A. Chi, Feb 2012.
- RFC 4272 “BGP Security Vulnerabilities Analysis” S. Murphy, Jan 2006
- RFC 4593 “Generic Threats to Routing Protocols”, A. Barbir, S. Murphy and Y. Yang, Oct 2006.
- MANRs
Papers
- Laurent Vanbever, Bruno Quoitin and Olivier Bonaventure, A Hierarchical Model for BGP Routing Policies, PRESTO’09 Friday, 21 Aug 2009
- R. Bush. (2011, Feb) BGPsec Operational Considerations.
- Kevin Butler, Toni R. Farley, Patrick McDaniel, and Jennifer Rexford, "A Survey of BGP Security Issues and Solutions," Proceedings of the IEEE, vol. 98, no. 1, pp. 100-122, Jan 2010
-
Debabrata Dash, Adrian Perrig, Hui Zhang Haowen Chan, "Modeling Adoptability of Secure BGP Protocols," in SIGCOMM'06, Pisa, Italy, 2006, pp. 279-290.
- BGP Case Studies, IP Routing, CISCO
- Nick Feamster, Zhuoquing Morley Mao, Jennifer Rexford, BorderGuard: Detecting Cold Potatoes from Peers (2004)
- Mike Freedman, Interdomain Routing Policy, COS 461: Computer Networks, (Spring 2011)
- Sharon Goldberg, Michael Schapira, Peter Hummon, and Jennifer Rexford, "How secure are secure interdomain routing protocols," in SIGCOMM '10, New Delhi, 2010.
- Geoffrey Goodell et al., "Working Around BGP: An Incremental Approach to Improving Security and Accuracy of Interdomain Routing," in NDSS Symposium 2003, Internet Society (ISOC), 2003.
- Yih-Chun Hu, Adrian Perrig, and Marvin Sirbu, "SPV: Secure Path Vector Routing for Securing BGP," in SIGCOMM’04, Portland, Oregon, 2004.
- Josh Karlin, Stephanie Forrest, and Jennifer Rexford, "Autonomous Security for Autonomous Systems," Computer Networks 52 (2008) 2908–2923, vol. 52, pp. 2908– 2923, June 2008.
- Stephen Kent
- S. Kent M. Lepinski. (2011, Feb) An Infrastructure to Support Secure Internet Routing.
- S. Kent. (2011, Feb) Threat Model for BGP Path Security.
- Stephen Kent, Charles Lynn, and Karen Seo, "Secure Border Gateway Protocol (S-BGP)," IEEE Journal on Selected Areas in Communications, vol. 18, no. 4, Apr 2000
.
- Stephen T. Kent, "Securing the Border Gateway Protocol," The Internet Protocol Journal, vol. 6, no. 3, Sept. 2003.
- R. Loomans, G. Michaelson G. Huston. (2011, Feb) A Profile for Resource Certificate Repository Structure.
- S. Murphy. (2006, Jan) RFC4272: BGP Security Vulnerabilities Analysis
- James Ng (Ed). (2004, Apr) Extensions to BGP to Support Secure Origin BGP (soBGP).
- John van Oppen, BGP Made Easy, TeamNANOG (June 15, 2016) (explaining how Internet networks announces routes to each other)
- Routing Attacks on the Privacy of Tor, 2015
- Barath Raghavan, Saurabh Panjwani, and Anton Mityagin, "Analysis of the SPV Secure Routing Protocol: Weaknesses and Lessons," ACM SIGCOMM Computer Communication Review, vol. 37, no. 2, Apr 2007.
- Monitor BGP Routes to and from Your Network, ThousandEyes
- S. Turner M. Lepinski. (2011, Mar) An Overview of BGPSEC
- Tao Wan, Evangelos Kranakis, and P.C. van Oorschot, "Pretty Secure BGP (psBGP)," in NDSS Symposium 2005, Internet Society (ISOC), 2005.
- "The Need for Routing in Complex Networking Systems or Why a Border Gateway Protocol," Hans Werner Braun, Jessica Yu, NSFNET LINK LETTER, Vol. 2, No. 4, September 1989
- Russ White, "Securing BGP Through Secure Origin BGP," The Internet Protocol Journal, vol. 6, no. 3, Sept. 2003.
- Meiyuan Zhao, Sean Smith, and David M. Nicol, "The Performance Impact of BGP Security," IEEE Network, Nov 2005.
News
- Chris C. Demchak & Yuvall Shavit, China’s Maxim: Leave No Access Point Unexploited: The Hidden Story of China Telecom’s BGP Hijacking, Military Cyber Affairs vol. 3 issue 1 (2018).
- Andrei Robachevsky, 14,000 Incidents: A 2017 Routing Security Year in Review, Internet Society jan 9, 2018
- Craig Timberg, The Long Life of a Quick Fix, Wash Post, May 31, 2015
- Sharon Goldberg, Why Is It Taking So Long to Secure Internet Routing? 12 Acmqueue 1 (2014)
