Google Global Cache for Carrier

Grade NAT - Configuration Guide

Release Beta

March 06, 2014

Google Global Cache

ggc@google.com

Contents

Introduction

Planning Your Network Topology

BGP Configuration

Notifying Google

Changing Routing for Privately Addressed Users

Testing and Diagnostics

6.1 Playing a Test Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Verifying your Public IP address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Verifying your Private IP address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11
11
12
12

ii

CHAPTER 1

Introduction

This document describes how to configure Google Global Cache (GGC) nodes to operate with your Carrier Grade
NAT (CGN) systems. Once configured, this will allow traffic between the GGC node and private user IPs to
bypass the CGN.
GGC has no requirement that private IP space be in the RFC1918 allocation. Any non-internet-routed space in
use in your network can bypass a CGN with this configuration.
The process for enabling CGN support for a GGC node has a number of steps. These are described in detail in the
remainder of this document.
1. Planning your network topology
2. BGP configuration
3. Notifying Google
4. Changing routing for privately addressed users
5. Testing and Diagnostics
Please see the Google Global Cache - Installation and Operations Guide 1 for installing a GGC node to serve
general users.

The GGC Installation and Operations Guide is available for download

Dell Poweredge R710 Guide: https://peering.google.com/static/downloads/GGCInstallation-R710.pdf
Dell Poweredge R720xd Guide: https://peering.google.com/static/downloads/GGCInstallation-R720.pdf

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

Chapter 1. Introduction

CHAPTER 2

Planning Your Network Topology

The following figure (CGN reference topology) shows a hypothetical network topology used as an example for
this document.
Traffic between the users in the private IP space and the general Internet is NATted. A route will be added to
allow traffic between these users and the High Traffic IPs in the GGC sub-net to bypass the CGN.
Google will provide you with a list of High Traffic IPs for each GGC node.

Figure 2.1: CGN reference topology

In the route illustration above:

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

A user with IP 192.168.1.2 can reach a High Traffic IP (e.g. 208.117.227.36) on the GGC servers, bypassing your CGN. The GGC server will see the users source IP as 192.168.1.2. It will be able to send traffic
back to 192.168.1.2, bypassing the CGN servers. [red path]
A user with IP 192.168.1.2 can reach all GGC IPs other than the High Traffic IPs (e.g., 208.117.227.31)
through the CGN. The GGC server on 208.117.227.31 will see the users source IP NATted as
208.117.8.0/24. It will be able to send traffic back to 192.168.1.2 through the CGN servers. [green path]
Traffic from the user 192.168.1.2 is directed towards sites on the public Internet through the CGN. The
public Internet will see the user coming from a NATted IP in 208.117.8.0/24. [green path]
Traffic between other users, in publicly routable space, and all GGC server IPs is routed independently of
the CGN. It is seen directly by the GGC servers (and any other server on the public Internet) with a public
non-NATted source address. [blue path]
Note: Do not configure private user IP addresses to bypass the CGN, until you have received confirmation from
Google that the GGC nodes are ready for CGN-bypassed traffic. Changing your routing without prior approval is
likely to lead to very poor performance for users. It may even cause complete failure for some user requests.

Chapter 2. Planning Your Network Topology

CHAPTER 3

BGP Configuration

Accepting traffic from private IP space requires special configuration of the prefixes advertised at GGC nodes.
We need to know which prefixes are private user IPs, and which prefixes contain public addresses of your CGN
servers. Our systems can then correctly serve these users from all Google data locations.
Note: Incorrectly configured community tags will mean Google systems will not be able to distinguish which
user requests originate in private IP space. This will cause traffic to be served from data locations other than the
GGC node.
The following community tags are REQUIRED:
Table 3.1: BGP communities for NAT devices & users
Community
15169:12000
15169:12100

NAT Configuration Range

Indicates a prefix contains a NAT device. These should be specified as /24 prefixes
Indicates a prefix is assigned to users who are NATed (it is either RFC1918, RFC6598,
or usurped globally unique address space)

In the example network, the advertisements would be:

Table 3.2: BGP communities example
CIDR
208.117.8.0/24
192.168.0.0/16
208.0.0.0/8

Usage
CGN device
Users routed via CGN
Users not routed via CGN

Community
15169:12000
15169:12100
none

We require all GGC nodes in the same Google Network Location (GNL)
advertised to them.

to have the same prefixes and tags

You can configure these community tags now. Until Google enable CGN support for the node, and you reconfigure
your routing, users will continue to be served via the public IP addresses of your CGN servers.
Changes to your BGP configuration may take up to 2 hours to be seen in all Google systems
For general details about configuring BGP for GGC, please see Google Global Cache - Installation and Operations Guide. For more detail on use of community tags, please see BGP Community Support for Google
Serving

1 A Google Network Location (GNL) is a set of GGC nodes serving the same users, with the same failover policy. GNLs for your network
are shown in the Google Peering Portal https://peering.google.com/failover_visualizer/

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

Chapter 3. BGP Configuration

CHAPTER 4

Notifying Google

Once you have planned your network topology and added community tags to your BGP advertisements, please inform GGC Operations that the node is ready for CGN support. You should do this by emailing <ggc@google.com>
Please include the following information:
GGC node names, for all nodes in this GNL 1
Confirmation that you have added community tags to your advertisements, at all nodes in the GNL
Confirmation that you are ready to bypass the CGN for connections to High Traffic IPs
Any special scheduling requirements
Google will verify your prefix advertisements and the GGC node configurations.
We will then provide you with:
A list of High Traffic IPs on the GGC servers, for you to use in your routing configuration
A proposed schedule for final configuration of CGN support on the GGC nodes

GGC node names are listed in the Google Peering Portal: https://peering.google.com/

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

Chapter 4. Notifying Google

CHAPTER 5

Changing Routing for Privately

Addressed Users

Once Google have enabled CGN support for the GGC nodes in this GNL, we will let you know that it is ready for
CGN-bypassed traffic.
When we have done this, you can then re-configure your routing, for all nodes in the GNL.
You should ensure that all traffic between private user IPs tagged with 15169:12100, and the High Traffic IPs on
the GGC servers, bypasses the CGN.
Let us know when you have made this change, and we will verify that we see traffic directly from private user IPs.

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

10

Chapter 5. Changing Routing for Privately Addressed Users

CHAPTER 6

Testing and Diagnostics

6.1 Playing a Test Video

Follow the procedure below to play a test video. If the system works as intended, the video traffic should be served
from the GGC node, bypassing the CGN. This procedure is similar to the test procedure in the GGC - Installation
and Operations Guide.
Using Google Chrome from a PC in one of the tagged private user IP ranges:
1. Open the Developers Tools (Menu Button > Tools > Developer Tools or Shift+Ctrl+I)
2. Open the Network tab
3. Play a popular video, such as http://www.youtube.com/watch?v=dQw4w9WgXcQ
4. Skip any ads. Ensure that the video starts to play properly.
5. Watch the item videoplayback request in the Network tab. This shows the name of server the video is
actually played from:

Figure 6.1: Developers Tools in Google Chrome

6. Resolve this server name using nslookup or similar tools:

11

Google Global Cache for Carrier Grade NAT - Configuration Guide, Release Beta

$ nslookup r3---sn-bjvg2-1gie.c.youtube.com
Server:
<your_name_server>
Address:
<your_name_server IP>
Non-authoritative answer:
r3---sn-bjvg2-1gie.c.youtube.com
canonical name = r3.sn-bjvg2-1gie.c.youtube.com.
Name:
r3.sn-bjvg2-1gie.c.youtube.com
Address: 193.142.125.14
If the resulting address (193.142.125.14 in this example) is an address in the sub-net allocated to the GGC
node, then video is playing correctly from the cache.
Note: The base web pages of www.youtube.com may not be served from the cache. These host names will
typically not resolve to the GGC node, and will not bypass the CGN
You can use FireFox as well to perform this test, but you will need to install the FireBug extension.

6.2 Verifying your Public IP address

The following URLs provide information on how Google systems see your IP address. This will assist us in
determining if CGN-bypass is working correctly.
This URL should display the public address of your CGN servers, and the GGC node name.
http://redirector.c.youtube.com/report_mapping
The response should look something like:
208.117.8.1 => yourispname-abc1

6.3 Verifying your Private IP address

You will need to specify one of the High traffic IPs from the list given to you by Google. In this example it is
208.117.227.36.
This URL should display your private IP: http://208.117.227.36/debug_info
The response should look something like:
client_ip = 192.168.1.2
private_ip_ranges match: true
exempted_ip_ranges match: false
url_ip =
nat_servers_ip_ranges match: false
destination_ip = 208.117.227.36
date_time = 2013/10/10-21:26:15.424
validation status code = SIGNATURE_MISMATCH
Check that the response includes a tagged private address for client_ip and has private_ip_ranges
match: true

12

Chapter 6. Testing and Diagnostics