Understanding Carrier Grade NAT
Understanding Carrier Grade NAT
The fact of the matter is that we actually ran out of globally unique IPv4
addresses a long time ago. You may ask (sounding a little like Tevye), “How
can this be? The IANA still has 28 /8 IPv4 address blocks that have not yet
been allocated. That’s about 469 million addresses. We have not run out of
IPv4 addresses!”
That worked fine in the 1990s, but more and more these days network-
connected devices want to be “always on.” That means devices hold on to the
addresses they’re given, reducing the number of addresses that can be shared
on an as-needed basis.
The other means of sharing globally unique IPv4 addresses, and by far the
more successful, is the venerable Network Address Translation (NAT).
Specifically NAT44, which translates an IPv4 address to another IPv4
address. Private (RFC1918) IPv4 addresses are used within a given network;
the assumption here being that within a network most IP devices only want to
talk to other IP devices in the same network.
But because RFC1918 addresses can be reused in different networks, they are
not globally unique. When a device on one network wants to talk to a device in
a different administrative domain, reachable across the public Internet, there is
no guarantee that the two devices are not using the same address. There is
also no way for routers in the public domain to differentiate between networks
using the same address blocks.
If NAT44 did no more than I have described so far, it would have the same
problem as dynamic address allocation: The pool of available addresses would
not scale to the demands of modern networks of “always-on” devices. The
assumption that most network-internal devices talk to other network-internal
devices most of the time is no longer valid, as more and more data exchanges
are across the public Internet.
NAT44 overcomes this scaling problem by using not only its pool of public IPv4
addresses but also the port numbers available with each of the addresses. TCP
and UDP headers support up to 65,536 port numbers, most of which are
unused. So by mapping an internal [private address, port] tuple to an outside
[public address, port] tuple, NAT44 is really mapping sessions rather than
devices and can support a very large number of sessions with each public
address.
NAT has been around since the early to mid 1990s, when it was one of several
short-term solutions (along with DHCP, RFC1918 addresses, and CIDR)
created to slow the depletion of IPv4 while a next-generation Internet Protocol –
which eventually became IPv6 – was developed. Back then it was just called
NAT, not NAT44, because translating an IPv4 address to another IPv4 address
was the only option there was. There was no need to distinguish it from, say,
NAT64, because IPv6 did not yet exist and so something that could translate
between IPv4 and IPv6 did not yet exist either.
Getting back to the dual stack dilemma that service providers face: Why not
NAT the IPv4 half of dual stack connections to customers? That gets customers
started on IPv6 while still providing IPv4 connectivity.
This is the basis for Carrier Grade NAT (CGN). Traditional NAT – CPE NAT
– appears at the edge of the customer network where it connects to a service
provider, and translates between private IPv4 addresses within the customer
network and one or a few public addresses assigned by the provider. CGN
appears within the service provider network. It, too, translates between private
and public IPv4 addresses; the private side of the CGN faces the provider’s
customers.
I other words, CGN enables service providers to assign private RFC 1918 IPv4
addresses to their customers rather than public, globally unique IPv4
addresses. Once again NAT comes to the rescue of a dwindling address
supply.
It also raises the question: Why not just use CGN and forget about IPv6? I’ll
address that question (forgive the pun) in a later post, after we’ve looked further
into the various CGN issues and architectural options.
The first issue around CGN is the name. There is no real distinction between a
CGN and any other NAT, other than the assumption that if it appears in a
service provider network it is probably going to be on a carrier class router. But
there are no specified qualities that make CGN “carrier grade” in and of itself.
Because nothing really defines the “Carrier Grade” part of CGN, a movement is
afoot to change the name to “Large Scale NAT” (LSN). I kicked off this article
using the term CGN, because that’s still the term most people are familiar with.
But I’m going to pull a switcheroo and use “Large Scale NAT” in subsequent
articles, now that I’ve had a chance to explain why the name is changing: LSN
better reflects the intent of a NAT that serves a large number of service provider
customers, say in a single PoP, and must scale to a massive amount of
customer traffic.
Also by taking the “carrier” reference out of the name it better reflects the fact
that LSN is a NAT solution that can be used in more than just service provider
scenarios. It can be useful in any large-scale network.
Apart from the need to scale, the other big issue with LSN is the fact that it is
used to assign RFC 1918 addresses to customers, which then use the
addresses on the “public” side of their own NATs. Those CPE NATs are then
translating from customer-side RFC 1918 addresses to provider-side RFC 1918
addresses. So end-user traffic is now passing through a two-tiered or
hierarchical NAT architecture rather than a single device.
That architecture, and the issues that arise from having private IPv4 addresses
between the LSN and the CPE NATs, are the subjects of my next post
Large Scale NAT Architectures
NAT444 and NAT 464
LSN adds another layer of translation, so that just as private IPv4 addresses are
used on the inside of the CPE NAT they can also be used to assign addresses
to the outside of the CPE NAT.
LEARN MORE
Accurate routing from the Internet to the correct device within the correct
customer network depends on two things:
1) The session was initiated from within the customer network, so that the
CPE NAT and the LSN have the correct address and port mappings; and
2) The route view from the outside is always toward a destination that can be
uniquely identified. So a packet from the public Internet can be routed to the
service provider’s unique IPv4 aggregate address; once within the service
provider network, a more-specific route is used to send the packet to a
particular LSN (which is usually a software process on some edge or
aggregation router). The LSN has an outside-to-inside address/port mapping
that points to a specific CPE NAT, and the CPE NAT has another outside-to-
inside address/port mapping that either takes the packet to a directly connected
destination or matches a route that is unique within the boundaries of that
customer’s network.
While it is simple, NAT444 is not without issues. One concern common not only
to this architecture but any LSN solution is scalability. For a broadband provider,
each customer network can represent several devices behind the CPE NAT in a
home or perhaps a dozen devices behind the CPE NAT in a small office. And
each of those devices can produce multiple application streams. There is not
yet enough production experience to know the upper boundaries of how many
customer networks a single LSN can handle, either in terms of performance or
in terms of mapping steams to a single public IPv4 address.
Another issue with NAT444 occurs when a customer wants to send packets to
another customer behind the same LSN, as illustrated in Figure 2. Filtering
policies in firewalls, router ACLs, and even servers often block packets from
outside the network that have private source addresses. To circumvent this
problem packets must pass through the LSN so that their source addresses can
be translated to a public address and then “hairpinned” back through the LSN to
the destination. LSN resources are consumed even though the packets are not
going to a destination beyond the LSN.
Another solution to these problems is to use public IPv6 addresses between the
LSN and the CPE NATs, as shown in Figure 3. IPv4 packets originating in the
customer network are translated into IPv6 packets for the hop between the CPE
NAT and the LSN, and then translated back to IPv4 (with a public address) by
the LSN. This architecture is NAT464.
While NAT464 simplifies things in the intermediate zone between the LSN and
CPE NATs, it is more problematic for the LSN and CPE NATs themselves. The
most obvious problem is that both these devices must now be NAT64 – that is,
they must translate between the two protocol versions. Very few current CPE
NATs support NAT64, so providers adopting this solution are faced with
requiring their customers to change their equipment: an administrative and
customer relations headache most providers would rather avoid.
Practical advice for you to take full advantage of the benefits of APM and keep
your IT environment
LEARN MORE
· Almost all services accessible on the public Internet are still IPv4 only
LSN – the newer, more accurate name for what was previously called Carrier
Grade NAT (CGN) – is simply a NAT that is located in a service provider
network rather than in a customer network. It is almost always a service on a
router rather than a standalone device; whether LSNs live up to “carrier grade”
performance or scaling expectations is yet to be seen.
An alternative to using IPv4 shared addresses between the provider and the
customers is to use IPv6 addresses. This solves the same problems that are
solved by shared addresses, and eases the administrative burden of the
provider having to assign and manage both an IPv4 address and an IPv6
address on every link. It also gets a bit closer to where the provider ultimately
wants to be: An IPv6-only infrastructure. The downside to this NAT464
architecture is that both the CPE NAT and the LSN must perform translation
between IPv4 and IPv6 (NAT64), which is complex and introduces
performance, scaling, and redundancy problems.
Dual-Stack Lite is a promising approach that takes the best of NAT464 while
avoiding its problems: It uses IPv6-only links between the provider and the
customer, but does not use NAT64 translation. When a device in the customer
network sends an IPv4 packet to an external destination, the IPv4 packet is
encapsulated in an IPv6 packet for transport into the provider network. At the
LSN, the packet is decapsulated and NAT44 is performed (Figure 1). Tunneling
IPv4 over IPv6 is far simpler than translation, so the performance and
redundancy concerns are eliminated.
If a simple mapping between inside IPv4 source address / port to outside IPv4
source address / port was performed on outgoing packets, as is done with
regular NAT44, the LSN would have no way to differentiate between
overlapping RFC1918 IPv4 addresses in different customer networks. Therefore
an additional element is added to the address mapping: The source address of
the encapsulating IPv6 packet (the address of the customer end of the IPv6
link) is added to the inside IPv4 source address and port. Because the IPv6
address is unique to each customer, the combination of IPv6 source address +
IPv4 source address + port makes the mapping unambiguous. When a
responding IPv4 packet is received from the outside, its IPv4 destination
address and port can be correctly matched to a specific customer behind the
NAT based on the IPv6 address in the mapping table; the packet’s IPv4
destination address and port can then be mapped to the inside IPv4 destination
address and port, encapsulated in IPv6 using the mapped IPv6 address as the
IPv6 destination address, and forwarded to the customer.
In other words, the mapped IPv6 address not only disambiguates the customer
RFC1918 address, it provides the reference for the tunnel endpoint.
Assuming there are multiple end systems in the customer network, the DS Lite
function occurs on a CPE device such as a home gateway as shown in Figure
2. If a device sends an IPv6 packet, the packet is routed normally to the IPv6
destination. If a device sends an IPv4 packet, the CPE gateway performs the
IPv4-in-IPv6 encapsulation, setting the destination address of the IPv6 packet to
the address of the DS Lite enabled LSN. This model allows use of dual stacked,
IPv4-only, and IPv6-only devices behind the gateway.
Comcast is the driving force behind DS Lite, and through Cablelabs DS Lite
CPE development is well underway. Rumor has it that on the LSN end, the “big
two” router vendors are developing DS Lite support. It’s a smart, reasonably
simple solution to the dual stack problem, and I think we can expect to begin
seeing DS Lite deployments soon.
NAT444 (CGN/LSN) and What it Breaks
by CHRIS GRUNDEMANN on 02/14/2011
Before we look at what breaks, I should probably make sure that you know
what it is that I’m talking about here. If you already know all
about traditional NAT and address overloading, skip to the NAT444 section. If
you are familiar with that as well, feel free to skip right to the list ofwhat it
breaks. In any case; enjoy! =)
On the far left of this diagram we have two hosts in our local network (a
laptop and a tablet), the Internet is on the right, and there’s a NAT overload
(NAPT) router in the middle connecting them. You can see that both of our
local network hosts have locally unique (“private”) inside addresses
(192.168.1.23 and .42) and the router has a single globally unique (“public”)
outside address (203.0.113.57).
When the laptop wants to connect to another host out on the Internet, it
sends its traffic to the router (it’s default gateway). The router takes that
traffic and swaps it’s outside (global/public) address in place of the laptop’s
inside (local/private) address and sends the traffic on, to the Internet. Now
all Internet devices see this traffic as if it came from the router itself, so they
send return traffic directly to the router. When the router receives that
return traffic, it swaps the laptop’s inside address back in place of it’s own
outside address and sends the traffic on, to the laptop. Simple enough.
Things get more interesting when both the laptop and the tablet need to talk
to the Internet at the same time though. Now the router needs to know what
return traffic goes to the tablet and which traffic is supposed to go to the
laptop. This is where those port numbers I mentioned earlier come into play.
Each two-way connection between network hosts is called a session (or
sometimes a flow) and to keep these sessions from getting mixed up, the
router uses a unique port number to identify each one.
Here you can see this in action. In this diagram, both the laptop and the
tablet are sending traffic to the Internet (initiating one session each). When
the router does the address translation (the swap), it not only changes the
address but also adds a unique port number when sending the traffic out to
the Internet (:2001 and :2002 in the diagram). In this way, when the router
receives return traffic that is addressed to 203.0.113.57:2001 it knows that
this should be swapped for 192.168.1.42 (which is sent to the laptop) and that
203.0.113.57:2002 translates to 192.168.1.23 and goes to the tablet. Easy
peasy. It’s kind of like having a post office box. All the mail comes to the
same physical street address (the outside IP address) but each letter is
addressed to a unique box number (the port number), so everybody gets the
right mail.
Of course, this example is very simple. In real life most networks have more
than two hosts and almost all hosts open up many more than one session.
Since each IP address has about 65,000 ports available, that is the absolute
maximum number of sessions that one address can handle. This is plenty for
most small networks, and larger ones can typically add a few outside
addresses to what is called a NAT pool, so this scales pretty well at the LAN
level. There are problems with this approach however.
The primary issue with NAT (we are still talking about NAPT/NAT-overlaod but
just saying NAT is easier, it’s what most folks mean when they say NAT
anyway) is that it breaks the end to end principle of networking. WTF does
that mean? Well, basically, the end to end principle states that network
communication should happen between the devices at the very edges of the
network and that all devices in the middle should just pass that traffic along.
This is a primary design goal of the Internet and for good reason. Without
delving into all the gory philosophical details, we can over-simplify again and
say that, in practical terms, the end to end principle allows network hosts to
talk to each other unobstructed.
Aha! Now we start to see the problem with NAT: It introduces an obstruction
into the communication path. In order for NAT to work, the device doing the
NATing has to mangle each and every packet of data being sent and received.
It must dig into those packets’headers and change the source or destination
address (and port number). The primary effect of this is that network hosts
behind an overloaded NAT must initiate all communication. That is; inbound
communication attempts are impossible because there is no way for a host
outside of the local network to know or understand the inside (local/private)
address of that host.
If we look at our laptop in the examples above, it has an RFC 1918 address of
192.168.1.42. Because this is a “private” address from a shared pool, it can’t
be advertised on the public Internet. This means that other hosts on the
Internet have no way to know the laptops address (and even if they did, there
would be no route to get to it). See, we need that outside address and port
combination set up on the router before any traffic can get in, to the laptop,
so that the router knows where the traffic is supposed to go. This is fine for
web browsing and many other client side applications but it is a major
problem for server and peer to peer applications (VoIP, gaming, webcams,
VPNs, bittorrent, video streaming, chat, etc) where communication needs to
be initiated from the outside in. Major meaning they don’t work.
Because there are simply not enough IPv4 addresses to provide a globally
unique (public) address to all of the devices already connected to the
Internet, network operators have been forced to deploy NAT in the LAN. As
such, tools have been developed to help combat the problems it causes.
One notable hack is enabled through UPnP (Universal Plug and Play) and is
called Internet Gateway Device (IGD) Protocol. UPnP-IGD is a protocol that
gives the hosts behind a NAT some control over the NAT device. Things like
discovering the router’s outside IP address and setting up static port
mappings. This allows many of the applications that would otherwise be
broken by the NAT to work in spite of it. NAT Port Mapping Protocol (NAT-
PMP) offers similar functionality.
There are several other NAT traversal techniques and protocols that have
been developed to get around the problems that NAT causes. These various
tools are together a looming testament to two things:
1. Human ingenuity and our ability to overcome obstacles.
2. The brokenness that NAT introduces into inter-networking.
Security
Stateful packet inspection is what you want for network access security, not
NAT brokenness.
The NAT444 Model
So far we have discussed traditional NAT (mainly NAPT/NAT overload) which is
starting to be called NAT44 because it translates one IPv4 address for another
(4 to 4). Now let’s explore what I (still) call NAT444, which was also called
Carrier Grade NAT (CGN) at one point and is currently called Large Scale NAT
(LSN) by all the cool geeks. I like NAT444 because it explains what is really
going to occur in most places when LSN is implemented; a triple NAT (IPv4 to
IPv4 to IPv4). Sounds like a nightmare already, doesn’t it? We just doubled the
NAT, which means doubling the interference with network traffic and further
impeding the end to end principle. Let’s see what that looks like.
This diagram illustrates the NAT444 model as envisioned in the IETF NAT444
draft and as it will most likely be deployed (unless we can figure out a way
around it altogether). In this diagram you should notice two things right off
the bat:
1. Large Scale NAT for IPv4 will be deployed dual-stacked with global
(“public”) IPv6.
2. Large Scale NAT adds a second layer of NAT and thus a second area of
“private”/inside addressing.
As you can guess from point two, NAT444 exacerbates all the problems that
traditional NAT44 introduced. What may not be obvious at first is that
LSN/NAT444 aggravates those issues in new ways as well.
In addition to adding a second layer of NAT that creates major problems with
law enforcement and abuse logging as well as geolocation and others (all
because many distinct customers are behind one provider address), we have
to deal with the fact that the second layer of NAT is not going to participate
in UPnP, NAT-PMP or other LAN-based NAT traversal protocols. No ISP
(Internet Service Provider) in their right mind is going to open up their own
routers (or other network devices) to customer control – which is exactly what
these protocols require. They are simply not secure and the risk of one
customer being able to impact other customers’ service is too great. So where
does that leave us?
What NAT444 Breaks
We are left with a number of applications (and application types) that
currently break when Large Scale NAT is introduced. To avoid the doom and
gloom feeling that is sure to follow a list of just the broken stuff, let’s start
with a list of what isn’t broken by NAT444/LSN:
Web browsing
Email
FTP download
Small files
BitTorrent and Limewire
Leeching (download)
Skype video and voice calls
Instant messaging
Facebook and Twitter chat
Not too shabby really, all things considered. That is quite a bit of
functionality for being behind a fairly large kludge. If that were the end of the
story I wouldn’t have written this article though. So, without further adieu,
here is the list you’ve been waiting for; what NAT444 breaks:
FTP download
Large files
BitTorrent and Limewire
Seeding (upload)
On-line gaming
Xbox
PlayStation
Etc.
Video streaming
Hulu
Netflix
Slingcatcher
Etc.
Webcam
Remote viewing
Tunneling
6to4
Teredo
Etc.
VPN & Encryption
IPSec
SSL
VoIP
Limited ALG/SIP support
All custom applications with the IP embedded
Lack of ALGs
Wow, is it just me or is that list a bit longer? There’s that doom and gloom
feeling creeping up.
For our purposes here, “breaks” means that the service was degraded or
completely failed. The data behind this list primarily comes from Assessing
the Impact of NAT444 on Network Applications, an IETF draft which
documents testing that was done by CableLabs, Time Warner Cable, and
Rogers Communication on “many popular Internet services using a variety of
test scenarios, network topologies, and vendor equipment.” If this kind of
thing interests you at all, I highly recommend checking out the full draft, it’s
a quick and informative read. I also have a bit of experience dealing with
NAT444 myself, but that’s a story for another day.
The Solution
You knew I was going to say it eventually: The only true solution is to deploy
IPv6. This is why the NAT444 model includes global IPv6; the only way to get
around the brokenness introduced by NAT is to eliminate it. Luckily we have
the means to do so; Internet Protocol version 6 (IPv6). So, dual-stack today,
with NAT444 if you must, and then do everything you can to get everyone you
do business with to do the same.
Carrier Grade NAT
1 Defectos
2 Escenario de utilización
3 Véase también
4 Referencias
5 Enlaces externos
Defectos
Sus críticos le encuentran los siguientes defectos:
Escenario de utilización
Un escenario de utilización del NAT masivo se puede describir como NAT444,3 porque las
conexiones de algunos clientes a los servidores públicos pueden llegar a pasar por tres
dominios de direccionamiento IPv4 diferentes: el propio de la red privada del usuario, la
red privada del operador e Internet.
Otro escenario de NAT masivo es Dual-Stack Lite, donde la red del operador utiliza IPv6 y
por tanto sólo se necesitan dos dominios de direccionamiento IPv4.
LEARN MORE
The “awkward period” we are entering is caused by several unavoidable facts:
1) With few exceptions, the only new globally unique unicast addresses you
can get from your regional address registries are IPv6. Those organizations are
becoming very protective of the few IPv4 blocks they have left, and even those
will be gone in short order.
2) The vast majority of the user devices connecting to theInternet are IPv4.
3) The vast majority of content accessible over the Internet is IPv4 only.
5) Although modern operating systems are fully IPv6 capable (Windows Vista,
MAC OS, most flavors of Unix), there is still a huge installed base of operating
systems that either have limited IPv6 support (Windows XP) or none at all.
6) Service providers are faced with growing their network using IPv6, while
continuing to serve legacy IPv4 customers. They are going tobe coping with
some unwieldy logical architectures for a while.
Coping with the early transitional years, in which most content and users are
still on IPv4 but the only new addresses are IPv6, falls heavily on service
providers. They cannot continue giving customers globally routable IPv4
addresses, they cannot get new globally routable IPv4 addresses for expanding
their own networks, and yet they must continue to serve both legacy IPv4
customers and new customers – all of whom are primarily trying to reach IPv4
destinations.
Keep in mind that the vast majority of broadband customers could care less
whether their application traffic is riding over IPv4 or IPv6, or even know what IP
is. Customers care about services, and how well those services are delivered.
So any effort by a service provider to introduce IPv6 that reduces service quality
or inconveniences their customers is going to be costly. In an intensely
competitive market, even perceptions matter: A customer might not have run
Windows 95 for years, and his IPv4-only game system might be gathering dust
in the closet, but tell him that he cannot use them and he might switch
providers.
Therefore IPv4 and IPv6 must coexist for some number of years, and their
coexistence must be transparent to end users. In fact if an IPv4-to-IPv6
transition is successful, the end users should not even notice it.
The tools available to networkers for IPv4/IPv6 coexistence fall into one of four
categories:
· Dual stack: A dual stack device is “bilingual;” that is, it can originate and
understand both IPv4 and IPv6 packets.
· Automatic tunnels: 6to4, ISATAP, DSTM, and tunnel brokers are the most
prominent examples of automatic tunnels; these technologies are best applied
when temporary tunneling is required. Unlike manual tunnels, automatic tunnels
identify their own endpoints and are set up and torn down as needed.
Dual stacking is by far the preferable solution in most scenarios. The dual
stacked device can speak equally to IPv4 devices, IPv6 devices, and other
dual-stacked devices (with the two devices agreeing on which IP version to
speak). And the entire transition can be driven by DNS: If a dual-stacked device
queries the name of a destination and DNS gives it an IPv4 address (a DNS A
Record), it sends IPv4 packets. If DNS responds with an IPv6 address (a DNS
AAAA Record), it sends IPv6 packets.
Dual stacking would have been the ideal transition strategy about ten years
ago, when there were still plenty of IPv4 addresses to go around. But we didn’t:
Ten years ago there were not that many people who believed we would be
running out of IPv4 addresses this soon, and few saw a compelling case for
going to the expense and trouble to make the transition. Most of us who were
advocating IPv6 at that time – myself included– were talking about killer
applications and exploding mobile markets and exploding Asian populations
and getting back to an end-to-end Internet model, not address depletion.
Ironically, even now the network operators who are the least inclined to start
planning for IPv6 are those who are still sitting on an abundant supply of IPv4
addresses. They’re in the best position to use dual stacking and avoiding the
eventual pain of transition, but they are not feeling the sense of urgency that
comes from analyzing their usagerates and seeing a big hard wall just around
the next curve.
This does not by a long shot mean that dual stacking is no longer an option for
IPv4/IPv6 coexistence. It just means that we must continue handling the IPv4
address challenges the way we’ve been handling them since the mid-1990s:
with Network Address Translation. Building dual stacked networks that mix
global IPv6 addresses and NAT-ed IPv4 addresses is feasible; it’s just, as I said
in the beginning, awkward.
There is no shortage of proposals for stretching what’s left of the IPv4 address
space into dual stacked networks: Carrier-Grade NAT (CGN), NAT444,
NAT464, Dual-Stack Lite, IVI, A+P… The challenge is deciding which of the
proposed solutions are the most practical for the most networks. None of them
have yet gone into widespread deployment, and the details of some are still
being worked out in the IETF.
Over the next few posts I’m going to be looking into these various “global IPv6
plus private IPv4” combinations for dual stack deployment.