Documentation

Enabling IPv6 with Internet2

Scope: Note that this isn't about IPv6 in general, but aimed narrowly at what a participant site has to do on a Cisco or Juniper router to connect to Internet2 using IPv6. For more information about IPv6 in general, IPv6 protocols mentioned here, and for further help, see the "For Further Information" section at the end of this document.

There are two parts to a "native IPv6" peering arrangement with Internet2: Addressing and BGP peering. This short document describes how to enable each and a few things to look for to determine whether they're working at all. It doesn't discuss debugging or troubleshooting strategies.

General Comments for these Examples

The configurations below represent what a peer might minimally do to enable IPv6 peering with Internet2. For these examples, we'll assume a neighbor with address 2001:468:ff:1b04::2 in AS 555, and that Internet2 has assigned the peer the 2001:468:0400::/40 network prefix.

		Neighbor		Internet2 router
Addresses	2001:468:ff:1b04::2/64	2001:468:ff:1b04::1/64
AS number	555			11537
Prefix		2001:468:0400::/40	N/A

---

Cisco Global commands

• There is one global router command just to enable IPv6 routing:

  ipv6 unicast-routing

Cisco Addressing

• Your IPv6 neighbor address will likely be assigned to you by the Internet2 NOC. You will need to add this IPv6 address to the interface you already peer with Internet2 on. Perform the following command from interface or sub-interface configuration mode:

   ipv6 address 2001:468:ff:1b04::2/64

• To verify that IPv6 is enabled on an interface issue the “show ipv6interface” command. The output for an IPv6 enabled interface should looksomething like this:

Router>sh ipv6 interface
POS2/0 is up, line protocol is up
 IPv6 is enabled, link-local address is FE80::210:1FFF:FE44:E3FF
 Description: point-to-point connection to a v6 site 
 Global unicast address(es): 
   2001:468:FF:1b04::2, subnet is 2001:468:FF:1b04::/64
 Joined group address(es): 
   FF02::1
   FF02::1:FF44:E3FF
   FF02::1:FF00:2
   FF02::2
 MTU is 9180 bytes
 ICMP error messages limited to one every 500 milliseconds
 ICMP redirects are enabled
 ND DAD is enabled, number of DAD attempts: 1
 ND reachable time is 30000 milliseconds
 Hosts use stateless autoconfig for addresses.

---

Cisco BGP

Enable IPv6 BGP:

• Within 'router BGP' context, configure a new neighbor in your “address-family ipv6 unicast” context.

  Address-family ipv6 unicast
   Neighbor 2001:468:ff:1b04::1 remote-as 11537
   Neighbor 2001:468:ff:1b04::1 activate
   Neighbor 2001:468:ff:1b04::1 description Internet2
   Neighbor 2001:468:ff:1b04::1 prefix-list To-Internet2 out
   Network 2001:468:0400::/40

• The router will move several of your commands to other areas of the BGP configuration. The above commands will result in the following showing up in your BGP router configs:

Router bgp 555
   Neighbor 2001:468:ff:1b04::1 remote-as 11537
   No Neighbor 2001:468:ff:1b04::1 activate
Address-family ipv6
   Neighbor 2001:468:ff:1b04::1 activate
   Neighbor 2001:468:ff:1b04::1 description Internet2
   Neighbor 2001:468:ff:1b04::1 prefix-list To-Internet2 out
   Network 2001:468:0400::/40

Note in particular how the "remote-as" command was moved to a separate portion of the config. Note also the "no neighbor 2001:468:ff:1b04::1 activate”command that was inserted automatically by the router. This keeps the router from attempting to bring up an IPv4 unicast BGP session to the 2001:468:ff:1b04::1 peer. This is, after all, IPv6 BGP session!

• BGP will not advertise a route if the route is not in its routing table. The 2001:468:0400::/40 is an aggregate route, and while several /64 subnets may be in the routing table, the aggregate route of 2001:468:0400::/40 will not be so unless some additional action is taken the aggregated prefix will not be advertised via BGP. There are several ways to correct this. We will get the aggregate route into the routing table by creating a static route to the NULL0 pseudo-interface. In global config context, perform the following command:

	ipv6 route 2001:468:0400::/40 null0

• We wish to ensure that BGP only advertises to Internet2 the aggregate route and not the more-specifics. This is done by applying an outbound prefix-list filter to the BGP session. We have performed half of this step already by specifying the prefix-list “To-Internet2" in the BGP configuration above. We must now create the “To-Internet2" prefix-list. Issue the following command in Global configuration:

	ipv6 prefix-list To-Internet2 permit 2001:468:0400::/40

• To verify the status of the IPv6 BGP session you can issue the“show bgp summary” command. You should see something like this:

Router>sh bgp summ
BGP router identifier 134.68.253.150, local AS number 555
BGP table version is 966002, main routing table version 966002
247 network entries and 247 paths using 49647 bytes of memory
199 BGP path attribute entries using 11940 bytes of memory
195 BGP AS-PATH entries using 6492 bytes of memory
6 BGP community entries using 144 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory

This indicates that BGP has been established to the 2001:468:ff:1b04::1 peer and that this router is receiving 246 routes from that peer.

• We can ensure that we are passing a route to the BGP peer by looking at the routes this BGP session is sending to the 2001:468:ff:1b04::1 peer.

Router>sh bgp ipv6 neighbor 2001:468:FF:1b04::1 advertised

BGP table version is 966017, local router ID is 134.68.253.150
Status codes:s suppressed,d damped,h history,* valid,> best,i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete
   Network                Next Hop            Metric LocPrf Weight Path
*> 2001:468:0400::/40  2001:468:FF:1b04::2                  0    i

• That’s it. All Done.

---

Juniper Addressing

• We will assume that the interface that peers with Internet2 is ge-2/0/0. Also, don’t forget to save and engage your changes by doing “commit”. From the interface ge-2/0/0 unit 0 edit mode, issue the following command:

	set family inet6 address 2001:468:ff:1b04::2/64

• To verify that IPv6 is configured for the interface, issue the“show interface terse” command. The output should be something like this:

User> show interfaces at-1/2/0 terse
Interface  Admin Link Proto Local		 Remote
at-1/2/0   up    up
at-1/2/0.0 up    up   inet  192.12.206.250/30
                      inet6 2001:468:ff:1b04::2/64
                            fe80::2a0:a5ff:fe3d:76c/64 

---

Juniper BGP

• A new BGP session must be established using IPv6 addresses. The easiest way to do this is to create a new group. From the Protocols BGP edit mode issue set commands to create a group that looks like this:

   group Peers-V6 {
	type external;
	export To-Internet2; 
	family inet6 {
		any;
	} 
	peer-as 11537; 
	neighbor 2001:468:FF:1b04::1 {  
		description "Internet2 v6 BGP Session";
	}
   } 

• The “To-Internet2 policy, referred to in the export command above, will need to be created. It should perform two functions. First, it should ensure that only the aggregate prefix 2001:468:0400::/40 is advertised to Internet2, and not the more specific /64s. In addition, the policy will set the Next-Hop for the route to Self. The following policy will do this:

   policy-statement To-Internet2 {
	term allow {
		from { 
			route-filter 2001:468:0400::0/40 exact; 
		then { 
			next-hop self; 
			accept; 
		}
	}
	term deny {
		then reject; 
	} 

• We will need to create an aggregate route, 2001:468:0400::/40, in order for BGP to advertise this route and not the more specific /64s. This can be done in the “routing-options” edit mode:

   rib inet6.0 {
        static {
            rib-group IF6-RG6;
            route 2001:468:0400::0/40 {
                reject;
                install;
                readvertise;
            }
        }
   }

• Note that this static route was created in the rib-group IF6-RG6.This should be replaced with the name of your inet6.0 rib-group. If you do not have an inet6.0 rib-group then you will need to create one BEFOREyou create the static route. You can do this by issuing the following command in “routing-options” edit mode:

   rib-groups {
      IF6-RG6 {
            import-rib inet6.0;
         }
      }
    }

• To verify that BGP has been established issue the “show bgp summary” command. The output should be something like this:

User> show bgp summary
Groups: 8 Peers: 10 Down peers: 0 

Table	Tot Paths	Act Paths	Suppressed	History	Damp State	Pending
inet.0	6806	6789	0	0	0	0
inet.2	3625	3621	0	0	0	0
inet6.0	247	247	0	0	0	0
inet6.2	0	0	0	0	0	0

Peer	AS	InPkt	OutPkt	OutQ	Flaps	Last Up/Down	State|#Active/Received/Damped...
2001:468:ff:1b04::1	11537	63536	42563	0	0	2w0d18h	inet6.0: 246/246/0

• You can verify that you are advertising the correct aggregate route to Internet2 by issuing the “show route advertising-protocol bgp 2001:468:ff1b04::1” command. The output should be something like this:

User> show route advertising-protocol bgp 2001:468:ff:644::1 
inet6.0: 258 destinations, 258 routes (258 active,0 holddown,0 hidden) 
Prefix             Nexthop                MED    Lclpref AS path
2001:468:400::/40  Self                     0            I

---

For Further Information:

1. Cisco web pages: including recommended releases, tutorials, sample configurations, and much more. http://www.cisco.com/en/US/products/ps6553/products_ios_technology_home.html
2. Juniper web pages: including recommended releases, tutorials, sample configurations, and much more. http://www.juniper.net/techpubs/software/junos/junos53/swconfig53-ipv6/frameset.htm
3. Internet2 IPv6 working group: http://www.internet2.edu/ipv6
4. NCNE web pages: http://www.ncne.org/documentation/doc_repository.html including FAQ and many tutorials from users and vendors.
5. NCNE user help: NCNE, the National Center for Network Engineering at Pittsburg Supercomputer Center, has agreed to help sites bringing up or having troubles with IPv6. Write to ncne@ncne.org and ask for help specifically with IPv6. Indicate that you're an Internet2 site.

Enabling IP Multicast with Internet2

Scope: Note that this isn't about multicast in general, but aimed narrowly at what a participant site has to do on a Cisco or Juniper router to connect to Internet2. It assumes basic familiarity with multicast concepts. For more information about multicast in general, multicast protocols mentioned here, and for further help or information, including some excellent troubleshooting and debugging documents, see the "For Further Information" section at the end of this document.

There are three parts to a "native multicast" peering arrangement with Internet2: MBGP, PIM-Sparse Mode, and MSDP. This short document describes how to enable each and a few things to look for to determine whether they're working at all. It doesn't discuss debugging or troubleshooting strategies, which is another subject not widely understood and very much needed.  Configuration examples are available here for Cisco (MBGP, PIM, and MSDP) or Juniper (MBGP, PIM, and MSDP).

---

Cisco configuration:

Global commands

There is one global router command just to enable multicast:

  ip multicast-routing

The configurations below represent what a peer might minimally do to enable multicast peering with Internet2. For these examples, we'll assume a neighbor with address 5.5.5.5 in AS 555:

                 Neighbor      Abilene router
IP addresses    5.5.5.5/30      5.5.5.6/30
AS number       555             11537

---

MBGP for Cisco

Enable MBGP:

• Within 'router BGP' context, change your network and neighbor statements  from implicit unicast-only to unicast and multicast. Include the nlri unicast multicast phrase for each network and neighbor with whom you want both types of BGP negotiation:
  
  • pre-12.1:

    change from:
  
    network 5.5.5.0
    neighbor 5.5.5.6 remote-as 11537
  
    to:
  
    network 5.5.5.0 nlri unicast multicast
    neighbor 5.5.5.6 remote-as 11537 nlri unicast multicast

  • 12.1 and later:

    change from:
  
    network 5.5.5.0
    neighbor 5.5.5.6 remote-as 11537
  
    to:
  
    neighbor 5.5.5.6 remote-as 11537
    address-family ipv4 unicast
      neighbor 5.5.5.6 activate
    neighbor 5.5.5.6 remote-as 11537
    address-family ipv4 multicast
      neighbor 5.5.5.6 activate

• To verify that MBGP is talking for both unicast and multicast, see output from "show ip mbgpsum" (look for state/prefixes) and/or "show ip bgp neighbor 5.5.5.5", looking for lines like these (this is for 12.0, but 12.1 will have similar indications):

&; Neighbor NLRI negotiation:
    Configured for unicast and multicast routes
    Peer negotiated unicast and multicast routes
    Exchanging unicast and multicast routes

and:

  Number of unicast/multicast prefixes received 3/3

---

PIM for Cisco

• Enable PIM on the point-to-point interface and add multicast- boundary administrative scoping:

interface ATMx/y.1 point-to-point
 description to I2/Abilene,AS11537,
     contact noc@abilene.iu.edu,317-278-6622

 ip address 5.5.5.5 255.255.255.252
 ip pim sparse-mode
 ip multicast boundary multicast-boundary

• This is the multicast-boundary list we presently use in Internet2, and there have been some suggestions for additional values for which we're soliciting feedback/consensus from the I2 community. The expressions below block sending RP announce and discovery packets and set the accepted administrative scoping to block 'local' multicast.

ip access-list standard multicast-boundary
 deny   224.0.1.39
 deny   224.0.1.40
 deny   239.0.0.0 0.255.255.255
 permit any

• If PIM is configured on both sides of the connection, then each should see the other as a "PIM neighbor", e.g.:

abilene-gsr>sho ip pim interface

Address      Interface        Version/Mode    Nbr   Query     DR
                                              Count Intvl
5.5.5.6      POS0/0           v2/Sparse        1    30     0.0.0.0
1.1.1.1      ATM1/0.1         v2/Sparse       *0    30     0.0.0.0

First of all, if you have PIM enabled for an interface it will appear in this list, so that's a good first sanity check that you've turned it on at all. Note that the "

Mode

" column indicates

v2/Sparse

,which are both good. In the column "

Nbr count

", look for non-zero neighbor counts:

0

means that no neighbor negotiation has occured (there is no PIM neighbor there; PIM is not turned on at your neighbor's side of the connection), while

1

means PIM *is* on at your neighbor's side of the connection.

---

MSDP for Cisco

• Finally, enable MSDP to your peer's RP. USUALLY (and this is the Internet2 preference),the MSDP peer address is the other end of the point-to-point connection. This will be the address to be used for Internet2. Some sites prefer to use a loopback address for their MSDP peer address, in which case they must also indicate to MSDP to use that interface. (the address in the 'peer' and the 'sa-filter' statements are the same, and are the address of your MSDP peer, usually the same as the MBGP peer.)
  • (alternatively, if you're using e.g. a loopback address for
    your MSDP source:)

 ip msdp peer 5.5.5.6
 ip msdp sa-filter out 5.5.5.6 list 111

 ip msdp peer 5.5.5.6 connect-source Loopback0
 ip msdp sa-filter out 5.5.5.6 list 111

• Also, it's a good idea to have a MSDP filter applied to the connection,to keep the peer from sending improper source-active announcements. You may also implement SA-count limits to keep the neighbor from flooding you with advertisements.   This list is referred to by the "ip msdp sa-filter ... list 111" command above; the list of prefixes Internet2 blocks are listed elsewhere in this document, but this is the syntax:

 access-list 111 deny   ip any host 224.0.1.2
 access-list 111 deny   ip any host 224.0.1.3

           ... etc ...

           (add the rest of the list of prefixes below and remember to add these RFC1918 addresses:)

 access-list 111 deny   ip 10.0.0.0 0.255.255.255 any
 access-list 111 deny   ip 127.0.0.0 0.255.255.255 any
 access-list 111 deny   ip 172.16.0.0 0.15.255.255 any
 access-list 111 deny   ip 192.168.0.0 0.0.255.255 any
 access-list 111 permit ip any any

• To see if the MSDP peering is working bidirectionally, "show ip msdp sum" and look for a state of "up". If they're joined to multicasts through this path, you should see SAs (source advertisements) from their AS in "show ip msdp count".

---

Juniper router configuration:

Global considerations

No global router commands to enable multicast are necessary, however your router may need a tunnel PIC in order to do multicast (if it is acting as an RP or will have directly-connected multicast sources.) Some routers, for instance the 7i or J series, have the tunnel-PIC function built in or don't need them.

The configurations below represent what an Internet2 peer must minimally do to enable multicast peering with Internet2. For these examples, we'll assume a neighbor with address 5.5.5.5 in AS 555, and the Internet2 router's address for that connection is 5.5.5.6.  Abilene's AS is 11537.

                 Neighbor      Abilene router
IP addresses    5.5.5.5/30      5.5.5.6/30
AS number       555             11537

---

MBGP for Juniper

JunOS assumes both unicast and multicast for ipv4 BGP peering, so if a BGP neighbor is configured, the Juniper will try to negotiate both unicast and multicast NLRI and will settle on whatever the partner is willing to do.  So if you want to do both unicast and multicast peering with the Internet2 router, the configuration is simply:

protocols {
   bgp {
      group Abilene {
          neighbor 5.5.5.6 {
              family inet {
                   any;    (this does unicast and multicast;

                                     alternatively, you could specify only "multicast")
              }
              type external;
              description "Abilene";
              peer-as 11537;
          }

PIM for Juniper

• Enable PIM on the point-to-point interface toward Internet2 and specify the RP:

protocols {
   pim {
      rp {		(use this to specify your RP)
          static {	(if RP is elsewhere. "local" if it is this router)
             family inet {
                 address <address of your RP here>;
                 group-ranges {
                     224.0.0.0/4;
                 }
             }
          }
      interface so-0/0/0 {	

("all" also works if you want PIM everywhere.
This is the interface facing Abilene)
mode sparse;
version 2;
}

• If PIM is working on both sides of the connection, then each should see the other as a "PIM neighbor", e.g.:

juniper>sho pim interface
Name                   Stat Mode       IP V State Count DR address
so-0/0/0.0             Up   Sparse      4 2 P2P       1 

First of all, if you have PIM enabled for an interface it will appear in this list, so that's a good first sanity check that you've turned it on at all.

"Stat(us)"

shows "up", which is what you're looking for. Note that the "

V(ersion)"

and

"Mode

" columns indicate 2 and

Sparse

, which should both be configured. In the column "

Count

", look for non-zero neighbor counts:

0

means that no neighbor negotiation has occured (there is no PIM neighbor there; PIM is not turned on at your neighbor's side of the connection), while

1

means PIM *is* working from your neighbor's side of the connection to you.  A DR ("designated router") isn't needed for a point-to-point connection.

---

MSDP for Juniper

• Finally,enable MSDP to your peer's RP. USUALLY (and this is the Internet2 preference),the MSDP peer address is the other end of the point-to-point connection. This will be the address to be used for Internet2. Some sites prefer to use a loopback address for their MSDP peering address, in which case they must also indicate to MSDP to use that interface. (The address in the 'peer' and the 'sa-filter' statements are the same, and are the address of your MSDP peer, usually the same as the MBGP peer.)
• Also, it's a good idea to have a MSDP filter applied to the connection, to keep the peer from sending improper source-active announcements.  You may also implement SA-count limits to keep the neighbor from flooding you with advertisements.

 protocols {
    msdp {
      group Abilene {
         export MSDP-FILTER;
         import MSDP-FILTER;
         peer 5.5.5.6 {
 local-address 5.5.5.5;  
(you could use your loopback address here if you need to)

}
...
policy-options {
policy-statement MSDP-FILTER {
term bad-groups {
from {
route-filter 224.0.1.2/32 exact;
(etc... the content of MSDP-FILTER is listed separately in this document)
}
then reject;
}
term bad-sources {
from {
source-address-filter 10.0.0.0/8 orlonger;
source-address-filter 127.0.0.0/8 orlonger;
source-address-filter 172.16.0.0/12 orlonger;
source-address-filter 192.168.0.0/16 orlonger;
}
then reject;
}
term allow {
then accept;
}
}

• You can verify that MSDP peering is up with the "show msdp brief" command. Look for "State: Established".

juniper> show msdp brief
Peer address Local address State Last up/down Peer-Group SA Count
5.5.5.6 5.5.5.5 Established 3w1d18h Abilene 1038/2101

---

MSDP Filter contents

It's a good idea to limit the source-active advertisements your peer can send to you to legitimate multicast group addresses.  This can be done on both Cisco and Juniper platforms with filters applied to your MSDP peers, as shown above.  Internet2's current MSDP filter blocks the prefixes listed below.  Its contents have input from the Internet2 Multicast Working Group, the Internet Assigned Numbers Authority (IANA) official list of multicast networks, and recommendations in www.cisco.com/warp/customer/105/49.html and draft-nickless-ipv4-mcast-unusable-03 (December 2003).
This version of the list uses Juniper syntax; Cisco's is "access-list 111 deny ip any host 224.0.2.2" for /32s or "deny ip any 224.77.0.00.0.255.255" for networks, etc. for this same set of prefixes).

   route-filter 224.0.1.2/32 exact;	! SGI-DOGFIGHT
   route-filter 224.0.1.3/32 exact;     ! RWHOD
   route-filter 224.0.1.8/32 exact;     ! SUB-NIS
   route-filter 224.0.1.22/32 exact;    ! SRVLOC
   route-filter 224.0.1.24/32 exact;    ! MICROSOFT-DS--WINS locator service
   route-filter 224.0.1.25/32 exact;    ! NBC-PRO
   route-filter 224.0.1.35/32 exact;    ! SRVLOC-DA
   route-filter 224.0.1.39/32 exact;    ! AUTORP-ANNOUNCE
   route-filter 224.0.1.40/32 exact;    ! AUTORP-DISCOVERY
   route-filter 224.0.1.60/32 exact;    ! HP-DEVICE-DISC
   route-filter 224.0.2.1/32 exact;     ! HP-DEVICE-DISC
   route-filter 224.0.2.2/32 exact;     ! SUN-RPC
   route-filter 224.1.0.38/32 exact;   ! Retrospect
   route-filter 224.77.0.0/16 orlonger;	! NORTON GHOST
   route-filter 225.1.2.3/32 exact;     ! ALTIRIS
   route-filter 226.77.0.0/16 orlonger;     ! NORTON GHOST
   route-filter 229.55.150.208/32 exact;     ! NORTON GHOST
   route-filter 234.42.42.40/30 orlonger;     ! IMAGECAST
   route-filter 234.142.142.42/31 orlonger;     ! IMAGECAST
   route-filter 234.142.142.44/30 orlonger;     ! IMAGECAST
   route-filter 234.142.142.48/28 orlonger;     ! IMAGECAST
   route-filter 234.142.142.64/26 orlonger;     ! IMAGECAST
   route-filter 234.142.142.128/29 orlonger;    ! IMAGECAST
   route-filter 234.142.142.136/30 orlonger;    ! IMAGECAST
   route-filter 234.142.142.140/31 orlonger;    ! IMAGECAST
   route-filter 234.142.142.142/32 exact;     ! IMAGECAST
   route-filter 232.0.0.0/8 orlonger;   ! SSM range--should be no MSDP here
   route-filter 239.0.0.0/8 orlonger;   ! admin scoped

---

For Further Information:

1. Cisco web pages: ftp://ftpeng.cisco.com/ipmulticast/index.html including recommended releases, tutorials, sample configurations, and much more.
2. Internet2 Multicast working group: multicast.internet2.edu, including references, debugging tutorial, and contents of I2 hands-on multicast workshops.
3. NCNE web pages: www.ncne.org/documentation/faq/multicast.html including FAQs and many tutorials from users and vendors.
4. NCNE multicast introduction at I2 member meeting Spring 2000: www.internet2.edu/presentations/200003228-I2MM-Goodwin.htm
5. NCNE user help: NCNE, the National Center for Network Engineering at Pittsburg Supercomputer Center, has agreed to provide technical support for sites bringing up or having troubles with multicast. Write to ncne@ncne.org and ask for help specifically with Multicast. Indicate that you're an Internet2 site.
6. "Best current practices for enabling Multicast networks": a presentation by Bill Nickless at the February 2003 Joint Techs workshop.
7. "Protecting multicast-enabled networks": a presentation by Matt Davy at the July 2004 Joint Techs workshop.
8. IPv4 Multicast Unusable Group and Source Addresses, version 3 of a draft by Bill Nickless (Dec 2003).
9. Multicast troubleshooting methodology: Bill Nickless presentation. This version is from 2003. (PDF no longer available)
10. Multicast troubleshooting: presentation given at Multicast workshop held in Vancouver, Canada, May 2004. (PDF no longer available)
11. Triumf AG Multicast references: several good, current (2004), multicast references.
12. Internet Assigned Numbers Authority (IANA) official list of multicast networks.
13. A book, Interdomain Multicast Routing: Practical Juniper Networks and Cisco Systems Solutions (2002), available here at Amazon.
14. A book, Developing IP Multicast Networks: The Definitive Guide to Designing and Deploying Cisco Multicast Networks (2000), available here at Amazon.