Internet2 IP Network Operations Report
12/16/2013 - 12/22/2013

Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

220362-High 2-High Brief Outage Resolved - I2 IP Backbone CLEV-WASH Circuit - Bumped Fiber 12/16/2013 6:54 PM 12/16/2013 6:55 PM
220382-High 2-High Outage Resolved - I2 IP Backbone CHIC-CLEV Hardware 12/17/2013 8:00 AM 12/17/2013 6:53 PM
220682-High 2-High Brief Outage - I2 IP Backbone KANS-SALT Undetermined 12/20/2013 4:28 PM 12/20/2013 4:29 PM

Unscheduled Outages Detail


Ticket No.:22036:45
Subject:Brief Outage Resolved - I2 IP Backbone CLEV-WASH
Affected:Backbone CLEV-WASH
ION Backbone CLEV-WASH
Start Time:Monday, December 16, 2013, 6:54 PM (1854) UTC
End Time:Monday, December 16, 2013, 6:55 PM (1855) UTC
Description:Backbone CLEV-WASH was briefly unavailable. Internet2's fiber provider has reported that the RFO is a fiber bump. IP traffic briefly re-routed.

Ticket No.:22038:45
Subject:Outage Resolved - I2 IP Backbone CHIC-CLEV
Affected:Backbones CHIC-CLEV
NetPlus Peer Code 42
Participant UIUC
Start Time:Tuesday, December 17, 2013, 8:00 AM (0800) UTC
End Time:Tuesday, December 17, 2013, 6:53 PM (1853) UTC
Description:Backbone CHIC-CLEV is now available. The reason for the outage was do to hardware failure during troubleshooting on AL2S Core Node STAR. The failed card has been replaced and no further outages are to be expected. IP traffic re-routed during the outage.

Ticket No.:22068:45
Subject:Brief Outage - I2 IP Backbone KANS-SALT
Affected:Backbone KANS-SALT
Start Time:Friday, December 20, 2013, 4:28 PM (1628) UTC
End Time:Friday, December 20, 2013, 4:29 PM (1629) UTC
Description:Backbone KANS-SALT was briefly unavailable. Internet2 Engineers are investigating the cause; however, the reason for the outage is pending. IP traffic rerouted during the event.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

22037 4-Normal 3-Elevated Emergency Maintenance Completed - I2 IP Backbones CHIC-CLEV Hardware 12/17/2013 6:49 AM 12/17/2013 7:59 AM

Scheduled Maintenances Detail

Ticket No.:22037:45
Subject:Emergency Maintenance Completed - I2 IP Backbones CHIC-CLEV
Affected:Backbones CHIC-CLEV
Start Time:Tuesday, December 17, 2013, 6:49 AM (0649) UTC
End Time:Tuesday, December 17, 2013, 7:59 AM (0759) UTC
Description: Backbones CHIC-CLEV was unavailable while Internet2 Engineers performed maintenance. Maintenance has been completed. Traffic re-routed during this maintenance.


Downtime and Availability



Internet2 IP Core Nodes Down TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98501% 99.99981%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98216% 99.99905%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99848% 99.99924%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96983% 100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886% 100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97059% 100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98729% 99.99867%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97875% 100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96604% 100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95086% 99.99962%

Aggregate Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%99.97879%99.99964%

** Less than 52 weeks of data available

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99488%99.99943%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99545%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94877%99.99848%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 12 hr 2 min10 hr 52 min 92.83730%93.53175% 99.83151%99.87591%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98444%100.00000%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99355%99.99943%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99241%100.00000%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%100.00000%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.76624%99.81785%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97476%99.99924%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95541%99.95636%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%99.99924%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75353%99.81538%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05638 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75220%99.81538%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.90760%99.96983%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05562 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.90646%99.97002%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05133 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.76947%99.86092%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.77080%99.86092%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-6366 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89109%99.89754%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08892 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89109%99.89754%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95712%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94801%100.00000%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08893 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91177%99.95731%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91803%99.95731%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.94441%99.99715%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08890 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.93852%99.99715%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.90077%99.90570%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05561 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.90077%99.90570%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.44084%99.59396%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05581 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.44046%99.59358%
Legacy 10G Backbone: I2-KANS-CHIC-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05138 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.79546%99.90930%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.79565%99.90949%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.84403%99.90930%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75694%99.80666%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05571 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.76017%99.80666%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.73778%99.82297%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.73778%99.82620%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.84422%99.90228%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07613 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.77174%99.82980%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96604%99.98596%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05573 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97951%99.99943%

Aggregate All Internet2 IP Backbone Circuits 12 hr 7 min1 hr 10 min 99.84974%99.86421% 99.88263%99.92394%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

* Internet2 IP 100G Backbone Circuits Ride over AL2S for connectivity.



Upcoming Scheduled Maintenances

Ticket No.:22075:45
Subject:Maintenance - I2 IP Core Node (CLEV)
Affected:Core Node (CLEV) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:01 AM (0501) UTC
End Time:Friday, December 27, 2013, 7:00 AM (0700) UTC
Description: Core Node (CLEV) will be unavailable while Internet2 Engineers perform maintenance to swap the Routing engines. A brief outage is expected; however, the entire window is reserved. All connections to the Core Node will be affected.

Ticket No.:22076:45
Subject:Maintenance - I2 IP Core Node (NEWY32AoA)
Affected:Core Node (NEWY32AoA) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:01 AM (0501) UTC
End Time:Friday, December 27, 2013, 7:00 AM (0700) UTC
Description: Core Node (NEWY32AoA) will be unavailable while Internet2 Engineers perform maintenance to swap the Routing engines. A brief outage is expected; however, the entire window is reserved. All connections to the Core Node will be affected.

Ticket No.:22077:45
Subject:Maintenance - I2 IP Core Node (WASH)
Affected:Core Node (WASH) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:01 AM (0501) UTC
End Time:Friday, December 27, 2013, 7:00 AM (0700) UTC
Description: Core Node (WASH) will be unavailable while Internet2 Engineers perform maintenance to swap the Routing engines. A brief outage is expected; however, the entire window is reserved. All connections to the Core Node will be affected.

Ticket No.:22078:45
Subject: Maintenance - I2 IP Core Node (ATLA)
Affected:Core Node (ATLA) and its Connectors and Participants
Start Time:Friday, December 27, 2013, 5:01 AM (0501) UTC
End Time:Friday, December 27, 2013, 7:00 AM (0700) UTC
Description: Core Node (ATLA) will be unavailable while Internet2 Engineers perform maintenance to swap the Routing engines. A brief outage is expected; however, the entire window is reserved. All connections to the Core Node will be affected.

Ticket No.:22047:45
Subject:Maintenance - I2 IP Backbones NEWY-WASH and ATLA-WASH
Affected:Backbones NEWY-WASH and ATLA-WASH
Connector OARnet (WASH)
Connector 3ROX (WASH)
Connector MAGPI (WASH)
Start Time:Tuesday, January 7, 2014, 4:00 AM (0400) UTC
End Time:Tuesday, January 7, 2014, 8:00 AM (0800) UTC
Description: The above listed will be unavailable while Level(3) Communications performs maintenance between Mclean, VA and Millersville, MD. Outages are expected throughout the maintenance window. IP traffic will re-route during this maintenance.

Ticket No.:22031:45
Subject:Maintenance 1 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Wednesday, January 8, 2014, 5:00 AM (0500) UTC
End Time:Wednesday, January 8, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the first of three maintenance windows.

Ticket No.:22032:45
Subject:Maintenance 2 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Thursday, January 9, 2014, 5:00 AM (0500) UTC
End Time:Thursday, January 9, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the second of three maintenance windows.

Ticket No.:21972:45
Subject:Maintenance - I2 IP Backbone HOUS-KANS
Affected:Backbone HOUS-KANS
Net+ Peer Dell (KANS)
Connector LEARN (KANS)*
Start Time:Friday, January 10, 2014, 6:01 AM (0601) UTC
End Time:Friday, January 10, 2014, 12:00 PM (1200) UTC
Description: Backbone HOUS-KANS will be unavailable while Level(3) Communications implement scheduled maintenance to splice backbone cable as mandated by the Oklahoma Department of Transportation due road widening project. An outage is expected, and the entire window is reserved. * indicates connector is multihomed and therefore should remain available. Backbone traffic will re-route during this maintenance.

Ticket No.:22033:45
Subject:Maintenance 3 of 3 - I2 IP Backbone ATLA-HOUS
Affected:Backbone ATLA-HOUS
Start Time:Saturday, January 11, 2014, 5:00 AM (0500) UTC
End Time:Saturday, January 11, 2014, 11:00 AM (1100) UTC
Description: Backbone ATLA-HOUS will be unavailable while vendor Engineers perform circuit maintenance in Jacksonville, Florida. Outages are expected, and the entire window is reserved. IP traffic will re-route during this maintenance. This is the third of three maintenance windows.