Internet2 IP Network Operations Report
09/19/2011 - 09/25/2011

Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

179162-High 2-High Internet2 IP Network Backbone Circuit ATLA-CHIC Brief Outage Resolved Circuit - Damaged Fiber 09/22/2011 2:00 PM 09/22/2011 2:02 PM
179313-Elevated 3-Elevated Internet2 IP Network Backbone Circuit CHIC-NEWY Brief Outage Unannounced Maintenance 09/25/2011 8:38 PM 09/25/2011 8:50 PM

Unscheduled Outages Detail


Ticket No.:17916:45
Subject:Internet2 IP Network Backbone Circuit ATLA-CHIC Brief Outage Resolved
Affected:Backbone Circuit ATLA-CHIC
Connector Indiana GigaPOP(ATLA)*
Connector KyRON
Start Time:Thursday, September 22, 2011, 2:00 PM (1400) UTC
End Time:Thursday, September 22, 2011, 2:02 PM (1402) UTC
Description:The above listed was briefly unavailable. Level(3) Communications investigated and determined the reason for outage was fiber damage outside of Birmingham, AL. Fiber splicing of the entire conduit will not be required. All services have been restored and no further outages are expected. (*denotes multihomed status and, therefore, remained reachable)

Ticket No.:17931:45
Subject:Internet2 IP Network Backbone Circuit CHIC-NEWY Brief Outage
Affected:Backbone Circuit CHIC-NEWY
Start Time:Sunday, September 25, 2011, 8:38 PM (2038) UTC
End Time:Sunday, September 25, 2011, 8:50 PM (2050) UTC
Description:Backbone circuit CHIC-NEWY was briefly unavailable. Level(3) Communications technicians report they were performing testing to troubleshoot a degraded light level. GRNOC Engineers are now investigating the cause of the degraded light levels.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

17711 3-Elevated 2-High Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Maintenance Completed Circuit 09/20/2011 11:35 AM 09/20/2011 12:06 PM
17892 3-Elevated 1-Critical Internet2 IP Network Various Core Node(s) Emergency Maintenance Completed Software 09/22/2011 6:02 AM 09/22/2011 6:34 AM
17887 3-Elevated 2-High Internet2 IP Network Backbone Circuit ATLA-WASH Maintenance Completed Hardware 09/23/2011 6:09 AM 09/23/2011 6:27 AM

Scheduled Maintenances Detail

Ticket No.:17711:45
Subject:Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Maintenance Completed
Affected:Backbone Circuit LOSA-SALT
Backbone Circuit LOSA-SEAT
IXP PacWave
Connector Oregon Gigapop (LOSA) and its Participants
Start Time:Tuesday, September 20, 2011, 11:35 AM (1135) UTC
End Time:Tuesday, September 20, 2011, 12:06 PM (1206) UTC
Description: The above listed were unavailable while Level(3) Communications performed maintenance to relocate fiber due to UPRR bridge project at Hollister and UPRR, Goleta, CA. This maintenance has been completed. Alternate night is reserved for 9/21/2011 for the same time, if required.

Ticket No.:17892:45
Subject:Internet2 IP Network Various Core Node(s) Emergency Maintenance Completed
Affected:Core Nodes ATLA, CHIC, SEAT, SALT, NEWY, LOSA, KANS, HOUS, WASH
Start Time:Thursday, September 22, 2011, 6:02 AM (0602) UTC
End Time:Thursday, September 22, 2011, 6:34 AM (0634) UTC
Description: Routing in the above listed was unstable as network engineers restarted iBGP on the routers across the backbone routers. No external or customer peerings were affected, and traffic should rerouted as the internal peering sessions restarted. Maintenance has been completed.

Ticket No.:17887:45
Subject:Internet2 IP Network Backbone Circuit ATLA-WASH Maintenance Completed
Affected:Backbone Circuit ATLA-WASH
Connector MCNC and its Participants (WASH)
Start Time:Friday, September 23, 2011, 6:09 AM (0609) UTC
End Time:Friday, September 23, 2011, 6:27 AM (0627) UTC
Description: Backbone Circuit ATLA-WASH and connector MCNC's connection to WASH were unavailable while Level(3) Communications replaced failing hardware in King George, VA. Maintenance has been completed.


Downtime and Availability



Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability
CHIC T16000 hr 32 min99.68254%99.89261%
NEWY T16000 hr 32 min99.68254%99.97021%
SEAT MX9600 hr 32 min99.68254%99.99336%
WASH T16000 hr 32 min99.68254%99.94403%
HOUS MX9600 hr 32 min99.68254%99.99279%
LOSA T16000 hr 32 min99.68254%99.96888%
ATLA T16000 hr 32 min99.68254%99.98008%
SALT T16000 hr 32 min99.68254%99.96793%
KANS T16000 hr 32 min99.68254%99.95351%

Aggregate Internet2 IP Core Nodes4 hr 48 min99.68254%99.96260%

** Less than 52 weeks of data available

Internet2 IP Backbone CircuitsDown TimeReporting Period Availability52 Week Availability
I2-HOUS-LOSA-10GE-055590 hr 0 min100.00000%99.95959%
I2-ATLA-CHIC-10GE-054190 hr 2 min99.98016%99.87876%
** I2-HOUS-KANS-10GE-055600 hr 0 min100.00000%99.81500%
I2-CHIC-NEWY32AOA-10GE-052390 hr 0 min100.00000%99.84498%
** I2-CHIC-NEWY32AOA-10GE-056670 hr 12 min99.88095%99.84252%
I2-NEWY32AOA-WASH-10GE-052420 hr 0 min100.00000%99.91063%
I2-CHIC-WASH-10GE-052500 hr 0 min100.00000%99.91765%
** I2-HOUS-KANS-10GE-055610 hr 0 min100.00000%99.81557%
** I2-NEWY32AOA-WASH-10GE-046430 hr 0 min100.00000%99.91101%
I2-ATLA-WASH-10GE-052510 hr 18 min99.82143%99.85124%
I2-ATLA-WASH-10GE-051330 hr 18 min99.82143%99.85371%
I2-ATLA-HOUS-10GE-054230 hr 0 min100.00000%99.91120%
** I2-ATLA-HOUS-10GE-055620 hr 0 min100.00000%99.91120%
** I2-KANS-SALT-10GE-055660 hr 0 min100.00000%99.95845%
I2-KANS-SALT-10GE-051380 hr 0 min100.00000%99.95845%
** I2-SALT-SEAT-10GE-055650 hr 0 min100.00000%99.95712%
** I2-SALT-SEAT-10GE-055730 hr 0 min100.00000%99.95712%
** I2-LOSA-SALT-10GE-055710 hr 31 min99.69246%99.93815%
** I2-LOSA-SALT-10GE-055630 hr 31 min99.69246%99.93815%
I2-ATLA-CHIC-10GE-056380 hr 2 min99.98016%99.88009%
I2-CHIC-KANS-10GE-055670 hr 0 min100.00000%99.82278%
I2-CHIC-KANS-10GE-055680 hr 0 min100.00000%99.82032%
I2-CHIC-KANS-10GE-059760 hr 0 min100.00000%99.89223%
I2-CHIC-WASH-10GE-056370 hr 0 min100.00000%99.97439%
I2-LOSA-SEAT-10GE-055640 hr 31 min99.69246%99.98653%
I2-LOSA-SEAT-10GE-055720 hr 31 min99.69246%99.98653%
I2-HOUS-LOSA-10GE-055810 hr 0 min100.00000%99.95959%

Aggregate All Internet2 IP Backbone Circuits2 hr 56 min99.93533%99.90567%





Upcoming Scheduled Maintenances

Ticket No.:17913:45
Subject:Internet2 IP Network Backbone Circuit CHIC-WASH Maintenance
Affected:Backbone Circuit CHIC-WASH
Start Time:Thursday, September 29, 2011, 4:00 AM (0400) UTC
End Time:Thursday, September 29, 2011, 10:00 AM (1000) UTC
Description: Backbone Circuit CHIC-WASH will be unavailable while Level(3) Engineers perform fiber maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. An alternate window has been reserved for 09/30/2011 from 0400 to 1000 UTC, should it be required.

Ticket No.:17914:45
Subject:Internet2 IP Network Backbone Circuit CHIC-WASH Maintenance Alternate Window
Affected:Backbone Circuit CHIC-WASH
Start Time:Friday, September 30, 2011, 4:00 AM (0400) UTC
End Time:Friday, September 30, 2011, 10:00 AM (1000) UTC
Description: Backbone Circuit CHIC-WASH will be unavailable while Level(3) Engineers perform fiber maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. This is an alternate window for ticket 17913:45, should it be required.