Internet2 IP Network Operations Report
08/12/2013 - 08/18/2013

Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

213452-High 2-High Internet2 IP Network Backbone Circuit KANS-SALT Outage Resolved Circuit - Damaged Fiber 08/14/2013 1:08 PM 08/14/2013 9:04 PM
213512-High 2-High Internet2 IP Network Backbone Circuit HOUS-LOSA Outage Resolved Circuit - Damaged Fiber 08/15/2013 1:52 PM 08/16/2013 6:50 AM
213541-Critical 1-Critical Internet2 IP Network Core Node NEWY Brief Outage Resolved Hardware 08/16/2013 6:39 PM 08/16/2013 6:41 PM

Unscheduled Outages Detail


Ticket No.:21345:45
Subject:Internet2 IP Network Backbone Circuit KANS-SALT Outage Resolved
Affected:Backbone Circuit KANS-SALT
Connector CENIC (KANS)
*Participant FRGP IPv4 and IPv6
Start Time:Wednesday, August 14, 2013, 1:08 PM (1308) UTC
End Time:Wednesday, August 14, 2013, 9:04 PM (2104) UTC
Description:The above listed is now stable. Level (3) Communications has indicated that the instability was caused by contractor work 4 miles from the Denver POP. The Field Crew has determined the fiber was too badly damaged to straighten and moved forward with a plan to cut the fiber completely, then re-splice each buffer tube individually. Splicing has been completed and no further outages are expected. *Participant FRGP remains unavailable and is now being tracked in ticket 21348

Ticket No.:21351:45
Subject:Internet2 IP Network Backbone Circuit HOUS-LOSA Outage Resolved
Affected:Backbone Circuit HOUS-LOSA
Peer CUDI (HOUS)
Participant University of Arizona (HOUS)
Start Time:Thursday, August 15, 2013, 1:52 PM (1352) UTC
End Time:Friday, August 16, 2013, 6:50 AM (0650) UTC
Description:The above listed is now available to the community. Level(3) Communications technicians reported a fiber cut between Comstock, TX and Del Rio, TX. Crews arrived onsite determined the damage was due to drilling being done at the site by workers who were installing a billboard. The splice crew has completed repairs and no further outages are expected.

Ticket No.:21354:45
Subject:Internet2 IP Network Core Node NEWY Brief Outage Resolved
Affected:Core Node NEWY and its Connector(s) and Participants.
Start Time:Friday, August 16, 2013, 6:39 PM (1839) UTC
End Time:Friday, August 16, 2013, 6:41 PM (1841) UTC
Description:Core Node NEWY was briefly unavailable. Internet2 Vendor Engineers have replaced a failed Routing Engine which had been responsible for the earlier outage. No further outages are anticipated.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

None during this report period

Downtime and Availability



Internet2 IP Core Nodes Down TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.98501% 99.99981
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.98197% 99.99905
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99924% 100.00000
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.97040% 100.00000
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99886% 100.00000
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.97382% 100.00000
NEWY MX960 0 hr 2 min0 hr 2 min 99.98016%99.98016 99.98729% 99.99867
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.97875% 100.00000
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.96604% 100.00000
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.95067% 99.99962

Aggregate Internet2 IP Core Nodes0 hr 4 min0 hr 2 min99.99802%99.99802%99.97921%99.99971%

** Less than 52 weeks of data available

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.60439%99.67687%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05638 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.65790%99.73038%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.80020%99.86548%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05562 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.80020%99.86548%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05133 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75543%99.85694%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75543%99.85694%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-6366 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89754%99.89754%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08892 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89754%99.89754%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95712%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93056%100.00000%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08893 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94630%99.95731%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94630%99.95731%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95029%99.99734%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08890 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95029%99.99734%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89659%99.90570%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05561 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.69547%99.70458%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 16 hr 58 min16 hr 58 min 89.90079%89.90079% 99.43913%99.56189%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05581 16 hr 58 min16 hr 58 min 89.90079%89.90079% 99.43894%99.56170%
Legacy 10G Backbone: I2-KANS-CHIC-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05138 7 hr 56 min7 hr 56 min 95.27778%95.27778% 99.81652%99.90949%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 7 hr 56 min7 hr 56 min 95.27778%95.27778% 99.81671%99.90968%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 7 hr 56 min7 hr 56 min 95.27778%95.27778% 99.88065%99.90949%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95294%100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05571 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95219%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75068%99.82240%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75429%99.82563%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91936%99.97742%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07613 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.84688%99.90494%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96623%99.98653%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05573 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97970%100.00000%

Aggregate All Internet2 IP Backbone Circuits 57 hr 44 min57 hr 44 min 99.28406%99.28406% 99.89491%99.92783%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

* Internet2 IP 100G Backbone Circuits Ride over AL2S for connectivity.



Upcoming Scheduled Maintenances

Ticket No.:21335:45
Subject:Internet2 IP Network Backbone Circuit ATLA-CHIC Maintenance
Affected:Backbone Circuit ATLA-CHIC
ION Backbone Circuit ATLA-CHIC
Connector SOX
Connector KyRON
Connector Indiana GigaPOP
Connector University of Memphis
Start Time:Tuesday, August 20, 2013, 5:00 AM (0500) UTC
End Time:Tuesday, August 20, 2013, 11:00 AM (1100) UTC
Description: Backbone circuit ATLA-CHIC will be unavailable while vendor Engineers perform scheduled maintenance to replace damaged fiber in Springfield, TN. An outage of 15 minutes is expected; however the entire window is reserved. Backbone traffic will re-route during the maintenance.

Ticket No.:21320:45
Subject:Internet2 IP Network Backbone Circuit LOSA-SEAT Maintenance Window 1 of 2
Affected:Backbone Circuit LOSA-SEAT
Connector Oregon Gigapop
Various Peers via IXP PacWave
Start Time:Wednesday, August 21, 2013, 7:00 AM (0700) UTC
End Time:Wednesday, August 21, 2013, 1:00 PM (1300) UTC
Description: Backbone Circuit LOSA-SEAT will be unavailable while Level(3) Engineers perform circuit migration maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. This is the first of two maintenance windows.

Ticket No.:21321:45
Subject:Internet2 IP Network Backbone Circuit LOSA-SEAT Maintenance Window 2 of 2
Affected:Backbone Circuit LOSA-SEAT
Connector Oregon Gigapop
Various Peers via IXP PacWave
Start Time:Thursday, August 22, 2013, 7:00 AM (0700) UTC
End Time:Thursday, August 22, 2013, 1:00 PM (1300) UTC
Description: Backbone Circuit LOSA-SEAT will be unavailable while Level(3) Engineers perform circuit migration maintenance. An outage of 2 hours is expected; however, the entire window is reserved. IP traffic will re-route during this maintenance. This is the second of two maintenance windows.