Unified - Internet2 IP Network Operations Report
11/03/2014 - 11/09/2014

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Source Of Impact Start Time (UTC) End Time (UTC)

13681-Critical 1-Critical Brief Outage Resolved - I2 Optical DWDM System CHIC-SAIN Human ErrorVendor 11/06/2014 6:23 AM 11/06/2014 6:24 AM
13912-High Brief Outage Resolved - I2 AL2S Core Node WASH HardwareVendor 11/07/2014 11:57 AM 11/07/2014 11:58 AM

Unscheduled Outages Detail


Ticket No.:1368:144
Subject:Brief Outage Resolved - I2 Optical DWDM System CHIC-SAIN
Source Of Impact:Vendor
Affected:DWDM System CHIC-SAIN
AL2S Backbones CHIC-COLU4 and CHIC-KANS
IP Backbone CHIC-KANS
NDDI Backbone CHIC-SEAT
Start Time:Thursday, November 6, 2014, 6:23 AM (0623) UTC
End Time:Thursday, November 6, 2014, 6:24 AM (0624) UTC
Description:DWDM System CHIC-SAIN was very briefly unavailable to the community. Level (3) Communications technicians advised that the source of the disturbance was a fiber bump due to maintenance outside of Broadview, IL.

Ticket No.:1391:144
Subject:Brief Outage Resolved - I2 AL2S Core Node WASH
Source Of Impact:Vendor
Affected:AL2S Core Node WASH and its participants
AL2S Backbone PHIL-WASH
AL2S Backbone ASHB-WASH
AL2S Backbone RALE-WASH
IP Backbone NEWY32AOA-WASH
IP Backbone CLEV-WASH
IP Backbone ATLA-WASH
IP Backbone CHIC-WASH
IP Backbone LOSA-WASH
Net+ Provider Amazon (WASH)
Net+ Provider Blackboard (WASH)
Start Time:Friday, November 7, 2014, 11:57 AM (1157) UTC
End Time:Friday, November 7, 2014, 11:58 AM (1158) UTC
Description:AL2S Core Node WASH experienced a brief outage. Internet2 Engineers have indicated that the outage was due to a hardware failure. The hardware has been replaced and no further outages are expected. Layer2 and Layer3 backbones re-routed during the event. Workgroup administrators receive notifications from OESS regarding circuits re-routed on the backbone.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Source Of Impact Start Time (UTC) End Time (UTC)

1292 3-Elevated Maintenance Completed - I2 AL2S Backbone CHIC-KANS Circuit Internal11/04/2014 6:03 AM 11/04/2014 6:06 AM

Scheduled Maintenances Detail

Ticket No.:1292:144
Subject:Maintenance Completed - I2 AL2S Backbone CHIC-KANS
Source Of Impact:Internal
Affected:AL2S Backbone CHIC-KANS
IP Backbone CHIC-KANS
Start Time:Tuesday, November 4, 2014, 6:03 AM (0603) UTC
End Time:Tuesday, November 4, 2014, 6:06 AM (0606) UTC
Description: The above listed circuits were unavailable while Level (3) engineers with the help of Internet 2 engineers performed a maintenance to clean fiber in Kansas. Maintenance has been completed.


Downtime and Availability



Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%

Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
I2-ATLA-CLEV-VLAN-14832 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-CLEV-VLAN-14837 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.82544%99.97268%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75979%99.99924%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.99734%99.99981%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96110%99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 0 hr 3 min0 hr 0 min 99.97024%100.00000% 99.99412%99.99962%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95219%99.99753%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.97287%99.99488%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93871%99.99981%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75675%99.76396%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91272%99.98975%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99488%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98368%99.99943%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98368%99.99943%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.99355%99.99867%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.99734%99.99981%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.82677%99.97306%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.60155%99.83796%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99621%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99621%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96452%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.99981%99.99981%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94346%99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.80609%99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93150%99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93150%99.99924%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99488%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97875%99.99962%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99431%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886%100.00000%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%

Aggregate All Unified - Internet2 IP Backbone Circuits 0 hr 8 min0 hr 3 min 99.99815%99.99885% 99.95238%99.98483%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.



Upcoming Scheduled Maintenances

Ticket No.:1422:144
Subject:Maintenance - I2 Optical DWDM System PHOE-SAND
Affected:DWDM System PHOE-SAND
AL2S Backbone LOSA-PHOE
IP Backbone HOUS-LOSA
IP Backbone HOUS-KANS
IP Connector Sun Corridor (LOSA)*
IP Connector CENIC (HOUS)*
Start Time:Saturday, November 15, 2014, 7:00 AM (0700) UTC
End Time:Saturday, November 15, 2014, 1:00 PM (1300) UTC
Description: DWDM System PHOE-SAND will be unavailable while Level(3) Communications relocates cable in Winterhaven, CA. An outage of 6 hours is expected; however the entire window is reserved. * Indicates entity is multihomed and therefore should remain reachable.

Ticket No.:1327:144
Subject:Maintenance *Tracking* - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 I2-EQCH-MISS-OC192-09205 (Northern Tier)
Affected:Northern Tier Optical Circuits
Start Time:Thursday, November 20, 2014, 7:00 AM (0700) UTC
End Time:Friday, November 21, 2014, 12:00 PM (1200) UTC
Description: The above listed will remain available while Zayo Engineers perform power maintenance. No outage is expected; however, the entire window is reserved.

Ticket No.:1474:144
Subject:Maintenance - I2 Optical Member ESnet (CHIC)
Affected:Optical Member ESnet (CHIC)
Start Time:Thursday, November 20, 2014, 11:00 PM (2300) UTC
End Time:Friday, November 21, 2014, 2:00 AM (0200) UTC
Description: Member ESnet's above listed Optical circuits listed above will be unavailable while ESnet Engineers perform software upgrade maintenance. A brief outage is expected, however the entire maintenance window has been reserved.

Ticket No.:1475:144
Subject:Maintenance - I2 Optical Member ESnet (BOST)
Affected:Optical Member ESnet (BOST)
Start Time:Thursday, November 20, 2014, 11:00 PM (2300) UTC
End Time:Friday, November 21, 2014, 2:00 AM (0200) UTC
Description: Member ESnet's above listed Optical circuits listed above will be unavailable while ESnet Engineers perform software upgrade maintenance. A brief outage is expected, however the entire maintenance window has been reserved.

Ticket No.:1339:144
Subject:Maintenance - I2 Optical DWDM System BATO-HOUH
Affected:DWDM System BATO-HOUH
AL2S Backbones ATLA-HOUH, ATLA-HOUS, and BATO-HOUH
IP Backbone ATLA-HOUS
Customer Circuits for LONI
Start Time:Friday, November 21, 2014, 6:00 AM (0600) UTC
End Time:Friday, November 21, 2014, 12:00 PM (1200) UTC
Description: DWDM System BATO-HOUH will be unavailable while Level(3) Communications relocates backbone cable due to service road project in Crowley, LA. An outage of one (1) hour is expected; but should prepare for an impact up to three (3) hours during the maintenance window; however the entire window is reserved. Layer 2 and 3 backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:1365:144
Subject:Maintenance *Tracking* - I2 IP Circuit BBDB4927 for oob.newy
Affected:BBDB4927-Level(3) High Speed IP Ethernet for OOB router in New York
Start Time:Tuesday, December 2, 2014, 5:00 AM (0500) UTC
End Time:Tuesday, December 2, 2014, 11:00 AM (1100) UTC
Description: The above listed will become unavailable while Level(3) communications performs maintenance to migrate circuits to a new path. This maintenance is not impacting to customer circuits.

Ticket No.:1409:144
Subject:Maintenance - I2 Optical DWDM System HOUS-JCSN
Affected:DWDM System HOUS-JCSN
AL2S Backbone HOUH-JCSN
NDDI Backbone LOSA-RALE
NDDI Backbone LOSA-WASH2
IP Connector MissION (HOUS) and its Participants*
Start Time:Friday, December 5, 2014, 6:00 AM (0600) UTC
End Time:Friday, December 5, 2014, 12:00 PM (1200) UTC
Description: DWDM System HOUS-JCSN will be unavailable while Level(3) Communications conduct maintenance to repair damaged cable due to construction equipment in Huffman, TX. An outage of 4 hours is expected; however the entire window is reserved.