Unified - Internet2 IP Network Operations Report
10/27/2014 - 11/02/2014

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Services

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





12281-Critical
1-Critical

Brief Outage - I2 Optical Various Circuits
UndeterminedVendor
10/28/2014 6:23 PM
10/28/2014 6:24 PM


12821-Critical


Brief Outage - I2 Optical DWDM System ALBA-BUFF
InvestigatingVendor
10/31/2014 12:34 PM
10/31/2014 12:35 PM






Unscheduled Outages Detail






Ticket No.:1228:144



Subject:Brief Outage - I2 Optical Various Circuits



Source Of Impact:Vendor



Affected:Optical Various Circuits
AL2S Backbone LOSA-PHOE
IP Backbone HOUS-LOSA
NDDI Backbone LOSA-WASH2



Start Time:Tuesday, October 28, 2014, 6:23 PM (1823) UTC



End Time:Tuesday, October 28, 2014, 6:24 PM (1824) UTC


Description:The above listed were briefly unavailable to the community.  Level 3 Engineers have
indicated that the reason for the outage is a fiber issue, the cause of which is
indeterminate.  Layer 2 and 3 backbone traffic re-routed during the outage.  Workgroup
administrators will receive notifications from OESS regarding circuits re-routed on the
backbone.







Ticket No.:1282:144



Subject:Brief Outage - I2 Optical DWDM System ALBA-BUFF



Source Of Impact:Vendor



Affected:Optical DWDM System ALBA-BUFF
AL2S Backbone ALBA-CLEV
IP Backbone CLEV-NEWY32AOA
Connector NOX



Start Time:Friday, October 31, 2014, 12:34 PM (1234) UTC



End Time:Friday, October 31, 2014, 12:35 PM (1235) UTC


Description:The above listed assets are available after a brief outage.  Level(3)
Engineers have been contacted and are investigating.  Layer2 and
Layer3 backbones re-routed during the event.  Workgroup administrators
receive notifications from OESS regarding circuits re-routed on the
backbone.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





1212
4-Normal
3-Elevated

Emergency Maintenance Completed - I2 AL2S Backbone BOST-HART2 
Hardware
Vendor10/28/2014 4:04 AM
10/28/2014 4:08 AM


1211
4-Normal
3-Elevated

Emergency Maintenance Completed- I2 AL2S Backbone ELPA-HOUH
HardwareVendor
10/28/2014 8:02 AM
10/28/2014 8:26 AM




Scheduled Maintenances Detail





Ticket No.:1212:144



Subject:Emergency Maintenance Completed - I2 AL2S Backbone BOST-HART2 



Source Of Impact:Vendor



Affected:AL2S Backbone BOST-HART2
IP Backbone CLEV-NEWY32AOA



Start Time:Tuesday, October 28, 2014, 4:04 AM (0404) UTC



End Time:Tuesday, October 28, 2014, 4:08 AM (0408) UTC


Description:   The above listed was briefly unavailable while Level 3 Engineers, working with Internet2
Engineers, performed hardware replacement maintenance.  A brief outage was 
experienced. This maintenance has been completed.



  







Ticket No.:1211:144



Subject:Emergency Maintenance Completed- I2 AL2S Backbone ELPA-HOUH



Source Of Impact:Vendor



Affected:AL2S Backbone ELPA-HOUH
IP Backbone HOUH-HOUS



Start Time:Tuesday, October 28, 2014, 8:02 AM (0802) UTC



End Time:Tuesday, October 28, 2014, 8:26 AM (0826) UTC


Description:   The above listed was unavailable while Level 3 Engineers, working with
Internet2
Engineers, performed hardware replacement maintenance. Maintenance has
been completed.  Layer 2 and 3 backbone traffic re-routed during this
maintenance.  Workgroup administrators will receive notifications from
OESS regarding circuits re-routed on the backbone.



  










Downtime and Availability







Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability


ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
      
Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Unified - Internet2 IP Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.82544%99.97268%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.75979%99.99924%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99753%100.00000%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96110%99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99469%99.99962%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177
               0 hr 5 min0 hr 1 min
               99.95040%99.99008%
               99.95219%99.99753%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97306%99.99507%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93871%99.99981%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189
               0 hr 25 min0 hr 1 min
               99.75198%99.99008%
               99.75675%99.76396%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.91272%99.98975%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99488%100.00000%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98368%99.99943%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98368%99.99943%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99374%99.99886%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99753%100.00000%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.82677%99.97306%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.60155%99.83796%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99621%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99621%100.00000%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96452%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.94346%99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.80609%99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93150%99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.93150%99.99924%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99488%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97875%99.99962%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99431%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99886%100.00000%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
      
Aggregate All Unified - Internet2 IP Backbone Circuits
                   0 hr 30 min0 hr 28 min
                   99.99196%99.99946%
                   99.94470%99.98239%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  

Upcoming Scheduled Maintenances





Ticket No.:1300:144



Subject:Maintenance 2 of 2 - I2 AL2S Backbones COLU4-KANS and CHIC-COLU4



Affected:AL2S Backbones COLU4-KANS and CHIC-COLU4



Start Time:Friday, November 7, 2014, 6:00 AM (0600) UTC



End Time:Friday, November 7, 2014, 12:00 PM (1200) UTC



Description:   The above listed backbones will become unavailable while MOREnet
engineers conduct fiber maintenance near Peculiar, MO.  Outages are
anticipated; The entire window has been reserved. Workgroup
administrators will receive notifications from OESS regarding circuits
re-routed on the backbone.



  







Ticket No.:1369:144



Subject:Maintenance - I2 IP Backbone ATLA-HOUS



Affected:Backbone ATLA-HOUS



Start Time:Monday, November 10, 2014, 6:00 AM (0600) UTC



End Time:Monday, November 10, 2014, 7:00 AM (0700) UTC



Description:   The above assets will be unavailable while Internet2 Engineers perform maintenance to 
Backbone ATLA-HOUS. An outage of 1 hour is expected; however, the entire window is 
reserved.  IP traffic will re-route during this maintenance.



  







Ticket No.:1240:144



Subject:Maintenance - I2 Optical DWDM System ATLA-CHAT



Affected:Optical DWDM System ATLA-CHAT



Start Time:Tuesday, November 11, 2014, 5:00 AM (0500) UTC



End Time:Tuesday, November 11, 2014, 11:00 AM (1100) UTC



Description:   The above listed will be unavailable while Level 3 Communications performs fiber 
relocation maintenance.  An outage is expected, and the entire window is reserved.  
Layer 2 and 3 backbone traffic will re-route during this maintenance.  Workgroup 
administrators will receive notifications from OESS regarding circuits re-routed on the 
backbone.



  







Ticket No.:1328:144



Subject:Maintenance - I2 Optical Member ESnet  



Affected:



Start Time:Tuesday, November 11, 2014, 7:00 PM (1900) UTC



End Time:Tuesday, November 11, 2014, 10:00 PM (2200) UTC



Description:   Member ESnet's above listed optical circuits should remain available  while ESnet 
engineers perform a maintenance in order to upgrade an operating system. No 
outages are expected; however the entire window has been reserved.



  







Ticket No.:1333:144



Subject:Maintenance - I2 Optical DWDM System ATLA-CHAR



Affected:IP Backbone ATLA-WASH
AL2S Backbone ATLA-CHAR
AL2S Backbone ATLA-JACK
Connector MCNC (ATLA)



Start Time:Wednesday, November 12, 2014, 4:00 AM (0400) UTC



End Time:Wednesday, November 12, 2014, 6:00 AM (0600) UTC



Description:   The above listed will remain available while Level 3 Communications performs
maintenance to verify route distribution in Atlanta, GA.  No outage is expected;
however, the entire window is reserved.



  







Ticket No.:1327:144



Subject:Maintenance *Tracking* - I2 Optical Circuits I2-FARG-SEAT-10GE-09204 I2-EQCH-MISS-OC192-09205 (Northern Tier)



Affected:Northern Tier Optical Circuits



Start Time:Wednesday, November 12, 2014, 7:00 AM (0700) UTC



End Time:Friday, November 14, 2014, 12:00 PM (1200) UTC



Description:   The above listed will remain available while Zayo Engineers perform power
maintenance.  No outage is expected; however, the entire window is reserved.



  







Ticket No.:1339:144



Subject:Maintenance - I2 Optical DWDM System BATO-HOUH



Affected:DWDM System BATO-HOUH
AL2S Backbones ATLA-HOUH, ATLA-HOUS, and BATO-HOUH
IP Backbone ATLA-HOUS
Customer Circuits for LONI



Start Time:Friday, November 21, 2014, 6:00 AM (0600) UTC



End Time:Friday, November 21, 2014, 12:00 PM (1200) UTC



Description:   DWDM System BATO-HOUH will be unavailable while Level(3) Communications relocates backbone 
cable due to service road project in Crowley, LA. An outage of one (1) hour is expected; but should 
prepare for an impact up to three (3) hours during the maintenance window; however the entire 
window is reserved. Layer 2 and 3 backbone traffic will re-route during this maintenance. 
Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the
backbone.



  







Ticket No.:1365:144



Subject:Maintenance *Tracking* - I2 IP Circuit BBDB4927 for oob.newy



Affected:BBDB4927-Level(3) High Speed IP Ethernet for OOB router in New York



Start Time:Tuesday, December 2, 2014, 5:00 AM (0500) UTC



End Time:Tuesday, December 2, 2014, 11:00 AM (1100) UTC



Description:   The above listed will become unavailable while Level(3) communications 
performs maintenance to migrate circuits to a new path. This maintenance 
is not impacting to customer circuits.