Internet2 Advanced Layer 2 Service Network Operations Report
07/04/2014 - 07/10/2014

Internet2 Advanced Layer 2 Service Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Network Resources

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





19432-High
2-High

Brief Outage Resolved - I2 AL2S Backbone ATLA-JCSN
Cut or Damaged FiberVendor
07/05/2014 8:48 PM
07/05/2014 8:49 PM


19442-High
2-High

Outage Resolved - I2 AL2S Backbones ATLA-CHIC and ATLA-JCSN
Cut or Damaged FiberVendor
07/06/2014 7:19 PM
07/07/2014 2:35 PM


19452-High
2-High

Outage Resolved - I2 AL2S Backbone DALL-HOUH
Cut or Damaged FiberVendor
07/07/2014 3:21 PM
07/07/2014 8:13 PM


19482-High
2-High

Brief Outage Resolved - I2 AL2S Backbone ATLA-CHIC
UndeterminedUndetermined
07/07/2014 6:27 PM
07/07/2014 6:28 PM


19502-High
2-High

Brief Outage Resolved - I2 AL2S Backbone ATLA-JCSN
HardwareVendor
07/09/2014 4:01 AM
07/09/2014 4:02 AM


19753-Elevated
3-Elevated

Brief Outage - I2 AL2S Backbone BATO-JACK
UndeterminedUndetermined
07/10/2014 7:44 PM
07/10/2014 7:45 PM






Unscheduled Outages Detail






Ticket No.:1943:135



Subject:Brief Outage Resolved - I2 AL2S Backbone ATLA-JCSN



Source Of Impact:Vendor



Affected:Backbone ATLA-JCSN



Start Time:Saturday, July 5, 2014, 8:48 PM (2048) UTC



End Time:Saturday, July 5, 2014, 8:49 PM (2049) UTC


Description:Backbone ATLA-JCSN was briefly unavailable. The Internet2's Optical
Vendor reports the reason for outage was due to damaged fiber.
Workgroup administrators receive notifications from OESS regarding
circuits re-routed on the backbone.







Ticket No.:1944:135



Subject:Outage Resolved - I2 AL2S Backbones ATLA-CHIC and ATLA-JCSN



Source Of Impact:Vendor



Affected:Backbones ATLA-CHIC and ATLA-JCSN



Start Time:Sunday, July 6, 2014, 7:19 PM (1919) UTC



End Time:Monday, July 7, 2014, 2:35 PM (1435) UTC


Description:Backbones ATLA-CHIC and ATLA-JCSN is currently available to the
community. Internet2 Optical Vendor technicians repaired  a fiber cut in
Chattanooga, TN. Level(3) has completed splicing to restore service.
Workgroup administrators received notifications from OESS regarding
circuits re-routed on the
backbone.







Ticket No.:1945:135



Subject:Outage Resolved - I2 AL2S Backbone DALL-HOUH



Source Of Impact:Vendor



Affected:Backbone DALL-HOUH



Start Time:Monday, July 7, 2014, 3:21 PM (1521) UTC



End Time:Monday, July 7, 2014, 8:13 PM (2013) UTC


Description:Backbone DALL-HOUH is now available to the community.  Fiber provider
Engineers report the reason for outage was damaged fiber near
Montgomery, TX which was caused by a company digging on private
property who then dug too close to railroad access.  The fiber has
been rolled  to spare fiber and all services have been restored.  No
more outages are expected. A maintenance may be scheduled to provide a
permanent fix. Notifications will be sent once a  maintenance window
has been determined.

Workgroup administrators received notifications from OESS regarding
circuits re-routed on the backbone.







Ticket No.:1948:135



Subject:Brief Outage Resolved - I2 AL2S Backbone ATLA-CHIC



Source Of Impact:Undetermined



Affected:Backbone ATLA-CHIC



Start Time:Monday, July 7, 2014, 6:27 PM (1827) UTC



End Time:Monday, July 7, 2014, 6:28 PM (1828) UTC


Description:Backbone ATLA-CHIC was unavailable. Internet2's fiber vendor reports the reason for outage is undetermined. Workgroup administrators receive notifications from OESS
regarding circuits re-routed on the backbone.







Ticket No.:1950:135



Subject:Brief Outage Resolved - I2 AL2S Backbone ATLA-JCSN



Source Of Impact:Vendor



Affected:Backbone ATLA-JCSN



Start Time:Wednesday, July 9, 2014, 4:01 AM (0401) UTC



End Time:Wednesday, July 9, 2014, 4:02 AM (0402) UTC


Description:Backbone ATLA-JCSN  was briefly unavailable. Internet2's fiber provider Engineers report the reason for outage was due to hardware failure. The card has been replaced, and no further outages are expected. Workgroup administrators received notifications
from OESS regarding circuits re-routed on the backbone.







Ticket No.:1975:135



Subject:Brief Outage - I2 AL2S Backbone BATO-JACK



Source Of Impact:Undetermined



Affected:Backbone BATO-JACK



Start Time:Thursday, July 10, 2014, 7:44 PM (1944) UTC



End Time:Thursday, July 10, 2014, 7:45 PM (1945) UTC


Description:Backbone BATO-JACK was briefly unavailable. Internet2 Engineers are
investigating the cause; however, the reason for the outage remains
pending. Workgroup administrators receive notifications from OESS
regarding circuits re-routed on the backbone.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





1934
3-Elevated
2-High

Maintenance Completed - I2 AL2S Various Backbones
Software
Vendor07/07/2014 7:01 AM
07/07/2014 7:10 AM


07/07/2014 7:43 AM
07/07/2014 8:02 AM


07/07/2014 8:28 AM
07/07/2014 8:44 AM




Scheduled Maintenances Detail





Ticket No.:1934:135



Subject:Maintenance Completed - I2 AL2S Various Backbones



Source Of Impact:Vendor



Affected:Backbone ALBA-BOST
Backbone ALBA-CLEV
Backbone PORT-SUNN
Backbone PORT-SEAT
Backbone SALT-SUNN



Start Time:Monday, July 7, 2014, 7:01 AM (0701) UTC



End Time:Monday, July 7, 2014, 7:10 AM (0710) UTC


Start Time:Monday, July 7, 2014, 7:43 AM (0743) UTC


		    
End Time:Monday, July 7, 2014, 8:02 AM (0802) UTC


Start Time:Monday, July 7, 2014, 8:28 AM (0828) UTC


		    
End Time:Monday, July 7, 2014, 8:44 AM (0844) UTC


Description:   The above Backbones were unavailable while Internet2's Fiber Provider
performed maintenance. The maintenance has been successfully completed.
Workgroup administrators received notifications from OESS regarding
circuits re-routed on the backbone.

*Please note the adjusted outage times*



  










Downtime and Availability







Internet2 Advanced Layer 2 Service Core NodesDown TimeReporting Period Availability52 Week Availability


sdn-sw.albu.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.alba.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696%  100.00000%
sdn-sw.ashb.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696%  100.00000%
sdn-sw.atla.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.bato.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99905%  100.00000%
sdn-sw.bost.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99772%  100.00000%
sdn-sw.char.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.chic.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96376%  99.98975%
sdn-sw.clev.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.colu4.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.dall.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.denv.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.85770%  99.85902%
sdn-sw.elpa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%  100.00000%
sdn-sw.eqch.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.houh.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92752%  99.92885%
sdn-sw.jack.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99829%  100.00000%
sdn-sw.jcsn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98425%  99.98520%
sdn-sw.kans.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99810%  99.99943%
sdn-sw.losa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886%  100.00000%
sdn-sw.minn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98596%  100.00000%
sdn-sw.newy32aoa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99905%  100.00000%
sdn-sw.phil.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.phoe.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99715%  100.00000%
sdn-sw.pitt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99962%  100.00000%
sdn-sw.port.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.rale.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98956%  99.99886%
sdn-sw.salt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99829%  100.00000%
sdn-sw.seat.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99279%  100.00000%
sdn-sw.star.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99791%  100.00000%
sdn-sw.sunn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.tuls.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.wash.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99810%  100.00000%
      
Aggregate Internet2 Advanced Layer 2 Service Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%99.98967%99.99253%

 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Internet2 Advanced Layer 2 Service Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


I2-ALBA-BOST-100GE-09210
               0 hr 44 min0 hr 0 min
               99.56349%100.00000%
               99.98634%100.00000%
I2-ALBA-CLEV-100GE-07735
               0 hr 44 min0 hr 0 min
               99.56349%100.00000%
               99.97932%99.99981%
I2-ASHB-CHIC-100GE-11803
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89887%99.98975%
I2-ASHB-PITT-100GE-07737
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.77117%99.88065%
I2-ASHB-WASH-100GE-09106
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90874%99.99981%
I2-ASHB-WASH-100GE-11823
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90987%100.00000%
I2-ATLA-CHAR-100GE-07738
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96812%99.97932%
I2-ATLA-CHIC-100GE-07740
               19 hr 17 min19 hr 17 min
               88.52183%88.52183%
               99.52603%99.71577%
I2-ATLA-HOUH-10GE-07742
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-JACK-100GE-07739
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99241%100.00000%
I2-ATLA-JCSN-100GE-10734
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-JCSN-100GE-10784
               19 hr 18 min19 hr 18 min
               88.51190%88.51190%
               99.44843%99.50364%
I2-ATLA-WASH-100GE-07738
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-BATO-HOUH-100GE-08860
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90361%99.92847%
I2-BATO-JACK-100GE-11005
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.70268%99.73304%
I2-BOST-NEWY32AOA-100GE-09211
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98160%100.00000%
I2-CHAR-RALE-100GE-10633
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96774%99.97970%
I2-CHIC-COLU4-100GE-11554
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95978%99.97856%
I2-CHIC-KANS-100GE-07745
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.74480%99.91006%
I2-CHIC-STAR-100GE-07743
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.67099%99.78370%
I2-CLEV-EQCH-100GE-07736
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-CLEV-NEWY32AOA-100GE-07735
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99962%100.00000%
I2-CLEV-PITT-100GE-10334
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90323%100.00000%
I2-CLEV-STAR-100GE-07736
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.86737%99.98596%
I2-COLU4-KANS-100GE-11548
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98197%99.98235%
I2-DALL-HOUH-100GE-07751
               4 hr 52 min4 hr 52 min
               97.10317%97.10317%
               99.85940%99.87325%
I2-DALL-TULS-100GE-11300
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.84384%99.84460%
I2-DENV-ELPA-100GE-07748
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97913%99.99260%
I2-DENV-KANS-100GE-07746
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.83910%99.88141%
I2-DENV-SALT-100GE-07747
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90494%99.96983%
I2-ELPA-HOUH-100GE-07749
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.31960%99.39132%
I2-ELPA-PHOE-100GE-07750
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89507%99.91595%
I2-ELPA-TUCS-100GE-07750
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-EQCH-STAR-100GE-09299
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.96300%99.97439%
I2-EQCH-MINN-100GE-12184
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99753%100.00000%
I2-HOUH-JCSN-100GE-10783
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.91177%99.91405%
I2-HOUH-TULS-100GE-07751
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.68788%99.75068%
I2-KANS-TULS-100GE-07753
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.90912%99.99981%
I2-LOSA-PHOE-100GE-09190
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.78408%99.80874%
I2-LOSA-SALT-100GE-07757
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.88293%99.93226%
I2-LOSA-SUNN-100GE-07755
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99355%99.99943%
I2-MINN-SEAT-100GE-09130
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.35318%99.54425%
I2-NEWY23AOA-PHIL-100GE-07759
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99431%99.99981%
I2-PHIL-WASH-100GE-10867
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.88881%99.99981%
I2-PORT-SEAT-100GE-08997
               0 hr 44 min0 hr 0 min
               99.56349%100.00000%
               99.98900%100.00000%
I2-PORT-SUNN-100GE-10343
               0 hr 44 min0 hr 0 min
               99.56349%100.00000%
               99.98843%99.99810%
I2-RALE-WASH-100GE-08888
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.78806%99.82810%
I2-SALT-SUNN-100GE-07766
               0 hr 44 min0 hr 0 min
               99.56349%100.00000%
               99.97287%99.99545%
      
Aggregate All Internet2 Advanced Layer 2 Service Backbone Circuits
                   47 hr 8 min3 hr 40 min
                   99.41551%99.46098%
                   99.87121%99.91593%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  

Upcoming Scheduled Maintenances





Ticket No.:2007:135



Subject:Maintenance Completed - I2 AL2S Backbone COLU4-KANS



Affected:Backbone COLU4-KANS



Start Time:Tuesday, July 29, 2014, 5:00 AM (0500) UTC



End Time:Tuesday, July 29, 2014, 10:00 AM (1000) UTC



Description:   Backbone COLU4-KANS was involved in a maintenance being executed by
Internet2 fiber vendor. This was not supposed to be service impacting,
however it was.  Maintenance has been completed and services have
restored.



  







Ticket No.:2068:135



Subject:Maintenance - I2 AL2S Backbone PORT-SEAT



Affected:Backbone PORT-SEAT



Start Time:Friday, August 15, 2014, 7:00 AM (0700) UTC



End Time:Friday, August 15, 2014, 9:00 AM (0900) UTC



Description:   Backbone PORT-SEAT will be unavailable while Level 3 Communications 
performs hardware maintenance. An outage is expected, and the entire 
window is reserved. Backbone traffic will re-route during this maintenance. 
Workgroup administrators will receive notifications from OESS regarding 
circuits re-routed on the backbone.



  







Ticket No.:2052:135



Subject:Maintenance 1 of 2 - I2 AL2S Various Backbones



Affected:Backbone ASHB-CHIC
Backbone CHIC-CLEV
Backbone CLEV-EQCH



Start Time:Thursday, August 28, 2014, 4:00 AM (0400) UTC



End Time:Thursday, August 28, 2014, 10:00 AM (1000) UTC



Description:   Tha above listed backbones will be unavailable while
Internet2's Fiber Provider Engineers perform
maintenance.  An outage of 2 hours is expected;
however, the entire window is reserved.  Backbone traffic
will re-route during this maintenance. Workgroup administrators will
receive notifications from OESS regarding circuits re-routed on the
backbone.An alternate window has been reserved for date/time, should it be required.



  







Ticket No.:2053:135



Subject:Maintenance 2 of 2 - I2 AL2S Various Backbones



Affected:Backbone ASHB-CHIC
Backbone CHIC-CLEV
Backbone CLEV-EQCH



Start Time:Friday, August 29, 2014, 4:00 AM (0400) UTC



End Time:Friday, August 29, 2014, 10:00 AM (1000) UTC



Description:   Tha above listed backbones will be unavailable while
Internet2's Fiber Provider Engineers perform
maintenance.  An outage of 2 hours is expected;
however, the entire window is reserved.  Backbone traffic
will re-route during this maintenance. Workgroup administrators will
receive notifications from OESS regarding circuits re-routed on the
backbone. This is an alternate window for ticket 2052:135, should it be required.