Internet2 Advanced Layer 2 Service Network Operations Report
05/19/2014 - 05/25/2014

Internet2 Advanced Layer 2 Service Network Operations Center
at Indiana University

New Connections

None


Upgrades

None

New Network Resources

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Source Of Impact
Start Time (UTC)

End Time (UTC)





17453-Elevated
2-High

Brief Outage Resolved - I2 AL2S Backbone LOSA-PHOE
HardwareVendor
05/20/2014 4:44 PM
05/20/2014 4:45 PM


17532-High
2-High

Brief Outage - I2 AL2S Backbone ASHB-WASH
UndeterminedUndetermined
05/22/2014 12:05 AM
05/22/2014 12:06 AM


17562-High
2-High

Brief Outage - I2 AL2S Backbone ATLA-CHIC
UndeterminedUndetermined
05/22/2014 4:06 PM
05/22/2014 4:07 PM


17652-High
2-High

Brief Outage Resolved - I2 AL2S Backbones ATLA-CHIC and ATLA-CHAR
SoftwareVendor
05/23/2014 8:35 PM
05/23/2014 8:37 PM






Unscheduled Outages Detail






Ticket No.:1745:135



Subject:Brief Outage Resolved - I2 AL2S Backbone LOSA-PHOE



Source Of Impact:Vendor



Affected:Backbone LOSA-PHOE



Start Time:Tuesday, May 20, 2014, 4:44 PM (1644) UTC



End Time:Tuesday, May 20, 2014, 4:45 PM (1645) UTC


Description:Backbone LOSA-PHOE is now available following a brief outage.
Internet2's Optical Vendor engineers have advised that this was
related to a hardware failure in their network. Workgroup
administrators received notifications from OESS regarding circuits
re-routed on the backbone.







Ticket No.:1753:135



Subject:Brief Outage - I2 AL2S Backbone ASHB-WASH



Source Of Impact:Undetermined



Affected:Backbone ASHB-WASH



Start Time:Thursday, May 22, 2014, 12:05 AM (0005) UTC



End Time:Thursday, May 22, 2014, 12:06 AM (0006) UTC


Description:Backbone ASHB-WASH was briefly unavailable. Internet2 Engineers and
Level(3) Communications have been contacted,
however the reason for the outage is pending. Workgroup administrators
receive notifications from OESS
regarding circuits re-routed on the backbone.







Ticket No.:1756:135



Subject:Brief Outage - I2 AL2S Backbone ATLA-CHIC



Source Of Impact:Undetermined



Affected:Backbone ATLA-CHIC



Start Time:Thursday, May 22, 2014, 4:06 PM (1606) UTC



End Time:Thursday, May 22, 2014, 4:07 PM (1607) UTC


Description:Backbone ATLA-CHIC was briefly unavailable. Internet2's Optical Vendor
Engineers are investigating the cause; however, the reason for the outage
is pending. Workgroup administrators receive notifications from OESS
regarding circuits re-routed on the backbone.







Ticket No.:1765:135



Subject:Brief Outage Resolved - I2 AL2S Backbones ATLA-CHIC and ATLA-CHAR



Source Of Impact:Vendor



Affected:Backbone ATLA-CHIC
Backbone ATLA-CHAR



Start Time:Friday, May 23, 2014, 8:35 PM (2035) UTC



End Time:Friday, May 23, 2014, 8:37 PM (2037) UTC


Description:Backbones ATLA-CHIC and ATLA-CHAR was briefly unavailable. While
Internet2 Vendors were debugging a non-service impacting error, a
hardware reset occurred on the Internet2 AL2S Core Node in Atlanta.
Service has been restored and no further outages are expected. 
Workgroup administrators receive notifications from OESS regarding
circuits re-routed on the backbone.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Source Of Impact
Start Time (UTC)
End Time (UTC)





1732
3-Elevated
2-High

Emergency Maintenance Completed - I2 AL2S Code Node CHIC (Line Card 1)
Hardware
Internal05/20/2014 4:57 PM
05/20/2014 5:37 PM


1761
3-Elevated
2-High

Emergency Maintenance Completed - I2 AL2S Backbone ATLA-CHIC
CircuitInternal
05/22/2014 9:19 PM
05/22/2014 9:22 PM


1763
3-Elevated
2-High

Emergency Maintenance Completed - I2 AL2S Backbone ATLA-CHIC
Circuit
Internal05/23/2014 9:17 PM
05/23/2014 9:19 PM


1752
3-Elevated
2-High

Maintenance Completed - I2 AL2S Backbone LOSA-PHOE
CircuitVendor
05/24/2014 7:42 AM
05/24/2014 9:34 AM




Scheduled Maintenances Detail





Ticket No.:1732:135



Subject:Emergency Maintenance Completed - I2 AL2S Code Node CHIC (Line Card 1)



Source Of Impact:Internal



Affected:Code Node CHIC (Line Card 1)
Participant Indiana GigaPOP



Start Time:Tuesday, May 20, 2014, 4:57 PM (1657) UTC



End Time:Tuesday, May 20, 2014, 5:37 PM (1737) UTC


Description:   The above listed are available after Internet2 Engineers performed
maintenance to replace a failed line card. Backbone traffic re-routed
during this maintenance and workgroup administrators received
notifications from OESS regarding circuits re-routed on the backbone.

*Please note this maintenance was rescheduled from 5/30/14 to occur at
1530 UTC Today, 5/20/14, to resolve an ongoing outage*



  







Ticket No.:1761:135



Subject:Emergency Maintenance Completed - I2 AL2S Backbone ATLA-CHIC



Source Of Impact:Internal



Affected:Backbone ATLA-CHIC



Start Time:Thursday, May 22, 2014, 9:19 PM (2119) UTC



End Time:Thursday, May 22, 2014, 9:22 PM (2122) UTC


Description:   Backbone ATLA-CHIC was unavailable while Level 3 Engineers,
working with Internet2 Engineers, performed fiber cleaning maintenance, 
which is completed. Backbone traffic will re-route during this maintenance. 
Workgroup administrators will receive notifications from OESS regarding 
circuits re-routed on the backbone.



  







Ticket No.:1763:135



Subject:Emergency Maintenance Completed - I2 AL2S Backbone ATLA-CHIC



Source Of Impact:Internal



Affected:Backbone ATLA-CHIC



Start Time:Friday, May 23, 2014, 9:17 PM (2117) UTC



End Time:Friday, May 23, 2014, 9:19 PM (2119) UTC


Description:   Backbone ATLA-CHIC was unavailable while Level 3 Engineers, working
with Internet2 Engineers, performed fiber cleaning maintenance, which is
completed. Backbone traffic will re-route during this maintenance.
Workgroup administrators will receive notifications from OESS regarding
circuits re-routed on the backbone.



  







Ticket No.:1752:135



Subject:Maintenance Completed - I2 AL2S Backbone LOSA-PHOE



Source Of Impact:Vendor



Affected:Backbone LOSA-PHOE



Start Time:Saturday, May 24, 2014, 7:42 AM (0742) UTC



End Time:Saturday, May 24, 2014, 9:34 AM (0934) UTC


Description:   Backbone  LOSA-PHOE was unavailable while Level(3) Engineers 
performed hardware maintenance in Los Angeles, CA.  Maintenance has 
been completed. Backbone traffic re-routed during this maintenance. No 
further outages are expected.



  










Downtime and Availability







Internet2 Advanced Layer 2 Service Core NodesDown TimeReporting Period Availability52 Week Availability


sdn-sw.albu.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.alba.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696%  100.00000%
sdn-sw.ashb.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99677%  100.00000%
sdn-sw.atla.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.bost.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99564%  100.00000%
sdn-sw.char.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%  100.00000%
sdn-sw.chic.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96376%  99.98975%
sdn-sw.clev.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.dall.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.denv.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.85770%  99.85902%
sdn-sw.elpa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99943%  100.00000%
sdn-sw.houh.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92752%  99.92885%
sdn-sw.jack.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99829%  100.00000%
sdn-sw.jcsn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98425%  99.98520%
sdn-sw.kans.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99810%  99.99943%
sdn-sw.losa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886%  100.00000%
sdn-sw.minn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98596%  100.00000%
sdn-sw.newy32aoa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99905%  100.00000%
sdn-sw.phil.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.phoe.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99715%  100.00000%
sdn-sw.pitt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99962%  100.00000%
sdn-sw.port.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99924%  100.00000%
sdn-sw.rale.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98937%  99.99886%
sdn-sw.salt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99829%  100.00000%
sdn-sw.seat.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99279%  100.00000%
sdn-sw.star.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99791%  100.00000%
sdn-sw.sunn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.tuls.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%  100.00000%
sdn-sw.wash.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99810%  100.00000%
      
Aggregate Internet2 Advanced Layer 2 Service Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%99.98855%99.99176%

 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  
	    

	    
Internet2 Advanced Layer 2 Service Backbone CircuitsDown Time
Reporting Period Availability52 Week Availability


I2-ALBA-BOST-100GE-09210
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99393%99.99924%
I2-ALBA-CLEV-100GE-07735
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98767%99.99981%
I2-ASHB-CHIC-100GE-11803
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98900%99.98975%
I2-ASHB-PITT-100GE-07737
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.86130%99.88065%
I2-ASHB-WASH-100GE-09106
               0 hr 1 min0 hr 1 min
               99.99008%99.99008%
               99.99886%99.99981%
I2-ASHB-WASH-100GE-11823
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-CHAR-100GE-07738
               0 hr 2 min0 hr 2 min
               99.98016%99.98016%
               99.97932%99.97932%
I2-ATLA-CHIC-100GE-07740
               0 hr 8 min0 hr 3 min
               99.92063%99.97024%
               99.80988%99.88122%
I2-ATLA-HOUH-10GE-07742
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-JACK-100GE-07739
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99412%100.00000%
I2-ATLA-JCSN-100GE-10734
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-ATLA-JCSN-100GE-10784
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.67953%99.72355%
I2-ATLA-WASH-100GE-07738
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               100.00000%100.00000%
I2-BATO-HOUH-100GE-08860
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.92543%99.92847%
I2-BATO-JACK-100GE-11005
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.88350%99.89488%
I2-BOST-NEWY32AOA-100GE-09211
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98634%100.00000%
I2-CHAR-RALE-100GE-10633
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97894%99.97970%
I2-CHIC-COLU4-100GE-11554
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98881%99.98956%
I2-CHIC-KANS-100GE-07745
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.78085%99.92164%
I2-CHIC-STAR-100GE-07743
               0 hr 40 min0 hr 0 min
               99.60317%100.00000%
               99.84100%99.86358%
I2-CLEV-NEWY32AOA-100GE-07735
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98691%100.00000%
I2-CLEV-PITT-100GE-10334
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99336%100.00000%
I2-CLEV-STAR-100GE-07736
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.95921%99.98596%
I2-COLU4-KANS-100GE-11548
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.98254%99.98254%
I2-DALL-HOUH-100GE-07751
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.92600%99.92866%
I2-DALL-TULS-100GE-11300
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.84403%99.84479%
I2-DENV-KANS-100GE-07746
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.83910%99.88141%
I2-DENV-SALT-100GE-07747
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.92676%99.96983%
I2-ELPA-HOUH-100GE-07749
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.40991%99.48163%
I2-ELPA-PHOE-100GE-07750
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89716%99.91595%
I2-HOUH-JCSN-100GE-10783
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.91310%99.91405%
I2-HOUH-TULS-100GE-07751
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.68788%99.75068%
I2-KANS-TULS-100GE-07753
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99924%99.99981%
I2-LOSA-PHOE-100GE-09190
               1 hr 53 min0 hr 1 min
               98.87897%99.99008%
               99.78616%99.80874%
I2-LOSA-SALT-100GE-07757
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.88464%99.93226%
I2-LOSA-SUNN-100GE-07755
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99829%99.99943%
I2-MINN-SEAT-100GE-09130
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.37557%99.54463%
I2-MINN-STAR-100GE-09299
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.88293%99.90475%
I2-NEWY23AOA-PHIL-100GE-07759
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99905%99.99981%
I2-PHIL-WASH-100GE-10867
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.89375%100.00000%
I2-PORT-SEAT-100GE-08997
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99829%100.00000%
I2-PORT-SUNN-100GE-10343
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.99829%99.99962%
I2-RALE-WASH-100GE-08888
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.79926%99.82829%
I2-SALT-SUNN-100GE-07766
               0 hr 0 min0 hr 0 min
               100.00000%100.00000%
               99.97533%99.99658%
      
Aggregate All Internet2 Advanced Layer 2 Service Backbone Circuits
                   2 hr 44 min2 hr 37 min
                   99.96302%99.99842%
                   99.90080%99.92501%


 NOTE: Numbers in italics represent downtime with scheduled  maintenance excluded. Internet2's definition of scheduled
  maintenance is any anticipated outage announced at least 72
  hours in advance of the outage start time.  

Upcoming Scheduled Maintenances





Ticket No.:1758:135



Subject:Maintenance - I2 AL2S Backbone MINN-STAR



Affected:Backbone MINN-STAR



Start Time:Friday, May 30, 2014, 5:00 AM (0500) UTC



End Time:Friday, May 30, 2014, 7:00 AM (0700) UTC



Description:   Backbone MINN-STAR will be unavailable while Internet2 Engineers 
perform maintenance to insert a new core node between MINN-STAR, 
making EQCH-MINN and EQCH-STAR.  An outage is expected, and the 
entire window is reserved.  Backbone traffic will re-route during this 
maintenance. Workgroup administrators will receive notifications from 
OESS regarding circuits re-routed on the backbone.



  







Ticket No.:1596:135



Subject:Maintenance - I2 AL2S Backbone MINN-SEAT



Affected:Backbone MINN-SEAT



Start Time:Saturday, June 14, 2014, 1:00 PM (1300) UTC



End Time:Saturday, June 14, 2014, 11:00 PM (2300) UTC



Description:   Backbone MINN-SEAT will be unavailable while our fiber provider's third
party provider performs a cable relocation in Judson and Dickinson, ND.
This work must take place during daylight hours as the cable is located
along the railroad. An outage of up to 3 hours is expected; however the
entire window has been reserved.