Unified - Internet2 Advanced Layer 2 Service Network Operations Report
03/23/2015 - 03/29/2015

Unified - Internet2 Advanced Layer 2 Service Network Operations Center
at Indiana University

New Connections

ConnectorTypeDateSponsor
USDA (DENV)10GE3/23/2015
USDA (KANS)10GE3/26/2015


Upgrades

None


New Network Resources

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Source Of Impact Start Time (UTC) End Time (UTC)

31994-Normal 1-Critical Brief Outage Resolved - I2 Optical DWDM System CHIC-EQCH Human ErrorVendor 03/24/2015 5:59 PM 03/24/2015 6:00 PM
32234-Normal Brief Outage Resolved - I2 Optical Various Circuits UndeterminedVendor 03/26/2015 5:38 PM 03/26/2015 5:39 PM
32274-Normal 1-Critical Brief Outage - I2 Optical DWDM System EUGE-PORT UndeterminedUndetermined 03/26/2015 9:52 PM 03/26/2015 9:53 PM
32484-Normal Outage Resolved - I2 AL2S Core Node (ASHB) HardwareVendor 03/29/2015 1:02 PM 03/29/2015 2:30 PM

Unscheduled Outages Detail


Ticket No.:3199:144
Subject:Brief Outage Resolved - I2 Optical DWDM System CHIC-EQCH
Source Of Impact:Vendor
Affected:DWDM System CHIC-EQCH
AL2S Backbone ASHB-CHIC
IP Backbone CHIC-CLEV
Start Time:Tuesday, March 24, 2015, 5:59 PM (1759) UTC
End Time:Tuesday, March 24, 2015, 6:00 PM (1800) UTC
Description:DWDM System CHIC-EQCH was briefly unavailable to the community. Level (3) Communications Engineers determined that the outage was caused due to a fiber bump.

Ticket No.:3223:144
Subject:Brief Outage Resolved - I2 Optical Various Circuits
Source Of Impact:Vendor
Affected:AL2S Backbone LOSE-PHOE
IP Backbone HOUS-LOSA
Optical Member ESnet
Start Time:Thursday, March 26, 2015, 5:38 PM (1738) UTC
End Time:Thursday, March 26, 2015, 5:39 PM (1739) UTC
Description:The above listed circuits were briefly unavailable to the community. Level(3) Engineers reviewed the issue and were not able to confirm an RFO.

Ticket No.:3227:144
Subject:Brief Outage - I2 Optical DWDM System EUGE-PORT
Source Of Impact:Undetermined
Affected:Optical DWDM EUGE-PORT
AL2S Backbone PORT-SUNN
IP Backbone LOSA-SEAT
Start Time:Thursday, March 26, 2015, 9:52 PM (2152) UTC
End Time:Thursday, March 26, 2015, 9:53 PM (2153) UTC
Description:DWDM System EUGE-PORT was briefly unavailable to the community. Level (3) Communications Engineers are currently investigating the reason for the outage.

Ticket No.:3248:144
Subject:Outage Resolved - I2 AL2S Core Node (ASHB)
Source Of Impact:Vendor
Affected:AL2S Backbone ASHB-CHIC
AL2S Backbone ASHB-PITT
AL2S Backbone ASHB-WASH
IP Backbone CLEV-WASH
AL2S Participant MARIA
AL2S Participant NIH
AL2S Participant Smithsonian
AL2S Participant Merit
AL2S Net+ Provider Amazon
AL2S Net+ Provider Blackboard
IP Connector MARIA (WASH)*
IP Participant NIH (WASH)*
IP Connector Smithsonian (WASH)*
IP Participant Merit (WASH)*
IP Net+ Provider Amazon
IP Net+ Provider Blackboard
Start Time:Sunday, March 29, 2015, 1:02 PM (1302) UTC
End Time:Sunday, March 29, 2015, 2:30 PM (1430) UTC
Description:The above listed assets are available after an outage. Internet2 Engineers have investigated the cause and determined that it was caused by a routing engine failure. All connections to the Core Node were affected. This outage has been resolved and no further outages are expected.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Source Of Impact Start Time (UTC) End Time (UTC)

None during this report period

Downtime and Availability



Unified - Internet2 Advanced Layer 2 Service Core NodesDown TimeReporting Period Availability52 Week Availability
sdn-sw.albu.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.alba.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.ashb.net.internet2.edu 1 hr 28 min1 hr 28 min 99.12698%99.12698% 99.94365% 99.94365%
sdn-sw.atla.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.bato.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.bost.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.char.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99639% 100.00000%
sdn-sw.chic.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99962% 99.99962%
sdn-sw.clev.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867% 100.00000%
sdn-sw.colu4.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98900% 100.00000%
sdn-sw.dall.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.denv.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98710% 99.99051%
sdn-sw.elpa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99772% 100.00000%
sdn-sw.eqch.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99753% 100.00000%
sdn-sw.houh.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.jack.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99791% 100.00000%
sdn-sw.jcsn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.kans.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99298% 100.00000%
sdn-sw.losa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99715% 100.00000%
sdn-sw.minn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.miss2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.newy32aoa.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.phil.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.phoe.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99905% 99.99924%
sdn-sw.pitt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99886% 100.00000%
sdn-sw.port.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
sdn-sw.rale.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99602% 99.99848%
sdn-sw.salt.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696% 100.00000%
sdn-sw.seat.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99848% 99.99848%
sdn-sw.star.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99013% 100.00000%
sdn-sw.sunn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99583% 100.00000%
sdn-sw.tuls.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99526% 100.00000%
sdn-sw.wash.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981% 99.99981%
sdn-srv2.chic.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.86282% 100.00000%
sdn-srv3.chic.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.86282% 100.00000%

Aggregate Unified - Internet2 Advanced Layer 2 Service Core Nodes1 hr 28 min1 hr 28 min99.97506%99.97506%99.98839%99.99799%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 Advanced Layer 2 Service Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
I2-ALBA-BOST-100GE-09210 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99905%99.99905%
I2-ALBA-CLEV-100GE-07735 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99165%99.99715%
I2-ASHB-CHIC-100GE-11803 1 hr 29 min1 hr 29 min 99.11706%99.11706% 99.91082%99.98235%
I2-ASHB-PITT-100GE-07737 1 hr 28 min1 hr 28 min 99.12698%99.12698% 99.95921%99.97837%
I2-ASHB-WASH-100GE-09106 1 hr 28 min1 hr 28 min 99.12698%99.12698% 99.96395%99.97780%
I2-ASHB-WASH-100GE-11823 1 hr 28 min1 hr 28 min 99.12698%99.12698% 99.96395%99.97799%
I2-ATLA-CHAR-100GE-07739 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-CINC-100GE-07740 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-HOUH-10GE-07742 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-JACK-100GE-07739 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93207%99.99734%
I2-ATLA-JCSN-100GE-10734 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-JCSN-100GE-10784 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.81121%99.99848%
I2-ATLA-WASH-100GE-07738 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-BATO-HOUH-100GE-08860 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94289%99.99715%
I2-BATO-JACK-100GE-11005 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.28962%99.56151%
I2-BOST-HART2-100GE-09211 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99108%99.99791%
I2-CHAR-RALE-100GE-10633 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99241%99.99848%
I2-CHIC-INDI-100GE-50993 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CINC-INDI-100GE-51692 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-COLU4-100GE-11554 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93056%99.97476%
I2-CHIC-KANS-100GE-07745 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96869%99.99848%
I2-CHIC-STAR-100GE-07743 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97666%99.99962%
I2-CLEV-EQCH-100GE-07736 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95048%99.99962%
I2-CLEV-NEWY32AOA-100GE-07735 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CLEV-PITT-100GE-10334 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98767%100.00000%
I2-CLEV-STAR-100GE-07736 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-COLU4-KANS-100GE-11548 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91689%99.98729%
I2-DALL-HOUH-100GE-07751 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89526%99.99469%
I2-DALL-TULS-100GE-11300 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99526%100.00000%
I2-DENV-ELPA-100GE-07748 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98444%99.99051%
I2-DENV-KANS-100GE-07746 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93018%99.99051%
I2-DENV-SALT-100GE-07747 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97742%99.98975%
I2-ELPA-HOUH-100GE-07749 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.80798%99.92904%
I2-ELPA-PHOE-100GE-07750 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ELPA-TUCS-100GE-07750 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99772%100.00000%
I2-EQCH-STAR-100GE-09299 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99374%100.00000%
I2-EQCH-MINN-100GE-12184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97230%99.99981%
I2-HART2-NEWY32AOA-100GE-12947 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%99.99981%
I2-HOUH-JCSN-100GE-10783 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.74139%99.82184%
I2-HOUH-TULS-100GE-07751 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-KANS-TULS-100GE-07753 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98577%99.99981%
I2-LOSA-PHOE-100GE-09190 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.80248%99.82620%
I2-LOSA-SALT-100GE-07757 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98425%100.00000%
I2-LOSA-SUNN-100GE-07755 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99298%100.00000%
I2-MINN-MISS2-100GE-09130 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.67479%99.69869%
I2-MISS2-SEAT-100GE-13237 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97002%99.97287%
I2-NEWY23AOA-PHIL-100GE-07759 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99070%100.00000%
I2-PHIL-WASH-100GE-10867 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99450%99.99867%
I2-PHOE-TUCS-100GE-11990 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99658%99.99924%
I2-PORT-SEAT-100GE-08997 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98805%99.99810%
I2-PORT-SUNN-100GE-10343 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.98975%99.99962%
I2-RALE-WASH-100GE-08888 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99279%99.99829%
I2-SALT-SUNN-100GE-07766 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96357%99.97078%

Aggregate All Unified - Internet2 Advanced Layer 2 Service Backbone Circuits 5 hr 55 min0 hr 0 min 99.93355%99.93355% 99.94527%99.97361%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.



Upcoming Scheduled Maintenances

Ticket No.:2943:144
Subject:Maintenance 3 of 4 - I2 Optical OneControl Upgrade
Affected:oc-denv.net.internet2.edu
Start Time:Monday, March 30, 2015, 2:00 PM (1400) UTC
End Time:Wednesday, April 1, 2015, 9:00 PM (2100) UTC
Description: During the above time frame, system engineers will be upgrading oc-denv.net.internet2.edu to OneControl 4.2. Users should switch to using oc-indi.net.internet2.edu as the primary OneControl server during this time frame. A second notification will be sent when oc-denv has been upgrade to 4.2 and is ready for use.

Ticket No.:3295:144
Subject:Emergency Maintenance - I2 AL2S Backbones CHIC-KANS and DENV-KANS
Affected:AL2S Backbones CHIC-KANS and DENV-KANS
IP Backbones CHIC-KANS and KANS-SALT
Start Time:Thursday, April 2, 2015, 5:00 AM (0500) UTC
End Time:Thursday, April 2, 2015, 6:00 AM (0600) UTC
Description: The above listed assets will be unavailable while Internet2 Engineers, with the assistance of Level3 Technicians, perform emergency maintenance. An outage is expected and the entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:3232:144
Subject:Maintenance - I2 Optical Member ESNET (CHIC-NASH)
Affected:ESNET-CHIC-NASH-100GE-07807
Start Time:Friday, April 3, 2015, 3:00 AM (0300) UTC
End Time:Friday, April 3, 2015, 5:00 AM (0500) UTC
Description: Member ESnet's Optical circuit ESNET-CHIC-NASH-100GE-07807 will be unavailable while Level3 Engineers perform maintenance. A brief outage is expected. The entire maintenance window has been reserved.

Ticket No.:3245:144
Subject:Maintenance - I2 Optical Member ESnet
Affected:Optical Member ESnet
Start Time:Friday, April 3, 2015, 3:00 AM (0300) UTC
End Time:Friday, April 3, 2015, 5:00 AM (0500) UTC
Description: Member ESnet's Optical circuit ESNET-CHIC-NASH-100GE-07807 will be unavailable while Level(3) Engineers perform maintenance to troubleshoot a "Pre-FEC Signal Degrade" on an OCLD. . A brief outage is expected, however the entire maintenance window has been reserved.

Ticket No.:3297:144
Subject:Maintenance *Tracking* - I2 Optical Node opt.sacr
Affected:Optical Node opt.sacr
Start Time:Saturday, April 4, 2015, 7:00 AM (0700) UTC
End Time:Saturday, April 4, 2015, 11:00 AM (1100) UTC
Description: *Tracking Ticket The above listed assets should remain available while Internet2 Engineers perform maintenance. An outage is not expected; however, the entire window is reserved.

Ticket No.:2978:144
Subject:Maintenance - I2 AL2S Connector CAAREN
Affected:Connector CAAREN and its Participants
Start Time:Tuesday, April 7, 2015, 4:00 AM (0400) UTC
End Time:Tuesday, April 7, 2015, 5:00 AM (0500) UTC
Description: The above listed will be unavailable while Internet2 and CAAREN Engineers migrate VLANs carrying CAAREN peerings. A brief outage is expected; however, the entire maintenance window is reserved.

Ticket No.:3219:144
Subject:Maintenance 1 of 2- I2 Optical Member ESnet
Affected:I2 Optical Member ESnet
Start Time:Tuesday, April 7, 2015, 4:00 AM (0400) UTC
End Time:Tuesday, April 7, 2015, 9:00 AM (0900) UTC
Description: *Please note that this maintenance has been rescheduled* Member ESnet's Optical circuit ESNET-CHAT-ORNL-100GE-06347 will be unavailable while Level (3) Engineers perform maintenance. A brief outage is expected, however the entire maintenance window has been reserved. This is maintenance window 1 of 2.

Ticket No.:3221:144
Subject:Maintenance 2 of 2- I2 Optical Member ESnet
Affected:I2 Optical Member ESnet
Start Time:Wednesday, April 8, 2015, 4:00 AM (0400) UTC
End Time:Wednesday, April 8, 2015, 9:00 AM (0900) UTC
Description: *Please note that this maintenance has been rescheduled* Member ESnet's Optical circuit ESNET-CHAT-ORNL-100GE-06347 will be unavailable while Level (3) Engineers perform maintenance. A brief outage is expected, however the entire maintenance window has been reserved. This is maintenance window 2 of 2.