Internet2 Advanced Layer 2 Service Network Operations Report
09/09/2013 - 09/15/2013

Internet2 Advanced Layer 2 Service Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

7992-High 2-High Internet2 Advanced Layer 2 Services Backbone Circuit SALT-SEAT Brief Outage Undetermined 09/11/2013 7:22 PM 09/11/2013 7:25 PM
8022-High 2-High Internet2 Advanced Layer 2 Services Backbone Circuit MINN-SEAT Outage Resolved Unannounced Maintenance 09/12/2013 4:57 AM 09/12/2013 10:56 AM
8092-High 2-High Internet2 Advanced Layer 2 Services Backbone Circuit DENV-KANS Brief Outage Undetermined 09/13/2013 9:56 AM 09/13/2013 9:57 AM
8122-High 2-High Internet2 Advanced Layer 2 Services Backbone Circuit CHIC-STAR Brief Outage Undetermined 09/13/2013 1:51 PM 09/13/2013 1:52 PM
8302-High 2-High Internet2 Advanced Layer 2 Services Backbone Circuit SALT-SUNN Brief Outage Undetermined 09/15/2013 9:44 PM 09/15/2013 9:45 PM

Unscheduled Outages Detail


Ticket No.:799:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit SALT-SEAT Brief Outage
Affected:Backbone Circuit SALT-SEAT
Start Time:Wednesday, September 11, 2013, 7:22 PM (1922) UTC
End Time:Wednesday, September 11, 2013, 7:25 PM (1925) UTC
Description:Backbone Circuit SALT-SEAT was briefly unavailable. Internet2 Engineers are investigating the cause; however, the reason for the outage is pending. Backbone traffic re-routed during the brief outage.

Ticket No.:802:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit MINN-SEAT Outage Resolved
Affected:Backbone Circuit MINN-SEAT
Start Time:Thursday, September 12, 2013, 4:57 AM (0457) UTC
End Time:Thursday, September 12, 2013, 10:56 AM (1056) UTC
Description:Backbone Circuit MINN-SEAT is unstable. Vendor Zayo has been contacted, and the instability was due to unannounced maintenance. Maintenance has been completed and no further outages are expected.

Ticket No.:809:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit DENV-KANS Brief Outage
Affected:Backbone Circuit DENV-KANS
Start Time:Friday, September 13, 2013, 9:56 AM (0956) UTC
End Time:Friday, September 13, 2013, 9:57 AM (0957) UTC
Description:Backbone Circuit DENV-KANS was briefly unavailable. Internet2 Engineers are investigating the cause; however, the reason for the outage is pending. Backbone traffic re-routed.

Ticket No.:812:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit CHIC-STAR Brief Outage
Affected:Backbone Circuit CHIC-STAR
Start Time:Friday, September 13, 2013, 1:51 PM (1351) UTC
End Time:Friday, September 13, 2013, 1:52 PM (1352) UTC
Description:The above listed circuit was briefly unavailable. Vendor Engineers have been contacted. No reason for outage has been provided. Updates will be provided as received.

Ticket No.:830:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit SALT-SUNN Brief Outage
Affected:Backbone Circuit SALT-SUNN
Start Time:Sunday, September 15, 2013, 9:44 PM (2144) UTC
End Time:Sunday, September 15, 2013, 9:45 PM (2145) UTC
Description:Backbone Circuit SALT-SUNN is now available following a brief outage. Internet2 Engineers are investigating the cause; however, the reason for the outage is pending. Backbone traffic should have re-routed.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

778 3-Elevated 2-High Internet2 Advanced Layer 2 Services Backbone Circuit ATLA-CHIC Maintenance Completed Circuit 09/12/2013 5:56 AM 09/12/2013 5:57 AM
804 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node DENV Emergency Maintenance Completed Software 09/13/2013 6:01 AM 09/13/2013 6:03 AM
805 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node CHIC Emergency Maintenance Completed Software 09/13/2013 6:16 AM 09/13/2013 6:18 AM
806 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node KANS Emergency Maintenance Completed Software 09/13/2013 6:06 AM 09/13/2013 6:09 AM
813 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (ATLA) Emergency Maintenance Completed Software 09/15/2013 8:07 AM 09/15/2013 8:10 AM
815 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (CLEV) Emergency Maintenance Completed Software 09/15/2013 8:21 AM 09/15/2013 8:24 AM
816 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (ELPA) Emergency Maintenance Completed Software 09/15/2013 8:56 AM 09/15/2013 8:59 AM
817 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (HOUH) Emergency Maintenance Completed Software 09/15/2013 8:34 AM 09/15/2013 8:37 AM
818 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (JACK) Emergency Maintenance Completed Software 09/15/2013 8:14 AM 09/15/2013 8:17 AM
819 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (LOSA) Emergency Maintenance Completed Software 09/15/2013 9:03 AM 09/15/2013 9:06 AM
820 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (NEWY) Emergency Maintenance Completed Software 09/15/2013 8:00 AM 09/15/2013 8:03 AM
821 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (PITT) Emergency Maintenance Completed Software 09/15/2013 8:21 AM 09/15/2013 8:23 AM
822 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (SALT) Emergency Maintenance Completed Software 09/15/2013 8:49 AM 09/15/2013 8:52 AM
823 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (STAR) Emergency Maintenance Completed Software 09/15/2013 8:23 AM 09/15/2013 8:30 AM
825 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (SUNN) Emergency Maintenance Completed Software 09/15/2013 8:10 AM 09/15/2013 9:13 AM
826 3-Elevated 1-Critical Internet2 Advanced Layer 2 Services Core Node (TULS) Emergency Maintenance Completed Software 09/15/2013 8:42 AM 09/15/2013 8:45 AM

Scheduled Maintenances Detail

Ticket No.:778:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit ATLA-CHIC Maintenance Completed
Affected:Backbone Circuit ATLA-CHIC
Start Time:Thursday, September 12, 2013, 5:56 AM (0556) UTC
End Time:Thursday, September 12, 2013, 5:57 AM (0557) UTC
Description: Backbone Circuit ATLA-CHIC was unavailable while Level(3) Engineers performed maintenance. Maintenance has been completed.

Ticket No.:804:135
Subject:Internet2 Advanced Layer 2 Services Core Node DENV Emergency Maintenance Completed
Affected:Core Node DENV and its Connector(s) and Participants
Start Time:Friday, September 13, 2013, 6:01 AM (0601) UTC
End Time:Friday, September 13, 2013, 6:03 AM (0603) UTC
Description: Core Node DENV was briefly unavailable while Internet2 Engineers performed Core Node software maintenance to enable error correction. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:805:135
Subject:Internet2 Advanced Layer 2 Services Core Node CHIC Emergency Maintenance Completed
Affected:Core Node CHIC and its Connector(s) and Participants
Start Time:Friday, September 13, 2013, 6:16 AM (0616) UTC
End Time:Friday, September 13, 2013, 6:18 AM (0618) UTC
Description: Core Node CHIC was briefly unavailable while Internet2 Engineers performed Core Node software maintenance to enable error correction. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:806:135
Subject:Internet2 Advanced Layer 2 Services Core Node KANS Emergency Maintenance Completed
Affected:Core Node KANS and its Connector(s) and Participants
Start Time:Friday, September 13, 2013, 6:06 AM (0606) UTC
End Time:Friday, September 13, 2013, 6:09 AM (0609) UTC
Description: Core Node KANS was unavailable while Internet2 Engineers performed Core Node software maintenance to enable error correction. All connections to the Core Node were affected.

Ticket No.:813:135
Subject:Internet2 Advanced Layer 2 Services Core Node (ATLA) Emergency Maintenance Completed
Affected:Core Node ATLA and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:07 AM (0807) UTC
End Time:Sunday, September 15, 2013, 8:10 AM (0810) UTC
Description: Core Node ATLA was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:815:135
Subject:Internet2 Advanced Layer 2 Services Core Node (CLEV) Emergency Maintenance Completed
Affected:Core Node CLEV and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:21 AM (0821) UTC
End Time:Sunday, September 15, 2013, 8:24 AM (0824) UTC
Description: Core Node CLEV was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:816:135
Subject:Internet2 Advanced Layer 2 Services Core Node (ELPA) Emergency Maintenance Completed
Affected:Core Node ELPA and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:56 AM (0856) UTC
End Time:Sunday, September 15, 2013, 8:59 AM (0859) UTC
Description: Core Node ELPA was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:817:135
Subject:Internet2 Advanced Layer 2 Services Core Node (HOUH) Emergency Maintenance Completed
Affected:Core Node HOUH and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:34 AM (0834) UTC
End Time:Sunday, September 15, 2013, 8:37 AM (0837) UTC
Description: Core Node HOUH was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:818:135
Subject:Internet2 Advanced Layer 2 Services Core Node (JACK) Emergency Maintenance Completed
Affected:Core Node JACK and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:14 AM (0814) UTC
End Time:Sunday, September 15, 2013, 8:17 AM (0817) UTC
Description: Core Node JACK was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected..

Ticket No.:819:135
Subject:Internet2 Advanced Layer 2 Services Core Node (LOSA) Emergency Maintenance Completed
Affected:Core Node LOSA and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 9:03 AM (0903) UTC
End Time:Sunday, September 15, 2013, 9:06 AM (0906) UTC
Description: Core Node LOSA was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed.Traffic re-routed during this maintenance.

Ticket No.:820:135
Subject:Internet2 Advanced Layer 2 Services Core Node (NEWY) Emergency Maintenance Completed
Affected:Core Node NEWY and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:00 AM (0800) UTC
End Time:Sunday, September 15, 2013, 8:03 AM (0803) UTC
Description: Core Node NEWY was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:821:135
Subject:Internet2 Advanced Layer 2 Services Core Node (PITT) Emergency Maintenance Completed
Affected:Core Node PITT and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:21 AM (0821) UTC
End Time:Sunday, September 15, 2013, 8:23 AM (0823) UTC
Description: Core Node PITT was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:822:135
Subject:Internet2 Advanced Layer 2 Services Core Node (SALT) Emergency Maintenance Completed
Affected:Core Node SALT and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:49 AM (0849) UTC
End Time:Sunday, September 15, 2013, 8:52 AM (0852) UTC
Description: Core Node SALT was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:823:135
Subject:Internet2 Advanced Layer 2 Services Core Node (STAR) Emergency Maintenance Completed
Affected:Core Node STAR and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:23 AM (0823) UTC
End Time:Sunday, September 15, 2013, 8:30 AM (0830) UTC
Description: Core Node STAR was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:825:135
Subject:Internet2 Advanced Layer 2 Services Core Node (SUNN) Emergency Maintenance Completed
Affected:Core Node SUNN and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:10 AM (0810) UTC
End Time:Sunday, September 15, 2013, 9:13 AM (0913) UTC
Description: Core Node SUNN was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.

Ticket No.:826:135
Subject:Internet2 Advanced Layer 2 Services Core Node (TULS) Emergency Maintenance Completed
Affected:Core Node TULS and its Connector(s) and Participants
Start Time:Sunday, September 15, 2013, 8:42 AM (0842) UTC
End Time:Sunday, September 15, 2013, 8:45 AM (0845) UTC
Description: Core Node TULS was briefly unavailable while Internet2 Engineers performed software maintenance to resolve switch fabric errors on the switch. Maintenance has been completed. All connections to the Core Node were affected.


Downtime and Availability



Internet2 Advanced Layer 2 Service Core NodesDown TimeReporting Period Availability52 Week Availability
sdn-sw.albu.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 100.00000% 100.00000
sdn-sw.ashb.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99772% 100.00000
sdn-sw.atla.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97401% 100.00000
sdn-sw.bost.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99791% 100.00000
sdn-sw.chic.net.internet2.edu 0 hr 2 min0 hr 0 min 99.98016%100.00000 99.97856% 100.00000
sdn-sw.clev.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97856% 100.00000
sdn-sw.denv.net.internet2.edu 0 hr 2 min0 hr 0 min 99.98016%100.00000 99.97913% 100.00000
sdn-sw.elpa.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97951% 100.00000
sdn-sw.houh.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97894% 100.00000
sdn-sw.jack.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97970% 100.00000
sdn-sw.kans.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97875% 100.00000
sdn-sw.losa.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97913% 100.00000
sdn-sw.minn.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 100.00000% 100.00000
sdn-sw.newy32aoa.net.internet2.edu 0 hr 2 min0 hr 0 min 99.98016%100.00000 99.97932% 100.00000
sdn-sw.pitt.net.internet2.edu 0 hr 2 min0 hr 0 min 99.98016%100.00000 99.99962% 100.00000
sdn-sw.phoe.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99791% 100.00000
sdn-sw.port.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 100.00000% 100.00000
sdn-sw.rale.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.69756% 99.73664
sdn-sw.salt.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97894% 100.00000
sdn-sw.seat.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.99374% 100.00000
sdn-sw.star.net.internet2.edu 0 hr 7 min0 hr 0 min 99.93056%100.00000 99.97799% 100.00000
sdn-sw.sunn.net.internet2.edu 1 hr 3 min0 hr 0 min 99.37500%100.00000 99.96718% 100.00000
sdn-sw.tuls.net.internet2.edu 0 hr 3 min0 hr 0 min 99.97024%100.00000 99.97761% 100.00000
sdn-sw.wash.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000 99.97818% 100.00000

Aggregate Internet2 Advanced Layer 2 Service Core Nodes1 hr 45 min0 hr 0 min99.95660%100.00000%99.97292%99.98903%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Internet2 Advanced Layer 2 Service Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
I2-ASHB-PITT-100GE-07737 0 hr 2 min0 hr 0 min 99.98016%100.00000% 99.99962%100.00000%
I2-ATLA-CHIC-100GE-07740 0 hr 6 min0 hr 0 min 99.94048%100.00000% 99.51598%99.59244%
I2-ATLA-HOUH-10GE-07742 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.36058%99.36608%
I2-ATLA-JACK-100GE-07739 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.36058%99.36608%
I2-ATLA-WASH-100GE-07738 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.91329%99.91803%
I2-CHIC-KANS-100GE-07745 0 hr 2 min0 hr 0 min 99.98016%100.00000% 99.97173%100.00000%
I2-CHIC-STAR-100GE-07743 0 hr 10 min0 hr 1 min 99.90079%99.99008% 99.88521%99.89375%
I2-CLEV-NEWY32AOA-100GE-07735 0 hr 2 min0 hr 0 min 99.98016%100.00000% 99.57366%99.60819%
I2-CLEV-PITT-100GE-10334 0 hr 5 min0 hr 0 min 99.95040%100.00000% 99.99905%100.00000%
I2-CLEV-STAR-100GE-07736 0 hr 12 min0 hr 0 min 99.88095%100.00000% 99.95921%99.99658%
I2-DENV-KANS-100GE-07746 0 hr 1 min0 hr 1 min 99.99008%99.99008% 99.85428%99.90930%
I2-DENV-SALT-100GE-07747 0 hr 3 min0 hr 0 min 99.97024%100.00000% 99.96129%100.00000%
I2-ELPA-HOUH-100GE-07749 0 hr 6 min0 hr 0 min 99.94048%100.00000% 99.42775%99.49795%
I2-ELPA-LOSA-100GE-07750 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.87743%99.88369%
I2-HOUH-TULS-100GE-07751 0 hr 6 min0 hr 0 min 99.94048%100.00000% 99.76529%99.79356%
I2-KANS-TULS-100GE-07753 0 hr 3 min0 hr 0 min 99.97024%100.00000% 99.91462%99.91519%
I2-LOSA-SALT-100GE-07757 0 hr 6 min0 hr 0 min 99.94048%100.00000% 99.99412%99.99962%
I2-LOSA-SUNN-100GE-07755 1 hr 6 min0 hr 0 min 99.34524%100.00000% 99.88426%99.89735%
I2-NEWY23AOA-WASH-100GE-07759 0 hr 2 min0 hr 0 min 99.98016%100.00000% 99.98141%99.99336%
I2-PORT-SEAT-100GE-08997 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-PORT-SUNN-100GE-10343 1 hr 3 min0 hr 0 min 99.37500%100.00000% 99.98805%100.00000%
I2-SALT-SUNN-100GE-07766 1 hr 7 min0 hr 1 min 99.33532%99.99008% 99.88654%99.91158%

Aggregate All Internet2 Advanced Layer 2 Service Backbone Circuits 8 hr 41 min4 hr 22 min 99.88186%99.99865% 99.82154%99.84285%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.



Upcoming Scheduled Maintenances

Ticket No.:796:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit DENV-SALT Maintenance
Affected:Backbone Circuit DENV-SALT
Start Time:Wednesday, September 25, 2013, 6:00 AM (0600) UTC
End Time:Wednesday, September 25, 2013, 12:00 PM (1200) UTC
Description: Backbone Circuit DENV-SALT will be unavailable while Level(3) Communications fiber relocation maintenance in Broomfield, CO. An outage of 2 hours is expected; however the entire window is reserved. Backbone traffic will re-route during this maintenance.

Ticket No.:797:135
Subject:Internet2 Advanced Layer 2 Services Backbone Circuit DENV-SALT Maintenance Alternate Window
Affected:Backbone Circuit DENV-SALT
Start Time:Thursday, September 26, 2013, 6:00 AM (0600) UTC
End Time:Thursday, September 26, 2013, 12:00 PM (1200) UTC
Description: Backbone Circuit DENV-SALT will be unavailable while Level(3) Communications fiber relocation maintenance in Broomfield, CO. An outage of 2 hours is expected; however the entire window is reserved. Backbone traffic will re-route during this maintenance. This is an alternate window for 796:135 should it be required.