Unified - Internet2 IP Network Operations Report
03/09/2015 - 03/15/2015

Unified - Internet2 IP Network Operations Center
at Indiana University

New Connections

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Source Of Impact Start Time (UTC) End Time (UTC)

29524-Normal Brief Outage - I2 AL2S Backbone CHIC-CINC UndeterminedVendor 03/09/2015 12:09 AM 03/09/2015 12:10 AM
29684-Normal Availability - I2 AL2S Various Services SoftwareInternal 03/10/2015 8:55 AM 03/10/2015 9:11 AM
30653-Elevated 3-Elevated Brief Outage - I2 Optical Various Circuits InvestigatingVendor 03/14/2015 5:15 PM 03/14/2015 5:16 PM

Unscheduled Outages Detail


Ticket No.:2952:144
Subject:Brief Outage - I2 AL2S Backbone CHIC-CINC
Source Of Impact:Vendor
Affected:AL2S Backbone CHIC-CINC
IP Backbone ATLA-CHIC
Start Time:Monday, March 9, 2015, 12:09 AM (0009) UTC
End Time:Monday, March 9, 2015, 12:10 AM (0010) UTC
Description:The above listed was briefly unavailable. Level(3) Engineers have reviewed the issue and have not been able to determine the cause. The reason for the outage is undetermined.

Ticket No.:2968:144
Subject:Availability - I2 AL2S Various Services
Source Of Impact:Internal
Affected:AL2S OESS Controller
IP Backbone NEWY32AOA-WASH
IP Backbone ATLA-NEWY32AOA
IP Backbone ATLA-CHIC
IP Backbone HOUS-LOSA
CAAREN (NEWY)
UPENN (NEWY)
CENIT (NEWY)
MAGPI (NEWY)
Caribnet/CKLN (NEWY)
UPENN (WASH)
NYSERNet (WASH)
MAGPI (WASH)
CENIC (CalREN-HPR) (HOUS)
Sun Corridor (HOUS)
CENIC (CHIC)
Start Time:Tuesday, March 10, 2015, 8:55 AM (0855) UTC
End Time:Tuesday, March 10, 2015, 9:11 AM (0911) UTC
Description:The above listed assets are available after an outage. Internet2 Engineers are reviewing the issue for a root cause. No AL2S backbones were down, so no AL2S re-route advisements were distributed. IP Backbone traffic re-routed. No further outages are expected. Updates will be provided as they are developed.

Ticket No.:3065:144
Subject:Brief Outage - I2 Optical Various Circuits
Source Of Impact:Vendor
Affected:AL2S Backbone CHIC-CINC
IP Backbone ATLA-CHIC
Optical Member Indiana GigaPOP
Optical Member NOAA N-Wave
Start Time:Saturday, March 14, 2015, 5:15 PM (1715) UTC
End Time:Saturday, March 14, 2015, 5:16 PM (1716) UTC
Description:The above listed circuits were briefly unavailable to the community. Level (3) Communications Engineers continue to investigate the issue. All affected circuits are now available.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Source Of Impact Start Time (UTC) End Time (UTC)

2692 3-Elevated 1-Critical Maintenance Completed 1 of 2 - I2 Optical DWDM System LASV-LOSA Circuit Vendor03/10/2015 9:13 AM 03/10/2015 9:38 AM

Scheduled Maintenances Detail

Ticket No.:2692:144
Subject:Maintenance Completed 1 of 2 - I2 Optical DWDM System LASV-LOSA
Source Of Impact:Vendor
Affected:DWDM LASV-LOSA
AL2S Backbone LOSA-SALT
IP Backbone LOSA-SALT
Start Time:Tuesday, March 10, 2015, 9:13 AM (0913) UTC
End Time:Tuesday, March 10, 2015, 9:38 AM (0938) UTC
Description: DWDM System LASV-LOSA was unavailable while Level(3) Communications relocated cable. Backbone traffic rerouted during this maintenance. Workgroup administrators received notifications from OESS regarding circuits re-routed on the backbone. Maintenance has been completed.


Downtime and Availability



Unified - Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability
ATLA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CHIC MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
CLEV MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
HOUS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
KANS MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
LOSA MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
NEWY MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SALT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
SEAT MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
WASH MX960 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw0.sing1.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%
rtsw.sing2.net.internet2.edu 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000% 100.00000%

Aggregate Unified - Internet2 IP Core Nodes0 hr 0 min0 hr 0 min100.00000%100.00000%100.00000%100.00000%

NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.

Unified - Internet2 IP Backbone CircuitsDown Time Reporting Period Availability52 Week Availability
I2-ATLA-CLEV-VLAN-14832 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-CLEV-VLAN-14837 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14833 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-ATLA-NEWY32AOA-VLAN-14838 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14831 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
I2-CHIC-NEWY32AOA-VLAN-14834 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-CHIC-VLAN-09181 0 hr 18 min0 hr 18 min 99.82143%99.82143% 99.65126%99.93682%
R&E 100G Backbone: I2-ATLA-HOUS-VLAN-09182 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.60876%99.99602%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92676%99.99696%
R&E 100G Backbone: I2-CHIC-CLEV-VLAN-09180 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94004%99.99924%
R&E 100G Backbone: I2-CHIC-KANS-VLAN-09183 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96926%99.99905%
R&E 100G Backbone: I2-CLEV-NEWY32AOA-VLAN-09177 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94213%99.99469%
R&E 100G Backbone: I2-CLEV-WASH-VLAN-09178 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96319%99.99336%
R&E 100G Backbone: I2-HOUS-KANS-VLAN-09184 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.90627%99.99677%
R&E 100G Backbone: I2-HOUS-LOSA-VLAN-09189 0 hr 16 min0 hr 16 min 99.84127%99.84127% 99.62621%99.75144%
R&E 100G Backbone: I2-KANS-SALT-VLAN-09185 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.89507%99.98975%
R&E 10G Backbone: I2-LOSA-PAIX-VLAN-23105 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Commodity 10G Backbone: I2-LOSA-PAIX-VLAN-23106 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-LOSA-SALT-VLAN-09188 0 hr 25 min0 hr 0 min 99.75198%100.00000% 99.98330%99.99905%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09187 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97666%99.99943%
R&E 100G Backbone: I2-LOSA-SEAT-VLAN-09328 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97666%99.99943%
R&E 100G Backbone: I2-LOSA-WILC-VLAN-23103 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Commodity 100G Backbone: I2-LOSA-WILC-VLAN-23104 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-NEWY32AOA-WASH-VLAN-09176 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.98216%99.99734%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09186 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99677%99.99981%
R&E 100G Backbone: I2-SALT-SEAT-VLAN-09327 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99677%99.99981%
R&E 100G Backbone: I2-WASH-ATLA-VLAN-09179 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
R&E 100G Backbone: I2-ATLA-WASH-VLAN-09318 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92676%99.99696%
Legacy 10G Backbone: I2-ATLA-CHIC-10GE-05419 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.73399%99.97287%
Legacy 10G Backbone: I2-ATLA-HOUS-10GE-05423 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.20993%99.56208%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-05251 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93871%100.00000%
Legacy 10G Backbone: I2-ATLA-WASH-10GE-09142 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.93739%99.99867%
Legacy 10G Backbone: I2-CHIC-CLEV-10GE-08891 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.95807%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05781 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-CHIC-KANS-10GE-05976 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%
Legacy 10G Backbone: I2-CLEV-NEWY32AOA-10GE-08894 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.96376%99.99962%
Legacy 10G Backbone: I2-CLEV-WASH-10GE-08889 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99867%99.99867%
Legacy 10G Backbone: I2-HOUS-KANS-10GE-05560 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.94346%99.99753%
Legacy 10G Backbone: I2-HOUS-KANS-LAG-08934 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-HOUS-LOSA-10GE-05559 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.75789%99.82772%
Legacy 10G Backbone: I2-KANS-SALT-10GE-05566 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92505%99.99924%
Legacy 10G Backbone: I2-KANS-SALT-LAG-08935 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.92505%99.99924%
Legacy 10G Backbone: I2-LOSA-PAIX-10GE-05853 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99621%100.00000%
Legacy 10G Backbone: I2-LOSA-SALT-10GE-05563 0 hr 25 min0 hr 0 min 99.75198%100.00000% 99.99013%100.00000%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05564 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.97837%99.99924%
Legacy 10G Backbone: I2-LOSA-SEAT-10GE-05572 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99431%100.00000%
Legacy 100G Backbone: I2-LOSA-WILC-100GE-23893 0 hr 0 min0 hr 0 min 100.00000%100.00000% 100.00000%100.00000%
Legacy 10G Backbone: I2-NEWY32AOA-WASH-10GE-07612 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99696%99.99810%
Legacy 10G Backbone: I2-SALT-SEAT-10GE-05565 0 hr 0 min0 hr 0 min 100.00000%100.00000% 99.99981%99.99981%

Aggregate All Unified - Internet2 IP Backbone Circuits 1 hr 24 min0 hr 50 min 99.98299%99.99312% 99.92889%99.97956%


NOTE: Numbers in italics represent downtime with scheduled maintenance excluded. Internet2's definition of scheduled maintenance is any anticipated outage announced at least 72 hours in advance of the outage start time.



Upcoming Scheduled Maintenances

Ticket No.:2998:144
Subject:Maintenance - I2 AL2S Core Node Insertion INDI
Affected:AL2S Core Node INDI
AL2S Backbone CHIC-CINC
IP Backbone ATLA-CHIC
Start Time:Friday, March 20, 2015, 5:00 AM (0500) UTC
End Time:Friday, March 20, 2015, 7:00 AM (0700) UTC
Description: The above listed assets will be unavailable while Internet2 Engineers perform insertion maintenance. Internet2 Engineers will be inserting the AL2S Core Node INDI between CHIC and CINC. OESS administrators will receive notifications of circuits re-routing. An outage of forty-five minutes is expected, however the entire window is reserved.

Ticket No.:2447:144
Subject:Maintenance - I2 Optical Various Circuits (Northern Tier)
Affected:AL2S Backbone MINN-MISS2
Net+ Provider Code42 (CHIC)
I2 Optical Member CA*Net/Canarie
Start Time:Saturday, March 21, 2015, 2:00 PM (1400) UTC
End Time:Saturday, March 21, 2015, 7:00 PM (1900) UTC
Description: The above listed circuits will be unavailable while Zayo Engineers perform maintenance. An outage of one hour is expected. The entire window is reserved. *Net+ Provider Code42 (CHIC) is multihomed and will remain reachable.*

Ticket No.:2875:144
Subject:Maintenance 1 of 2 - AL2S Backbone Circuits (Northern Tier)
Affected:AL2S Backbone EQCH-MINN

Net+ Provider Code42 (CHIC)
Start Time:Sunday, March 22, 2015, 5:00 AM (0500) UTC
End Time:Sunday, March 22, 2015, 10:00 AM (1000) UTC
Description: The above listed assets will be unavailable while Zayo Engineers perform maintenance. An outage of up to 3 hours is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone. *denotes multihomed service and will remain available

Ticket No.:2876:144
Subject:Maintenance 2 of 2 - I2 Optical Various Circuits (Northern Tier)
Affected:AL2S Backbone EQCH-MINN
Net+ Provider Code42 (CHIC)
Start Time:Monday, March 23, 2015, 5:00 AM (0500) UTC
End Time:Monday, March 23, 2015, 10:00 AM (1000) UTC
Description: This is 2 of 2. The above listed assets will be unavailable while Zayo Engineers perform maintenance. An outage of up to 3 hours is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone. This is back up maintenance window to be used, if necessary. *denotes multihomed service and will remain available

Ticket No.:3099:144
Subject:Maintenance - I2 IP Backbone CHIC-KANS
Affected:Backbone CHIC-KANS
Start Time:Tuesday, March 24, 2015, 5:00 AM (0500) UTC
End Time:Tuesday, March 24, 2015, 8:00 AM (0800) UTC
Description: The above assets will be unavailable while Internet2 Engineers, with the assistance of Level3 Technicians, perform maintenance to swap fiber on the AL2S to IP interconnect in Kansas City. An outage is expected. The entire window is reserved. IP traffic will re- route during this maintenance.

Ticket No.:3121:144
Subject:Maintenance - I2 Optical Circuits I2-COLU4-KANS-100GE-11579 and I2-COLU4-SAIN-100GE-11580
Affected:AL2S Backbone CHIC-COLU4
AL2S Backbone COLU4-KANS
Start Time:Tuesday, March 24, 2015, 6:00 AM (0600) UTC
End Time:Tuesday, March 24, 2015, 10:00 AM (1000) UTC
Description: The above listed will be unavailable while MOREnet Engineers perform maintenance. An outage of 5 hours is expected. The entire window is reserved. Backbone traffic will re-route during this maintenance. Workgroup administrators will receive notifications from OESS regarding circuits re-routed on the backbone.

Ticket No.:2978:144
Subject:Maintenance - I2 AL2S Connector CAAREN
Affected:Connector CAAREN and its Participants
Start Time:Tuesday, April 7, 2015, 4:00 AM (0400) UTC
End Time:Tuesday, April 7, 2015, 5:00 AM (0500) UTC
Description: The above listed will be unavailable while Internet2 and CAAREN Engineers migrate VLANs carrying CAAREN peerings. A brief outage is expected; however, the entire maintenance window is reserved.