Internet2 IP Network Operations Report
01/07/2008 - 01/13/2008

Internet2 IP Network Operations Center
at Indiana University

New Customers

None


Upgrades

None


New Services

None


Unscheduled Outages Summary

Ticket Number Customer Impact Network Impact Title Outage Type Start Time (UTC) End Time (UTC)

115241-Critical 2-High Internet2 IP Network Backbone Circuit CHIC-KANS Available Hardware 01/07/2008 3:26 AM 01/07/2008 3:51 PM
115252-High 1-Critical Internet2 IP Network Core Node KANS Outage Hardware 01/07/2008 3:26 AM 01/07/2008 3:28 AM
115422-High 2-High Internet2 IP Network Backbone Circuit NEWY-WASH Outage Resolved Circuit - Damaged Fiber 01/09/2008 6:47 PM 01/09/2008 10:19 PM
01/09/2008 10:29 PM 01/09/2008 10:29 PM

Unscheduled Outages Detail


Ticket No.:11524:45
Subject:Internet2 IP Network Backbone Circuit CHIC-KANS Available
Affected:Backbone Circuit CHIC-KANS
Start Time:Monday, January 7, 2008, 3:26 AM (0326) UTC
End Time:Monday, January 7, 2008, 3:51 PM (1551) UTC
Description:Internet2 IP Network backbone circuit CHIC-KANS was unavailable to the community. Level(3) Communications has determined the outage was due to a card failure in Bloomington, IL. A new card was installed at 1100 EST (1500 UTC). Traffic re-routed.

Ticket No.:11525:45
Subject:Internet2 IP Network Core Node KANS Outage
Affected:Core Node KANS
Start Time:Monday, January 7, 2008, 3:26 AM (0326) UTC
End Time:Monday, January 7, 2008, 3:28 AM (0328) UTC
Description:Internet2 IP Network Core Router KANS RE crashed due to circuit related issue within Level3.

Ticket No.:11542:45
Subject:Internet2 IP Network Backbone Circuit NEWY-WASH Outage Resolved
Affected:I2 IP Network Backbone Circuit NEWY-WASH
I2 Grooming Backbone Circuit PHIL-NEWY
ESNet IP Backbone Circuit NEWY-WASH
ESNet SDN Backbone Circuit NEWY-WASH
HOPI Backbone Circuit NEWY-WASH
Connector MAGPI

Start Time:Wednesday, January 9, 2008, 6:47 PM (1847) UTC
End Time:Wednesday, January 9, 2008, 10:19 PM (2219) UTC
Start Time:Wednesday, January 9, 2008, 10:29 PM (2229) UTC
End Time:Wednesday, January 9, 2008, 10:29 PM (2229) UTC
Description:Internet2 IP Network Backbone Circuit NEWY-WASH was unavailable. Level(3) reports that this was due to an aerial fiber cut between Princeton, NJ and Newark, NJ. The damaged fiber has been spliced and service has been restored.

Scheduled Maintenances Summary

Ticket Number Customer Impact Network Impact Title Maintenance Type Start Time (UTC) End Time (UTC)

11526 3-Elevated 1-Critical Internet2 IP Network Core Node KANS Emergency Maintenance Completed Software 01/08/2008 6:01 AM 01/08/2008 6:03 AM
11539 3-Elevated 2-High Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Emergency Maintenance Completed Circuit 01/08/2008 11:38 PM 01/08/2008 11:39 PM
01/09/2008 7:32 AM 01/09/2008 7:39 AM

Scheduled Maintenances Detail

Ticket No.:11526:45
Subject:Internet2 IP Network Core Node KANS Emergency Maintenance Completed
Affected:Core Node KANS
Start Time:Tuesday, January 8, 2008, 6:01 AM (0601) UTC
End Time:Tuesday, January 8, 2008, 6:03 AM (0603) UTC
Description: Internet2 IP Network Core Node KANS was briefly unavailable as Internet2 Engineers performed an emergency software maintenance. Maintenance has been completed. * Traffic re-routed.

Ticket No.:11539:45
Subject:Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT Emergency Maintenance Completed
Affected:Internet2 IP Network Backbone Circuit LOSA-SALT
Internet2 IP Network Backbone Circuit LOSA-SEAT
IXP PAIX
Connector Oregon GigaPoP
Peer ESNET
ELPA-SUNN Ciena Backbone
LOSA-SUNN Ciena Backbone
Start Time:Tuesday, January 8, 2008, 11:38 PM (2338) UTC
End Time:Tuesday, January 8, 2008, 11:39 PM (2339) UTC
Start Time:Wednesday, January 9, 2008, 7:32 AM (0732) UTC
End Time:Wednesday, January 9, 2008, 7:39 AM (0739) UTC
Description: Internet2 IP Network Backbone Circuits LOSA-SALT and LOSA-SEAT and several customer connections (listed above) underwent emergency maintenance window while Level(3) Communications technicians performed circuit maintenance on an OAM in Santa Barbara, CA. Card has been replaced and maintenance has been completed.


Downtime and Availability



Internet2 IP Core NodesDown TimeReporting Period Availability52 Week Availability
** CHIC T6400 hr 0 min100.00000%100.00000%
** NEWY T6400 hr 0 min100.00000%99.99627%
** SEAT T6400 hr 0 min100.00000%99.99851%
** WASH T6400 hr 0 min100.00000%100.00000%
** HOUS T6400 hr 0 min100.00000%99.99888%
** LOSA T6400 hr 0 min100.00000%99.99925%
** ATLA T6400 hr 0 min100.00000%99.99589%
** SALT T6400 hr 0 min100.00000%99.99832%
** KANS T6400 hr 4 min99.96032%99.99739%

Aggregate Internet2 IP Core Nodes0 hr 4 min99.99559%99.99828%

** Less than 52 weeks of data available

Internet2 IP Backbone CircuitsDown TimeReporting Period Availability53 Week Availability
** I2-CHIC-ATLA0 hr 0 min100.00000%99.32908%
** I2-CHIC-NEWY0 hr 0 min100.00000%99.50941%
** I2-CHIC-KANS12 hr 25 min92.60913%99.85663%
** I2-CHIC-WASH0 hr 0 min100.00000%99.87287%
** I2-NEWY-WASH3 hr 32 min97.89683%99.94269%
** I2-WASH-ATLA0 hr 0 min100.00000%99.98096%
** I2-ATLA-HOUS0 hr 0 min100.00000%99.42876%
** I2-KANS-ATLA0 hr 0 min100.00000%99.83964%
** I2-KANS-SALT0 hr 0 min100.00000%100.00000%
** I2-SALT-SEAT0 hr 0 min100.00000%100.00000%
** I2-HOUS-LOSA0 hr 0 min100.00000%99.96584%
** I2-LOSA-SALT0 hr 8 min99.92063%99.92290%
** I2-LOSA-SEAT0 hr 8 min99.92063%99.99701%
** I2-KANS-HOUS0 hr 0 min100.00000%99.97965%

Aggregate All Internet2 IP Backbone Circuits16 hr 13 min99.31052%99.83039%





Upcoming Scheduled Maintenances

Ticket No.:11544:45
Subject:Internet2 IP Network Backbone Circuit LOSA-SALT Maintenance
Affected:I2 IP Backbone Circuit LOSA-SALT
I2 Grooming Backbone Circuit SALT-SUNN
ESNet Backbone Circuit DENV-SUNN
Start Time:Tuesday, January 29, 2008, 8:00 AM (0800) UTC
End Time:Tuesday, January 29, 2008, 2:00 PM (1400) UTC
Description: Level(3) will be re-splicing broken fibers, and then rolling fibers to permanent assignments in Sacramento, CA.