Internet2 Network Operations Report
07/30/2007 - 08/05/2007

Internet2 Network Operations Center
at Indiana University

New Customers

None

Upgrades

None

New Services

None



Unscheduled Outages Summary







Ticket Number
Customer Impact
Network Impact
Title
Outage Type
Start Time (UTC)

End Time (UTC)





105971-Critical
1-Critical

Internet2 IP Network Various Core Node(s) Outage Update
Software
08/02/2007 8:13 PM
08/02/2007 8:24 PM






Unscheduled Outages Detail






Ticket No.:10597:45



Subject:Internet2 IP Network Various Core Node(s) Outage Update



Affected:Various Internet2 IP Network Core Nodes



Start Time:Thursday, August 2, 2007, 8:13 PM (2013) UTC



End Time:Thursday, August 2, 2007, 8:24 PM (2024) UTC


Description:Internet2 engineers isolated the problem to an IS-IS
data collection process that triggered a JunOS routing
bug, causing the routing process daemon to reset on the
T640 core nodes. Engineers disabled the monitoring
process and the backbone stabilized. Juniper has
identified the bug and it has been corrected in future
releases. Internet2 engineers will be moving the
network to a version of code that doesn't contain the
bug. In the meantime, the monitoring process will
remain disabled.







Scheduled Maintenances Summary





Ticket Number
Customer Impact
Network Impact
Title

Maintenance Type
Start Time (UTC)
End Time (UTC)





10576
2-High
1-Critical

Internet2 IP Network STTL-SNVA Backbone Circuit Maintenance Completed
Circuit
07/31/2007 1:47 AM
07/31/2007 1:48 AM


07/31/2007 6:40 AM
07/31/2007 8:16 AM


07/31/2007 8:27 AM
07/31/2007 8:28 AM


10578
3-Elevated
2-High

Internet2 IP Network Backbone Circuit NEWY-WASH Maintenance Complete
Circuit
07/31/2007 7:09 AM
07/31/2007 7:19 AM




Scheduled Maintenances Detail





Ticket No.:10576:45



Subject:Internet2 IP Network STTL-SNVA Backbone Circuit Maintenance Completed



Affected:Backbone Circuit STTL-SNVA



Start Time:Tuesday, July 31, 2007, 1:47 AM (0147) UTC



End Time:Tuesday, July 31, 2007, 1:48 AM (0148) UTC


Start Time:Tuesday, July 31, 2007, 6:40 AM (0640) UTC


		    
End Time:Tuesday, July 31, 2007, 8:16 AM (0816) UTC


Start Time:Tuesday, July 31, 2007, 8:27 AM (0827) UTC


		    
End Time:Tuesday, July 31, 2007, 8:28 AM (0828) UTC


Description:   Backbone Circuit STTL-SNVA was unavailable.
Qwest Engineers were performing scheduled
maintenance.  This maintenance has been completed.



  






Ticket No.:10578:45



Subject:Internet2 IP Network Backbone Circuit NEWY-WASH Maintenance Complete



Affected:Backbone circuit NEWY-WASH



Start Time:Tuesday, July 31, 2007, 7:09 AM (0709) UTC



End Time:Tuesday, July 31, 2007, 7:19 AM (0719) UTC


Description:   Backbone circuit NEWY-WASH was briefly unavailable
to the Internet2 IP community while Level(3)
Communications performed emergency circuit maintenance
in Baltimore, MD. Maintenance has been completed.



  









Downtime and Availability







Internet2 Core NodesDown TimeReporting Period Availability52 Week Availability


** CHIC T6400 hr 0 min100.00000%100.00000%
DNVR T6400 hr 0 min100.00000%99.99962%
HSTN T6400 hr 11 min99.89087%99.99618%
LOSA T6400 hr 11 min99.89087%98.95700%
** NEWY T6400 hr 0 min100.00000%99.98760%
SNVA T6400 hr 11 min99.89087%99.99752%
STTL T6400 hr 11 min99.89087%99.99695%
** WASH T6400 hr 0 min100.00000%100.00000%
ATLA M50 hr 0 min100.00000%100.00000%
** ATLA T6400 hr 11 min99.89087%99.99790%
** SALT T6400 hr 0 min100.00000%100.00000%
** KANS T6400 hr 0 min100.00000%100.00000%
      
Aggregate Internet2 Core Nodes0 hr 55 min99.95453%99.91106%

** Less than 52 weeks of data available

Internet2 Backbone CircuitsDown TimeReporting Period Availability31 Week Availability




DNVR-SNVA OC-1920 hr 0 min100.00000%100.00000%
DENV-STTL OC-1920 hr 0 min100.00000%99.96352%
HSTN-LOSA OC-1920 hr 0 min100.00000%99.97760%
LOSA-SNVA OC-1920 hr 0 min100.00000%100.00000%
ATLA-HSTN OC-1920 hr 0 min100.00000%99.52605%
SNVA-STTL OC-1921 hr 38 min99.02778%99.75614%
** I2-CHIC-ATLA0 hr 0 min100.00000%99.98464%
** I2-CHIC-NEWY0 hr 0 min100.00000%99.77279%
** I2-CHIC-KANS0 hr 0 min100.00000%100.00000%
** I2-CHIC-WASH0 hr 0 min100.00000%100.00000%
** I2-NEWY-WASH0 hr 10 min99.90079%99.99648%
** I2-WASH-ATLA0 hr 0 min100.00000%100.00000%
** I2-ATLA-HOUS0 hr 0 min100.00000%99.90527%
** I2-KANS-ATLA0 hr 0 min100.00000%99.72510%
** I2-KANS-SALT0 hr 0 min100.00000%100.00000%
** I2-SALT-SEAT0 hr 0 min100.00000%100.00000%
      
Aggregate All Internet2 Backbone Circuits1 hr 48 min99.93304%99.91297%






Upcoming Scheduled Maintenances





Ticket No.:10566:45



Subject:Internet2 IP Network Backbone Circuit ATLA-WASH Maintenance (second window)



Affected:I2-ATLA-WASH-I2-00097
I2-ATLA-WASH-ESNET-00008
I2-ATLA-WASH-ESNET-00009



Start Time:Thursday, August 9, 2007, 4:00 AM (0400) UTC



End Time:Thursday, August 9, 2007, 10:00 AM (1000) UTC



Description:   Internet2 IP Network backbone circuit ATLA-WASH will be
unavailable to the Internet2 community as Level (3)
Communications technicians perform a fiber
maintenance from Richmond VA, to Dahlgren, VA.  Traffic
will re-route