Every once in awhile I am asked where I get the topics for my blog. This one came to me in the form of an email. The message, from one of our customer agencies, reminded me of how well we have embraced our mission statement at the OCIO, Respect for the Taxpayers of the State of Nebraska.
Hardware and its related components are predisposed to fail, hence you have to plan before that failure occurs. The OCIO decided after consolidation to eliminate its disaster recovery site, which was essentially a secure copy of our data backed up at an offsite location. Primarily we did this in favor of two active data centers to focus on the availability of applications for our customers and our citizens. I wrote several months ago about Plan to Fail(over) and a few days ago we had another chance to test our plan.
What happened was this, one of our midrange servers in the Lincoln Data Center crashed which hosts all the applications that support the Courts located in Counties across Nebraska. Having two active data centers, we were prepared to respond to the incident rapidly and efficiently. We had a plan in place to react regardless of which system was having an impact. No time was wasted debating whether we can or should failover. We simply followed the plan. This could be better explained by our customer’s emails, received from the Office of the Courts and Probation.
Good morning,
I just wanted to send a note to compliment the OCIO midrange team on their response to the AS/400 server outage yesterday. Kevin Smith’s communication with us and assistance throughout the day was extremely helpful, and the process to failover to the secondary data center was very smooth with relatively few issues. I also know that Donna Ruhl went above and beyond by working late into the night to ensure not only that the JUSTICE program was online, but that we still were able to complete our backups. I am sure there were other teams involved in the recovery efforts, so our thanks go to them as well. The minimal downtime we experienced yesterday was only possible because of the existence of the replicated backup server in Omaha. Because of the OCIO’s disaster recovery measures, and the dedication of the midrange team, the judicial branch was able to function nearly as per normal yesterday despite a major hardware failure.
Thank you,
Jennifer Rasmussen
The OCIO has a different process in place to help us determine which applications and services are most critical to the State of Nebraska. For guidance to manage this process, we reference our Application Portfolio, which I discussed in yet another blog. We continue to evaluate all of the State’s applications being used throughout the agencies and identify those critical to business operations. These applications get replicated across our Lincoln and Omaha Data Centers. Our diligence to the Application Portfolio Management project has a purpose, to ensure that critical applications have the ability to failover to a secondary location and so that this type of operation results in zero data loss.
The OCIO’s team efforts to lead projects that support our mission have enhanced the customer service we are capable of now of providing. Our confidence in our disaster recovery strategy continues to grow because of the support that you provide. As always I appreciate the work you do each day for the citizens of Nebraska.
Ed Toner