Skip to main content

If this job doesn't kill me, I might live till I'm old

        It’s been one of those days - again!



        It’s feeling a lot like ground hog day.



        A strange problem, corrupted our production AIX server        and we are restoring the OS from a mksysb (just the AIX naming for the root disk backup). I am hoping that the recovery will restore all the security information, which we were able to save from the old system. What exactly went wrong on it is still to be determined. Right now, it’s all hands on deck to get the system back.


        Plans were made to switch to the DR (Disaster Recovery) site but we are not sure that the other systems will be able to continue operating on this side of the great divide, while the core system is sitting in the DR computer room.


        First signs that things were not normal? There were none. A customer support agent came storming in waving arms wildly about his head, babbling about transactions not going through. When I investigated I found that I could not login (switch user) to the application users. It popped up an error that I’ve only ever seen when the system was installed the first time and you forgot to configure the five profile files to the correct values.


        Then it got worse. No one could establish new sessions to the system which is bad. All the automation is on an external system that launches it through the application users. So, the little problem suddenly got much larger.


        I started checking the system. Almost all the files in the /dev file system were deleted.


        Ah-ha.. that’s why I can’t SSH into it. Manually create the random device files and SSH out works, but not in. Great! Neither the memory, kmem or pty device files where there… then the Titanic feeling set in.. there’s no net0 driver, nor any of the disk and rhdisk pairs…


        That’s where we are now. Restoring another systems OS. Hoping the security information will transfer correctly from the backups we made and make the amount of work to recover less.


        Now, to get the rest of the banks systems to get their act together so we can fully switch components over to the DR site.


        Another late, late night… no golf (I’m really upset about that) and a ruined week… best laid plans and all.

Labels

Labels:

Comments

Post a Comment

Popular posts from this blog

There is a crack in everything. That’s how the light gets in.

As writers we forget that we are human first, writers second, and if your married, being a writer moves lower down on that list. Add children to this mix and you slide even lower on the priority list. Unless of course you are of the lucky few who actually earn a living by writing then you need to schedule writing in between all the other tasks that you have to attend to. In my attempts at becoming a better writer I’ve read a large volume of material, some of the pieces from great writers, others from writers in the making. Today I’ve read that Gore Vidal had died. Did I know who Gore Vidal was before today… nope. Does that make me a bad person, or worse, a bad writer? Should we not try and learn from the best to improve ourselves. I’d say yes, but since I’ve never read a line, correction, thanks to Writer’s Digest , several quotes from Mr Vidal, but also none of his actual books. I am saved from having to find some of his works since it seems Mr Vidal would have chosen a vocation ins...
Post a Comment
Read more