Skip to main content

If this job doesn't kill me, I might live till I'm old

        It’s been one of those days - again!



        It’s feeling a lot like ground hog day.



        A strange problem, corrupted our production AIX server        and we are restoring the OS from a mksysb (just the AIX naming for the root disk backup). I am hoping that the recovery will restore all the security information, which we were able to save from the old system. What exactly went wrong on it is still to be determined. Right now, it’s all hands on deck to get the system back.


        Plans were made to switch to the DR (Disaster Recovery) site but we are not sure that the other systems will be able to continue operating on this side of the great divide, while the core system is sitting in the DR computer room.


        First signs that things were not normal? There were none. A customer support agent came storming in waving arms wildly about his head, babbling about transactions not going through. When I investigated I found that I could not login (switch user) to the application users. It popped up an error that I’ve only ever seen when the system was installed the first time and you forgot to configure the five profile files to the correct values.


        Then it got worse. No one could establish new sessions to the system which is bad. All the automation is on an external system that launches it through the application users. So, the little problem suddenly got much larger.


        I started checking the system. Almost all the files in the /dev file system were deleted.


        Ah-ha.. that’s why I can’t SSH into it. Manually create the random device files and SSH out works, but not in. Great! Neither the memory, kmem or pty device files where there… then the Titanic feeling set in.. there’s no net0 driver, nor any of the disk and rhdisk pairs…


        That’s where we are now. Restoring another systems OS. Hoping the security information will transfer correctly from the backups we made and make the amount of work to recover less.


        Now, to get the rest of the banks systems to get their act together so we can fully switch components over to the DR site.


        Another late, late night… no golf (I’m really upset about that) and a ruined week… best laid plans and all.

Labels:

Comments

Post a Comment

Popular posts from this blog

Life's Luxuries, Good Food, Great Wine and Better conversation.

It’s amazing how much you see, but don’t see. You know what I mean. You notice something different. Be it in your house, at the office, with a friend, the kids or around town. You file it away for later reference and investigation, because invariably, at the time you notice it you are too busy rushing between the events in your life. Today is one of those days that follows after one of those days where you noticed something, which got filed for later investigation - for me, it’s the appearance of a wine cooler in the house. I noticed the little rectangular box with the blue light display some weeks ago. Filed it in the ‘to find out what it is, whose it is and why it’s here’ file and summarily forgot about it. Till a few minutes ago when I wanted to open some red wine. I buy wine from a merchant and have developed a taste for certain makers wines. An appreciation of the fine art that is wine making and the incredible bliss of paring good wine, with good food and great conversation. I wa
Post a Comment
Read more

Calitzdorp - Smoking Pipe

Today I had some time on my hands, and decided to check in on this website. I've bought a Afrika wood pipe from EbenHarts in 2008 and still smoke it. It's a heavy pipe. Solid in the hand. Not one you want to clamp down on though. Unless you like spending time in the dentist chair. It's with great joy and a little grump (hate the exchange rate) that I looked over the pipes on the site again. There are some seriously beautiful pipes here. Some, in fact, I don't think you'd want to smoke. I hope to soon see sold on all these pipes and a new batch of remarkable creations to sit and drool over. Sterkte met die pyp maak en ek hoop:  'hulle verkoop so lekker, want hulle rook so lekker.' cheap : 'via Blog this'
Post a Comment
Read more

10 Ways To Hack Into Your Happy Brain Chemicals

10 Ways To Hack Into Your Happy Brain Chemicals Life gets busy. 2016 for us in South Africa is going to be tough. Emotional black mail and black mail in general is a socially acceptable way to deny some their constitutional rights. So take a read above and stick it to all those office clones that make your day hell. All those wannabes on the roads that can't drive to safe their lives (it's been one of those days). So, to quote Bobby McFarren: "Don't worry. Be Happy."
Post a Comment
Read more