General Consultant Discussion

My story for the day...... Last night I was about to install a server pack on a MAS90 system.. So following standard procedure, I started to make a complete copy of the MAS90 folder.... about 3 minutes into the copy the server locks up and the remote session closed. I attempt to reconnect and it doesn't find it. I asked the customer to look at the server and Uh oh ...Blue memory dump screen on the server. So next the customer attempts to reboot the server and he receives the ""No O/S present."" massage. Immediately the he calls their IT guy. The IT guy arrives at 7am bright and early this morning and gets to work. About an hour later he then tell that client that Hard disk is toast and then drops ""the bomb"" that the night backup failed and worse yet has been failing for the past 1 week. Now to add insult to injury, the client has been using paperless office so most of the journals aren't printed and the problem is that those PDF's were on the crashed drive....... Not a good start to the day. Luckily, The IT guy has been able to run some utilities on the disk to recover some of the files. Most of the PDF files have been recovered but his hasn't been so lucky recovering the MAS files. The latest is that they restored the week old MAS onto a different server and are starting the process of entering a week's worth of data. Definitely not a get way to start a month!!! OUCH!!! This brings up a whole bunch of things to think about. Just to name a few... 1) There needs to be a better way for a client to know when their their system backup has failed. 2) Should you store the PDF's on the same disk as the MAS folder.

UPDATE: IT guy ran another set of utilities and is finding MAS files, there may be hope!!

Good luck!

Answer to 1): With current technology and best practices, there really should not be an absolute failed backup or lack of notification. I would go so far as to say the IT guy is simply irresponsible. Using NAS appliances and including redundant Internet backup methods, such as Mozy, should provide sufficient backup with minimal failure, unless there is a physical issue or extreme power outage. There should also be some type of automated monitoring tool in place, such as Level Platforms, which provides 24/7 coverage and notification of exceptions. These are the current best practices. None of the above are overly expensive, especially when comparing the cost to loss of critical business data.

I was on an Accpac customer's system (main company database is 40GB+) recently and found that the drive where the SQL backups were going had filled up a MONTH ago. Fortunately no hardware failures happened. And for those of you who think hosting will get you away from this kind of stupid IT trick, this was on a HOSTED server. A better system at another customer makes a full backup nightly, then hourly incremental backups all day. These backups go first to an external drive and are then (almost immediately - too quickly one evening when I made a backup to try something Sage support recommended that I had doubts about and was right) moved to offsite storage.

Should be selling all clients cloud backup in addition to their regular backup

I realize that this may be old school thinking but aren't mirrored or RAID drives part of server configurations these days?

Yikes, Jim! Good luck 'cuz you know you will be helping them unravel everything...

Just one of those occasions where being on an hourly billing arrangement may make sense......

1. Agree on selling cloud backup as a secondary. What are people using? 2. I have often thought as part of a ""Gold"" plan that we could monthly or weekly copy off data to a USB drive. These things are dirt cheap. Leave drive with customer. Could even be a free included unannounced service. 3. Countdown to Jim being accused of frying the drive (under last to touch it broke it theory) - 3,2,1

Oh and I've seen all of these fail mostly due to nobody checking them until shit hits fan.

@JeffSchwenk With respect to hourly billing being better = not necessarily. Over the summer my price was about $20,000 to become involved in a system restore. The customer needed an emergency ""access at a moment's notice"" response. Hourly would have been far less because I would have had a hard time computing a special rate on a moment's notice. And I did not include sitting at their desk doing restoration -- that's a whole separate engagement if they want to pay me to do keypunch. I merely advised their staff on the re-keying.

Having a cloud backup reminds me of the T-shirt: ""186,000 miles per second isn't just a good idea . . . it's the law!"" In the case of cloud backup, it should be. Also there are two kinds of people in the world - those who have had a hard drive failure, and those who will. Even if our group is not maintaining the client's network, I inform them and their IT support group of best practices. And, yes, there are many IT support people who want to disagree with us.

I too get the feeling the IT support doesn't have the client's well being in mind. I cringe when I find out about some practices. But then again, management doesn't seem to appreciate the need for ""insurance""..............

Back to the question posed by @WayneSchulz: What are people using for offsite backup? We have a few clients on Carbonite, and I've seen Mozy on a few as well. A lot of clients are still doing tape backups because there just isn't enough bandwidth in some places to get everything off site. I use JungleDisk at home (based on Amazon S3), and I've been very happy with their service. Are there others?