Are your backup systems robust enough?

My previous blog talks about the problems getting Windows 7 workstations to attach to a Windows 2003 domain server. In that blog we talked a problem we had with our domain server. This blog is to add more detail to those comments.
Our server failed unexpectedly. No warning signs, just a shutdown of access for workstations. Believeing this to be one of thos unusual things we reset the server. After 10 minutes we had no service and after attaching a monitor we found that the server had not completed POST and BIOS setup.
Several attempts met with similar failure.
We rebuilt the server but when we attached the disks from the old server to the new one to copy off the data we could not access any data.
The last time we had problems with this disk system we had set it up without the mirrored RAID so one level of protection was missing.
However we do have an external hard disk which was set up to do a complete backup of the data EVERY night. No problem then. Attach the external disk. We see a perfectly good looking .bkf file but when we tried to recover it with the Windows Backup & Recovery tool it told us there was no recoverable data on the disk.
Never mind, we still have our online, in the Cloud, backup system. We have not stored everything but the major items are still up there. We download the backup client software to do a restore. It tells us that we have used 2GB of our 3GB allowance but wait.... When we try to recover the files it says there are no files selected for recovery or that there is no data to collect. The help desk lived up to the antithesis of their name and I still have no satisfactory answer from them.
From this it appears that we have lost all our Client files from the last 6 years, all the accounts data, all the management files (policy documents, logos, letter templates, contract templates etc), all our supplier files, and all the downloaded data (device drivers and software used to build client machines).
We have called in the services of a Disk specialist who has been able to recover most of our data but before you say 'What is all the fuss for' then consider this.
We limped along on Friday. We paid overtime on Saturday to rebuild the server (luckily we did not have to pay the cost of new hardware as we could use some kit that was redundant). We have not been able to work on Monday, Tuesday or Wednesday.
This doesn't affect many of us as there are only 3 of us in the organisation but how would it affect you?
What is the cost to you for paying someone to replace and rebuild your computer system?
What is the cost to have your staff being idle for 3 days?
What is the cost to you not being able to fulfill client orders?
Add to that the cost of calling in a specialist to attempt to recover you data.
This is not cheap, It could easily run into 5 figures. Put into perspective what it would cost to have a completely robust backup and recovery system in place.
This has happened to us. We might have got through it because we had second and third line backup systems in place. Don't let it happen to you. Having gone through all this we are now better placed to help you avoid the pitfalls that we have fallen into.
Call us!

The security database on the server does not have a computer account for this workstation trust relationship.

We recently has this error message appear when trying to log on Windows 7 workstations to our Windows 2003 domain server.
"The security database on the server does not have a computer account for this workstation trust relationship."
We trawled through the Google searches for this message and there are pages of reports of this error.
Let me give you the circumstances.
Our domain server failed. We could not get it to get through POST and BIOS to try to find and operating system. That was the first big 'Ouch'.
So we rebuilt the server on to new hardware and configured it, as we thought, the same. We initially had some issues with DNS and NICs not doing as they should but the upshot was that our 2 Windows 7 workstations could not connect to the domain. Please note this did not affect our remaining XP workstation which logged just fine.
After looking at the Goggle responses we noticed that we had rebuilt the domain but missed off the '.local' on the domain name.
So we rebuilt being more careful next time. Even so we got a capital letter in the wrong place in the domain name. So we rebuilt a third time. This time we are sure we got everything right. It made not a jot of difference.
Other websites had suggested we disconnect from the domain and reconnect to the new domain. Not an iota of difference.
Finally we had to reinstall the operating system on both our Windows 7 machines.
One side effect of this, despite having installed onto completely new hard disks to preserve the data on the old disk, is that on one of the machines we have lost ALL out Outlook pst files. Some emails going back 6 years with important information.
Couple this with the total data loss of all information on the server did not make us very happy.
I will finish with this comment. Why, oh why, Microsoft is it so hard to get your systems to talk to each other. This is hardly a ringing endorsement for the improvements in Windows 7 when XP performs better in what was already a very fraught situation. Please try to do better next time.