Hard drive needs to be replaced

[veronicabend]

Hi,

The hosting company is telling us they need to replace the hard drive. But they will not move the current server data along. They say we need to take care of backup and restore.

The server has over 100 domains in there. Each cPanel account has mailboxes, databases, files, cron jobs, and so on. Over time, there have been some additions installed into it like FDFToolkit for Php and Image Magick, to name 2 examples.

We don't have experience in backing up an entire disk to load the data back on the new drive. Can anyone who has done this before help us? What should we do to be able to restore the server as it is now the soonest?

We don't have physical access to the server, just remote via SSH.

Thanks for any help on this matter.

Veronica

 

[chirpy]

That's extremely unreasonable of your host and I'd suggest that once you're beyond this crisis that you look for a more customer responsive datacenter, of which there are many.

Presumably, you have some other disk you can backup to? If so, then you can perform a cPanel full backup by configuring it in WHM and then running:

/scripts/cpbackup

And then use the resultant backup to restore from.

If you don't have a separate drive to backup to, then you'll probably still have to do the full backup and store the results (usually in /backup/cpbackup/daily/*) off-server using FTP. Then FTP them backup to the same equivalent place and restore from the backups on the new drive.

 

[veronicabend]

Hi chirpy,
Thanks for your response.

We don't have another drive where to backup. Maybe I can talk to the hosting company about adding another drive to run the backup? I'm not sure if this will be possible.

About backing up all to the drive and download with FTP, I just tried doing tarball of some public_html directories. One of them is 1GB, another one is over 600 MB and there are much more. I don't think the server will even have physical space to make a full backup to itself. And then how will I be able to download it with FTP?

I am very disappointed with the hosting company, I don't think they should leave us on our own with the data. They say they don't copy or move data because if liability issues. But this needs someone experienced to move this data. They have been very good so far and I didn't expect this.

This WHM backup you mention will just move over the cpanel accounts, right? Any ideas about how to proceed with other installed apps?

Thanks for your help.

 

[chirpy]

The WHM full backup creates tarballs (accountname.tar.gz files) which contains everything cPanel needs to restore the account in its entirety, including all the account data and files. It's the tarballs you would then FTP off the server (or copy to a spare drive) and then restore through the WHM restore functions.

 

[AndyReed]

We don't have another drive where to backup. Maybe I can talk to the hosting company about adding another drive to run the backup? I'm not sure if this will be possible.

Yes, it is possible to add a 2nd HD. We have done with many clients.

I don't think the server will even have physical space to make a full backup to itself. And then how will I be able to download it with FTP?

FTPing your data to a remote server is not a very good idea. I suggest you find an offline backup service, if the you are not cherishing the idea of a 2nd HD.

I am very disappointed with the hosting company, I don't think they should leave us on our own with the data. They say they don't copy or move data because if liability issues. But this needs someone experienced to move this data.

The vast majority of data centers hold their clients responsible for the maintenance and integrity of their own backups and restorations. It is in your best interest and strongly encouraged that you maintain backup copies of your clients' files in an alternative location such as an offline backup system, or a 2nd HD, or a tape.

During the replacement of your HD, the server will be down for several hours. If there are unforeseen difficulties, your server might be shut down for a little long time. If this is not plausible for you, then you might want to ask your data center whether it is possible to get a new server. If not that, you may want to consider an offline backup, or a 2nd HD.

 

[chirpy]

Yes, it is possible to add a 2nd HD. We have done with many clients.

That's out of your control. It's up to the server provider if they will allow the addition of a 2nd drive. I think you misunderstood the question.

FTPing your data to a remote server is not a very good idea. I suggest you find an offline backup service, if the you are not cherishing the idea of a 2nd HD.

Eh? That makes no sense. An offline backup service is just as likely to offer FTP as any other protcol. Nothing at all wrong with using FTP to whatever resource you can use to store backups off-server Again, you don't seem to have understood the OP's problem at hand.

The vast majority of data centers hold their clients responsible for the maintenance and integrity of their own backups and restorations. It is in your best interest and strongly encouraged that you maintain backup copies of your clients' files in an alternative location such as an offline backup system, or a 2nd HD, or a tape.

Which is irrelevant and nothing to do with this situation.

The vast majority of data centers hold their clients responsible for the maintenance and integrity of their own backups and restorations

Which also isn't relevant, since it's poor customer service managment for any datacenter not to offer to move data from a failing drive for them, even if there is a cost. Most of the respectable datacenters will offer such a service.

 

[veronicabend]

Thanks for all your replies.

We have got the datacenter to add 2 new hard drives to the server. They are at it, hopefully, now. They will leave the failing drive also there for now, as secondary, until we are able to move the data from it to the new drives.

From now on we will be using one of these new drives for backup. I will have to configure cPanel backup so it performs full backup to this drive. I asked them to make a big backup partition in there.

I agree we need to keep backups. But I also think it would have been very nice of them to help us moving the data from the failing drive to the new drive, especially since it is not a consequence of something we did, and they are the ones who have more knowledge about how to use the server, copy files, which files can be copied directly and which ones cannot, for which files services need to be stopped, how to stop and then restart services, etc. The hard drive got bad and that's it. It was a surprise to me when they said they would not move any of the data.

Once they have added the new drives, I will have to see how I can move the data over.

 

[chirpy]

I'm glad you got some movement from your datacentre.

For restoring full backups, I'd suggest a read of this:
http://forums.cpanel.net/showthread.php?p=253694#post253694

To migrate data from a replaced system drive that's been mounted as a slave drive:
http://forum.ev1servers.net/showthread.php?s=&threadid=38797

 

[veronicabend]

Hi Chirpy,

Thanks for those links. They will be very helpful.

Since I was not able to do full backups with cPAnel, I will have to go with copying from the slave disk. I have been reading that post and it is a good explanation. The part that makes me feel rather nervous is when they mention that after executing all those commands, I'll find myself fixing little things here and there. Since my server knowledge is not vast, I'm fearing that. Did you ever do something like this?

Thanks again,
Veronica

 

[chirpy]

I do several a week

So long as you follow the main post you should be OK. The main things to remember to do once all the rsync commands have been done is to:

/scripts/upcp --force

Then rebuild apache+php

Then run:

/scripts/fixeverything

That usually does all that you need.

 

[jsnape]

That's out of your control. It's up to the server provider if they will allow the addition of a 2nd drive. I think you misunderstood the question.


Eh? That makes no sense. An offline backup service is just as likely to offer FTP as any other protcol. Nothing at all wrong with using FTP to whatever resource you can use to store backups off-server Again, you don't seem to have understood the OP's problem at hand.

That's what I thought too. Glad someone else thought that was a little odd. The data center saying they don't move customers data due to liability sounds weird too. Ask if they can put a replacement drive in and at least mount the old one as a last ditch "Hail Mary" effort to move the data over. I that works, or even if it doesn't I'd be telling the data center its a liability risk to you by having them as a supplier soon after things calm down and you have the sites in a better data center.

 

[veronicabend]

Hi chirpy,

how do I rebuild apache + php?
could you give me some guidance on that as well?

Thank you so much!

Veronica

 

[chirpy]

WHM > Software > Update Apache

 

[veronicabend]

Hi chirpy,

Thanks so much for your guidance on this.

Datacenter says OS Reload is finished but I don't see the old drive so I have asked them how I can access it. I had asked for it to be mounted at /old.

Once I know how to access the old drive, do I need to stop any processes before I start copying the data over as in the post you reference, with rsync? The new drive has a new copy of cpanel running.

Thanks again,
Veronica

 

[chirpy]

Yes, you should sop pretty much everything. This is the command set I use:

/etc/init.d/chkservd stop
/etc/init.d/httpd stop
/etc/init.d/exim stop
/etc/init.d/pure-ftpd stop
/etc/init.d/proftpd stop
/etc/init.d/mysql stop
/etc/init.d/xinetd stop
killall -9 eximstats
/etc/init.d/cpanel stop
/etc/init.d/crond stop
/etc/init.d/courier-authlib stop
/etc/init.d/courier-imap stop


When you've finished all the work, a reboot is usually a good idea.

Mounting the old drive can be a little tricky and is best done by your provider. If you do need help with it, post the output of the following commands:

df -h

fdisk -l

 

[veronicabend]

Thanks again, chirpy.

I'm discussing mounting the old drive with the datacenter people but they only told me to be careful so I don't break the system with duplicate labels.

This is the output of the commands:

 df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3            1012M  244M  717M  26% /
/dev/sdb1              68G  760M   64G   2% /backup
/dev/sda1              99M   26M   69M  27% /boot
/dev/sda8              47G  257M   44G   1% /home
none                 1004M     0 1004M   0% /dev/shm
/dev/sda2            1012M   33M  928M   4% /tmp
/dev/sda6             9.9G  2.2G  7.2G  24% /usr
/dev/sda7             6.9G  151M  6.4G   3% /var
/tmp                 1012M   33M  928M   4% /var/tmp

fdisk -l

Disk /dev/sda: 73.4 GB, 73407900160 bytes
255 heads, 63 sectors/track, 8924 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sda1   *         1        13    104391   83  Linux
/dev/sda2            14       144   1052257+  83  Linux
/dev/sda3           145       275   1052257+  83  Linux
/dev/sda4           276      8924  69473092+   5  Extended
/dev/sda5           276       529   2040223+  82  Linux swap
/dev/sda6           530      1834  10482381   83  Linux
/dev/sda7          1835      2748   7341673+  83  Linux
/dev/sda8          2749      8924  49608688+  83  Linux

Disk /dev/sdb: 73.4 GB, 73407900160 bytes
255 heads, 63 sectors/track, 8924 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdb1   *         1      8924  71681998+  83  Linux

Disk /dev/sdc: 73.4 GB, 73407820800 bytes
255 heads, 63 sectors/track, 8924 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdc1   *         1        13    104391   83  Linux
/dev/sdc2            14      1318  10482412+  83  Linux
/dev/sdc3          1319      2232   7341705   83  Linux
/dev/sdc4          2233      8924  53753490    f  Win95 Ext'd (LBA)
/dev/sdc5          2233      8408  49608688+  83  Linux
/dev/sdc6          8409      8539   1052226   83  Linux
/dev/sdc7          8540      8670   1052226   83  Linux
/dev/sdc8          8671      8924   2040223+  82  Linux swap

I issued this command they provided:
e2label /dev/sdc6
/

So I know /dev/sdc6 is the old / partition.

About stopping the services, I issue all this commands in the SSH box. THen run all the rsync stuff and updates, all from the original post, and then the final instructions you posted here :

/scripts/upcp --force

Then rebuild apache+php

Then run:

/scripts/fixeverything

I see some people found SSH access broken after doing this, and since I only access the server via SSH I am worried. ANy precautions about this?

Thanks, real real thanks again,
Veronica

 

[brianoz]

The WHM full backup creates tarballs (accountname.tar.gz files) which contains everything cPanel needs to restore the account in its entirety, including all the account data and files. It's the tarballs you would then FTP off the server (or copy to a spare drive) and then restore through the WHM restore functions.

As far as I'm aware, the cpanel tarballs don't backup or restore Mailman lists - so if you use cpanel mailing lists be aware they won't copy over. if someone knows whether this has changed I'd love to know.

Seems like you were going to use rsync, so the above caution may not apply, but thought it was worth mentioning in case anyone refers back to this thread.

 

[mctDarren]

Veronica, can you update us on how you fared? Been a few days so hopefully all is well. Also, and this is totally up to you - don't want any harsh feelings - can you mention who the DC is? Might be someone people should check carefully into before they consider them if they are apprehensive about a simple data move. Hope all worked out!

 

[veronicabend]

Well, I finally got chirpy to help me with moving the data since I was fearing I was not going to do it well. I don't have much server knowledge and there were so many commands and things to take care of. He did a great job and all data was moved successfully in a few hours.
The datacenter people added another hard drive, and he moved everything from old drive to new one.

 

[chirpy]

As far as I'm aware, the cpanel tarballs don't backup or restore Mailman lists - so if you use cpanel mailing lists be aware they won't copy over. if someone knows whether this has changed I'd love to know

Must have changed as I recently moved our own sites to a new server using backups and it transferred over the mailman mailing lists that we use. Indeed, in the backup tarball they can be found in mm/ and mma/ subdirs.