Friday, December 27, 2013

Setting up WD MyCloud with Crashplan

tl;dr skip to INSTRUCTIONS

Crashplan is a great backup solution: inexpensive, fast, easy to set up. I have it running under a multi-computer plan for a small home office which uses a NAS as a file server. It runs on each computer fine, but I wanted to also back up all the shared files on the NAS.

Additionally, we recently had a Buffalo TeraStation PRO failure, so I wanted our system to rely less on any one piece of hardware.

Enter the WD MyCloud. Cheap, fast, small, quiet. 4TB for $199 in Dec 2013. VERY easy to set up. Just plug it in. Seriously - if you don't need a lot of customization - this is how computer equipment should work.

Now to set it up took several hours and a lot of debugging. If you follow the instructions below though it should be quick. You can do this. You'll have a NAS accessable across your network which automatically backs itself up to Crashplan without any user intervention.

The instructions below cover ALL of the steps and hopefully don't require knowledge of Unix or Linux commands so feel free to skip if you know what you're doing. The indented items and blue links go into more detail. I used a PC, but instructions should be the same on a Mac though the tools used may be different.

Shoutouts to all who are linked, thank you for the help.

INSTRUCTIONS


  1. Plug the device in to power and your network
  2. Optional: Change the Device Name
    1. Load your device dashboard
    2. Click Settings and change Device Name and Device Description. Make sure to click Save.
  3. Enable SSH on your device & note the username and password given
  4. Log in to the MyCloud device using SSH (I like PuTTY, it is lightweight, free and has been around for 15 years). IMPORTANT: This voids your warranty. Proceed at your own risk.
    1. Download putty.exe from this site or wherever you can find it
    2. Run putty.exe from wherever you downloaded it and click "Session" in the "Category" browser on the left.
    3. Use your device IP address as the "Host Name (or IP address)" and click "Open" at the bottom
    4. If a Security Alert pops up, you can click yes to allow connection. This is YOUR server after all.
    5. Log-in by typing the username and password given on step 3 (Default is "root" and "welc0me" - yes, that's a zero)
  5. Your PuTTY screen should look like this:

  6. You may notice that at the end of the 3rd line, we are dealing with an armv71 processor, not an Intel. When we install crashplan in the next step we will have to modify a file to make it able to communicate with the ARM processor.
  7. IMPORTANT Install Java first (trust me - see this)
    1. First we need to update apt-get: sudo apt-get update
    2. Install JRE headless: sudo apt-get install openjdk-6-jre-headless
    3. The following may be optional, I have not tested without them:
    4. Get the java version for our OS: sudo apt-get install libjna-java
    5. Install the Java Runtime Environment (JRE) sudo apt-get install default-jre
  8. Install Crashplan using these instructions. 
    1. Using my windows machine, I downloaded the installer and then copied to a new folder called "tmp" I created in the Public folder on the MyCloud device
    2. Then in the PuTTY session from step 4 use the following commands to unzip and run the installer:
      1. cd /shares/Public/tmp
      2. tar zxvf CrashPlan_3.5.3_Linux.tgz
        1. This line may be slightly different depending on the filename of the Crashplan download, just change the part after "tar zxvf" to the filename you downloaded
      3. cd CrashPlan-install
      4. sudo ./install.sh
  9. Follow all the prompts for the Crashplan install. Accept the defaults. If you don't, you probably know what you're doing - just change the below instructions to point to the locations you installed the files. If it asked to install java, cancel and go back to step 7.
    1. The installer ends by saying the service is started, but if you check the status, after a few seconds it will not be running. We need to fix one file to work with the ARM processor.
      1. To check the status, run: /etc/init.d/crashplan status
  10. First, let's increase the number of files the Crashplan client will watch (while you remember how to log in to the MyCloud!). Follow these directions (section 3.2) - copied below.
    1. To see # of watches: cat /proc/sys/fs/inotify/max_user_watches
    2. To change the # of watches, run the following:
      1. sudo echo 1048576 > /proc/sys/fs/inotify/max_user_watches
      2. sudo nano /etc/sysctl.conf
      3. Then once the editor opens, add this to the end of the file:
      4. # Set up inotify kernel module to keep watch on up to 1M files
        fs.inotify.max_user_watches=1048576
  11. Then we need to patch the crashplan files as in step 2 of this tutorial, with some minor changes.
    1. Modify the crashplan launcher script to point to the new java version
      1. nano /usr/local/crashplan/bin/CrashPlanEngine
      2. Change the line starting with FULL_CP to:
        1. FULL_CP="/usr/share/java/jna.jar:$TARGETDIR/lib/com.backup42.desktop.jar:$TARGETDIR/lang"
  12. Now we need to change the libjtux.so file to a version that will work with the ARM processor. We will need to download it, fix it and build and compile it. This is the hard part. 
    1. Rename original libjtux.so file to get it out of the way
      1. mv /usr/local/crashplan/libjtux.so /usr/local/crashplan/libjtux.so.intel
    2. You can skip all of the following by downloading the file from here and replacing the original one.
      1. cd /shares/Public/tmp/
      2. wget https://dl.dropboxusercontent.com/u/21924000/libjtux.so
      3. cp /shares/Public/tmp/libjtux.so /usr/local/crashplan/libjtux.so
    3. If you'd rather compile from scratch (maybe the above doesn't work), then follow these directions:
      1. Download the jtux package to your tmp directory on the share and unzip it:
        1. cd /shares/Public/tmp
        2. wget http://www.basepath.com/aup/jtux/jtux.tar.gz
          1. if  you can't find it there, just do a google search for jtux and paste the link after wget
        3. tar -zxvf jtux.tar.gz
        4. sudo apt-get install build-essentials, using any packages from here if they don't match the version that is needed. I had to use version 2.17-6. Install them using dpkg -i <package-name>
          1. To install, download to your drive and then unzip as you did with previous files. Then use dpkg -i on the .deb file that you get after unzipping. I'll go into more detail if there are a lot of questions on this.
        5. Follow the patch instructions here (don't forget the #include line!)
  13. Once you have the right version of libjtux.so in place, start the CrashPlanEngine
    1. /usr/local/crashplan/bin/CrashPlanEngine start
  14. Check to make sure it is running
    1. /etc/init.d/crashplan status
  15. Now follow these Directions for Configuring a Headless Client
  16. You should be good to go!! Hopefully you have a fast internet connection for the first upload. Good luck!

18 comments:

  1. This is great, just set it up on my new WD drive and works a treat. thanks!

    ReplyDelete
    Replies
    1. Thanks Marcus!! I am glad to hear it. Just an FYI - I did a firmware update and had to re-install everything. The same recipe worked though. Good luck!

      Delete
    2. Yes, just realised that myself - had auto updates enabled on the MyCloud, so am now back to re-installing it all, and have now disabled auto updates!

      Delete
  2. NIce one, just set this up. I got errors during step 7 sub-tasks, I just ignored these and everything still seemed ok. Might be worth mentioning if these errors are expected and can be ignored.

    ReplyDelete
  3. Thanks! What kind of errors did you see?

    ReplyDelete
  4. Thank you for the detailed instructions. I just got a WDMyCloud, and look forward to trying this. Will we be able to use CrashPlans backup to Friend feature using this method? I could buy a 2nd drive, seed the backup locally and give the drive to a friend.

    ReplyDelete
    Replies
    1. I have not tried that - let me know if it works!

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Thanks, everything worked great for me. However, a couple weeks in I had to power cycle the mycloud and afterwards all the Crashplan files were gone. I had to reinstall the entire thing (note this is just the headless client, the backups were fine). Any ideas why or how to avoid that in the future? Thanks!

    ReplyDelete
  7. Actually, the plot thickens for the worse. After attempting to reinstall crashplan via the procedure above and running into permission problems (even though I was logged in as root) I decided to do a factory reset. Bad idea, I am now plagued with the problem where the dashboard is not accessable and therefore I cannot I change any settings. Worse is that the factory reset diasabled the SSH access so there is no fix for this problem. Luckily the retailer will exchange the unit since its relatively new. Do you have any recommendations for doing a complete uninstall and resinstall without having to lose user data?

    Also, I'm wondering if it wasn't the reboot that killed crashplan but rather maybe a WD automatic update kicked off which cleared the Crashplan install...

    ReplyDelete
    Replies
    1. Did you ever get this working again? I ran into the same problem. I haven't tried to reinstall yet because I saw your post.

      Delete
    2. Currently I am running into the same problem. It appears the latest firmware (v4.00) has issues running packages installed normally, and requires a workaround posted on their forums (copied below). I have not yet tested this, please let me know if you have and whether or not it is successful!

      Instructions to rebuild a Debian package

      The WD My Cloud device firmware versions 04.00.00 and later use a modified version of Debian Linux which uses 64K sized memory pages. In order to build Debian packages for use on a 64K page-size device, an "qemu-emulated native" build system is used, which contains the binutils package modified for 64K page-size alignment.

      Prerequisites:

      - x86 PC with 64-bit Ubuntu or Debian installed
      - packages qemu-user-static and binfmt-support installed on host build system
      - Modified binutils package with 64K page size support. A pre-built version is included in binutils directory. There is no need to re-build binutils, however, instructions are provided in the source package (see binutils/README-binutils-64K-pagesize.txt).

      Run the script:

      ./build-armhf64k-package.sh

      Where:

      is the name of the package you would like to build.
      is the Debian suite version. E.g., "wheezy" (i.e. stable) or "jessie" (i.e. testing).

      After a successful build, the Debian build artifacts (.deb files) for the package will appear in the directory "build/root". The artifacts may then be copied to a WD My Cloud device, and installed through the standard Debian install mechanism (dpkg -i).

      Delete
    3. Hi all,

      No, I never fixed it. Luckily for me I had just purchased it and just got a new one. I'm still on the fence if I will try crashplan on it.

      -e

      Delete
  8. Ok guys, I'm back and running.

    I downgraded using these instructions:

    http://community.wd.com/t5/WD-My-Cloud/Easy-method-to-downgrade-to-previous-version-of-the-firmware/td-p/773073

    Be sure that Auto Firmware Update is NOT enabled. =)

    ReplyDelete
  9. I ended up trying to downgrade mine, but it crashed the UI, so WD sent me a new one. I made sure the auto upgrade was turned off, so it won't upgrade to 4.0. When I started the install process on it, I got the following error on step 7.2 (installing JRE). Is this something that will cause problems or can I ignore it? If it is a problem, can someone tell me how to fix it? I am very new to linux, so I don't really know what I am looking at. Thanks for your help.

    Error info:
    update-alternatives: error: error creating symbolic link `/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file or directory
    dpkg: error processing openjdk-6-jre-headless:armhf (--configure):
    subprocess installed post-installation script returned error exit status 2
    dpkg: dependency problems prevent configuration of ca-certificates-java:
    ca-certificates-java depends on openjdk-6-jre-headless (>= 6b16-1.6.1-2) | java6-runtime-headless; however:
    Package openjdk-6-jre-headless:armhf is not configured yet.
    Package java6-runtime-headless is not installed.


    Thanks,
    Jason

    ReplyDelete
  10. Not sure but I had many errors as well and I just ignored them. Everything seems to be working fine. No problems for weeks.

    Now I did find that occasionally the crashplan engine would die on its own so I wrote a simple bash script which checks to see if the engine is running and if it isn't it starts it. I added the bash script as a cron job which gets run once a day. In fact, I set it up so that I backup at only specific times of day and use the bash script as a cron job to start the engine right before the backup and turn off the engine when the backup window is up. This way the engine isn't running when I don't need it.

    Good luck.

    ReplyDelete
  11. the default username is sshd but this user does not have root access. There do not appear to be any other accounts.

    ReplyDelete
  12. This sounds great. How do you select the folders on the WD myCloud that CrashPlan will back up?

    ReplyDelete