Month: July 2009

Using rsync for replication on OS X

I recently began a quest to replicate my house data with a hosting service so if my house goes up in a fire I still have my data elsewhere. The utility that seems to work the best for this is called rsync, and is available on most flavors of Unix, including Apple’s OS X. Why is it good?  Well, rsync talks to the remote host and compares file meta-data like modified times, checksums, etc. to determine if a file needs to be transmitted.  It can handle partial file retransmission which is very nice if your dealing with large files.  It’s much more efficient than other methods that clumsily send all data over the line to have the destination server determine if the data should be discarded or not.  Here’s an example of the command I use (disk1 is a directory with a bunch of directories and files):

superbooky:Sean$ rsync -P -z -a -v -e "ssh -l sshuser"           \
                       --partial --delete-after --max-delete=500 \
                       /Volumes/disk1 bubba.com:~/disk1

While the command switches look scary, they really aren’t that bad. Here’s the explanation:

-P = Display

-z = Compress the data stream

-a = Archive mode – enables switches -rlptgoD
recurse dirs, copy symlinks, preserve permissions:mod times:group:owner:special files

-v = Increase verbosity

-e “ssh -l sshuser” = Run rsync through ssh for encrypted transfers

–partial = Leave partial files on the destination side.  This is to allow rsync to simply continue the transfer if a transmission interuption occurs. If you have large multi-GB files this is invaluable.

–delete-after = Delete any files on the destination that were deleted locally after all files have been replicated rather that during the replication.

–max-delete=500 = If rsync starts it’s deletion and goes more than 50 files, exit with an error.  This is a safegaurd in case something weird happens.

Using windows?  Sorry there’s no simple equivalent that is just available.  You’ll need to install software to get this type functionality

The 4th Electrical Element, Memristor

Last year HP Labs was able to create the long theorized electrical element, the Memristor.  While most people have not heard anything about the memristor, it will soon change the whole face of our electronics world.  The Memristor was a theoretical element in electronics that was postulated in the late 1960’s and was finally created in 2008.  Our entire electronics work today is only based on the other 3 elements – Resistor, Capacitor, and Inductor.

Why should you care?

Imagine your next computer with 10 terabytes of RAM, with no hard drive and no need to “load” programs.  Imagine the same computer powered off without loosing any information, requiring no voltage draw and at a fraction of the size of the smallest computers today – heck I could have described the iPhone of the future.

That’s just the tip of the iceberg.  Analog processors, extremely fast crossbars, and others are on the near horizon. 20-50 years?  No.  I’m talking about the next 2-5 years.

Here’s a few links regarding the memristor: