- Open Source
- Open Standards
- Open Minds
Use Amazon S3 with Ubuntu Hardy
Amazon S3 is an online storage web service offered by Amazon. I tested it back in 2006, since I was quite surprised that Amazon offered such an innovative product.
I found JungleDisk to be a nice and free utility to access your storage space, since Amazon doesn't offer standardized WebDAV access. While I was still hoping that Jungle Tools would release their software as free and open source, they decided to charge a (reasonable) fee of $20 for this software. I lost interest in continuing testing, since I intended to use S3 as backup space and still prefer open source software for critical tasks.
There were some open source projects trying to replace JungleDisk, but I couldn't get one to work reliable enough for a backup solution.
I was really pleased when I read that Steven Harms reviewed s3fs, a FUSE file system for Amazon S3.
Assumed you already own a Amazon Web Service Account, just make sure you've got Git and Python bindings for fuse installed:
sudo apt-get install python-fuse python-boto git-coreSince there is no Ubuntu package of s3fs available, checkout the source code:
git clone git://git.fedorahosted.org/s3fs ~/s3fsTo play around with s3fs, just start it up with:
# create a bucket, while providing your key pair.
~/s3fs/src/s3fs -C -c <bucketname> -p "<aws access key>" -s "<aws secret access key>"
# format bucket
~/s3fs/src/s3fs -C -f <bucketname> -p "<aws access key>" -s "<aws secret access key>"
# create mount point
mkdir ~/backupS3
# Unfortunately, I couldn't convince s3fs to accept the key pair as a command line option when mounting.
# So, just export them:
export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>
# mount it!
~/s3fs/src/s3fs -o bucket=<bucketname> ~/backupS3/Finally you're able to use your S3 space with GUI-tools.
"WARNING You should not yet store any data that you do not have otherwise backed up on s3fs! development on this filesystem is early enough that data loss/corruption may occur!! "
It works quite nice for me. I just wished s3fs could provide some visual feedback when it finishes copying. An integration with gvfs would be great, so I could get an simple progress bar for the copy process.
Comments
I used JungleDisk and I
I used JungleDisk and I really liked it--it worked great--but a few things turned me away from it.
1) It's non-free, which I struggled with the whole time.
2) It was very poorly labeled as a trial, so I ended up putting all my data on s3 via JungleDisk and then the trial ran out. I was then reminded one of the key reasons why I prefer free software: If I did not pay for JungleDisk I would not be able to access my data again. EPIC FAIL!
I ended up emailing them and explaining the situation. They were very professional and let me access my data for a short while so I could retrieve it again. I think that experience however reminded me that non-free software only traps the user it never frees the user. I'll continue to look for a free-software solution, I just hope I can find one as robust as JungleDisk.
Let's hope s3fs will fill
Let's hope s3fs will fill that gap. It seems a Ubuntu package is on it's way: "Received some email with some interest from Ubuntu. Looks like there may be .deb packages of s3fs available soon :)"
How do you find the
How do you find the "bucketname"?
Just choose an arbitrary one.
Just choose an arbitrary one. Bucket names are unique through the whole S3 system.
After running the "mount"
After running the "mount" command I get the following error:
Need to specify AWS_ACCESS_KEY_ID
Need to specify AWS_SECRET_ACCESS_KEY
Unable to establish configure this mount
I used the export commands. What did I do wrong?
Thanks
I had the same problem two
I had the same problem two days before I posted this article. I tried it again with an updated version of s3fs, so I thought it was only a temporary problem. You might want to try it again in a few days (just delete the s3fs directory and get a new copy).
A very helpful mini howto.
A very helpful mini howto. Thanks!
However, when i try to mount the bucket, the following error is thrown on the CLI:
fuse: failed to open /dev/fuse: Permission denied
I have created a group called fuse, added my default regular user to it, set chown root:fuse /dev/fuse and chmod 770 /dev/fuse
No luck so far.
Iif i follow the tutorial as root, the bucket is mount without a glitch.
Didn't work for me either.
Didn't work for me either. Requested the Secret Key, and when I provided it, got some nonsense about passing single quotes. Thought you had something here at first. Thanks.
I have this error: Unable to
I have this error:
Unable to establish this mount
I have the most recent git version. I tried to do it as root. I read the code in src/s3fs. It fails on this line:
if fs.setup() == False
print "Unable to establish this mount"
It's sad, I was excited to use S3 with fuse.
Hadn't considered the
Hadn't considered the integration with gvfs. I'll look into that ASAP. Thanks!
Also, for those having mount issues, if you'd like to open up tracking bugs (https://fedorahosted.org/s3fs/), I'd be happy to take a look into your various issues.
Thanks for all the interest!
How I got it set up for a
How I got it set up for a non-root user:
0) Assume a non-root user account already set up. We'll call this user 'bob'.
1) Created a directory structure under /home/bob for Jungle Disk, like so:
/home/bob/JungleDisk/cache
/home/bob/JungleDisk/mount
/home/bob/JungleDisk/setup
I gave the above folders 777 access (chmod), though /home/bob does not have 777 access.
1) Copied the jungledisk-settings.xml from my Mac (found ???) to /home/bob/JungleDisk/setup on my linux box. Give it 777 access, and changed the element inside the file to point to /home/bob/JungleDisk/cache.
2) Installed fuse-utils. I'm running ubuntu server and had to edit /etc/apt/sources.list to open up the universe repository, then ran
sudo apt-get update
sudo apt-get install fuse-utils
3) Per the post at https://lists.ubuntu.com/archives/ubuntu-users/2006-June/080745.html, fixed fusermount permissions like so
sudo chmod 4755 /usr/bin/fusermount
4) Downloaded the linux install from http://www.jungledisk.com/desktop/download.aspx, untarred it somewhere, copied the 'jungledisk' binary out to /usr/bin, then
sudo chown root:root /usr/bin/jungledisk
sudo chmod 4755 /usr/bin/jungledisk
5) Ran Jungle Disk like so:
jungledisk /home/bob/JungleDisk/mount/ -o config=/home/bob/JungleDisk/setup/jungledisk-settings.xml
You can check to see if it worked by running
less ~/.jungledisk/jungledisk.log
If it doesn't exactly state "Jungle Disk started", then it broke and you get to try to fix it.
6) Test your setup:
cd /home/bob/JungleDisk/mount
ls -la
You should see any files you've bucketed. And you should be able to stick stuff out there too.
Post new comment