How can I access AmazonS3 through the Linux command line?

AmazonS3 is a cloud storage provided by Amazon Web Services (AWS. AmazonS3 released a set of network service interfaces on top of commercial services or client software developed by many third parties. This tutorial describes how to use Linux command lines to access Amazon tags: Linux command cloud storage Amazon

Amazon S3 is a cloud storage provided by Amazon Web Services (AWS. Amazon S3 releases a set of network service interfaces on top of commercial services or client software developed by many third parties. This tutorial describes how to use Linux command lines to access Amazon S3 cloud storage.

The most famous Amazon S3 command line client is s3cmd written in python. As a simple AWS S3 command line tool, s3cmd is used to run script-based cron tasks, such as daily backup.

Install s3cmd on Linux

Install s 3cm on Ubuntu or Debian

 
 

  
  
___FCKpd___0nbsp;sudo apt-get install s3cmd 
 
 

Install s3cmd on Fedora

 
 

  
  
___FCKpd___1nbsp;sudo yum install s3cmd 
 
 

Install s 3cm on CentOS or RHEL

 
 

  
  
___FCKpd___2nbsp;s3cmd put –acl-public 4.png s3://dev99 
  
  
4.png -> s3://dev99/4.png [1 of 1] 
  
  
30778 of 30778 100% in 8s 3.34 kB/s done 
 
 

Download the rpm package from the official site and install it manually. Run the following command to run 64-bit CentOS or RHEL 6:

 
 

  
  
___FCKpd___3nbsp;sudo rpm -ivh s3cmd-1.0.0-4.1.x86_64.rpm 
 
 

Configure s3cmd

To run s3cmd for the first time, run the following command for configuration:

 
 

  
  
___FCKpd___4nbsp;s3cmd –configure 
 
 

It will ask you a series of questions:

• AWS S3 access key and security key
• AWS S3 two-way transmission of encrypted passwords and encrypted data
• Set the path of the GPG program for encrypted data (for example,/usr/bin/gpg)
• Whether to use https
• If http proxy is used, set the name and port

The configuration will be saved in ~ /. S3cfg.

Basic usage of s3cmd

List all existing buckets in your account:

 
 

  
  
___FCKpd___5nbsp;s3cmd ls 
  
  
2011-05-28 22:30 s3://mybucket1 
  
  
2011-05-29 00:14 s3://mybucket2 
 
 

Create a new bucket:

 
 

  
  
___FCKpd___6nbsp;s3cmd mb s3://dev99 
  
  
Bucket ’s3://dev99/’ created 
 
 

Upload a file to an existing bucket:

 
 

  
  
___FCKpd___7nbsp;s3cmd put 1.png 2.png 3.png s3://dev99 
  
  
1.png -> s3://dev99/1.png [1 of 3] 
  
  
26261 of 26261 100% in 5s 4.33 kB/s done 
  
  
2.png -> s3://dev99/2.png [2 of 3] 
  
  
201430 of 201430 100% in 2s 98.05 kB/s done 
  
  
3.png -> s3://dev99/3.png [3 of 3] 
  
  
46630 of 46630 100% in 0s 56.62 kB/s done 
 
 

The default access permission for uploaded files is private, that is, only you can access the file by using the correct access and security password.

Upload a file with public access permission to an existing bucket:

 
 

  
  
___FCKpd___8nbsp;s3cmd put --acl-public 4.png s3://dev99 
  
  
4.png -> s3://dev99/4.png  [1 of 1] 
  
  
30778 of 30778   100% in    8s     3.34 kB/s  done 
  
  
Public URL of the object is: http://dev99.s3.amazonaws.com/4.png 
 
 

If the uploaded file is granted public access, anyone in the browser can access it through the http://dev99.s3.amazonaws.com/4.png.

View the content of an existing bucket:

 
 

  
  
___FCKpd___9nbsp;s3cmd ls s3://dev99 
  
  
2013-06-02 02:52 26261 s3://dev99/1.png 
  
  
2013-06-02 02:52 201430 s3://dev99/2.png 
  
  
2013-06-02 02:52 46630 s3://dev99/3.png 
  
  
2013-06-02 02:56 30778 s3://dev99/4.png 
 
 

Download the files containing the bucket package (for example, all .png files ):

 
 

  
  
___FCKpd___10nbsp;s3cmd get s3://dev99/*.png 
  
  
s3://dev99/1.png -> ./1.png [1 of 4] 
  
  
26261 of 26261 100% in 0s 39.39 kB/s done 
  
  
s3://dev99/2.png -> ./2.png [2 of 4] 
  
  
201430 of 201430 100% in 7s 24.64 kB/s done 
  
  
s3://dev99/3.png -> ./3.png [3 of 4] 
  
  
46630 of 46630 100% in 1s 39.34 kB/s done 
  
  
s3://dev99/4.png -> ./4.png [4 of 4] 
  
  
30778 of 30778 100% in 0s 97.01 kB/s done 
 
 

Delete files in an existing bucket:

 
 

  
  
___FCKpd___11nbsp;s3cmd del s3://dev99/*.png 
  
  
File s3://dev99/1.png deleted 
  
  
File s3://dev99/2.png deleted 
  
  
File s3://dev99/3.png deleted 
  
  
File s3://dev99/4.png deleted 
 
 

Obtain information about an existing bucket, including the storage location and access control list (ACL ):

 
 

  
  
___FCKpd___12nbsp;s3cmd info s3://dev99 
  
  
s3://dev99/ (bucket): 
  
  
Location: us-east-1 
  
  
ACL: dan.nanni: READ 
  
  
ACL: dan.nanni: WRITE 
  
  
ACL: dan.nanni: READ_ACP 
  
  
ACL: dan.nanni: WRITE_ACP 
 
 

Before uploading data to an existing bucket, encrypt the file:

 
 

  
  
___FCKpd___13nbsp;s3cmd -e put encrypt.png s3://dev99 
  
  
/tmp/tmpfile-pzT1zV3kLZlxDwqA0kwy -> s3://dev99/encrypt.png [1 of 1] 
  
  
196890 of 196890 100% in 1s 99.51 kB/s done 
 
 

When you use s3cmd to download an encrypted file, it automatically detects the encryption and decrypts the file during the download process. Therefore, when you download and access the encrypted file, it is as usual:

 
 

  
  
___FCKpd___14nbsp;s3cmd get s3://dev99/encrypt.png 
  
  
s3://dev99/encrypt.png -> ./encrypt.png [1 of 1] 
  
  
196890 of 196890 100% in 1s 131.29 kB/s done 
 
 

Delete an existing bucket:

 
 

  
  
___FCKpd___15nbsp;s3cmd rb s3://dev99 
  
  
Bucket ’s3://dev99/’ removed 
 
 

Note: You cannot delete a non-empty bucket.