Hello, I have some files and folders saved on Amazon web services S3 Bucket, now I would like to download entire bucket files and folders locally, how can I do it using CLI? Which is an easy way to get all contents of the Amazon S3 bucket? Thanks.
You can use
aws s3 sync command using CLI to download all files of bucket, here is the sample code
aws s3 sync s3://<source_bucket> <local_destination>
In the above command, replace the following fields:
<source_bucket>-> your S3 bucket that you want to download.
<local_destination>-> path in your local system where you want to download all the files.
But yes, first you need to run
aws configure to add
your access key and
secret key, to grant access of bucket to CLI user.
You can check details on how to get access key here https://console.aws.amazon.com/iam/home?#/security_credentials
Before doing that you will need to download and install AWS CLI (if you haven't done it already.)
If you don't like CLI commands, you can use https://cyberduck.io/ tool and use it's GUI to download S3 Buckets files, you need to simply enter your credentials there and download files.
I will suggest to use below command to configure concurrent request in AWS CLI
aws configure set default.s3.max_concurrent_requests 1000 aws configure set default.s3.max_queue_size 100000
The above settings will helps you download large bucket more quickly using concurrent request settings.
and then you can
s3cmd in aws cli
s3cmd --configure s3cmd sync s3://bucketnamehere/folder /destination/folder
S3 command to download bucket
aws s3 sync s3://source-bucket/source-path c:\my\local\data\path
Subscribe to our weekly Newsletter & Keep getting latest article/questions in your inbox weekly