How can I download complete AWS S3 Bucket Locally?


Hello, I have some files and folders saved on Amazon web services S3 Bucket, now I would like to download entire bucket files and folders locally, how can I do it using CLI? Which is an easy way to get all contents of the Amazon S3 bucket? Thanks.


Asked by:- bhanu
1
: 2575 At:- 11/29/2021 12:18:14 PM
AWS Amazon S3 bucket AWS CLI







2 Answers
profileImage Answered by:- vikas_jk

You can use aws s3 sync command using CLI to download all files of bucket, here is the sample code

aws s3 sync s3://<source_bucket> <local_destination>

In the above command, replace the following fields:

  • <source_bucket> -> your S3 bucket that you want to download.
  • <local_destination> -> path in your local system where you want to download all the files.

But yes, first you need to run aws configure to add your access key and secret key, to grant access of bucket to CLI user.

You can check details on how to get access key here https://console.aws.amazon.com/iam/home?#/security_credentials

Before doing that you will need to download and install AWS CLI (if you haven't done it already.)

OR

If you don't like CLI commands, you can use https://cyberduck.io/ tool and use it's GUI to download S3 Buckets files, you need to simply enter your credentials there and download files.

1
At:- 11/29/2021 2:55:47 PM Updated at:- 8/4/2022 12:50:19 AM


profileImage Answered by:- pika

I will suggest to use below command to configure concurrent request in AWS CLI

aws configure set default.s3.max_concurrent_requests 1000
aws configure set default.s3.max_queue_size 100000

The above settings will helps you download large bucket more quickly using concurrent request settings.

and then you can s3cmd in aws cli

s3cmd --configure
s3cmd sync s3://bucketnamehere/folder  /destination/folder

OR

S3 command to download bucket

aws s3 sync s3://source-bucket/source-path c:\my\local\data\path

thanks

0
At:- 2/21/2022 9:33:37 AM






Login/Register to answer
Or
Register directly by posting answer/details

Full Name *

Email *




By posting your answer you agree on privacy policy & terms of use