If you want to take backup using the aws s3 sync on a large number of files than it can result in the excessive cpu and network utilization. You can limit the same using the s3api as follows
trickle -u 10 -d 20 aws s3 sync source destination
This limits the upload speed to 10kbps and download to 20kbps
$ aws configure set default.s3.max_concurrent_requests 5
$ aws configure set default.s3.multipart_threshold 6MB
$ aws configure set default.s3.multipart_chunksize 6MB
If you want to limit the network than you can use the Trickle in linux to limit the network upload and download speedtrickle -u 10 -d 20 aws s3 sync source destination
This limits the upload speed to 10kbps and download to 20kbps
0 comments:
Post a Comment