Upload file larger than 5GB to Amazon S3 with Ruby

Several days ago I noticed one of my backup to Amazon S3 was failed because the file is larger than 5GB and I didn’t use multipart upload. Code snippet as below. Works on AWS SDK for Ruby v3.

#!/usr/bin/env ruby
# encoding: utf-8

require "aws-sdk-s3"

bucket_name = ARGV[0]
object_name = ARGV[1]
filename    = ARGV[2]

if ARGV.size != 3
  raise ArgumentError, "Missing argument"
end

Aws.config.update({ region: "ap-southeast-2"}) # Sydney

s3 = Aws::S3::Resource.new

bucket = s3.bucket(bucket_name)
object = bucket.object(object_name)
object.upload_file(filename) # Support file up to 5TB

# Not multipart upload - up to 5GB only
#File.open(filename, "rb"){|file|
#  object.put({:body => file})  
#}

Ruby gems version as below

$ gem list | grep aws
aws-partitions (1.57.0)
aws-sdk-core (3.14.0)
aws-sdk-kms (1.4.0)
aws-sdk-s3 (1.8.0)
aws-sigv4 (1.0.2)

Reference

Leave a Reply

Your email address will not be published. Required fields are marked *