Permanently Restore S3 Objects from Glacier

Glacier is a cost effective long term storage class for S3 objects. S3 objects stored in S3 are not available for retrieval immediately and require a restore if needed.

I had to convert several Glacier objects from Glacier to STANDARD storage class lately, here are my notes.

If you have millions of objects you are better off using aws batch for these operations.

Lets begin.

1. Generate a list of all files with the Glacier class using the storage class filter.

aws s3api list-objects-v2 \
 — profile <aws-profile-name> \
 — bucket <bucket-name> \
 — query “Contents[?StorageClass==’GLACIER’]” \
 — output text | awk ‘{print $2}’ > glacier_files.txt

2. Initiate a restore.

cat glacier_files.txt| while read file
do 
  echo $file && aws s3api restore-object \
   --profile <aws-profile-name> \
   --restore-request Days=25 \
   --bucket <bucket-name> --key $file
done

3. Check restore status.

cat glacier_files.txt| while read file
do
echo -n "$file: " && aws s3api head-object \
  --profile <aws-profile-name> \
  --bucket <bucket-name> \
   --key $file | jq .Restore
done

A completed restore will show the request as pending=False

"ongoing-request=\"false\", expiry-date=\"Sat, 04 Mar 2023 00:00:00 GMT\"" 

4. Overwrite the file to change the tier, we are copying the same object over itself to set this up.

cat glacier_files.txt| while read file
do
aws s3 cp --profile <aws-profile-name> \
   s3://<bucket-name>/$file s3://<bucket-name>/$file  \
   --storage-class STANDARD \
   --force-glacier-transfer

5. Now re-run step 1 and make sure your list of files is empty, done.