Question

Spaces Lifecycle is not expiring files

Posted October 12, 2018 1.4k views
DigitalOcean

I am trying to setup a lifecycle rule to delete all files older than 5 days. I can successfully PUT the lifecycle and confirmed it is up there with a GET. However, it doesn’t look like any files are getting removed.

Below is the rules.json file:

{
“Rules”: [
{
“ID”: “Remove old files”,
“Status”: “Enabled”,
“Prefix”: “”,
“Expiration”: {
“Days”: 5
}
}
]
}

And here is the AWS command I used:

aws s3api –endpoint-url=https://nyc3.digitaloceanspaces.com \
put-bucket-lifecycle-configuration \
–bucket REDACTED \
–lifecycle-configuration \
file://rules.json

Am I missing something obvious?

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers

I get the same issue. The lifecycle policy is not deleting the files. Setting a prefix or keeping it empty does not make a difference.

Did you get it working? API returns all the rules I’ve set (see below) but files are not being deleted even after 11 days and trash just keeps piling. Not cool…

{
    "Rules": [
        {
            "Status": "Enabled", 
            "Prefix": "/", 
            "Expiration": {
                "Days": 2
            }, 
            "ID": "Remove old tmp files"
        }, 
        {
            "Status": "Enabled", 
            "Prefix": "", 
            "ID": "Remove uncompleted uploads", 
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}

There might be an issue with overlapping rules. Try creating a single rule with 2 actions (expire + abort incomplete):

{
    "Rules": [
        {
            "ID": "Remove old backups",
            "Status": "Enabled",
            "Prefix": "/",
            "Expiration": {
                "Days": 2
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}

(I’m not sure whether the prefix should be “/” or “” (empty string), though. Might help to try with both.)

  • I tried empty string, but it’s mandatory and can’t be empty (it’s ok for AbortIncompleteMultipartUpload though). I’ll try to put everything in a folder e.g. “tmp” instead of root and give it prefix “tmp/”, then we will see…

    • The documentation says:

      A string specifying the objects to which the rule will be applied. When provided, only objects whose keys begin with the prefix will be acted upon. If empty or not present, all object in the Space will be affected.

      So, you should be able to omit the prefix entirely.

      But the most important part is that you use a single rule with Expiration & AbortIncompleteMultipartUpload actions, not 2 separate rules.

      Also (I just remembered), for me, I actually had to use an XML configuration file, not JSON.

      For example:

      <LifecycleConfiguration>
          <Rule>
              <ID>MyRuleID123</ID>
              <Status>Enabled</Status>
              <Filter>
                <Prefix>xxx-</Prefix>
              </Filter>
              <AbortIncompleteMultipartUpload>
                <DaysAfterInitiation>2</DaysAfterInitiation>
              </AbortIncompleteMultipartUpload>
              <Expiration>
                <Days>15</Days>
              </Expiration>
          </Rule>
      </LifecycleConfiguration>
      

      Using the command:

      s3cmd setlifecycle /path/to/lifecycle.xml s3://bucket-name
      
      • Thank you! Single rule with empty prefix did the trick. Json worked fine for me.

        {
            "Rules": [
                {
                    "ID": "Remove expired temporary files",
                    "Status": "Enabled",
                    "Prefix": "",
                    "Expiration": {
                        "Days": 1
                    },
                    "AbortIncompleteMultipartUpload": {
                        "DaysAfterInitiation": 1
                    }
                }
            ]
        }
        
Submit an Answer