Spaces Lifecycle is not expiring files

October 12, 2018 778 views
DigitalOcean
robotastic
By:
robotastic

I am trying to setup a lifecycle rule to delete all files older than 5 days. I can successfully PUT the lifecycle and confirmed it is up there with a GET. However, it doesn’t look like any files are getting removed.

Below is the rules.json file:

{
“Rules”: [
{
“ID”: “Remove old files”,
“Status”: “Enabled”,
“Prefix”: “”,
“Expiration”: {
“Days”: 5
}
}
]
}

And here is the AWS command I used:

aws s3api –endpoint-url=https://nyc3.digitaloceanspaces.com \
put-bucket-lifecycle-configuration \
–bucket REDACTED \
–lifecycle-configuration \
file://rules.json

Am I missing something obvious?

Sign In to Comment
3 Answers
yanislav November 27, 2018

I get the same issue. The lifecycle policy is not deleting the files. Setting a prefix or keeping it empty does not make a difference.

Report
DimL December 2, 2019

Did you get it working? API returns all the rules I’ve set (see below) but files are not being deleted even after 11 days and trash just keeps piling. Not cool…

{
    "Rules": [
        {
            "Status": "Enabled", 
            "Prefix": "/", 
            "Expiration": {
                "Days": 2
            }, 
            "ID": "Remove old tmp files"
        }, 
        {
            "Status": "Enabled", 
            "Prefix": "", 
            "ID": "Remove uncompleted uploads", 
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}
Report
joellm December 2, 2019

There might be an issue with overlapping rules. Try creating a single rule with 2 actions (expire + abort incomplete):

{
    "Rules": [
        {
            "ID": "Remove old backups",
            "Status": "Enabled",
            "Prefix": "/",
            "Expiration": {
                "Days": 2
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}

(I’m not sure whether the prefix should be “/” or “” (empty string), though. Might help to try with both.)

Report
  • DimL December 2, 2019

    I tried empty string, but it’s mandatory and can’t be empty (it’s ok for AbortIncompleteMultipartUpload though). I’ll try to put everything in a folder e.g. “tmp” instead of root and give it prefix “tmp/”, then we will see…

    Report
    • joellm December 2, 2019

      The documentation says:

      A string specifying the objects to which the rule will be applied. When provided, only objects whose keys begin with the prefix will be acted upon. If empty or not present, all object in the Space will be affected.

      So, you should be able to omit the prefix entirely.

      But the most important part is that you use a single rule with Expiration & AbortIncompleteMultipartUpload actions, not 2 separate rules.

      Also (I just remembered), for me, I actually had to use an XML configuration file, not JSON.

      For example:

      <LifecycleConfiguration>
    <Rule>
        <ID>MyRuleID123</ID>
        <Status>Enabled</Status>
        <Filter>
          <Prefix>xxx-</Prefix>
        </Filter>
        <AbortIncompleteMultipartUpload>
          <DaysAfterInitiation>2</DaysAfterInitiation>
        </AbortIncompleteMultipartUpload>
        <Expiration>
          <Days>15</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

      Using the command:

      s3cmd setlifecycle /path/to/lifecycle.xml s3://bucket-name
      Report
      • DimL December 3, 2019

        Thank you! Single rule with empty prefix did the trick. Json worked fine for me.

        {
    "Rules": [
        {
            "ID": "Remove expired temporary files",
            "Status": "Enabled",
            "Prefix": "",
            "Expiration": {
                "Days": 1
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}
        Report
Have another answer? Share your knowledge.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Related