Related

Does DigitalOcean preserve snapshots of deleted droplets? Question Question
Is there a DigitalOcean Free Trial Available? Question Question

Question

Spaces Lifecycle is not expiring files

Posted October 12, 2018 1.8k views
DigitalOcean

I am trying to setup a lifecycle rule to delete all files older than 5 days. I can successfully PUT the lifecycle and confirmed it is up there with a GET. However, it doesn’t look like any files are getting removed.

Below is the rules.json file:

{
“Rules”: [
{
“ID”: “Remove old files”,
“Status”: “Enabled”,
“Prefix”: “”,
“Expiration”: {
“Days”: 5
}
}
]
}

And here is the AWS command I used:

aws s3api –endpoint-url=https://nyc3.digitaloceanspaces.com \
put-bucket-lifecycle-configuration \
–bucket REDACTED \
–lifecycle-configuration \
file://rules.json

Am I missing something obvious?

Add a comment
robotastic
By:
robotastic
Add a comment

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
3 answers
yanislav November 27, 2018

I get the same issue. The lifecycle policy is not deleting the files. Setting a prefix or keeping it empty does not make a difference.

Reply Report
DimL December 2, 2019

Did you get it working? API returns all the rules I’ve set (see below) but files are not being deleted even after 11 days and trash just keeps piling. Not cool…

{
    "Rules": [
        {
            "Status": "Enabled", 
            "Prefix": "/", 
            "Expiration": {
                "Days": 2
            }, 
            "ID": "Remove old tmp files"
        }, 
        {
            "Status": "Enabled", 
            "Prefix": "", 
            "ID": "Remove uncompleted uploads", 
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}
Reply Report
joellm December 2, 2019

There might be an issue with overlapping rules. Try creating a single rule with 2 actions (expire + abort incomplete):

{
    "Rules": [
        {
            "ID": "Remove old backups",
            "Status": "Enabled",
            "Prefix": "/",
            "Expiration": {
                "Days": 2
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}

(I’m not sure whether the prefix should be “/” or “” (empty string), though. Might help to try with both.)

Reply Report
  • DimL December 2, 2019

    I tried empty string, but it’s mandatory and can’t be empty (it’s ok for AbortIncompleteMultipartUpload though). I’ll try to put everything in a folder e.g. “tmp” instead of root and give it prefix “tmp/”, then we will see…

    Reply Report
    • joellm December 2, 2019

      The documentation says:

      A string specifying the objects to which the rule will be applied. When provided, only objects whose keys begin with the prefix will be acted upon. If empty or not present, all object in the Space will be affected.

      So, you should be able to omit the prefix entirely.

      But the most important part is that you use a single rule with Expiration & AbortIncompleteMultipartUpload actions, not 2 separate rules.

      Also (I just remembered), for me, I actually had to use an XML configuration file, not JSON.

      For example:

      <LifecycleConfiguration>
    <Rule>
        <ID>MyRuleID123</ID>
        <Status>Enabled</Status>
        <Filter>
          <Prefix>xxx-</Prefix>
        </Filter>
        <AbortIncompleteMultipartUpload>
          <DaysAfterInitiation>2</DaysAfterInitiation>
        </AbortIncompleteMultipartUpload>
        <Expiration>
          <Days>15</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

      Using the command:

      s3cmd setlifecycle /path/to/lifecycle.xml s3://bucket-name
      Reply Report
      • DimL December 3, 2019

        Thank you! Single rule with empty prefix did the trick. Json worked fine for me.

        {
    "Rules": [
        {
            "ID": "Remove expired temporary files",
            "Status": "Enabled",
            "Prefix": "",
            "Expiration": {
                "Days": 1
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 1
            }
        }
    ]
}
        Reply Report
        • akash69711 October 17, 2020

          How did you apply this policy to the bucket?

          Reply Report
          • DimL October 17, 2020

            I used a docker container to do the job, just change variables:

            sudo docker run --rm -it \
    -e "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" \
    -e "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" \
    -v "${LOCAL_LIFICYCLE_JSON_PATH}:/lifecycle.json" \
    mesosphere/aws-cli \
    s3api --endpoint-url="https://${YOUR_REGION}.digitaloceanspaces.com" \
    "put" \
    --bucket "${YOUR_BUCKET_NAME}" \
    --lifecycle-configuration file:///lifecycle.json

            You can confirm it by changing “put” to “get”, it should return the status

            Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help