AWS S3: Bucket permissions required for successful ingestion or protection of buckets

Scenario

You connected your AWS account to the Clumio portal, but are not able to view all the buckets in your environment

or

You can see the bucket in the environment, but backups fail due to an unauthorized error.

Bucket-Level Policy errors

This may be due to the explicit "Deny" permissions on the bucket-level policy. To resolve this issue, please add the below NotPrincipals to your "Deny" bucket policy, to ensure that these principals have appropriate access to the bucket: 

"ArnNotLike": {
	"aws:PrincipalArn": [
    	"arn:aws:iam::<account_number>:role/clumio/<Clumio_role_id>",
    	"arn:aws:sts::<account_number>:assumed-role/<Clumio_role_id>/daebaksrv-S3Backup"
    	"arn:aws:sts::<account_number>:assumed-role/<Clumio_role_id>/daebaksrv-RestoreTargetCheck",
    	"arn:aws:sts::<account_number>:assumed-role/<Clumio_role_id>/daebaksrv-createS3CustomerSession"
    	"arn:aws:sts::<account_number>:assumed-role/<Clumio_role_id>/Ingestion-ingesttool",
    	"arn:aws:iam::<account_number>:root"
	]
}

Replace the <aws_acct_number> and <Clumio_role_id> with appropriate values. 
The Clumio_role_id is of the format ClumioRole-<aws_account_region>-<token>; The token can be found in the environment details on the Clumio portal.

For example, if you have a policy similar to the one below that limits access to one VPC,

{
  "Version": "2012-10-17",
  "Id": "Policy1415115909153",
  "Statement": [
    {
    "Sid": "Access-to-specific-VPC-only",
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:*",
    "Resource": [
      "arn:aws:s3:::hd-test",
      "arn:aws:s3:::hd-test/*"
      ],
    "Condition": {
    "StringNotEquals": {
        "aws:SourceVpc": "vpc-12345678"
        }
      }
    }
  ]
}

Change it to, 

{
  "Version": "2012-10-17",
  "Id": "Policy1415115909153",
  "Statement": [
    {
    "Sid": "Access-to-specific-VPC-only",
    "Effect": "Deny",
    "NotPrincipal": {
      "AWS": [
        "arn:aws:iam::<AWS_ACCT_NUMBER>:root",
				"arn:aws:iam::<AWS_ACCT_NUMBER>:role/clumio/<CLUMIO_ROLE_ARN>",
				"arn:aws:sts::<AWS_ACCT_NUMBER>:assumed-role/<CLUMIO_ROLE_ARN>/daebaksrv-*"
      ]
      },
    "Action": "s3:*",
    "Resource": [
    	"arn:aws:s3:::hd-test",
    	"arn:aws:s3:::hd-test/*"
    ],
    "Condition": {
      "StringNotEquals": {
        "aws:SourceVpc": "vpc-12345678"
        }
      }
    }
  ]
  

KMS key policy access to encrypted objects

If you have encrypted objects in your bucket, Clumio will skip these objects during the backup process. You must either intentionally exclude these objects or add the Clumio role to your KMS key policy, to successfully process the backups.

For KMS key policies, the default behavior is to deny access unless it explicitly gives a role access. Verify that the Clumio Role has permissions to the key policy by using the following permission block as a guide.

{
	"Sid": "...",
	"Effect": "Allow",
	"Principal": {
		"AWS": [
			"<CLUMIO_ROLE_ARN>"
			]
	},
	"Action": [
		"kms:Decrypt",
		"kms:GenerateDataKey*"
	],
	"Resource": "*"
}

If you have already added the Clumio role to the key policy and a "Deny" policy exists which prevents access, add the ArnNotLike exception to the Clumio role using the following as a guide.

"ArnNotLike": {
	"aws:PrincipalArn": [
		"arn:aws:iam::<AWS_ACCT_NUMBER>:role/clumio/<CLUMIO_ROLE_ARN>",
		"arn:aws:sts::<AWS_ACCT_NUMBER>:assumed-role/<CLUMIO_ROLE_ARN>/daebaksrv-*"
	]
}

Object and Bucket owner mismatch

Another possible issue that you may encounter is an 'Object and Bucket Owner Mismatch' error. If the bucket owner does not have ownership of the objects, then backups will continue to fail with the 'Access Denied status code: 403' message followed by objects skipped during the backup operation. The most likely cause is due to an ACL applied to the bucket. To easily avoid failing backups, remove the ACL as defaulted by AWS in the following excerpt.

ACLs disabled

• Bucket owner enforced (default)– ACLs are disabled, and the bucket owner automatically owns and has full control over every object in the bucket. ACLs no longer affect permissions to data in the S3 bucket. The bucket uses policies to define access control.

ACLs enabled

• Bucket owner preferred– The bucket owner owns and has full control over new objects that other accounts write to the bucket with thebucket-owner-full-controlcanned ACL.
  Object writer (default)– The Amazon Web Services account that uploads an object owns the object, has full control over it, and can grant other users access to it through ACLs.

If you continue to experience backup failures after using this article as a guide, contact [email protected] for assistance.