This week I will explain the AWS S3 buckets details. In my previous post I explained the fundamentals of S3 and created a sample bucket and object. We use S3 buckets to store our objects. Buckets have properties like permissions, versioning, life cycling etc. The most important thing about the buckets are bucket permission and today’s post will cover this subject.
Ok let’s start…
Bucket permission:
First of all I should say that Amazon has three S3 permission sets.
Bucket policies: Attached to bucket , json based
ACLS: Grant access to specific AWS account or anonymous
IAM : Attached to user , json based
Bucket policies and IAM policies gives us fine level of control. ACLs are used for general permissions( high-level).
First of all I will explain ACLs (bucket permissions). Basically, ACLs define the grantee and the actions it can take on the bucket. When we create a bucket by default all actions permitted to our root account. We can add or remove additional permissions and can use AWS built-in groups as the grantee.
AWS built-in groups:
- Authenticated Users – This should be an AWS account’s either email address or canonical user ID.
- Everyone – For anonymous access
- Log Delivery – This group is used if you enabled logging on your bucket.
Bucket permissions:
- List – Permission to view a list of the objects in the bucket.
- Upload/Delete – Permission to upload and delete the object if the grantee is logged in
- View Permissions – Permission to see the permissions for objects
- Edit Permissions – Permission to edit the permissions for objects
So if we want a public bucket that has only permission to view the object, we should add more permission and use “Everyone” as grantee and “List” as the action.
Ok now let’s make some tests with ACL. First of all I will create two buckets named “wekanban.demo.s3” and “wekanban.everyone.access” as shown below.
For “wekanban.demo.s3” bucket I will give permission “List” ( read ) and as a grantee I will use another account’s canonical user ID.
For “wekanban.everyone.access” I will give permission “List” again and as a grantee I will use “Everyone”.
Let’s test and see the result using AWS S3 cli ( I won’t explain how to setup aws cli tool but you can find it here ) :
I try to access to the permitted bucket with a permitted account and a denied account:
aws s3 ls s3://wekanban.demo.s3 --profile permittedaccount 2014-07-09 09:49:29 4 s3demo.txt aws s3 ls s3://wekanban.demo.s3 --profile deniededaccount A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied
Now I try to access to anonymous bucket:
aws s3 ls s3://wekanban.everyone.access --profile deniedaccount 2014-07-09 09:49:37 4 s3demo.txt
As you see, ACLs gives us to configure permissions at a high level. You can also try to give upload/delete permission and test the result again.
Now we can talk about the bucket policies. As I mentioned before, bucket policies give us fine level of control. For example, we can permit that an ACL can be attached to bucket or restore an object in the bucket. To create a bucket policy, we will use AWS Policy generator.
In S3 console, when I click the “Add bucket policy” , I can paste my policy or use the policy generator. Let’s use the generator.
In my first example, I will give everyone to view the ACL associated with my “wekanban.demo.s3” bucket. To do this I will create a policy as shown below:
The settings for the policy:
Select type of policy: S3 Bucket Policy
Effect: Allow
Principal: * ( Everyone )
AWS Service: S3
Actions: GetBucketAcl ( I want everyone can list my bucket’s ACL )
ARN: arn:aws:s3:::wekbanban.demo.s3
If I click “Add Statement” and then “Generate Policy”, policy generator will create my json based policy.
My bucket policy:
{
"Id": "Policy1404965986822",
"Statement": [
{
"Sid": "Stmt1404965817445",
"Action": [
"s3:GetBucketAcl"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::wekanban.demo.s3",
"Principal": {
"AWS": [
"*"
]
}
}
]
}
Now let’s try if it works. Now I will use a basic python script to test. I use another account’s key and secret in my code to test if policy works.
This is my code:
import boto
key="awskey"
secret="awssecret"
s3conn = boto.connect_s3(key,secret)
bucket=s3conn.get_bucket("wekanban.demo.s3")
print "bucket shared with my account: ",bucket.name
list_acl = bucket.get_acl()
print "ACL list: ",list_acl
for grant in list_acl.acl.grants:
print grant.permission, grant.display_name
And the result as expected, I can list the ACL associated with the bucket even if I am not the user.
bucket shared with my account: wekanban.demo.s3 ACL list: READ myuser WRITE myuser READ_ACP myuser WRITE_ACP myuser READ other_account
Now let’s change it and give permission to list the bucket tags: Again I click “Add bucket policy” and change the policy.
{
"Id": "Policy1404967883586",
"Statement": [
{
"Sid": "Stmt1404967881890",
"Action": [
"s3:GetBucketTagging"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::wekanban.demo.s3",
"Principal": {
"AWS": [
"*"
]
}
}
]
}
I will create two tags as shown below ( by the way tags are used for billing and you can track your costs by using tags ):
I will also change my script.
import boto
key="awskey"
secret="awssecret"
s3conn = boto.connect_s3(key,secret)
bucket=s3conn.get_bucket("wekanban.demo.s3")
print "bucket shared with my account: ",bucket.name
list_tags = bucket.get_tags()
for tags in list_tags:
for mytag in tags:
print "Key:",mytag.key," Value:",mytag.value
And test again:
bucket shared with my account: wekanban.demo.s3 Key: Name Value: demo Key: Env Value: Development
So our last step is giving permission based on IAM. There are different scenarios we can have but in my example I will give permission to an IAM user of other account again. First of all, I will delete all ACL and bucket policy created for other account. Then I will create an IAM user named “s3test” and give him to list and put objects in my “wekanban.demo.s3” bucket. In order to do this, I have to create a bucket policy for “wekanban.demo.s3” and an IAM policy for “s3test”. Let’s start…
IAM user of other account:
I’ve created it and the arn of the user is arn:aws:iam::111122223333:user/s3test
The policy attached to the “s3test” user: It gives the list permission on “wekanban.demo.s3” bucket and put permission in it as “wekanban.demo.s3/*”
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1404970427000",
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::wekanban.demo.s3/*"
]
},
{
"Sid": "Stmt1404970443000",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::wekanban.demo.s3"
]
}
]
}
Bucket policy:
Here again I give permission for the “s3test” user: As you see I use the same resources for list and put actions.
{
"Version": "2008-10-17",
"Id": "Policy1404969578022",
"Statement": [
{
"Sid": "Stmt1404969562571",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/s3test"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::wekanban.demo.s3"
},
{
"Sid": "Stmt1404969576654",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/s3test"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::wekanban.demo.s3/*"
}
]
}
Ok it is time to test now. Again I will use AWS S3 cli.
First I will try to list the objects in the bucket.
aws s3 ls s3://wekanban.demo.s3 --profile s3test 2014-07-10 08:34:42 127 c.py 2014-07-10 08:36:35 30 demo_file 2014-07-09 16:59:34 5849 mem.txt 2014-07-09 09:49:29 4 s3demo.txt
Now I will try to put an object.
aws s3 cp it_works_on_my_computer.txt s3://wekanban.demo.s3 --profile s3test upload: ./it_works_on_my_computer.txt to s3://wekanban.demo.s3/it_works_on_my_computer.txt
And list again.
aws s3 ls s3://wekanban.demo.s3 --profile s3test 2014-07-10 08:45:29 9 c.py 2014-07-10 08:36:35 30 demo_file 2014-07-10 08:46:19 9 it_works_on_my_computer.txt 2014-07-09 16:59:34 5849 mem.txt 2014-07-09 09:49:29 4 s3demo.txt
If I use an account not permitted to list and put , I will get error as shown below.
aws s3 ls s3://wekanban.demo.s3 --profile deniedaccount A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied
Ok that was the S3 bucket permissions. If you have any question or comment, please feel free to write and don’t forget to share please.
You can also check Udemy’s s3 course if you want.
