<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>boto3 | FLRNKS</title><link>https://flrnks.netlify.app/tag/boto3/</link><atom:link href="https://flrnks.netlify.app/tag/boto3/index.xml" rel="self" type="application/rss+xml"/><description>boto3</description><generator>Source Themes Academic (https://sourcethemes.com/academic/)</generator><language>en-us</language><copyright>© 2024</copyright><lastBuildDate>Mon, 03 Feb 2020 11:11:00 +0000</lastBuildDate><image><url>https://flrnks.netlify.app/images/icon_hu0b7a4cb9992c9ac0e91bd28ffd38dd00_9727_512x512_fill_lanczos_center_2.png</url><title>boto3</title><link>https://flrnks.netlify.app/tag/boto3/</link></image><item><title>Identity &amp; Access Management</title><link>https://flrnks.netlify.app/post/aws-iam/</link><pubDate>Mon, 03 Feb 2020 11:11:00 +0000</pubDate><guid>https://flrnks.netlify.app/post/aws-iam/</guid><description>&lt;h2 id="introduction">INTRODUCTION&lt;/h2>
&lt;p>In this post I show how the Identity and Access Management service in the AWS Public Cloud works to secure resources and workloads. It is a very important topic, because it underpins all of the security that is needed for hosting one&amp;rsquo;s resources in the public cloud.&lt;/p>
&lt;p>At the end of the day, the cloud is just a concept that offers a convenient illusion of dedicated resources, but in reality it&amp;rsquo;s just some process that runs on someone else&amp;rsquo;s hardware, so one has to be absolutely sure about security before trusting it and running their business-critical workloads on it.&lt;/p>
&lt;p>It is enough to do a quick google search for
&lt;a href="https://www.google.com/search?q=unsecured%20s3%20bucket" target="_blank" rel="noopener">unsecured s3 bucket&lt;/a> to see plenty of examples of administrators failing to properly harden and configure their AWS resources, and falling victim to accidental disclosure of often business-critical information.&lt;/p>
&lt;p>
&lt;a href="https://docs.aws.amazon.com/iam/?id=docs_gateway" target="_blank" rel="noopener">IAM&lt;/a> exists in the realm of AWS Cloud as a standalone service, providing various ways in which access to resources and workloads can be restricted. For example, if someone has an S3 bucket for storing arbitrary data, one can use IAM policies to restrict access to data stored in the bucket based on various criteria such as user identity, connection source IP, VPC environment and so on. S3 is a convenient service to demonstrate IAM capabilities, because it is very easy to grasp the result of restrictions: access to files in an S3 bucket is either granted or denied.&lt;/p>
&lt;h2 id="how-it-works">HOW IT WORKS&lt;/h2>
&lt;p>In order to illustrate how IAM works, I decided to create a Python Lambda function, which is just an AWS service offering server-less functions, and implemented a routine that tries to access some data stored in a particular S3 bucket. By default the Lambda starts running with an
&lt;a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html" target="_blank" rel="noopener">IAM role&lt;/a> that has only read-only permission to the bucket. This is verified by making an API call with the
&lt;a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html" target="_blank" rel="noopener">boto3&lt;/a> package, which returns without any error. Next the Lambda tries to write some new data to the bucket, but this fails because the IAM role is not equipped with Write permission to the S3 bucket.&lt;/p>
&lt;p>To mitigate this problem, I use boto3 to make an AWS Secure Token Service (
&lt;a href="https://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html" target="_blank" rel="noopener">STS&lt;/a>) call and assume a new role which is equipped with the necessary read-write access. Using this new role the program demonstrates that it can write to the bucket as expected. Below is a sample output of the Lambda Function in action:&lt;/p>
&lt;div class="highlight">&lt;pre class="chroma">&lt;code class="language-yml" data-lang="yml">===&lt;span class="w"> &lt;/span>Checking&lt;span class="w"> &lt;/span>IAM&lt;span class="w"> &lt;/span>Identity&lt;span class="w"> &lt;/span>===&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>&lt;span class="k">ARN&lt;/span>&lt;span class="p">:&lt;/span>&lt;span class="w"> &lt;/span>arn&lt;span class="p">:&lt;/span>aws&lt;span class="p">:&lt;/span>sts&lt;span class="p">::&lt;/span>ACCOUNT_ID&lt;span class="p">:&lt;/span>assumed-role/Base-Lambda-Custom-Role/lambda&lt;span class="w">
&lt;/span>&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>===&lt;span class="w"> &lt;/span>Testing&lt;span class="w"> &lt;/span>Read&lt;span class="w"> &lt;/span>access&lt;span class="w"> &lt;/span>to&lt;span class="w"> &lt;/span>S3&lt;span class="w"> &lt;/span>file&lt;span class="w"> &lt;/span>in&lt;span class="w"> &lt;/span>bucket&lt;span class="w"> &lt;/span>===&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>{&lt;span class="w">
&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">&amp;#34;field1&amp;#34;: &lt;/span>&lt;span class="kc">true&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">&amp;#34;field2&amp;#34;: &lt;/span>&lt;span class="m">1.&lt;/span>4107917E7&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>}&lt;span class="w">
&lt;/span>&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>===&lt;span class="w"> &lt;/span>Testing&lt;span class="w"> &lt;/span>Write&lt;span class="w"> &lt;/span>access&lt;span class="w"> &lt;/span>to&lt;span class="w"> &lt;/span>S3&lt;span class="w"> &lt;/span>bucket&lt;span class="w"> &lt;/span>===&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>&lt;span class="k">Error&lt;/span>&lt;span class="p">:&lt;/span>&lt;span class="w"> &lt;/span>AccessDenied!&lt;span class="w">
&lt;/span>&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>===&lt;span class="w"> &lt;/span>Assumed&lt;span class="w"> &lt;/span>New&lt;span class="w"> &lt;/span>IAM&lt;span class="w"> &lt;/span>Identity&lt;span class="w"> &lt;/span>===&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>&lt;span class="k">ARN&lt;/span>&lt;span class="p">:&lt;/span>&lt;span class="w"> &lt;/span>arn&lt;span class="p">:&lt;/span>aws&lt;span class="p">:&lt;/span>sts&lt;span class="p">::&lt;/span>ACCOUNT_ID&lt;span class="p">:&lt;/span>assumed-role/S3-RW-Role/lambda&lt;span class="w">
&lt;/span>&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>===&lt;span class="w"> &lt;/span>Testing&lt;span class="w"> &lt;/span>Write&lt;span class="w"> &lt;/span>access&lt;span class="w"> &lt;/span>to&lt;span class="w"> &lt;/span>S3&lt;span class="w"> &lt;/span>bucket&lt;span class="w"> &lt;/span>(using&lt;span class="w"> &lt;/span>new&lt;span class="w"> &lt;/span>role)&lt;span class="w"> &lt;/span>===&lt;span class="w">
&lt;/span>&lt;span class="w">&lt;/span>...&lt;span class="w"> &lt;/span>file&lt;span class="w"> &lt;/span>was&lt;span class="w"> &lt;/span>written&lt;span class="w"> &lt;/span>successfully!&lt;span class="w">
&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>To get a better understanding how this all worked in code, feel free to check out the source code repository in Github (
&lt;a href="https://github.com/florianakos/aws-iam-exercise" target="_blank" rel="noopener">link&lt;/a>). Because I am a big fan of Terraform, I defined all resources (S3, IAM, Lambda) in code which makes it very simple and straightforward to deploy and test the code if you feel like!&lt;/p>
&lt;h2 id="advanced-iam">ADVANCED IAM&lt;/h2>
&lt;p>Besides providing the basic functionality to restrict access to resources base on user identity, there are some cool and more advanced features of AWS IAM that I wanted to touch upon. For example, to show how simple it is to give read-only permissions to a bucket for an IAM role:&lt;/p>
&lt;div class="highlight">&lt;pre class="chroma">&lt;code class="language-shell" data-lang="shell">data &lt;span class="s2">&amp;#34;aws_iam_policy_document&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;s3_ro_access_policy_document&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
statement &lt;span class="o">{&lt;/span>
&lt;span class="nv">effect&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;Allow&amp;#34;&lt;/span>
&lt;span class="nv">actions&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span>
&lt;span class="s2">&amp;#34;s3:GetObject&amp;#34;&lt;/span>,
&lt;span class="s2">&amp;#34;s3:ListBucket&amp;#34;&lt;/span>,
&lt;span class="o">]&lt;/span>
&lt;span class="nv">resources&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span>
&lt;span class="s2">&amp;#34;arn:aws:s3:::my-bucket&amp;#34;&lt;/span>,
&lt;span class="s2">&amp;#34;arn:aws:s3:::my-bucket/*&amp;#34;&lt;/span>
&lt;span class="o">]&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;span class="o">}&lt;/span>
resource &lt;span class="s2">&amp;#34;aws_iam_policy&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;s3_ro_access_policy&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
&lt;span class="nv">name&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;S3-ReadOnly-Access&amp;#34;&lt;/span>
&lt;span class="nv">policy&lt;/span> &lt;span class="o">=&lt;/span> data.aws_iam_policy_document.s3_ro_access_policy_document.json
&lt;span class="o">}&lt;/span>
resource &lt;span class="s2">&amp;#34;aws_iam_role_policy_attachment&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;Allow_S3_ReadOnly_Access&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
&lt;span class="nv">role&lt;/span> &lt;span class="o">=&lt;/span> aws_iam_role.aws_custom_role_for_lambda.name
&lt;span class="nv">policy_arn&lt;/span> &lt;span class="o">=&lt;/span> aws_iam_policy.s3_ro_access_policy.arn
&lt;span class="o">}&lt;/span>
resource &lt;span class="s2">&amp;#34;aws_iam_role&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;aws_s3_readwrite_role&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
&lt;span class="nv">name&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;S3-RW-Role&amp;#34;&lt;/span>
&lt;span class="nv">description&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;Role to allow full RW to bucket&amp;#34;&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;/code>&lt;/pre>&lt;/div>&lt;p>Full source code on
&lt;a href="https://github.com/florianakos/aws-iam-exercise/blob/master/terraform/s3.tf" target="_blank" rel="noopener">GitHub&lt;/a>.&lt;/p>
&lt;p>With this short Terraform code, I created a role, and assigned an IAM policy to it, which has RO access to &lt;code>my-bucket&lt;/code> resource in S3. To spice this up a bit, it is possible to add extra restrictions based on various elements of the request context to restrict access based on Source IP for example:&lt;/p>
&lt;div class="highlight">&lt;pre class="chroma">&lt;code class="language-shell" data-lang="shell">data &lt;span class="s2">&amp;#34;aws_iam_policy_document&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;s3_ro_access_policy_document&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
statement &lt;span class="o">{&lt;/span>
&lt;span class="nv">effect&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;Deny&amp;#34;&lt;/span>
&lt;span class="nv">actions&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span>
&lt;span class="s2">&amp;#34;s3:*&amp;#34;&lt;/span>
&lt;span class="o">]&lt;/span>
&lt;span class="nv">resources&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span> &lt;span class="s2">&amp;#34;*&amp;#34;&lt;/span>&lt;span class="o">]&lt;/span>
condition &lt;span class="o">{&lt;/span>
&lt;span class="nb">test&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;IpAddress&amp;#34;&lt;/span>
&lt;span class="nv">variable&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;aws:SourceIp&amp;#34;&lt;/span>
&lt;span class="nv">values&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span> &lt;span class="s2">&amp;#34;192.168.2.0/24&amp;#34;&lt;/span> &lt;span class="o">]&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;/code>&lt;/pre>&lt;/div>&lt;p>All of a sudden, even if the user who makes the request to S3 has correct credentials, but is connecting from a subnet which is outside the one specified above, the request will be &lt;strong>denied&lt;/strong>! This can be very useful for example, when trying restricting access to resources to be possible only from within a corporate network with specific CIDR range.&lt;/p>
&lt;p>One small issue with this source IP restriction is that it can cause issues for certain AWS services that run on behalf of a principal/user. When using the AWS Athena service for example, triggering a query on data stored in S3 means Athena will make S3 API requests on behalf of the user who initiated the Athena query, but will have a source IP address from some Amazon AWS CIDR range and the request will fail. For this purpose, there is an extra condition that can be added to remediate this issue:&lt;/p>
&lt;div class="highlight">&lt;pre class="chroma">&lt;code class="language-shell" data-lang="shell">data &lt;span class="s2">&amp;#34;aws_iam_policy_document&amp;#34;&lt;/span> &lt;span class="s2">&amp;#34;s3_ro_access_policy_document&amp;#34;&lt;/span> &lt;span class="o">{&lt;/span>
statement &lt;span class="o">{&lt;/span>
&lt;span class="nv">effect&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;Deny&amp;#34;&lt;/span>
&lt;span class="nv">actions&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span>
&lt;span class="s2">&amp;#34;s3:*&amp;#34;&lt;/span>
&lt;span class="o">]&lt;/span>
&lt;span class="nv">resources&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span> &lt;span class="s2">&amp;#34;*&amp;#34;&lt;/span>&lt;span class="o">]&lt;/span>
condition &lt;span class="o">{&lt;/span>
&lt;span class="nb">test&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;IpAddress&amp;#34;&lt;/span>
&lt;span class="nv">variable&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;aws:SourceIp&amp;#34;&lt;/span>
&lt;span class="nv">values&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span> &lt;span class="s2">&amp;#34;192.168.2.0/24&amp;#34;&lt;/span> &lt;span class="o">]&lt;/span>
&lt;span class="o">}&lt;/span>
condition &lt;span class="o">{&lt;/span>
&lt;span class="nb">test&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;Bool&amp;#34;&lt;/span>
&lt;span class="nv">variable&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="s2">&amp;#34;aws:ViaAWSService&amp;#34;&lt;/span>
&lt;span class="nv">values&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="o">[&lt;/span> &lt;span class="s2">&amp;#34;false&amp;#34;&lt;/span> &lt;span class="o">]&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;span class="o">}&lt;/span>
&lt;/code>&lt;/pre>&lt;/div>&lt;p>The &lt;code>aws:viaAWSService = false&lt;/code> condition will ensure that this Deny will only take effect when the request context does not come from an AWS Service Endpoint. For additional info on what other possibilities exist that can be used to grant or deny access, please consult the AWS
&lt;a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html" target="_blank" rel="noopener">documentation&lt;/a>.&lt;/p>
&lt;h2 id="conclusion">CONCLUSION&lt;/h2>
&lt;p>In this post I demonstrated how to use the boto3 python package to make AWS IAM and STS calls to access resources in the AWS cloud protected by IAM policies. I also discussed some advanced features of AWS IAM that can help you implement more granular IAM policies and access rights. The linked repository also contains an example which may be run locally and does not need the Lambda function to be created (it still, however, requires the Terraform resources to be deployed).&lt;/p></description></item></channel></rss>