Skip to content

Commit 1304497

Browse files
author
Ruslan Korniichuk
committed
Initial commit
0 parents  commit 1304497

18 files changed

+319
-0
lines changed

‎LICENSE

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
This is free and unencumbered software released into the public domain.
2+
3+
Anyone is free to copy, modify, publish, use, compile, sell, or
4+
distribute this software, either in source code form or as a compiled
5+
binary, for any purpose, commercial or non-commercial, and by any
6+
means.
7+
8+
In jurisdictions that recognize copyright laws, the author or authors
9+
of this software dedicate any and all copyright interest in the
10+
software to the public domain. We make this dedication for the benefit
11+
of the public at large and to the detriment of our heirs and
12+
successors. We intend this dedication to be an overt act of
13+
relinquishment in perpetuity of all present and future rights to this
14+
software under copyright law.
15+
16+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19+
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
20+
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
21+
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22+
OTHER DEALINGS IN THE SOFTWARE.
23+
24+
For more information, please refer to <https://unlicense.org>

‎README.md

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# ci-cd-pipeline-on-aws-for-python
2+
**Name:** ci-cd-pipeline-on-aws-for-python
3+
**Description:** AWS CloudFormation template for CI/CD pipeline on AWS for Python
4+
**GitHub:** https://github.com/korniichuk/ci-cd-pipeline-on-aws-for-python
5+
6+
## Table of Contents
7+
* **[Intro](#intro)**
8+
* **[Create CI/CD pipeline. CLI](#create-cicd-pipeline-cli)**
9+
* **[Create CI/CD pipeline. Console](#create-cicd-pipeline-console)**
10+
* **[Validate created pipeline](#validate-created-pipeline)**
11+
12+
## Intro
13+
This repository includes AWS CloudFormation template for CI/CD pipeline on AWS for Python. CI/CD pipeline based on [AWS CodeCommit](https://aws.amazon.com/codecommit/) (à la GitHub), [AWS CodeBuild](https://aws.amazon.com/codebuild/) (à la Jenkins), and [AWS CodePipeline](https://aws.amazon.com/codepipeline/) (continuous delivery) services. You can see visualization below:
14+
![aws_code_services_-_mvp.jpg](img/aws_code_services_-_mvp.jpg "AWS Code services. MVP")
15+
16+
CI/CD pipeline validates your Python code against coding style (aka [PEP8](https://www.python.org/dev/peps/pep-0008/)), programming errors, and [cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity). CI/CD pipeline runs your unit tests. Solution based on [Flake8](https://flake8.pycqa.org/en/latest/) and [Pytest](https://docs.pytest.org/en/stable/) libraries. For more details see `buildspec.yml` file in [ci_cd_pipeline_init_code_20200827.zip](https://s3-eu-west-1.amazonaws.com/korniichuk.share/code/ci_cd_pipeline_init_code_20200827.zip) archive.
17+
18+
## Create CI/CD pipeline. CLI
19+
In a terminal, we can enter the following command:
20+
```
21+
$ aws cloudformation create-stack --stack-name <value> --template-body file://<value> --capabilities CAPABILITY_NAMED_IAM
22+
```
23+
24+
Example:
25+
```
26+
$ git clone https://github.com/korniichuk/ci-cd-pipeline-on-aws-for-python.git
27+
$ cd ci-cd-pipeline-on-aws-for-python
28+
$ aws cloudformation create-stack --stack-name medium --template-body file://ci_cd_pipeline.yaml --capabilities CAPABILITY_NAMED_IAM
29+
```
30+
31+
Example output:
32+
```
33+
{
34+
"StackId": "arn:aws:cloudformation:eu-west-1:999999999999:stack/medium/4576d190-e731-11ea-925a-0ab221334260"
35+
}
36+
```
37+
38+
**Source:** https://docs.aws.amazon.com/cli/latest/reference/cloudformation/create-stack.html
39+
40+
## Create CI/CD pipeline. Console
41+
Navigate to https://console.aws.amazon.com/cloudformation/. Click `Create stack`:
42+
![aws_cloudformation.png](img/aws_cloudformation.png "AWS CloudFormation")
43+
44+
Select `Upload a template file`. Click `Choose file` and select AWS CloudFormation template from your local machine (e.g. [ci_cd_pipeline.yaml](ci_cd_pipeline.yaml)):
45+
![aws_cloudformation_-_specify_template.png](img/aws_cloudformation_-_specify_template.png "AWS CloudFormation. Specify template")
46+
47+
Click `Next`:
48+
![aws_cloudformation_-_specify_template_next.png](img/aws_cloudformation_-_specify_template_next.png "AWS CloudFormation. Specify template. Next")
49+
50+
Enter `Stack name` (e.g. `medium`). Click `Next`:
51+
![aws_cloudformation_-_stack_name.png](img/aws_cloudformation_-_stack_name.png "AWS CloudFormation. Stack name")
52+
53+
Scroll down. Click `Next`:
54+
![aws_cloudformation_-_advanced_options_next.png](img/aws_cloudformation_-_advanced_options_next.png "AWS CloudFormation. Advanced options. Next")
55+
56+
Scroll down. Select `I acknowledge that AWS CloudFormation might create IAM resources with custom names.` checkbox. Click `Create stack`:
57+
![aws_cloudformation_-_create_stack.png](img/aws_cloudformation_-_create_stack.png "AWS CloudFormation. Create stack")
58+
59+
## Validate created pipeline
60+
Navigate to https://console.aws.amazon.com/cloudformation/. Click `Stacks` and validate status of your stack:
61+
![aws_cloudformation_-_stacks.png](img/aws_cloudformation_-_stacks.png "AWS CloudFormation. Stacks")
62+
63+
Navigate to https://console.aws.amazon.com/codepipeline/. Validate status of your pipeline:
64+
![aws_codepipeline_-_pipelines.png](img/aws_codepipeline_-_pipelines.png "AWS CodePipeline. Pipelines")
65+
66+
Navigate to https://console.aws.amazon.com/codecommit/. Copy `URL` of your repository:
67+
![aws_codecommit_-_repositories.png](img/aws_codecommit_-_repositories.png "AWS CodeCommit. Repositories")
68+
69+
In a terminal, we need to enter the following command:
70+
```
71+
$ git clone URL
72+
$ cd demo-ci-cd-pipeline
73+
```
74+
75+
Where:
76+
- `URL` -- URL of your repository from the previous step.
77+
78+
Example:
79+
80+
```
81+
$ git clone https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/demo-ci-cd-pipeline
82+
$ cd demo-ci-cd-pipeline
83+
```
84+
85+
Next we need to change our Python code and send update to AWS CodeCommit repository. For example, you can change code of `sample.py` file to new code with typo:
86+
```
87+
# -*- coding: utf-8 -*-
88+
# Name: sample
89+
# Version: 0.1a2
90+
# Owner: Ruslan Korniichuk
91+
92+
def hello_world():
93+
return 'Hello, World1' # Typo is here
94+
```
95+
96+
Commit changes and send to remote repo:
97+
```
98+
$ git commit -am "Test failure"
99+
$ git push origin master
100+
```
101+
102+
Navigate to https://console.aws.amazon.com/codepipeline/. **Wait a few minutes.** You can expect `In progress` status first:
103+
![aws_codepipeline_-_pipeline_in_progress.png](img/aws_codepipeline_-_pipeline_in_progress.png "AWS CodePipeline. Pipeline in progress")
104+
105+
Wait a few minutes more. You can expect `Failed` status:
106+
![aws_codepipeline_-_pipeline_failed.png](img/aws_codepipeline_-_pipeline_failed.png "AWS CodePipeline. Pipeline failed")
107+
108+
Select your pipeline and click `Details` in red `Build` block:
109+
![aws_codepipeline_-_build_details.png](img/aws_codepipeline_-_build_details.png "AWS CodePipeline. Build details")
110+
111+
Click `Link to execution details`:
112+
![aws_codepipeline_-_link_to_exec_details.png](img/aws_codepipeline_-_link_to_exec_details.png "AWS CodePipeline. Link to execution details")
113+
114+
Finally, you can see reason of failure. Our new Python code with typo cannot pass unit tests:
115+
![aws_codepipeline_-_build_failure_reason.png](img/aws_codepipeline_-_build_failure_reason.png "AWS CodePipeline. Build failure reason")

‎ci_cd_pipeline.yaml

+180
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
AWSTemplateFormatVersion: "2010-09-09"
2+
Description: AWS CloudFormation template for CI/CD pipeline on AWS for Python
3+
Resources:
4+
5+
CodeBuildIAMRole:
6+
Type: AWS::IAM::Role
7+
Properties:
8+
RoleName: demo-ci-cd-pipeline-codebuild-role
9+
AssumeRolePolicyDocument:
10+
Version: "2012-10-17"
11+
Statement:
12+
-
13+
Effect: "Allow"
14+
Principal:
15+
Service:
16+
- "codebuild.amazonaws.com"
17+
Action:
18+
- "sts:AssumeRole"
19+
Path: "/service-role/"
20+
Policies:
21+
-
22+
PolicyName: demo-ci-cd-pipeline-codebuild-policy
23+
PolicyDocument:
24+
Version: "2012-10-17"
25+
Statement:
26+
-
27+
Effect: "Allow"
28+
Action:
29+
- "s3:GetBucketAcl"
30+
- "s3:GetBucketLocation"
31+
Resource:
32+
- !GetAtt CodePipelineArtifacts.Arn
33+
-
34+
Effect: "Allow"
35+
Action:
36+
- "s3:PutObject"
37+
- "s3:GetObject"
38+
- "s3:GetObjectVersion"
39+
Resource:
40+
- !Sub ${CodePipelineArtifacts.Arn}/*
41+
-
42+
Effect: "Allow"
43+
Action:
44+
- "logs:CreateLogGroup"
45+
- "logs:CreateLogStream"
46+
- "logs:PutLogEvents"
47+
Resource:
48+
- !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/demo-ci-cd-pipeline*
49+
- !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/demo-ci-cd-pipeline:*
50+
51+
CodePipelineIAMRole:
52+
Type: AWS::IAM::Role
53+
Properties:
54+
RoleName: demo-ci-cd-pipeline-codepipeline-role
55+
AssumeRolePolicyDocument:
56+
Version: "2012-10-17"
57+
Statement:
58+
-
59+
Effect: "Allow"
60+
Principal:
61+
Service:
62+
- "codepipeline.amazonaws.com"
63+
Action:
64+
- "sts:AssumeRole"
65+
Path: "/service-role/"
66+
Policies:
67+
-
68+
PolicyName: demo-ci-cd-pipeline-codepipeline-policy
69+
PolicyDocument:
70+
Version: "2012-10-17"
71+
Statement:
72+
-
73+
Effect: "Allow"
74+
Action:
75+
- "s3:ListBucket"
76+
- "s3:GetBucketPolicy"
77+
Resource:
78+
- !GetAtt CodePipelineArtifacts.Arn
79+
-
80+
Effect: "Allow"
81+
Action:
82+
- "s3:PutObject"
83+
- "s3:GetObject"
84+
- "s3:GetObjectVersion"
85+
- "s3:DeleteObject"
86+
Resource:
87+
- !Sub ${CodePipelineArtifacts.Arn}/*
88+
-
89+
Effect: "Allow"
90+
Action:
91+
- "codecommit:CancelUploadArchive"
92+
- "codecommit:GetBranch"
93+
- "codecommit:GetCommit"
94+
- "codecommit:GetUploadArchiveStatus"
95+
- "codecommit:UploadArchive"
96+
Resource: !GetAtt CodeCommitRepository.Arn
97+
-
98+
Effect: "Allow"
99+
Action:
100+
- "codebuild:BatchGetBuilds"
101+
- "codebuild:StartBuild"
102+
Resource:
103+
- !GetAtt CodeBuildProject.Arn
104+
- !Sub arn:aws:codebuild:${AWS::Region}:${AWS::AccountId}:build/demo-ci-cd-pipeline:*
105+
106+
CodePipelineArtifacts:
107+
Type: AWS::S3::Bucket
108+
109+
CodeCommitRepository:
110+
Type: AWS::CodeCommit::Repository
111+
Properties:
112+
RepositoryName: demo-ci-cd-pipeline
113+
RepositoryDescription: CI/CD pipeline on AWS for Python
114+
Code:
115+
S3:
116+
Bucket: korniichuk.share
117+
Key: code/ci_cd_pipeline_init_code_20200827.zip
118+
119+
CodeBuildProject:
120+
Type: AWS::CodeBuild::Project
121+
Properties:
122+
Name: demo-ci-cd-pipeline
123+
Source:
124+
Type: CODEPIPELINE
125+
Artifacts:
126+
Type: CODEPIPELINE
127+
Environment:
128+
Type: LINUX_CONTAINER
129+
Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0 # https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html
130+
ComputeType: BUILD_GENERAL1_SMALL
131+
ServiceRole: !GetAtt CodeBuildIAMRole.Arn
132+
TimeoutInMinutes: 15
133+
QueuedTimeoutInMinutes: 60
134+
LogsConfig:
135+
CloudWatchLogs:
136+
Status: ENABLED
137+
138+
CodePipelinePipeline:
139+
Type: AWS::CodePipeline::Pipeline
140+
Properties:
141+
Name: demo-ci-cd-pipeline
142+
ArtifactStore:
143+
Location: !Ref CodePipelineArtifacts
144+
Type: S3
145+
RoleArn: !GetAtt CodePipelineIAMRole.Arn
146+
Stages:
147+
-
148+
Name: Source
149+
Actions:
150+
-
151+
Name: Source
152+
ActionTypeId:
153+
Category: Source
154+
Owner: AWS
155+
Provider: CodeCommit
156+
Version: '1'
157+
Configuration:
158+
RepositoryName: demo-ci-cd-pipeline
159+
BranchName: master
160+
PollForSourceChanges: true
161+
OutputArtifacts:
162+
-
163+
Name: SourceOutput
164+
RunOrder: 1
165+
-
166+
Name: Build
167+
Actions:
168+
-
169+
Name: Build
170+
InputArtifacts:
171+
-
172+
Name: SourceOutput
173+
ActionTypeId:
174+
Category: Build
175+
Owner: AWS
176+
Provider: CodeBuild
177+
Version: '1'
178+
Configuration:
179+
ProjectName: !Ref CodeBuildProject
180+
RunOrder: 1

‎img/aws_cloudformation.png

332 KB
Loading

‎img/aws_cloudformation_-_advanced_options_next.png

116 KB
Loading

‎img/aws_cloudformation_-_create_stack.png

105 KB
Loading

‎img/aws_cloudformation_-_specify_template.png

112 KB
Loading

‎img/aws_cloudformation_-_specify_template_next.png

120 KB
Loading

‎img/aws_cloudformation_-_stack_name.png

80.3 KB
Loading

‎img/aws_cloudformation_-_stacks.png

97.3 KB
Loading

‎img/aws_code_services_-_mvp.jpg

64.1 KB
Loading

‎img/aws_codecommit_-_repositories.png

106 KB
Loading

‎img/aws_codepipeline_-_build_details.png

125 KB
Loading

‎img/aws_codepipeline_-_build_failure_reason.png

193 KB
Loading

‎img/aws_codepipeline_-_link_to_exec_details.png

19.5 KB
Loading

‎img/aws_codepipeline_-_pipeline_failed.png

92.8 KB
Loading

‎img/aws_codepipeline_-_pipeline_in_progress.png

93.6 KB
Loading

‎img/aws_codepipeline_-_pipelines.png

96.7 KB
Loading

0 commit comments

Comments
 (0)