Terraform is a great open-source Infrastructure as Code(IaC) tool. Terraform is used for defining and provisioning the complete infrastructure in the cloud, maintaining and versioning this safely and efficiently.
Sometime ago, I participated of a DevOps/Terraform/AWS exercise where we should deploy a static website using terraform under DevOps best practices in less than 4 hours.
Challenge
" In a few lines create a very simple API in a language of your choice that returns a secret message. The message is very sensitive so you’ll need to figure out how to best store it. We anticipate that we’ll also need to store some data in a relation database in the future, so our application will also need access to RDS though setting up the connection in your application is extra credit.
Once your API is put together, write a Terraform script to provision and upload the API to be hosted in Elastic Beanstalk along with an RDS instance. We are looking for the use of best security practices in setting up both. "
Approach
We deployed a NodeJS Express application able of retrieving a secret from AWS Secret Manager and to connect to a MySQL database.
All mentioned resources should be deployed on AWS using Terraform.
Some of them are:
- AWS Elastic Beanstalk Environment, Application and Versioning
- AWS IAM rules to allow multi-service communication
- AWS S3 Bucket to keep the application code versioning
- AWS Secret Manager to keep a application secret
- AWS VPC/Subnets/Gateway - To allow Public and Private network access of services
- AWS RDS to deploy a private MySQL database
During Terraform deployment, the application code is copied to S3 and then the new version is deployed into the EBS Environment.
Code
All the Terraform scripts can be found in the following repository. https://github.com/mvitor/aws-api-rds-secret
I’m highlighting the most relevant code below.
AWS Elastic Beanstalk Environment, Application and Versioning
AWS Beanstalk Application
Creates an Elastic Beanstalk Application Resource. AWS EBS is a service that allows us to deploy and manage web applications in the AWS cloud without managing the web server infrastructure.
1
2
3
4
|
resource "aws_elastic_beanstalk_application" "app" {
name = "${var.service_name}"
description = "${var.service_description}"
}
|
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_application
AWS Beanstalk Environment
Creates an Elastic Beanstalk Environment Resource. Elastic Beanstalk Environment is a collection of AWS resources running one or more applications versions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
resource "aws_elastic_beanstalk_environment" "app-prod" {
name = "app-prod"
application = "${aws_elastic_beanstalk_application.app.name}"
solution_stack_name = "64bit Amazon Linux 2 v5.4.5 running Node.js 14"
setting {
namespace = "aws:ec2:vpc"
name = "VPCId"
value = "${aws_vpc.main.id}"
}
setting {
namespace = "aws:ec2:vpc"
name = "Subnets"
value = "${aws_subnet.subnet-private-1.id},${aws_subnet.subnet-private-2.id}"
}
setting {
namespace = "aws:ec2:vpc"
name = "AssociatePublicIpAddress"
value = "false"
}
setting {
namespace = "aws:autoscaling:launchconfiguration"
name = "IamInstanceProfile"
value = "app-ec2-rolepf"
}
setting {
namespace = "aws:autoscaling:launchconfiguration"
name = "SecurityGroups"
value = "${aws_security_group.app-prod.id}"
}
setting {
namespace = "aws:autoscaling:launchconfiguration"
name = "EC2KeyName"
value = "${aws_key_pair.app.id}"
}
setting {
namespace = "aws:autoscaling:launchconfiguration"
name = "InstanceType"
value = "t2.micro"
}
setting {
namespace = "aws:elasticbeanstalk:environment"
name = "ServiceRole"
value = "aws-elasticbeanstalk-service-role"
}
setting {
namespace = "aws:ec2:vpc"
name = "ELBScheme"
value = "public"
}
setting {
namespace = "aws:ec2:vpc"
name = "ELBSubnets"
value = "${aws_subnet.subnet-public-1.id},${aws_subnet.subnet-public-2.id}"
}
setting {
namespace = "aws:elb:loadbalancer"
name = "CrossZone"
value = "true"
}
setting {
namespace = "aws:elasticbeanstalk:command"
name = "BatchSize"
value = "30"
}
setting {
namespace = "aws:elasticbeanstalk:command"
name = "BatchSizeType"
value = "Percentage"
}
setting {
namespace = "aws:autoscaling:asg"
name = "Availability Zones"
value = "Any 2"
}
setting {
namespace = "aws:autoscaling:asg"
name = "MinSize"
value = "1"
}
setting {
namespace = "aws:autoscaling:updatepolicy:rollingupdate"
name = "RollingUpdateType"
value = "Health"
}
# LB healthcheck
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "HealthyThreshold" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_healthy_threshold : var.environmentType}"
}
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "UnhealthyThreshold" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_unhealthy_threshold : var.environmentType}"
}
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "Interval" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_interval : var.environmentType}"
}
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "Timeout" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_timeout : var.environmentType}"
}
# Autoscaling group triggers.
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "BreachDuration" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.as_breach_duration : var.environmentType}"
}
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "LowerBreachScaleIncrement" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.as_lower_breach_scale_increment : var.environmentType}"
}
setting {
namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
name = "${var.environmentType == "UseLoadBalancer" ? "LowerThreshold" : "EnvironmentType"}"
value = "${var.environmentType == "UseLoadBalancer" ? var.as_lower_threshold : var.environmentType}"
}
setting {
namespace = "aws:elasticbeanstalk:application:environment"
name = "RDS_USERNAME"
value = "${aws_db_instance.rds-mysqldb.username}"
}
setting {
namespace = "aws:elasticbeanstalk:application:environment"
name = "RDS_PASSWORD"
value = "${aws_db_instance.rds-mysqldb.password}"
}
setting {
namespace = "aws:elasticbeanstalk:application:environment"
name = "RDS_DATABASE"
value = "${aws_db_instance.rds-mysqldb.name}"
}
setting {
namespace = "aws:elasticbeanstalk:application:environment"
name = "RDS_HOSTNAME"
value = "${aws_db_instance.rds-mysqldb.address}"
}
setting {
namespace = "aws:elasticbeanstalk:application:environment"
name = "AWS_REGION"
value = "${var.AWS_REGION}"
}
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_environment
AWS S3 Bucket to keep the application code versioning
First, create a zip of deployment files with terraform ‘archive_file’ utility, then we create a new bucket with this zip file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
data "archive_file" "api_dist_zip" {
type = "zip"
source_dir = "../${path.root}/${var.api_dist}"
output_path = "../${path.root}/${var.api_dist}.zip"
}
resource "aws_s3_bucket" "dist_bucket" {
bucket = "${var.namespace}-elb-dist"
acl = "private"
}
resource "aws_s3_bucket_object" "dist_item" {
key = "${var.env}/dist-${uuid()}"
bucket = "${aws_s3_bucket.dist_bucket.id}"
source = "${var.dist_zip}"
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object
AWS Beanstalk Versioning
Creates an Elastic Beanstalk Application Version Resource. Elastic Beanstalk Version is the service which allow uploading and maintaining our code in the EBS Environment.
1
2
3
4
5
6
7
8
9
10
|
resource "aws_elastic_beanstalk_application_version" "app-prod" {
name = "${var.namespace}-${var.env}-${uuid()}"
application = "My application"
description = "application version created by terraform"
bucket = "${aws_s3_bucket.dist_bucket.id}"
key = "${aws_s3_bucket_object.dist_item.id}"
provisioner "local-exec" {
command = "aws --region ${var.AWS_REGION} elasticbeanstalk update-environment --environment-name app-prod --version-label ${aws_elastic_beanstalk_application_version.app-prod.name}"
}
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_application
AWS Secret Manager to keep the application secret
That secret will be dynamically queried during Webapp Index loading.
1
2
3
4
5
6
7
|
resource "aws_secretsmanager_secret" "mysecretmanager" {
name = "mysecret003"
}
resource "aws_secretsmanager_secret_version" "mysecretmanager" {
secret_id = aws_secretsmanager_secret.mysecretmanager.id
secret_string = "mysecretvalue003"
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret
AWS RDS to deploy a private MySQL database
We need to provide a private MySQL database and connect with the MySQL driver on the NodeJS application.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
resource "aws_db_instance" "rds-mysqldb" {
allocated_storage = 10
engine = "mysql"
engine_version = "5.7"
instance_class = "db.t2.micro"
identifier = "mysqldb"
name = "mydb"
username = "root"
password = "change1intPws"
db_subnet_group_name = "${aws_db_subnet_group.rds-mysqldb.name}"
parameter_group_name = "default.mysql5.7"
multi_az = "false"
vpc_security_group_ids = ["${aws_security_group.rds-mysqldbsg.id}"]
storage_type = "gp2"
backup_retention_period = 30
publicly_accessible = false
deletion_protection = false
skip_final_snapshot = true
tags = {
Name = "rds-appprod"
Type = "course_exam"
}
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance
AWS RDS - Security Groups and Subnets
We need Network Security groups to allow communication through the Private Subnet resources.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
resource "aws_security_group" "rds-mysqldbsg" {
vpc_id = "${aws_vpc.main.id}"
name = "rds-mysqldb-sg"
description = "Allow inbound mysql traffic"
tags = {
Name = "rds-mysqldb"
Type = "course_exam"
}
}
resource "aws_security_group_rule" "allow-mysql" {
type = "ingress"
from_port = 3306
to_port = 3306
protocol = "tcp"
security_group_id = "${aws_security_group.rds-mysqldbsg.id}"
source_security_group_id = "${aws_security_group.app-prod.id}"
}
resource "aws_security_group_rule" "allow-outgoing" {
type = "egress"
from_port = 0
to_port = 0
protocol = "-1"
security_group_id = "${aws_security_group.rds-mysqldbsg.id}"
cidr_blocks = ["0.0.0.0/0"]
}
resource "aws_db_subnet_group" "rds-mysqldb" {
name = "rds-mysqldb"
description = "RDS subnet group"
subnet_ids = ["${aws_subnet.subnet-private-1.id}", "${aws_subnet.subnet-private-2.id}"]
}
|
Terraform resource link:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group
IAM Rules, VPC, NAT and Subnet configuration can be found in the GitHub repository.