mvitor.com
Posts Tags Categories Projects About English
mvitor.com

Contents

Terraform - Deploying serverless Nodejs Web application, Secret Mgmt and RDS database in AWS using Terraform

Marcio Vitor included in Projects Automation
   1518 words   8 minutes    views  
Views
/2021/08/10/terraform-uploading-static-content-to-cloud/featured-image.jpg

Terraform is a great open-source Infrastructure as Code(IaC) tool. Terraform is used for defining and provisioning the complete infrastructure in the cloud, maintaining and versioning this safely and efficiently.

Sometime ago, I participated of a DevOps/Terraform/AWS exercise where we should deploy a static website using terraform under DevOps best practices in less than 4 hours.

Challenge

" In a few lines create a very simple API in a language of your choice that returns a secret message. The message is very sensitive so you’ll need to figure out how to best store it. We anticipate that we’ll also need to store some data in a relation database in the future, so our application will also need access to RDS though setting up the connection in your application is extra credit.

Once your API is put together, write a Terraform script to provision and upload the API to be hosted in Elastic Beanstalk along with an RDS instance. We are looking for the use of best security practices in setting up both. "

Approach

We deployed a NodeJS Express application able of retrieving a secret from AWS Secret Manager and to connect to a MySQL database.

All mentioned resources should be deployed on AWS using Terraform.

Some of them are:

  • AWS Elastic Beanstalk Environment, Application and Versioning
  • AWS IAM rules to allow multi-service communication
  • AWS S3 Bucket to keep the application code versioning
  • AWS Secret Manager to keep a application secret
  • AWS VPC/Subnets/Gateway - To allow Public and Private network access of services
  • AWS RDS to deploy a private MySQL database

During Terraform deployment, the application code is copied to S3 and then the new version is deployed into the EBS Environment.

Code

All the Terraform scripts can be found in the following repository. https://github.com/mvitor/aws-api-rds-secret I’m highlighting the most relevant code below.

AWS Elastic Beanstalk Environment, Application and Versioning

AWS Beanstalk Application

Creates an Elastic Beanstalk Application Resource. AWS EBS is a service that allows us to deploy and manage web applications in the AWS cloud without managing the web server infrastructure.

1
2
3
4

resource "aws_elastic_beanstalk_application" "app" {
  name        = "${var.service_name}"
  description = "${var.service_description}"
}

Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_application

AWS Beanstalk Environment

Creates an Elastic Beanstalk Environment Resource. Elastic Beanstalk Environment is a collection of AWS resources running one or more applications versions.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

resource "aws_elastic_beanstalk_environment" "app-prod" {
  name = "app-prod"
  application = "${aws_elastic_beanstalk_application.app.name}"
  solution_stack_name = "64bit Amazon Linux 2 v5.4.5 running Node.js 14"
  setting {
    namespace = "aws:ec2:vpc"
    name      = "VPCId"
    value     = "${aws_vpc.main.id}"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "Subnets"
    value = "${aws_subnet.subnet-private-1.id},${aws_subnet.subnet-private-2.id}"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "AssociatePublicIpAddress"
    value = "false"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "IamInstanceProfile"
    value = "app-ec2-rolepf"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "SecurityGroups"
    value = "${aws_security_group.app-prod.id}"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "EC2KeyName"
    value = "${aws_key_pair.app.id}"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "InstanceType"
    value = "t2.micro"
  }
  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name = "ServiceRole"
    value = "aws-elasticbeanstalk-service-role"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "ELBScheme"
    value = "public"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name = "ELBSubnets"
    value = "${aws_subnet.subnet-public-1.id},${aws_subnet.subnet-public-2.id}"
  }
  setting {
    namespace = "aws:elb:loadbalancer"
    name = "CrossZone"
    value = "true"
  }
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name = "BatchSize"
    value = "30"
  }
  setting {
    namespace = "aws:elasticbeanstalk:command"
    name = "BatchSizeType"
    value = "Percentage"
  }
  setting {
    namespace = "aws:autoscaling:asg"
    name = "Availability Zones"
    value = "Any 2"
  }
  setting {
    namespace = "aws:autoscaling:asg"
    name = "MinSize"
    value = "1"
  }
  setting {
    namespace = "aws:autoscaling:updatepolicy:rollingupdate"
    name = "RollingUpdateType"
    value = "Health"
  }
  # LB healthcheck
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "HealthyThreshold" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_healthy_threshold : var.environmentType}"
  }
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "UnhealthyThreshold" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_unhealthy_threshold : var.environmentType}"
  }
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "Interval" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_interval : var.environmentType}"
  }
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:elb:healthcheck" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "Timeout" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.healthcheck_timeout : var.environmentType}"
  }
  # Autoscaling group triggers.
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "BreachDuration" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.as_breach_duration : var.environmentType}"
  }
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "LowerBreachScaleIncrement" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.as_lower_breach_scale_increment : var.environmentType}"
  }
  setting {
    namespace = "${var.environmentType == "UseLoadBalancer" ? "aws:autoscaling:trigger" : "aws:elasticbeanstalk:environment"}"
    name      = "${var.environmentType == "UseLoadBalancer" ? "LowerThreshold" : "EnvironmentType"}"
    value     = "${var.environmentType == "UseLoadBalancer" ? var.as_lower_threshold : var.environmentType}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_USERNAME"
    value = "${aws_db_instance.rds-mysqldb.username}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_PASSWORD"
    value = "${aws_db_instance.rds-mysqldb.password}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_DATABASE"
    value = "${aws_db_instance.rds-mysqldb.name}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "RDS_HOSTNAME"
    value = "${aws_db_instance.rds-mysqldb.address}"
  }
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name = "AWS_REGION"
    value = "${var.AWS_REGION}"
  }
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_environment

AWS S3 Bucket to keep the application code versioning

First, create a zip of deployment files with terraform ‘archive_file’ utility, then we create a new bucket with this zip file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

data "archive_file" "api_dist_zip" {
  type        = "zip"
  source_dir = "../${path.root}/${var.api_dist}"
  output_path = "../${path.root}/${var.api_dist}.zip"
}
resource "aws_s3_bucket" "dist_bucket" {
  bucket = "${var.namespace}-elb-dist"
  acl    = "private"
}
resource "aws_s3_bucket_object" "dist_item" {
  key    = "${var.env}/dist-${uuid()}"
  bucket = "${aws_s3_bucket.dist_bucket.id}"
  source = "${var.dist_zip}"
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object

AWS Beanstalk Versioning

Creates an Elastic Beanstalk Application Version Resource. Elastic Beanstalk Version is the service which allow uploading and maintaining our code in the EBS Environment.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

resource "aws_elastic_beanstalk_application_version" "app-prod" {
  name        = "${var.namespace}-${var.env}-${uuid()}"
  application = "My application"
  description = "application version created by terraform"
  bucket      = "${aws_s3_bucket.dist_bucket.id}"
  key         = "${aws_s3_bucket_object.dist_item.id}"
  provisioner "local-exec" {
    command = "aws --region ${var.AWS_REGION} elasticbeanstalk update-environment --environment-name app-prod --version-label ${aws_elastic_beanstalk_application_version.app-prod.name}"
  }
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_application

AWS Secret Manager to keep the application secret

That secret will be dynamically queried during Webapp Index loading.

1
2
3
4
5
6
7

resource "aws_secretsmanager_secret" "mysecretmanager" {
  name = "mysecret003"
}
resource "aws_secretsmanager_secret_version" "mysecretmanager" {
  secret_id     = aws_secretsmanager_secret.mysecretmanager.id
  secret_string = "mysecretvalue003"
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret

AWS RDS to deploy a private MySQL database

We need to provide a private MySQL database and connect with the MySQL driver on the NodeJS application. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

resource "aws_db_instance" "rds-mysqldb" {
  allocated_storage    = 10
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.micro"
  identifier           = "mysqldb"
  name                 = "mydb"
  username             = "root"
  password             = "change1intPws"
  db_subnet_group_name = "${aws_db_subnet_group.rds-mysqldb.name}"
  parameter_group_name = "default.mysql5.7"
  multi_az             = "false"
  vpc_security_group_ids = ["${aws_security_group.rds-mysqldbsg.id}"]
  storage_type         = "gp2"
  backup_retention_period = 30
  publicly_accessible =  false
  deletion_protection = false
  skip_final_snapshot = true
  tags = {
      Name = "rds-appprod"
      Type = "course_exam"
  }
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance

AWS RDS - Security Groups and Subnets

We need Network Security groups to allow communication through the Private Subnet resources. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

resource "aws_security_group" "rds-mysqldbsg" {
  vpc_id = "${aws_vpc.main.id}"
  name = "rds-mysqldb-sg"
  description = "Allow inbound mysql traffic"
  tags = {
    Name = "rds-mysqldb"
    Type = "course_exam"

  }
}
resource "aws_security_group_rule" "allow-mysql" {
    type = "ingress"
    from_port = 3306
    to_port = 3306
    protocol = "tcp"
    security_group_id = "${aws_security_group.rds-mysqldbsg.id}"
    source_security_group_id = "${aws_security_group.app-prod.id}"
}
resource "aws_security_group_rule" "allow-outgoing" {
    type = "egress"
    from_port = 0
    to_port = 0
    protocol = "-1"
    security_group_id = "${aws_security_group.rds-mysqldbsg.id}"
    cidr_blocks = ["0.0.0.0/0"]
}
resource "aws_db_subnet_group" "rds-mysqldb" {
    name = "rds-mysqldb"
    description = "RDS subnet group"
    subnet_ids = ["${aws_subnet.subnet-private-1.id}", "${aws_subnet.subnet-private-2.id}"]
}
Terraform resource link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group

IAM Rules, VPC, NAT and Subnet configuration can be found in the GitHub repository.

Updated on 2021-12-20  2fd67f0
Read Markdown
 aws, terraform, serverless, upload, ebs, elasticbeanstalk, webapp, content, nodejs, javascript, nodejs, devops, deploy, nodejsexpress
Back | Home