Now that you understand how the Reference Architecture works, the next step is to begin migrating your apps to it. The
process we recommend is as follows:
Most "big bang" migrations—where you try to pick up everything and move it over to a new cloud (AWS), new tools
(Terraform, Docker, Packer), and new processes (CI, CD, DevOps), all in one huge step—almost always fail. The only way
to succeed at a big migration is to do it incrementally. Note that "incrementally" doesn't mean you just chop up the
work into small parts; it means you chop it up into small parts so that each part is worth doing, even if the other
parts never happen.
If the only value you get from the migration is at the very end of a long process (and lets be honest, migrations
always take longer than you expect), then there's a good chance management will lose patience, developers will get
frustrated, funding for the project will be cut, corners will be cut, and the project will be a disaster.
Therefore, you want to arrange the work so that you can complete one small piece at a time and get value out of that
piece immediately. That way, management stays happy, developers stay motivated, and the project is worth doing, even
if the end of the migration is still a long ways off.
You'll find a number of suggestions in this document on how to do the migration incrementally. The short version is:
Migrate one or a small number of apps.
Allow the apps to keep using the data stores and other dependencies in your original data center via a VPN
connection.
Test.
Once things are working, switch over the DNS config to point to the apps running in AWS.
Repeat the previous steps with the rest of your apps.
Once all apps have been migrated, begin migrating your data stores.
Read on for the long version!
Set up account access
The first step to migrating to the Reference Architecture is to make sure everyone on your team has access to your AWS
accounts. This consists of the following items:
We strongly recommend that everyone on your team reads through the following:
Gruntwork Security Best
Practices,
doc, which covers critical concepts such as how to securely store secrets, how to lock down servers, and how to
securely use AWS.
Each of your AWS accounts has a root user. Whoever
created the account should have access to the root user. If you created the account using the gruntwork
CLI or AWS Organizations, then
you will know the email address associated with the root user, but not the password. To get the password:
If you had previously signed into some other AWS account, click "Sign-in using root account credentials."
Enter the email address and click "Forgot your password" to reset the password.
Check the email address associated with the root user account for a link you can use to create a new password.
Please note that the root user account can do just about anything in your AWS account, bypassing almost all security
restrictions you put in place, so you need to take extra care with protecting this account. We very strongly
recommend that you:
Use a strong password. Preferably 30+ characters, randomly generated, and stored in a secrets manager.
Only use the root user to create an IAM User for yourself and then switch to the IAM User for all other operations
thereafter. You should NOT use the root user account on a day-to-day basis; it's there only for very rare
circumstances (e.g., if you get locked out of your IAM User account).
IAM users
You should create an IAM User for each
developer. The Reference Architecture enforces a
password policy: see
the iam-user-password-policy module in infrastructure-live-acme for the details. We also strongly recommend
requiring every developer to enable multi-factor auth
for their IAM users.
IAM groups
Give each IAM User permissions by adding them to the appropriate IAM
Groups. The following IAM Groups may be of especial
interest (see the iam-groups module
for a more complete list):
full-access: This gives full admin privileges within this AWS account. Use this only for a small number of
trusted admins.
read-only: This gives read-only access within this AWS account. This is especially useful for production
accounts where you want to be able to debug, but not necessarily make changes.
openvpn-users: This allows users to use openvpn-admin
to request VPN certificates in this AWS account. See SSH and VPN for more info.
ssh-grunt-users: Users in this group will be able to access servers via SSH. See SSH and VPN for
more info.
ssh-grunt-sudo-users: Users in this group will be able to access servers via SSH with sudo permissions. See
SSH and VPN for more info.
SSH Access
We have configured ssh-grunt on every
EC2 Instance, so developers will be able to SSH to the servers using their own usernames and SSH keys. See
SSH and VPN for instructions.
VPN Access
All the EC2 Instances run in private subnets, so to access them, you must first connect via VPN. See SSH and
VPN for instructions.
This will involve setting up Virtual Private Gateway,
and you will need to configure the VPCs to propagate routes for these Gateways using the private_propagating_vgws
and persistence_propagating_vgws variables.
Deploy your apps
Once account access is set up, the next step is to migrate your apps—or even just a single app—to the Reference
Architecture, while continuing to use whatever data stores and other dependencies you already have deployed in your old
data center (e.g., via a VPN connection). Once you can get one app running in AWS successfully, you can use it as a
template to move over the others, getting value from the migration right away (i.e., more security, scalability, etc)
without having to wait weeks or months to move everything over (incrementalism!).
To move over an app, you'll need to do the steps listed below. The Reference Architecture includes two sample apps
(sample-app-frontend-acme and sample-app-backend-acme) that show how to implement all of these steps, so use them as a
reference!
Note that you do NOT need to do all of these steps at once! You can take care of just a few of them, deploy the app
into the Reference Architecture, see that some basic things work, move on to the next few steps, and so on.
Each app should expose a health check endpoint (e.g., /health) that the load balancer can use to see if the app is
running and ready to receive requests. The health check endpoint should return a 200 OK when the app is healthy.
The definition of "healthy" depends on the app; for some apps, healthy might mean the app has booted; for others,
healthy may mean the app has booted, can talk to databases, and has warmed up its cache.
Update the load balancer config
Now that your app is packaged and has a health check endpoint, you can try deploying it! The sample apps are deployed
via Terraform code in the infrastructure-live-acme repo, under the services folder (e.g.,
stage/services/sample-app-frontend-acme). Make a copy of that folder and customize the terragrunt.hcl file within it
to deploy your app instead. This will include specifying which paths in the load balancer your app should claim.
This is because the Reference Architecture uses a single Application Load Balancer
(ALB) for several apps in
each environment, and each of those apps can claim certain paths (e.g., /foo goes to app foo, /bar goes to app bar)
and/or certain domain names (e.g., foo.acme.com goes to app foo, bar.acme.com goes to app bar). If you want your
app to handle all paths, you can use regular expressions (e.g., *).
Set up config files for each environment
If your app needs to be configured differently in each environment (e.g., in stage and prod), we recommend creating
config files for each environment, checking those configs into version control, and packaging them with the app. That
way, config is updated, versioned, tested, and deployed exactly like the rest of your code. This tends to be easier
to reason about and far less error prone than storing configs externally (e.g., in a DB).
The sample apps use JSON files for config, but any format supported by your apps will be fine, so long as it is
file-based. See the config folder in the sample apps for an example, and check out the bin/run-app.sh script for
how the sample apps select the proper config file for the current environment.
Encrypt secrets
If your apps need access to secrets, such as the password to a database, we recommend encrypting those secrets with
KMS and adding the ciphertext to your config files. Your apps can then use IAM Roles to
decrypt the secrets just before booting.
If your apps talk to a relational database, you will need a way to evolve the database schema over time. The best way
to do that is to use a schema migration tool such as Flyway or
Liquibase to define the schema migrations as code, check that code into version control,
package it with your apps, and have your apps apply the schema migrations just before booting (Flyway and Liquibase
will use locks to ensure there are no concurrency issues).
See the sql folder and bin/run-app.sh of sample-app-backend-acme for an example.
Configure service discovery
If you need to run multiple apps ("microservices") that talk to each other, those apps will need a way to discover each
other's IP and port. In the dynamic world of AWS, IPs and ports change all the time, so you don't want to hard code
them. Instead, use one of the following options:
Use an internal load balancer. Each of your apps can register at a different path with the load balancer, and each
app gets the load balancer domain name as an environment variable (via Terraform), so, for example, you can talk to
app foo by sending a request to <LOAD_BALANCER_DOMAIN_NAME>/foo and to app bar by sending a request to
<LOAD_BALANCER_DOMAIN_NAME>/bar. This is how sample-app-frontend-acme talks to sample-app-backend-acme.
For Docker containers running in ECS, you can use ECS Service
Discovery. As of May, 2018,
Gruntwork has not yet added support for this, but it should be added to the ECS
modules soon.
Use a tool such as Consul. This does require running extra infrastructure, which is made
easier via the Gruntwork Consul module (not currently part of
the Reference Architecture).
Set up static content
The best way to serve CSS, JS, images, and fonts is by putting them in an S3 bucket and
using CloudFront as a CDN in front of it. Your Reference Architecture may already
contain an example of this under the services/static-website folder. Your CloudFront distribution will have its own
domain name (e.g., static.<your-company>.com), so you'll need to ensure that any code of yours that links to static
content (e.g., server-side rendered HTML) makes it possible to configure a different static content domain name in
each environment.
Configure CI and CD
The sample apps have Continuous Integration (CI) and Continuous Delivery (CD) configured for them. A CI job will run
after every commit to build, test, and package the apps, and, for certain commits (e.g., to specific branches or with
specific tags), the CI job will automatically deploy the app to stage or prod.
Once you have your apps running in AWS, the next step is to migrate the data stores they depend on. We again recommend
doing this incrementally, moving over one data store at a time, if possible. There are two ways to move over your data
stores:
Take a downtime
Do a zero-downtime migration
Option #2 typically takes 100x longer (this is NOT an exaggeration) than option #1 and has a much higher risk of
data loss or data corruption. If at all possible, we very strongly recommend taking a brief downtime to do the
migration, as it will make your life much easier.
If you are migrating to the Reference Architecture from another AWS account, and in that account you are using an
AWS-managed data store such as RDS or ElastiCache,
the migration process (with downtime!) is:
Put your app in read-only mode or take a downtime.
Share the snapshot with the prod account where the Reference Architecture is deployed (e.g., see Sharing a DB
Snapshot).
Deploy a new data store in the Reference Architecture account from the snapshot (e.g., set the
snapshot_identifier parameter in the rds module).
Bring your app back up.
Migrating from external data stores
If you are migrating data stores from non-AWS managed systems (e.g., from your own data center), the process is the
same as in the previous section, except instead of using the snapshotting mechanism built into AWS, you will need to
use one of these mechanisms instead:
Use your data store's native snapshot functionality (e.g., mysqldump)
to make a backup of your database, copy the backup into AWS (e.g., into an S3 bucket), and fire up an EC2 Instance
that connects to the data store and loads the backup into it.
Note that all of these assume at least some amount of downtime!
If you absolutely must do a zero-downtime migration
If you absolutely cannot take any downtime whatsoever (unlikely, as you most likely have unplanned outages anyway!),
things get a lot more complicated. The typical strategy is:
Take an initial snapshot of your data store and load it into AWS. The idea is to move most of the data over early on
and then "catch up" whatever new data is written in the following steps.
Find a way to send all new writes to both your original data store and the new one in AWS.
Once the new data store has "caught up", start using it directly instead of the old database.
Step #2 is the tricky one, as dual writes are notoriously error prone. For example, what happens if the process doing
dual writes succeeds with the first write, but crashes before the second? What if it sends the requests to both data
stores, but one of them is temporarily down? What if both requests make it to the data stores, but then a transaction
gets rolled back? See why dual writes are a bad
idea
for more info.
The most common solutions to these problems are:
Use a change capture system. This is a system that effectively replicates your data store's underlying log: you
write to the original data store, and if this write has succeeded and been committed, it triggers an "event" in the
change capture system, which an event processor then writes to the second data store. For example,
see databus and bottledwater-pg.
Send all writes initially to a log system such as Apache Kafka. You then have one
Kafka consumer that writes the messages to your original data store and another that writes it to the new data store.
Kafka guarantees at-least once delivery, so as long as the database writes are idempotent, this ensures no data
will be lost.
Whatever option you go with, we strongly recommend adding "integrity checking" to verify no data was lost or corrupted.
Some example integrity checks:
Count the number of rows in each table and make sure they are identical in both data stores.
Perform a checksum or hash on all the data in a table (or some randomly selected subset if there's too much data)
and make sure you get identical values in both data stores.
Configure monitoring and alerting
Once your apps and data stores are working, you'll want to make sure you have the following monitoring and alerting in
place:
Make sure all the metrics you expect show up in CloudWatch. All AWS services
automatically send metrics to CloudWatch and the Reference Architecture includes a few important ones that are not
available by default (e.g., memory and disk space usage on EC2 Instances). Create
Dashboards for the
most important metrics, such as CPU usage in your ECS cluster or request count in the ALB.
Note that the Reference Architecture uses CloudWatch for metrics as it's built-into AWS, very inexpensive, and requires
very little work to maintain. However, the CloudWatch UI is fairly basic, so if you need more powerful tools for slicing
and dicing your metrics, consider tools such as DataDog,
New Relic, or Prometheus. If you'd like help with this, please reach
out to support@gruntwork.io.
Alerts
Configure alerts to go off for key metrics. All the services in the Reference Architecture have basic alarms
configured directly in the Terraform code, such as low disk space alarms on RDS DBs and high CPU usage alarms for the
ECS cluster. All of the Terraform modules allow you to tweak the settings for these alarms (e.g., should the CPU
usage alarm go off at 80% or 90%?), so take some time to go through each module in infrastructure-modules-acme and
make sure the settings match your use cases so you don't have too many false positives or negatives.
Logs
The logs from all servers get automatically sent to CloudWatch
Logs. The user-data.sh scripts
in infrastructure-modules-acme call run-cloudwatch-logs-agent.sh (from the cloudwatch-log-aggregation-scripts
module)
to configure which log files get sent to CloudWatch. By default, syslog is sent to CloudWatch on every server, and
ECS Services are configured to send their logs to syslog. Make sure you can find the logs for your apps there
before you go live!
Note that the Reference Architecture uses CloudWatch Logs as it's built-into AWS, very inexpensive, and requires
very little work to maintain. However, the CloudWatch UI is fairly basic, so if you need more powerful tools for slicing
and dicing your log data, consider tools such as ELK (as of May, 2018, the ELK
support is under development in the package-elk repo and should be
ready soon), Loggly, and Papertrail. If you'd like help with
this, please reach out to support@gruntwork.io.
Test
As you go through the process of migrating your apps and data stores, you should continuously be testing your
infrastructure to make sure things are working. Here are the key types of tests to perform:
Manual testing: Hit URLs by hand, look at log files, and check the metrics. You already know how to do this!
Automated testing: We strongly recommend writing automated tests that (a) deploy your infrastructure into an
AWS account dedicated for testing, (b) check the infrastructure works as expected, and (c) tears the infrastructure
back down at the end of the test. You can use the Terratest library
to write these tests.
Load testing: You should make sure your code performs well under the kind of loads you expect in production.
Use tools such as Apache Bench,
JMeter to throw traffic at your infrastructure and when things fall over, make sure
your monitoring and alerting can help you find the bottlenecks! Load testing across several different EC2 Instance
Types can also be a good way to find the sweet spot between price and performance (see How Netflix Tunes EC2
Instances for Performance).
Security testing: Bring in an outside firm to pen test and audit your code to look for security vulnerabilities.
Gruntwork customers have done this several times with the Reference Architecture already, and we've fixed all issues
that were found, but if you find any new vulnerabilities, please report them to support@gruntwork.io!
Switch over DNS
The Reference Architecture is typically deployed with "placeholder" domain names (e.g., your-company-aws.com) to make
testing easy and safe. Once your apps start passing tests, you can switch over your real domain names to point to
the Reference Architecture as follows:
If you have multiple AWS accounts (e.g., dev, stage, prod), we recommend that each one use its own, completely
separate domain names (e.g., my-company-dev.com, my-company-stage.com, etc.). This reduces the chances of
accidental errors (e.g., breaking prod DNS entries while trying to make changes to stage) and makes it harder for
attackers to do damage (e.g., using cookies from pre-prod environment in a prod environment).
Before switching the domain names on your apps, make to use AWS Certificate Manager
(ACM) to request TLS certificates for your prod domain names and
associate those certificates with your Load Balancers and CloudFront distributions. ACM is a great choice for
TLS certificates, because they are (a) free and (b) issue in 1-2 minutes for domain names you own in Route 53, and
(c) renew automatically!
To switch over your ALBs or ECS services to use different domain names, simply update the domain_name variables in
the corresponding terragrunt.hcl.
{"treedata":{"name":"root","toggled":true,"children":[{"name":".gitignore","path":".gitignore","sha":"1c27fc6013cba46cd301a7c8bf951694670153a3"},{"name":"CODEOWNERS","path":"CODEOWNERS","sha":"6bddb3ff6e1b3dfaba7cf180e56bca12c245be56"},{"name":"README.md","path":"README.md","sha":"74507349a9d5cd784540410365adc61ba68000f1"},{"name":"_docs","children":[{"name":"01-architecture-overview.md","path":"_docs/01-architecture-overview.md","sha":"fa091294c3f4cfb2e7bd1d7df78907faf076996b"},{"name":"02-whats-deployed.md","path":"_docs/02-whats-deployed.md","sha":"8bf4519132e2ea43cbcf1e1d67eff3f961471af2"},{"name":"03-security-compliance-compatibility.md","path":"_docs/03-security-compliance-compatibility.md","sha":"9342617f42adb28e440cc2161f3fee56205c150e"},{"name":"04-how-code-is-organized.md","path":"_docs/04-how-code-is-organized.md","sha":"64b9396f54fb0b791d39b93919a6416ab8215f0d"},{"name":"05-dev-environment.md","path":"_docs/05-dev-environment.md","sha":"9209da466b0f9afee5e1afb36f01a2ba8149012f"},{"name":"06-ci-cd.md","path":"_docs/06-ci-cd.md","sha":"0685cbe746fa0271357db34ebd39d76397ea19c4"},{"name":"07-monitoring-alerting-logging.md","path":"_docs/07-monitoring-alerting-logging.md","sha":"619c810c6e60418b3a46fa3d903bc76dc6d48e41"},{"name":"08-ssh-vpn.md","path":"_docs/08-ssh-vpn.md","sha":"0f526549f4b0d08cf2a914def239f8ff872ec2d1"},{"name":"09-accounts-and-auth.md","path":"_docs/09-accounts-and-auth.md","sha":"9e6d2c1f8b9a7cc7c5b3ce3386eea22daac6cc17"},{"name":"10-gruntwork-tools.md","path":"_docs/10-gruntwork-tools.md","sha":"d08b1fe7cfbb9ad91155bfff9e3a05525c39c127"},{"name":"11-deploying-a-docker-service.md","path":"_docs/11-deploying-a-docker-service.md","sha":"d2123b688287557c1c38cd415a729b3a445a45ad"},{"name":"12-migration.md","path":"_docs/12-migration.md","sha":"6e46bf752f330de978a8927858a716f04db13f60","toggled":true},{"name":"13-deploying-the-reference-architecture-from-scratch.md","path":"_docs/13-deploying-the-reference-architecture-from-scratch.md","sha":"9eb702bb6da48a97b2c9eead594a3474a1ee6703"},{"name":"14-undeploying-the-reference-architecture.md","path":"_docs/14-undeploying-the-reference-architecture.md","sha":"3ed0569cdd0e3d32079ab537e1697fbcb3ee27d8"},{"name":"15-adding-new-environments-regions-and-accounts.md","path":"_docs/15-adding-new-environments-regions-and-accounts.md","sha":"6a0372a843a9245570379e1beaad452e67d234c3"},{"name":"README.md","path":"_docs/README.md","sha":"785d2b0b36b10e75c96e4eaa7414c1c71d78e222"},{"name":"_images","children":[{"name":"cw-logs-1.png","path":"_docs/_images/cw-logs-1.png","sha":"84c86f014751844fbd777b5139ed61f749b5ed32"},{"name":"cw-logs-2.png","path":"_docs/_images/cw-logs-2.png","sha":"9a0a80b20490fdc1b9014040cc0bbc87c9cf6f68"},{"name":"cw-logs-3.png","path":"_docs/_images/cw-logs-3.png","sha":"bda49dc4e947658e0ceb9ba592b4e314d9db61e9"},{"name":"cw-logs-4.png","path":"_docs/_images/cw-logs-4.png","sha":"54bcc44c4b0701620b7f20c4e6fc0a9fd8f38049"},{"name":"ecs-console-1.png","path":"_docs/_images/ecs-console-1.png","sha":"afe452278d5f107e6ec225a235c587de7cb53510"},{"name":"ecs-console-2.png","path":"_docs/_images/ecs-console-2.png","sha":"40609b98015d781b9e1de801c131fadc323337ae"},{"name":"ecs-console-3.png","path":"_docs/_images/ecs-console-3.png","sha":"87ad40d291b7e9e6f6caa0389b846392bdb93ee0"},{"name":"ref-arch-full.png","path":"_docs/_images/ref-arch-full.png","sha":"8c17eef52be06757553a1f3ee4e387e6dc820016"},{"name":"ref-arch-icon.png","path":"_docs/_images/ref-arch-icon.png","sha":"05876962e6877df911674237ca1b793d9f4f04b3"},{"name":"terraform-code-provenance.png","path":"_docs/_images/terraform-code-provenance.png","sha":"e2a9d6bfbd8b963b057d4341dd0ec93e3823d834"}]}],"toggled":true},{"name":"main","children":[{"name":"_global","children":[{"name":"README.md","path":"main/_global/README.md","sha":"d1b8a96c00211751f079fa13cac1b3417d29bf09"},{"name":"cloudtrail","children":[{"name":"README.md","path":"main/_global/cloudtrail/README.md","sha":"7bf54b13e60f80416bbe3ee5b22328ee47a2532a"},{"name":"terragrunt.hcl","path":"main/_global/cloudtrail/terragrunt.hcl","sha":"5d4953cb81dc711c1f5641e26b54b2de8dedf62e"}]},{"name":"iam-groups","children":[{"name":"README.md","path":"main/_global/iam-groups/README.md","sha":"b6797c786d3c914257fc0e0bb4680e9fc861ab38"},{"name":"terragrunt.hcl","path":"main/_global/iam-groups/terragrunt.hcl","sha":"f14f3f59fb104ffc4f9925dd8565f43d665600c7"}]},{"name":"iam-user-password-policy","children":[{"name":"README.md","path":"main/_global/iam-user-password-policy/README.md","sha":"1ddbc02253cb2fb3971fdbf1b3e09758f8eede9a"},{"name":"terragrunt.hcl","path":"main/_global/iam-user-password-policy/terragrunt.hcl","sha":"482cdc172a7e5f033acf9d808bef3e8bd3ef8f1e"}]},{"name":"machine-user","children":[{"name":"README.md","path":"main/_global/machine-user/README.md","sha":"0d676a50c24d954dab8a57794e6790eb03d03e4e"},{"name":"terragrunt.hcl","path":"main/_global/machine-user/terragrunt.hcl","sha":"50b845d531ee85c0a109c6abba695a3fe7d5b89e"}]},{"name":"region.yaml","path":"main/_global/region.yaml","sha":"18b7823ed017b97431d58da7bcb9a4e31299272a"},{"name":"route53-public","children":[{"name":"README.md","path":"main/_global/route53-public/README.md","sha":"4757db7a8adde3d4af6a86f3ea20e050ae946a08"},{"name":"terragrunt.hcl","path":"main/_global/route53-public/terragrunt.hcl","sha":"06c315b032005358d2a3e6c5774f5b99ab64e681"}]},{"name":"service-linked-roles","children":[{"name":"README.md","path":"main/_global/service-linked-roles/README.md","sha":"45c1919cc5667b8d8ae25f09b3baf3078a7b36f9"},{"name":"terragrunt.hcl","path":"main/_global/service-linked-roles/terragrunt.hcl","sha":"398538ac8d92ed0160f303b029720592f81af280"}]}]},{"name":"empty.yaml","path":"main/empty.yaml","sha":"5aa66daa40faeaef37eccb7b4b0fcc792233cd7b"},{"name":"terragrunt.hcl","path":"main/terragrunt.hcl","sha":"3f362bb353e2d6e02f59294d7f7044da3a62b565"},{"name":"us-east-1","children":[{"name":"_global","children":[{"name":"README.md","path":"main/us-east-1/_global/README.md","sha":"37b828b038945a50e2e571ef1e755c4f9170e7cf"},{"name":"ecr-repos","children":[{"name":"README.md","path":"main/us-east-1/_global/ecr-repos/README.md","sha":"e7215127ffcf141002796e83ed1b9e9647ddbe22"},{"name":"terragrunt.hcl","path":"main/us-east-1/_global/ecr-repos/terragrunt.hcl","sha":"20c8ae6bd26f8cfd8fddcb8676cc109201bc49eb"}]},{"name":"sns-topics","children":[{"name":"README.md","path":"main/us-east-1/_global/sns-topics/README.md","sha":"05eb8a853eccf6465dc558bb9c57637fb4e9ccd3"},{"name":"terragrunt.hcl","path":"main/us-east-1/_global/sns-topics/terragrunt.hcl","sha":"81c92f821daf7077881a43678c717d485fc36401"}]}]},{"name":"mgmt","children":[{"name":"README.md","path":"main/us-east-1/mgmt/README.md","sha":"8a131a11632b97fec18a5e344d5c721fce24b652"},{"name":"env.yaml","path":"main/us-east-1/mgmt/env.yaml","sha":"b514ab3187ebfb5bf467c632f27a21f5a9611bfc"},{"name":"kms-master-key","children":[{"name":"README.md","path":"main/us-east-1/mgmt/kms-master-key/README.md","sha":"2affa3417a1b76f670e407330cb9dc62d01a521e"},{"name":"terragrunt.hcl","path":"main/us-east-1/mgmt/kms-master-key/terragrunt.hcl","sha":"5e24ee41d1565d3e026354e71a5ae7a362a206ea"}]},{"name":"openvpn-server","children":[{"name":"README.md","path":"main/us-east-1/mgmt/openvpn-server/README.md","sha":"93a2465ed5c720dd30b434aa36a2d28c4e0c7fcf"},{"name":"terragrunt.hcl","path":"main/us-east-1/mgmt/openvpn-server/terragrunt.hcl","sha":"aaf206a2c80987a888872ef7812bb93984cf26d1"}]},{"name":"vpc","children":[{"name":"README.md","path":"main/us-east-1/mgmt/vpc/README.md","sha":"495e2b8828a490c03ea7e153e695239f2bb92512"},{"name":"terragrunt.hcl","path":"main/us-east-1/mgmt/vpc/terragrunt.hcl","sha":"d6fad3a7e2397ea7bc62fa77ea6e138ec27e4335"}]}]},{"name":"prod","children":[{"name":"README.md","path":"main/us-east-1/prod/README.md","sha":"f15da18661ef3624d5f63deb288bad072e93df57"},{"name":"cloudwatch-dashboard","children":[{"name":"README.md","path":"main/us-east-1/prod/cloudwatch-dashboard/README.md","sha":"766cff97af8b2bbbdb90c2262c150b4d0bc88c62"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/cloudwatch-dashboard/terragrunt.hcl","sha":"ff055251d5427c0116d0e382f38c537b09db96ee"}]},{"name":"data-stores","children":[{"name":"elasticsearch","children":[{"name":"README.md","path":"main/us-east-1/prod/data-stores/elasticsearch/README.md","sha":"de10ddf77c3ae0b341ebbd7152f8d3c086d7ba20"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/data-stores/elasticsearch/terragrunt.hcl","sha":"736a4bdbdcc1b54dcd909671a76f09988c226436"}]},{"name":"kafka","children":[{"name":"README.md","path":"main/us-east-1/prod/data-stores/kafka/README.md","sha":"3681db5950b18676e92d6f00df190ff553c06404"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/data-stores/kafka/terragrunt.hcl","sha":"89d57ee0e689e1f28141c383924b03937b46fb3b"}]},{"name":"mysql","children":[{"name":"README.md","path":"main/us-east-1/prod/data-stores/mysql/README.md","sha":"3ff802dea2beeb94b34a9d2087fa1ce332702ba0"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/data-stores/mysql/terragrunt.hcl","sha":"681f993523ec8336c4f6be26a41ab1148b127ce8"}]},{"name":"redis","children":[{"name":"README.md","path":"main/us-east-1/prod/data-stores/redis/README.md","sha":"7f5426659066280ce18fad93eb14dd573e3de1b0"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/data-stores/redis/terragrunt.hcl","sha":"e7d7171695ad1ca039db9171ac31f1fb210f51f5"}]},{"name":"zookeeper","children":[{"name":"README.md","path":"main/us-east-1/prod/data-stores/zookeeper/README.md","sha":"3aa643354b946d75610e3a8d10e616e1080717bc"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/data-stores/zookeeper/terragrunt.hcl","sha":"ed94a3d0277beb71cd9500f1d29053e2f5b99f28"}]}]},{"name":"env.yaml","path":"main/us-east-1/prod/env.yaml","sha":"90e2d18e481b6e35ddc57391f752874ffc0058cf"},{"name":"kms-master-key","children":[{"name":"README.md","path":"main/us-east-1/prod/kms-master-key/README.md","sha":"7bf1a8da34427b8314d99904b01c20937604e1e0"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/kms-master-key/terragrunt.hcl","sha":"fb4afa8978a7b719c852756ac51883c5e87f5ab8"}]},{"name":"lambda","children":[{"name":"long-running-scheduled","children":[{"name":"README.md","path":"main/us-east-1/prod/lambda/long-running-scheduled/README.md","sha":"a6a7503b1168dd015618028f30b74aeb1ba7baf3"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/lambda/long-running-scheduled/terragrunt.hcl","sha":"c05243749ec7d3bd386c0563936b0f5e0b4bbbe6"}]},{"name":"s3-image-processing","children":[{"name":"README.md","path":"main/us-east-1/prod/lambda/s3-image-processing/README.md","sha":"1b149a62078c71549d77e59b0ea995f7181a7d8b"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/lambda/s3-image-processing/terragrunt.hcl","sha":"68b62dc42154055b1eaf522fd2240992df4d7814"}]}]},{"name":"networking","children":[{"name":"alb-internal","children":[{"name":"README.md","path":"main/us-east-1/prod/networking/alb-internal/README.md","sha":"5880e9468a3bf336b613dc7132b052ea89a0f99a"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/networking/alb-internal/terragrunt.hcl","sha":"7ad684cfd8fc90140bf149e0f7fa689a2bd6ace0"}]},{"name":"alb-public","children":[{"name":"README.md","path":"main/us-east-1/prod/networking/alb-public/README.md","sha":"5880e9468a3bf336b613dc7132b052ea89a0f99a"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/networking/alb-public/terragrunt.hcl","sha":"ad8fd584b78fdd026ac76554e60653f2c23cf765"}]},{"name":"route53-private","children":[{"name":"README.md","path":"main/us-east-1/prod/networking/route53-private/README.md","sha":"9160c66a0b04a407981db7bf9ee40dad8c5d9434"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/networking/route53-private/terragrunt.hcl","sha":"df4f82c4dc568fbe251eef10adbc28f1b1ccc263"}]}]},{"name":"services","children":[{"name":"ecs-cluster","children":[{"name":"README.md","path":"main/us-east-1/prod/services/ecs-cluster/README.md","sha":"560d730485383f188833313b77599681d146bdd7"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/ecs-cluster/terragrunt.hcl","sha":"38c41c4c8583f8569c2d37e51321acf4a0af6f5f"}]},{"name":"eks-cluster","children":[{"name":"README.md","path":"main/us-east-1/prod/services/eks-cluster/README.md","sha":"84d53a02559d844d8c62ee9d11a558265b3ae5b5"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/eks-cluster/terragrunt.hcl","sha":"e80ff0c733463d17c33d8c7062121e985d1cd1e8"}]},{"name":"eks-core-services","children":[{"name":"README.md","path":"main/us-east-1/prod/services/eks-core-services/README.md","sha":"298969aaf6db9c7e972cd735cef2d43cda899e3f"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/eks-core-services/terragrunt.hcl","sha":"1e8a88f4c8330f10848fea04daa719ab92c74d53"}]},{"name":"k8s-applications-namespace","children":[{"name":"README.md","path":"main/us-east-1/prod/services/k8s-applications-namespace/README.md","sha":"5e0a3640be89e96aece9e4ac8f8b7967e4d57056"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/k8s-applications-namespace/terragrunt.hcl","sha":"120cab356ca4590c262bd3b727d60f6c8294792b"}]},{"name":"k8s-sample-app-backend-acme","children":[{"name":"README.md","path":"main/us-east-1/prod/services/k8s-sample-app-backend-acme/README.md","sha":"4737c31d75b6d6c7565fa3d391177c9d3d022a5e"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/k8s-sample-app-backend-acme/terragrunt.hcl","sha":"c4e9d2aaad8a61ebcb1148800e50858e1023d959"}]},{"name":"k8s-sample-app-frontend-acme","children":[{"name":"README.md","path":"main/us-east-1/prod/services/k8s-sample-app-frontend-acme/README.md","sha":"43161374e0c977f15e8af04913e3554894ed2a82"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/k8s-sample-app-frontend-acme/terragrunt.hcl","sha":"9ab415e18bdf994e3b78f9a6ade6cd3b0fb87e5f"}]},{"name":"sample-app-backend-acme-asg","children":[{"name":"README.md","path":"main/us-east-1/prod/services/sample-app-backend-acme-asg/README.md","sha":"90c4b69a937239b4b51e99c4404360f4aba4edde"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/sample-app-backend-acme-asg/terragrunt.hcl","sha":"940d42befd46a394d63124cfd0d5830c6cea44b2"}]},{"name":"sample-app-backend-acme","children":[{"name":"README.md","path":"main/us-east-1/prod/services/sample-app-backend-acme/README.md","sha":"db3e1635aae0577080a9e2518633d4b5830a259a"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/sample-app-backend-acme/terragrunt.hcl","sha":"c2f4509348ff2f45c10901bbd06f7e409b888e14"}]},{"name":"sample-app-beanstalk","children":[{"name":"README.md","path":"main/us-east-1/prod/services/sample-app-beanstalk/README.md","sha":"27f6c3930f0621262b5dd0400f59e1122631da65"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/sample-app-beanstalk/terragrunt.hcl","sha":"2eaf051ae2cb3f4871be2cb78d3c9edec2da212e"}]},{"name":"sample-app-frontend-acme-asg","children":[{"name":"README.md","path":"main/us-east-1/prod/services/sample-app-frontend-acme-asg/README.md","sha":"ab9db3714676828cb0b766b5a3329f37115b1729"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/sample-app-frontend-acme-asg/terragrunt.hcl","sha":"2d6811f17d17c94bdf9f06c74bf74a3424555c3b"}]},{"name":"sample-app-frontend-acme","children":[{"name":"README.md","path":"main/us-east-1/prod/services/sample-app-frontend-acme/README.md","sha":"c0bc1f93aef1ab1bcf3e22aaec14a1744da74dfb"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/sample-app-frontend-acme/terragrunt.hcl","sha":"84c663da537b9e7485f4e3eff18ffaabfc90ff4b"}]},{"name":"static-website","children":[{"name":"README.md","path":"main/us-east-1/prod/services/static-website/README.md","sha":"16d262eab6ca158ae60d84010bc4804a948673cf"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/services/static-website/terragrunt.hcl","sha":"bb0530a96785e00136cb13da4dd2aca6583ba393"}]}]},{"name":"vpc","children":[{"name":"README.md","path":"main/us-east-1/prod/vpc/README.md","sha":"eb0cfd86345b2983b4a9c9572501728493bfcde4"},{"name":"terragrunt.hcl","path":"main/us-east-1/prod/vpc/terragrunt.hcl","sha":"5a5f20f83d3935f43b48c71b88c0f44443eb11ed"}]}]},{"name":"region.yaml","path":"main/us-east-1/region.yaml","sha":"d56afa3d82e6cea0d792e84748de56dafb0bad70"},{"name":"stage","children":[{"name":"README.md","path":"main/us-east-1/stage/README.md","sha":"b24ba21bf01baf19ff84a2de457697a757d905c5"},{"name":"cloudwatch-dashboard","children":[{"name":"README.md","path":"main/us-east-1/stage/cloudwatch-dashboard/README.md","sha":"766cff97af8b2bbbdb90c2262c150b4d0bc88c62"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/cloudwatch-dashboard/terragrunt.hcl","sha":"feb471fd97d340798fd7ea565195faf4c4328f82"}]},{"name":"data-stores","children":[{"name":"elasticsearch","children":[{"name":"README.md","path":"main/us-east-1/stage/data-stores/elasticsearch/README.md","sha":"de10ddf77c3ae0b341ebbd7152f8d3c086d7ba20"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/data-stores/elasticsearch/terragrunt.hcl","sha":"b65b647edff6298b3a5a010d8ebe13f022b7c54d"}]},{"name":"kafka","children":[{"name":"README.md","path":"main/us-east-1/stage/data-stores/kafka/README.md","sha":"3681db5950b18676e92d6f00df190ff553c06404"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/data-stores/kafka/terragrunt.hcl","sha":"5ba7184e22ee433462c2d05c97ad6bb47c13786b"}]},{"name":"mysql","children":[{"name":"README.md","path":"main/us-east-1/stage/data-stores/mysql/README.md","sha":"3ff802dea2beeb94b34a9d2087fa1ce332702ba0"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/data-stores/mysql/terragrunt.hcl","sha":"d5d7ac741e73afbce47d99af1cbb16219e8af208"}]},{"name":"redis","children":[{"name":"README.md","path":"main/us-east-1/stage/data-stores/redis/README.md","sha":"7f5426659066280ce18fad93eb14dd573e3de1b0"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/data-stores/redis/terragrunt.hcl","sha":"224dc41a716053b231af5c557d7fe95a67af6d51"}]},{"name":"zookeeper","children":[{"name":"README.md","path":"main/us-east-1/stage/data-stores/zookeeper/README.md","sha":"3aa643354b946d75610e3a8d10e616e1080717bc"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/data-stores/zookeeper/terragrunt.hcl","sha":"c084625c9b909a6e5c7cc45995752645e1a9a577"}]}]},{"name":"env.yaml","path":"main/us-east-1/stage/env.yaml","sha":"5767506e27e978f52524dadbbd8fb9f8ad115599"},{"name":"kms-master-key","children":[{"name":"README.md","path":"main/us-east-1/stage/kms-master-key/README.md","sha":"0fc848e518ff6551caae5f234b89f3f2c2a3b015"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/kms-master-key/terragrunt.hcl","sha":"49762d286ee1ed5b999973a5f82688c01bdcf679"}]},{"name":"lambda","children":[{"name":"long-running-scheduled","children":[{"name":"README.md","path":"main/us-east-1/stage/lambda/long-running-scheduled/README.md","sha":"a6a7503b1168dd015618028f30b74aeb1ba7baf3"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/lambda/long-running-scheduled/terragrunt.hcl","sha":"c05243749ec7d3bd386c0563936b0f5e0b4bbbe6"}]},{"name":"s3-image-processing","children":[{"name":"README.md","path":"main/us-east-1/stage/lambda/s3-image-processing/README.md","sha":"1b149a62078c71549d77e59b0ea995f7181a7d8b"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/lambda/s3-image-processing/terragrunt.hcl","sha":"2ab4fe1b159bfe0797662dee0c24aba8b9b3e106"}]}]},{"name":"networking","children":[{"name":"alb-internal","children":[{"name":"README.md","path":"main/us-east-1/stage/networking/alb-internal/README.md","sha":"c1c8edd637ebd686cc5d7675013d750b8ca4ad52"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/networking/alb-internal/terragrunt.hcl","sha":"d90749b2f63ce2187c0fb1ee95723a57fb7e4097"}]},{"name":"alb-public","children":[{"name":"README.md","path":"main/us-east-1/stage/networking/alb-public/README.md","sha":"c1c8edd637ebd686cc5d7675013d750b8ca4ad52"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/networking/alb-public/terragrunt.hcl","sha":"fc8f137af22da522f85614dc22de19f962bdb285"}]},{"name":"route53-private","children":[{"name":"README.md","path":"main/us-east-1/stage/networking/route53-private/README.md","sha":"9160c66a0b04a407981db7bf9ee40dad8c5d9434"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/networking/route53-private/terragrunt.hcl","sha":"df4f82c4dc568fbe251eef10adbc28f1b1ccc263"}]}]},{"name":"services","children":[{"name":"ecs-cluster","children":[{"name":"README.md","path":"main/us-east-1/stage/services/ecs-cluster/README.md","sha":"94916a4be4208e4c1e1e7b7ee0d6dfa2fbfaf38c"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/ecs-cluster/terragrunt.hcl","sha":"744840965cbe10f02eb8d363ee2407201be2baf9"}]},{"name":"eks-cluster","children":[{"name":"README.md","path":"main/us-east-1/stage/services/eks-cluster/README.md","sha":"6db608bd68f9b1d4ee2741d72b2949d0dcf3e33d"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/eks-cluster/terragrunt.hcl","sha":"40da930fe85423d42896eac63f0a084847d85f53"}]},{"name":"eks-core-services","children":[{"name":"README.md","path":"main/us-east-1/stage/services/eks-core-services/README.md","sha":"0311f74f5aca78b831407ac5396907ae792d2297"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/eks-core-services/terragrunt.hcl","sha":"b8d6845c4bbca33df0a3cde51cbe1ef597095050"}]},{"name":"k8s-applications-namespace","children":[{"name":"README.md","path":"main/us-east-1/stage/services/k8s-applications-namespace/README.md","sha":"c96645baedf7aba16fc04d003608c61f9353ff4a"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/k8s-applications-namespace/terragrunt.hcl","sha":"dbd60d465e5cbb46c8a9d21620dca06f9b70dd63"}]},{"name":"k8s-sample-app-backend-acme","children":[{"name":"README.md","path":"main/us-east-1/stage/services/k8s-sample-app-backend-acme/README.md","sha":"430e8d870a1f1895093df8cb4e88e0bf2bb82bc1"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/k8s-sample-app-backend-acme/terragrunt.hcl","sha":"479f85df9855c21e4f095d6251ce4ca6d127b08c"}]},{"name":"k8s-sample-app-frontend-acme","children":[{"name":"README.md","path":"main/us-east-1/stage/services/k8s-sample-app-frontend-acme/README.md","sha":"3efad94b3379996e29355319fbe2de5c426c71dc"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/k8s-sample-app-frontend-acme/terragrunt.hcl","sha":"e0035433fce291da1ce64e77370c20790a4f063d"}]},{"name":"sample-app-backend-acme-asg","children":[{"name":"README.md","path":"main/us-east-1/stage/services/sample-app-backend-acme-asg/README.md","sha":"e2d00c960fa47118b3010550ecddf1133518d5fa"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/sample-app-backend-acme-asg/terragrunt.hcl","sha":"91940573b9a659202289c6815bd9871e50ac4d20"}]},{"name":"sample-app-backend-acme","children":[{"name":"README.md","path":"main/us-east-1/stage/services/sample-app-backend-acme/README.md","sha":"4d0bc076e9d8a51bde996c7c9f77e91bcc3e6125"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/sample-app-backend-acme/terragrunt.hcl","sha":"e240269caa4c5e60e932ebed487cf688a4c22227"}]},{"name":"sample-app-beanstalk","children":[{"name":"README.md","path":"main/us-east-1/stage/services/sample-app-beanstalk/README.md","sha":"1b3f68cb5baac277414f45585221f7048b801c26"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/sample-app-beanstalk/terragrunt.hcl","sha":"4deb3dbc95ed927e33211e473868915c1f86b5b3"}]},{"name":"sample-app-frontend-acme-asg","children":[{"name":"README.md","path":"main/us-east-1/stage/services/sample-app-frontend-acme-asg/README.md","sha":"c4df19131aafdf152f08998a2d1121ccd58c4207"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/sample-app-frontend-acme-asg/terragrunt.hcl","sha":"1d8aaccc8608aa1c2b46338c592a10f27e5b3aea"}]},{"name":"sample-app-frontend-acme","children":[{"name":"README.md","path":"main/us-east-1/stage/services/sample-app-frontend-acme/README.md","sha":"20e39909320df4775e21dd2cea70f3dabdd94c1e"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/sample-app-frontend-acme/terragrunt.hcl","sha":"2597d32b7c9d865e9f3701aa7c02a30ee4deeb01"}]},{"name":"static-website","children":[{"name":"README.md","path":"main/us-east-1/stage/services/static-website/README.md","sha":"16d262eab6ca158ae60d84010bc4804a948673cf"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/services/static-website/terragrunt.hcl","sha":"9a9cb3c9830902ef05fd497bfd21028a7fac38df"}]}]},{"name":"vpc","children":[{"name":"README.md","path":"main/us-east-1/stage/vpc/README.md","sha":"d1e3f82de82c4b8f4eaca81b3a8ec63ed040a0b5"},{"name":"terragrunt.hcl","path":"main/us-east-1/stage/vpc/terragrunt.hcl","sha":"c2429ca5412fd6f0ff00e89dc4a1541e46b07d00"}]}]}]}]}]},"detailsContent":"<h1 class=\"preview__body--title\" id=\"migration\">Migration</h1><div class=\"preview__body--border\"></div><p>Now that you understand how the Reference Architecture works, the next step is to begin migrating your apps to it. The\nprocess we recommend is as follows:</p>\n<ol>\n<li><a href=\"#do-the-migration-incrementally\" class=\"preview__body--description--blue\">Do the migration incrementally</a></li>\n<li><a href=\"#set-up-account-access\" class=\"preview__body--description--blue\">Set up account access</a></li>\n<li><a href=\"#deploy-your-apps\" class=\"preview__body--description--blue\">Deploy your apps</a></li>\n<li><a href=\"#migrate-your-data-stores\" class=\"preview__body--description--blue\">Migrate your data stores</a></li>\n<li><a href=\"#configure-monitoring-and-alerting\" class=\"preview__body--description--blue\">Configure monitoring and alerting</a></li>\n<li><a href=\"#test\" class=\"preview__body--description--blue\">Test</a></li>\n<li><a href=\"#switch-over-dns\" class=\"preview__body--description--blue\">Switch over DNS</a></li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"do-the-migration-incrementally\">Do the migration incrementally</h2>\n<p>Most "big bang" migrations—where you try to pick up everything and move it over to a new cloud (AWS), new tools\n(Terraform, Docker, Packer), and new processes (CI, CD, DevOps), all in one huge step—almost always fail. The only way\nto succeed at a big migration is to do it <em>incrementally</em>. Note that "incrementally" doesn't mean you just chop up the\nwork into small parts; it means you chop it up into small parts so that <em>each part is worth doing, even if the other\nparts never happen</em>.</p>\n<p>If the only value you get from the migration is at the very end of a long process (and lets be honest, migrations\n<em>always</em> take longer than you expect), then there's a good chance management will lose patience, developers will get\nfrustrated, funding for the project will be cut, corners will be cut, and the project will be a disaster.</p>\n<p>Therefore, you want to arrange the work so that you can complete one small piece at a time and get value out of that\npiece immediately. That way, management stays happy, developers stay motivated, and the project is worth doing, even\nif the end of the migration is still a long ways off.</p>\n<p>You'll find a number of suggestions in this document on how to do the migration incrementally. The short version is:</p>\n<ol>\n<li>Migrate one or a small number of apps.</li>\n<li>Allow the apps to keep using the data stores and other dependencies in your original data center via a VPN\nconnection.</li>\n<li>Test.</li>\n<li>Once things are working, switch over the DNS config to point to the apps running in AWS.</li>\n<li>Repeat the previous steps with the rest of your apps.</li>\n<li>Once all apps have been migrated, begin migrating your data stores.</li>\n</ol>\n<p>Read on for the long version!</p>\n<h2 class=\"preview__body--subtitle\" id=\"set-up-account-access\">Set up account access</h2>\n<p>The first step to migrating to the Reference Architecture is to make sure everyone on your team has access to your AWS\naccounts. This consists of the following items:</p>\n<ol>\n<li><a href=\"#security-primer\" class=\"preview__body--description--blue\">Security primer</a></li>\n<li><a href=\"#root-user\" class=\"preview__body--description--blue\">Root user</a></li>\n<li><a href=\"#iam-users\" class=\"preview__body--description--blue\">IAM users</a></li>\n<li><a href=\"#iam-groups\" class=\"preview__body--description--blue\">IAM groups</a></li>\n<li><a href=\"#ssh-access\" class=\"preview__body--description--blue\">SSH access</a></li>\n<li><a href=\"#vpn-access\" class=\"preview__body--description--blue\">VPN access</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"security-primer\">Security primer</h3>\n<p>We <strong>strongly</strong> recommend that everyone on your team reads through the following:</p>\n<ol>\n<li><a href=\"https://docs.google.com/document/u/1/d/e/2PACX-1vTikva7hXPd2h1SSglJWhlW8W6qhMlZUxl0qQ9rUJ0OX22CQNeM-91w4lStRk9u2zQIn6lPejUbe-dl/pub\" class=\"preview__body--description--blue\" target=\"_blank\">Gruntwork Security Best\nPractices</a>,\ndoc, which covers critical concepts such as how to securely store secrets, how to lock down servers, and how to\nsecurely use AWS.</li>\n<li><a href=\"https://blog.gruntwork.io/a-comprehensive-guide-to-authenticating-to-aws-on-the-command-line-63656a686799\" class=\"preview__body--description--blue\" target=\"_blank\">A Comprehensive Guide to Authenticating to AWS on the Command\nLine</a>,\nwhich covers all the basics of AWS auth, including all the different options for how to authenticate on the CLI.</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"root-user\">Root user</h3>\n<p>Each of your AWS accounts has a <a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_root-user.html\" class=\"preview__body--description--blue\" target=\"_blank\">root user</a>. Whoever\ncreated the account should have access to the root user. If you created the account using the <a href=\"/repos/gruntwork\" class=\"preview__body--description--blue\">gruntwork\nCLI</a> or <a href=\"https://aws.amazon.com/organizations/\" class=\"preview__body--description--blue\" target=\"_blank\">AWS Organizations</a>, then\nyou will know the email address associated with the root user, but not the password. To get the password:</p>\n<ol>\n<li>Go to the <a href=\"https://console.aws.amazon.com/console/home\" class=\"preview__body--description--blue\" target=\"_blank\">AWS Console</a></li>\n<li>If you had previously signed into some other AWS account, click "Sign-in using root account credentials."</li>\n<li>Enter the email address and click "Forgot your password" to reset the password.</li>\n<li>Check the email address associated with the root user account for a link you can use to create a new password.</li>\n</ol>\n<p>Please note that the root user account can do just about <em>anything</em> in your AWS account, bypassing almost all security\nrestrictions you put in place, so you need to take extra care with protecting this account. We <strong>very strongly</strong>\nrecommend that you:</p>\n<ol>\n<li>Use a strong password. Preferably 30+ characters, randomly generated, and stored in a secrets manager.</li>\n<li><a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable_virtual.html#enable-virt-mfa-for-root\" class=\"preview__body--description--blue\" target=\"_blank\">Enable Multi-Factor Auth</a>\nfor the root user.</li>\n<li><em>Only</em> use the root user to create an IAM User for yourself and then switch to the IAM User for all other operations\nthereafter. You should NOT use the root user account on a day-to-day basis; it's there only for very rare\ncircumstances (e.g., if you get locked out of your IAM User account).</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"iam-users\">IAM users</h3>\n<p>You should create an <a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html\" class=\"preview__body--description--blue\" target=\"_blank\">IAM User</a> for each\ndeveloper. The Reference Architecture enforces a\n<a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_passwords_account-policy.html\" class=\"preview__body--description--blue\" target=\"_blank\">password policy</a>: see\nthe <code>iam-user-password-policy</code> module in <code>infrastructure-live-acme</code> for the details. We also <strong>strongly</strong> recommend\nrequiring every developer to <a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa.html\" class=\"preview__body--description--blue\" target=\"_blank\">enable multi-factor auth</a>\nfor their IAM users.</p>\n<h3 class=\"preview__body--subtitle\" id=\"iam-groups\">IAM groups</h3>\n<p>Give each IAM User permissions by adding them to the appropriate <a href=\"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html\" class=\"preview__body--description--blue\" target=\"_blank\">IAM\nGroups</a>. The following IAM Groups may be of especial\ninterest (see the <a href=\"/repos/module-security/modules/iam-groups\" class=\"preview__body--description--blue\">iam-groups</a> module\nfor a more complete list):</p>\n<ol>\n<li><strong>full-access</strong>: This gives full admin privileges within this AWS account. Use this only for a small number of\ntrusted admins.</li>\n<li><strong>read-only</strong>: This gives read-only access within this AWS account. This is especially useful for production\naccounts where you want to be able to debug, but not necessarily make changes.</li>\n<li><strong>openvpn-users</strong>: This allows users to use <a href=\"/repos/package-openvpn/modules/openvpn-admin\" class=\"preview__body--description--blue\">openvpn-admin</a>\nto request VPN certificates in this AWS account. See <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/08-ssh-vpn.md\" class=\"preview__body--description--blue\">SSH and VPN</a> for more info.</li>\n<li><strong>ssh-grunt-users</strong>: Users in this group will be able to access servers via SSH. See <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/08-ssh-vpn.md\" class=\"preview__body--description--blue\">SSH and VPN</a> for\nmore info.</li>\n<li><strong>ssh-grunt-sudo-users</strong>: Users in this group will be able to access servers via SSH with sudo permissions. See\n<a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/08-ssh-vpn.md\" class=\"preview__body--description--blue\">SSH and VPN</a> for more info.</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"ssh-access\">SSH Access</h3>\n<p>We have configured <a href=\"/repos/module-security/modules/ssh-grunt\" class=\"preview__body--description--blue\">ssh-grunt</a> on every\nEC2 Instance, so developers will be able to SSH to the servers using their own usernames and SSH keys. See\n<a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/08-ssh-vpn.md\" class=\"preview__body--description--blue\">SSH and VPN</a> for instructions.</p>\n<h3 class=\"preview__body--subtitle\" id=\"vpn-access\">VPN Access</h3>\n<p>All the EC2 Instances run in private subnets, so to access them, you must first connect via VPN. See <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/08-ssh-vpn.md\" class=\"preview__body--description--blue\">SSH and\nVPN</a> for instructions.</p>\n<p>Note that if you are migrating to AWS from some other data center, you may also want to set up VPN access back to that\ndata center. See <a href=\"https://aws.amazon.com/getting-started/projects/connect-data-center-to-aws/\" class=\"preview__body--description--blue\" target=\"_blank\">How to connect your data center to\nAWS</a> and <a href=\"https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/SetUpVPNConnections.html\" class=\"preview__body--description--blue\" target=\"_blank\">Setting Up an AWS VPN\nConnection</a> for instructions.</p>\n<p>This will involve setting up <a href=\"https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_VPN.html#VPNGateway\" class=\"preview__body--description--blue\" target=\"_blank\">Virtual Private Gateway</a>,\nand you will need to configure the VPCs to propagate routes for these Gateways using the <code>private_propagating_vgws</code>\nand <code>persistence_propagating_vgws</code> variables.</p>\n<h2 class=\"preview__body--subtitle\" id=\"deploy-your-apps\">Deploy your apps</h2>\n<p>Once account access is set up, the next step is to migrate your apps—or even just a single app—to the Reference\nArchitecture, while continuing to use whatever data stores and other dependencies you already have deployed in your old\ndata center (e.g., via a VPN connection). Once you can get one app running in AWS successfully, you can use it as a\ntemplate to move over the others, getting value from the migration right away (i.e., more security, scalability, etc)\nwithout having to wait weeks or months to move everything over (incrementalism!).</p>\n<p>To move over an app, you'll need to do the steps listed below. The Reference Architecture includes two sample apps\n(<code>sample-app-frontend-acme</code> and <code>sample-app-backend-acme</code>) that show how to implement all of these steps, so use them as a\nreference!</p>\n<p>Note that you do NOT need to do all of these steps at once! You can take care of just a few of them, deploy the app\ninto the Reference Architecture, see that some basic things work, move on to the next few steps, and so on.</p>\n<ol>\n<li><a href=\"#package-the-app\" class=\"preview__body--description--blue\">Package the app</a></li>\n<li><a href=\"#add-a-health-check-endpoint\" class=\"preview__body--description--blue\">Add a health check endpoint</a></li>\n<li><a href=\"#update-the-load-balancer-config\" class=\"preview__body--description--blue\">Update the load balancer config</a></li>\n<li><a href=\"#set-up-config-files-for-each-environment\" class=\"preview__body--description--blue\">Set up config files for each environment</a></li>\n<li><a href=\"#encrypt-secrets\" class=\"preview__body--description--blue\">Encrypt secrets</a></li>\n<li><a href=\"#configure-schema-migrations\" class=\"preview__body--description--blue\">Configure schema migrations</a></li>\n<li><a href=\"#configure-service-discovery\" class=\"preview__body--description--blue\">Configure service discovery</a></li>\n<li><a href=\"#set-up-static-content\" class=\"preview__body--description--blue\">Set up static content</a></li>\n<li><a href=\"#configure-ci-and-cd\" class=\"preview__body--description--blue\">Configure CI and CD</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"package-the-app\">Package the app</h3>\n<p>To deploy into the Reference Architecture, you will need to package your app using one of the following tools:</p>\n<ol>\n<li>\n<p><a href=\"https://www.packer.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Packer</a>: If you are deploying your apps directly on EC2 Instances, use Packer to package\nyour apps as <a href=\"https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html\" class=\"preview__body--description--blue\" target=\"_blank\">Amazon Machine Images</a>.</p>\n</li>\n<li>\n<p><a href=\"https://www.docker.com/\" class=\"preview__body--description--blue\" target=\"_blank\">Docker</a>: If you are deploying your apps as Docker containers, create a <code>Dockerfile</code> to\npackage your app as a Docker image.</p>\n</li>\n</ol>\n<p>See the <a href=\"/repos/sample-app-frontend-acme\" class=\"preview__body--description--blue\">sample-app-frontend-acme</a> and\n<a href=\"/repos/sample-app-backend-acme\" class=\"preview__body--description--blue\">sample-app-backend-acme</a> repos for examples.</p>\n<h3 class=\"preview__body--subtitle\" id=\"add-a-health-check-endpoint\">Add a health check endpoint</h3>\n<p>Each app should expose a health check endpoint (e.g., <code>/health</code>) that the load balancer can use to see if the app is\nrunning and ready to receive requests. The health check endpoint should return a <code>200 OK</code> when the app is healthy.\nThe definition of "healthy" depends on the app; for some apps, healthy might mean the app has booted; for others,\nhealthy may mean the app has booted, can talk to databases, and has warmed up its cache.</p>\n<h3 class=\"preview__body--subtitle\" id=\"update-the-load-balancer-config\">Update the load balancer config</h3>\n<p>Now that your app is packaged and has a health check endpoint, you can try deploying it! The sample apps are deployed\nvia Terraform code in the <code>infrastructure-live-acme</code> repo, under the <code>services</code> folder (e.g.,\n<code>stage/services/sample-app-frontend-acme</code>). Make a copy of that folder and customize the <code>terragrunt.hcl</code> file within it\nto deploy your app instead. This will include specifying which paths in the load balancer your app should claim.</p>\n<p>This is because the Reference Architecture uses a single <a href=\"https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html\" class=\"preview__body--description--blue\" target=\"_blank\">Application Load Balancer\n(ALB)</a> for several apps in\neach environment, and each of those apps can claim certain paths (e.g., <code>/foo</code> goes to app foo, <code>/bar</code> goes to app bar)\nand/or certain domain names (e.g., <code>foo.acme.com</code> goes to app foo, <code>bar.acme.com</code> goes to app bar). If you want your\napp to handle all paths, you can use regular expressions (e.g., <code>*</code>).</p>\n<h3 class=\"preview__body--subtitle\" id=\"set-up-config-files-for-each-environment\">Set up config files for each environment</h3>\n<p>If your app needs to be configured differently in each environment (e.g., in stage and prod), we recommend creating\nconfig files for each environment, checking those configs into version control, and packaging them with the app. That\nway, config is updated, versioned, tested, and deployed exactly like the rest of your code. This tends to be easier\nto reason about and far less error prone than storing configs externally (e.g., in a DB).</p>\n<p>The sample apps use JSON files for config, but any format supported by your apps will be fine, so long as it is\nfile-based. See the <code>config</code> folder in the sample apps for an example, and check out the <code>bin/run-app.sh</code> script for\nhow the sample apps select the proper config file for the current environment.</p>\n<h3 class=\"preview__body--subtitle\" id=\"encrypt-secrets\">Encrypt secrets</h3>\n<p>If your apps need access to secrets, such as the password to a database, we recommend encrypting those secrets with\n<a href=\"https://aws.amazon.com/kms/\" class=\"preview__body--description--blue\" target=\"_blank\">KMS</a> and adding the ciphertext to your config files. Your apps can then use IAM Roles to\ndecrypt the secrets just before booting.</p>\n<p>See <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/05-dev-environment.md#encrypt-a-secret\" class=\"preview__body--description--blue\">encrypt a secret</a> for instructions.</p>\n<h3 class=\"preview__body--subtitle\" id=\"configure-schema-migrations\">Configure schema migrations</h3>\n<p>If your apps talk to a relational database, you will need a way to evolve the database schema over time. The best way\nto do that is to use a schema migration tool such as <a href=\"https://flywaydb.org/\" class=\"preview__body--description--blue\" target=\"_blank\">Flyway</a> or\n<a href=\"https://www.liquibase.org/\" class=\"preview__body--description--blue\" target=\"_blank\">Liquibase</a> to define the schema migrations as code, check that code into version control,\npackage it with your apps, and have your apps apply the schema migrations just before booting (Flyway and Liquibase\nwill use locks to ensure there are no concurrency issues).</p>\n<p>See the <code>sql</code> folder and <code>bin/run-app.sh</code> of <code>sample-app-backend-acme</code> for an example.</p>\n<h3 class=\"preview__body--subtitle\" id=\"configure-service-discovery\">Configure service discovery</h3>\n<p>If you need to run multiple apps ("microservices") that talk to each other, those apps will need a way to discover each\nother's IP and port. In the dynamic world of AWS, IPs and ports change all the time, so you don't want to hard code\nthem. Instead, use one of the following options:</p>\n<ol>\n<li>\n<p>Use an internal load balancer. Each of your apps can register at a different path with the load balancer, and each\napp gets the load balancer domain name as an environment variable (via Terraform), so, for example, you can talk to\napp <code>foo</code> by sending a request to <code><LOAD_BALANCER_DOMAIN_NAME>/foo</code> and to app <code>bar</code> by sending a request to\n<code><LOAD_BALANCER_DOMAIN_NAME>/bar</code>. This is how <code>sample-app-frontend-acme</code> talks to <code>sample-app-backend-acme</code>.</p>\n</li>\n<li>\n<p>For Docker containers running in ECS, you can use <a href=\"https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html\" class=\"preview__body--description--blue\" target=\"_blank\">ECS Service\nDiscovery</a>. As of May, 2018,\nGruntwork has not yet added support for this, but it should be added to the <a href=\"/repos/module-ecs\" class=\"preview__body--description--blue\">ECS\nmodules</a> soon.</p>\n</li>\n<li>\n<p>Use a tool such as <a href=\"https://www.consul.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Consul</a>. This does require running extra infrastructure, which is made\neasier via the Gruntwork <a href=\"/repos/terraform-aws-consul\" class=\"preview__body--description--blue\">Consul module</a> (not currently part of\nthe Reference Architecture).</p>\n</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"set-up-static-content\">Set up static content</h3>\n<p>The best way to serve CSS, JS, images, and fonts is by putting them in an <a href=\"https://aws.amazon.com/s3/\" class=\"preview__body--description--blue\" target=\"_blank\">S3 bucket</a> and\nusing <a href=\"https://aws.amazon.com/cloudfront/\" class=\"preview__body--description--blue\" target=\"_blank\">CloudFront</a> as a CDN in front of it. Your Reference Architecture may already\ncontain an example of this under the <code>services/static-website</code> folder. Your CloudFront distribution will have its own\ndomain name (e.g., <code>static.<your-company>.com</code>), so you'll need to ensure that any code of yours that links to static\ncontent (e.g., server-side rendered HTML) makes it possible to configure a different static content domain name in\neach environment.</p>\n<h3 class=\"preview__body--subtitle\" id=\"configure-ci-and-cd\">Configure CI and CD</h3>\n<p>The sample apps have Continuous Integration (CI) and Continuous Delivery (CD) configured for them. A CI job will run\nafter every commit to build, test, and package the apps, and, for certain commits (e.g., to specific branches or with\nspecific tags), the CI job will automatically deploy the app to stage or prod.</p>\n<p>See <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/06-ci-cd.md\" class=\"preview__body--description--blue\">Build, tests, and deployment (CI/CD)</a> and the sample app repos for details on how to set up the same\nCI/CD workflow for your real apps.</p>\n<h2 class=\"preview__body--subtitle\" id=\"migrate-your-data-stores\">Migrate your data stores</h2>\n<p>Once you have your apps running in AWS, the next step is to migrate the data stores they depend on. We again recommend\ndoing this incrementally, moving over one data store at a time, if possible. There are two ways to move over your data\nstores:</p>\n<ol>\n<li>Take a downtime</li>\n<li>Do a zero-downtime migration</li>\n</ol>\n<p>Option #2 typically takes 100x longer (this is NOT an exaggeration) than option #1 and has a much higher risk of\ndata loss or data corruption. If at all possible, we <strong>very strongly</strong> recommend taking a brief downtime to do the\nmigration, as it will make your life much easier.</p>\n<p>A few guidelines about migrating data stores:</p>\n<ol>\n<li><a href=\"#migrating-from-another-aws-managed-data-store\" class=\"preview__body--description--blue\">Migrating from another AWS-managed data store</a></li>\n<li><a href=\"#migrating-from-external-data-stores\" class=\"preview__body--description--blue\">Migrating from external data stores</a></li>\n<li><a href=\"#if-you-absolutely-must-do-a-zero-downtime-migration\" class=\"preview__body--description--blue\">If you absolutely must do a zero-downtime migration</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"migrating-from-another-aws-managed-data-store\">Migrating from another AWS-managed data store</h3>\n<p>If you are migrating to the Reference Architecture from another AWS account, and in that account you are using an\nAWS-managed data store such as <a href=\"https://aws.amazon.com/rds/\" class=\"preview__body--description--blue\" target=\"_blank\">RDS</a> or <a href=\"https://aws.amazon.com/elasticache/\" class=\"preview__body--description--blue\" target=\"_blank\">ElastiCache</a>,\nthe migration process (with downtime!) is:</p>\n<ol>\n<li>Put your app in read-only mode or take a downtime.</li>\n<li>Use the built-in snapshotting mechanism to take a snapshot of the data store (e.g., <a href=\"https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateSnapshot.html\" class=\"preview__body--description--blue\" target=\"_blank\">RDS\nsnapshots</a> or\n<a href=\"https://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_Snapshot.html\" class=\"preview__body--description--blue\" target=\"_blank\">ElastiCache snapshots</a>).</li>\n<li>Share the snapshot with the prod account where the Reference Architecture is deployed (e.g., see <a href=\"https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ShareSnapshot.html\" class=\"preview__body--description--blue\" target=\"_blank\">Sharing a DB\nSnapshot</a>).</li>\n<li>Deploy a new data store in the Reference Architecture account from the snapshot (e.g., set the\n<code>snapshot_identifier</code> parameter in the <a href=\"/repos/module-data-storage/modules/rds\" class=\"preview__body--description--blue\">rds module</a>).</li>\n<li>Bring your app back up.</li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"migrating-from-external-data-stores\">Migrating from external data stores</h3>\n<p>If you are migrating data stores from non-AWS managed systems (e.g., from your own data center), the process is the\nsame as in the previous section, except instead of using the snapshotting mechanism built into AWS, you will need to\nuse one of these mechanisms instead:</p>\n<ol>\n<li>Use the <a href=\"https://aws.amazon.com/dms/\" class=\"preview__body--description--blue\" target=\"_blank\">AWS database migration service</a> to migrate relational databases.</li>\n<li>Use a mechanism supported by RDS and your database, such as <a href=\"https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Importing.html\" class=\"preview__body--description--blue\" target=\"_blank\">restoring MySQL into RDS using Percona\nXtraBackup</a>.</li>\n<li>Use your data store's native snapshot functionality (e.g., <a href=\"https://dev.mysql.com/doc/refman/5.5/en/mysqldump.html\" class=\"preview__body--description--blue\" target=\"_blank\">mysqldump</a>)\nto make a backup of your database, copy the backup into AWS (e.g., into an S3 bucket), and fire up an EC2 Instance\nthat connects to the data store and loads the backup into it.</li>\n</ol>\n<p>Note that all of these assume at least some amount of downtime!</p>\n<h3 class=\"preview__body--subtitle\" id=\"if-you-absolutely-must-do-a-zero-downtime-migration\">If you absolutely must do a zero-downtime migration</h3>\n<p>If you absolutely cannot take any downtime whatsoever (unlikely, as you most likely have unplanned outages anyway!),\nthings get a lot more complicated. The typical strategy is:</p>\n<ol>\n<li>Take an initial snapshot of your data store and load it into AWS. The idea is to move most of the data over early on\nand then "catch up" whatever new data is written in the following steps.</li>\n<li>Find a way to send all new writes to both your original data store and the new one in AWS.</li>\n<li>Once the new data store has "caught up", start using it directly instead of the old database.</li>\n</ol>\n<p>Step #2 is the tricky one, as dual writes are notoriously error prone. For example, what happens if the process doing\ndual writes succeeds with the first write, but crashes before the second? What if it sends the requests to both data\nstores, but one of them is temporarily down? What if both requests make it to the data stores, but then a transaction\ngets rolled back? See <a href=\"https://www.confluent.io/blog/using-logs-to-build-a-solid-data-infrastructure-or-why-dual-writes-are-a-bad-idea/\" class=\"preview__body--description--blue\" target=\"_blank\">why dual writes are a bad\nidea</a>\nfor more info.</p>\n<p>The most common solutions to these problems are:</p>\n<ol>\n<li>\n<p>Use a change capture system. This is a system that effectively replicates your data store's underlying log: you\nwrite to the original data store, and if this write has succeeded and been committed, it triggers an "event" in the\nchange capture system, which an event processor then writes to the second data store. For example,\nsee <a href=\"https://github.com/linkedin/databus\" class=\"preview__body--description--blue\" target=\"_blank\">databus</a> and <a href=\"https://github.com/confluentinc/bottledwater-pg\" class=\"preview__body--description--blue\" target=\"_blank\">bottledwater-pg</a>.</p>\n</li>\n<li>\n<p>Send all writes initially to a log system such as <a href=\"https://kafka.apache.org/\" class=\"preview__body--description--blue\" target=\"_blank\">Apache Kafka</a>. You then have one\nKafka consumer that writes the messages to your original data store and another that writes it to the new data store.\nKafka guarantees at-least once delivery, so as long as the database writes are idempotent, this ensures no data\nwill be lost.</p>\n</li>\n</ol>\n<p>Whatever option you go with, we strongly recommend adding "integrity checking" to verify no data was lost or corrupted.\nSome example integrity checks:</p>\n<ol>\n<li>Count the number of rows in each table and make sure they are identical in both data stores.</li>\n<li>Perform a checksum or hash on all the data in a table (or some randomly selected subset if there's too much data)\nand make sure you get identical values in both data stores.</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"configure-monitoring-and-alerting\">Configure monitoring and alerting</h2>\n<p>Once your apps and data stores are working, you'll want to make sure you have the following monitoring and alerting in\nplace:</p>\n<ol>\n<li><a href=\"#metrics\" class=\"preview__body--description--blue\">Metrics</a></li>\n<li><a href=\"#alerts\" class=\"preview__body--description--blue\">Alerts</a></li>\n<li><a href=\"#logs\" class=\"preview__body--description--blue\">Logs</a></li>\n</ol>\n<h3 class=\"preview__body--subtitle\" id=\"metrics\">Metrics</h3>\n<p>Make sure all the metrics you expect show up in <a href=\"https://aws.amazon.com/cloudwatch/\" class=\"preview__body--description--blue\" target=\"_blank\">CloudWatch</a>. All AWS services\nautomatically send metrics to CloudWatch and the Reference Architecture includes a few important ones that are not\navailable by default (e.g., memory and disk space usage on EC2 Instances). Create\n<a href=\"https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html\" class=\"preview__body--description--blue\" target=\"_blank\">Dashboards</a> for the\nmost important metrics, such as CPU usage in your ECS cluster or request count in the ALB.</p>\n<p>Note that the Reference Architecture uses CloudWatch for metrics as it's built-into AWS, very inexpensive, and requires\nvery little work to maintain. However, the CloudWatch UI is fairly basic, so if you need more powerful tools for slicing\nand dicing your metrics, consider tools such as <a href=\"https://www.datadoghq.com/\" class=\"preview__body--description--blue\" target=\"_blank\">DataDog</a>,\n<a href=\"https://newrelic.com/\" class=\"preview__body--description--blue\" target=\"_blank\">New Relic</a>, or <a href=\"https://prometheus.io/\" class=\"preview__body--description--blue\" target=\"_blank\">Prometheus</a>. If you'd like help with this, please reach\nout to support@gruntwork.io.</p>\n<h3 class=\"preview__body--subtitle\" id=\"alerts\">Alerts</h3>\n<p>Configure alerts to go off for key metrics. All the services in the Reference Architecture have basic alarms\nconfigured directly in the Terraform code, such as low disk space alarms on RDS DBs and high CPU usage alarms for the\nECS cluster. All of the Terraform modules allow you to tweak the settings for these alarms (e.g., should the CPU\nusage alarm go off at 80% or 90%?), so take some time to go through each module in <code>infrastructure-modules-acme</code> and\nmake sure the settings match your use cases so you don't have too many false positives or negatives.</p>\n<h3 class=\"preview__body--subtitle\" id=\"logs\">Logs</h3>\n<p>The logs from all servers get automatically sent to <a href=\"https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html\" class=\"preview__body--description--blue\" target=\"_blank\">CloudWatch\nLogs</a>. The <code>user-data.sh</code> scripts\nin <code>infrastructure-modules-acme</code> call <code>run-cloudwatch-logs-agent.sh</code> (from the <a href=\"/repos/module-aws-monitoring/modules/logs/cloudwatch-log-aggregation-scripts\" class=\"preview__body--description--blue\">cloudwatch-log-aggregation-scripts\nmodule</a>)\nto configure which log files get sent to CloudWatch. By default, <code>syslog</code> is sent to CloudWatch on every server, and\nECS Services are configured to send their logs to <code>syslog</code>. Make sure you can find the logs for your apps there\n<em>before</em> you go live!</p>\n<p>Note that the Reference Architecture uses CloudWatch Logs as it's built-into AWS, very inexpensive, and requires\nvery little work to maintain. However, the CloudWatch UI is fairly basic, so if you need more powerful tools for slicing\nand dicing your log data, consider tools such as <a href=\"https://www.elastic.co/elk-stack\" class=\"preview__body--description--blue\" target=\"_blank\">ELK</a> (as of May, 2018, the ELK\nsupport is under development in the <a href=\"/repos/package-elk\" class=\"preview__body--description--blue\">package-elk</a> repo and should be\nready soon), <a href=\"https://www.loggly.com/\" class=\"preview__body--description--blue\" target=\"_blank\">Loggly</a>, and <a href=\"https://papertrailapp.com/\" class=\"preview__body--description--blue\" target=\"_blank\">Papertrail</a>. If you'd like help with\nthis, please reach out to support@gruntwork.io.</p>\n<h2 class=\"preview__body--subtitle\" id=\"test\">Test</h2>\n<p>As you go through the process of migrating your apps and data stores, you should continuously be testing your\ninfrastructure to make sure things are working. Here are the key types of tests to perform:</p>\n<ol>\n<li>\n<p><strong>Manual testing</strong>: Hit URLs by hand, look at log files, and check the metrics. You already know how to do this!</p>\n</li>\n<li>\n<p><strong>Automated testing</strong>: We strongly recommend writing automated tests that (a) deploy your infrastructure into an\nAWS account dedicated for testing, (b) check the infrastructure works as expected, and (c) tears the infrastructure\nback down at the end of the test. You can use the <a href=\"/repos/terratest\" class=\"preview__body--description--blue\">Terratest</a> library\nto write these tests.</p>\n</li>\n<li>\n<p><strong>Load testing</strong>: You should make sure your code performs well under the kind of loads you expect in production.\nUse tools such as <a href=\"https://httpd.apache.org/docs/2.4/programs/ab.html\" class=\"preview__body--description--blue\" target=\"_blank\">Apache Bench</a>,\n<a href=\"https://jmeter.apache.org/\" class=\"preview__body--description--blue\" target=\"_blank\">JMeter</a> to throw traffic at your infrastructure and when things fall over, make sure\nyour monitoring and alerting can help you find the bottlenecks! Load testing across several different EC2 Instance\nTypes can also be a good way to find the sweet spot between price and performance (see <a href=\"https://www.slideshare.net/brendangregg/how-netflix-tunes-ec2-instances-for-performance\" class=\"preview__body--description--blue\" target=\"_blank\">How Netflix Tunes EC2\nInstances for Performance</a>).</p>\n</li>\n<li>\n<p><strong>Security testing</strong>: Bring in an outside firm to pen test and audit your code to look for security vulnerabilities.\nGruntwork customers have done this several times with the Reference Architecture already, and we've fixed all issues\nthat were found, but if you find any new vulnerabilities, please report them to support@gruntwork.io!</p>\n</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"switch-over-dns\">Switch over DNS</h2>\n<p>The Reference Architecture is typically deployed with "placeholder" domain names (e.g., <code>your-company-aws.com</code>) to make\ntesting easy and safe. Once your apps start passing tests, you can switch over your real domain names to point to\nthe Reference Architecture as follows:</p>\n<ol>\n<li>\n<p>We recommend managing all DNS entries using <a href=\"https://aws.amazon.com/route53/\" class=\"preview__body--description--blue\" target=\"_blank\">Route 53</a>. If you bought your\nproduction domain name using some other registrar, you can either configure <a href=\"https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/MigratingDNS.html\" class=\"preview__body--description--blue\" target=\"_blank\">Route 53 as the DNS\nService</a> for your existing domain name\nor <a href=\"https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-transfer-to-route-53.html\" class=\"preview__body--description--blue\" target=\"_blank\">transfer the domain name entirely to Route\n53</a>.</p>\n</li>\n<li>\n<p>If you have multiple AWS accounts (e.g., dev, stage, prod), we recommend that each one use its own, completely\nseparate domain names (e.g., <code>my-company-dev.com</code>, <code>my-company-stage.com</code>, etc.). This reduces the chances of\naccidental errors (e.g., breaking prod DNS entries while trying to make changes to stage) and makes it harder for\nattackers to do damage (e.g., using cookies from pre-prod environment in a prod environment).</p>\n</li>\n<li>\n<p>Before switching the domain names on your apps, make to use <a href=\"https://aws.amazon.com/certificate-manager/\" class=\"preview__body--description--blue\" target=\"_blank\">AWS Certificate Manager\n(ACM)</a> to request TLS certificates for your prod domain names and\nassociate those certificates with your Load Balancers and CloudFront distributions. ACM is a great choice for\nTLS certificates, because they are (a) free and (b) issue in 1-2 minutes for domain names you own in Route 53, and\n(c) renew automatically!</p>\n</li>\n<li>\n<p>To switch over your ALBs or ECS services to use different domain names, simply update the <code>domain_name</code> variables in\nthe corresponding <code>terragrunt.hcl</code>.</p>\n</li>\n</ol>\n<h2 class=\"preview__body--subtitle\" id=\"next-steps\">Next steps</h2>\n<p>Now that you know how to migrate to the Reference Architecture, the next thing to learn is <a href=\"/repos/v0.0.1-01172020/infrastructure-live-acme/_docs/13-deploying-the-reference-architecture-from-scratch.md\" class=\"preview__body--description--blue\">how to deploy the\nReference Architecture from scratch</a>.</p>\n","repoName":"infrastructure-live-acme","repoRef":"v0.0.1-01172020","serviceDescriptor":{"serviceName":"Single-account Reference Architecture","serviceRepoName":"infrastructure-live-acme","serviceRepoOrg":"gruntwork-io","cloudProviders":["aws"],"description":"End-to-end tech stack designed to deploy into a single AWS account. Includes VPCs, EKS, ALBs, CI / CD, monitoring, alerting, VPN, DNS, and more.","imageUrl":"grunt.png","licenseType":"subscriber","technologies":["Terraform","Go","Bash","Python"],"compliance":[],"tags":[""]},"serviceCategoryName":"Reference Architecture","fileName":"12-migration.md","filePath":"/_docs/12-migration.md","title":"Repo Browser: Single-account Reference Architecture","description":"Browse the repos in the Gruntwork Infrastructure as Code Library."}
_docs
.gitignore
