Javascript

Putting Your NodeJS App in a Docker Container

Published Feb 22, 2021

Updated Feb 14, 2023

12 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Putting Your App in a Docker Container

The days of having to provision servers and VMs by hand or by using complicated and heavy handed toolchains like Chef, Puppet, and Ansible are over. Docker simplifies the process by providing developers with a simple domain specific language for creating pre-configured virtual machine images, and simple tools for building, publishing and running them on the (virtual) hardware you're already using. In this guide, I will show you how to install your NodeJS application into a Docker container.

The Dockerfile Language

Docker containers are built using a single file called a Dockerfile. This file uses a simple domain specific language (think SQL) to define how to configure a virtual machine to run your application. The language provides a small number of commands called "instructions" that can be used to define the steps required to build a new virtual machine called a "container". First, I'll explain which instructions we'll be using and what they'll be used for.

FROM

The FROM instruction is used to define a base image to use as the foundation for your custom image. You can use any local or published image with FROM. There are published images that only contain popular Linux distributions (or even Windows!) and there are also images that come preinstalled with popular software development stacks like NodeJS, Python, or .NET.

RUN

The RUN instruction is used in the image build process to run commands required to bootstrap your application environment. We'll use it mostly to install dependencies, but it's capable of running any command that your container OS supports.

COPY

The COPY instruction is used to copy files from the local filesystem into the container. We will use this instruction to copy our application code, etc., into our image.

ENTRYPOINT

The ENTRYPOINT instruction contains a command that will be run when your container is launched. It is different from RUN because the command passed to ENTRYPOINT does not run at build time. Instead the command passed to ENTRYPOINT will run when your container is started via docker run (check out my Docker CLI Deep Dive post). Only a single ENTRYPOINT instruction per Dockerfile is allowed. If used multiple times, only the last usage will be operative. The value of ENTRYPOINT can be overridden when running a container image.

CMD

The CMD instruction is an extension of the ENTRYPOINT instruction. The content passed to CMD is tacked onto the end of the command passed to ENTRYPOINT to create a complete command to start your application. Like with ENTRYPOINT, only the final usage of CMD in a Dockerfile is operative and the value given can be overridden at runtime.

EXPOSE

EXPOSE is a little different from the other instructions in that it doesn't have a practical purpose. It only exists to provide metadata about what port the container expects to expose. You don't need to use EXPOSE in your container, but anyone who has to understand how to connect to it will appreciate it.

And more...

More details about these Dockerfile instructions and some others that I didn't cover are available via the official Dockerfile reference.

Choosing a Base Image

Generally, choosing a base image will be simple. If you're using a popular language, there is most likely already an official image available that installs the language runtimes. For instance, NodeJS application developers will most likely find it easiest to use the official NodeJS image provided via Dockerhub. This image is developed by the NodeJS team and comes pre-installed with everything you need to run basic NodeJS applications. Similarly, users of other popular languages will find similar images available (ex: Python, Ruby). Once you've chosen your base image, you also need to choose which specific version you will use. Generally, images are available with any supported version of a language's toolchain so that a wide range of applications can be supported using official images. You can usually find a list of all available version tags on an image's DockerHub page.

In addition to offering images with different versions of language tools installed, there are also typically additional images available with different operating systems as well. Unless specified otherwise, images usually use the most recent Debian Linux release as their base image. Since it's considered best practice to keep the size of your images as low as possible, most languages also offer variants of their images built with a "slim" version of Debian Linux, or built with Alpine Linux, a Linux distribution designed for building Docker containers with tiny footprints. Both Debian Slim and Alpine ship with fewer system packages installed than the typical Debian Linux base image. They only include the packages that are required to run the language tools. This will make your Docker images more compact, but may result in more work to build your containers if you require specific system dependencies that are not preinstalled in those versions. Some languages, like .NET Core, even offer Windows-based images.

Though it's typically not necessary, you can choose to use a base operating system image without any additional language specific tools installed by default. Images containing only Debian Linux, Debian Slim, and Alpine Linux are available. However, the most popular images contain many other operating systems like Ubuntu Linux, Red Hat Linux or Windows are available as well. Choosing one of these images will add much more complexity to your Dockerfile. It is highly reccommended that you use a more specific official image if your use case allows.

In the interest of keeping things simple for our example NodeJS app, we will choose the most recent (at the time of writing) Debian Linux version of the official NodeJS image. This image is called node:15.

Note that we have only included a major version number in the image's version tag (The "version tag" is the part after the colon that specifies a specific version of an image). The NodeJS team (as well as most other maintainers of official images) also publishes images with more specific versions of Node. Using node:15 instead of node:15.5.1 means that my image with be automatically upgraded to new versions of NodeJS 15 at build time when an update is available. This is good for development, but for production workloads, you may want to use a more specific version so you don't get surprised with upgrades to NodeJS that your application can't support.

Starting Your Dockerfile

Now that we've chosen an image, we will create our Dockerfile. This part is very easy since the FROM instruction is going to do most of the work for us. To get started, simply create a new file in your project's root folder called Dockerfile. To this file, we will add this one simple line:

FROM node:15

Now we have installed everything we need to run a basic NodeJS application along with any other system dependencies that come pre-installed in Debian Linux.

Installing Additional Depenencies

If your application is simple and only requires NodeJS binaries to be installed and run, congratulations! You get to skip this section. Many developers won't be so lucky. If you use a tool like Image Magick to process images or wkhtmltopdf for generating PDFs, or any other libraries or tools that are not included by your chosen language or don't come installed by default on Debian Linux, you will need to add instructions to your Dockerfile so that they will be installed by Docker when your image is built.

We will primarily use the RUN instruction to specify the operating system commands required to install our desired packages. If you recall, RUN is used to give Docker commands to run when building your image. We will use RUN to issue the commands required to install our dependencies. You may choose to use a package management system like Debian's apt-get (or Alpine's apm) or you may install via source. Installing via package manager is always the simplest route, but thanks to the simplicity of the RUN instruction, it's fairly straightforward to install from source if your required package isn't available to install via package management.

Installing Package Dependencies

Using a package manager is the easiest way to install dependencies. The package manager handles most of the heavy lifting like installing dependencies. The node:15 image is based on Debian, so we will use the RUN instruction with the apt-get package manager to install ImageMagick for image processing. Add the following lines to the bottom of our Dockerfile:

RUN apt-get update && \ apt-get install -y imagemagick

This is all the code you need in your Dockerfile to use the RUN instruction to install ImageMagic via apt-get. It's really not very different from how you would install it by hand on an Ubuntu or Debian host. If you've done that before, you probably noticed that there are some unfamiliar instructions. Before we installed using apt-get install, we had to run apt-get update. This is required because in order to keep the docker images small, Debian linux containers don't come with any of the package manager metadata pre-downloaded. apt-get update bootstraps the OS with all the metadata it needs to install packages.

We've also added the -y option to apt-get install. This option automatically answers affimatively to any yes/no prompts when apt-get would otherwise ask for a user response. This is necessary because you will not be able to respond to prompts when Docker is building your image.

Finally, we use the && operator to run both commands within the same shell context. When installing dependencies, it's a good practice to combine commands that are part of the same procedure under the same RUN instruction. This will ensure that the whole procedure is contained in the same "layer" in the container image so that Docker can cache and reuse it to save time in future builds. Check out the official documentation for more information on image layering and caching.

Installing Source Dependencies

Sometimes, since they use pre-compiled binaries, package managers will contain a version of a dependency that doesn't line up with the version you need. In these cases, you'll need to install the dependency from source. If you've done it by hand before, the commands used will be familiar. As with package installs, it's only different in that we use && to combine the whole procedure into a single RUN instruction. Let's install ImageMagick from source this time.

RUN wget https://download.imagemagick.org/ImageMagick/download/ImageMagick-7.0.10-60.tar.gz && \ tar -xzf ImageMagick-7.0.10-60.tar.gz && \ cd ImageMagick-7.0.10-60 && \ ./configure --prefix /usr/local && \ make install && \ ldconfig /usr/local/lib && \ cd .. && \ rm -rf ImageMagick*

As you can see, there's a lot more going on in this instruction. First, we need Docker to download the code for the specific ImageMagic version we want to install with wget, and unpack it using tar. Once the source is unpacked, we have it navigate to the source directory with cd and use ./configure to prepare the code for compilation. Then, make install and ldconfig are used to compile and install the binaries from source. Afterward, we navigate back to the root directory and clean the source tarball and directory since they are no longer needed.

Installing Your App

Now that we've installed dependencies, we can start installing our own application into the container. We will use the COPY instruction to add our own node app's source code to the container, and RUN to install npm dependencies. We'll install NPM dependencies first.

In order to get the most out of Docker's build caching, it's best to install external dependencies first, since your dependency tree will change less often than your application code. A single cache miss will disable caching for the remainder of the build instructions. Application code typically changes more often between builds, so we will apply it as late in the process as we possibly can. To install your application's NPM packages, add these lines to the end of your Dockerfile:

WORKDIR /var/lib/my-app COPY package*.json . RUN npm install

First, we use the WORKDIR instruction to change the Dockerfile's working directory to /var/lib/my-app. This is similar to using the cd command in a shell environment. It changes the working directory for all of the following Docker instructions. Then we use COPY to copy our package.json and package-lock.json from the local filesystem to the working directory within the container. We used the wildcard operator (*), to copy both files with a single instruction. After the package files have been copied, use RUN to execute npm install

Finally, we will use COPY to bring the rest of our application code into the container:

COPY * .

This will copy the rest of your NodeJS app's source code to the container, again using COPY and a much more broad usage of the wildcard. However, since we're using * to copy everything, we need to introduce a new configuration file called .dockerignore to prevent some local files from being copied to the container at this time. For example, we want to make sure that we aren't copying the contents of our local node_modules folder so that the modules we installed previously don't get overwritten by the ones we've installed on our development machine.

It's likely that your local build platform is different from the one in the container, so copying your local node_modules folder will likely cause your app to malfunction or not run at all. The .dockerignore file is very simple. Simply add the names of files or folders that Docker should ignore at build time. You can use the * character as a wildcard just like you can in COPY instructions. Create a .dockerignore with this content:

node_modules/

You may wish to add additional entries to the .dockerignore. For example, if you're using git for version control, you'll want to add the .git/ folder since it's not needed and will unnecessarily increase the size of your image. Any file or directory name you add will be skipped over when copying files via COPY at build time.

Running Your App

Now that we've installed all our external dependencies and copied our application code into the container, we're ready to tell docker how to run our application. We will run our app using node index.js. Docker provides the ENTRYPOINT and CMD instructions for this purpose. Both instructions have nearly the same behavior of defining the command that the container should use to start the application when our container is run, but ENTRYPOINT is less straightforward to override. They can be used together and Docker will concatenate their values and run them as a single command. In this case, you would provide the main application command to ENTRYPOINT (in our case, node) and any arguments to CMD (index.js in our case). However, we're just going to use CMD. Using both would make sense if NodeJS was our main process, but really, our main command is the whole node index.js expression. We could use only ENTRYPOINT but it's more complicated to override an ENTRYPOINT instruction's value at runtime, and we will want to be able to override the main command simply so that it's easier to troubleshoot issues within the conatainer when they arise. With all that said, add the following to the end of your Dockerfile:

CMD ["node", "index.js"]

Now Docker understands what to do to start our application. We provide our command to CMD (and ENTRYPOINT if it's used) in a different way than we supply commands to the RUN instruction. The form we're using for CMD is called "exec form" and the form used for RUN is called "shell form". Using shell form for RUN allows you to access all of the power of the sh shell environment. You can use variable and wildcard substitution in shell form, in addition to other shell features like piping and chaining commands using && and ||. When using exec form, you do not have access to any of these shell features. When passing a command via exec form, each element within the square brackets is joined with a space in between and run exactly as is. Using shell form is preferred for RUN so that you can leverage build arguments and chaining (recall we did that above for better layering/caching). It's better to use exec form for CMD or ENTRYPOINT so that it's always straightforward to understand which action the container takes at runtime.

Conclusion

I hope this article has helped to demystify the process of getting your app into a container. Docker can have a steep learning curve if you're not already a seasoned systems administrator, but features like portability, distribution, and reproducible builds make getting over the hump totally worth it, especially for developers working in teams.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Adam Clarke
@vaxinate

How to host a full-stack app with AWS CloudFront and Elastic Beanstalk

How to host a full-stack JavaScript app with AWS CloudFront and Elastic Beanstalk Let's imagine that you have finished building your app. You have a Single Page Application (SPA) with a NestJS back-end. You are ready to launch, but what if your app is a hit, and you need to be prepared to serve thousands of users? You might need to scale your API horizontally, which means that to serve traffic, you need to have more instances running behind a load balancer. Serving your front-end using a CDN will also be helpful. In this article, I am going to give you steps on how to set up a scalable distribution in AWS, using S3, CloudFront and Elastic Beanstalk. The NestJS API and the simple front-end are both inside an NX monorepo The sample application For the sake of this tutorial, we have put together a very simple HTML page that tries to reach an API endpoint and a very basic API written in NestJS. The UI The UI code is very simple. There is a "HELLO" button on the UI which when clicked, tries to reach out to the /api/hello endpoint. If there is a response with status code 2xx, it puts an h1 tag with the response contents into the div with the id result. If it errors out, it puts an error message into the same div. ` The API We bootstrap the NestJS app to have the api prefix before every endpoint call. ` We bootstrap it with the AppModule which only has the AppController in it. ` And the AppController sets up two very basic endpoints. We set up a health check on the /api route and our hello endpoint on the /api/hello route. ` Hosting the front-end with S3 and CloudFront To serve the front-end through a CMS, we should first create an S3 bucket. Go to S3 in your AWS account and create a new bucket. Name your new bucket to something meaningful. For example, if this is going to be your production deployment I recommend having -prod in the name so you will be able to see at a glance, that this bucket contains your production front-end and nothing should get deleted accidentally. We go with the defaults for this bucket setting it to the us-east-1 region. Let's set up the bucket to block all public access, because we are going to allow get requests through CloudFront to these files. We don't need bucket versioning enabled, because these files will be deleted every time a new front-end version will be uploaded to this bucket. If we were to enable bucket versioning, old front-end files would be marked as deleted and kept, increasing the storage costs in the long run. Let's use server-side encryption with Amazon S3-managed keys and create the bucket. When the bucket is created, upload the front-end files to the bucket and let's go to the CloudFront service and create a distribution. As the origin domain, choose your S3 bucket. Feel free to change the name for the origin. For Origin access, choose the Origin access control settings (recommended). Create a new Control setting with the defaults. I recommend adding a description to describe this control setting. At the Web Application Firewall (WAF) settings we would recommend enabling security protections, although it has cost implications. For this tutorial, we chose not to enable WAF for this CloudFront distribution. In the Settings section, please choose the Price class that best fits you. If you have a domain and an SSL certificate you can set those up for this distribution, but you can do that later as well. As the Default root object, please provide index.html and create the distribution. When you have created the distribution, you should see a warning at the top of the page. Copy the policy and go to your S3 bucket's Permissions tab. Edit the Bucket policy and paste the policy you just copied, then save it. If you have set up a domain with your CloudFront distribution, you can open that domain and you should be able to see our front-end deployed. If you didn't set up a domain the Details section of your CloudFront distribution contains your distribution domain name. If you click on the "Hello" button on your deployed front-end, it should not be able to reach the /api/hello endpoint and should display an error message on the page. Hosting the API in Elastic Beanstalk Elastic beanstalk prerequisites For our NestJS API to run in Elastic Beanstalk, we need some additional setup. Inside the apps/api/src folder, let's create a Procfile with the contents: web: node main.js. Then open the apps/api/project.json and under the build configuration, extend the production build setup with the following (I only ) ` The above settings will make sure that when we build the API with a production configuration, it will generate a package.json and a package-lock.json near the output file main.js. To have a production-ready API, we set up a script in the package.json file of the repository. Running this will create a dist/apps/api and a dist/apps/frontend folder with the necessary files. ` After running the script, zip the production-ready api folder so we can upload it to Elastic Beanstalk later. ` Creating the Elastic Beanstalk Environment Let's open the Elastic Beanstalk service in the AWS console. And create an application. An application is a logical grouping of several environments. We usually put our development, staging and production environments under the same application name. The first time you are going to create an application you will need to create an environment as well. We are creating a Web server environment. Provide your application's name in the Application information section. You could also provide some unique tags for your convenience. In the Environment information section please provide information on your environment. Leave the Domain field blank for an autogenerated value. When setting up the platform, we are going to use the Managed Node.js platform with version 18 and with the latest platform version. Let's upload our application code, and name the version to indicate that it was built locally. This version label will be displayed on the running environment and when we set up automatic deployments we can validate if the build was successful. As a Preset, let's choose Single instance (free tier eligible) On the next screen configure your service access. For this tutorial, we only create a new service-role. You must select the aws-elasticbeanstalk-ec2-role for the EC2 instance profile. If can't select this role, you should create it in AWS IAM with the AWSElasticBeanstalkWebTier, AWSElasticBeanstalkMulticontainerDocker and the AWSElasticBeanstalkRoleWorkerTier managed permissions. The next step is to set up the VPC. For this tutorial, I chose the default VPC that is already present with my AWS account, but you can create your own VPC and customise it. In the Instance settings section, we want our API to have a public IP address, so it can be reached from the internet, and we can route to it from CloudFront. Select all the instance subnets and availability zones you want to have for your APIs. For now, we are not going to set up a database. We can set it up later in AWS RDS but in this tutorial, we would like to focus on setting up the distribution. Let's move forward Let's configure the instance traffic and scaling. This is where we are going to set up the load balancer. In this tutorial, we are keeping to the defaults, therefore, we add the EC2 instances to the default security group. In the Capacity section we set the Environment type to Load balanced. This will bring up a load balancer for this environment. Let's set it up so that if the traffic is large, AWS can spin up two other instances for us. Please select your preferred tier in the instance types section, We only set this to t3.micro For this tutorial, but you might need to use larger tiers. Configure the Scaling triggers to your needs, we are going to leave them as defaults. Set the load balancer's visibility to the public and use the same subnets that you have used before. At the Load Balancer Type section, choose Application load balancer and select Dedicated for exactly this environment. Let's set up the listeners, to support HTTPS. Add a new listener for the 443 port and connect your SSL certificate that you have set up in CloudFront as well. For the SSL policy choose something that is over TLS 1.2 and connect this port to the default process. Now let's update the default process and set up the health check endpoint. We set up our API to have the health check endpoint at the /api route. Let's modify the default process accordingly and set its port to 8080. For this tutorial, we decided not to enable log file access, but if you need it, please set it up with a separate S3 bucket. At the last step of configuring your Elastic Beanstalk environment, please set up Monitoring, CloudWatch logs and Managed platform updates to your needs. For the sake of this tutorial, we have turned most of these options off. Set up e-mail notifications to your dedicated alert e-mail and select how you would like to do your application deployments. At the end, let's configure the Environment properties. We have set the default process to occupy port 8080, therefore, we need to set up the PORT environment variable to 8080. Review your configuration and then create your environment. It might take a few minutes to set everything up. After the environment's health transitions to OK you can go to AWS EC2 / Load balancers in your web console. If you select the freshly created load balancer, you can copy the DNS name and test if it works by appending /api/hello at the end of it. Connect CloudFront to the API endpoint Let's go back to our CloudFront distribution and select the Origins tab, then create a new origin. Copy your load balancer's URL into the Origin domain field and select HTTPS only protocol if you have set up your SSL certificate previously. If you don't have an SSL certificate set up, you might use HTTP only, but please know that it is not secure and it is especially not recommended in production. We also renamed this origin to API. Leave everything else as default and create a new origin. Under the Behaviors tab, create a new behavior. Set up the path pattern as /api/* and select your newly created API origin. At the Viewer protocol policy select Redirect HTTP to HTTPS and allow all HTTP methods (GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE). For this tutorial, we have left everything else as default, but please select the Cache Policy and Origin request policy that suits you the best. Now if you visit your deployment, when you click on the HELLO button, it should no longer attach an error message to the DOM. --- Now we have a distribution that serves the front-end static files through CloudFront, leveraging caching and CDN, and we have our API behind a load balancer that can scale. But how do we deploy our front-end and back-end automatically when a release is merged to our main branch? For that we are going to leverage AWS CodeBuild and CodePipeline, but in the next blog post. Stay tuned....

Sep 11, 2023

12 mins

NestJSAWSTypeScriptNodeJS

Deploying apps and services to AWS using AWS Copilot CLI

Copilot has become a household name for developers, all thanks to GitHub’s popular AI tooling. Before GitHub released Copilot, AWS already had a developer tool in the wild, also named Copilot. I stumbled across AWS Copilot a year or two ago and found it to be a really great tool for easily deploying Serverless applications and services to AWS infrastructure. Since deploying to AWS has been such a huge pain point for so many developers, Copilot CLI is one of many tools that is designed to make this process a lot easier. AWS Copilot is a command-line interface (CLI) that simplifies the process of deploying and managing containerized applications on AWS. It abstracts away the complexity of managing infrastructure, allowing us to focus on writing code. This blog post will provide an overview of AWS Copilot CLI and explore its practical use cases. AWS Copilot CLI is designed to simplify the building, releasing, and operating of production-ready containerized applications on Amazon ECS and AWS Fargate. It offers an intuitive interface for developers to launch and manage environments, jobs, pipelines, and services in the cloud. Although ECS and Fargate are considered ‘Serverless’; you are actually deploying to more traditional, stateful, web servers on EC2 _(these are cool again)_. Installation You can install the Copilot CLI using Homebrew. ` Otherwise use one of the scripts from the installation page. Credentials You need to add AWS credentials with proper permissions to your ~/aws/.credentials file to use Copilot. See the credentials docs for more details on this. ` Overview All you need is a Dockerfile that knows how to build and run your application and Copilot will handle the rest. Copilot provides a simple declarative set of commands, including examples and guided experiences built-in. These commands make it easy to start from scratch and get a containerized application running in the cloud in just a few steps. The configuration files that Copilot generates (called manifests) allow us to easily configure and adjust the amount of compute resources available to our web service _(cpu, memory, instances, etc)_. Here’s an architecture diagram for a load-balanced web service running on AWS and deployed with Copilot. The outermost layer is the region your application is deployed to on AWS ie: us-east-1, us-west-1, etc. Copilot handles all the networking for your application which includes a VPC, and public subnets that your server instances can be reached from. The load balancer (ALB) sits at the top, listens for requests, and directs traffic to the ECS Cluster (instances of your application server). Typically, the work involved with setting all of these pieces up and getting them working together is cumbersome to put it lightly. Concepts Before we dive into some of the commands, a quick look at the concepts that Copilot is built on so we have a clear understanding of what we can achieve with it. Applications Applications in Copilot is the top-level parent of all the AWS infrastructure managed by your Copilot setup. In this article I will generally refer to an application as any kind of software that you are deploying (api server, etc). In Copilot your ‘Application’ is the entire collection of all the environments and services that you have configured. Here’s the diagram from the documentation. The Vote application can consist of a number of different services, jobs, pipelines, etc. Services In AWS Copilot, “Services” refer to the type of application (not to be confused with the definition of Application in the previous section) that you’re deploying and the underlying AWS service infrastructure that supports it. Using the Load Balanced Web Service service from the diagram above, the service consists of the ECS (Elastic Container Service) service, the application load balancer, and the network load balancer. These are some of the AWS infrastructure that Copilot orchestrates for you when deploying this type of “Service”. There are a few main types of services that you can deploy with Copilot - Internet-facing web services (Static Site, Load Balanced Web Service, etc) - Backend services (services that can only be accessed internally) - Worker services (pub/sub queues, etc) Environments When setting up your project and your first service you will supply Copilot with the name of the environment you want to deploy to. For example, you can create a production environment initially to deploy your application and services to and then later on add additional environments like a staging environment. Jobs You might also know these as ‘crons’ but these are just tasks or some code that runs on a schedule. Pipelines Pipelines are for automating tests and releases. This is AWS’s CI/CD service somewhat similar to something like GitHub actions. Copilot can initialize and create new pipelines for you. We can configure this as a manifest in our codebase that declares instructions on how we build and deploy our application(s). Configuration Copilot is configured through various configuration files called ‘Manifests’. The documentation contains examples on how to configure your application, services, environments, jobs, and pipelines. You can also refer to it to learn what options are available. Since Copilot is a CLI tool they make a lot of the configuration process pretty painless by providing walkthrough prompts and generating configuration files for you. The CLI Now that we have a pretty good idea of what AWS Copilot is and the types of things we can accomplish with it, let's walk through using the CLI. ` This is where you will most likely want to start to get your application ready for deployment to AWS. The init command will prompt you with questions like what kind of service you want to generate and once you’re done, it will generate all the initial configuration files for you. Example setup: ` You’ll pick your service type, tell Copilot where your Dockerfile lives, name your environment, and let it do its magic from there. Copilot will generate your Manifest/config files which in turn become CloudFormation stacks that set up all your infrastructure on AWS. Other Commands There are commands for each concept that we covered above. There are Create, Delete, and Deploy operation commands for Environments, Services, Jobs, and Pipelines. If you wanted to add a staging environment to your application you could just run copilot env init . There are also commands to manage your Application or things like tasks. To see details about your Application you can run copilot app show -n [name] Conclusion AWS Copilot CLI is a powerful tool that simplifies the deployment and management of containerized applications on AWS. It abstracts away the complexities of the underlying infrastructure, allowing developers to focus on writing code and delivering value to their customers. Whether you're deploying a simple web service or managing multiple environments, AWS Copilot CLI can make your cloud development process more efficient and enjoyable....

Jan 3, 2024

6 mins

AWSToolingDocker

Build IT Better - DevOps - Monitoring Roundup

Build IT Better DevOps - Monitoring Roundup On This Dot's Build IT Better show, I talk to people who make popular tools that help developers make great software. In my most recent series, we looked at application monitoring tools. Marcus Olssen from Grafana and Ben Vinegar from Sentry showed us how the tools they work on can help developers keep their applications running smoothly. Grafana Grafana is an organization that builds a number of open source observation and monitoring tools for collecting and visualizing application metrics. Their namesake product is a platform for aggregating and visualizing any kind of data from a near limitless number of sources via their rich plugin library. Grafana's commercial counterpart, Grafana Labs, maintains this plugin library as well as educational resources for the Grafana ecosystem and paid products and services for companies that are looking for help managing their own Grafana tooling. Flexibility Grafana is a platform for application monitoring and analytics that offers a really huge amount of flexibility for collecting and analyzing application data. Instead of providing a hyper-focused application monitoring solution, Grafana provides unparallelled flexibility for collecting almost any kind of data. Grafana offers built in integrations for all the most popular SQL and non-SQL databases, as well as Grafana's own popular application monitoring tools, Prometheus, Loki, and Tempo (and a handful of other popular sources). Community developed plugins can be used to add support for most other platforms. This flexibility allows Grafana to have applications outside the traditional application monitoring use cases. Some are even using Grafana to track their own home energy usage and health data. You can really analyze almost any kind of data in Grafana. *A Grafana dashboard with custom metrics* Datasource Compatibility While flexibility allows Grafana to reach across industries to find users and use cases, it still excels at traditional application monitoring. Developers can use Prometheus to pull data out of their own applications. Most popular host operating systems and appliation development frameworks offer community developed integrations with Prometheus that will provide useful system and application data like resource usage and response time, as well as the ability to publish your own custom application data. Loki is a tool for aggregating and querying system and application logs. Also, you can use Tempo for aggregating distributed application trace data from tools like Jaeger, OpenTelemetry, and Zipkin. If you use all 4 tools together, you can visually trace transactions all the way through your application, even as the user shifts between different components of your microservice architecture. Visualization and Analysis All of this flexible data collection technology would be useless without Grafana's equally flexible visualization platform. Once you've integrated all your data sources, you can use Grafana to explore and visualize data you've collected. You can use dashboards to create an array of vizualizations of your data. As a DevOps engineer, one of my favorite things about Grafana is their Dashboard library. The dashboard library contains community developed dashboards for a number of popular (and not so popular) application frameworks and backend tools and systems. Instead of needing to make your own dashboards from scratch for monitoring Rails apps and PostgreSQL databases, you can simply add and modify community Dashboards, saving you time and providing insights you may not have considered on your own. Finally, we have to mention the Explore tool. It can be easy to overlook with everything that's possible with Dashboards, but it allows users to easily view, query, and analyze data streams on the fly without needing to create permanent dashboards. *Grafana Nginx Dashboard - available from the dashboard library* This big tent collection of features makes Grafana a great platform for observing any amount of any kind data. The flexibility does come with the overhead of needing to know a lot about a number of different tools like Prometheus and Loki, which have a non-trivial amount of overhead on their own. As with any community-developed content, plugins and dashboards from the library don't always work as expected out of the box and will often need to be modified to line up with your devops procedures and environments. Sentry Sentry, like Grafana, is a tool for monitoring application health and performance. However, unlike Grafana, Sentry is laser-focused on providing curated experiences with deep first party integrations for popular application development tools and provides some additional tools for tracking user errors and code changes, which it uses as the framing narrative for all of the data the Sentry platform surfaces for developers. Integratons are available for most popular frontend JavaScript frameworks (React, Angular, Vue, etc) and backend applications in Python, Ruby, Go, and more. Sentry gives developers a huge amount of visibility without the overhead of more complex devops driven platforms like Grafana. Developer Focused Sentry's primary goal is to help you understand what's wrong with all of the parts of your application. They do this by giving you a view into what errors your users are experiencing in real time. Sentry collects data on all of the exceptions thrown by applications which have a Sentry integration. As you investigate individual issues, Sentry provides you with a curated collection of datapoints to cross reference with the specific error. Sentry provides some very traditional data, such as user like browser agent, OS, their geographical location, and the url they were visiting, but it also connects that error back to the code. Not only can you see the stack trace and easily see the lines of code where the error manifested, but Sentry also uses its deep integration to provide what they call "Breadcrumbs." These are pieces of data about what actual activity led up to the error. Depending on the what type of application you're troubleshooting, this might be things like log output, events fired from UI elements, or your own custom breadcrumb events. These can give you a better idea of the actions the user took leading up to the error. *Sentry's Issue (aka Error) View* *A sample of Sentry's Breadcrumbs* Integrations In addition to helping you identify the root cause of your errors, Sentry also aggregates errors to make it easier for you to understand which errors have the highest impact on your application. You can easily identify errors that are happening frequently and on critical paths. If you've enabled integration with a source control platform like GitHub, Sentry will even make suggestions as to which code commits introduced the problem. All these features together will help you tackle application health like a devops expert, without needing to be a devops expert. Application Performance Debugging and error surfacing aren't the only place where Sentry shines. I'm really excited to talk about Sentry's performance and application tracing platform. Using their deep framework and platform integrations, you're able to collect a lot of performance data from your applicaitons and to coallate them with user behaviors. Similar to the debugging experience, Sentry starts you from a broad view of your performance picture, and shows you the slowest pages and endpoints of your application, and provides you with another curated experince for investigating and resolving performance problems. The most interesting aspect of the performance investigation tools are transactions, or traces. When you choose a slow page to begin investigating, alongside the individual performance metrics for that page, are transactions. These transactions allow you to see the performance of your pages broken into waterfall graphs, like you might already be used to from browser dev tools. However, Sentry adds some really cool tricks since they're deeply integrated into all the parts of your application. If you analyze a transaction that starts from your javascript app and see that there's a fetch request that's taking a long time, assuming the API is part of your stack that's integrated with Sentry, you can click down into that fetch request within the Sentry UI and switch contexts to the API application and see a waterfall graph of what the API did to handle that request, allowing you to simply traverse your whole application to identify the exact source of performance problems. These transactions also benefit from the same Breadcrumb and code change data that's provided in the error analysis tools. Conclusions Sentry and Grafana are both strong tools to add to your DevOps toolbelt. While they both provide great features for observing application health and analyzing data, they really fill two pretty different niches. Sentry provides curated developer experiences and deep integrations that will help developers dive head first into error and performance monitoring for their applications without needing to be experts. However for experts and "data scientists" Grafana provides an incredibly powerful and flexible platform for not only analyzing application metrics and health, but really any data you can manage to get into a Dashboard. Some organizations may even benefit from using both tools for different use cases....

Apr 20, 2021

7 mins

DevOps

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Building a Stripe App: A Step-by-Step Guide to QR Code Generation Why Build a Stripe App? I recently participated in an audio space with the Stripe team, and something they said really stuck with me: the Stripe app store is a growing area that isn't overly saturated yet. There's a lot of potential for new apps, and companies can use this opportunity to grow. I work at a company called This Dot Labs, and we've created several Stripe apps and even own one. After looking at the data, I can confirm that the Stripe team was right! Creating a QR Code Generator App For this tutorial, we'll build a QR code app that can take a URL and generate a code for it. This is a good use case to help you understand the ins and outs of Stripe's developer tools. Why QR Codes? QR codes are useful tools that have become common in e-commerce, restaurants, and other industries. While Stripe already has a QR code tool, we'll make our own to familiarize ourselves with their syntax and problem-solving approaches. Project Structure Before we dive into the implementation, let's look at the structure of our Stripe App QR Code project: * .vscode: Contains settings for Visual Studio Code * source/views: Holds the main application views * .gitignore: Specifies files to ignore in version control * stripe-app.json: Defines the Stripe app configuration * ui-extensions.d.ts: TypeScript declaration file for UI extensions * .build: this is where the built Stripe app gets placed. Step-by-Step Implementation 1\. Install Stripe Locally First, you need to install Stripe on your local machine. The documentation provides great instructions for this: * For Mac users: Use Brew to install * For Windows users: Download the package and add it to your environment variables You can find the details here in the stripe docs to install the Stripe CLI https://docs.stripe.com/stripe-cli When using Windows, you must do stripe login from Powershell, NOT from Git bash or any other tool. After the server is up, then you can continue using git bash for everything else. After stripe login, you need to enter stripe apps start. Once you do that, the server is up and running and you can go back to using git bash or any other tool. 2\. Install Dependencies We'll be using an extra package for QR code generation. Install it using npm: ` 3\. Set Up the Main Component Let's look at the home.tsx file, where we'll use Stripe's UI components: ` These components are similar to other UI libraries like Bootstrap or Tailwind CSS. 4\. Create the UI Structure Our app will have: * An input field for the URL * Validation using a regex pattern * Error handling for invalid URLs * QR code generation and display Here is the Home.tsx file that is located in the src/views folder ` * ContextView` is at the top level of the app where we see the Title and the link to the Stripe Docs that we placed in our Context View. * Box is how you use Divs. * Banners can be used to show notification errors or any other item you wish to display. * Textfields are input fields. * Everything else is pretty self-explanatory. 5\. Handle Content Security Policy One problem I personally ran into was when I tried to redirect users, the Stripe policies would block it since I did not express that I knew what it was doing. I had to go into the stripe-app.json file and mention the specific security policies. For this particular exercise, I kept these as null. This is my stripe-app.json file. ` 6\. Configure App Views As you can see here, the stripe-app.json file shows the views for each file I have. The Home.tsx file and the Invoice.tsx are also included This is our way of saying that for each view we have, show the app functionality on that page. Our stripe-app.json file will show it but also, the manifest.js file in our .build folder will also show the same. Any view that doesn't have a file will not show the application's functionality. So, if I were to go to transactions, the app would not show the same logic as the home or invoices page. By following these steps, you'll have a fully functional QR code generator app for Stripe. This is just a simple example, but the potential for Stripe apps is massive, especially for businesses serving e-commerce customers. If you need help or get stuck, don't hesitate to reach out, danny.thompson@thisdot.co. The Stripe team is also very active in answering questions, so leverage them as a resource. Happy coding!...

Dec 18, 2024

4 mins

stripe

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Putting Your NodeJS App in a Docker Container

Putting Your App in a Docker Container

The Dockerfile Language

FROM

RUN

COPY

ENTRYPOINT

CMD

EXPOSE

And more...

Choosing a Base Image

Starting Your Dockerfile

Installing Additional Depenencies

Installing Package Dependencies

Installing Source Dependencies

Installing Your App

Running Your App

Conclusion

Adam Clarke

You might also like

How to host a full-stack app with AWS CloudFront and Elastic Beanstalk

Deploying apps and services to AWS using AWS Copilot CLI

Build IT Better - DevOps - Monitoring Roundup

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Let's innovate together!

You might also like

How to host a full-stack app with AWS CloudFront and Elastic Beanstalk

Deploying apps and services to AWS using AWS Copilot CLI

Build IT Better - DevOps - Monitoring Roundup

Building a Stripe App: A Step-by-Step Guide to QR Code Generation