General

Remix Deployment with Architecture

Published Aug 30, 2022

Updated Feb 13, 2023

4 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Intro

Today’s article, will give a brief overview of the Architect framework and how to deploy a Remix app. I’ll cover a few different topics, such as what Architect is, why it’s good to use, and the issues I ran into while using it. It is a straightforward process, and I recommend using it with the Grunge Stack offered by Remix. So let’s jump on in and start talking about Architect.

Prerequisites

There are a few prerequisites and also some basic understandings that are expected going into this. The first is to have a GitHub account, then an AWS account, and finally some basic understanding of how to deploy. I also recommend checking out the Grunge Stack here if you run into any issues when we progress further.

What is Architect?

First off, Architect is a simple framework for Functional Web Apps (FWAs) on AWS. Now you might be wondering, "why Architect?" It offers the best developer experience, works locally, has infrastructure as code, is secured to the least privilege by default, and is open-source.

Architect prioritizes speed, smart configurable defaults, and flexible infrastructure. It also allows users to test things and debug locally. It defines a high-level manifest, and turns a complex cloud infrastructure into a build artifact. It complies the manifest into AWS CloudFormation and deploys it. Since it’s open-source, Architect prioritizes a regular release schedule, and backwards compatibility.

Remix deployment

Deploying Remix with Architect is rather straightforward, and I recommend using the Grunge Stack to do it. First, we’re going to head on over to the terminal and run npx create-remix --template remix-run/grunge-stack. That will get you a Remix template with almost everything you need.

Generating remix template

For the next couple of steps, you need to add your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your repo’s secrets. You’ll need to create the secrets on your AWS account, and then add them to your repo.

AWS secrets

GitHub secrets

The last steps before deploying include giving CloudFormation a SESSION_SECRET of its own for both staging and production environments. Then you need to add an ARC_APP_SECRET.

Adding env variables

npx arc env --add --env staging ARC_APP_SECRET $(openssl rand -hex 32)
npx arc env --add --env staging SESSION_SECRET $(openssl rand -hex 32)
npx arc env --add --env production ARC_APP_SECRET $(openssl rand -hex 32)
npx arc env --add --env production SESSION_SECRET $(openssl rand -hex 32)

With all of that out of the way, you need to run npx arc deploy, which will deploy your build to the staging environment. You could also do npx arc deploy —-dry-run beforehand to verify everything is fine.

Issues I Had in My Own Project

Now let's cover some issues I had, and my solution for my own project. So, I had a project far into development, and while looking at the Grunge Stack, I was scratching my head, trying to figure out what was necessary to bring over to my existing project.

I had several instances of trial and error as I plugged in the Architect related items. For instance: the arc file, which has the app name, HTTP routes, static assets for S3 buckets, and the AWS configuration. I also had to change things like the remix.config file, adding a server.ts file, and adding Architect related dependencies in the package.json.

During this process, I would get errors about missing files or missing certain function dirs(), and I dumped a good chunk of time into it. So while continuing to get those errors, I concluded that I would follow the Grunge Stack, and their instructions for a new project. Then I would merge my old project with the new project; and that resolved my issues. While not the best way, it did get me going, and did not waste any more time.

That resolved my immediate issues of trying to get my Remix app deployed to AWS, but then I had to figure out what got pushed. That was easy, since it created a lambda, API gateway, and bucket based on the items in my arc file. Then I realized, while testing it out live, that my environmental variables hadn’t carried over, so I had to tweak those on AWS. My project also used a GitHub OAuth app. So that needed to be tweaked with the new URL. Then it was up and running.

Conclusion

Today’s article covered a brief overview of what Architect is, why it’s good to use, how to deploy a Remix app and the issues you might have. I hope it was useful and will give others a good starting point.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

William Hutt
I'm a software engineer with an interest in web development.
@willHutt

Why is It so HARD to Break Into Tech with Jason Torres

Tracy Lee and Rob Ocel chat with Jason Torres about the challenges of breaking into tech, especially for self-taught and underrepresented developers. Jason shares his journey from a 15-year career in the film industry to pursuing software development, discussing the emotional and financial hurdles involved. They also discuss the importance of networking, finding a niche rather than mastering everything, and the impact of the tech downturn on junior developers. Chapters 1. Introduction and Tech Career Journeys (00:00 - 02:30) 2. The Struggles of Breaking Into Tech (02:31 - 06:00) 3. Jason’s Career Pivot from Film to Tech (06:01 - 10:30) 4. The Importance of Networking and Community (10:31 - 15:00) 5. Specializing vs. Being a Generalist in Tech (15:01 - 20:00) 6. Finding Your Path in Tech (20:01 - 25:30) 7. Dealing with Imposter Syndrome and Belonging (25:31 - 30:00) 8. Final Thoughts and Tips for Breaking Into Tech (30:01 - 33:00) 9. Closing Remarks and Tech Talk Humor (33:01 - End) Follow Jason Torres on Social Media Twitter: https://x.com/TasonJorres Linkeidn: https://www.linkedin.com/in/thejasontorres/ Sponsored by Wix Studio: https://www.wix.com/studio...

Oct 8, 2024

1 min

Software EngineerOpen Source

The Dangers of ORMs and How to Avoid Them

Background We recently engaged with a client reporting performance issues with their product. It was a legacy ASP .NET application using Entity Framework to connect to a Microsoft SQL Server database. As the title of this post and this information suggest, the misuse of this Object-Relational Mapping (ORM) tool led to these performance issues. Since our blog primarily focuses on JavaScript/TypeScript content, I will leverage TypeORM to share examples that communicate the issues we explored and their fixes, as they are common in most ORM tools. Our Example Data If you're unfamiliar with TypeORM I recommend reading their docs or our other posts on the subject. To give us a variety of data situations, we will leverage a classic e-commerce example using products, product variants, customers, orders, and order line items. Our data will be related as follows: The relationship between customers and products is one we care about, but it is optional as it could be derived from customer->order->order line item->product. The TypeORM entity code would look as follows: ` With this example set, let's explore some common misuse patterns and how to fix them. N + 1 Queries One of ORMs' superpowers is quick access to associated records. For example, we want to get a product and its variants. We have 2 options for writing this operation. The first is to join the table at query time, such as: ` Which resolves to a SQL statement like (mileage varies with the ORM you use): ` The other option is to query the product variants separately, such as: ` This example executes 2 queries. The join operation performs 1 round trip to the database at 200ms, whereas the two operations option performs 2 round trips at 100ms. Depending on the location of your database to your server, the round trip time will impact your decision here on which to use. In this example, the decision to join or not is relatively unimportant and you can implement caching to make this even faster. But let's imagine a different situation/query that we want to run. Let's say we want to fetch a set of orders for a customer and all their order items and the products associated with them. With our ORM, we may want to write the following: ` When written out like this and with our new knowledge of query times, we can see that we'll have the following performance: 100ms * 1 query for orders + 100ms * order items count + 100ms * order items' products count. In the best case, this only takes 100ms, but in the worst case, we're talking about seconds to process all the data. That's an O(1 + N + M) operation! We can eagerly fetch with joins or collection queries based on our entity keys, and our query performance becomes closer to 100ms for orders + 100ms for order line items join + 100ms for product join. In the worst case, we're looking at 300ms or O(1)! Normally, N+1 queries like this aren't so obvious as they're split into multiple helper functions. In other languages' ORMs, some accessors look like property lookups, e.g. order.orderItems. This can be achieved with TypeORM using their lazy loading feature (more below), but we don't recommend this behavior by default. Also, you need to be wary of whether your ORM can be utilized through a template/view that may be looping over entities and fetching related records. In general, if you see a record being looped over and are experiencing slowness, you should verify if you've prefetched the data to avoid N+1 operations, as they can be a major bottleneck for performance. Our above example can be optimized by doing the following: ` Here, we prefetch all the needed order items and products and then index them in a hash table/dictionary to look them up during our loops. This keeps our code with the same readability but improves the performance, as in-memory lookups are constant time and nearly instantaneous. For performance comparison, we'd need to compare it to doing the full operation in the database, but this removes the egregious N+1 operations. Eager v Lazy Loading Another ORM feature to be aware of is eager loading and lazy loading. This implementation varies greatly in different languages and ORMs, so please reference your tool's documentation to confirm its behavior. TypeORM's eager v lazy loading works as follows: - If eager loading is enabled for a field when you fetch a record, the relationship will automatically preload in memory. - If lazy loading is enabled, the relationship is not available by default, and you need to request it via a key accessor that executes a promise. - If neither is enabled, it defaults to the behavior ascribed above when we explained handling N+1 queries. Sometimes, you don't want these relationships to be preloaded as you don't use the data. This behavior should not be used unless you have a set of relations that are always loaded together. In our example, products likely will always need product variants loaded, so this is a safe eager load, but eager loading order items on orders wouldn't always be used and can be expensive. You also need to be aware of the nuances of your ORM. With our original problem, we had an operation that looked like product.productVariants.insert({ … }). If you read more on Entity Framework's handling of eager v lazy loading, you'll learn that in this example, the product variants for the product are loaded into memory first and then the insert into the database happens. Loading the product variants into memory is unnecessary. A product with 100s (if not 1000s) of variants can get especially expensive. This was the biggest offender in our client's code base, so flipping the query to include the ID in the insert operation and bypassing the relationship saved us _seconds_ in performance. Database Field Performance Issues Another issue in the project was loading records with certain data types in fields. Specifically, the text type. The text type can be used to store arbitrarily long strings like blog post content or JSON blobs in the absence of a JSON type. Most databases use a technique that stores text fields off rows, which requires a special file system lookup operation to fetch that data. This can make a typical database lookup that would take 100ms under normal conditions to take 200ms. If you combine this problem with some of the N+1 and eager loading problems we've mentioned, this can lead to seconds, if not minutes, of query slowdown. For these, you should consider not including the column by default as part of your ORM. TypeORM allows for this via their hidden columns feature. In this example, for our product description, we could change the definition to be: ` This would allow us to query products without descriptions quickly. If we needed to include the product description in an operation, we'd have to use the addSelect function to our query to include that data in our result like: ` This is an optimization you should be wary of making in existing systems but one worth considering to improve performance, especially for data reports. Alternatively, you could optimize a query using the select method to limit the fields returned to those you need. Database v. In-Memory Fetching Going back to one of our earlier examples, we wrote the following: ` This involves loading our data in memory and then using system memory to perform a group-by operation. Our database could have also returned this result grouped. We opted to perform this operation like this because it made fetching the order item IDs easier. This takes some performance challenges away from our database and puts the performance effort on our servers. Depending on your database and other system constraints, this is a trade-off, so do some benchmarking to confirm your best options here. This example is not too bad, but let's say we wanted to get all the orders for a customer with an item that cost more than $20. Your first inclination might be to use your ORM to fetch all the orders and their items and then filter that operation in memory using JavaScript's filter method. In this case, you're loading data you don't need into memory. This is a time to leverage the database more. We could write this query as follows in SQL: ` This just loads the orders that had the data that matched our condition. We could write this as: ` We constrained this to a single customer, but if it were for a set of customers, it could be significantly faster than loading all the data into memory. If our server has memory limitations, this is a good concern to be aware of when optimizing for performance. We noticed a few instances on our client's implementation where the filter operations were applied in functions that appeared to run the operation in the database but were running the operation in memory, so this was preloading more data into memory than needed on a memory-constrained server. Refer to your ORM manual to avoid this type of performance hit. Lack of Indexes The final issue we encountered was a lack of indexes on key lookups. Some ORMs do not support defining indexes in code and are manually applied to databases. These tend not to be documented, so an out-of-sync issue can happen in different environments. To avoid this challenge, we prefer ORMs that support indexes in code like TypeORM. In our last example, we filtered on the cost of an order item, but the cost field does not contain an index. This leads to a full table scan of our data collection filtered by the customer. The query cost can be very expensive if a customer has thousands of orders. Adding the index can make our query super fast, but it comes at a cost. Each new index makes writing to the database slower and can exponentially increase the size of our database needs. You should only add indexes to fields that you are querying against regularly. Again, be sure you can notate these indexes in code so they can be replicated across environments easily. In our client's system, the previous developer did not include indexes in the code, so we retroactively added the database indexes to the codebase. We recommend using your database's recommended tool for inspection to determine what indexes are in place and keep these systems in sync at all times. Conclusion ORMs can be an amazing tool to help you and your teams build applications quickly. However, they have gotchas for performance that you should be aware of and can identify while developing or during code reviews. These are some of the most common examples I could think of for best practices. When you hear the horror stories about ORMs, these are some of the challenges typically discussed. I'm sure there are more, though. What are some that you know? Let us know!...

Jul 3, 2024

8 mins

Architectural DesignDesign PatternsArchitecture

Setting Up a Shopify App and Getting Order Data

Today, we are going on an adventure! We’re starting a three-part guide on creating a Shopify app and updating a customer's order with tracking information. For this article, it's assumed that you already have a Shopify store. If you want to skip ahead and look at the code, it can be found here. To start us off, we’ll use the Shopify Create app and then follow it up with retrieving customer orders. Shopify Create app Getting started with the Shopify Create app will be a quick and easy process. We’ll start by navigating to a directory where we want to create our app and run the following ` We’ll be greeted by a few different prompts asking for the project name, and building your app selection. Success! Now let's navigate into our new directory and do a yarn dev, and we’ll get a few options. We’ll choose to create it as a new app, add an app name, config name, and select the store we want to use. With that out of the way, we’ll open the preview by pressing the p button. It should automatically open it up in a window and show us an app install screen where we will click the Install app button. This should redirect us to our Remix app template screen Install Template Perfect! We now have a basic Shopify Create app up and running. Next, we will move on to adding in the ability to retrieve our customer orders. Orders query Alright, it’s customer order time! We’re going to be leaving the template mostly as is. We are going to focus on adding the call and verifying the data comes back. We’ll navigate over to our app/routes/app._index.jsx file and start changing the loader function. Start by removing: ` And replacing it with: ` Next, swap Wrap change { shop } in: ` With ` Follow that up with changing ` To ` Then, we’ll remove the View product button that has the old shop variable in it. When you go back and look at your application, you should see the Error: Access denied for fulfillmentOrders field. This is due to scopes that we haven’t updated. To fix this, we’ll head over to our shopify.app.toml file and replace ` with ` Here is what you should now have: ` We’ll now do another yarn dev which will tell us that our scopes inside the TOML don’t match the scopes in our Partner Dashboard. To fix this, we simply need to run: ` And then we’ll be prompted to confirm our changes with the difference shown. It will give us a success response, and now we can do another yarn dev to look at our application. Doing so brings us back to our old friend the app install page. Only this time it’s telling us to update the app and redirecting us back to the application page. Huh, seems like we’re getting a new error this time. For some reason, the app is not approved to access the FulfillmentOrder object. No worries, follow the link in that message. It should be https://partners.shopify.com/[partnerId]/apps/[appId]/customer_data Here, we select the Protected customer data and just toggle Store management and save. After this, go down to the Protected customer fields (optional) and do the same thing for Name, Email, Phone, and Address. With that all said and done, we’ll exit the page. After that, go back to our application, and refresh it. Tada! It works! We will see a toast notification at the bottom of the screen that says Orders received and in our terminal console, we will see our orders returning. Conclusion That was an exciting start to our three-part adventure. We covered a lot, from setting up a Shopify app to getting our orders back and everything up and running! Next time, we’ll be digging into how to get our fulfillment ids, which will be needed to update a customer's order with tracking information....

Aug 23, 2023

5 mins

Shopify

How Nuxt Studio is Redefining Developer & User Experience with Baptiste Leproux & Ferdinand Coumau

In this episode of the Modern Web Podcast, Danny Thompson sits down with Ferdinand Como and Baptiste Leproux from Nuxt Labs to uncover the story behind Nuxt Studio, a tool that's transforming how developers and non-technical users interact with Nuxt applications. Ferdinand and Baptiste share how Nuxt Studio bridges the gap between developer customization and user-friendly content management. Built to empower agencies, freelancers, and their clients, Nuxt Studio combines powerful features like live previews, Vue component integration, and schema-driven forms to make managing content seamless. The conversation also explores the broader mission of Nuxt Labs—building sustainable open-source tools that enhance developer experience and meet real-world needs. With insights into the future of Nuxt Studio and its potential to scale for larger organizations, this episode is a must-listen for anyone passionate about innovation in web development. Chapters 1. Introduction and Setting the Stage (00:00:00) 2. The Vision Behind Nuxt Studio (00:03:10) 3. Nuxt Studio’s Core Features (00:08:45) 4. Challenges in Building Nuxt Studio (00:16:20) 5. Target Audience and Use Cases (00:22:35) 6. Sustainability in Open Source (00:29:00) 7. The Future of Nuxt Studio (00:35:10) 8. Nuxt Studio’s Role in the Nuxt Ecosystem (00:42:30) 9. Closing Thoughts and What’s Next (00:48:00) 10. Sponsor Shoutout and Wrap-Up (00:53:20) Follow Baptiste Leproux and Ferdinand Coumau Baptiste Twitter: https://x.com/_larbish Ferdinand Twitter: https://x.com/CoumauFerdinand Baptiste Linkedin: https://www.linkedin.com/in/baptiste-leproux-618842b0/ Ferdinand Linkedin: https://www.linkedin.com/in/ferdinand-coumau-nuxt/ Sponsored by This Dot: thisdot.co...

Nov 20, 2024

2 mins

VueNuxtJS

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Remix Deployment with Architecture

Intro

Prerequisites

What is Architect?

Remix deployment

Issues I Had in My Own Project

Conclusion

William Hutt

You might also like

Why is It so HARD to Break Into Tech with Jason Torres

The Dangers of ORMs and How to Avoid Them

Setting Up a Shopify App and Getting Order Data

How Nuxt Studio is Redefining Developer & User Experience with Baptiste Leproux & Ferdinand Coumau

Let's innovate together!

You might also like

Why is It so HARD to Break Into Tech with Jason Torres

The Dangers of ORMs and How to Avoid Them

Setting Up a Shopify App and Getting Order Data

How Nuxt Studio is Redefining Developer & User Experience with Baptiste Leproux & Ferdinand Coumau