When coding, I never seem to have enough time to complete everything that I want to, so I love discovering tips and tricks to speed some of my processes up.
In this article, we are going to learn how to set up automatic posting to The Practical Dev site (dev.to), saving you significant time and effort.
The following solution uses a static website built on Gatsby, and powered by contentful CMS, but this can actually be applied to any site that can generate an RSS feed.
The content of this article is going to be divided into 3 different sections:
- What is an RSS feed?
- Set up an RSS into a gatsby website
- Configure dev.to to use an RSS feed
What is an RSS feed
RSS has been around for quite some time. In fact, its first usable version was released in 1999 by Dan Libby (wikipedia).
When first created, RSS stood for RDF Site Summary, but then it started to be referred to as Rich Site Summary, or Really Simple Syndication, after the removal of the RDF element from version 0.91.
A simple definition of an RSS feed is:
RSS (Rich Site Summary) is a format for delivering regularly changing web content. Many news-related sites, weblogs, and other online publishers syndicate their content as an RSS Feed to whoever wants it. http://www.whatisrss.com/
RSS provides a way to consume a specific website's data that is structured according to RSS specification. You can have a glimpse of what it looks like for reddit
For the past 2 decades, RSS has mostly been used to generate News aggregator, but it has many other uses, as we are going to later show in our article.
Set up an RSS into a gatsby website
In the following section, we are going to set up the feed in our Gatsby site, but as previously mentioned, RSS feed generators are available for all major frameworks and languages.
Gatsby-plugin-feed
As you are probably aware, Gatsby has many fantastic plugins and great documentation, that makes it so powerful and easy to use.
To achieve our goal, we are going to use a plugin called Gatsby-plugin-feed. There are different custom plugins that can support you in creating an RSS feed, but I have decided to use the above mentioned, as it is listed within the official gatsby site, and is well documented (gatsby-plugin-feed documentation).
Installation
It is now time to install the plugin using a your preferred package manager.
npm install --save gatsby-plugin-feed
//or
yarn add gatsby-plugin-feed
Configuration
Configuration of a Gatsby plugin can be achieved within the "Gatsby-config.js" file in the root folder of our site.
Default config
If your Gatsby site follows the suggested configuration, you may be able to easily set the plugin using the built-in configuration, by just loading the plugin:
module.exports = {
siteMetadata: {
siteUrl: `https://www.example.com`,
},
plugins: [
...,
`gatsby-plugin-feed`
],
}
Custom config
Unfortuantely, not everyone is able to use the default configuration of this plugin. To help you understand how the plugin actually works, we are going to cover the steps necessary to achieve a full custom configuration for a website powered by Contenful CMS Contentful CMS.
Custom config can be a bit more complex to set up, than the default configuration detailed above, and it requires a number of options to be set for the plugin to work:
- query
- feeds
- title
- output
- query
- serialize
Query
In this options paramenter, we are going to define a GraphQL query. This is necessary to fetch all the metadata required to fully define our feeds.
In our case, we are just going to fetch the siteUrl:
{
resolve: `gatsby-plugin-feed`,
options: {
query: `
{
site {
siteMetadata {
siteUrl
}
}
}
`,
}
}
note: The plugin declaration within the gatsby-config.js file has been changed from a simple string to an object, because of the need to use a custom configuration.
Feeds - title and output
The next step involves configuring a title and an output name for our file. The file name (output) is important to us, as it will later be used to dynamically fetch the content of our site.
{
resolve: `gatsby-plugin-feed`,
options: {
query: `
{
site {
siteMetadata {
siteUrl
}
}
}
`,
feeds:[
{
output: "/rss.xml",
title: "My custom feed",
}
]
}
}
feeds - query
It is now time to fetch the data necessary for our RSS feed. This is accomplished in the query paramether within the feeds option, in the form of a Graphql query:
...,
feeds: [
{
query: `
{
allContentfulPost(filter: { project: { eq: "blogPost" } }) {
edges {
node {
createdAt
title
slug
project
markdown {
markdown
}
}
}
}
}
`,
output: "/rss.xml",
title: "My custom feed"
}
]
feeds - serialize
So far, we have been able to set up some basic config (such as file name and title), and create a query to collate all the data necessary for our configuration.
In this final step, we are going to define the structure of our RSS items entry, using the different collection of data available.
Depending of the usage of your RSS feed, you may be required to provide a specific structure for your RSS feed (hence the possibility to create more than one feed within the Feeds array).
In our case, for the feed to be fully consumed by dev.to, we will require our RSS feed XML file to have the following information for each of the items:
title: The title of the post link: The absolute URL of our blog post pubDate: The date in which the article was first published on your website content: The markdown or HTMl content of your post
The serialize method, will expose the queried data as its parameters, and it expects an array to be returned.
It took me some time to understand the relationship between the parameters exposed by the serialize method and the XML generated, but after some playing and research, I have found out that the following structure is the correct one to produce the above mentioned architecture.
...,
feeds: [
{
serialize: ({ query: { site, allContentfulPost } }) => {
return allContentfulPost.edges.map(edge => {
return {
title: edge.node.title,
date: edge.node.createdAt,
url: `${site.siteMetadata.siteUrl}/${edge.node.slug}`,
custom_elements: [
{ 'content:encoded': edge.node.markdown.markdown },
],
}
})
},
query: `
...
`,
output: "/rss.xml",
title: "My custom feed"
]
note: Dev.to provides a small amount of documentation on publishing, using RSS feeds (publishing_from_rss_guide, but the information on the site seems to be a little bit outdated. In fact, it doesn't mention the use of a pubDate even though it actually uses this property.
Usage
The configuration of our Gatsby site powered by Contentful is now completed, and it is now time to see the result of our effort.
Due to the nature of feeds, the actual feed XML file is not avaialble in a development environment. Instead, it gets generated during a build command execution. For this, we need to run "Server Side Rendering" build option:
npm run ssr
Behind the scene, this command is going to build the site, and then serve it.
Once build is completed, you should be able to analyse the output of your feed at the location defined by the output option of our plugin's config.
You can access the generated feed at http://localhost:9000/rss.xml
As you can see from the image above, our feed includes title, item, and other elements necessary to be consumed by dev.to.
Configure dev.to to use an RSS feed
Now that our RSS feed is ready to be consumed, we are able to complete our post automation directly within Dev.to.
Setup the autofeed
The rest of the configuration will require you to modify your personal settings, so our first step is to log in to dev.to.
While in the home page, you can access the settings page by hovering on your avatar on the top right of the main navbar, and selecting Settings from the dropdown menu.
The settings to publish from RSS are available within the main sidebar menu, as shown in the screenshot below:
Setting up the RSS (after in posses of the RSS feed endpoint) is just two clicks away.
Automatically feed our articles just requires three simple options:
- RSS feed URL: This is going to be the URL defined before for our XML file. It is important to note that the URL that we are going to paste here cannot be localhost, and needs to be your LIVE url
- Canonical_URL: If you have not heard of this before, it is a good time for your learn about it. Duplicating content on the web usually result in a bad SEO ranking. To support real authors, posting their content in multiple places, we can make use of a canonical url (Canonical URLs: What Are They and Why Are They Important?. This makes sure to inform search engines, which URL should really be indexed for a specific page. So in our case, setting a canonical_URL from dev.to to our site, will inform google that all posts found on dev.to, should actually refer to the one published on our site.
- Replace links: This option will provide you the ability to "replace" all the links of your posts. As mentioned in the description, this is suggested, when you are planning to migrate an entire blog over.
With all settings completed, it is now time to click the Submit button. Doing so will initiate an immediate fetch of your content, and also schedule subsequent automatic watch over your file.
From this point onward, all the posts published on your website, are going to be automatically available within your dev.to account as Draft.
Publish your posts
As mentioned above, the imported posts are not automatically posted, but just created in a DRAFT state. This is done, to prevent posts that do not render successfully, or that need final touches (eg. define more FrontMatter) to be published.
There are a couple of things I have personally learned in the past from enabling RSS feed:
- ALL RSS feed items are imported This feature imports ALL posts. In our case, this was not what we wanted, as we were already used to do a manual fetch of all our posts. This resulted in a LOT of draft posts within our dashboard. The best solution for this problem is to "refine" the GraphQL query and/or the serialize method used, while configuring your RSS feed, to export posts from a specific date, or with specific values.
- Imported posts will reappear if deleted RSS feed will create a draft post, if it does not find one that match the same information. So if you encountered the problem above, removing them from the dashboard would not solve your problem, as they will magically reappear.
- Rendering differences Your imported posts may not look precisely the way they appear on your site. This may be due to custom styles you may have, unsupported embed and/or media images usage. So it is greatly suggested to always check the preview of your post, even if it looks good on your original post.
Conclusion
Implementing RSS on our Gatsby site has been a great success. Not only, has it allowed us to focus our content development in a single platform, but it has also supported our site with great SEO gains.
As already mentioned, RSS has been around for a long time, and no matter what techstack your website is built in, it will surely have the ability (natively or with the use of a plugin), to produce an RSS feed.
I hope the above guide will help you to find out of this fantastic feature, and please do not hesitate to contact me if you require further details, or to inform me about your succesful configuration.
