Mapping Returned HTTP Data with RxJS

In frontend development, it's likely that you will make API requests to retrieve data from some backend resource. However, some backends are set up in such a way that they either send too much data or not enough data back.

When we combine what we know already about Reactive Programming and RxJS, we can handle these situations in a very elegant manner using some useful operators that RxJS provides us.

In this article, we will look at handling both scenarios and look at four operators in particular: map, mergeMap, concatMap and switchMap. We will also provide links to some StackBlitz examples so you can see these live.

For fun, we'll use the Star Wars API to fetch data and use Angular as our Frontend framework of choice as its HTTP library has Reactive Bindings out of the box.

Handling too much data (part 1)

Ok, let's set up the scenario. We want to find out some details about Luke Skywalker such as his name, birth year, height, weight, and eye color.

With Angular, we can do this by simply making an HTTP request:

this.http.get("https://swapi.dev/api/people/1")
    .subscribe(response => console.log(response));

If we do this, we do get back the info we need, but we also get back a lot of info we don't need such as when the API entry was created and edited, Luke's species, his Vehicles etc. Right now, we don't care about those details.

Let's see how we could use RxJS's map operator to only return the data we want.

this.http.get("https://swapi.dev/api/people/1")
    .pipe(map(response => ({
        name: response.name,
        birthYear: response.birth_year,
        height: Number(response.height),
        weight: Number(response.mass),
        eyeColor: response.eye_color
    })))
    .subscribe(luke => console.log(luke))

We can see from the example above that it's super easy to only pull the values you need from the response object!

Note

You can also see how we managed to map some of the fields in the response from snake_case to camelCase which is more of a convention in JavaScript as well as mapping some fields in the response to a new name (mass -> weight). We can also convert some types to other types, such as mass as string to weight as number. This can be super helpful to keep the domain language of your frontend codebase intact.

Live Example

You can check this code out in action over at the live example here on StackBlitz. Feel free to fork it and play with it.

Handling too much data (part 2)

Another common use case of having too much data is search results. Some times we only want to display the first result from a bunch of search results.

Let's set up a new scenario.

We'll create a typeahead search that only shows the most relevant search result. We want to request search results after every key press the user makes; however, we also don't want to make requests that are not needed. Therefore, we want to cancel any inflight requests created as the user types.

For this scenario, we'll be using a mix of the map and switchMap operators.

We'll also use Angular's Reactive Forms Module to provide us with some reactive bindings.

Our code for this is very simple:

this.searchResult$ = this.search.valueChanges.pipe(
    // We get the search term the user has typed
    // And use switchMap to cancel any inflight requests
    // then create a new request and switch to that
    // Observable stream
    switchMap(term =>
      this.http.get<any>(`https://swapi.dev/api/people/?search=${term}`)
    ),
    // Next we check that there is results, if so we pick the first one
    // If not we create an object to show there are no results
    map(response =>
      response.count > 0 ? response.results[0] : { name: "No results" }
    ),
    // We then map the full response data down to only the fields we 
    // care about.
    map(
      response =>
        ({
          name: response.name,
          birthYear: response.birth_year,
          height: Number(response.height),
          weight: Number(response.mass),
          eyeColor: response.eye_color
        } as PeopleData)
    )
);

It's that simple to create an eager typeahead search with Angular and RxJS.

You can see the live example of this here on StackBlitz.

Handling too little data

We've looked at how to handle too much data in the API response with RxJS, but how about when we have too little data?

We can build this scenario out perfectly with the Star Wars API. Let's say we want to see what films the characters we are searching for appear in. The API response for the characters does contain what films they are in, but it does not give us details, just a URL that we can send a new request to get that film's data. This is also an Array of films so we may want to get the details for all the films they appear in at this time.

Let's see how we can transform our search code to allow us to fetch more data related to the results and map it into the final object that we use to render the data.

this.searchResult$ = this.search.valueChanges.pipe(
    // Again we take the search term and map to a new api request
    switchMap(term =>
      this.http.get<any>(`https://swapi.dev/api/people/?search=${term}`)
    ),
    // Now we use mergeMap as we do not need cancellation
    mergeMap(response =>
        // We use from to iterate over each film for the character
        from(response.films).pipe(
            // We need to massage the url to be correct as SW API returns http rather 
            // than https in the character details
            map(
              (film: string) => `${film.substring(0, 4)}${film.substring(4)}`
            ),
            // Now we use concatMap as this will force RxJS to wait for each request
            // to complete before starting the next one, ensuring we have all the 
            // data needed for each film
            concatMap((film: string) => this.http.get<any>(film)),
            // The film API also returns more data than we care about so we map it down
            // to only the fields we care about
            map(film => ({
              title: film.title,
              releaseDate: film.release_date
            })),
            // We then need to collect each of these API responses and map them back into
            // a single array of films
            reduce((films, film) => [...films, film], []),
            // Finally we then map the character data and the film data into one percise
            // object that we care about
            map(
              films =>
                ({
                  name: response.name,
                  birthYear: response.birth_year,
                  height: Number(response.height),
                  weight: Number(response.mass),
                  eyeColor: response.eye_color,
                  films
                } as PeopleData)
            )
        )
    )
);

I highly recommend reading the code above and the comments to understand exactly how to achieve this scenario. This pattern is very powerful, especially for example if you need to iterate over an array of user IDs and fetch the user details associated with those IDs.

There is also a live example of this here on StackBlitz that you can use to try it out.

Conclusion

This is a small introduction to mapping data returned from HTTP requests with RxJS, but hopefully you can see use this as a reference point if you ever need to perform complex data mapping that involves additional API requests.