development

Take control of your backend with Firebase Cloud Functions

Sprint Board Guardian and Project Happiness Keeper. Dog walker, husband and enthusiastic sci-fi reader.

As a software engineer building Android apps, I inevitably run into problems with server API’s not designed with mobile clients in mind. In this post I’ll explain how we can use Firebase Cloud Functions to clean up and transform those API’s responses before they reach our app.


Backend API responses that are hard to work on mobile apps usually happen in two types of companies:

This is why I love Firebase: it's the bicycle of mobile apps developers. It gives you freedom and autonomy with almost no investment. And you can get it up and running even if you know nothing about servers, lambdas, node.js or hosting.

Take control of your backend API

What is the worst thing you've ever found on a server API? I can think of a bunch:

Wouldn't be great if you could fix all those problems before they even reach your app? It's not only convenient but more efficient; mainly for two reasons:

The example: a news app

Let's define a practical case for this example. We want to build a news app that reads an edition feed from a company's backend. Now imagine there are a series of problems with that feed:

  1. The structure is heavily nested, with lots of unnecessary data
  2. The feed is messy and contains different elements identified with the same key, instead of using arrays
  3. The response is not JSON

Parsing this is not a pleasant job on Android. If you use Retrofit you'll need a separate converter, lot of Java classes to act as Server API models for the unnecessary received data and helper classes to transform that Server models into your tidied up Domain models.

HTTP Triggers to the rescue

Firebase HTTP triggers let you trigger a function through an HTTP request. We'll use them to proxy our backend. Instead of calling the problematic API directly, we'll call our HTTP trigger endpoint, that will execute a function to fetch today's news edition from the backend, clean it up and return it as beautiful small JSON. Have in mind Cloud Functions have a limited time of 540 seconds to run before they are forcibly terminated, so we need to keep them small.

1. Define a function to be triggered through HTTP

exports.fetch = functions.https.onRequest((req, res) => {  
    (...) // TODO Fetch from API
    (...) // TODO Clean up
    (...) // TODO Return result
})

That's it. When deployed to your Firebase project this will generate a URL you can call using an HTTP request.

Our mobile app network requests will go through this URL, the code in the function's body will get executed, fetching, cleaning and returning the data in the format we need.

2. Fetch raw backend data

We need a little bit more code for this. We also need to make sure your Firebase project has billing enabled.

It is necessary to do external HTTP requests and since our backend lives outside Google's servers we need to switch to the Blaze plan, which is surprisingly cheap. The first 2,000.000 function executions a month are free and, to put it on perspective, I only did 68 while developing this project. After that you pay $0.40/million requests. At that price point, it’s a cost I am happy to underwrite to save us the pain of dealing with problematic and inconsistent APIs..

The other thing we need is a REST client to call our backend. I like node rest client because it’s super simple to use and comes with beautiful features out of the box, like converting XML to JSON automatically.

2.1 Adding a REST client

const Client = require('node-rest-client').Client  
const client = new Client()

exports.fetch = functions.https.onRequest((req, res) => {  
    client.get(BACKEND_URL, function (data, response) {

        (...) // TODO Clean up

        return res.status(200)
                .type('application/json')
                .send(data)
        })
    })  
})

This snippet is already doing a lot: we haven't written much, but this code fetches our backend XML edition feed and returns it as JSON with minimum effort from us.

You could leave this example as it is and you would already be saving yourself from converting XML to JSON in the app.

2.2 Cleaning up raw data

const Client = require('node-rest-client').Client  
const client = new Client()

exports.fetch = functions.https.onRequest((req, res) => {  
    client.get(URL_THE_GUARDIAN, function (data, response) {
        const items = cleanUp(data)
        res.status(201)
            .type('application/json')
            .send(items)
    })
})

At this point all the heavy work of cleaning up is extracted into a cleanUp function, that takes the raw data we get from the backend and extracts only the bits we are interested in.
This is the function you'll need to change for your particular case. In this sample we are going to use the format returned by The Guardian public RSS feed as an example of a convoluted raw backend response.

So let's imagine your backend returns something like this:

<rss xmlns:media="http://search.yahoo.com/mrss/" xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">  
  <channel>
    <title>London | The Guardian</title>
    <link>https://www.theguardian.com/uk/london</link>
    <description>Latest news and features from theguardian.com, the world's leading liberal voice</description>
    <language>en-gb</language>
    <copyright>Guardian News and Media Limited or its affiliated companies. All rights reserved. 2017</copyright>
    <pubDate>Mon, 31 Jul 2017 15:32:49 GMT</pubDate>
    <dc:date>2017-07-31T15:32:49Z</dc:date>
    <dc:language>en-gb</dc:language>
    <dc:rights>Guardian News and Media Limited or its affiliated companies. All rights reserved. 2017</dc:rights>
    <image>
      <title>The Guardian</title>
      <url>https://assets.guim.co.uk/images/guardian-logo-rss.c45beb1bafa34b347ac333af2e6fe23f.png</url>
      <link>https://www.theguardian.com</link>
    </image>
    <item>
      <title>'Leaving London means I can afford kids': readers on why the capital lost its sparkle</title>
      <link>https://www.theguardian.com/uk-news/2017/jul/31/leaving-london-means-i-can-afford-kids-readers-on-why-the-capital-lost-its-sparkle</link>
      <description>&lt;p&gt;Almost 100,000 Londoners moved out last year. Here they, and others who are avoiding the city altogether, explain why it is no longer the place to be&lt;br&gt;&lt;/p&gt;&lt;p&gt;The rate of Londoners leaving the capital is more than &lt;a href="https://www.theguardian.com/uk-news/2017/jul/24/bloated-london-property-prices-fuelling-exodus-from-capital"&gt;80% higher than five years ago&lt;/a&gt;, according to Savills, with people in their thirties the age group most likely to leave. &lt;/p&gt;&lt;p&gt;We asked readers why they’re leaving London, or avoiding moving to the capital altogether. Here’s what you said:&lt;/p&gt;&lt;p&gt;All my salary was being spent on living costs in London&lt;/p&gt; &lt;a href="https://www.theguardian.com/uk-news/2017/jul/31/leaving-london-means-i-can-afford-kids-readers-on-why-the-capital-lost-its-sparkle"&gt;Continue reading...&lt;/a&gt;</description>
      <category domain="https://www.theguardian.com/uk/london">London</category>
      <category domain="https://www.theguardian.com/money/family-finances">Family finances</category>
      <category domain="https://www.theguardian.com/society/housing">Housing</category>
      <category domain="https://www.theguardian.com/society/communities">Communities</category>
      <category domain="https://www.theguardian.com/money/work-and-careers">Work &amp; careers</category>
      <category domain="https://www.theguardian.com/cities/commuting">Commuting</category>
      <category domain="https://www.theguardian.com/lifeandstyle/parents-and-parenting">Parents and parenting</category>
      <category domain="https://www.theguardian.com/cities/cities">Cities</category>
      <category domain="https://www.theguardian.com/money/pay">Pay</category>
      <category domain="https://www.theguardian.com/lifeandstyle/family">Family</category>
      <category domain="https://www.theguardian.com/money/money">Money</category>
      <category domain="https://www.theguardian.com/lifeandstyle/lifeandstyle">Life and style</category>
      <category domain="https://www.theguardian.com/society/society">Society</category>
      <category domain="https://www.theguardian.com/uk/uk">UK news</category>
      <pubDate>Mon, 31 Jul 2017 12:43:16 GMT</pubDate>
      <guid isPermaLink="false">http://www.theguardian.com/uk-news/2017/jul/31/leaving-london-means-i-can-afford-kids-readers-on-why-the-capital-lost-its-sparkle</guid>
      <media:content width="140" url="https://i.guim.co.uk/img/media/33387252cc566ca74f8c0ccd4eef1f85ca81233c/0_141_3500_2102/master/3500.jpg?w=140&amp;q=55&amp;auto=format&amp;usm=12&amp;fit=max&amp;s=b3f65cf4bdb2597a770f32e3c49cf223">
        <media:credit scheme="urn:ebu">Photograph: Dominic Lipinski/PA</media:credit>
      </media:content>
      <media:content width="460" url="https://i.guim.co.uk/img/media/33387252cc566ca74f8c0ccd4eef1f85ca81233c/0_141_3500_2102/master/3500.jpg?w=460&amp;q=55&amp;auto=format&amp;usm=12&amp;fit=max&amp;s=0b078928a97ab43f0add6a9bf46c0b50">
        <media:credit scheme="urn:ebu">Photograph: Dominic Lipinski/PA</media:credit>
      </media:content>
      <dc:creator>Rachel Banning-Lover and Guardian readers</dc:creator>
      <dc:date>2017-07-31T12:43:16Z</dc:date>
    </item>
    <item> ... </item>
    <item> ... </item>

Let’s say we are only interested in building a list of items with a few of those properties, like title, description, pubDate, creator and what seems to be a list of media.
Our target will be to produce a JSON response like this:

[{
    "title": "'Leaving London means I can afford kids': readers on why the capital lost its sparkle",
    "description": "Almost 100,000 Londoners moved out last year. Here they, and others who are avoiding the city altogether, explain why it is no longer the place to be&lt;br&gt;&lt;/p&gt;&lt;p&gt;The rate of Londoners leaving the capital is more than &lt;a href="https://www.theguardian.com/uk-news/2017/jul/24/bloated-london-property-prices-fuelling-exodus-from-capital"&gt;80% higher than five years ago&lt;/a&gt;, according to Savills, with people in their thirties the age group most likely to leave. &lt;/p&gt;&lt;p&gt;We asked readers why they’re leaving London, or avoiding moving to the capital altogether. Here’s what you said:&lt;/p&gt;&lt;p&gt;All my salary was being spent on living costs in London&lt;/p&gt; &lt;a href="https://www.theguardian.com/uk-news/2017/jul/31/leaving-london-means-i-can-afford-kids-readers-on-why-the-capital-lost-its-sparkle"&gt;Continue reading...&lt;/a&gt;",
    "date": "Mon, 31 Jul 2017 15:32:49 GMT",
    "creator": "Rachel Banning-Lover and Guardian readers",
    "media": [{
        "url": "https://i.guim.co.uk/img/media/33387252cc566ca74f8c0ccd4eef1f85ca81233c/0_141_3500_2102/master/3500.jpg?w=140&amp;q=55&amp;auto=format&amp;usm=12&amp;fit=max&amp;s=b3f65cf4bdb2597a770f32e3c49cf223",
        "credit": "Photograph: Dominic Lipinski/PA"
    }, {
        "url": "https://i.guim.co.uk/img/media/33387252cc566ca74f8c0ccd4eef1f85ca81233c/0_141_3500_2102/master/3500.jpg?w=460&amp;q=55&amp;auto=format&amp;usm=12&amp;fit=max&amp;s=0b078928a97ab43f0add6a9bf46c0b50",
        "credit": "Photograph: Dominic Lipinski/PA"
    }]
}, 
{<item2>}, 
{<item3>},
...
]

This is something really easy to achieve in JS:

function cleanUp(data) {  
    // Empty array to add cleaned up elements to
    const items = []
    // We are only interested in children of the 'channel' element
    const channel = data.rss.channel

    channel.item.forEach(element => {
        item = {
            title: element.title,
            description: element.description,
            date: element.pubDate,
            creator: element['dc:creator'],
            media: []
        }
        // Iterates through all the elements named '<media:content>' extracting the info we care about
        element['media:content'].forEach(mediaContent => {
            item.media.push({
                url: mediaContent.$.url,                // Parses media:content url attribute
                credit: mediaContent['media:credit']._  // Parses media:credit tag content
            })
        })
        items.push(item)
    })
    return items
}

Putting it all together: all the code you need for the clean up Firebase Cloud Function

This is all you need to use a Firebase HTTP trigger as clean up proxy for a mobile unfriendly API.
Here's the full code for this example:

const functions = require('firebase-functions')  
const URL_THE_GUARDIAN = "https://www.theguardian.com/uk/london/rss"

const Client = require('node-rest-client').Client  
const client = new Client()

exports.fetch = functions.https.onRequest((req, res) => {  
    client.get(URL_THE_GUARDIAN, function (data, response) {
        const items = cleanUp(data)
        res.status(201)
            .type('application/json')
            .send(items)
    })
})

function cleanUp(data) {  
    // Empty array to add cleaned up elements to
    const items = []
    // We are only interested in children of the 'channel' element
    const channel = data.rss.channel

    channel.item.forEach(element => {
        item = {
            title: element.title,
            description: element.description,
            date: element.pubDate,
            creator: element['dc:creator'],
            media: []
        }
        // Iterates through all the elements named '<media:content>' extracting the info we care about
        element['media:content'].forEach(mediaContent => {
            item.media.push({
                url: mediaContent.$.url,                // Parses media:content url attribute
                credit: mediaContent['media:credit']._  // Parses media:credit tag content
            })
        })
        items.push(item)
    })
    return items
}

Stay tuned!

This is the first post in a new series. In the next article we’ll learn about using Firebase Database as an intermediate cache for all this data, so you can save the result and only request fresh data from the backend after a set period of time.


I’d love to hear your thoughts on this, please do reach me at @lgvalle


Special thanks and ❤️ to @takecare for making my JS makes sense, to @seebrock3r for making my English makes sense and to nnnneeeiil, @blundell, @niamh__power and @electryc for their reviews & suggestions.

Enjoyed this article? There's more...

We send out a small, valuable newsletter with the best stories, app design & development resources every month.

No spam, no giving your data away, unsubscribe anytime.

About Novoda

We plan, design, and develop the world’s most desirable software products. Our team’s expertise helps brands like Sony, Motorola, Tesco, Channel4, BBC, and News Corp build fully customized Android devices or simply make their mobile experiences the best on the market. Since 2008, our full in-house teams work from London, Liverpool, Berlin, Barcelona, and NYC.

Let’s get in contact

Stay in the loop!

Hear about our events, blog posts and inspiration every month

Subscribe to our newsletter