Step-by-Step Guide: App Analytics and Visualizations

Polls for iMessage

About Us

Polls for iMessage is an iOS app that lets you send polls in your iMessage group chats.

People use our application to make group decisions as small as where to go for dinner or as big as where to throw the bachelorette party.

In This Guide

In this guide I’ll share some of my experience working with various analytics stacks and the one I’ve landed on which is working well for me.

The exercise we’ll follow as an example is generating a Bar Chart Race as shown above for your new users of your iOS app by region.

Disclaimer: I’m no expert! I’d love to hear your opinion and I’m always welcome to advice from others. This article is just meant to share what I have learned and my personal preferences for solving these problems.

Sending Analytics Events

The first step in the journey is to collect Analytics Events on the frontend (in this case, your iOS App), and send them to the backend. Unfortunately this is a very manual process for now. Essentially, any time something interesting happens in your app (such as the user tapping a button), you will need to call some method which records the event. The event is simply a string for the event name and a key value payload of additional context.

Guidance

This is one of those things that’s much easier to do right from the beginning than it is to go back and add in later. My recommendation is even if you aren’t going to add analytics now, you should set it up in your codebase.

Add an empty/stubbed method for tracking events. The method should accept the event name and payload. Now as you develop your app, whenever anything interesting happens, call your function with an event name and payload. Now if you ever choose to add analytics, your events are already tracked throughout your codebase and you just need to add an implementation to the empty/stubbed method. The bonus here is that you can also swap between different analytics libraries since you already have this layer of abstraction.

I recommend the same style of starting from the beginning for other things like error reporting and remote logging.

Pro-Tip: Use a string enum to keep your event list organized.

What to look for in an analytics client library

1. Make sure it’s compatible with the backend you are using, most backends (e.g. Google Analytics) have their own SDK you can use if you wish.
2. Supporting routing to multiple backends is nice, although you can easily implement this yourself in the frontend. Also take a look at Segment.io, which allows you to route to multiple destinations server-side so you’re not duplicating your analytics event network traffic on the frontend.
3. Batching/Caching events is a must. Analytics data is important but not critical path for your app’s functionality, so you don’t want it to interfere with normal usage. Analytics events should be batched together and sent when the user activity in your app is low (for example when user goes to the background). This should be handled for you by the framework. This is extra important if your app functions offline, the library should cache the events and upload later when the device is online.
4. Check out Heap.io, they are doing some amazing things with automated event collection. Huge time saver, although a bit expensive IMO.
5. Check the limitations (client side and server side) on event name length, number of unique event names, number of key-value pairs in the payload, and key and value lengths in the payload.

Notes for Firebase

I use Firebase and I’m very happy with it. It’s free and easy to use, but make sure to watch out for the limitations.

You can find their limitations here.

Viewing the data

Out of the box, Firebase gives you some nice visualizations of the data for free. You can see this chart with event count, user count, and count per user for any event.

In addition to this, Firebase automatically tracks a rolling trend of monthly active users (MAU), weekly active users (WAU), and daily active users (DAU).

Warning: Firebase has their own definition of what is considered an active user, and that may not match your definition so these numbers could be lower than you expect.

I highly recommend browsing the Firebase dashboard, there are some very powerful tools in there, such as Funnels to see where your users are dropping off in a critical workflow. You can segment users by analytics events, use machine learning to predict behavior based on event history, and more.

Google Analytics

As of recently, Firebase analytics events are now sent to Google Analytics, if you want to get more advanced there are even more tools available in Google Analytics.

Querying

I love the out-of-the-box visualizations from Firebase and Google Analytics, but I often find when browsing the data that to really learn from the data I need to be able to ask extremely complex and detailed questions which can only come form the power of SQL.

Linking Firebase to BigQuery

Firebase offers an integration with BigQuery that is very easy to use. Head to Project Settings -> BigQuery to enable it.

Important Notes!

1. Transferring the data will incur a very small cost, so I had to enable a billing account.
2. Enabling a billing account automatically upgrades you to the Firebase Blaze (pay as you go) plan. Your Firebase price will not increase (other than the transfer fees for BigQuery), but this means if you begin to do anything in Firebase that incurs a cost you will be charged since they now have your billing info.
3. To reduce your costs, limit what data is sent. For me that means just sending Google Analytics data.
4. IMPORTANT: the data has a default expiration date of 60 days. I didn’t realize this until data started disappearing.

Update Dataset Expiration in BigQuery

Navigate to your dataset (project_name -> analytics_12345) and edit the dataset info to update the expiration. Notice this only affects tables moving forward. The way the Firebase <> BigQuery integration works is that each day’s events are represented by a new table in the dataset. So I had to go back to each of my 60 tables and update their expiration as well since I didn’t catch this until later. If you set this up from the beginning though it will work for new days.

Querying the data

The structure of the events data is pretty odd and takes some getting used to, but trust that it’s done this way for a reason.

Here’s an example of a query that shows you how to get nested user parameters and event parameters from an event.

Note: When extracting user params or event params, change value.string_value to the appropriate type based on your data, e.g. value.int_value, or value_double_value.

Notice that BigQuery supports a wildcard in the table name, this allows us to query all the tables. Remember that each day gets its own table in BigQuery.

Here is the query I used to get new users by region:

Note: I used week number because all of my data was in one calendar year. If your data spans multiple years you will need a different date format that is unique and can be alphabetically sorted such as YYYY-mm.

Advanced Visualization

You’re in a great place now. Your app is collecting events, you can get easy insights from the dashboard, and you can drill into detailed questions with the full power of SQL.

However, you don’t want to have to run a SQL query every time you want to check the results. What we need next is a way to combine the nice auto-updating charts of Firebase with the power of BigQuery.

Enter DataStudio

Data Studio is a dashboard/report builder . Think of it as Google Slides + Google Sheets Charts powered by BigQuery data instead of a Google Sheet.

I won’t go into it in detail here, but this is where it all came together for me. I setup advanced SQL queries, and saved them as a BigQuery View. Then I linked the BigQuery View to DataStudio and wired it up to some charts (time series line graphs, bar chart, pie charts, histograms, etc.). The charts will autoupdate when the data in BigQuery updates.

Now instead of logging into the Firebase dashboard I check my own custom dashboard every day. I spend roughly $40 per month which is much cheaper than solutions like Periscope/Sisense and Looker. I also find the BigQuery and DataStudio UX much easier to work with and more powerful. I think this price could be lowered if I wrote my queries more efficiently and re-used views to build other views.

Creating the Bar Chart Race

I used Flourish to create the bar chart race. It’s a free tool and they have some really great ways of visualizing data temporally or geographically.

The format of the data needs to match as it is above:

1. Each bar in the race get it’s own row. For us that means region.
2. The name of each bar is in column A.
3. Columns B an onward will hold the data over time, for me each column represents 1 week.

I’m sure there’s a fancy way to get your data in this format using SQL, but I found it much easier to pull the data into Google Sheets and use a pivot table. BigQuery allows you to export the data in various ways. I believe exporting to CSV in Google Drive allowed me to avoid any limits. Once in Google Drive the CSV can be converted to a Google Sheet.

Select all the data and create a pivot table to match the format that Flourish needs.

Note: Before exporting to CSV, copy the data to a new sheet, excluding the first row and the totals (last row and lat column).

Export as CSV and upload to Flourish. Set the settings on Flourish as I have in the screenshot above and you should be up and running. From there Flourish is pretty self explanatory, you can customize to your liking!

What’s next

The bar chart race met our needs, but if you want to take your visualization to the next level here are some of our ideas:

1. Plot your growth on a map by correlating cities to coordinates using public data

1. Visualize social interactions from one region to another as an arc map

Thank you

Hope you enjoyed this article and it helps get you set up to start visualizing your data. Never underestimate the power of analytics data to fuel your product development. Even for something as simple as a Polls app, we have gained so many powerful insights from reviewing the data, I hope the same is true for you!