Recording user sessions in closed environments: hybrid mobile and desktop apps, and browser extensions

The components of the non-web JavaScript app

We are all well aware of the building blocks of the modern website/web app:

• HTML defines the structure and textual content of the page
• CSS states the page look and feel
• JavaScript determines page behavior
• Other static resources like images, fonts, videos, etc. enrich overall user experience

These, however, are also the integral parts of browser extensions, hybrid mobile applications (e.g. Cordova-based ones), as well as Electron-based desktop applications (or alternatives to the Electron framework).

The beauty of it is that developers can reuse their web tech skills to craft hybrid mobile and desktop apps, or even browser plugins: tools like Cordova and Electron play the role of a bridge between the web and native/desktop technology. Cordova applications, for example, “execute within wrappers targeted to each platform and rely on standards-compliant API bindings to access each device’s capabilities such as sensors, data, and network status. The application itself is implemented as a web page that references CSS, JavaScript, images, media files, or other resources necessary for it to run.” (See Cordova’s docs for more details on app architecture)

Electron, on the other hand, is a framework for creating native applications with web technologies like JavaScript, HTML, and CSS, as well as various Node libraries. The interface container is handled by Chromium and uses a bundled Node.JS server to “serve” the app within the container. (See more on building apps with Electron)

A browser extension can be authored using web technologies too. Chrome extensions, for example, are just simple web apps that run in a little page that appears when the user clicks the icon. The only difference is that an extension requires a manifest file to define the structure, permissions, icons and other metadata. (Check out Aidan Breen’s blog post on building Chrome extensions.)

The challenges cross-platform tools bring to tracking user behavior

Session replay tools ‘record’ applications by listening for changes in the DOM (see how we do it at SessionStack by using MutationObserver) as well as for all user events like mouse clicks, mouse moves, page scrolls, window resizing and so on. All static resources referenced from the page that’s been recorded need to be accessed by the 3rd party tool so that it’s able to offer a pixel-perfect replay of the session. The collected data is used by the tool to recreate every single user activity on the page in the form of a video recording.

As you can tell, a common trait across the above development platforms and frameworks is the way static resources such as imagery, fonts, multimedia, etc. get served in a closed environment, where there’s an intermediary between the browser and the application, be it a WebView or a Manifest file. In essence, all static files are embedded in the application and loaded as files that live on a local server which makes it impossible for 3rd party tools to access them from outside the network.

This is what you’d be hoping to see on user session replay.

That’s how a session replay would look like if static files have not been loaded (and recorded).

And that’s precisely why most session replay tools hit a wall trying to record and replay user activities on mobile hybrid and desktop applications, or capture browser extension usage.

A workaround

A solution for this problem would be to serve the resources referenced from the app on a remote server which can be accessed externally. Also, the session recording tool should provide a way to change the original resource addresses to point to the remotely accessible server.

For example, the recording tool might have the ability to map the original URLs to different ones based on some rules defined by the user.

Another option would be if the tool allows changing the server origin of the app. This method would also require that all of the URLs in the app are relative and the resources on the remote server keep the same structure as in the app. The latter method is the one we employ to make sure SessionStack is able to record and replay user sessions in hybrid mobile and desktop apps, as well as browser add-ins:

Specify a new location for your app’s static resources to allow SessionStack to reproduce user behavior in your hybrid mobile and desktop apps.

tl;dr: if you’re looking for a session replay tool to track user behavior in hybrid mobile and desktop apps, as well as browser plugins, make sure it offers a way to reach the static resource files of the app.

This post was originally shared on the SessionStack blog.