Collecting browser console logs in Stackdriver
TL;DR: This post explains how to improve the supportability of your browser JavaScript by exporting browser console logs to a central location in Google Stackdriver Logging, why you would want to do it, and discusses implementation options.
Background
As better tools like ES2015 modules, Angular, and TypeScript have become available for developing large scale browser-based applications, large applications have been developed, and it has become increasingly important to support them well. Unfortunately, today it is not a common practice to collect browser logs and errors in a central location. The typical support experience with a commercially supported web application is: an end user hits a problem, submits an error report, a developer or support engineer looks at server logs and finds no error on the server, the engineer then requests that the end user send browser logs. The end user may or may not have a support contract and may or may not know how to send browser logs. Even worse, console.log() messages may be stripped out by compilation tools. Worst of all, end users might not even bother to report the problem and just move on to a competing product. This results in a frustrating user experience and the developers being left in the dark about the problems being encountered by end users.
However, browser console logs and unhandled errors can be fairly easily and economically collected in a central location, for example with Stackdriver Logging on Google Cloud Platform. Not collecting the logs in a central location is a missed opportunity. This post will explain how it can be done with an example application that defines new log() and error() methods that send the logs and errors to the backend server running Node.js, which then logs to them at the server. It also optionally redefines the browser console log and error functions to call these implementations. This is optional because some developers may not want to log messages to the console and so actively strip out calls to console.log() from their code base. There is no dependence on Stackdriver proprietary APIs but the ease of doing with with Stackdriver is demonstrated.
See stackdriver-errors-js for code that does error reporting from browser JavaScript to the Stackdriver Error Reporting API. The advantage of that project is that it does not require a backend server. However, it does not send informational log messages, only errors.
Besides logs and errors, there are other interesting events that you may want to record to the server. For example, the browser history. Modern JavaScript frameworks like Angular make possible to develop a large set of views that appear to the user like multiple pages in a single page application. But there will be no record of the user’s navigation to the different views in web server request logs, which is another new gap in understanding of application usage patterns. This post does not cover that but similar principles to those described here apply.
Approach
The approach to collect console logs and errors is shown in the schematic diagram below.
In the diagram red indicates sample application code, blue indicates Google Cloud Platform services, and green indicates open source or a web browser.
The possible approaches to writing logs to Stackdriver or to another central location, in order of more to less obvious are
- Create a new API, say LogCollector,
- Send the logs to our server and save them to Stackdriver there
- Add some buffering so that a remote call is not needed for every log message
- Make it installable as an ES6 module for convenience and code maintainability
- Listen to GlobalEventHandlers.onerror to collect uncaught errors
- A disadvantage to this approach is that we cannot catch messages that are logged via console.log().
2. Redefine the console logging function
- This can be combined with the approach above.
- The flexible nature of JavaScript makes redefining functions on objects easy
- Follow the same points about buffering and sending to our server as above.
- This assumes that we have our own server, ie it is not a static web site. You could run a collector process on another server and configure Cross-Origin Resource Sharing (CORS) to allow the logs to be sent from a static web site.
3. Writing to Stackdriver with the Stackdriver APIs
- You can use the Stackdriver APIs Client Logging Libraries directly from the browser, although they are primarily intended for Node.js. However, you need to use OAuth 2.0 for this and execute the API on the user’s behalf. This is not a feasible approach if you cannot register and give access to GCP for the users of your application. See the GCP Authentication Overview for more details.
In this post I will combine the first and second approaches: define new log() and error() methods, redefine the console log and error functions, and send the logs and errors to a Node.js server where they will be logged. The demo app uses Cloud Run with logging using the standard console.log() and console.error() functions, which are directed to Stackdriver by Cloud Run. If you do not use Cloud Run then you can easily set up Stackdriver Logging by following the steps in Setting Up Stackdriver Logging for Node.js. Stackdriver Logging integrates with the Node.js Bunyan and Winston logging frameworks, which can also insulate you from proprietary APIs. Similar Stackdriver integrations are available for other languages.
Sample App
The sample application creates the log collector and drives a simple page with a web form allowing users to enter in their name and favorite color into text fields. The browser TypeScript to do this is shown shown below (file public/app.ts). The TypeScript compiler generates very similar JavaScript using (file public/app.js).
import { fromEvent } from "rxjs"; import { LogCollector, LogCollectorBuilder } from "./index.js";// Start the log collector
const buildId = "v0.01";
const builder = new LogCollectorBuilder().setBuildId(buildId).setReplaceConsole(true);
const logCollector = builder.makeLogCollector();
logCollector.start();// Handle events for the name form
const nameForm = document.getElementById("nameForm");
const nameTF = document.getElementById("nameTF") as HTMLInputElement;
if (nameForm) {
const events = fromEvent(nameForm, "submit");
events.subscribe( (event) => {
event.preventDefault();
const name = nameTF.value as string;
logCollector.log(`Your name is ${name}`);
return false;
});
}// Handle events for the favorite color form
const favColorForm = document.getElementById("favColorForm");
const favColorTF = document.getElementById("favColorTF") as HTMLInputElement;
if (favColorForm) {
const events = fromEvent(favColorForm, "submit");
events.subscribe( (event) => {
event.preventDefault();
const favColor = favColorTF.value as string;
console.log(`Your favorite color is ${favColor}`);
// recklessly generate a null pointer error
let nullValue: string | null = "";
nullValue = null;
nullValue!.toUpperCase();
return false;
});
}
The code first imports the LogCollector class, which is the main focus of this post, and the rxjs module which is used for responding to user events for form submission. I used Webpack to resolve the import statements and bundle them together into the file public/dist/bundle.js. A build id is used to identify the version of the app. After instantiating an instance of LogCollector, the start method is called which start a log flushing server, which will send buffered logs to the server at fixed intervals. The script’s response to form events demonstrates the console log method and also intentionally generates an uncaught TypeError to show how uncaught errors can be collected.
The JavaScript app drives a HTML page with a textfield in a form shown below. When a developer clicks on the button he or she can open the Developer Tools Console in Chrome or equivalent in another browser and see the message logged, as shown below.
The code for the log collection is contained in a JavaScript module exported in file public/index.ts:
export { LogCollectorBuilder } from './lib/LogCollectorBuilder';
export { LogCollector } from './lib/LogCollector';The module exports two classes, which are contained in their own files. Most of the implementation is in the class LogCollector: in file lib/LogCollector.ts:
const defaultLog = console.log.bind(console);export class LogCollector {
private buildId: string;
private logs = new Array<string>();
private errors = new Array<string>(); constructor(buildId: string, replaceConsole: boolean) {
this.buildId = buildId;
if (replaceConsole) {
console.log = (msg: string, ...args: object[]) => {
if (args && args.length) {
defaultLog(msg, args);
} else {
defaultLog(msg);
}
this.log(msg, args);
};
}
....
}
The constructor optionally replaces the console.log and console.error functions. There are separate callable LogCollector methods log() and error, which store the log and error messages to a buffer:
public log(msg: string, ...args: object[]) {
let message = msg;
if (args) {
message += args.join(", ");
}
this.logs.push(`browser app ${this.buildId}: ${message}`);
}Class LogCollectorBuilder is a builder class to create LogCollector instances.
The TypeScript files are compiled to JavaScript, which are also bundled into a single file dist/bundle.js by Webpack.
Inside the module the log messages and errors are stored in string arrays. Then every interval of 10 seconds or so the logs to the server with an AJAX call. For simplicity, this neglects the fact that the native console function takes a variable list of arguments, not just a single message argument.
Uncaught exceptions are collected by listening to GlobalEventHandlers.onerror
window.onerror = (msg, url, lineNo, columnNo, error) => {
if (error && error.stack) {
errors.push(`Uncaught error: ${msg} in url ${url}\n${error.stack}`);
} else {
errors.push(`Uncaught error: ${msg}\n url ${url}\n Line: ${lineNo}`);
}
};The server Node.js app includes a handler to receive these data and log them at the server, contained in file app.ts:
import * as express from "express";const app = express();
app.use(express.static("public"));
app.use(express.json());app.post("/log", (req: express.Request, res: express.Response) => {
if ("logs" in req.body) {
const logs = req.body["logs"];
if (logs && logs instanceof Array) {
logs.forEach( (log) => {
if (typeof log === "string") {
console.log(log);
} else {
sendError(`Log has wrong type: ${log}`, res);
}
});
}
...
When the application in our local environment using Node.js, the logs can be seen on the server command line:
npm run start
> browser-logs@0.0.1 start
> node app.js
App listening on port 8080
browser app v0.01: Your name is Alex
browser app v0.01: Your favorite color is red
Uncaught error: Uncaught TypeError: Cannot read property 'toUpperCase' of null in url webpack:///./node_modules/rxjs/_esm5/internal/util/hostReportError.js?
TypeError: Cannot read property 'toUpperCase' of null
at SafeSubscriber.eval [as _next] (webpack:///./app.js?:57:15)
at SafeSubscriber.__tryOrUnsub
...The uncaught browser exception can be seen in the output above.
The application can be run in the cloud with no code changes, for example, by deploying to Cloud Run. A Docker image for the Node.js app can be built with the command below. Se the shell variable PROJECT_ID to the appropriate value first.
gcloud builds submit --tag gcr.io/$PROJECT_ID/logging-demoThe app can be deployed to Cloud Run with the command
gcloud run deploy --image gcr.io/$PROJECT_ID/logging-demo --platform managedThen we can see the log entries appear in the Stackdriver console, as shown below
Errors are also tracked in Stackdriver Error Reporting, which gives details on when the error was first and last see and charts the timeline of frequency for the error.
The JavaScript Error object includes a stack trace, which is implemented on Chrome and Firefox. Logging errors with multi-line payloads, such as with a multi-line stack trace, will lead to the errors to be recorded by Stackdriver Error Reporting. Single line errors are not reported.
├── app.js
├── app.ts
├── Dockerfile
├── package.json
├── public
│ ├── app.js
│ ├── app.scss
│ ├── app.ts
│ ├── dist
│ │ ├── bundle.css
│ │ └── bundle.js
│ ├── index.html
│ ├── index.js
│ ├── index.ts
│ ├── lib
│ │ ├── LogCollectorBuilder.js
│ │ ├── LogCollectorBuilder.ts
│ │ ├── LogCollector.js
│ │ └── LogCollector.ts
│ ├── package.json
│ ├── tsconfig.json
│ ├── tslint.json
│ └── webpack.config.js
├── README.md
├── tsconfig.json
└── tslint.jsonNext Steps
- Do Stackdriver Error Reporting from the browser with the Client-side JavaScript library for Stackdriver Error Reporting
- Measure performance data from your web client with collection to Stackdriver Trace using opencensus-web.
- The log collection module does not have all the features of the browser console API. See the console documentation in the links below for more details.
Resources
- Setting Up Stackdriver Logging for Node.js at Google Cloud Platform
- Chrome console documentation at Google Web Fundamentals
- Mozilla console documentation at the Mozilla Developer Connection
