How JavaScript Works: the evolution of graphics

Published in

SessionStack Blog

11 min readJul 15, 2021

This is post # 36 of the series, dedicated to exploring JavaScript and its building components. In the process of identifying and describing the core elements, we also share some rules of thumb we use when building SessionStack, a JavaScript tool for developers to identify, visualize, and reproduce web app bugs through pixel-perfect session replay.

Overview

In this article, we will look at the fascinating evolution of graphics in browsers from the prehistoric days of the early browsers. We will start with simple images and make our way to the vast graphics offerings of modern browsers. Along the way, we will meet long-forgotten friends like Java Applets and ActiveX controls as well as newer friends like SVG, HTML 5 Canvas, and WebGL. Let’s get started with some historical perspective.

The web browser from 1990 until today

In 1989, Tim Berners-Lee (AKA TimBL) worked as a contractor at CERN, one of Europe’s largest research centers. Researchers all had their own computers with their own document formats and it was very difficult to share information. Tim Berners-Lee had to write several conversion programs to translate and copy information between systems. That was frustrating. Internet technologies and protocols like DNS and TCP already existed, as well as ideas like hypertext. Tim Berners Lee decided to put them together to solve the babel tower of information sharing. In his words: “I just had to take the hypertext idea and connect it to the TCP and DNS ideas and — ta-da! — the World Wide Web.”

Many more web browsers were created in the following years. Marc Andreessen and Eric Bina were two programmers at NCSA that were excited by the possibilities of the web. They developed Mosaic that was released in 1993 and became very popular. The two major innovations of Mosaic were introducing images to the text-only web and porting their browser to Windows. The combination of both turned the web from a niche for scientists sharing information into a mass-market application.

In 1994 Jim Clark and Marc Andressen created “Mosaic Communications Corporation” later renamed to “Netscape Communications Corporation”. The Mosaic Netscape 0.9 released in 1995 took over 75% of the market within a few months. The internet boom was real. Later versions were known as Netscape Navigator.

Most importantly, JavaScript was developed by Brendam Eich in 1995 for Netscape Navigator 2.

In 1995, Microsoft also released Internet Explorer 1.0. This signaled the beginning of the browsers wars that saw Netscape and Microsoft battle it out head to head. Netscape couldn’t compete with the massive resources of Microsoft and by the time IE 4 came out Microsoft dominated the market. By 2001, when IE 6 was launched Microsoft had an astounding 96% of the market. Microsoft won the first browser war.

**https://upload.wikimedia.org/wikipedia/commons/9/90/Browser_Wars_(en).png**

For 3 years innovation on the web practically stood still.

Then, came the second browser war. Mozilla Firefox has risen from the ashes of Netscape and its first version was released in 2004. Google released the Chrome browser in 2008 and the fight was on. By 2010, IE dropped below 50%. By 2012, Chrome superseded IE. Mobile browsers became important as more and more people started to use their phones to access the web.

**https://upload.wikimedia.org/wikipedia/commons/7/70/BrowserUsageShare.png**

There were a few other actors along the way like Opera and Apple’s Safari (still relevant), but they never made it above the 10% market share.

These days Chrome dominates the desktop with 77%, all others are below 10%.

Microsoft’s latest browser — Edge — is based on Chromium since 2020.

Ok. Let’s get into graphics and images.

Images in the browser

The venerable <img> tag was introduced by Mosaic in 1993 and later included in the first HTML 2.0 standard. Images are of course a staple of the web.

**https://upload.wikimedia.org/wikipedia/commons/d/de/Bad_cat_hides_lolcat.png**

In the early days of the web that’s all we had. People were still very creative and combined images with Javascript or Microsoft’s JScript :-) to move images around and create animations and even games.

Of course, the problem is performance. Fetching multiple images and/or large high-resolution images slows page loading. There are many tricks such as using image maps, lazy loading in the background, level of detail images, etc. But, the bottom line is that images are a crude tool for many applications.

Java Applets

Enter Java. Java was the language of the internet. I was a C++ programmer at the time and I was very excited reading the Java documentation and playing around with it. Somehow, I never got to write Java professionally and remained on the C side of C++, a little C# and these days Go (with plenty of Python too).

Java applets were a major force in the browsers world from 1995 until 2017 when browsers no longer supported it due to serious security issues. Java had a lot going on for it. It was cross-platform and supported by all browsers (via NPAPI) across all operating systems. It provided access to the 3D hardware accelerators of the underlying machine and initially, it was much faster than JavaScript. In the early days, Java applets were the way to go for interactive and visually rich applications.

Slowly but surely Java applets lost all their advantages until browser vendors phased them out completely.

Java applets are embedded on the web page using the `<applet>` tag just like any other HTML element. For example:

You provide the applet code as a Java class. The Java class must extend the java.applet. Applet class and it’s responsible for rendering whatever content it likes in its area on the page.

Here is a simple Java applet that just draws a circle:

ActiveX controls

Microsoft’s IE always supported Java applets for compatibility reasons, but Microsoft tried to bring its own technology stack based on COM/OLE into the browser. This is where ActiveX controls come in. Starting with IE 3.0 all the way to IE 11 you could embed ActiveX controls in web pages much like Java applets. ActiveX controls were supported by other browsers through plugins too. Modern browsers dropped support for ActiveX controls along with Java applets for the same reasons. ActiveX is not supported by Microsoft Edge either.

ActiveX controls are identified by a GUID called clsid. They are embedded into a web page using an <object> tag. For example:

Macromedia/Adobe Shockwave and Flash

Macromedia (1992–2005) was a web tooling and development powerhouse. It was eventually acquired by Adobe in 2005. Macromedia Shockwave player was a Netspace plugin that allows publishing to the web multimedia, graphics, and animations created using Adobe Director. Shockwave saw a lot of success and remained in the market as recently as 2019. In 1996, Macromedia released The Flash Player 1.0. Flash had its own editor (although later Director 8.5 had integration with Flash). Shockwave and Flash allowed the development of rich web applications, games, and visualizations. In 2000, Flash 5 was released with ActionScript 2.0 and it became ubiquitous. Adobe also made efforts to open-source components of the Flash platform like the SWF file format and donated the ActionScript VM to the Mozilla foundation.

Over the years, Flash enjoyed tremendous success and popularity, but there was criticism too. In 2010, Steve Jobs published an open letter to Adobe where he complained about it being a proprietary platform and its security issues (similar to Java applets and ActiveX controls).

In 2017, Adobe declared the end of life to Flash in 2020. In 2021, browsers blocked Flash content completely.

HTML 5 and the modern web

HTML5 was released in 2008 by W3C. It included support for multimedia applications as well as consideration for low-level devices like mobile browsers. HTML5 ushered in the modern era of the web where full-fledged rich web applications can be developed without plugins. It took more than a decade for the older technologies like Java applets, ActiveX controls, and Flash to completely fade out.

These days HTML is maintained by the WHATWG and a consortium of browser vendors and is known as the HTML living standard.

But, it all started with HTML5 that incorporated multiple web standards like HTML4, XHTML 1.0, and DOM Level 2. In addition, it added a plethora of new elements for managing page structure such as <main>, <section>, <article>, <header>, <footer>, <aside>, <menu>, and <nav>. Of course, the most prominent elements were <canvas>, <video>, and <audio> for graphics and multimedia as well as <svg> and <mathml> for vector graphics and mathematical formulas.

The adoption of HTML5 by all browser vendors opened the door for developing cross-browser and cross-platform applications using standard web technologies only.

The adoption of HTML 5 and standard web APIs for rich web applications also brought JavaScript to the forefront. Previously, a variety of programming languages were used to program the web. Java for Java applets, any language for ActiveX control, and ActionScript for Flash. Previously JavaScript was used primarily to add some interaction to web pages.

Let’s look at some of the web standards for graphics that are supported by modern browsers.

SVG (Scalable Vector Graphics)

SVG is a sophisticated XML dialect to describe vector graphics and much more. It is supported by all modern browsers. SVG content can be embedded in a web page just like other HTML elements.

Here is an example that renders a golden rectangle:

When embedding in a web page the XML metadata is not needed. Here is the same golden rectangle inside a web page:

SVG has the following properties:

Standard (unlike VML)
Scalable vectors graphics
Embedded in the markup
Scripted like any other element in the DOM

Canvas 2D API

The Canvas 2D API is arguably the most popular HTML 5 feature. It allows developers to render 2D graphics in the browser and program them using JavaScript. Unlike Java applets, ActiveX controls, or Flash the Canvas is not plugin-based and relies on web standards. The Canvas 2D API is one of the primary reasons all the plugin-based methods to render graphics in the browser could be retired.

It all starts with the <canvas> element, which as the name implies is just an area where you can draw some graphics. Once you have a <canvas> element you can get a 2D context, which exposes various methods to render shapes and text on your canvas.

Let’s see it in action. First, we need, an HTML document with a <canvas> element:

Then, using JavaScript we get the canvas from the document by its id and request a 2D context. The returned context is an object that implements the CanvasRenderingContext2D interface. This is the gateway for rendering shapes, paths, text as well as images on the canvas. Then, we can draw the golden rectangle as we did with SVG:

Essentially we provide the same information: the position, the dimensions, the stroke color, and the fill color. But, here everything is done programmatically. The context provides support for additional features like:

line styles
text styles
shadows
gradients
patterns
transformations
composites
images
pixel manipulation
canvas state
hit regions
filters

Overall, the 2D capabilities provided by the Canvas are comprehensive. Many libraries and frameworks have been built on top of the Canvas API. My favorite is the Phaser game framework.

WebGL

WebGL brings to the browser high-performance 3D and 2D APIs inspired by OpenGL — the leading industry API for interacting with GPUs. WebGL lets you benefit from hardware-accelerated rendering.

It is part of HTML5 just like the Canvas 2D API. To make things even more confusing WebGL also uses the <canvas> element and can render 2D graphics just like the Canvas 2D API.

However, only WebGL supports 3D graphics. WebGL has more moving parts and is considerably more complicated than the Canvas 2D API. Playing directly with the WebGL API is a non-trivial task. I suggest working with a library that presents a more user-friendly facade such as Three.js. Three.js brings a lot of concepts and capabilities that are familiar to 3D designers like:

Scenes
Cameras
Geometry
3D Model Loaders
Lights
Materials
Shaders
Particles
Animations
Math Utilities

Let’s look at an example. First, the HTML wrapper, where we pull the `three.js` library:

Then in the script block let’s add the following to set up a scene with a camera and add the renderer to the page.

Next, let’s add a golden cube, similar to the golden rectangle we created with SVG and Canvas 2D, except in 3D now:

To add some fun, let’s animate our cube and make it spin:

You can see the live example here:

As the technologies used to build web apps evolved, the tools needed to troubleshoot and monitor them changed as well. Nowadays, web apps are built with JavaScript which can be quite challenging to troubleshoot. A solution like SessionStack will make the troubleshooting process highly efficient as it allows you to replay customer journeys as videos. This allows you to see how customers interacted with your web app and what happened on their screen when the issue occurred. Combining this visual information with all of the technical details from the browser, device, network, and environment will give you all of the context needed to reproduce the problem and resolve it efficiently.

There is a free trial if you’d like to give SessionStack a try.

SessionStack replaying a session

Conclusion

The graphics capabilities of the browsers evolved over decades from simple images, through a variety of non-secure plugins to a healthy mix of safe and browser native APIs like SVG, 2D Canvas API, and WebGL to accommodate any need. Web developers today can write uniform code for advanced graphics using a variety of standard APIs. The code is executed natively by the browser, can take advantage of GPUs for top-notch performance, and doesn’t compromise security. Check out SVG, Canvas 2D, and WebGL. Whatever your use case is one of these APIs will help you accomplish your goals.

If you missed the previous chapters of the series, you can find them here: