Understanding how browsers work by creating one
In this article, I will demonstrate how browsers work by creating a simple mini-browser. This toy example can show the basic rendering principles. After all, learning by doing is always effective, so let’s start the journey by building from scratch a browser that supports HTTP requests, HTML/CSS parsing, and flexbox layout. If you want to view the complete code, please refer to this repository.
A browser’s basic work process
To display a page, the browser takes the following steps:
- Requesting the page from the server using HTTP or HTTPS protocol
- Parsing the HTML to build a DOM tree
- Parsing the CSS to build a CSSOM tree
- Combine the DOM and CSSOM into a render tree
- Rendering the elements according to the CSS properties to get the bitmaps in memory
- Compositing bitmap for better performance (optional)
- Painting to display the content on the screen
The contents can be displayed on the screen as soon as they are individually ready, because the rendering engine will not wait until all HTML is parsed before starting to build and layout the DOM tree. In this way, the contents can be displayed on the screen gradually as soon as possible.
In our mini-browser project, we follow the steps below to implement it.
.
├── client
│ ├── index.js (step one)
│ ├── parser.js (step two and three)
│ ├── layout.js (step four)
│ ├── render.js (step five)
│ ├── package-lock.json
│ ├── package.json
│ └── viewport.jpg (the final result)
└── server
├── package-lock.json
├── package.json
└── server.jsStep one: network request
Let’s start with this basic directory structure:
.
├── client
│ └── index.js
└── server
└── server.jsSet up a server
First of all, let’s start a simple HTTP server which can send back a simple sentence “ Hello World\n”.
./server/server.js
const http = require("http");http.createServer((request, response) => {
let body = []; request.on('error', error => {
console.log(error);
})
.on('data', chunk => {
body.push(chunk.toString());
console.log(body)
})
.on('end', () => {
body = body.join('');
response.writeHead(200, { 'Content-Type': 'text/html' });
response.end(' Hello World\n');
});
}).listen(8080);console.log('The server is running!');
After installing the ‘http’ library and run the index.js file, the server is running.
npm install http
node server.jsI would recommend using the node monitor, so you don’t have to restart the server by hand every time you change something.
nodemon server.js
Construct and send an HTTP request
To send an HTTP request, we need to
- specify the host for IP
- specify the port for TCP
- specify the method, path, headers and body for HTTP
We put them in an IIFE (Immediately Invoked Function Expression) like this:
./client/index.js
void async function () {
const request = new Request({
host: '127.0.0.1',
port: 8080,
method: 'POST',
path: '/',
headers: {
['X-Jobpal']: 'chatbots',
},
body: {
name: 'Jobpal',
}
}); const response = await request.send();
console.log(response);
}()
According to these requirements, we can implement the request class as follows.
./client/index.js
const net = require('net');class Request {
constructor({
method,
host,
port,
path,
headers,
body
}) {
this.host = host;
this.port = port || 80;
this.method = method || 'GET';
this.path = path || '/';
this.headers = headers || {};
this.body = body || {}; if (!this.headers['Content-Type']) {
this.headers['Content-Type'] = 'application/x-www-form-urlencoded';
}
// encode the http body.
if (this.headers['Content-Type'] === 'application/json') {
this.bodyText = JSON.stringify(this.body);
} else if (this.headers['Content-Type'] === 'application/x-www-form-urlencoded') {
this.bodyText = Object.keys(this.body).map(
key => `${key}=${encodeURIComponent(this.body[key])}`
).join('&');
} this.headers["Content-Length"] = this.bodyText.length;
}
// The parameter is a TCP connection.
send(connection) {
return new Promise((resolve, reject) => {
if (connection) {
connection.write(this.toString());
} else {
// create a TCP connection
connection = net.createConnection(
{ host: this.host, port: this.port },
() => connection.write(this.toString())
);
} connection.on('data', data => {
console.log(data.toString());
console.log(JSON.stringify(data.toString()));
}); connection.on('error', error => {
console.log(error);
reject(error);
connection.end();
})
})
} toString() {
// the HTTP request
return `${this.method} ${this.path} HTTP/1.1\r
${Object.keys(this.headers).map(key => `${key}: ${this.headers[key]}`).join('\r\n')}\r
\r
${this.bodyText}`;
}
}
In the construction function, we save the parameters and set the default values. When dealing with headers, pay attention to content-type. It is required; otherwise, the body can not be parsed correctly, because different formats require different parsing methods.
These are the four most used format:
- application/json
- application/x-www-form-urlencoded (when submitting form)
- multipart/form-data (when uploading files)
- text/xml
In the project, we set the default value as application/x-www-form-urlencoded. In this format, the submitted data will be encoded like key1=value1&key2=value2.
In toString() method, the HTTP request is build. It is composed of a request line, request head, and request body.
Then, in the send() method, we send this request and print the response data when success.
Don’t forget to install the ‘net’ library, we need it to create a TCP connection.
As we can see, the structure of response data conforms to the above figure.
Maybe you have noticed that the response body is different from what we sent on the server-side.
- what we sent: ‘ Hello World\n’
- what we get: ‘d\r\n Hello World\n\r\n0\r\n\r\n’
The reason is the Transfer-Encoding: chunked in the response headers.
As described here:
chunked
Data is sent in a series of chunks. The Content-Length header is omitted in this case and at the beginning of each chunk you need to add the length of the current chunk in hexadecimal format, followed by
\r\nand then the chunk itself, followed by another\r\n. The terminating chunk is a regular chunk, with the exception that its length is zero. It is followed by the trailer, which consists of a (possibly empty) sequence of entity header fields.
So in the first chunk, d is a hexadecimal number, indicating that there are 13 characters, which are ‘HelloWorld’, two spaces, and a \n. The second chunk, the empty one, means the body part is over.
We have successfully got the response data. Next, let’s parse it.
Parse HTTP response
Parse response line and response head
As you can see, the response is such a string:
- the request line and the request header are separated by
\r\n - there are two
\r\nbetween the last request header and the request body
HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nDate: Sun, 02 Aug 2020 17:05:31 GMT\r\nConnection: keep-alive\r\nTransfer-Encoding: chunked\r\n\r\nd\r\n Hello World\n\r\n0\r\n\r\n
We can use a state machine to parse this string. The states can be designed according to the format of the response, like the code below.
In the index.js, we add the ResponseParser class.
./client/index.js
class ResponseParser {
constructor() {
this.WAITING_STATUS_LINE = 0;
this.WAITING_STATUS_LINE_END = 1; this.WAITING_HEADER_NAME = 2;
this.WAITING_HEADER_SPACE = 3;
this.WAITING_HEADER_VALUE = 4;
this.WAITING_HEADER_LINE_END = 5;
this.WAITING_HEADER_BLOCK_END = 6; this.WAITING_BODY = 7; // initial state
this.current = this.WAITING_STATUS_LINE; this.statusLine = '';
this.headers = {};
this.headerName = '';
this.headerValue = '';
this.bodyParser = null;
}
get isFinished() { } get response() { } receive(string) {
for (const char of string) {
this.receiveChar(char);
}
console.log(this.statusLine, '\n', this.headers)
}
// a state machine
receiveChar(char) {
switch (this.current) {
case this.WAITING_STATUS_LINE:
if (char === '\r') {
this.current = this.WAITING_STATUS_LINE_END;
} else {
this.statusLine += char;
}
break; case this.WAITING_STATUS_LINE_END:
if (char === '\n') {
this.current = this.WAITING_HEADER_NAME;
}
break; case this.WAITING_HEADER_NAME:
if (char === ':') {
this.current = this.WAITING_HEADER_SPACE;
} else if (char === '\r') {
// no key-value pairs at all or the end of k-v paris (no more)
this.current = this.WAITING_HEADER_BLOCK_END;
} else {
this.headerName += char;
}
break; case this.WAITING_HEADER_SPACE:
if (char === ' ') {
this.current = this.WAITING_HEADER_VALUE;
}
break;
case this.WAITING_HEADER_VALUE:
if (char === '\r') {
this.current = this.WAITING_HEADER_LINE_END;
} else {
this.headerValue += char;
}
break; case this.WAITING_HEADER_LINE_END:
if (char === '\n') {
this.current = this.WAITING_HEADER_NAME;
this.headers = {
...this.headers,
[this.headerName]: this.headerValue
}
this.headerName = '';
this.headerValue = '';
}
break; case this.WAITING_HEADER_BLOCK_END:
if (char === '\n') {
this.current = this.WAITING_BODY;
}
break; case this.WAITING_BODY:
console.log(JSON.stringify(char));
break; default:
break;
}
}
}
Then, in the send() method of the Request class, we pass the data to the parser.
send(connection) {
return new Promise((resolve, reject) => {
const parser = new ResponseParser; if (connection) {
connection.write(this.toString());
} else {
// create a TCP connection
connection = net.createConnection(
{ host: this.host, port: this.port },
() => connection.write(this.toString())
);
} connection.on('data', data => {
console.log(data.toString());
console.log(JSON.stringify(data.toString())); // receive the data and pass it to the parser
parser.receive(data.toString());
if (parser.isFinished) {
resolve(parser.response);
connection.end();
}
}); connection.on('error', error => {
console.log(error);
reject(error);
connection.end();
})
})
}
Executing the file, we can see that the response line and response headers have been parsed successfully, and the content of the response body has been printed.
Different types of response bodies require different parsing methods. Next, we only take chunked as an example.
Parse response body
In the newly added ChunkedBodyParser class, we also use a state machine to parse the response body.
The hexadecimal number and the content are separated by \r\n, so we design the states like this.
./client/index.js
class ChunkedBodyParser {
constructor() {
this.WAITING_LENGTH = 0;
this.WAITING_LENGTH_LINE_END = 1; this.READING_CHUNK = 2;
this.WAITING_BLANK_LINE = 3;
this.WAITING_BLANK_LINE_END = 4; this.current = this.WAITING_LENGTH; this.content = [];
this.length = 0;
this.isFinished = false;
}receiveChar(char) {
switch (this.current) {
case this.WAITING_LENGTH:
if (char === '\r') {
this.current = this.WAITING_LENGTH_LINE_END;
if (this.length === 0) {
this.isFinished = true;
}
} else {
// operation of hexadecimal numbers, for example, 2AF5, 2 => A => F => 5
this.length *= 16;
this.length += parseInt(char, 16);
}
break; case this.WAITING_LENGTH_LINE_END:
if (char === '\n' && !this.isFinished) {
this.current = this.READING_CHUNK;
}
break; case this.READING_CHUNK:
this.content.push(char);
this.length -= 1;
if (this.length === 0) {
this.current = this.WAITING_BLANK_LINE;
}
break; case this.WAITING_BLANK_LINE:
if (char === '\r') {
this.current = this.WAITING_BLANK_LINE_END;
}
break; case this.WAITING_BLANK_LINE_END:
if (char === '\n') {
this.current = this.WAITING_LENGTH;
}
break; default:
break;
}
}
}
Then, we use the parser in the receiveChar() method of the ResponseParser class.
receiveChar(char) {
...
case this.WAITING_HEADER_NAME:
if (char === ':') {
this.current = this.WAITING_HEADER_SPACE;
} else if (char === '\r') {
this.current = this.WAITING_HEADER_BLOCK_END;
if (this.headers['Transfer-Encoding'] === 'chunked') {
this.bodyParser = new ChunkedBodyParser();
}
} else {
this.headerName += char;
}
break;
...
case this.WAITING_BODY:
this.bodyParser.receiveChar(char);
break;
...
}At this point, the response line, response header, and response body are all parsed, we can complete the isFinished() and response() methods in ResponseParser class.
get isFinished() {
return this.bodyParser && this.bodyParser.isFinished;
}
get response() {
this.statusLine.match(/HTTP\/1.1 ([0-9]+) ([\s\S]+)/);
return {
statusCode: RegExp.$1,
statusText: RegExp.$2,
headers: this.headers,
body: this.bodyParser.content.join('')
}
}
This is the printed result.
Let’s send HTML from the server. After saving the changes, the node monitor should automatically restart the server. Requesting again from the client-side, the response is printed out.
./server/server.js
const http = require("http");http.createServer((request, response) => {
let body = []; request.on('error', error => {
console.log(error);
})
.on('data', chunk => {
body.push(chunk.toString());
console.log(body)
})
.on('end', () => {
body = body.join('');
response.writeHead(200, { 'Content-Type': 'text/html' });
response.end(
`<html maaa=a >
<head>
<style>
body div #myid{
width:100px;
background-color: #ff5000;
}
body div img{
width:30px;
background-color: #ff1111;
}
</style>
</head>
<body>
<div>
<img id="myid"/>
<img />
</div>
</body>
</html>`);
});
}).listen(8080);console.log('The server is running!');
So far, we have successfully received the HTML from the network request.
Now, we can try to parse HTML to get a DOM tree.
Step two: HTML parsing
Tokenization (lexical analysis)
A token represents the smallest meaningful unit in the compilation principle. In terms of HTML, 90% of the token that we need for daily development is only about startTag, attributes, endTag, comments, and CDATA nodes.
Take <p class="a">text</p> as an example, split it to the smallest meaningful units, we can get:
<pthe start of an opening tagclass="a"attribute>the end of an opening tag- text
- closing tag
Similar to the previous, we also use a state machine for parsing. Fortunately, the state machine has been well designed in the HTML standard, what we need to do is just translate it to JavaScript.
There are more than 80 states specified, but we don’t need so many states to parse our simple HTML. So in our sample code, some simplifications are made.
Setup an empty state machine
We design the state as a function so that the state transition part is very simple.
let state = data; // the initial state is called 'data' in the HTML standardfor (const char of html) {
state = state(char);
}
Add a new file “parser.js”.
./client/parser.js
const EOF = Symbol('EOF'); // end of file tokenmodule.exports.parseHTML = function (html) {
let state = data; for (const char of html) {
console.log(JSON.stringify(char), state.name)
state = state(char);
}
state = state(EOF);
}// There are three kinds of HTML tags: opening tag <div>, closing tag </div>, self-closing tag <div/>// the initial state is called 'data' in the HTML standard
function data(char) {
if (char === '<') {
return tagOpen;
} else if (char === EOF) {
return;
} else {
// text node
return data;
}
}// when the state is tagOpen, we don't know what kind of tag it is yet. like '<'
function tagOpen(char) {
if (char === '/') {
// '</' ,e.g. '</div>'
return endTagOpen;
} else if (char.match(/^[a-zA-Z]$/)) {
// the char is a letter, the tag could be a opening tag or a self-closing tag. e.g. '<d' => '<div>' or '</div>'
return tagName(char); // reconsume the char
} else {
// Parse error
return;
}
}
function endTagOpen(char) {
if (char.match(/^[a-zA-Z]$/)) {
return tagName(char);
} else if (char === '>') {
// error /> It's html, not JSX
// Parse error
} else if (char === EOF) {
// Parse error
} else {
// Parse error
}
}function tagName(char) {
if (char.match(/^[\t\n\f ]$/)) {
// tagname start from a '<', end with a ' ' e.g. '<div prop'
return beforeAttributeName;
} else if (char === '/') {
return selfClosingStartTag;
} else if (char.match(/^[a-zA-Z]$/)) {
return tagName;
} else if (char === '>') {
// the current tag is over, go back to the initial state to parse the next tag
return data;
} else {
return tagName;
}
}// Only '>' is valid after '<div/'
function selfClosingStartTag(char) {
if (char === ">") {
return data;
} else if (char === EOF) { } else { }
}function beforeAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/" || char === ">" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
// Parse error
} else {
return attributeName(char);
}
}function attributeName(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === EOF) { // end of a pair of attribute, e.g. "<div class='abc' "
return afterAttributeName(char);
} else if (char === "=") { // e.g. 'class='
return beforeAttributeValue;
} else if (char === "\u0000") { // null } else if (char === "\"" || char === "'" || char === "<") { } else {
return attributeName;
}
}
function beforeAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === ">" || char === EOF) {
return beforeAttributeValue;
} else if (char === "\"") {
return doubleQuotedAttributeValue; // e.g. <html attribute="
} else if (char === "\'") {
return singleQuotedAttributeValue; // e.g. <html attribute='
} else if (char === ">") {
// return data
} else {
return UnquotedAttributeValue(char); // e.g. <html attribute=
}
}function doubleQuotedAttributeValue(char) {
if (char === "\"") { // the second double quotes
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function singleQuotedAttributeValue(char) {
if (char === "\'") {
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return singleQuotedAttributeValue;
}
}// when the attribute part is over, save attribute name and value to the current token
function UnquotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) { // end of unquoted Attribute value
return beforeAttributeName; // to parse a new pair of attribute
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
return data;
} else if (char === "\u0000") { } else if (char === "\"" || char === "'" || char === "<" || char === "=" || char === "`") { } else if (char === EOF) { } else {
return UnquotedAttributeValue;
}
}function afterQuotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
return data;
} else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function afterAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return afterAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === ">") {
return data;
} else if (char === EOF) { } else {
return attributeName(char);
}
}
Then, use the new parser in the index.js.
./client/index.js
const parser = require('./parser');
...void async function () {
const request = new Request({
...
}); const response = await request.send();
const dom = parser.parseHTML(response.body);
console.log(JSON.stringify(dom, null, 2));
}()
Run the index.js, now the state machine is running! You can see the correspondence between characters and states at each step in the printed log.
The dom is undefined because now this state machine is empty, it does nothing. In the next step, let's add the corresponding logic code in these states to build the DOM tree.
Emit tokens
The tag types are nothing more than the starting-tag, the closing tag, and the self-closing tag. In the state machine, we emit the tag token when we encounter the end of a tag.
The emit() function takes the token generated from the state machine. In this step, it does nothing but prints the received token.
So the parser.js looks like this.
./client/parser.js
const EOF = Symbol('EOF');
let currentToken = null;function emit(token) {
console.log(token);
}module.exports.parseHTML = function (html) {
let state = data; for (const char of html) {
state = state(char);
}
state = state(EOF);
}function data(char) {
if (char === '<') {
return tagOpen;
} else if (char === EOF) {
emit({ type: 'EOF' });
return;
} else {
// emit the text nodes one by one, we can join them later
emit({
type: 'text',
content: char,
});
return data;
}
}function tagOpen(char) {
if (char === '/') {
return endTagOpen;
} else if (char.match(/^[a-zA-Z]$/)) {
currentToken = {
type: 'startTag',
tagName: '',
}
return tagName(char); // reconsume
} else {
// Parse error
return;
}
}
function endTagOpen(char) {
if (char.match(/^[a-zA-Z]$/)) {
currentToken = {
type: 'endTag',
tagName: ''
}
return tagName(char);
} else if (char === '>') {
// Parse error
} else if (char === EOF) {
// Parse error
} else {
// Parse error
}
}function tagName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === '/') {
return selfClosingStartTag;
} else if (char.match(/^[a-zA-Z]$/)) {
currentToken.tagName += char;
return tagName;
} else if (char === '>') {
// the current tag is over, go back to the initial state to parse the next tag
emit(currentToken);
return data;
} else {
return tagName;
}
}function selfClosingStartTag(char) {
if (char === ">") {
currentToken.isSelfClosing = true;
emit(currentToken);
return data;
} else if (char === EOF) { } else { }
}function beforeAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/" || char === ">" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
// parser error
} else {
return attributeName(char);
}
}function attributeName(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === "\u0000") { // null } else if (char === "\"" || char === "'" || char === "<") { } else {
return attributeName;
}
}function beforeAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === ">" || char === EOF) {
return beforeAttributeValue;
} else if (char === "\"") {
return doubleQuotedAttributeValue;
} else if (char === "\'") {
return singleQuotedAttributeValue;
} else if (char === ">") {
// return data
} else {
return UnquotedAttributeValue(char);
}
}function doubleQuotedAttributeValue(char) {
if (char === "\"") {
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function singleQuotedAttributeValue(char) {
if (char === "\'") {
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return singleQuotedAttributeValue;
}
}function UnquotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === "\u0000") { } else if (char === "\"" || char === "'" || char === "<" || char === "=" || char === "`") { } else if (char === EOF) { } else {
return UnquotedAttributeValue;
}
}function afterQuotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function afterAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return afterAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === EOF) { } else {
return attributeName(char);
}
}
Now, we get these tokens. You might have noticed that the attributes are missing. For example, the token of the img tag with id "myid" is { type: 'startTag', tagName: 'img', isSelfClosing: true }. The id is not included.
The way of adding attributes is similar to the previous step. Add the currentAttribute variable and complete the logic in the state machine like below.
./client/parser.js
const EOF = Symbol('EOF');
let currentToken = null;
let currentAttribute = null;...function beforeAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/" || char === ">" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
// parse error
} else {
return attributeName(char);
}
}function attributeName(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === "\u0000") { } else if (char === "\"" || char === "'" || char === "<") { } else {
return attributeName;
}
}
function attributeName(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === EOF) {
return afterAttributeName(char);
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === "\u0000") { // null } else if (char === "\"" || char === "'" || char === "<") { } else {
return attributeName;
}
}function beforeAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/) || char === "/" || char === ">" || char === EOF) {
return beforeAttributeValue;
} else if (char === "\"") {
return doubleQuotedAttributeValue;
} else if (char === "\'") {
return singleQuotedAttributeValue;
} else if (char === ">") {
// return data
} else {
return UnquotedAttributeValue(char);
}
}function doubleQuotedAttributeValue(char) {
if (char === "\"") {
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function singleQuotedAttributeValue(char) {
if (char === "\'") {
return afterQuotedAttributeValue;
} else if (char === "\u0000") { } else if (char === EOF) { } else {
return singleQuotedAttributeValue;
}
}function UnquotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === "\u0000") { } else if (char === "\"" || char === "'" || char === "<" || char === "=" || char === "`") { } else if (char === EOF) { } else {
return UnquotedAttributeValue;
}
}function afterQuotedAttributeValue(char) {
if (char.match(/^[\t\n\f ]$/)) {
return beforeAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === EOF) { } else {
return doubleQuotedAttributeValue;
}
}function afterAttributeName(char) {
if (char.match(/^[\t\n\f ]$/)) {
return afterAttributeName;
} else if (char === "/") {
return selfClosingStartTag;
} else if (char === "=") {
return beforeAttributeValue;
} else if (char === ">") {
emit(currentToken);
return data;
} else if (char === EOF) { } else {
return attributeName(char);
}
}
Run the index again, you can find out that now the attributes come back!
At this point, we have split the character stream into tokens, and then let’s use these tokens to build a DOM tree.
DOM tree construction (syntactic analysis)
In real browsers, the HTML nodes inheritance from different subclasses of Node. To simplify our code, we only divide Node into Element and Text.
For elements, we construct the DOM tree by using a stack to match the tags.
Specifically, when the emit function receives the token, it starts to build the DOM tree. When it encounters a startTag, it pushes the element into the stack, and when it encounters the matching endTag, it pops the top element out of the stack. Self-closing tags do not need to be pushed into the stack.
It is worth mentioning that when a tag mismatch occurs, the real browser will do fault-tolerant processing, while we just throw an error here.
When all tokens are received, the top element (the document element) of the stack is the root node of the DOM tree.
For Text nodes, we need to merge them when they are adjacent. When it is pushed into the stack, check whether the top node of the stack is a Text node. If so, merge the Text nodes and then add the text nodes to the DOM tree.
./client/parser.js
const EOF = Symbol('EOF');
let currentToken = null;
let currentAttribute = null;
let currentTextNode = null;// a stack containing the root node
const stack = [{ type: 'document', children: [] }];module.exports.parseHTML = function (html) {
let state = data; for (const char of html) {
state = state(char);
}
state = state(EOF);
// return the DOM tree
return stack[0];
}...function emit(token) {
let top = stack[stack.length - 1]; if (token.type === 'startTag') {
// create the element
let element = {
type: 'element',
children: [],
attributes: [],
tagName: token.tagName
}for (const prop in token) {
if (prop !== "type" && prop !== "tagName") {
element.attributes.push({
name: prop,
value: token[prop],
})
}
}
top.children.push(element);
// comment out this line to avoid the circular structure error when print the DOM tree
// element.parent = top; if (!token.isSelfClosing) {
stack.push(element);
} currentTextNode = null;
} else if (token.type === 'endTag') {
if (top.tagName !== token.tagName) {
throw new Error('Tag does not match');
} else {
stack.pop();
}
currentTextNode = null;
} else if (token.type === 'text') {
if (currentTextNode === null) {
currentTextNode = {
type: "text",
content: "",
}
top.children.push(currentTextNode);
}
currentTextNode.content += token.content;
}
}
Run the index again, instead of undefined, now we can see a DOM tree here! Under the document node is the html node, below it is the head node and the body node.
{
"type": "document",
"children": [
{
"type": "element",
"children": [
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [
{
"type": "text",
"content": "\n body div #myid{\n width:100px;\n background-color: #ff5000;\n }\n body div img{\n width:30px;\n background-color: #ff1111;\n }\n "
}
],
"attributes": [],
"tagName": "style"
},
{
"type": "text",
"content": "\n "
}
],
"attributes": [],
"tagName": "head"
},
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [],
"attributes": [
{
"name": "id",
"value": "myid"
},
{
"name": "isSelfClosing",
"value": true
}
],
"tagName": "img"
},
{
"type": "text",
"content": "\n "
},
{
"type": "element",
"children": [],
"attributes": [
{
"name": "isSelfClosing",
"value": true
}
],
"tagName": "img"
},
{
"type": "text",
"content": "\n "
}
],
"attributes": [],
"tagName": "div"
},
{
"type": "text",
"content": "\n "
}
],
"attributes": [],
"tagName": "body"
},
{
"type": "text",
"content": "\n "
}
],
"attributes": [
{
"name": "maaa",
"value": "a"
}
],
"tagName": "html"
}
]
}Look at the DOM tree we have here. The structure is correct, but it is bare, without any decoration.
Step three: CSS computing
Let’s make it a beautiful Christmas tree with CSS in this step!
During the CSS computation step, parsed CSS rules are added to the matched DOM elements. Note that CSS computing also occurs during the construction of the DOM tree.
We need to install the css library which can parse the CSS code into a CSS AST (abstract syntax tree). To simplify, we use the AST to replace the role of CSSOM in our sample project.
npm install css
Gather the CSS rules
First, we need to gather all CSS rules together from the CSS AST.
./client/parser.js
const css = require("css");
...// to save the CSS rules
const rules = []; // gather all the CSS rules
function addCSSRules(text) {
const ast = css.parse(text);
rules.push(...ast.stylesheet.rules);
}
Then, execute it at the end of the style tag (ignore CSS in other locations).
So, in the emit() method:
...
else if (token.type === 'endTag') {
if (top.tagName !== token.tagName) {
throw new Error('Tag does not match');
} else {
if (top.tagName === 'style') {
addCSSRules(top.children[0].content); // The children of the top element in the stack are the current element
}
stack.pop();
}
currentTextNode = null;
}
...If you print the rules, you’ll find that the CSS rules have been collected.
Match CSS rules and elements
When to do it? Generally speaking, we will try to ensure that all selectors can be judged when startTag is entered. With the later addition of advanced selectors, this rule has been loosened. Due to the limited length of the article, we only use simple selectors in our sample code. So when there is a startTag, it is already possible to determine which CSS rules the element matches.
In real browsers, there may be style tags in the body that require recalculation of CSS. We ignore this kind of situation.
Call the computeCSS in the emit function.
./client/parser.js
function emit(token) {
let top = stack[stack.length - 1]; if (token.type === 'startTag') {
let element = {...} for (const prop in token) {
...
}
// CSS computing happens during the DOM tree construction
computeCSS(element);
top.children.push(element);
...
In the computeCSS function, in order to match the selector with the corresponding element, we need to get the ancestor element sequence of the current element.
const elements = stack.slice().reverse();
All ancestor elements of the current element are stored in the stack. Because the stack is changing, the slice() method is used to save a copy of the current state.
When we try to determine whether an element matches the selector, we start with the current element, then its parent element, and step by step outward. Take “div #myid” as an example, the div can be any ancestor element, but #myid must be the current element.
In addition, we need a way to calculate whether the selector matches the element. Also for simplicity, we only deal with the case of descendant selectors composed of simple selectors and spaces, like ‘div #myid’.
./client/parser.js
...// assuming selector is a simple selector (.class #id tagname)
function match(element, selector) {
if (!selector || !element.attributes) return false; if (selector.charAt(0) === '#') { // id selector
const attr = element.attributes.filter(
({ name }) => (name === 'id')
)[0];
if (attr && attr.value === selector.replace('#', '')) {
return true;
}
} else if (selector.charAt(0) === '.') { // class selector
const attr = element.attributes.filter(
({ name }) => (name === 'class')
)[0];
if (attr && attr.value === selector.replace(".", "")) {
return true;
}
} else { // type selector
if (element.tagName === selector) return true;
}
}
function computeCSS(element) {
const elements = stack.slice().reverse();for (const rule of rules) {
const selectors = rule.selectors[0].split(' ').reverse(); // the structure of the ast and don't forget to do the reversing
// selectors are like '[ '#myid', 'div', 'body' ]' if (!match(element, selectors[0])) continue; let matched = false;let selectorIndex = 1; // elements is parent element or ancestor element, so here we start from 1
for (let elementIndex = 0; elementIndex < elements.length; elementIndex++) {
if (match(elements[elementIndex], selectors[selectorIndex])) {
selectorIndex++;
}
}
if (selectorIndex >= selectors.length) {
// all selectors are matched
matched = true;
}
if (matched) {
console.log(`Selector "${rule.selectors[0]}" has matched element ${JSON.stringify(element)}`)
}
}
}...
From the printed result, we can see that the element and the selector have been correctly matched.
Selector "body div #myid" has matched element {"type":"element","children":[],"attributes":[{"name":"id","value":"myid"},{"name":"isSelfClosing","value":true}],"tagName":"img"}
After the successful match, we should add CSS rules to the corresponding elements.
This step is very simple. In the computeCSS function, we add a computedStyle property to the element and save the CSS rules into it when matching.
function computeCSS(element) {
if (!element.computedStyle) {
element.computedStyle = {};
} const elements = stack.slice().reverse(); for (const rule of rules) {
...
if (matched) {
const { computedStyle } = element;
for (const declaration of rule.declarations) {
const { property, value } = declaration;
if (!computedStyle[property]) {
computedStyle[property] = {};
}
computedStyle[property] = value;
}
}
}
console.log(element.computedStyle)
}
Now the elements have their CSS rules, but it’s not enough yet. Go check these two img elements, you’ll find that they have the same computedstyle property, which is wrong.
{ width: '30px', 'background-color': '#ff1111' }
Why? The reason is the CSS specificity. The rules with higher specificity should always have higher priority. The currently wrong result is because the later low-specificity rules overwrite the previous high-specificity rules.
To calculate the CSS specificity, we need a quaternion: [inline, id, class/attribute, tagName]（Inline styles have the highest priority, and the rest is number of each type of selectors）.
For example, for the case[a, b, c, d], the calculation formula is:
specificity = a * N³ + b * N² + c * N + d
N is a big number. In the old version of IE, in order to save memory, the value of N is 255, which is not large enough, resulting in a funny bug that 256 classes are equivalent to one id. Of course, any sane developer would not write 256 selectors. Nowadays, most browsers set N as 65536.
Based on this, we can implement functions for calculating and comparing CSS specificity. Again, this is also a simplified version.
function getSpecificity(selector) {
const specificity = [0, 0, 0, 0];
//Similarly, assuming that selector is composed of simple selectors.
const selectors = selector.split(' '); for (const item of selectors) {
if (item.charAt(0) === '#') {
specificity[1] += 1;
} else if (item.charAt(0) === '.') {
specificity[2] += 1;
} else {
specificity[3] += 1;
}
}
return specificity;
}function compareSpecificity(sp1, sp2) {
if (sp1[0] - sp2[0]) {
return sp1[0] - sp2[0];
}
if (sp1[1] - sp2[1]) {
return sp1[1] - sp2[1];
}
if (sp1[2] - sp2[2]) {
return sp1[2] - sp2[2];
}
return sp1[3] - sp2[3];
}
In the computeCSS function, when an element matches a selector, instead of simply apply the CSS rules, we need to compare the CSS specificity and use the rules with higher specificity.
if (matched) {
const { computedStyle } = element;
const specificity = getSpecificity(rule.selectors[0]); for (const declaration of rule.declarations) {
const { property, value } = declaration;
if (!computedStyle[property]) {
computedStyle[property] = {};
}
// computedStyle[property] = value;
// CSS specificity
if (!computedStyle[property].specificity) {
computedStyle[property] = {
value,
specificity,
...computedStyle[property],
}
} else if (compareSpecificity(computedStyle[property].specificity, specificity) < 0) {
// current CSS selector have higher specificity than the previous, cover the previous rules
computedStyle[property] = {
value,
specificity,
...computedStyle[property],
}
}
}
}
Run it again, as it printed, the img element with ID is correctly displayed as '#ff5000'.
Now, if you print the DOM tree again, you will see a beautiful Christmas tree with all the CSS decoration.
Next, our job is to calculate the position of each element so we can get a DOM tree with position information.
Step four: layout
In CSS, there are three generations of layout technology:
- normal flow
- flexbox
- grid
We are going to implement the most popular one: flexbox. If you are not familiar with this technique, please check this link.
Main axis and cross axis
Before we start, we must first understand the concepts of the main axis and the cross axis. By default, the main axis is horizontal (from left to right) and the cross axis is vertical (from top to bottom). It can also change with our settings. For example, when the value of ‘flex-direction’ is ‘row’, the main axis is horizontal, when the value is ‘column’, the main axis is vertical. Using these concepts can help us reduce a lot of unnecessary if-else code.
When to do the layout
Suppose we already have the layout() function, when should we call it? To calculate the flexbox layout, we need to know all the children elements of the current element. So we should call the layout() function in the ‘endTag’ branch.
./client/parser.js
const css = require("css");
const layout = require("./layout.js");...function emit(token) {
...
else if (token.type === 'endTag') {
if (top.tagName !== token.tagName) {
throw new Error('Tag does not match');
} else {
if (top.tagName === 'style') {
addCSSRules(top.children[0].content);
}
// right here!
layout(top);
stack.pop();
}
currentTextNode = null;
}
...
}
Pre-processing
In the newly created layout.js file, let's add a getStyle() function to do some pre-processing work like filtering out unwanted elements and type conversion(string to number).
./client/layout.js
function getStyle(element) {
if (!element.style) element.style = {}; const { computedStyle, style } = element; for (const prop in computedStyle) {
if (style[prop]) continue;
element.style[prop] = computedStyle[prop].value; if (element.style[prop].toString().match(/px$/)) {
element.style[prop] = parseInt(element.style[prop]);
}
if (element.style[prop].toString().match(/^[0-9\.]+$/)) {
element.style[prop] = parseInt(element.style[prop]);
}
}
return element.style;
}
Set default values
Set default values in the layout function.
./client/layout.js
function layout(element) {
if (!element.computedStyle) return; const elementStyle = getStyle(element);
if (elementStyle.display !== 'flex') return; // filter out text nodes
const elementItems = element.children.filter(
el => el.type === 'element'
); // to support the order property
elementItems.sort((a, b) => (a.order || 0) - (b.order || 0)); const style = elementStyle; ['width', 'height'].forEach(size => {
if (style[size] === 'auto' || style[size] === '') {
style[size] = null;
}
}) // set default values
if (!style['flex-direction'] || style['flex-direction'] === 'auto') {
style['flex-direction'] = 'row';
}
if (!style['align-items'] || style['align-items'] === 'auto') {
style['align-items'] = 'stretch';
}
if (!style['justify-content'] || style['justify-content'] === 'auto') {
style['justify-content'] = 'flex-start';
}
if (!style['flex-wrap'] || style['flex-wrap'] === 'auto') {
style['flex-wrap'] = 'nowrap';
}
if (!style['align-content'] || style['align-content'] === 'auto') {
style['align-content'] = 'stretch';
} let mainSize,
mainStart,
mainEnd,
mainSign,
mainBase,
crossSize,
crossStart,
crossEnd,
crossSign,
crossBase; if (style['flex-direction'] === 'row') {
mainSize = 'width';
mainStart = 'left';
mainEnd = 'right';
mainSign = +1;
mainBase = 0; crossSize = 'height';
crossStart = 'top';
crossEnd = 'bottom';
} else if (style['flex-direction'] === 'row-reverse') {
mainSize = 'width';
mainStart = 'right';
mainEnd = 'left';
mainSign = -1;
mainBase = style.width; crossSize = 'height';
crossStart = 'top';
crossEnd = 'bottom';
} else if (style['flex-direction'] === 'column') {
mainSize = 'height';
mainStart = 'top';
mainEnd = 'bottom';
mainSign = +1;
mainBase = 0; crossSize = 'width';
crossStart = 'left';
crossEnd = 'right';
} else if (style['flex-direction'] === 'column-reverse') {
mainSize = 'height';
mainStart = 'bottom';
mainEnd = 'top';
mainSign = -1;
mainBase = style.height; crossSize = 'width';
crossStart = 'left';
crossEnd = 'right';
} if (style['flex-wrap'] === 'wrap-reverse') {
const [crossEnd, crossStart] = [crossStart, crossEnd];
crossSign = -1;
} else {
crossBase = 0;
crossSign = +1;
}}
Collect elements into flex line
Put the elements in one line if there is enough room or no-wrap, otherwise, start a new line.
There might be a special case: If the parent element does not have a main axis size (like width), the parent element is stretched by the child element. This mode is called the “auto main size”. In this case, the elements are gathered in one line.
./client/layout.js (continue with the previous code block)
// the special case
let isAutoMainSize = false;
if (!style[mainSize]) {
// auto sizing
elementStyle[mainSize] = 0;
for (let i = 0; i < elementItems.length; i++) {
const itemStyle = getStyle(elementItems[i]);
if (itemStyle[mainSize] !== null || itemStyle[mainSize] !== (void 0)) {
elementStyle[mainSize] = elementStyle[mainSize] + itemStyle[mainSize];
}
}
isAutoMainSize = true;
}
// normal case
let flexLine = [];
const flexLines = [flexLine]; // mainSpace is the current remaining space, set it to the mainsize of the parent element
let mainSpace = elementStyle[mainSize];
let crossSpace = 0; // loop all the flex items
for (let i = 0; i < elementItems.length; i++) {
const item = elementItems[i];
const itemStyle = getStyle(item);
if (itemStyle[mainSize] === null) {
itemStyle[mainSize] = 0;
}
if (itemStyle.flex) { // one line
flexLine.push(item);
} else if (style['flex-wrap'] === 'nowrap' && isAutoMainSize) {
mainSpace -= itemStyle[mainSize];
if (itemStyle[crossSize] !== null && itemStyle[crossSize] !== (void 0)) {
crossSpace = Math.max(crossSpace, itemStyle[crossSize]);
}
// because of 'nowrap'
flexLine.push(item);
} else { // multiple lines
// if the mainsize of the element is larger than it's parent element, it will be compressed to the same size as the parent element.
if (itemStyle[mainSize] > style[mainSize]) {
itemStyle[mainSize] = style[mainSize];
}
// start a new line
if (mainSpace < itemStyle[mainSize]) {
flexLine.mainSpace = mainSpace;
flexLine.crossSpace = crossSpace;
// create the new line
flexLine = [item];
flexLines.push(flexLine);
// reset these two props
mainSpace = style[mainSize];
crossSpace = 0;
} else {
flexLine.push(item);
}
if (itemStyle[crossSize] !== null && itemStyle[crossSize] !== (void 0)) {
crossSpace = Math.max(crossSpace, itemStyle[crossSize]);
}
mainSpace -= itemStyle[mainSize];
}
}
flexLine.mainSpace = mainSpace;
Calculate the main axis
Find all flex items and assign the remaining space mainspace in the main axis direction to these flex items proportionally. Special case: if the remaining space is a negative number (like in the “no-wrap” mode), set the size of the main axis of all flex items elements to 0, and compress the remaining elements proportionally.
If there are no flex items, calculate the position of each element according to the value of justify-content.
./client/layout.js (continue with the previous code block)
if (style['flex-wrap'] === "nowrap" || isAutoMainSize) {
flexLine.crossSpace = (style[crossSize] !== undefined)
? style[crossSize]
: crossSpace;
} else {
flexLine.crossSpace = crossSpace;
}if (mainSpace < 0) {
// overflow (happens only if container is single line), scale every item
// the special case, proportional compression
let scale = style[mainSize] / (style[mainSize] - mainSpace);
let currentMain = mainBase;for (let i = 0; i < elementItems.length; i++) {
const itemStyle = getStyle(elementItems[i]);
if (itemStyle.flex) {
// flex elements do not participate in proportional compression
itemStyle[mainSize] = 0;
} itemStyle[mainSize] = itemStyle[mainSize] * scale; // if the flex direction is row，this part is to calculate left and right after compression
itemStyle[mainStart] = currentMain;
itemStyle[mainEnd] = itemStyle[mainStart] + mainSign * itemStyle[mainSize];
currentMain = itemStyle[mainEnd];
}
} else {
// multiple lines
// process each flex line
flexLines.forEach(flexLine => {
const mainSpace = flexLine.mainSpace;
let itemStyle;
let flexTotal = 0;
for (let i = 0; i < flexLine.length; i++) {
itemStyle = getStyle(flexLine[i]);
// find the flex items to get the value of flexTotal
if ((itemStyle.flex !== null) && (itemStyle.flex !== (void 0))) {
flexTotal += itemStyle.flex;
continue;
}
} // if flex elements exist, distribute mianSpace evenly to each flex element
if (flexTotal > 0) {
let currentMain = mainBase;
for (let i = 0; i < flexLine.length; i++) {
itemStyle = getStyle(flexLine[i]);
if (itemStyle.flex) {
itemStyle[mainSize] = (mainSpace / flexTotal) * itemStyle.flex;
} itemStyle[mainStart] = currentMain;
itemStyle[mainEnd] = itemStyle[mainStart] + mainSign * itemStyle[mainSize];
currentMain = itemStyle[mainEnd];
}
} else {
// if there is no flex element, the remaining space in the main axis direction is allocated according to the rules of justifyContent
let currentMain, gap;
if (style['justify-content'] === 'flex-start') {
currentMain = mainBase;
gap = 0; // there is no space between each element
}
if (style['justify-content'] === 'flex-end') {
currentMain = mainBase + mainSpace * mainSign;
gap = 0;
}
if (style['justify-content'] === 'center') {
currentMain = mainBase + mainSpace / 2 * mainSign;
gap = 0;
}
if (style['justify-content'] === 'space-between') {
currentMain = mainBase;
gap = mainSpace / (elementItems.length - 1) * mainSign;
}
if (style['justify-content'] === 'space-around') {
currentMain = gap / 2 + mainBase;
gap = mainSpace / elementItems.length * mainSign;
}
if (style['justify-content'] === 'space-evenly') {
gap = mainSpace / (elementItems.length + 1) * mainSign
currentMain = gap + mainBase
}
// calculate mainend based on mainstart and mainsize
for (let i = 0; i < flexLine.length; i++) {
itemStyle[mainStart] = currentMain;
itemStyle[mainEnd] = itemStyle[mainStart] + mainSign * itemStyle[mainSize];
currentMain = itemStyle[mainEnd] + gap;
}
}
})
}
If the flex-direction is “row”, at this point we have the value of width, left, and right. The next step is to calculate the cross axis to get the value of height, top, and bottom. When these are determined, the position of the element is also determined.
Calculate the cross axis
- the height of the line based on the height of the largest element
- the specific position of elements can be calculated according to the value of
flex-alignanditem-align
./client/layout.js (continue with the previous code block)
if (!style[crossSize]) {
// if the parent element has no crossSize, crossSpace will always be zero
crossSpace = 0;
elementStyle[crossSize] = 0;
// add the height of the expansion
for (let i = 0; i < flexLines.length; i++) {
elementStyle[crossSize] = elementStyle[crossSize] + flexLines[i].crossSpace;
}
} else {
crossSpace = style[crossSize];
for (let i = 0; i < flexLines.length; i++) {
crossSpace -= flexLines[i].crossSpace;
}
} // wrap-reverse this affects crossBase
if (style['flex-wrap'] === 'wrap-reverse') {
crossBase = style[crossSize];
} else {
crossBase = 0;
} let lineSize = style[crossSize] / flexLines.length;
let gap;
if (style['align-content'] === 'flex-start') {
crossBase += 0;
gap = 0;
}
if (style['align-content'] === 'flex-end') {
crossBase += crossSpace * crossSign;
gap = 0;
}
if (style['align-content'] === 'center') {
crossBase += crossSpace * crossSign / 2;
gap = 0;
}
if (style['align-content'] === 'space-between') {
crossBase += 0;
gap = crossSpace / (flexLines.length - 1);
}
if (style['align-content'] === 'space-around') {
crossBase += crossSign * gap / 2;
gap = crossSpace / (flexLines.length);
}
if (style['align-content'] === 'stretch') {
crossBase += 0;
gap = 0;
} flexLines.forEach(flexLine => {
let lineCrossSize = style['align-content'] === 'stretch'
? flexLine.crossSpace + crossSpace / flexLines.length
: flexLine.crossSpace;
for (let i = 0; i < flexLine.length; i++) {
let itemStyle = getStyle(flexLine[i]);
let align = itemStyle['align-self'] || style['align-items']; if (itemStyle[crossSize] === null) {
itemStyle[crossSize] = align === 'stretch'
? lineCrossSize
: 0;
} if (align === 'flex-start') {
itemStyle[crossStart] = crossBase;
itemStyle[crossEnd] = itemStyle[crossStart] + crossSign * itemStyle[crossSize];
}
if (align === 'flex-end') {
itemStyle[crossStart] = crossBase + crossSign * lineCrossSize;
itemStyle[crossEnd] = itemStyle[crossEnd] - crossSign * itemStyle[crossSize];
}
if (align === 'center') {
itemStyle[crossStart] = crossBase + crossSign * lineCrossSize[crossSize] / 2;
itemStyle[crossEnd] = itemStyle[crossStart] + crossSign * itemStyle[crossSize];
}
if (align === 'stretch') {
itemStyle[crossStart] = crossBase;
itemStyle[crossEnd] = crossBase + crossSign * ((itemStyle[crossSize] !== null && itemStyle[crossSize] !== (void 0)) ? itemStyle[crossSize] : lineCrossSize);
itemStyle[crossSize] = crossSign * (itemStyle[crossEnd] - itemStyle[crossStart]);
}
}
crossBase += crossSign * (lineCrossSize + gap);
})
Now, we have a DOM tree with the position data!
Send an HTML with flexbox layout from the server
./server/server.js
...
response.end(
`<html maaa=a >
<head>
<style>
#container {
width:500px;
height:300px;
display:flex;
background-color:rgb(139,195,74);
}
#container #myid {
width:200px;
height:100px;
background-color:rgb(255,235,59);
}
#container .c1 {
flex:1;
background-color:rgb(56,142,60);
}
</style>
</head>
<body>
<div id="container">
<div id="myid"/>
<div class="c1" />
</div>
</body>
</html>`);
...
As you can see from the new DOM tree, the positions of the elements are already there. It’s ready to be rendered!
Step five: render
In the field of computer graphics, the term “render” specifically refers to the process of turning a model into a bitmap. Note that some frameworks, like React, call the “process from data to HTML code” as to render, which is different.
The bitmap here is a two-dimensional table built in memory, and the color corresponding to each pixel of a picture is saved.
Here we use the “images” library for painting in the viewport, it supports painting background-color, border, background-image, etc.
npm install images
Render one flex item
In the beginning, let’s try to render one flex item.
./client/render.js
const images = require("images");function render(viewport, element) {
if (element.style) {
let img = images(element.style.width, element.style.height);
// to simplify, only deal with background color
if (element.style["background-color"]) {
let color = element.style["background-color"] || "rgb(0, 0, 0)";
color.match(/rgb\((\d+),(\d+),(\d+)\)/);
img.fill(Number(RegExp.$1), Number(RegExp.$2), Number(RegExp.$3));
viewport.draw(
img,
element.style.left || 0,
element.style.top || 0
);
}
}
}module.exports = render;
./client/index.js
const net = require('net');
const images = require("images");
const parser = require('./parser');
const render = require("./render");...void async function () {
const request = new Request({...}); const response = await request.send();
const dom = parser.parseHTML(response.body); const viewport = images(800, 600);
render(viewport, dom.children[0].children[3].children[1].children[3]); // the element to render
viewport.save('viewport.jpg');
}()
Go check the viewport.jpg, we rendered the item out successfully!
Full rendering
This step is quite easy, calling the render() method of the children elements recursively can render the whole DOM tree.
Text rendering is difficult, it relying on the font library to turn the font into a picture. We ignore it in our mini-browser project.
./client/render.js
function render(viewport, element) {
if (element.style) {
...
} if (element.children) {
for (const child of element.children) {
render(viewport, child);
}
}
}
Run the index.js and check the picture again, you’ll find that the elements are rendered perfectly.
Let’s change some flex rules to see whether the browser can handle it, like this:
The answer is yes!
We have successfully developed a toy mini-browser that supports HTTP requests, HTML/CSS parsing, and flexbox layout.
After going through these steps, I hope you have a deeper understanding of how browsers works. If you want to view the complete code, please refer to this repository. The index.js in the client-side is separated into a few more files, but the basic structure remains.
src
├── client
│ ├── index.js
│ ├── Request.js
│ ├── HTMLparser.js
│ ├── ResponseParser.js
│ ├── ChunkedBodyParser.js
│ ├── layout.js
│ ├── render.js
│ ├── package.json
│ ├── package-lock.json
│ └── viewport.jpg
└── server
├── server.js
├── package.json
└── package-lock.json
