On The Road To Professional Web Development | How The Internet Works

Part I: The Internet

How It Works: The Short Answer

We use the words “internet” and “world wide web” (or just “web”) interchangeably sometimes, but there is a distinction. The internet is the hardware and network layer backbone that the web sits on top of. The web is a blend of software components that work together — including the IP packets that carry data to IP addresses (possibly obtained via DNS servers through a URL you typed into your browser), the TCP that keeps track of each packet to make sure they all arrived, and the HTTP and HTML data contained inside the TCP/IP packets. This is a very loose description. The main takeaway is that when we think “internet”, we should think hardware + networking, but when we think “web”, we should think software.

For a passive exposure to the inner workings of the internet, I strongly recommend checking out the Crash Course CS series (networking, the internet, and the web), followed by the Khan Academy CS series (each video here). For extra credit, check out How the internet Works in 5 minutes, and How does the Internet work?

Slightly longer answers include:

• This compact discussion on how the internet works
• This dated but insightful whitepaper from 2002
• And this surprisingly comprehensive discussion on Wikipedia

What Is HTTP?

If you understand the meaning of GET and POST, you probably don’t need to dig deeper for the purposes of this article. If not, you’re in luck: I recently posted an article covering HTTP, HTTPS, and what they mean to APIs.

Browsers And How They Work

We care how web browsers work because it informs how we can write and optimize our front end code. The main takeaway is that it’s important to understand what’s going on under the hood, but there are still many processes that are complex enough that rules-of-thumb won’t be good enough for professional web development. Dev tools are our friend when it comes to monitoring and optimizing site performance. We should keep an eye on the Time To First Byte (TTFB), Time To Interactive (TTI).

Multi-process architecture: The browser starts several processes (browser, render, GPU, utility). Each processes may have multiple threads (browser has UI, storage, network). They talk to other processes via IPC.

Navigation flow: The UI thread in the browser process accepts a URL as user input, asks the network thread to take care of the request, then takes the result from the network thread and hands it to the main thread of a new render process. Security measures and optimizations take place along the way, but this discussion covers the basics.

Rendering: The render process is responsible for everything that happens inside a tab. Its main job is to convert HTML/CSS/JS into a page the user can interact with. The main thread parses HTML to construct a DOM tree, then a layout tree, then a layer tree, then the paint order for each element. Lastly, it uses a technique called composting to separate parts of a page into separate layers, rasterizing (painting) each separately, then hands this information to a compositor thread to put it all together. Understanding the bottlenecks and optimizations involved may be a web developer’s best friend — since (on the front end) it’s our goal to make pixel perfect sites perform well.

• For more information, start with this post outlining essential web browser concepts
• Get a look inside Chrome browser with this excellent 4 part introduction
• Lastly, check out this in-depth resource that considers multiple browsers

What Domain Names Are

A domain name is an alias for an IP address. There are tables that keep track of the mappings between domain names and IP addresses — that’s the topic of DNS. Domain names can contain a gTLD (generic top level domain) such as .com or .org, or they can contain a ccTLD (country code top level domain) such as .us or .jp. The second level domain would be google in google.com. We can buy domain names from various companies, but they actually are just saving us the trouble of registering it ourselves with ICANN (the organization keeping track of all domain names).

• More than we’ll ever need to know about domain names is contained in ICANNs 18 page guide here

DNS And How It Works

It may come as no surprise but buying a domain name doesn’t mean we have a website. The website needs to live on a server somewhere. That server has an IP address. When we bought the domain name, the company that sold it to us (NameCheap, for the upcoming example) always gives us a way to select the IP address of our server. Say someone wants to go to oursite.com. Their browser sends a request to one of only 13 “DNS root nameservers”. The DNS root nameserver forwards the request to one of NameCheap’s DNS servers. NameCheap has access to the information we gave them — and that’s how they navigate to our site.

• There are practical considerations we should learn such as setting up domain name records. Read the comprehensive guide here

What Hosting Is

Websites need to live on hardware (a physical network-connected place where site data is stored). We can own this infrastructure or rent it from someone else — whoever owns it is the host. Whoever programs and maintains the site is called the webmaster. Site reliability, uptime, and security are primary considerations for hosting providers. Webmasters are interested in host-provided support for databases, operating systems, and site management interfaces. There isn’t much to this topic in the abstract sense, but like all technical professionals, we will need to dive into intricate practical details. There are tons of sites willing to sell you hosting plans packed with additional features — this doesn’t fundamentally change what “hosting” is, but if we’re paying for a service, we’ll just need to be careful to read up on exactly what they say they’re offering.

• Read this list of 10 types of hosting providers

Part II: Learning Internet Technologies

The concepts above will sink in with some hands-on experience, but there are many tools we’ll need to rely on. It will require some digging.

1. To understand how packets move from place to place, I strongly recommend downloading and understanding the traceroute tool (tracert on Windows)
2. There are all kinds of goodies on this site — use each tool and try to really think deeply about how they tie into internet-related concepts we’ve mentioned
3. If you’re on desktop using Firefox or Chrome, navigate to the Network Inspector (or similar) tool. f12 or right-click inspect source should get you close. This provides a way for you to inspect all of the HTTP requests associated with the site you’re on. For several requests, check out the headers, read the content, and generally try to follow what it’s doing
4. While you’re at it, try to understand the site performance monitoring tools builtin to your browser. Use it on several sites — comparing the results between a professional website and your friend’s hastily made WordPress blog will be especially insightful
5. If you have the resources, buy a domain name. They can be as cheap as a dollar or two per year. Open a free account on GitHub and build a website using GitHub pages (they offer stock sites — you don’t need to learn how to code anything yourself for the purposes of this exercise). The key step is to learn how to connect your domain name to your GitHub Pages site. This will expose you to domain names, DNS, records, and (in a limited sense) website hosting

Part III: Interview Questions

• compare modems, switches and routers
• discuss address resolution
• discuss application protocols
• discuss authoritative nameservers
• discuss caching in browsers
• discuss client-server models
• discuss cookies in browsers
• discuss deep web
• discuss DHCP
• discuss the DOM
• discuss IP addresses
• discuss ISP responsibilities
• discuss network service providers
• discuss protocol stack used by the world wide web
• discuss root nameservers
• discuss the WWW prefix
• discuss TLD nameservers
• discuss traceroute

Closing Thoughts

I hope you’ve enjoyed this post as much as I enjoyed writing it. If you have thoughts you’d like to share, your editorial suggestions are always welcome. This post is part of a series on becoming a professional web developer. If you’d like to see more content like this, please click here to navigate to the table of contents.