Teapot: Web Programming Made Easy


On the heels of my recently published Smalltalk tutorial, I wanted to write yet another web tutorial, this time highlighting a wonderful micro web framework in Pharo. (If you’ve not followed the original tutorial, I suggest you at least read Chapters 2, 3, and 4. Note that it’s not necessary to use the Raspberry Pi for this tutorial; any PC will do. Install Pharo from here.)

Teapot is a nice, simple, easy-to-use framework for creating web services and web applications. It’s much like Python’s Flask and Ruby’s Sinatra and Java/Kotlin’s Spark.

(Teapot is based on the Zinc HTTP Components framework for which Sven Van Caekenberghe has written an excellent tutorial.)

Our first Teapot application will be a fairly extensive login management front end. For this, we need four things:

  1. a password encryption mechanism (use Pierce Ng’s PasswordCrypt)
  2. a database store for login credentials (use MongoDB and VoyageMongo)
  3. a send email mechanism (use Zodiac’s #ZdcSecureSMTPClient)
  4. a method of generating UUID (use #UUIDGenerator)

To install Teapot, perform this in Playground:

Gofer it
smalltalkhubUser: 'zeroflag' project: 'Teapot';
configuration;
loadStable.

We will use PBKDF2 for password encryption. We will use FFI to call into a C library for this purpose because the C code is much faster. If your website needs to handle dozens of logins per minute, performance will be a real issue.

We will need the OpenSSL C libraries:

sudo apt-get install libssl-dev

The C library for PasswordCrypt has to be compiled. Use the supplied Makefile. (Depending on your development system, you may or may not be able to use the ‘-m32’ flag. If not, just remove the flag.) With the C files in the same folder, just execute:

make

Place the library file into the Pharo folder (or wherever the Pharo VM is).

To load PasswordCrypt into Pharo:

Metacello new 
baseline: 'PasswordCrypt';
repository: 'github://PierceNg/PasswordCrypt/src-st';
load

We will use the popular MongoDB for our database store. To setup MongoDB in Debian Linux,

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install mongodb-server
mongo # run mongo shell
sudo service mongodb start # start mongodb as a service
sudo service mongodb stop # stop the service

Voyage is an object persistence abstraction layer which you can use with Mongo. Install VoyageMongo from the Catalog Browser (click on the green check mark to install the stable version). Here is some introductory material about VoyageMongo.

To notify Voyage about Mongo, perform this in Playground:

|repo|
repo := VOMongoRepository
host: VOMongoRepository defaultHost
database: 'NCTDB'.
VORepository setRepository: repo.

Our database is called ‘NCTDB’ and the database model is represented by the #NCTUser class. We create the model thus:

Object subclass: #NCTUser
instanceVariableNames: 'name user pwdHash pwdSalt uuid
creationDate accessDate'
classVariableNames: ''
poolDictionaries: ''
category: 'NCT-Tutorial'
NCTUser class>>isVoyageRoot "a class-side method"
^ true
NCTUser class>>voyageCollectionName "a class-side method"
^ 'NCTUsers'

As well, we want to create all the “accessors” for the instance variables.

We want the following pieces of information stored in the database “collection” (consisting of “documents,” to use Mongo parlance):

  • name — the full name of the user (optional)
  • user — this is the email address of the user, guaranteed to be unique
  • pwdHash and pwdSalt — the encrypted password along with its associated salt
  • uuid — a UUID is a 128-bit number used to (almost) uniquely identify something or someone (in our case, the user)
  • creationDate — the date when the user registered; potentially useful for auditing purposes or account expiry
  • accessDate — the date when the user last logged in; potentially useful for determining how “stale” the account is

Creating a document for a new user is as simple as:

pwd := 'Amber2017'. "Amber Heard will be my girlfriend in 2017!"
salt := 'et6jm465sdf9b1sd'.
(NCTUser new)
name: 'Richard Eng';
user: 'horrido.hobbies@protonmail.com';
pwdHash: PasswordCrypt sha256Crypt: pwd withSalt: salt;
pwdSalt: salt;
uuid: UUID new hex asUppercase;
creationDate: DateAndTime today;
save.

To use #ZdcSecureSMTPClient, you must turn on ‘Allow less secure apps’ for the Gmail account that you’re using to send emails. This is used for email verification of user accounts.

Tea Horse Road

Teapot is based on the idea of routes. A route consists of three parts: 1) HTTP method; 2) URL pattern; 3) Action — it can be a block or a message send or an object. A list of routes basically comprises your web application.

initialize
Teapot stopAll. "reset everything"
Teapot on
GET: '/register' -> [ self registerPage: 0 name: ''
user: '' pwd: '' pwd2: '' ];
POST: '/register' -> [ :req | self verifyRegistration: req ];
GET: '/verify/<uuid>' -> [ :req | self verifyUUID: req ];

GET: '/login' -> [ self loginPage: 0 user: '' pwd: '' ];
POST: '/login' -> [ :req | self verifyLogin: req ];

before: '/welcome/*' -> [ :req |
req session attributeAt: #user
ifAbsent: [ req abort: (TeaResponse redirect
location: '/login') ] ];
GET: '/welcome/<name>' -> [ :req | self mainPage: req ];

GET: '/forgot' -> [ self forgotPage: '' ];
POST: '/forgot' -> [ :req | self handleForgot: req ];

before: '/profile/*' -> [ :req |
req session attributeAt: #user
ifAbsent: [ req abort: (TeaResponse redirect
location: '/login') ] ];
GET: '/profile' -> [ :req | self profilePage: req ];
POST: '/profile' -> [ :req | self handleProfile: req ];

GET: '/logout' -> [ :req | self logout: req ];
GET: '/books' -> [ :req | 'Check ',(req at: #title),' and ',
(req at: #limit) ]; "this route demonstrates how to pass
parameters in the URL, eg,
/books?title=The Expanse&limit=8"
start

For example, if you visit the login page (for a fictitious URL) in your web browser…

http://nct.gov/login "nct.gov is a fictitious domain;
normally, you will register your own domain
and configure your web app to use it"

…you will see a web form for logging in with your username and password. This activates the route:

GET: '/login' -> [ self loginPage: 0 user: '' pwd: '' ]

The method #loginPage:user:pwd: presents HTML code to render the web page. When you submit the form information to the web server, this activates the route…

POST: '/login' -> [ :req | self verifyLogin: req ]

…where #verifyLogin: processes the form information and, upon successful verification, you are taken to the main web page:

GET: '/welcome/<name>' -> [ :req | self mainPage: req ]

The method #mainPage: presents HTML code to render the Welcome page.

A similar process applies to the registration page (‘/register’) and “Forgot your password?” page (‘/forgot’) and user profile page (‘/profile’).

Some routes don’t present a web page, for example, ‘/logout’ which simply logs you out and redirects you to the login page.

The argument ‘req’ is the HTTP request associated with the HTTP method. It contains a ‘session’ which can be used to store “global” information. In our case, we will store ‘user’ (or email address) in order to determine when a user has logged in. We will also store ‘uuid’ because it’s fun!

The route #before: is a filter that is evaluated before the GET: request immediately following. The filter is used to ensure that a user is logged in before they can access the webpage.

What the user sees

The various webpages represent the public-facing view of our application. It consists of a “stylesheet” (which contains CSS instructions) and a whole bunch of HTML code. The following is the Login page:

loginPage: code user: user pwd: pwd
^ '<html> <head>',self stylesheet,'</head>
<body>
<h2>Login</h2>
<div>
<form method="POST">
Email:<br>', (self errCode: (code bitAnd:
self class ErrBadEmail)), '
<input type="text" name="user" value="',user,'"><br>
Password:<br>', (self errCode: (code bitAnd:
self class ErrBadPassword)), '
<input type="password" name="pwd" value="',pwd,'"><br><br>
<input type="submit" value="Submit">
</form>
<p><a href="/forgot">Forgot your password?</a></p>
<p><a href="/register">Sign up now!!</a></p>
</div>
</body>
</html>'

Here is the Registration page:

registerPage: code name: name user: user pwd: pwd pwd2: pwd2
^ '<html> <head>',self stylesheet,'</head>
<body>
<h2>Register</h2>
<div>
<form method="POST">
Fullname:<br>
<input type="text" name="name" value="',name,'"><br>
Email:<br>', (self errCode: (code bitAnd:
self class ErrBadEmail)), '
<input type="text" name="user" value="',user,'"><br>
Password:<br>', (self errCode: (code bitAnd:
self class ErrBadPassword)), '
<input type="password" name="pwd" value="',pwd,'"><br>
Password (confirm):<br>', (self errCode: (code bitAnd:
self class ErrNoPasswordMatch)), '
<input type="password" name="pwd2" value="',pwd2,'">
<br><br>
<input type="submit" value="Submit">
</form>
</div>
</body>
</html>'

The pages for Profile and “Forgot your password?” are similar. Regarding #bitAnd:, it’s a bitwise operator that treats numbers as sequences of binary digits. For each corresponding bit pair from the operands, an ‘and’ (or ‘&’) operator produces the following results:

  • 0 & 0 -> 0
  • 0 & 1 -> 0
  • 1 & 0 -> 0
  • 1 & 1 -> 1

There are similar operators for ‘or’ (or ‘|’), ‘xor’ (or ‘^’), and ‘not’ (or ‘~’).

Note, however, that Smalltalk numbers the bits from 1 to 16, not 0 to 15! (This is analogous to arrays where Smalltalk starts with element 1, not element 0.) By the way, in Smalltalk, integers are not limited to 32 nor 64 bits, so these bitwise operators can be used with very large numbers!

The variable ‘code’ is a convenient way to encode multiple error messages.

Here is the stylesheet:

stylesheet
^ '<style>
body {
background-image:
url(https://cdn-images-1.medium.com/max/2000/1*QVTC39_gW_wMXKwNxUvooA.jpeg);
background-size: 100%;
/*font-family: arial, helvetica, sans-serif;*/
text-align: center;
}
/* from https://www.w3schools.com/howto/howto_js_sidenav.asp */
body {
font-family: "Lato", sans-serif;
}
.sidenav {
height: 100%;
width: 0;
position: fixed;
z-index: 1;
top: 0;
left: 0;
background-color: #111;
overflow-x: hidden;
transition: 0.5s;
padding-top: 60px;
}
.sidenav a {
padding: 8px 8px 8px 32px;
text-decoration: none;
font-size: 25px;
color: #818181;
display: block;
transition: 0.3s;
}
.sidenav a:hover, .offcanvas a:focus{
color: #f1f1f1;
}
.sidenav .closebtn {
position: absolute;
top: 0;
right: 25px;
font-size: 36px;
margin-left: 50px;
}
@media screen and (max-height: 450px) {
.sidenav {padding-top: 15px;}
.sidenav a {font-size: 18px;}
}
</style>'

HTML and CSS are beyond the scope of this tutorial, but there are plenty of online learning resources for them.

Handling POSTs and special requests

The most important function of a web application is to handle or process HTTP requests beyond merely presenting webpages. Our application has several types of requests that need to be handled:

  • Login — the user has submitted username and password
  • Logout — the user wants to terminate their login session
  • Registration — a potential user has submitted username and password
  • Account verification — a potential user has clicked on a verification link sent to them via email
  • User profile update — the user wants to change the password
  • Recovering from a forgotten password — the user needs a temporary password sent to them via email

Here, for example, is the handler for user login:

verifyLogin: req
| code name user pwd doc tries |
user := req at: #user.
pwd := req at: #pwd.
code := 0.
(self validateEmail: user)
ifFalse: [ code := code + self class ErrBadEmail ].
(self validatePassword: pwd)
ifFalse: [ code := code + self class ErrBadPassword ].
code > 0 ifTrue: [ ^ self loginPage: code user: user pwd: pwd ].
doc := NCTUser selectOne: [ :each | each user = user ].
doc ifNil: [ ^ req abort: (TeaResponse redirect
location: '/register') ].
(PCPasswordCrypt sha256Crypt: pwd withSalt: doc pwdSalt) ~=
doc pwdHash ifTrue: [
tries := req session attributeAt: #tries
ifAbsentPut: [ tries := 0 ].
tries = 3 ifTrue: [ ^ self messagePage: 'Login'
msg: 'Exceeded limit. You''ve been locked out.' ].
tries := tries + 1.
req session attributeAt: #tries put: tries.
^ self messagePage: 'Login' msg: 'Wrong password.' ].
req session attributeAt: #user ifAbsentPut: user.
req session attributeAt: #uuid ifAbsentPut: doc uuid.
doc accessDate: DateAndTime today; save.
name := doc name.
^ TeaResponse redirect location: '/welcome/',
(name substrings = #() ifTrue: [ 'friend' ]
ifFalse: [ name ])

The basic pseudocode is:

Extract username and password from the HTTP request.
Validate username and password. If this fails, report the error(s)
back to the user.
Query the database for the user document.
Compare the password hash from the database to the hash of the
submitted password. If they don't match, report the error back
to the user. If this happens three times in succession, lock the
user out (presumably, a hacker is trying to breach security).
The passwords match, so keep track of login status by storing #user
and #uuid in the HTTP session.
Update the access date for the user in the database.
Redirect the user to the main page after successful login. If the
user didn't provide a full name, use the name "friend" instead.

The basic pseudocode for Registration is similar:

Extract username and password from the HTTP request.
Validate username and password. If this fails, report the error(s)
back to the potential user.
Query the database for the user document. If it exists, report to
the potential user that the user already exists.
We're ready to create a new user document, so generate a UUID and
encrypt the password. Create a new database document for the
user, storing the creation date, too.
Send an account verification email to the new user.

Here is the Registration handler:

verifyRegistration: req
| code name user pwd pwd2 uuid salt |
name := req at: #name.
user := req at: #user.
pwd := req at: #pwd.
pwd2 := req at: #pwd2.
code := 0.
(self validateEmail: user)
ifFalse: [ code := code + self class ErrBadEmail ].
(self validatePassword: pwd)
ifFalse: [ code := code + self class ErrBadPassword ].
pwd = pwd2
ifFalse: [ code := code + self class ErrNoPasswordMatch ].
code > 0 ifTrue: [ ^ self registerPage: code
name: name user: user pwd: pwd pwd2: pwd2 ].
(NCTUser selectOne: [ :each | each user = user ]) ifNotNil: [
^ req abort: (self messagePage: 'Register'
msg: 'User already exists.') ].
uuid := UUID new hex asUppercase.
salt := self generateSalt.
(NCTUser new)
name: name;
user: user;
pwdHash: (PCPasswordCrypt sha256Crypt: pwd withSalt: salt);
pwdSalt: salt;
uuid: uuid;
creationDate: DateAndTime today;
save.
self sendEmail: user subject: 'NCTDB Account Verification'
content: 'Please click on the following link to verify your email: http://nct.gov/verify/',uuid.
^ self messagePage: 'Register'
msg: 'Check your email for account verification.'

Utilities

We use regular expressions for validating passwords and email addresses. For example,

validatePassword: aPassword
(aPassword size >= 8) & "at least 8 characters"
(aPassword matchesRegex: '^.*[A-Z].*$') &
(aPassword matchesRegex: '^.*[a-z].*$') &
(aPassword matchesRegex: '^.*\d.*$')
ifFalse: [ ^ false ].
^ true

Here, we want to ensure that the password has at least 8 characters, has at least one uppercase character and at least one lowercase character and at least one numeric character.

We also have a method for generating a “salt”:

generateSalt
^ (String new: 16) collect: [ :each |
'0123456789abcdefghijklmnopqrstuvwxyz' atRandom ]

This allocates a new String object of size 16, and for each character in the string, we insert a character chosen at random from a string of digits and letters.

See NCTDB in action

After you’ve written the web application, you can run it to see what it looks like. From Playground, do this instruction:

NCTDB new

Then from your local web browser, enter this URL:

http://localhost:1701/login

You can make your web app available for your friends and family to play with by port-forwarding ‘localhost:1701’ in your LAN router (if your computer is behind a router). Just log into your router, find the port forwarding page, and add an HTTP entry (which includes your internal IP address, say, for example, 192.168.0.5, and a private port number of 1701).

Then, when your friends visit your router’s IP address ‘http://aaa.bbb.ccc.ddd/login’, they’ll see the NCTDB login page!

In a production setting, you’ll want a dedicated server machine for your web application and use something like Apache or Nginx with Linux, or IIS with Windows. This is beyond the scope of this tutorial.


As you can see, Teapot is a nice, simple way to write a web application. There really isn’t much to our tutorial app from the Pharo end (except you still have to learn HTML, CSS, a smidgen of JavaScript, and how to use MongoDB!).

Our Teapot application is a fairly common login management front end that can be used for just about any web application. Feel free to adapt it, modify it, expand on it for your own purposes.

You may want to take this opportunity to write an ‘admin’ module for it, a simple CRUD facility for browsing, adding, modifying, and removing users from the database. This function can be done securely over the Internet (make sure the administrator uses a very strong password).

Speaking of security, your Teapot application can be served over HTTPS by obtaining a SSL certificate and doing something like this (this is a Raspberry Pi Linux example):

initialize
| secureServer teapot |
Teapot stopAll.
secureServer := (ZnSecureServer on: 1443)
certificate: '/home/pi/server.pem';
logToTranscript;
start;
yourself.
teapot := Teapot configure: { #znServer -> secureServer }.
teapot
GET: '/register' -> [ self registerPage: 0 name: ''
user: '' pwd: '' pwd2: '' ];
POST: '/register' -> [ :req | self verifyRegistration: req ];
GET: '/verify/<uuid>' -> [ :req | self verifyUUID: req ];
" ... "
start

This code creates a secure server based on HTTPS and SSL. It then configures Teapot to use the secure server instead of the regular HTTP server. The optional #logToTranscript message lets you trace all server activity in the Transcript for diagnostic purposes. (Be sure that you change the verification link in the registration handler to use ‘https:’ instead of ‘http:’.)

I hope you found this tutorial useful.


Source code

This zip file contains the Pharo code for ‘FileIn’ with the System Browser. Unzip first. Then drag-and-drop the file into the Pharo window. FileIn the entire file.

Smalltalk