A Mock SMTP Server for remote mail delivery testing

Most of you probably know this already, the Simple Mail Transfer Protocol (SMTP) is a communication protocol for electronic mail transmission. It is then essential to make sure that remote mail delivery using SMTP protocol is well covered when developing and maintaining a mail server.

Until recently, in the Apache James project, we were relying on an external solution, a docker image of rest-smtp-sink, which is a fake SMTP and web server in Javascript, similar to FakeSMTP, for testing our back-end mail server regarding the remote delivery of emails.

This solution was enough at the beginning but for a while now we noticed that our remote delivery was lightly tested and numerous corner cases were missing. Some error codes in special cases scenarios with remote mail delivery and MX records were not being handled well, …

We tried to find replacements for it at first, by looking at other existing mock SMTP server projects. But we couldn’t really find one that was fitting all the things we needed from it.

Thus we finally decided to start implementing our own mock SMTP server, for the needs of our project. This post is about this journey.

What should our mock SMTP server be able to do?

The mock SMTP server should be able to cover different things at once:

• Can return error responses.
• Can return error responses but actually still success to deliver the mails (450 — “server retry and will be success”).
• Adding behaviors (mocks) to be able to simulate the case scenarios that we need.
• Act like a simple SMTP server if there is no mock setup => when mails come, save them into the memory.
• Load the JSON behaviors via HTTP to allow remote delivery setup.
• Able to rely on MX resolution.

Regarding the behaviors, they should answer the following requirements :

• A behavior can be applied to all given commands or a subset of them (condition).
• A number of answer can be specified, after X answers the behavior will no longer be applied.

Let’s add as well that we would need to be able to dockerize it to replace the current one and run it with our integration tests.

So how did we do that you might ask? Well let’s continue our journey by looking a bit more closely into the implementation of our mock SMTP server.

Mock SMTP server implementation

POJO models

We needed to modelize some POJO (Plain Old Java Object) models to be used by the mock SMTP server and its HTTP routes.

First, the mock behavior object that we called “MockSMTPBehavior”. We need the behavior to be able to simulate some type of responses depending of the commands and some matching criteria with the requests. The object at the end would look like this for example :

{
     "command": "MAIL FROM",
     "condition": {
       "operator": "contains",
       "matchingValue": "frodo"
     },
     "response": {
       "code": 250,
       "message": "OK"
     },
     "numberOfAnswer": 7
}

Let’s unwrap this and explain the fields one by one :

• command: it defines the SMTP functions requested by the client towards the server. Here we want to define a behavior for the MAIL FROM command.
• condition: the condition the request needs to meet on the above command to get the defined response. Here the condition will be matched if the mail address contains frodo.
• response: the response delivered to the client for a specific command if the condition is met. Here we return a successful 250 with an Ok message.
• numberOfAnswer: the number of times this behavior will be executed when matched before expiring. Note that you can set this value empty which means that the behavior will never expire !

Then, we would need an object for the mails being stored on the mock SMTP server :

{
  "envelope": {
    "from": "from@domain.tld",
    "recipients":["recipient1@domain.tld","recipient2@domain.tld"]
  },
  "message": "mail content here"
}

Here the fields are pretty straightforward :

• envelope: contains two sub-fields that are from (the sender address) and recipients (the addresses at the receiving end).
• message: the message content of the mail.

With this, we had finally a good base of objects to continue doing our implementation of the mock SMTP server.

The base SMTP server and the message handler

As the base of our mock SMTP server, we decided to use the library SubEtha SMTP, which is a Java library allowing your application to receive SMTP mails with a simple, easy-to-understand API.

So we are using the “SMTPServer” object as our base server for our “MockSMTPServer”, to which we bind a “MessageHandler”, that we implemented as the “MockMessageHandler”.

We overrided the few functions we needed to deal with when a client is communicating with the server, as :

• from: answering to the MAIL_FROM command, which defines the sender.
• recipient: answering to the RCPT_TO command, which adds a recipient to the mail.
• data: answering to the DATA command, which defines the content of the mail.
• done: makes sure that the commands above were called, so that we can build a complete mail and store it server side.

Of course we also first check in those functions if we have some behaviors matching. If it’s the case, we force the server to respond what has been defined by the behavior.

HTTP routes implementation

We have mainly implemented two main routes :

• /smtpBehaviors: the route to manage the behaviors defined above, where we can add, delete and get the list of behaviors. We can define them like this :

- PUT /smtpBehaviors
{
     "command": "MAIL FROM",
     "condition": {
       "operator": "contains",
       "matchingValue": "frodo"
     },
     "response": {
       "code": 250,
       "message": "OK"
     },
     "numberOfAnswer": 7
}
-> to set a new behavior

- GET /smtpBehaviors
-> to get the list of behaviors

- DELETE /smtpBehaviors
-> to clear all behaviors

• /smtpMails: the route to manage the mails stored in the mock SMTP server. We can get the list of all mails and clear all mails :

- GET /smtpMails
-> to get the list of mails stored in the server
- DELETE /smtpMails
-> to clear all mails

With this in place, we seem to have a minimal functional working mock SMTP server. We could finally go the final step.

Dockerization of the server

Well now to be able to really use it in our tests and make sure that our mail server is strong at covering corner cases related to SMTP remote mail delivery, we needed to have a container running it.

To dockerize it accordingly, we decided to investigate about Jib. Jib helps you build optimized Docker and OCI images for your Java applications without a Docker daemon — and without deep mastery of Docker best-practices. It is available as plugins for Maven and Gradle and as a Java library.

And truth is: it works like a charm! You just need to define a few things in your pom.xml and then the build is integrated into your maven build phase cycle.

For example here we decided to build the image into a tarball, that we can then later easily load into Docker and run.

Conclusion

Well this first implementation of our own mock SMTP server looks promising to be honest. It was a good and interesting experience to see a bit closer the basic mechanics of a SMTP server and implementing a mock of it on our own.

At the end, we did write some integration tests that we were missing with SMTP and it looks promising, but we still miss some cases, and need to migrate from the old SMTP server solution to the new one regarding the existing tests.

This means there is probably more work to come on the mock SMTP server as well, as you always have the risk to encounter issues while implementing new tests.

But we are confident! And who knows, maybe when it’s mature and stable enough we might just move the mock SMTP server out of James and make it standalone, so that other projects could use it and contribute to it perhaps… :)