End to End testing for Telegram bot
Speed it up with a simulator for Telegraf module.
I have been developing a Telegram bot that helps with points-based incentives management system to develop positive habits at home.
The exercise also serves as a simulation of a typical journey and challenges of software development at a smaller and relatable scale.
The prototype has been developed rapidly (Refer to [1]) but we will need to clear technical debts before adding more features or people to work on it.
One of the first few activities is to add tests to the software.
Why are we adding tests at this point of time and not earlier?
- We were experimenting with some ideas and time to market to validate product fit is important.
- Now that we have a prototype for testing, we could test if the idea is useful.
- With sufficient confidence that the code and features we develop are mostly to stay and not thrown away we could move on to extend our product.
- While requests for new features are coming in from testing, we have a short respite to tackle the most pressing technical debt.
- Tests allow us to safely make further changes without impacting existing features.
- Tests also document the design, features and behaviours of our software and make it readable by other developers.
For a start, we would want to first introduce some form of end to end testing instead of implementing unit tests on the existing code.
This allows us to continue to release features, bugfixes or refactored code to deployment with lower risk on disruption to our users’ experience.
Progressively, we will add unit tests to capture the design and behaviour of the code to enable continuous refactor of our code.
The naive approach to setup Telegram Bot end to end testing is to:
- Setup a test bot account and infrastructure.
- Setup a user account. This requires a mobile number.
- Deploy the current build to the infrastructure, seed the data.
- Launch a browser web client for Telegram and login with the user account.
- Run browser automation to perform the various interactions supported.
The architecture will look like the following.
However, this will run into the following issues.
- Running cost of maintaining a mobile number.
- Multi Factor Authentication (MFA) required for browser web client login will make browser automation complicated.
- Slow run time using browser automation.
- Only one instance could be ran at one time.
Clearly this will not be a scalable solution.
Alternative approach is to consider building a simulator for the interactions between the user and our chat bot.
The chat bot and server interactions are not based on Graphical User Interfaces (GUI) but rather backend Application Programming Interfaces (APIs).
Hence, it will be highly effective if we could mock or simulate the API interactions between our Telegram Bot and the Telegram server.
The deviation between simulated test from the production environment only occurs with changes to interface or behaviours of existing API servers.
Typically, the APIs service providers follows the Open Close Principle in software engineering and design to prevent breaking changes for their existing users.
Hence the risk of defects that cannot be caught from our end to end environment is low and conversely our confidence in regression testing is higher.
A simulator will also be able to run at faster speed and the isolation offer the possibility of parallel runs to speed up.
The diagram below illustrates the difference for earlier approach.
Done with concepts, we will now move on to implementation.
As we are using the Telegraf module, we could possibly introduce some changes or code to intercept the API interactions between our Telegram Bot code and the Telegram server at this module.
To be able to do this, we need two important capabilities:
- Ability to monkey patch the Telegraf library.
- Ability to log API inbound and outbound requests and responses.
The formal will determine the viability of this approach.
Without ability to monkey patch dynamically, we are limited to either implementing a man-in-middle proxy or fork the library to provide interception interface.
The latter is about having the utility to help us understand the Telegram protocol and payload required in the API exchanges.
This will speed up troubleshooting and development of our end to end test cases.
We are fortunate that Telegraf module offers ability to add middleware to its Express server webhook service.
There is a telegraf-throttler module that takes advantage of this feature (Refer to [2]).
Along the way, we have found that there is a gap when our bot initiates an outbound message instead of just responding to the user.
This is because the library uses a different dynamic copy of the same function to make that API call but we could repeat the same approach to address it.
The code excerpt below illustrates the entry points where we will install interceptors to shim out the API calls or replies to the server.
import * as functions from "firebase-functions";
import * as admin from "firebase-admin";
import express from "express";
import * as bodyParser from "body-parser";
import morganBody from "morgan-body";
import "dotenv/config";
import {Telegraf, Context} from "telegraf";
...
import {middlewareInterceptor, outgoingInterceptor}
from "./interceptor";
if (admin.apps.length === 0) {
admin.initializeApp();
}
const firestore = admin.firestore();
const app = express();
app.use(bodyParser.json());
app.use("/telegram", bodyParser.json());
morganBody(
app,
{
maxBodyLength: 30000,
noColors: true,
prettify: false,
includeNewLine: false,
logRequestBody: true,
logResponseBody: true,
});
let telegramStatus = "Bot is not loaded";
...
if (!process.env.YOURTOKEN) {
functions.logger.info("YOURTOKEN is not defined.", {structuredData: true});
telegramStatus = "YOURTOKEN is not defined.";
} else {
const bot = new Telegraf(process.env.YOURTOKEN, {
telegram: {webhookReply: true},
});
...
const oldCallApi = bot.telegram.callApi.bind(bot.telegram);
const newCallApi = outgoingInterceptor(oldCallApi);
bot.telegram.callApi = newCallApi.bind(bot.telegram);
const middleware = middlewareInterceptor();
bot.use(middleware);
...
app.use("/telegram", bot.webhookCallback("/",
{secretToken: process.env.API_TOKEN}));
telegramStatus = "Bot is loaded.";
}
...
export default app;The interceptors are 2 functions, middlewareInterceptor and outgoingInterceptor that we have imported from a separate module, “interceptor.ts”.
The reasons to keep the interceptor code on a separate module are hiding away the complexity of interception and enabling these functions to be swapped out by mocks during testing and simulation.
Additionally, we will have also added body-parser and morgan-body to log incoming API call and its payload.
These are used to capture APIs that are sent from the Telegram server to our bot.
The captured API calls are initiated by our user sending message to our bot and helps us formulate the required payload to simulate the interactions.
The detailed interceptor functions from “interceptor.ts” are listed below.
import * as functions from "firebase-functions";
import type {Context, Middleware} from "telegraf";
import {Opts, Telegram}
from "telegraf/typings/core/types/typegram";
import ApiClient from "telegraf/typings/core/network/client";
export function middlewareInterceptor(): Middleware<Context> {
const middleware: Middleware<Context> = async (ctx, next) => {
const oldCallApi = ctx.telegram.callApi.bind(ctx.telegram);
const newCallApi: typeof ctx.telegram.callApi =
async function newCallApi(
this: typeof ctx.telegram, method, payload, {signal} = {}) {
functions.logger.info(
"ctx-reply-interceptor-request",
method, payload, signal, {structuredData: true});
const response = await oldCallApi(method, payload, {signal});
functions.logger.info(
"ctx-reply-interceptor-response",
response, {structuredData: true});
return response;
};
ctx.telegram.callApi = newCallApi.bind(ctx.telegram);
return next();
};
return middleware;
}
export function outgoingInterceptor(oldCallApi: <M extends keyof Telegram>(
method: M,
payload: Opts<M>,
{signal}: ApiClient.CallApiOptions) =>
Promise<ReturnType<Telegram[M]>>):<M extends keyof Telegram>(
method: M,
payload: Opts<M>,
{signal}: ApiClient.CallApiOptions) => Promise<ReturnType<Telegram[M]>> {
async function newCallApi<M extends keyof Telegram>(
this: Telegram,
method: M,
payload: Opts<M>,
{signal} : ApiClient.CallApiOptions = {}):
// eslint-disable-next-line @typescript-eslint/no-explicit-any
Promise<ReturnType<(...args: any) => any>> {
functions.logger.info(
"bot-sendmessage-interceptor-request",
method, payload, signal, {structuredData: true});
const response = await oldCallApi(
method, payload, {signal}) as ReturnType<Telegram[M]>;
functions.logger.info(
"bot-sendmessage-interceptor-response",
response, {structuredData: true});
return response;
}
return newCallApi;
}
module.exports = {
middlewareInterceptor: middlewareInterceptor,
outgoingInterceptor: outgoingInterceptor,
};They preserve a copy of the original code that makes the actual API call to the Telegram server. Instead of calling the Telegram server immediately, the interceptor will log a copy of the payload content. After that, it invokes the original code to resume the API call. Hence, it behaves as a transparent middleman who spies on the exchange during operation.
Now that we have settled the fundamentals for simulation, we will need to have a test database storage too.
We could do so by setting up our Firestore and Cloud Function emulator (Refer to [3]).
my-project$ firebase init
The Cloud Function emulator allows us to perform exploratory tests on our code without having to deploy them while the Firestore emulator provides a backing storage for running tests.
Once done, use the following command to start the emulators in one of your terminal.
my-project$ firebase emulators:start
Add a “.env.local” file with the address of your firestore emulator and fake tokens.
Remember to add this file to your .gitignore.
This is to be needed when we start running our end to end tests.
YOURTOKEN=ABC
API_TOKEN=DEF
FIRESTORE_EMULATOR_HOST=localhost:8080Next, we will add Jest and Supertest modules (Refer to [4]) to enable unit testing and integration testing.
npm i --save-dev typescript supertest nodemon jest ts-jest ts-node @types/jest @types/supertest @types/expressWe would add “jest.config.js” to the functions root folder with the following content.
/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testPathIgnorePatterns: ["/node_modules/", "/lib/", "/__tests__/e2e"],
};Because we would like to use Jest for both unit test and end-to-end test suite, I have added another config file, “jest.e2e.config.js” to separate these two categories of tests.
/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
testPathIgnorePatterns: ["/node_modules/", "/lib/"],
testRegex: "/__tests__/e2e/.*\\.(test|spec)\\.[jt]sx?$",
};Now we update the following for “tsconfig.json”.
{
"exclude": ["coverage/**/*", "./lib", "jest.config.js", "jest.e2e.config.js"],
"compilerOptions": {
"esModuleInterop": true,
"module": "commonjs",
"noImplicitReturns": true,
"noUnusedLocals": true,
"outDir": "lib",
"sourceMap": true,
"strict": true,
"target": "es2017",
"types": [
"jest",
]
},
"compileOnSave": true,
"include": [
"src",
"__tests__/**/*",
"jest.config.js",
"jest.e2e.config.js",
]
}And update the following for “tsconfig.build.json” to not transpile test code during build.
{
"extends": "./tsconfig.json",
"exclude": ["coverage/**/*", "./lib", "jest.config.js", "jest.e2e.config.js", "__tests__/**/*",],
}The “tsconfig.build.json” will be used when we are building and not running tests.
And then we update our “package.json” file to add and update the various script actions.
{
"name": "functions",
"scripts": {
"lint": "eslint --ext .js,.ts .",
"lint-fix": "eslint --fix --ext .js,.ts .",
"build": "tsc --project tsconfig.build.json",
"build:watch": "tsc --project tsconfig.build.json --watch",
"test": "jest --coverage --testIgnorePathPattern=/__tests__/e2e",
"test:e2e": "jest --config=jest.e2e.config.js --coverage --detectOpenHandles",
...
},
...
}We would also need to update the “.eslintrc.js” file to stop running linters over test scripts during build step.
module.exports = {
root: true,
env: {
es6: true,
node: true,
},
extends: [
"eslint:recommended",
"plugin:import/errors",
"plugin:import/warnings",
"plugin:import/typescript",
"google",
"plugin:@typescript-eslint/recommended",
],
parser: "@typescript-eslint/parser",
parserOptions: {
project: ["tsconfig.json", "tsconfig.build.json"],
tsconfigRootDir: __dirname,
sourceType: "module",
},
ignorePatterns: [
"/lib/**/*", // Ignore built files.
"__tests__/**/*",
"coverage/**/*",
"jest.config.js",
"jest.e2e.config.js",
".eslintrc.js",
],
plugins: [
"@typescript-eslint",
"import",
],
rules: {
"quotes": ["error", "double"],
"import/no-unresolved": 0,
"indent": ["error", 2],
},
};Once done, we will implement our tests with Jest.
The interceptorMock module is a mocked variant of the interceptor module described earlier.
It does not forward the APIs call out to the Telegram server.
Instead, it will invoke a Jest mock function to capture the payload used.
Our code will now be able to recall these from the mock function to make assertions on the expected messages from the bot.
Helper functions for testing are written in testHelper module.
This hides away repetitive implementation specifics like data setup, making API post requests, payload setup and assertions.
By doing that, it makes our test code clear on the interactions and expected behaviours between our user and the Telegram Bot.
Now we are ready to flesh out the rest of the end to end tests for our bot and pare down some technical debt.
For the baseline of the source code refer to [5].
References:
[1] https://medium.com/@kaikok/telegram-bot-with-cloud-function-and-firestore-14cc2ec0bfc6
[2] https://github.com/telegraf/telegraf/discussions/1267
[3] https://firebase.google.com/docs/emulator-suite
