Integrating Cartesi dApps with XMTP: Bridging decentralised computation and secured communication.
Co-Author: Shaheen Ahmed
In this rapidly exploding era of interoperability and modular narrative, we’ve experienced increasing demands for integration between protocols supporting the modular narrative. We’ve also begun to see protocols built around the idea of modular architecture and interconnectivity. Execution layers, data availability layers, communication layers are becoming increasingly popular and we can see crazy projects and ideas around, leveraging the interconnection of these different modular components to build really exciting things.
In this article, we’ll be looking at one of such integrations between Cartesi an app-specific rollup and XMTP, a secure messaging layer for web3. By integrating dApps on Cartesi with XMTP, developers can create applications that not only leverage Cartesi’s off-chain computation capabilities but also incorporate secure, decentralised messaging. This combination opens up new possibilities for dApps on cartesi to implement more functionalities and also improve the user experience on these dApps. We will explore the process of integrating dApps on Cartesi with XMTP, highlighting the benefits and use cases of such integration. Whether you’re building a complex dApp that requires heavy computation or a messaging platform with robust privacy guarantees, this integration can provide a powerful foundation for your Cartesi project.
To keep this article as simple as possible we assume that you understand the basics of building on cartesi and also the importance, capabilities and limitations of Notices on Cartesi. We’ll be focusing on the components necessary for integrating Cartesi and XMTP and not on building a Cartesi dApp.
Architecture:
Integrating your Cartesi dApp with XMTP requires 3 different components, some of which you’ll need to write from scratch yourself while others you just integrate into your application through SDK. These components are;
- The Cartesi dApp backend: This is any dApp written by the programmer in any language supported by Cartesi. The logic and execution in this dApp do not affect the integration in any way though for this particular example, the backend needs to be able to send properly structured notice to the GraphQL server which will further be relayed to the XMTP network.
- The Relayer Server: As you already know, dApps on Cartesi are sandboxed and therefore unable to interact with external servers or protocols. The XMTP protocol, on the other end, is a network of its own, dedicated to providing a secure protocol for messaging; therefore, there is a need for an external component which would be able to read data from dApps on Cartesi in real time and then relay these data to the expected destination on the XMTP network. This component plays a vital role of being the bridge between dApps on Cartesi and the XMTP network. It’s able to listen actively for requests from the Cartesi dApp and then forward these messages promptly to the destination address on XMTP.
It’s important to note that while this component is crucial for this integration, it also introduces a degree of centralization to the project as this server is fully owned and controlled by the developer. We’re open to contributions and suggestions that help in decentralising this component. - The XMTP Network: This layer in the architecture is outside the control of the developer. This is the messaging protocol that handles and displays messages between different users. The relayer server interacts with this layer to relay messages from the Cartesi dApp, which the target user can pick up on the XMTP network.
At the end of the day, we have an architecture similar to what’s shown below, where a user interacts with a Cartesi dApp and if necessary, a specially structured Notice is sent to the GraphQL server, which is further picked up by the relayer, destructured, and then relayed to the destination address on XMTP.
Deep Dive into the Relayer Server:
At this point, it’s expected that you should have an understanding of the structure and functionalities of the relayer server. In this section, we’ll be taking things a little further by explaining in more detail the structure and components of the relayer server. For the sake of this integration, we developed a simple MVP for the relayer server. This should be seen as an MVP and not a fixed structure. We highly encourage developers to further modify this code to implement more complex functionalities and also better tailor the relayer to serve their dApp.
The relayer is divided into two (2) sections: the GraphQl client and the XMTP client. The GraphQl client utilises Apollo SDK to listen on the Cartesi dApp GraphQL server for notices meant to be relayed to a target user on XMTP. On finding such a notice, the GraphQL client destructures this notice, then sends it to the XMTP-Client section. This section utilises the Ethers Js library to control a private key belonging to a wallet address that’s registered on XMTP and will serve as the origin of all messages relayed from your Cartesi dApp to target users on XMTP. In summary, the relayer listens for a new notice from the Cartesi dApp, destructures this notice, and then uses its private key to relay the message via the XMTP network. The target user basically receives a message on XMTP from the relayer address containing the information in the notice (message) that was sent from the Cartesi dApp.
Sample Relayer Code:
Since we’ve laid out a good foundation on the importance, components and function of the relayer, we’ll be solidifying that knowledge with a sample relayer codebase detailing the different sections and their functions. We believe this would go a long way to serve as a template that you could directly use or better still build on to achieve the same functionality of integrating messaging via XMTP in your dApp.
Project Structure:
In this example, our relayer server is written in Node JS and leverages other libraries and SDKs like Ethers JS and Apollo SDK for additional functionalities, so it’s important that you have Node properly installed before attempting to code along. For this project, we’re using a simplified version of the MVC model to structure the repo like this:
We have a parent folder called RELAYER inside which we have the ‘graphqlClient’ and the ‘server.js’ files in a folder called ‘src’. Just beside the src folder, we also have a controllers folder, which would house the ‘graphql.controller’ and the ‘xmtp.controller’ files. In the same directory level as the src, we also have a .env file whose contents we would be explaining later in this article.
Project Dependencies:
For the scope of this project, we would be utilising a very streamlined set of dependencies selected to simply meet the requirements of integrating these two protocols. In your case, you might want to add more dependencies as you modify this code base further.
We’ll be using the below set of dependencies for now;
"@apollo/client": "^3.11.4",
"@xmtp/xmtp-js": "^12.1.0",
"cors": "^2.8.5",
"dotenv": "^16.4.5",
"ethers": "^6.13.2",
"express": "^4.19.2",
"graphql": "^16.9.0",
"graphql-tag": "^2.12.6",
"viem": "^2.19.6",- .env file:
To make it easier to quickly modify some important variables that are used in the server and also to protect some private data, we’ve pushed a couple of variables to the .env file present in the repo:
PRIVATE_KEY=<private key for an evm wallet>
PRIVATE_KEY2=<private key for another evm wallet>
URI='http://localhost:8080/graphql'
POOL_INTERVAL=6000The above structure is what the out .env file should look like. `PRIVATE_KEY` represents the private key for an Ethereum wallet, which the server would use to relay messages to target addresses on XMTP. The `PRIVATE_KEY2` variable is a substitute target address; messages that are meant to be sent to users who unfortunately are not registered on XMTP are relayed to this address. This way, these messages are not lost but sent to a secured point from which they can be picked up and handled by the developer in any ways necessary so as to get these notifications to the target users. Next is the `URI` This is the URI of our cartesi dApp graphql server, If you’re running on localhost, you should have the same URI as in the section above but once you move to testnet or mainnet, you should change the URI to that of your testnet or mainnet application. Finally, we have the `POOL_INTERVAL`; this sets the interval in milliseconds between successive calls to the GraphQL URI, a lower amount represents faster pooling intervals, while a larger number represents a much slower polling interval.
- Server.js file:
This is a simple express server running on PORT 3000. It also imports and calls the `fetchAndSendLatestNotice` function, which contains the main logic and functionalities of the server.
const express = require('express');
const cors = require('cors');
const app = express();
const [fetchAndSendLatestNotice] = require('./graphqlClient.js');
const PORT = 3000;
app.use(cors());
app.use(express.json());
app.get('/test', function (req, res) {
res.send('GraphQL listener is running. Check the console for notices.');
})
app.listen(PORT, function () {
console.log(`server listening on port ${PORT}`);
});
fetchAndSendLatestNotice();- graphqlClient.js file:
This contains the necessary Apollo sdk implementations and functionalities to poll notices from the graphQl url that was included in the .env file. Its sole purpose is to poll notices from the GraphQl server, then send these notices to be filtered and sorted out based on implementations in the ‘graphql.controller ’file.
const { ApolloClient, InMemoryCache, ApolloProvider, gql } = require('@apollo/client');
const filterNotices = require('./controllers/graphql.controller');
require('dotenv').config();
/// @variable This is the url of a cartesi dApp subgraph as contained in the .env file.
const URI = process.env.URI;
/// @variable This is the interval in milliseconds at which the fetchAndSendLatestNotice function should be called.
const POOL_INTERVAL = process.env.POOL_INTERVAL;
/// @title fetchAndSendLatestNotice
/// @notice This function is a continues loop that periodically calls the graphql server based on the time interval specified in the .env file
async function fetchAndSendLatestNotice() {
/// @notice Apollo SDK setup for an interaction client, it consimes the subgraph URI specified in the .env file
const client = new ApolloClient({
uri: URI,
cache: new InMemoryCache(),
fetch,
});
/// @notice Simplified query stucture to fetch all the notices from a Cartesi dApp graphql server
const NOTICES_QUERY = gql`
query notices {
notices {
edges {
node {
index
input {
index
}
payload
}
}
}
}
`;
/// @notice Simple Interval loop that utilizes Apollo SDK to periodically fetch all the notices from a Cartesi dApp
/// @notice Collected Notices are passed to the filterNotice function for further processing.
setInterval(() => {
client
.query({
query: NOTICES_QUERY,
fetchPolicy: 'network-only',
})
.then(response => {
const edges = response.data.notices.edges;
filterNotices(edges);
})
.catch(error => {
console.error('Polling error:', error);
});
}, POOL_INTERVAL);
}
module.exports = [fetchAndSendLatestNotice];- graphql.controller.js file:
This section of the code plays a vital role in the functionality of this project. It tracks the index of the last notice that was parsed so as to prevent excessive and repeated relaying of old notices. It also filters the notices picked up from the GraphQL server in a way that previously analysed notices are skipped while more recent notices are destructured and then compared against an expected object structure.
{
TxType: 'InAppMessage',
Origin: 'sender address',
Destination: 'receiver address',
Payload: 'message to be sent',
}Notices that meet this structure are then sent to the `xmtp.controller.js` contained functions.
const ethers = require('ethers');
const viem = require('viem');
const sendNotice = require('./xmtp.controller.js');
/// @variable This keeps track of the index of the last notice that was sent out.
let lastAnalysedNotice = 0;
/// @title filterNotices
/// @notice This function extracts out notifications that have already been previously handled by the server based on the stored index (lastAnalysedNotice).
/// @notice Notices that have not been previously handled by the server are sent to the decodeNotice function
/// @param notices An array of all Notices from a subgraph server
function filterNotices(notices) {
if (notices.length > 0) {
let slicedNotices = notices.slice(lastAnalysedNotice);
lastAnalysedNotice = notices.length;
for (let notice of slicedNotices) {
decodeNotice(notice);
}
} else {
console.log('No new notices found.');
}
}
/// @title decodeNotice
/// @notice This function converts notices from hex to object notations, destructures them, then checks if they're ment to be relayed via XMTP network
/// @notice notices containing payload to be relayed via XMTP network are sent to the sendNotices function
/// @param notice A single Notices from a subgraph server
function decodeNotice(notice) {
let payload = ethers.toUtf8String((notice.node.payload).toString());
payload = JSON.parse(payload);
console.log(payload);
let TxType = payload.TxType;
let Origin = payload.Origin;
let Destination = payload.Destination;
let Payload = payload.Payload;
if (TxType === 'InAppMessage') {
console.log('Received InAppMessage');
sendNotice(Origin, Destination, Payload);
}
}
module.exports = filterNotices;- xmtp.controller.js file:
This is the final part of the codebase; it is the connection point between our server and the XMTP network. In this file we use the xmtp sdk to relay messages from the Cartesi dApp to target addresses on XMTP using the private key contained in our .env file.
const { Client } = require('@xmtp/xmtp-js');
const { Wallet } = require('ethers');
require('dotenv').config();
/// @variable This is the private key belonging to the wallet that'll be used to relay messages from the dApp.
const PRIVATE_KEY = process.env.PRIVATE_KEY;
/// @variable This is the second private key belonging to a different wallet that'll be used to receive messages that were sent to addresses that are not active on XMTP this way they're not completely lost.
const PRIVATE_KEY2 = process.env.PRIVATE_KEY2;
let xmtp;
/// @title initializeXmtp
/// @notice This function uses a privatekey to get an account that will be used to create an active XMTP client for sending and receiving messages.
/// @param private_key Private key belonging to the wallet that's being used to setup an XMTP connection
async function initializeXmtp(private_key) {
if (!xmtp) {
try {
const wallet = new Wallet(private_key);
xmtp = await Client.create(wallet, { env: 'dev' });
return wallet.address;
} catch (error) {
console.error('Error initializing XMTP client:', error);
}
}
}
/// @title sendNotice
/// @notice This function checks if an address is active on XMTP then relays the message to destination adress else it relays the message to the fallback address.
/// @param Origin Address thats initiating the message
/// @param Destination Address the message is to be sent to
/// @param Payload message that's to be sent via XMTP
async function sendNotice(Origin, Destination, Payload) {
try {
await initializeXmtp(PRIVATE_KEY);
const isOnNetwork = await Client.canMessage(
Destination,
{ env: "dev" },
);
if (!isOnNetwork) {
console.log(`Destination wallet is not on the network. Sending message to Fallback wallet......`);
await initializeXmtp(PRIVATE_KEY2);
const conversation = await xmtp.conversations.newConversation((new Wallet(PRIVATE_KEY2)).address);
const messageContent = `Sender: ${Origin}, Message: ${Payload}`;
const message = await conversation.send(messageContent);
console.log('Message sent:', message);
} else {
const conversation = await xmtp.conversations.newConversation(Destination);
const messageContent = `Sender: ${Origin}, Message: ${Payload}`;
const message = await conversation.send(messageContent);
console.log('Message sent:', message);
}
} catch (error) {
console.error('Error sending message:', error);
}
}
module.exports = sendNotice;At this point we have a fully functional server ready to pick up and relay notices from our Cartesi dApp to target addresses on XMTP. You can simply start your server by navigating into the src folder and running the command `node server.js`. The complete code base for this project, along with the template used in the demo video, can be found HERE, while a demo video can be found HERE.
With this integration, you now have an endless list of possibilities that you can implement in your dApps, ranging from real-time notifications and callbacks in games to personalised alerts in Defi dApp. The use cases are just endless. Though it is important to note that target addresses for these messages need to be registered on XMTP, additional efforts might be needed in convincing and also helping your users register their addresses on XMTP.
Hopefully this article inspires and educates you enough to build a project leveraging the Cartesi and XMTP integration, If you’re ever in need of more ideas, education on other possible integrations or even facing challenges in your integrations, feel free to reach out or connect with the Developer Advocacy team on discord (Cartesi Community). We’ll be excited to hear from you.
