Tuning MongoDB transactions
Transactions are new in MongoDB but have been existed in SQL databases for more than 30 years. Transactions are used to maintain consistency and correctness in database systems that are subjected to concurrent changes issued by multiple users.
Transactions generally result in improved consistency at the cost of reduced concurrency. Therefore, transactions have a large bearing on database performance.
This post is not intended as a tutorial on transactions. To learn how to program transactions, see the MongoDB manual section on transactions[1]. In this post, we will concentrate on maximizing transaction throughput and minimizing transaction wait times.
TransientTransactionErrors
Unlike many other transactional databases such as MySQL or Oracle, MongoDB does not use blocking locks to prevent conflicts between transactions. Instead, it issues TransientTransactionErrors to abort transactions that might cause conflicts.
The figure below illustrates the MongoDB paradigm. When a session updates a document, it does not lock it. However, if a second session tries to modify the document in a transaction, a TransientTransactionError is issued.
Here is some code that illustrates the TransientTransactionError paradigm. The code snippet creates two sessions, each within its own transaction. We then attempt to update the same document within each transaction.
var session1=db.getMongo().startSession();
var session2=db.getMongo().startSession();var session1Collection=session1.getDatabase(db.getName()).transTest;
var session2Collection=session2.getDatabase(db.getName()).transTest;session1.startTransaction();
session2.startTransaction();session1Collection.update({_id:1},{$set:{value:1}});
session2Collection.update({_id:1},{$set:{value:2}});session1.commitTransaction();
session2.commitTransaction();
When the second update statement is encountered, MongoDB issues an error:
mongo>session1Collection.update({_id:1},{$set:{value:1}});
WriteCommandError({
"errorLabels" : [
"TransientTransactionError"
],
"operationTime" : Timestamp(1596785629, 1),
"ok" : 0,
"errmsg" : "WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.",
"code" : 112,
"codeName" : "WriteConflict",Transactions in the MongoDB drivers
From MongoDB 4.2 onwards, the MongoDB drivers hide transientTransationErrors from you, by automatically retrying the transaction. For instance, you can run multiple copies of this NodeJS code simultaneously, without encountering any TransientTransactionErrors:
async function myTransaction(session, db,
fromAcc, toAcc, dollars) {
try {
await session.withTransaction(async () => { await db.collection('accounts').
updateOne({ _id: fromAcc },
{ $inc: { balance: -1*dollars } },
{ session }); await db.collection('accounts').
updateOne({ _id: toAcc },
{ $inc: { balance: dollars } },
{ session }); }, transactionOptions);
} catch (error) {
console.log(error.message);
}
}
The NodeJS driver — and drivers for other languages such as Java, Python, Go, etc., — automatically handle any TransientTransactionErrors and resubmit any aborted transactions. However, the errors are still being issued by the MongoDB server, and you can see them recorded in the MongoDB log:
~$ grep -i 'assertion.*writeconflict' \
/usr/local/var/log/mongodb/mongo.log \
|tail -1|jq
{
"t": {
"$date": "2020-08-08T14:04:47.643+10:00"
},
…
"msg": "Assertion while executing command",
"attr": {
"command": "update",
"db": "MongoDBTuningBook",
"commandArgs": {
"update": "transTest",
"updates": [
{
"q": {
"_id": 1
},
"u": {
"$inc": {
"value": 2
}
},
"upsert": false,
"multi": false
}
],
/* Other transaction information */
},
"error": "WriteConflict: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction."
}
}At a global level, the retries are visible in the db.serverStatus counter transactions.totalAborted.
function txnCounts() {
var ssTxns = db.serverStatus().transactions;
print(ssTxns.totalStarted + 0, 'transactions started');
print(ssTxns.totalAborted + 0, 'transactions aborted');
print(ssTxns.totalCommitted + 0, 'transactions commited');
print(Math.round(ssTxns.totalAborted * 100 /
ssTxns.totalStarted) + '% txns aborted');
}mongo> txnCounts();
203628 transactions started
167989 transactions aborted
35639 transactions commited
82% txns aborted
The performance implications of TransientTransactionErrors
The retries that result from TransientTransactionErrors are expensive — they involve not just discarding any work done in the transaction so far, but also reverting database state back to the start of the transaction. It is the impact of transaction retries more than anything else that makes MongoDB transactions expensive. The chart below shows that as the percentage of transaction aborts increases, the elapsed time for transactions degrades rapidly.
Reducing transient transaction error impact
Given that TransientTransactionError retries have such a severe effect on transaction performance, it follows that we need to do whatever is possible to minimize these retries. There are a couple of strategies that we can employ:
- Avoid a transaction altogether.
- Partitioning “hot” documents that are subject to high levels of write conflict.
- Order operations to minimize the number of conflicting operations.
You can avoid transactions in a couple of ways, perhaps by putting related items into a single document so that you can update them atomically. You can reduce contention for documents by partitioning data across multiple documents. In our book, we give a few examples of these techniques.
A third option is to change the order of actions in a transaction. This approach can be surprisingly effective.
For example, consider the following transaction:
await session.withTransaction(async () => {
await db.collection('txnTotals').
updateOne({ _id: 1 },
{ $inc: { counter: 1 } },
{ session });
await db.collection('accounts').
updateOne({ _id: fromAcc },
{ $inc: { balance: -1*dollars } },
{ session });
await db.collection('accounts').
updateOne({ _id: toAcc },
{ $inc: { balance: dollars } },
{ session });}, transactionOptions);;
This transaction transfers funds between two accounts, but first, it updates a global “transaction counter”. Every transaction that tries to issue this transaction will attempt to update this counter, and many will encounter TransientTransactionError retries as a result.
If we move the contentious statement to the end of the transaction, then the chance of a TransientTransactionError will be reduced, since the window for conflict will be reduced to the final few moments in the execution of the transaction. Below we see the effect of this optimization:
Consider placing “hot” operations — those like to encounter TransientTransactionErrors — last in your transactions to reduce the conflict time window.
Conclusion
Transactions are a really useful addition to MongoDB, but they do have performance implications. In particular, transaction retries can have a significant negative effect on throughput. In this post, we discussed a few options for reducing the impact of transactions.
For more information, try our book MongoDB Performance Tuning — Chapter 9 is dedicated to use and tuning of transactions.
Guy Harrison and Michael Harrison are authors of MongoDB Performance Tuning (Apress, 2021). They both work at ProvenDB which, amongst other things, brings the immutability and trust of public Blockchains to the power and productivity of MongoDB.
