Deleting Multiple Rows in DynamoDB
Had to do this today and after struggling with it for a bit, I finally got it working. Here’s how…
First of, my table, talr-taskStatus, has a partition key called requestId and a sort key called eventTimestamp. My specific scenario is that the table grew to a fairly large size and I wanted to purge all the rows for a given requestId which was about 10K rows.
DynamoDB doesn’t have a command that deletes multiple rows so I ended up running a Query call, looping through the response to feed into a BatchWriteItem call. Here’s how in Python:
Worked quite well, just had to increase the write capacity units as well as the Lambda timeout (the request ran in a Lambda function).
One learning, as you can see from the code above, I’m using boto3.client. Originally I was using boto3.resource and was getting the following error:
An error occurred (ValidationException) when calling the BatchWriteItem operation: The provided key element does not match the schema
Unfortunately that error had nothing to do with the root cause of the problem and caused over an hour of unneeded troubleshooting in the wrong area but switching the resource to client did the trick.