Archiving Books and Records for a Broker-Dealer

Prerak Sanghvi
Proof Reading
Published in
7 min readSep 21, 2020

Unless you are a particular connoisseur of regulatory technology, “How do I archive the relevant books and records for my broker-dealer in compliance with SEC Rule 17a-4 and FINRA Rule 4511”, is probably not a question that gets you excited. And if it does, you probably have this all figured out, so this article is not for you; it is for the other technologists who have wondered but never dared to ask for fear of an extremely dull answer.

This post is a follow-up to our popular post on obtaining a broker-dealer license.

TL;DR

Here’s what we do at our broker-dealer subsidiary:

  1. Use Global Relay to archive communications
  2. Use rclone to copy files from AWS and Google Drive into a Wasabi bucket that is configured as Write-Once-Read-Many (WORM) storage, and use SIRS as the Designated Third Party (D3P).

Background

Without going into too many details, the SEC/FINRA rules require broker-dealers to archive records in a non-destructive format for a certain period of time, say 2–7 years, depending on what is being archived. More recent data is required to be stored in an “easily accessible” place, while older data may be archived away for slower retrieval. Also, the data should be indexed and a designated third party (D3P) should be able to access it independently if and when the regulator needs them to.

Back when the records were being archived on physical media (boxes full of files, then tape backups, then CD-R, DVD-R, BD-R disks, etc), it made sense that there should be a limit to how many tapes are kept in storage and that only recent ones are “easily accessible”. However, with the advent of inexpensive cloud storage, it seems to me that the simple answer is to just archive everything and keep it all easily accessible forever (or for foreseeable future, if forever feels like a really long time). The side benefit of this is — you don’t have to spend time categorizing pieces of information and figuring out how long a certain piece should be saved for.

Possible solutions

Vendors. It just makes sense — for any tedious but necessary activity that is non-core to a business, it often makes sense to offload it to a vendor. A vendor can aggregate the expertise needed and spread out the costs between its various clients, relieving you of the hassle and providing peace of mind. In this case, it makes even more sense because the rule already requires hiring a designated third party who has access to the archived data.

For archiving communications (chat, email), vendors like Global Relay provide an affordable and easy solution, especially if you use hosted chat/email like Slack and GSuite (or Microsoft 365). For Slack, you will need to upgrade to “Enterprise Grid” or a higher plan in order to turn on archiving through Global Relay.

For the rest of your records, like legal and financial documents, customer and employee data, and trading records, companies like Iron Mountain and Patrina charge a pretty penny to archive and index your data, and also act as the designated third party.

If you’re a cost-conscious startup and willing to do a bit of work yourself, you can split this activity into a few different services and save yourself a significant amount of money (like, over 90%, depending on the amount of data you are archiving):

  • WORM storage provider: You need a vendor that provides an attestation saying their storage is compliant with the relevant regulations. Wasabi provides this attestation, while AWS S3/Glacier does not, although AWS provides a third-party assessment confirming their compliance. Our understanding is that FINRA prefers the self-attestation approach, which is the biggest reason why we picked Wasabi instead of AWS.
  • Transfer services: You need a method of copying relevant data from your data center servers, shared drives, cloud storage (S3, GCP buckets, etc.), cloud drives (OneDrive, G-Drive, etc.) to your WORM storage
  • D3P: Designated third party services. SIRS is a vendor that is willing to work with any compliant WORM storage provider.

Wasabi

Wasabi is an “S3-compliant” storage solution. What this means is that the standard AWS S3 APIs can be used to interact with Wasabi buckets. Said differently, any client that works with AWS S3 can work with a Wasabi bucket (assuming the tool allows configuration of the bucket endpoint).

Caveat: if you choose Wasabi and your data is stored in AWS or at another cloud provider, keep in mind that all cloud providers charge a data egress fee, which can add up if you’re transferring terabytes of data.

Wasabi uses the immutability feature of its storage service to provide WORM storage. Here are the steps to set up a compliant bucket:

  • Sign up for an account online (30-day free trial, then charges based on data stored)
  • Create a bucket named, say, sec-17a-4. Ensure "public mode" is off, then turn on "Compliance mode" with retention set to, say, 10 years (or 100 years!). Lock "Compliance mode". Once locked, these properties cannot be changed by a user.
Wasabi bucket settings for Compliance mode
  • Create a group called sec-17a4-writers, which has read and write access to the bucket, but not admin access.
  • Create sub-users (in our case, prerak_proof and dan_proof) with their own password. You will be able to download a credentials file with "Access Key Id" and "Access Key" that should be used to archive files to the bucket.

That’s it. From that point on, anything copied into the bucket may not be erased for the retention period specified.

Some tips to keep in mind:

  • It is helpful to organize your official files that constitute books and records into a single or small number of folders on your network. For us, we organized all of our records into a single folder in Google Drive. We archive a copy of this folder on the first of each month. This way, we don’t have to think about what we need to archive — everything in the folder gets archived.
  • Given that the archive files may not be deleted, moved, or renamed, it is best to create a dated subfolder for each run of the archive job (as opposed to always copying into the same folder).
  • You can use Wasabi Explorer to manually check on the files or retrieve them.

Transferring Data to Wasabi

This sounds like it should be an easy thing, and it is, if you store your files locally or on network drives or even S3. In these cases, you can use one of the many S3 command-line client utilities available to transfer data to Wasabi. Our trading system records will be stored in AWS (EBS drives or S3), and will be transferred to Wasabi using this method. And of course, you can automate the whole process using cron or your favorite scheduler (you can use cron even if you use Windows!).

If you store your files in cloud drives though, this is much trickier than you’d expect, and we found that rclone does the job well. Here are the steps:

  • Install rclone (if you use Windows, I highly recommend using chocolatey to install and manage miscellaneous tools like this)
  • Use rclone config to configure two remotes - Wasabi and your cloud drive (e.g. Google Drive)
  • Alternatively, if you are an advanced user and know how to obtain the relevant credentials, you can create a config file directly. In our case, the rclone.conf file looks like below. Note the shared_with_me = true attribute under [Proof-GDrive]section; this is needed if you're not the owner of the directory you want to sync
[Proof-GDrive]
type = drive
client_id = 28751[...]b0ba.apps.googleusercontent.com
client_secret = xbEf...
scope = drive.readonly
token = {"access_token":"ya29...","token_type":"Bearer","refresh_token":"1//01YT...","expiry":"2020-07-15T08:59:18.330418-04:00"}
root_folder_id = 0ACL...
shared_with_me = true
[Proof-Wasabi]
type = s3
access_key_id = 5XG...
endpoint = s3.wasabisys.com
provider = Wasabi
region = us-east-1
secret_access_key = Q9e...
  • Use rclone to sync files using a command like this (obviously, replace Proof-GDrive and Proof-Wasabi with the names of the remotes you created in the previous step):
rclone -v --config <rclone.conf> copy "Proof-GDrive:Corporate Books and Records" "Proof-Wasabi:sec-17a-4/Corporate Books and Records/20200801-030002"
  • The complete shell script and the accompanying config file can be found here
  • You’ll notice the use of sendmail in the linked shell script to send notifications. Unless your company already has its own SMTP server or a mail-relay server, this can be its own can of worms. If you're using Linux or WSL (Windows Subsystem for Linux), you can configure postfix or ssmtp to send mail using the Gmail SMTP server, but you have to embed credentials in a config file. As an example, use this link to configure ssmtp. For AuthPass, use a Gmail "App Password". If there are better (but free) ways of sending automated emails on Windows, please let me know!

D3P

Setting up the designated third party was the easiest step of them all. All we had to do was contact SIRS and sign an annual contract with a completely reasonable fee.

Strangely enough, Wasabi itself provides a letter (see letter regarding Rule 17a-4(f)(3)(vii)) that seems to claim that it would provide the requisite D3P services, but we clarified with them and they do not, in fact, provide such services. They referred us to Iron Mountain for the same.

Closing Thoughts

If you actually read the whole article and got to this paragraph, I only have one question for you — why?? Just kidding, see you here next time hopefully with a more interesting technical topic.

If you think we’re missing something or if you know of easier ways to achieve the above, please let me know on Twitter: @preraksanghvi

--

--