When sending digital data is done physically and why

We are living in a digital world, we spend most of the time with computer and smartphone. Most communications are done by messengers, like Whatsapp, Telegram and Facebook, or emails. We share large files by putting them on cloud like Google Drive, iCloud, OneDrive and Dropbox. However, in some situations, sending digital data by the oldest and slowest way, physical delivery, is still preferred.

Image from https://dazeinfo.com

How to send digital data in a physical way? Simply copying a data to a thumb drive or HDD and ask someone (maybe yourself or courier) to bring the drive to the recipient.

When will we prefer to do it? One case is sharing files between close people, if the person who want to have the data is standing in front of me, why bother to use the Internet? Some cases may not be obvious to most of us, data tracking and huge data delivery.

If you are working in an enterprise, very likely your IT team will tell you that the USB ports of your computer are disabled and cannot insert thumb drive for security reason. You are forced to send or receive all data though network, under the company monitoring. This measure prevents people from putting malicious external drive to the computer, and hence preventing any virus and malware from infecting the computer and company network via USB. Another benefit is preventing confidential / company data leaking without record.

It is especially important in situations like auditing and forensic investigations. The flow of data must be closely monitored and recorded (chain of custody), and any possible way of leaking the data must be prevented. This is why they often have a air-gapped computer for accessing the confidential data. After auditing or investigations, they will need to send back the data by courier service, which provide a detailed delivery record showing that the data is sent to the expected party and no one else has accessed it.

Proof of data delivery and confidentiality is very important in this case.

Another case is huge data delivery. Here it is not about sending out a video or album, which can still be done by cloud drive. It is about sending a system image from vendors, database backups from websites, or sending a HDD image from a suspect PC. The size is about tens of or hundreds of gigabytes, or even terabytes.

Hitachi HDD, we often use it to keep client private data and harddisk images in forensic investigations

Obviously they cannot be sent via messengers or emails. Cloud drive may not work too as it is very expensive to have such large storage, especially if you only do it once or less than once a year. The bandwidth of cloud drive is also very narrow compared to the data size (<5MB/sec). It may take hours to transfer a data up to cloud, and more hours to download from cloud to the destination.

Take a 500GB data for example, it will take at least 102,400 seconds, or 1.19 days to upload to the cloud, and 1.19 days more to download to the destination, assuming the network does not have any problem or congestion during the transfer and the cloud provider can process and distribute the data on cloud in no time.

Some companies will do it by mapping a network drive from target computer to source computer and copying the data using the network drive. This method is faster than putting it on cloud, but still takes a very long time (>1.19 days). But it is very hard to keep a stable connection long enough to download the whole data. Any short disruption (<1 minute) on the connection will fail the whole download process (over a day).

To address this problem, there are programs like FileZilla which allows people to setup a temporary FTP server on the sender machine and let the destination machine to connect to it and download the data. The system also supports resuming download after connection disruption.

Another problem is on the file integrity checking. For huge files it is likely that some of the bits inside the file is changed during the transmission. There are error corrections like CRC on the communication level, but it may not be sufficient to fix all errors, especially when the error comes from connection disruptions. An extra file integrity check must be done on both sender and recipient side after sending the file to make sure not a single bit of the file is changed.

Is there anyway to transfer a data which can make sure no error during the transmission, no need to setup long and stable connection between computers, no need to keep both computers open at the same time for days, solid data delivery and confidentiality?

Yes, it is physical mailing, even it sounds stupid to most of us in this digital age.

BitTorrent also solves this problem for public data, but it does not work on private, or encrypted data, as the peers are not willingly to join the swarm and distribute data they cannot access.

Authpaper Delivery platform solves this problem by combining Blockchain and BitTorrent. Blockchain keeps the transaction records of the currency inside this platform, or stamp, as well as the data delivery requests and records in terms of multi-party-signed smart contracts. When someone would like to send out a private data, he/she needs to pay mailers stamps to handle the data transfer. Other peers will get paid in stamps for joining the swarm and assisting the data distributions. The data is confidential for the peers. A detailed delivery record is stored on Blockchain for public review, BitTorrent protocol makes sure no a single bit is changed during the transaction, the sender and recipient do not need to online together and fault tolerance. The delivery speed is also much higher, because there are many peers assisting the download speed. It has all benefit of physical mailing, but takes much less time.

This is why we think Authpaper Delivery will be the next generation P2P data Delivery system.