Python Pandemonium
Published in

Python Pandemonium

Getting started with Python and IPFS

In this post I am going to discuss how you can use decentralized IPFS in your Python apps for storing different kind of data.

What is IPFS?

From Wikipedia:

InterPlanetary File System (IPFS) is a protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system. IPFS was initially designed by Juan Benet, and is now an open-source project developed with help from the community.

In simple it’s Amazon S3 on Blockchain. All of your information is available on decentralized network across nodes thus not only make the system scaleable but reliable as well since data which is fed in it will be there hence no chance of 404-File not found kind of errors.

So there can be many possibilities how you can use it, essentially for file storage, you can use as web hosting, for messaging system, possibilities are endless.

Installation

Download the setup file from here. Since I am on OSX so I just unpacked go-ipfs_v0.4.13_darwin-amd64.tar.gz and executed the file install.sh which will transfer the binary file in /usr/local/bin

To make sure everything is setup properly, run ipfs --version or ipfs on your console. If all set then you can see something like below:

➜ IPFS ipfs --version

and it prints

ipfs version 0.4.13

OK it’s installed. Now, it’s time to initialize your local IPFS node. For that run the following command:

ipfs initWhen runs it prints following:

➜  IPFS ipfs init                      
initializing IPFS node at /Users/AdnanAhmad/.ipfs
generating 2048-bit RSA keypair...done
peer identity: QmaJcDSAceC6nDZ4CEz8F6mkiGcATjWbCE8JAfbiVxY7oW
to get started, enter:
ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme

And when you run ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme on command line it should show you something like below:

So, basically ipfs cat shows the content of the file. If you do ipfs get it will download it locally on your machine.

IPFS Daemon

When you run the command ipfs daemon it will start a node on your machine which will connect with other nodes available. The concept is similar to Blockchain that there are many nodes available and you start your own node which syncs with other nodes. If you want you can have a setup of private nodes by setting IPs in Swarm and Bootstrap section. For instance these are defaults:

"Addresses": {
"API": "/ip4/127.0.0.1/tcp/5001",
"Announce": [],
"Gateway": "/ip4/127.0.0.1/tcp/8080",
"NoAnnounce": [],
"Swarm": [
"/ip4/0.0.0.0/tcp/4001",
"/ip6/::/tcp/4001"
]
},
"Bootstrap": [
"/dnsaddr/bootstrap.libp2p.io/ipfs/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/ipfs/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
"/dnsaddr/bootstrap.libp2p.io/ipfs/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
"/dnsaddr/bootstrap.libp2p.io/ipfs/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
"/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
"/ip4/104.236.179.241/tcp/4001/ipfs/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
"/ip4/128.199.219.111/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
"/ip4/104.236.76.40/tcp/4001/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
"/ip4/178.62.158.247/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd",
"/ip6/2604:a880:1:20::203:d001/tcp/4001/ipfs/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
"/ip6/2400:6180:0:d0::151:6001/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
"/ip6/2604:a880:800:10::4a:5001/tcp/4001/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
"/ip6/2a03:b0c0:0:1010::23:1001/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd"
],

you can view it when you run ipfs config show

So, when you do ipfs daemon it shows following output:

Initializing daemon...
Adjusting current ulimit to 2048...
Successfully raised file descriptor limit to 2048.
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/192.168.1.3/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /p2p-circuit/ipfs/QmaJcDSAceC6nDZ4CEz8F6mkiGcATjWbCE8JAfbiVxY7oW
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/192.168.1.3/tcp/4001
Swarm announcing /ip4/39.57.97.236/tcp/49388
Swarm announcing /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

You can have a web interface by visiting http://localhost:8080/ipfs by adding Hash of the file object. So for instance when you run ipfs refs local it prints the content exist on your local node:

➜  IPFS ipfs refs local
QmdL9t1YP99v4a2wyXFYAQJtbD9zKnPrugFLQWXBXb82sn
QmZTR5bcpQD7cFgTorqxZDYaew1Wqgfbd2ud9QqGPAkK2V
QmXgqKTbzdh83pQtKFb19SpMCpDDcKR2ujqk3pKph9aCNF
QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv
Qma4NNR8dUSDt2BvLYYtgdMLF8J3usKrT9kDFhHzfpB7oq
QmNcNo8TXi92Da91fDfzCMbYF5ScaHEJmQG1jqCEbkS7Kt
QmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y
QmejvEPop4D7YUadeGqYWmZxHhLc4JBUCzJJHWMzdcMe2y
QmPhk6cJkRcFfZCdYam4c9MKYjFG9V29LswUnbrFNhtk2S
QmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7
QmSKboVigcD3AY4kLsob117KJcMHvMUu6vNFqk1PQzYUpp
QmQ5vhrL7uv6tuoN9KeVBwd4PwfQkXdVVmDLUZuTNxqgvm
QmZZRTyhDpL5Jgift1cHbAhexeE1m2Hw8x8g7rTcPahDvo
Qme7RW9zfGgYujJt6CJ5yKiMkvV9zPSSkbB4hgipee3j6S
QmYQoke9bEqzBLWPGqyjhUYc3TwBEkn4wed2kUmAbxvLFu
QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n

And you can view the content on your browser by doing http://localhost:8080/ipfs/QmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7 on your browser and can view the content. In my case it is the help file by IPFS itself.

Pinning

When you post a file on IPFS network, it’s spread across the connected nodes. IPFS provides garbage collection concept which eventually let the files get deleted. Or, if your node goes offline and not synced the data will be lost. In order for your data stay there forever, you pin your file by running the command:

ipfs pin add <hash_of_content> where hash of content are the entries you see above. When you do it, it’s not removed due to garbage collection process. You can also unpin by running ipfs pin rm <hash_content>

So far we see how you can interact with IPFS via command line but how to automate it or even use in your python, let’s get into the code!

Development

I am not doing anything fancy, just calling commands we called via CLI. By learning that you can come up with your own Image Hosting Service or Messaging App or your own decentralized Amazon S3!

First install IPFS Python API, run the command pip install ipfsapi

Start local daemon by running ipfs daemon and create a local Python file in your favorite IDE.

import ipfsapiif __name__ == '__main__':# Connect to local nodetry:api = ipfsapi.connect('127.0.0.1', 5001)print(api)except ipfsapi.exceptions.ConnectionError as ce:print(str(ce))

If connected you will see ipfsapi.client.Client object at 0x104af5a90 otherwise exception like: ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5001): Max retries exceeded with url: /api/v0/version?stream-channels=true (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 61] Connection refused',))

OK, it’s connected now let’s add a new file:

new_file = api.add('new.txt')

It will add a new file. If you get a print() of it you can see a return dict object like below:

{'Name': 'new.txt', 'Hash': 'QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq', 'Size': '28'}

Now let’s access this file via browser by visiting http://localhost:8080/ipfs/QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq/

and it will show the newly added file.

In order to see the content of file you call api.cat('QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq') and it will print content of the file, same thing you do via command line. You can learn more about IPFS and Python integration here.

Conclusion

IPFS is a new way to implement web, a new protocol that makes thing scaleable and reliable. You could consider it as a decentralized storage system running on blockchain. As a Python developer you can integrate new or existing Python apps by using these simple to use APIs. Like always, code is available on Github

This post originally published here.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adnan Siddiqi

Adnan Siddiqi

2.8K Followers

Pakistani | Husband | Father | Software Consultant | Developer | blogger. I occasionally try to make stuff with code. http://adnansiddiqi.me