Alpha implemented functions in ethearnal software:
- DHT (Kademlia distributed hash table — UDP ipv4 messaging up to MTU size(1500 bytes) )
- Push, Pull from DHT distributed document store ( document data model )
- Personal node ( localhost http for ui and udp DHT server )
- BSON flexible protocol
- RSA message signing and verify ( to guarantee ownership of messages )
- SHA256 hashing
- UPNP IDG punch firewall hole ( for udp service )
- Persist document DHT store based on SQL
- Auto Peer exchange, auto Public key exchange
- Search indexing on DHT, offers prefix or other property hashing search
- Support custom quantitative search rank.
- Replicating-distributed CDN Nodes ( udp and http content delivery nodes)
- Distributed linked lists based on DHT ( documents per owner )
- Per GUID owner properties
- OpenAPI REST specification, ( plugable single page app UI )
- Buildable and portable ( with pysetup )
- Based on mature software libraries: CherryPy ( http server), SQLite, Python 3.5
With simple words
We are implementing from one side distributed key -> value storage, peer to peer network communicate with fast/short UDP messages keys are hashed with SHA256 and values are up to 1500 bytes binary BSON ( lbinary json ) interoperable format. We are creating small documents which hyperlinks to other documents with their hash keys and subject URIs. DHT is the distributed hash table document database. DHT uses Kademlia algorithm to to contact with efficient complexity O(log(n)) nodes and find value for queried key.
Advantage of UDP communication is its simplicity and fast convergence and discovery ,the limitation is size of UDP datagram packet have to < 1500 bytes in order to not being fragmented ( routers default maximum transmission unit MTU ) UDP is unreliable session-less protocol but we make it reliable with our protocol with RSA signed and sha256 hashed bson messages ( key -> value short document )
To ensure distributed data integrity authenticity tampering protection and etc, we are using patent free RSA stable implementation ( rsa python library ). Our DHT implementation and bson formated protocol integrates directly RSA message signing ( now implemented ). Our application node generate itself public, private key pair ( currently only for signing, encryption is supported but not integrated) or could use exiting one ( e.g SSH RSA PEM keys ). We create our own automatic public key exchange infrastructure ( first pub keys are exchanged within DHT on joining nodes and later data authenticity and integrity is verified ) RSA message signing is critical for whole distributed document database.
As we implemented sha256 key->signed bson document distributed database we distribute/replicate our gigs posting from digital owners across network of peer to peer nodes. As much gigs are pushed to many nodes, data become permanent across network ( like IPFS)
The limitation of key -> value database is that you need to know the key to get the value, for that reason we created a light but efficient search engine that is capable to index up to 10K Gig documents. Now search engine is in heavy development but it has been tested to work with word-tem prefixes as you type. Search engine converts search terms into a hashes and it could be used from any party with higher degree of privacy.
In order to support big binary files like images and also HTTP search API we implemented CDN ( content delivery network) type of node. It is like personal node but also exports HTTP search and binary hashed upload/download resource. CDN nodes again are members to DHT database and uses it to exchange message for uploaded resources in order to replicate it and distribute it across other CDN nodes. The other rolle of CDN nodes is to give index search services — when other peers pushing to DHT Gigs ( service offers/request ) or other value-bson-documents are indexed by and then appears in search results. Network of CDN nodes create high available DHT, binary resource upload/retrieval and search services, which are rewarded with large amount of indexed knowledge and other crypto-economic stimulus we are going to create.
In next stages of project, we are going to interlink our digital RSA ids and network with other networks, and mainly we integrate ethereum platform. ERT tokens, will be used for payments and ERT turnover and hold will became a major search index indicator and rank. Thus in this case security is increased as the Gig ( service offer/request ) publishing/searching/processing is used with our autonomous DHT RSA PKI infrastructure and we are not using ethereum keys for that job. However in ethearnal DHT network our RSA identities could be linked with other mainly ethereum addresses, when sending small amount of tokens ( a.k.a 0.00002344 ) as code to verify cryptocurrency identity. Now we are totally open source, and implement OpenAPI (swagger) specification, so our swarm of cdn nodes will form autonomous distributed API cloud.
From the client perspective, now implementation of browser only client is possible where RSA private key for signing/encryption will stay only in browser. Currently our backend architecture allow pluggable UI implementations to REST exposed by node server, thus CDN nodes could be expanded to website portals where content will be exposed and indexed (from many different domains) to current centralized search engines like google to increase our popularity.
GitHub: ethearnal
