Decentralized File Storage with Kademlia and IOTA
It has been said that the most valuable resource in the 21st century is no longer oil but data. With most of the data being controlled by 6 or 7 tech companies many people have started developing peer to peer/ decentralized systems again to try and take back control of data and mitigate the risk of relying on cloud providers.
In this article I’ll explore the use of the Kademlia DHT and the Iota tangle for storing data on a decentralized network of nodes.
Software Libraries used
Kadence library
Iota javascript library
Overview
Kademlia is a distributed hash table. It uses key-value pairs similarly to regular hash maps but it stores them on various nodes connected to it. It searches for the node with an ID that closely resembles the key in the routing table of each node until it finds the closest match. This allows it to find and store data in O(log(n)) time while only needing a small amount of addresses in the routing table. I used the Kadence library to implement a Kademlia network.
Iota is a distributed ledger technology similar to blockchain but has a completely different structure. It is incredibly fast and is useful for projects where you use micro transactions.
A basic Kademlia node will be ran that allows us to store and retrieve small bits of data and the Iota token will be used to pay small amounts to any node that stores the data.
Project Discussion
Kademlia requires two or more nodes to store data on the network. In my code I set up two nodes running on localhost and used the join method so one would connect to the other.
const node = new kadence.KademliaNode({
identity: kadence.utils.getRandomKeyBuffer(),
transport: new kadence.UDPTransport(),
storage: levelup(encode(leveldown(‘database’))),
contact: { hostname: ‘localhost’, port: 1337 }
});
//Creating second node object//
const nodeTwo = new kadence.KademliaNode({
identity: kadence.utils.getRandomKeyBuffer(),
transport: new kadence.UDPTransport(),
storage: levelup(encode(leveldown(‘secondDB’))),
contact: { hostname: ‘localhost’, port: 8080 }
});
//Listening on ports of each node//
node.listen(1337);
nodeTwo.listen(8080);
//Node joins network through nodeTwo//
node.join([ nodeTwo.identity, {
hostname: ‘localhost’,
port: 8080
}], async () => {
//Rest of the article’s code
//goes in here besides the use
//Methods and the sendIota function
});
The next thing required is to use the iterativeStore() function. This takes a key and a value as parameters. It finds K closest node IDs to the key and stores them on that node’s local storage. The storage directory can be seen in more detail in the video I have at the top of the article
