Learn Internet Grow

Tech Skills

  • Measure
  • Search
  • Create
  • Secure
  • Share

IPFS? What is the InterPlanetary File System?

March 25, 2020 By Dmitri Larionov

It’s difficult to explain the importance of the internet to humanity. Many have stated that it’s our best invention ever.

So what’s IPFS and what does it have to do with the internet?

Is it the future of the internet? Who’s working on improving it’s protocols, systems, and tools? What is the Interplanetary file system aka IPFS and why is it important?

Protocol labs created this innovating tech. Essentially, it consists of a decentralized internet scheme on top of the existing one.

In more specific terminology, IPFS is a peer-to-peer version controlled file-system which seeks to model all data as part of the same Merkle Tree (hash tree) which is DAG (Directed acyclic graph) type of data structure.

Directed Acyclic Graph as the basis of IPFS Image

How does this work? What does this mean for government censorship? How can you start using it now? Are there any dangers?

This article will cover many topics regarding IPFS, however, the interplanetary file system is a very complex subject, so to really dig in, check out the IPFS white paper after you’re done with this article.

My goal with this article is to introduce you to many of the related concepts, give you an understanding of the value of IPFS, and a generalized view of how it works.

What is Decentralized Resource Sharing?

You computer is likely only using part of it’s available CPU cycles, disk storage, and RAM.

resource sharing for IPFS image

What if you could use these resources to speed up a network you and your organization depend on?

What if you could make money of off it?

Examples of Distributed File Systems

The concept of distributed computing and file storage is not new, but IPFS certainly is.

Here are some technology precursors and inspirations.

AFS mentioned in the White Paper

The AFS is such a system, developed at Carnegie Mellon University.

Andrew File System (AFS) has succeeded widely and is still in use today.

ARIA

A free AFS implementation.

OPEN AFS

Home of the OPEN AFS.

Dat Project

A distributed data community aka Dat is a nonprofit-backed data sharing protocol for apps of the future.

Perkeep

This set of open source formats, protocols, and software, previously known as Camlistore (Content-Addressable Multi-Layer Indexed Storage), is used to model, store, search, share, and sync data.

Peer-to-Peer Examples

Some other examples include popular publicly available peer-to-peer file sharing applications like Napster, KaZaA, and BitTorrent.

These decentralized platforms offered audio, video, and other file formats but it fell short on providing a way to build on top of it.

build on top of IPFS

Thus far, there hasn’t been a global low-latency, and decentralized distributed system that is designed as infrastructure to be built on.

MojoNation

MojoNation is an old version which attempted to implement a way to compensate people for contributing resources to a network.

It wasn’t successful due to the challenge of compensating contributors. This was before crypto-currencies, so now, it’s much simpler to compensate users of a decentralized network.

Introducing, IPFS.

What is InterPlanetary File System – IPFS?

IPFS is all about decentralized resource sharing.

It’s a technology that enables content to persist on the internet as long as someone on the network has a copy.

You can view IPFS as a single BitTorrent swarm, exchanging objects within one Git Repository (from the white paper).

The white paper goes on to put it in more technical terms.

IPFS provides a high throughput content-addressed block storage model, with content-addressed hyper links.

This forms a generalized Merkle DAG (directed acyclic graph), a data structure upon which one can build versioned file-systems, block-chains, and even a Permanent Web.

The IPFS stack uses IPNS and IPLD to define the data, and Libp2p to move the data.

The IPFS Stack

The stack utilizes Filecoin. This part of IPFS is to provide a cryptocurrency incentivized storage network.

The Filecoin block-chain is used to compensate contributors to the network.

How Does IPFS Work?

Getting a resource from the network starts with a hash that identifies a resource. If the original publisher source isn’t available, the protocol will find someone on the network with a copy.

The protocol will connect to the node, fetch the resource, verify it’s authenticity, and then send it to you.

Protocol labs created it and continues to improve and maintain it. They do research, development, and conduct experiments with network protocols.

They create open-source technologies, which means that you can take a look at their projects.

IPNS

IPNS is the InterPlanetary Naming System.

It uses a Distributed Hash Table.

DHT or the Distributed Hash Table

DHT distributed hash tables for IPFS image

DHTs are widely used to coordinate and maintain metadata about peer-to-peer system.

DHTs Mentioned in the White Paper

Kademlia DHT provides lookup through massive networks.

Coral DSHT extends Kademlia to address a handful of deficiencies.

  1. More efficient storage scheme to save storage and bandwidth.
  2. Increased efficiency of value distribution by only providing a subset.
  3. Creates regions, so queries can first apply only to a local region, and then if value isn’t found they can query more distant nodes.

S/Kademlia DHT extends Kademlia to address security.

  1. Prevents Sybill attacks in Node Id generation
  2. Enables lookup values over disjoint paths in order to ensure communication with a large number of adversarial nodes in the network

It provides a lookup service similar to a hash table. Key/value pairs are stored in the distributed hash table.

IPLD

Interplanetary Linked Data Logo

IPLD stands for interplanetary linked data. Although, my initial guess was interplanetary link directory. Here’s the github repo.

The data model of the content-addressable web. With it, you can treat all hash-linked data structures as subsets of a unified information space.

In essence this means that you can unify any data models that link data with hashes as instances of IPLD.

  • A common hash-chain format for distributed data structures.
  • Allows system interoperability..
    • Ex: Bitcoin to Git, Git the Ethereum, etc..
  • Merkle-links
    • secure, immutable
  • Merkle-paths
  • Universal nestable URIs
  • Canonical hashing safe

libp2p

This peer-to-peer networking stack is open source. It’s called a modular network stack because you can run network apps free from run-time and address services, independent of your location.

lib peer to peer logo for ipfs

What does this library enable you to do?

  • Use several transports
    • Newer and faster transports
      • Examples: QUIC and WebRTC
    • More established ones such as TCP
      • supporting multiple hosts and connectivity conditions.
  • Native Roaming
    • configuration step free migration between networks and machines
  • Runtime Freedom
    • Assume connectivity
  • Protocol Muxing
    • Reuse already established and secure connections with in-band protocol multiplexing.
  • Work Offline
    • libp2p is capable of discovering other peers without resourcing to centralized registries, enabling apps to work disconnected from the backbone.
  • Encrypted Connections
    • Set a crypto channel by default
  • Upgrade without Compromises
    • Upgrade your service while still supporting older versions.
  • Work in the Browser
    • Write your application to run in the browser.
  • Good for High Latency Scenarios
    • Not all connections are equal. Make sure your app is using the best transport for its environment.

Implementations in:

  • Javascript
  • NodeJS
  • GO

Implementations coming soon, as of 6-10-2018.

  • Java
  • Python
  • Rust
  • Haskell

Multiformats

This part of IPFS is a collection of protocols which aim to future-proof systems.

This is the name of the organization but also the name of the protocols.

In other words, self-describing values for future-proofing. This allows inter-operability, protocol agility, and helps us avoid lock in.

You can think of Multiformats as a specification that helps systems to evolve and grow.

The Multiformats github page, is a good source of examples and source of deeper understanding, for this part of the IPFS stack.

FileCoin

filecoin logo

FileCoin is a decentralized storage network created by Protocol Labs. It’s a decentralized network and a protocol.

Basically, you can contribute your file space to the network to earn FileCoins which can be traded for other crypto-currencies or a fiat-currency like the US Dollar.

The idea is that over time the cost of storage needs to go down, since the amount of data will continue to grow. Hence, FileCoin’s goal of reliably storing files at hyper-competitive prices.

Core Concepts

The above are technologies that together help IPFS work. There are a number of concepts that will help solidify your understanding of IPFS.

Content Versus Link Based Addressing

The web as we know it works as a set of links. If the server hosting the domain of a link is down, then the content can’t be retrieved. You can check out the internet archive but this isn’t ideal.

IPFS uses content based addressing where the address is a hash. If someone has a copy of the content that this hash represents, others can view it, even if the origin server is down.

Being IPFS Friendly

Make sure that all the links on your site are relative. Or in other-words, they don’t include the domain.

Interviews

David Dias

Worked on peer to peer and received a masters in it’s study. Say IPFS’s ambitious goals and joined the team. He say that they have similar goals and problems to solve. He created the IPFS Javascript implementation so it can run in the browser.

He has an interview on Software Engineering Daily Podcast all about IPFS and his contribution.

Or if you prefer reading, you can view the transcript of the IPFS software engineering daily interview with David Dias.

Karl Floersh

Software daily discussion on Decentralization.

Juan Benet

juan-benet image

Juan Benet, creator of IPFS and Filecoin, founder of Protocol Labs.

Juan Benet is the inventor of the InterPlanetary File System (IPFS). His goal is to make the web faster, safer, and more open. He studied Computer Science at Stanford University.

IPFS is a large open source revolution focused on security, and re-decentralizing the web.

Juan is the founder of Protocol Labs, the internet tech R&D lab that develops IPFS and Filecoin.

Youtube talk on IPLD called Enter the Merkle Forest. The concept is all about content and data moving on the web. The main data structure behind it is a Merkle Tree, which is a computer science concept that isn’t the easiest to understand.

He also has an interview on Software Engineering daily.

Government censorship

If the local government of a client blocks the original hosting server, the client can fetch the content from another node that the government hasn’t black listed. This can have earth changing consequences when it comes to dissemination and access to information on a global level.

Create Your Own IPFS Node

To join the network you have to create a public and private key pair. This will then register you on the IPFS network as a contributing node.

First your node will check your Local Area Network or LAN to find other local nodes. This is done using a multicast DNS or Domain Name System.

Resources, Demos and Tutorials

Collaborative Editing Application with IPFS using CRDT video on youtube.

Create a React App to Store Data on IPFS

Conclusion

This type of technology can change the world, so if you’re in the field, it’s a good idea to look into it, as you may be building on it in the future.

Stay tuned as more IPFS related article will be published here as I learn more.

Tweet

Filed Under: Search

Newsletter

Tags

Backlinks Bing Search Bots Browser Currency Design Email Facebook Files Funding Gmail Google Analytics Google Apps for Work Google Maps Google Now Google Search Internet Link Building LinkedIn Phone Photoshop Productivity SEO Server Social Media Twitter Websites WordPress Writing

Recent Articles

  • IPFS? What is the InterPlanetary File System? March 25, 2020
  • How Your Security on the Internet and Web Browser Data, Cookies, Cache, and Sessions Really Work. May 24, 2018
  • Why You Should Disavow Spam Domain Links to Your Site March 16, 2018
  • Is Follow and NoFollow Important When Link Building? March 16, 2018
  • Measure Social Media with Klout February 9, 2018
  • Amazon Echo What can it do? February 9, 2018
  • Facebook
  • LinkedIn
  • Twitter

Learn Internet Grow © 2023