This article was originally published on The Daily Chain,7th January 2020.
“Let me be explicit: the current NKN node code, without any modification, will work with one billion full nodes without any problem.”
— Dr. Yiliun Zhang, NKN CTO, November 20 2019
New Kind of Network (NKN) is a project aiming to create the world’s most efficient, scalable and decentralized communication network.
Earlier this month, after over a year of courting Binance and following a spectacular landslide victory in the exchange’s revamped Community Vote, NKN was finally listed. In high-spirits it is a great opportunity for us to catch up on what other progress NKN has made.
In the past I have written a detailed and lengthily primer for NKN, and published an interview with its modest yet brilliant founder Yanbo Li.
The primer details the tech behind NKN with a focus on the bewildering world of cellular automata, a stunning method for modelling the world around us. Cellular automata reveal with startling results the truth of how simple rules can create complex systems and patterns. These systems exist in life and nature, and with the language of cellular automata we are better able to grasp how even the seemingly chaotic or random are borne out of simple rules: like a pattern on a shell or the leaves of a tree, or even the Universe.
For NKN, understanding the behavior of these puzzling contradictions is the inspiration for its revolutionary consensus mechanism, called MOCA (Majority vOte Cellular Automata).
Thanks to MOCA and NKN’s take on mining called Proof-of-Relay, unparalleled results have been achieved by having well over 20,000 full consensus nodes, more than Bitcoin and Ethereum combined.
As impressive as those numbers are, NKN’s CTO has spoken publicly about millions of nodes reaching consensus (in the interview he will speak of a billion full consensus nodes).
Stephen Wolfram, the award-winning computer scientist and physicist who has led research in this field, culminating in a book titled New Kind of Science, acts as an advisor to NKN. That book, and indeed its title, are the inspiration for NKN (New Kind of Network). Founder Yanbo Li spoke in interview about this subject, and a great many more.
It is a credit to Yanbo, Bruce, Yilun and the NKN team that they have a working relationship not only with Wolfram but also Whitfield Diffie. Whit is the co-creator of public key cryptography, without which technologies like HTTPS, SSL and Bitcoin couldn’t exist.
In this article I have the pleasure of speaking with Dr. Yilun Zhang, NKN’s CTO, in order to review what advancements the World’s #1 full node consensus network has made since I last spoke with them.
Hi Yilun, great talking to you again. Since I published the NKN Primer I can see from discord and github there has been much progress, both in NKN core and its ecosystem of dApps. Could you itemize and briefly describe the most significant of these for us?
Oh sure! Indeed there has been tons of progress recently, both in NKN core and applications. Let me try to summarize them briefly:
On the NKN node side, we have basically implemented most of our design, and it has been tested thoroughly in our testnet and mainnet. Specifically, since your last publication, the MOCA consensus has been improved a lot especially on the security side, and has been battle-tested in public mainnet with 20k+ community nodes!
P2P messaging and Pub/Sub: We have improved the protocol quite a lot and have seen many more applications using it. Probably the most popular one is D-Chat. It’s a purely decentralized IM that runs as a browser add-on and we are using it every day for our internal communication. Also the NKN shell (nsh) is a much more convenient and safer replacement to ssh in some cases and we have been using it in production in nCDN node management. There are more applications that are available, like nLove, and more to come, like NKN hotel.
nCDN: this is our decentralized CDN solution announced recently. Our website and nknx api server are already using nCDN now and we’ve seen a significant performance boost compared to before. We are also doing public beta testing and everyone is able to use nCDN in just a few minutes.
TUNA: TUNA is a platform that allows any network based application to be served to other people and priced by bandwidth usage. We haven’t publicly released it yet but internally we are already using it everyday for a few services.
Before continuing the interview, let’s take a closer look at some of the technologies mentioned by Dr. Zhang.
A New Security Model (nsh)
“NKN Shell (nsh): A new kind of remote shell that is more secure and convenient than SSH” was recently published on Hackernoon by Yilun, accruing over 25k views. The article explains the advantages of running SSH over NKN. The problems with SSH are explained by NKN’s CTO in the article. It boils down to the issues with secure key exchange and public facing IP addresses which expose you to attack. HTTPS is also mentioned since NKN removes the need for trusted Certificate Authorities like Verisgn.
The security of ssh relies on the assumption that a user knows the public key of the remote machine. This is because ssh uses an IP address for packet delivery and a public key for encryption. Since there is no intrinsic relationship between the IP address and public key, some prior knowledge or PKI (public key infrastructure) is required to prevent man in the middle (MITM) attack, otherwise a middle man is able to see and modify all communication between you and the remote machine. HTTPS faces the same problem and solves it by predefined trusted certificates. But for ssh, either you know the remote public key yourself, or you have to pray that no middle man is attacking you. This is especially true the first time you are establishing an ssh connection when ssh cannot detect remote public key changes.
“NKN Shell (nsh): A new kind of remote shell that is more secure and convenient than SSH”
How does NKN fix these problems? Yilun’s article explains:
NKN client address contains public key and is used for both routing and encryption, which enables end to end encryption without any PKI.
Remote machine does not need to have a public IP address or publicly accessible. The only requirement is that it has Internet access and can establish outbound http and websocket connections.
Any software, as long as it can use the NKN client, can run a command using nkn-shell-daemon. For example, nkn-shell-daemon is compatible with d-chat and you can run a command on your node using d-chat Chrome/Firefox plugin or our mobile app (in dev) that supports d-chat.
All NKN dApps benefit from this trustless secure setup. Whether the messaging dApp (d-chat), the dating dApp (nLove) or the upcoming hotel reservation dApp. Any application can take advantage of NKN’s security benefits by hooking up with the API. This is potentially a huge deal as IPV4 address space runs out.
D-Chat
An October Medium article published by NKN describes the decentralized messaging application in greater detail.
D-Chat is one of the highlights of NKN consumer applications, developed by community developer Lynn. It utilizes and showcases a wide spectrum of NKN’s platform products and SDKs, including:
– Use NKN’s decentralized PubSub service to deliver group messages
– Use Multi-client SDK to optimize the speed and latency of delivery
– Use secure end to end encryption to enable private chat called Whisper.
Unlike any other mainstream consumer chat applications such as WhatsApp, Facebook Messenger or WeChat, D-Chat does not use or rely on a single centralized server. The entire application lives on your browser plug-in, and all your data are safe on your own device.
D-Chat ships as a browser plugin for all Chromium based browsers. The features are too numerous to detail in this article but both public group chats and private one-to-one chats (“Whispers” mode) are supported.
It is like snapchat, an encrypted messaging service between two clients (client can be human or machine).
D-Chat can even be used to control IoT devices in combination with nsh.
Better Performance (nCDN)
However NKN also offers performance benefits over legacy networks as exemplified with the signing of a commercial contract with NETNIC for NKN’s Content Delivery Service, nCDN, announced October 8th. Quoting from the announcement:
NETNIC provides application centric cloud solutions for enterprise customers and is the largest value added reseller for Baidu cloud and for Tencent Cloud in Northern China. nCDN, or new kind of Content Delivery Network from NKN, is a competitive new product that is more reliable, simpler to deploy at scale, and more adaptive to load variations than traditional CDN solutions.
On November 4th NKN launched the public beta of nCDN at ncdn.io for businesses interested in testing the solution. NKN’s latest bi-weekly report elaborates on nCDN’s recent beta launch.
This new service uses NKN’s global shared network of servers across many different platforms including data centers by many of the top cloud providers such as AWS, Google Cloud, and DigitalOcean as well as edge locations including home WiFi routers, NAS (Network Attached Storage) devices, and more. The result is a CDN with many points of presence and embedded quality controls to ensure availability and performance.
nCDN is currently in limited beta for selected enterprise and Small Medium Business customers. If you are interested in trying nCDN for your business, please visit https://www.ncdn.io/ and apply for a free trial account.
For full details of nCDN, please check our nCDN launch news article. To learn more about how nCDN self service portal works, please check out our video nCDN Self Service Portal Tutorial.
Back in the interview I wanted to pick Yilun’s brain on these topics.
What prevents NKN node operators from carrying out man-in-the-middle (MITM) attacks?
In NKN we eliminate MITM attacks from root. The root cause of MITM attack is the mismatch between communication addresses (e.g. ip address) and encryption pubic key (pk). If, by whatever means, I get your communication address with a middle man’s pk, then the middle man can see the message content and modify the message without being noticed. In NKN, a client address contains the encryption pk, and the same address is used for both communication address and encryption public key, so it’s not possible to get the correct communication address with a wrong encryption pk. When someone sends a message, he will (by default) authenticate and encrypt a message end-to-end (AEAD) with a recipient’s pk, so any middle man, as long as he doesn’t have sender or receiver’s private key, cannot see the message content (guaranteed by encryption). If he modifies the message, the receiver will also know upon receiving it (guaranteed by authentication).
Why can we trust NKN nodes any more than an ISP?
From the above question we know that it’s technically impossible to perform MITM attack by anyone, including node operators. So we don’t have to trust them at all!
How does NKN achieve better performance than a traditional CDN and how is the performance measured?
There are a lot of metrics when we talk about performance. A traditional CDN is definitely better at single node throughput as their “node” is typically a whole data center, but nCDN is often closer to end users given a large enough edge network so it can achieve better latency. If we make the analogy that fetching a website’s content is like buying some groceries, then a traditional CDN is like going to a big supermarket farther away — it has room for more people but it takes longer to get there; while nCDN is like going to a small store near your house — it’s closer to you so it saves you time.
Before joining NKN, and in addition to completing your doctorates in Philosophy and Computational Neuroscience, you also researched cellular automata.
The human brain, I suppose, is like a network? Could you tell us a little more about this area of study and how it has impacted your development of NKN’s unique solution?
When I learnt about blockchain, consensus algorithm and everything about decentralization, I realized that the brain is a very typical decentralized system. Take humans for example, there are tens of billions of neurons in the brain, but typically each neuron is only connected to thousands of other neurons, a very tiny portion, and only communicates with those direct neighbors. There is no single neuron(s) that controls the whole brain, just like every other decentralized system, but the whole brain is in an ordered state and capable of handling extreme complex tasks. This is very similar to a consensus algorithm, but in massive scale and much more complicated goals. After I realized this, the problem becomes much easier, as there are many existing methods and tools in physics already to solve mathematically similar problems.
How have cellular automata been leveraged to create NKN’s MOCA consensus, and could similar results have been achieved without knowledge of cellular automata? Could you please describe NKN’s “secret sauce” (MOCA) in plain English for our readers?
MOCA is utilizing a very basic cellular automata (CA) rule but in a complex topology. There is never any knowledge “required” to solve the problem, but with knowledge and experience in CA, Ising Model, or something similar, it’s much easier to understand why local rule is capable of achieving global consensus and thus much easier to find the solution.
MOCA, short for Majority vOte Cellular Automata, can be explained in plain English much easier than most other consensus algorithms. Say we try to elect a president from two candidates. Initially everyone has his/her own opinion about who to vote for. Then everyone asks about his/her neighbors’ votes, and checks if his/her own vote is different from the majority of neighbors’ vote. If so, he/she will change opinion and makes sure his/her vote is aligned with the majority of neighbors’ vote and tell his/her neighbor “hey, I’ve changed my mind and now I’m voting for xxx”. If everyone keeps doing this, then after finite (actually very short amount of) time, everyone will now votes for the same candidate, and we have achieved global consensus efficiently with only local communication (between neighbors).
Since NKN’s launch Emin Gün Sirer and AVA Labs have developed a consensus algorithm called Avalanche, and IOTA another named Coordicide. The former may not be so similar at all to MOCA, but certainly the latter makes use of cellular automata. Can you briefly describe the chief differences between these technologies?
There are some fundamental difference between MOCA and Avalanche/Coordicide: we are blockchain based, while they are DAG based, so we need to reach global consensus per block interval while they just need eventual consistency between transactions. Being a blockchain rather than DAG gives us a few advantages.
We have mining rewards, which has attracted 20k+ community consensus nodes into the network in the early stage (just a few months) and greatly enhanced security and decentralization. Let’s see how long it takes (if ever possible) for a DAG to reach such a level of decentralization
Transaction order are well and easily defined and thus much friendlier to smart contracts. In addition to the blockchain-DAG difference, MOCA is based on a verifiable overlay topology, while Avalanche/Coordicide didn’t say much about how to choose neighbors/samples securely and verifiably, or if ever possible. This makes a huge difference on security as for such type of consensus, secure neighbor/sample selection is one of the most important factors affecting security.
In addition to tamper-proofing data running across the network, NKN also introduces a level of privacy Internet traffic is not usually afforded. Could you discuss this please?
There are a few additional privacy features in addition to the tamper-proofing data you mentioned. An NKN address is pseudo-anonymous, just like a Bitcoin address, so knowing about an NKN address will not leak any information about the user unless the link has been made in other ways. Also, an NKN address contains a public key, so there is no need for unencrypted protocol negotiation when sending data end to end and further reduces information leak.
How has it been meeting and working with Stephen Wolfram and Whitfield Diffie?
Smart, knowledgeable and nice. They are very smart (as I expected of course) and can understand what I thought very easily without redundant communication; they know a lot of stuff inside and outside their field; and they both are very nice people.
Do you really believe NKN can handle millions of nodes, and do you expect to see the network grow so much? What do you see in NKN’s future?
Let me be explicit: the current NKN node code, without any modification, will work with 1 billion full node without any problem
Because everything scales with O(log N) at most in NKN, from the current network size (20k node) to 1 million node, each node only consumes 40% more resource; from 20k to 1 billion node, each node only consumes 110% more resource. Given that current NKN node consumes just a tiny amount of resource, scaling is not an issue at all, but adoption is. The real problem we are trying to solve is, how to attract more users (clients), miners (nodes), and value into the network, and really become the decentralized network infrastructure in the future.
In the simplest terms, what makes NKN special?
NKN is the only project I know that provides a ready-to-use decentralized data transmission network, and it’s also the largest blockchain network with 20k+ full consensus nodes, and can further scale horizontally to any size.
Conclusion
NKN is progressing at impressive speed, and the scope of the vision is coming into focus. Information is Power, and the control of that information is true power.
The barrier-to-entry is nominal. To run an NKN node one need only possess an Internet connection and a low-powered device, even a consumer-grade router. No longer is mining a question of “Why would I?”, but rather “Why wouldn’t I?”
Blockchain history begins with P2P money, and NKN is pressing hard to open up P2P communications and applications using blockchain in a world where many fear that Privacy is Dead.
There is, however, a light at the end of the (nsh) tunnel. It is startling and brave, it speaks of freedom.
“The thing that got me started on the science that I’ve been building now for about 20 years or so was the question of okay, if mathematical equations can’t make progress in understanding complex phenomena in the natural world, how might we make progress?”
— Stephen Wolfram, chief designer of Mathematica and advisor to NKN
“The decisions we make about communication security today will determine the kind of society we live in tomorrow.”
— Whitfield Diffie, co-creator of public key cryptography and advisor to NKN
Thanks
Thanks to Dr. Yilun Zhang for taking time to answer my questions.
Thanks to Zheng “Bruce” Li for helping coordinate the interview.
Thanks to Lama for the original artwork.
Thanks to community member Tom for the NKN splash logo.
