About decentralized swarms of asynchronous distributed cellular automata using inter-planetary file system’s publish-subscribe experimental implementation

This research describes the simple implementation of asynchronous distributed cellular automata and decentralized swarms of asynchronous distributed cellular automata built on top of inter-planetary file system’s publish-subscribe (IPFS PubSub) experimentation. Various publish-subscribe(PubSub) models are described. As an illustration, two distributed versions and a decentralized swarm version of a 2D elementary cellular automaton are thoroughly detailed to highlight the simplicity of implementation with IPFS and the inner workings of these kinds of cellular automata (CA). Both algorithms were implemented, and experiments were conducted throughout five datacenters of Grid’5000 testbed in France to obtain preliminary performance results in terms of network bandwidth usage. This work is prior to implementing a large-scale decentralized epidemic propagation modeling and prediction system based upon asynchronous distributed cellular automata with application to the current pandemic of SARS-CoV-2 coronavirus disease 2019 (COVID-19). This is an open access article under the CC BY-SA license. and communication increas-ing of interest, with a wide array of problems to solve and an even wider array of applications from free-speech protection to climate change modeling This article described a simple implementation of an asynchronous distributed cellular automata and a decentralized swarm of asynchronous distributed cellular automata built on top of inter-planetary file system’s (IPFS) [3] publish-subscribe (PubSub) experimentation. After briefly describing notions of cellular automata (CA), IPFS, and PubSub protocols, two distributed versions and a decentralized swarm version of a simple 2D cellular automaton are detailed in order to highlight the simplicity of implementation with IPFS and the inner workings of these kinds of CA. The algorithms described in this paper are intentionally straightforward to focus on the simplicity of such asynchronous and decentralized PubSub systems. Implementation and preliminary results were obtained with experiments conducted across five datacenters throughout France with Grid’5000 testbed. This paper is motivated as preliminary work to build a very large scale distributed computing application that models population dynamics and predicts de-mography at city/region/country level, with multi-level and inter-level interactions, in the scope of the current SARS-CoV-2 coronavirus disease 2019 (COVID-19) pandemic.

INTRODUCTION Decentralization and distribution of computing and communication systems represent an ever increasing topic of interest, with a wide array of problems to solve and an even wider array of applications from free-speech protection [1] to climate change modeling [2]. This article described a simple implementation of an asynchronous distributed cellular automata and a decentralized swarm of asynchronous distributed cellular automata built on top of inter-planetary file system's (IPFS) [3] publish-subscribe (PubSub) experimentation. After briefly describing notions of cellular automata (CA), IPFS, and PubSub protocols, two distributed versions and a decentralized swarm version of a simple 2D cellular automaton are detailed in order to highlight the simplicity of implementation with IPFS and the inner workings of these kinds of CA. The algorithms described in this paper are intentionally straightforward to focus on the simplicity of such asynchronous and decentralized PubSub systems. Implementation and preliminary results were obtained with experiments conducted across five datacenters throughout France with Grid'5000 testbed. This paper is motivated as preliminary work to build a very large scale distributed computing application that models population dynamics and predicts demography at city/region/country level, with multi-level and inter-level interactions, in the scope of the current SARS-CoV-2 coronavirus disease 2019 (COVID- 19) pandemic.

. Cellular automata
A cellular automaton is a dynamical system composed of a finite lattice of cells with local and straightforward communication capabilities. Each cell has a finite number of states evolving in discrete steps through time, and each cell state depends on the state of its neighbourhood. A transition function processes state changes. Originating from Ulam's works on dynamical systems [4], CA directly contributed to Von Neumann's theory of self-replicating automata [5]. CA and distributed cellular automata (DCA) represent an extensive field of study and notably interesting modelling and simulation tools for contemporary topics ranging from submicroscopic physics [6] to large scale macroscopic phenomena [7] such as climate change [2]. This paper will implement asynchronous distributed and decentralized swarms of cellular automata through IPFS PubSub capabilities.

Inter-planetary file system
Inter-planetary file system (IPFS) is a multipurpose, distributed, peer-to-peer, version-controlled file system with no single point of failure [3]. IPFS engenders a global Merkle directed acyclic graph data structure (Merkle DAG) [8], with a content-addressing storage block architecture driven by distributed hash tables (DHT), a block exchange system and a self-certifying namespace [3]. Stored content inside IPFS is accessible via content identifiers (CID) hyperlinks. Currently, IPFS is used for a wide variety of applications, such as distributed web applications and serverless applications [9], telecommunication, cloud data storage networks [10], content delivery networks (CDN), and blockchains [8], and even a crypto-currency based large scale decentralized file storage network called Filecoin [11], [12]. Its trustless and decentralized structure may lead to the development of a censorship-resistant and permanent web [1].

PubSub: IPFS and Libp2p implementations 2.3.1. About PubSub
Publish-subscribe models (PubSub) consists of asynchronous distributed and independent nodes, where each node can publish events or subscribe to related topics or contents through an overlay communication infrastructure [13]. Focusing on topic-based PubSub systems, when a node publishes an event to a related topic, all nodes that subscribed to this topic receive it asynchronously. Publishers and subscribers preserve their anonymity, as they do not interact directly and do not have to know each other to communicate. Specifically, the section below describes three topic-based PubSub implementations: IPFS FloodSub, Libp2p GossipSub, and Libp2p EpiSub.

IPFS FloodSub
FloodSub, also known as DumbSub or PubSub-Flood, is the first and the most simple PubSub implementation experiment in IPFS/Libp2p. FloodSub is based upon message routing by network flooding with no CastTree forming [14], and ambient peer discovery by the use of external distributed hash tables (DHT) [15]. Concerning message broadcasting on small networks, FloodSub has low latency and is ideal for instant messaging applications. However, it cannot scale to more significant sized networks due to its high overhead and bandwidth consumption.

Libp2p GossipSub
GossipSub is a gossip-based publication-subscribe protocol [16] relying on a mesh construction and a score function [17]. The network structure uses two types of bidirectional peering: full-message peerings, where peers send entire messages to sparsely connected peers called mesh members. The second type is metadata-only peerings, where peers gossip about message availability and maintain full-message peerings [18]. In a GossipSub network, any peer can change their peering type from full-message to metadata-only (pruning) and conversely (grafting). Additionally, each node scores its peers based upon each peer behaviour, and then this scoring limits message transmission only to peers reaching a certain score threshold [19]. Nodes can also publish to unsubscribed topics via fan-out peering. In this unidirectional mechanism, they send their message to 3 randomly picked peers that subscribed to the topic, called fan-out peers, and then redistribute the message to the network [18]. Thus by design, GossipSub is highly scalable, reliable, fast and efficient, resilient and attack-resistant [17].

Libp2p EpiSub
EpiSub is a proximity aware mono-source multicast optimized PubSub protocol [20] implementing epidemic broadcast trees Plumtree protocol [21] (Gossip-based spanning tree construction, tree repair and optimization). HyParView membership protocol [22] manages EpiSub peers. Each node has two different views in this protocol: a small active one that engenders a message-passing overlay. A larger passive one maintains active view network resilience in case of node failure. EpiSub implements GoCast proximity-aware overlay scheme [23] which dynamically maintains near neighbourhood connectivity, i.e. low latency neighbours, by a near degree node confinement method during add and drop connection phases. Libp2p EpiSub comprises two main protocols: the broadcast protocol (publish) that uses lazy multicast tree construction through Plumtree's epidemic broadcast [21]. The second protocol is membership management (subscribe), which maintains active and passive peers lists for a related topic. Eager and lazy active peers are distinguished, eager ones actively disseminating new messages at the edges of the multicast tree, and lazy ones only gossiping about message summaries and maintaining the multicast tree [20]. Multicast tree and peer proximity optimization [23] constantly optimize transmission latency and propagation latency. An implementation of EpiSub is currently under active development in the scope of this project.

Used cellular automata and conventions
The CA uses Conway's Game of Life: two states, grade IV [24], totalistic, Moore's neighborhood driven [25], elementary cellular automaton. This CA will run on a 2-dimensional circular grid. Each cell ran independently and asynchronously, as subscription events trigger its transition function: its own cell for the neighbour publishing version and its eight neighbouring cells for the neighbour subscribing version. A generic PubSub employed functions pubsub.pub as publish function and pubsub.sub as subscribe function. Publish function takes for argument the name of the topic and the message to publish. Subscribe function takes for argument the name of the topic to subscribe and a callback function that processes received messages. These two generic functions can be adapted to match FloodSub, GossipSub, and EpiSub versions. Each published message reflects the cell state or a start action request. Message can be 0 (dead cell), 1 (alive cell) or 2 (start request). Any cell of the CA can receive a start request, then the triggered cell broadcasts its state to its neighbours and effectively starts the CA. Let N ≥ 3, the CA will run on a N × N circular grid, and let C a cell of coordinates (x, y) ∈ N × N . Thus the pubsub topic of C will be "cell-x-y". For performance comparison purposes, two DCA versions, a neighbour publishing version (NP) and a neighbour subscribing version (NS), detailed below and then implemented.

Neighbour publishing version
In this version, on a N × N CA, N ≥ 3, each cell has one topic subscription, thus N 2 total subscriptions, and at each round, a cell publishes to its eight neighbours as shown in Figure 1(a), thus 8 × N 2 publications per round. Algorithm 1 shows a single cell class, instantiating a cell object member of the CA. The initialization parameter coord is in the form (x, y) ∈ N × N , the state parameter is either 0 or 1 and the length parameter is N. The neighb variable is an array of the cell neighbour coordinates, thus an array of length 8 as Moore's neighbourhood [25] is used. Variable alive neighb is the number of alive neighbours for this current round, and current neighb is the number of messages received by its neighbourhood. Variable subscribe keeps the pubsub mechanism, processes each event received by updating the cell and broadcasting each cell state update to its neighbourhood.

Neighbour subscribing version
In this version, on a N × N CA, N ≥ 3, each cell has 8 topics subscriptions as shown in Figure 1(b) thus 8 × N 2 total subscriptions, and at each round a cell publishes to its own topic, thus N 2 publications per round. Algorithm 2 shows a single cell class, instantiating a cell object member of the CA. The initialization parameter coord is in the form (x, y) ∈ N × N , the state parameter is either 0 or 1 and the length parameter is N. Variable alive neighb is the number of alive neighbours for this current round, and current neighb is the number of messages received by its neighbourhood. Variable subs keeps the pubsub mechanism as an array of neighbourhood subscriptions that processes each event received, updates the cell and broadcasts the cell state update to its current topic.

Publish-subscribe functions
PubSub functions of both CA versions used above can be described as: i) On a Neighbour publishing scheme; ii) Each cell uses the publish function to send a message to its neighbours; and iii) The subscribe function listens to its topic. In Algorithm 3, these are neighb publish and cell subscribe functions. On a neighbour subscribing scheme, each cell uses the publish function to update its topic and the subscribe function is used to listen to its neighbours' topics. In Algorithm 3, these are cell publish and neighb subscribe functions. It is essential to mention that the cell function parameter is just a reference to the cell object and not the cell object itself.

Message processing function
The message processing function plays the role of an asynchronous CA transition function and is used for both versions. As shown in Algorithm 4, this function takes for argument a cell reference and a pubsub publication callback, and then returns a function that processes a message, starts or updates a cell given its current neighbourhood state according to Conway's game of life rules: any alive cell surrounded by two alive  neighb ) mod len ν ← (len + y + ∆[β]) mod len neighb.push((µ, ν)) end if end for return neighb end function

CA initialization and known limitations
The CA can be initialized by instantiating the N ×N cells and then send a message msg=2 to any topic "cell-x-y" where (x, y) ∈ N × N . The triggered cell then propagates its current state and effectively starts the whole CA in an asynchronous domino effect. This straightforward implementation has known limitations, as a cell does not keep track of its current round and thus does not know if received messages are related to its current or next round. The following CA implementation tackle this problem.

Decentralized swarms of distributed CA 3.2.1. Description and conventions
For large scale purposes, the asynchronous distributed CA divided into swarms of autonomous smaller CA with inner-swarm and inter-swarm communication capabilities. Communication is necessary inside the swarm and between the cells located at the edges of two distinct swarms. This will be achieved through local cell-to-cell and global swarm-to-swarm communication channels as shown in Figure 2. Each cell has to keep track of its current round, and process its neighbourhood messages according to its round number. At round 0, each cell is unaware of its neighbours' swarm. Each cell identifies itself to its neighbourhood during this discovery phase through direct messaging as in the NP DCA version previously described. From round 1 begins the processing phase, where each cell is aware of the corresponding swarms of its neighbourhood, and communication is provided through inner-swarm and inter-swarm broadcast.

Cell and swarm objects
Algorithm 6 describes a single cell class, instantiating a cell object member of a swarm. The initialization parameter coord is in the form (x, y) ∈ N × N , state parameter is either 0 or 1, swarm parameter is a reference to the related swarm of the cell and length parameter is N. Variable neighb is an array of 8 neighbouring cells where each item has its swarm id, cell coordinates, and its two last state rounds stored. Variable subscribe keeps the pubsub mechanism at a local level, processing each event received, updating the cell and broadcasting cell state updates to its neighbourhood. Algorithm 7 describes a single swarm class, instantiating a swarm object member of the CA. The initialization parameter swarm id represents the id of the swarm, and cell array parameter assigned to cells variable is the array of cell objects from the swarm. Variable subscribe keeps the pubsub mechanism at both local and global levels, processing and dispatching each event received depending on its origin and destination.

Publish and subscribe functions
PubSub functions enable local and inter-swarm communication. On a local level, each cell uses the publish function to send a message to its swarm, and the subscribe function listens and processes messages from its topic. In Algorithm 8, these are the functions cell publish and cell subscribe. On a global level, each swarm uses the publish function to dispatch a received message, either to its cells or to another swarm, and the subscribe function listens and processes messages from its topic. In Algorithm 8, these are the functions swarm publish and swarm subscribe. Functions pubsub.sub and pubsub.sub are generic enough to implement either FloodSub, GossipSub or EpiSub PubSub models.

Cell message broadcast
The cell message broadcast procedure described in Algorithm 9 sends a message to all neighbours of a given cell through swarm publish procedure.

Swarm processing function
The swarm processing function plays the role of an asynchronous message router. As shown in Algorithm 10, this function takes for argument the current swarm reference. It returns a function that processes each message as follows: if the message is heading to an inner cell, the target cell processes it directly. If the message is heading to a known swarm, the pubsub swarm publish function will send it. In the initialization phase, the swarm destination is unknown. Thus the message is directly dispatched to the corresponding cell via pubsub cell publish function.

Cell processing function
The cell processing function plays the role of an asynchronous transition function. As shown in Algorithm 11, this function takes for argument a cell reference and a pubsub publication callback and returns a function that processes a message, starts or updates a cell depending on the message received. If the received swarm id is -1, the cell identifies itself on the network, broadcasting its swarm id, coordinates and current state to its neighbourhood. In other cases, the received message is processed in the neighbour array. Suppose the cell received enough neighbours for its round (processed via alive n function). In that case, Conway's game of life rules are applied, the neighbour state array is reset for this round (via reset neighb procedure), and the cell message broadcast procedure sends the next state to its neighbours. PRELIMINARY RESULTS -PERFORMANCE COMPARISONS 4.1. Implementation and experimentation testbed DCA and DSDCA algorithms were implemented in the NodeJS framework and interfaced with a tweaked version of IPFS, compatible with Linux x64 and OS X platforms. Source code is freely available on GitHub [26]. The following performance measures were carried out using Grid'5000 testbed [27], a large-scale testbed for experiment-driven research supported by a scientific interest group (GIS) hosted by Inria, including CNRS, RENATER, and several French Universities as well as other organizations. Only IPFS bandwidth was measured during preliminary experiments, but inconsistencies between IPFS and network bandwidth, including TCP/IP overhead, were observed. It was then decided that total network bandwidth usage, including TCP/IP overhead, was the most relevant resource to monitor.

DCA NP/NS Grid'5000 experimentation protocol
Experimentations with DCA NP and NS versions were carried out on various clusters from 5 different sites (Lyon, Lille, Nancy, Nantes, Grenoble). Each experiment was launched for 100 rounds on five machines in parallel, and network bandwidth was measured with vnstat. Various settings were used, such as the number of nodes (from 10 to 226), FloodSub and GossipSub router protocols, and NP / NS DCA versions. At the end of each experiment, the total network bandwidth used for IPFS nodes bootstrapping and CA processing was measured, and the average network bandwidth per node was processed. Each DCA ran on individual machines, and network bandwidth was measured through vnstat software in interactive mode, monitoring each machine's loopback (lo).

Distributed cellular automata performance
DCA performance was measured during 250 experiments with several nodes ranging from 10 to 226. The focus is kept on results for experiments with more than 50 nodes. As shown in Figures 3(a) and 3(b), there is no meaningful difference in network bandwidth between both PubSub protocols when comparing NP or NS implementations, 0.1% for NP version and 0.3% for NS version on average. Contrarily, it is much relevant to compare NP and NS implementations running with the same PubSub protocol, as shown in Figure 4(a) and 4(b). Technol   ISSN: 2252-8776  ❒  41 On average, there is a 24.7% bandwidth difference for Floodsub protocol and a 24.3% bandwidth difference for GossipSub protocol. In any case, and contrarily to what was supposed theoretically, the NP version seems to be the most bandwidth-efficient algorithm, even independently from FloodSub or GossipSub protocol. Experimentations for Decentralized Swarms of DCA were carried out across five sites with machines from various clusters with 10 GBps connectivity (Nova-Lyon, Chiclet-Lille, Gros-Nancy, Ecotype-Nantes, Dahu-Grenoble). Each swarm was running on a specific site, and network bandwidth was measured with vnstat software. Nova was used as the IPFS private network and swarm bootstrapping node, Chiclet, Gros, Ecotype, and Dahu as Swarms of DCA nodes. Various settings were tried, such as the number of swarms (3 and 4), nodes (from 13 to 404), and FloodSub and GossipSub router protocols. Each experiment was run five times for 1000 rounds. At the end of each experiment, the total network bandwidth used between sites was measured, and the average network bandwidth per node was processed.  Figures 5(a) and 5(b). On average, there is a 31.3% bandwidth difference for FloodSub vs GossipSub protocol on three swarms bootstrap phases, and 11.7% bandwidth difference for FloodSub vs GossipSub protocol on four swarms bootstrap phases.   About 500 different experiments were run during two weeks, about 43,000 IPFS nodes were fired up, and 30,000,000 rounds of CA were computed.

5.
CONCLUSION AND FURTHER WORKS This article introduced an experimental implementation of simple asynchronous distributed cellular automata and decentralized swarms of asynchronous distributed cellular automata driven by IPFS and Libp2p PubSub. PubSub functions genericity provides the possibility to switch between FloodSub, GossipSub and Episub protocols. Experiments were carried out on Grid'5000 testbed for either NP/NS DCA and 3/4 swarms DSDCA. Preliminary results showed the sustainability of NP version over NS DCA and FloodSub protocol over GossipSub for DCA and DSDCA algorithms.
Further experiments will be conducted with a larger number of Swarms and Nodes, and a comparison of protocol performance will be made with the EpiSub protocol. EpiSub implementation over Libp2p/IPFS is currently under active development, as an automated DSDCA experiment launcher and bandwidth monitoring over Grid'5000, and a real-time asynchronous distributed cellular automata visualization system. This work is prior to implementing a large-scale decentralized epidemic propagation modelling and prediction system based upon asynchronous distributed cellular automata applied to the current SARS-CoV-2 (COVID-19) epidemic. This system will use a hybrid adapted version of SIR models driven by DSDCA and fed by worldwide data, estimating population dynamics on various factors such as mobility, infection rates, vaccine rates, recoveries and deaths. In this DSDCA system, individual nodes will simulate population dynamics in cities, swarms for regions and upper-level swarms for countries. Although cellular automata can be computed faster in a centralized manner, decentralization will enable to free from memory and processing power limits induced by centralization in the context of such a very large scale simulation application.