Just-in-time connectivity for very large neuronal networks

Memory storage remains a limitation when running very large neuronal networks (VLNN). With number of neurons n, connectivity storage grows as n2. With connectivity densities of 0.1–10%, 1e6 neurons will require 1e9-1e11 synapses. A single connection requires at least an associated weight and delay as well as additional pointers or offsets to store the connectivity matrix. Conservatively, this will require 10 bytes (e.g., 2 floats and 2 chars) which will then bring the total synaptic memory load to 10 GB-1 TB. The former value may be barely executable on a single large machine.We have exploited an algorithmic space-time trade-off to build large event-driven artificial-cell simulations in the NEURON simulator by utilizing just-in-time connections (JITCONs) that are generated at the time of presynaptic cell spiking. JITCON utilizes a presynaptic-cell-specific random-number-generator seed based on presynaptic-cell serial number that permits it to generate a list of postsynaptic cell targets on the fly, and seeds based on a multiple of presynaptic-cell and postsynaptic-cell serial numbers for generating weights.We have utilized the JITCON algorithm to readily run simulations of >2e6 neurons. These simulations include a moderate level of cellular detail with AMPA, NMDA, GABAA and GABAB synapses, as well as multiple intrinsic properties such as bursting, depolarization blockade and an afterhyperpolarizing ``channel.'' Note that these are event-driven simulations and therefore do not utilize continuously integrated compartmental neurons. Since these simulations are event-driven, there is no overhead unless there is activity: simulation time varies widely depending on the level of network activity. An active network of 1.2e5 cells with >8.9e6 synapses, generating >1.1e7 spikes in 1 s simulation time, took 32.3 minutes to run on a 2.4 GHz AMD Opteron processor. Large, active simulations still develop space problems due to the need for a variable-size queue to accommodate varying delivery delays. This limitation is minimized by restrictions on the range and variability of permitted delays.We have begun to explore algorithms that permit a nuanced approach to the space-time trade-off. We permit individual presynaptic cells to store their list of postsynaptic targets in a compressed format. This additional storage can be turned on or off on a per-cell basis. We will explore making this storage dynamic so that a cell can maintain its connectivity list during a period of high activity and then return the memory when its activity is reduced. An additional direction for future development will be the incorporation of an entire encapsulated artificial-cell network as an independent piece of a compiled code (a mod file in NEURON). Such a network module could then be plugged in to other network modules or to a more detailed network that used compartmental models or compartmental/artificial cell hybrids, running in the main NEURON simulator. Running such simulations on parallel supercomputers will permit execution of very-VLNNs of order 100 million neurons.