Lynn Haney

GAIAN Database
A distributed federated database using a biologically inspired self-organization principle to minimize management.

What is GAIAN Database? GAIAN Database (hereafter referred to as GaianDB) is a small (four-megabyte) dynamic distributed federated database (DDFD) that was written in Java. GaianDB is an extension to Apache Derby that federates heterogenous back-end data sources (RDBMS databases, files, text indexes, and so on) using a logical table abstraction layer. However, its principal feature is its ability to autonomically discover and federate other GaianDBs, federating nodes such that a whole network of nodes can be formed. Its connectivity model is "biologically inspired" in that it strives to minimize network diameter and maximize connections to the most fit nodes. GaianDB advocates a flexible "store locally, query anywhere" (SLQA) paradigm. GaianDB also provides a wide variety of other features, such as dynamic re-configuration; an "explain" network-route query option; in-memory tables and indexing; MQTT (MQ Telemetry Transport) integration; and contextual metadata. GaianDB has already been used in complex, distributed, semantic join queries for text analytics applications and has drawn the attention of significant customers in the military. GaianDB combines multiple technologies to reach data wherever it resides, whether on sensors, file systems, text indexes or RDBMS. It also includes some concepts taken from federation techniques, mobile ad-hoc networks, grid computing, and autonomic computing technologies in order to manage the overall database with a minimum of human intervention. GaianDB's self-organizing implementation makes it easy to manage and to cope with failures or network topology changes, and it is easy to deploy GaianDB on thousands of devices and computers with no central management. An automatic query propagation mechanism allows an application to get to the specific data anywhere in the network just as if it were a local database. The fundamental research enabling the development of GaianDB was part of the USA-UK international technology alliance (ITA) research project. The research and subsequent development were performed by the Emerging Technology Services team in SWG Strategy and Technology. How does it work? After you unzip the GaianDB image on disk or on a USB drive, GaianDB is immediately ready to use and can propagate queries to other GaianDBs in the vicinity. As soon as you add new data sources locally by associating them with a logical table definition, GaianDB will be able to respond to SQL queries targeting that logical table and send the result sets back to the callers, wherever they are. A SQL API is available for managing logical tables, data sources, or specific, hard-wired connections to out-of-domain networks. GaianDB automatically recognizes other GaianDBs in the same network domain, so no manual connection is required in that case.