Overview

ADAPT’s backup storage system gives ADAPT nodes the ability to create encrypted backups of ADAPT packets and restore (roll back) packets to previous states.

Understanding the Backup Process

To understand which data needs to be stored in the backup storage, we need to first understand how ADAPT packet backups work. The MUFL language employs a transactional model, meaning the only way to transition a packet from one state to another is to perform a transaction within the packet. Transactions are always fully deterministic, meaning that from a specific packet state, the same transaction with the same input data always leads to the same result state. Thus, if the backup storage system retains the transactions performed in a packet and the packet’s initial state, ADAPT can replay the transactions to accurately recreate the packet’s correct result state. These items are essentially what gets stored as the backup.

To restore a packet to a previous state, ADAPT starts with the packet’s initial state and applies the transactions one-by-one, stopping at the desired restore point, effectively rolling the packet back to the previous state.

If only the initial packet state and the list of all transactions performed in the packet get stored, the restore process eventually becomes too time consuming due to a high number of transactions that have to be replayed. To improve performance, the backup storage system occasionally saves intermediate packet states called snapshots. For example, if a packet snapshot get saved every 100 transactions, at most 100 transactions are ever needed to restore the packet state. When restoring a packet from backup, the system starts with the most-recent snapshot taken before the desired restore point and then applies the transactions leading from the snapshot state to the desired state.

As previously mentioned, each transaction is fully deterministic, However, for security and reliability, the entire system as a whole is not deterministic. So, each transaction receives random entropy and a timestamp. By generating entropy for each transaction using a cryptographically-secure random number generator, the system’s reliability is assured. But entropy affects the final state of the transaction. So, the entropy and timestamp are also included in the replaying of each transaction, keeping the state transition of the packet entirely deterministic.

Thus, the backup storage holds snapshots and transaction logs that include entropy and timestamps.

The data stored is encrypted. Therefore, backups are only supported for environments that provide the ability to cryptographically secure the data encryption process, such as AWS Nitro Enclaves.

Data storage is implemented with an interface for communication with ADAPT nodes. The actual storage implementation can be of your choosing, as long as it correctly implements the provided TypeScript interface. Possibilities include an S3 bucket, an SQL database, or any other solution.

The data storage source files reside in the ./utilities/storage directory in the main git repository.

A script for hosting local data storage resides at /mufl/utilities/executables/dist/local_data_storage.js in the ADAPT Docker development kit. The script runs within the host file system and creates a hierarchy of directories to store the data in the host file system (rather than an S3 bucket or database). Local data storage is mainly used in tests.