diff --git a/docs/2.installation.md b/docs/a.installation.md
similarity index 100%
rename from docs/2.installation.md
rename to docs/a.installation.md
diff --git a/docs/3.using-repl.md b/docs/b.using-repl.md
similarity index 100%
rename from docs/3.using-repl.md
rename to docs/b.using-repl.md
diff --git a/docs/4.architecture.md b/docs/c.architecture.md
similarity index 87%
rename from docs/4.architecture.md
rename to docs/c.architecture.md
index b61069a86..bfe141817 100644
--- a/docs/4.architecture.md
+++ b/docs/c.architecture.md
@@ -3,16 +3,20 @@ id: architecture
title: Architecture
---
-Skytable is a modern NoSQL database that prioritises performance, scalability and reliability while providing a rich and powerful querying interface. We are generally targetting an audience that wants to build high performance, large-scale, low latency applications, such as social networking services, auth services, adtech and such. Skytable is designed to work with
-both **structured and semi-structured data**.
+Skytable is a modern NoSQL database that prioritises performance, scalability and reliability while providing a rich and powerful querying interface.
+We are generally targetting an audience that wants to build high performance, large-scale, low latency applications, such as social networking services,
+auth services, adtech and such. Skytable is designed to work with both **structured and semi-structured data**.
-Our goal is to provide you with a powerful and solid foundation for your application with no gimmicks — just a solid core. That's why, every component in Skytable has been engineered from the ground up, from scratch.
+Our goal is to provide you with a powerful and solid foundation for your application with no gimmicks — just a solid core. That's why, every component in
+Skytable has been engineered from the ground up, from scratch.
And all of that, without you having to be an expert, and with the least maintenance that you can expect.
## Fundamental differences from relational systems
-BlueQL kind of looks and feels like using SQL with a relational database but that doesn't make Skytable's internals the same, with the most important distinction being the fact that Skytable has a NoSQL engine! But Skytable's evaluation and execution of queries is fundamentally different from SQL counterparts and even NoSQL engines. Here are some key differences:
+BlueQL kind of looks and feels like using SQL with a relational database but that doesn't make Skytable's internals the same, with the most important
+distinction being the fact that Skytable has a NoSQL engine! But Skytable's evaluation and execution of queries is fundamentally different from SQL
+counterparts and even NoSQL engines. Here are some key differences:
- All DML queries are point queries and **not** range queries:
- This means that they will either return atleast one row or error
@@ -64,7 +68,7 @@ A `model` in Skytable is like a `table` in SQL but is vastly different because o
## Query language
-Skytable has it's own query language BlueQLTM which takes a lot of inspiration from SQL but makes several different (and sometimes vastly different) design choices, focused on clarity, speed, simplicity and most importantly, security.
+Skytable has its own query language BlueQLTM which takes a lot of inspiration from SQL but makes several different (and sometimes vastly different) design choices, focused on clarity, speed, simplicity and most importantly, security.
For example, Skytable's BlueQLTM *only* allows the parameterization of queries. All the queries you ran previously with strings and numbers directly were only possible because the REPL client smartly does the paramterization behind the scenes. This is done for security. You'll learn more about BlueQL next.
@@ -99,12 +103,15 @@ Skytable will use atleast as many threads as the number of logical CPUs present
## Networking
-Skytable its own in-house Skyhash protocol that is built on top of TCP enabling any programming language that has a TCP client to use it without issues. There are three phases in the connection:
+Skytable uses its own in-house Skyhash protocol for client-server communication. It is built on top of TCP, enabling any programming language that has a
+TCP client to use it without issues. There are three phases in the connection:
- Handshake: All auth data, compatibility information and other data is exchanged at this step
- Connection mode selection: based on the handshake parameters a connection mode is chosen and the server responds with the chosen exchange mode
- Data exchange: This is where the real querying happens
- Termination: there is no special step; just a `TCP FIN`
+You can [read more about the protocol here](protocol).
+
## Backwards compatibility
We make the promise to you that no matter what changes in Skytable, you will always be able to:
@@ -115,6 +122,7 @@ More technically:
- **For minor/patch releases**: The minor/patch is just in the name but it indicates that no data migration effort is needed. **No minor releases ever need data migration, and any migration is done automatically**
- **For major releases**: Major releases generally introduce breaking changes (just like the upgrade from `0.7.x` to `0.8.0` is a largely breaking change). **Major releases will either automatically upgrade the data files or require you to use a migration tool that is shipped with the bundle**.
- Definitions (closely following semantic versioning):
- - **A major release** is something like `1.0.0` to `2.0.0` or `0.8.0` to `0.9.0` (in development versions, 0.8.0 to 0.9.0 is a major version bump)
+ - **A major release** is something like `1.0.0` to `2.0.0` or `0.8.0` to `0.9.0` (in development versions, 0.8.0 to 0.9.0 is considered a major version
+ bump)
- **A minor release** is something like `1.0.0` to `1.1.0` or `0.8.0` to `0.8.1`
- **A patch release** is something like `1.0.0` to `1.0.1` or `0.8.0` to `0.8.1` (note that in development versions there is no distinction between a minor and patch release)
diff --git a/docs/15.deployment.md b/docs/deployment.md
similarity index 100%
rename from docs/15.deployment.md
rename to docs/deployment.md
diff --git a/docs/index.md b/docs/index.md
index 64578dd8a..c45cbd0b5 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -19,10 +19,12 @@ To develop using Skytable and maintain your deployment you will want to learn ab
- [**DCL**](blueql/dcl): Data control with BlueQL
- [**Querying**](querying): Introduces different query modes and when to choose a specific query mode
- [**System administration**](system):
- - [**Configuration**](system/configuration): Information to help you configure Skytable with custom settings such as custom ports, hosts, TLS, and etc.
- - [**User management**](system/user-management): Information on access control, user and other administration features
- - [**Global management**](system/global-management): Global settings management
- - [**Operations**](system/operations): Learn about administration operations
+ - [**Configuration**](system/configuration): Configuration modes (CLI, environment variables, configuration files) and options
+ - [**User management**](system/user-management): Account types, permissions, creating and managing multiple users
+ - [**Global management**](system/global-management): Learn how to check system health and manage the global state of your database instances
+ - [**Disk usage**](system/disk-usage): Understand disk usage and compaction
+ - [**Backup and restore**](system/backup-and-restore): Backing up data and restoring data from backups
+ - [**Data recovery**](system/recovery): Understanding data loss, mitigation and recovery options
- **Resources**:
- [**Useful links**](resources/useful-links): Links to helpful resources
- [**Migration**](resources/migration): For old our returning Skytable users who are coming from older versions
diff --git a/docs/13.libraries.md b/docs/libraries.md
similarity index 100%
rename from docs/13.libraries.md
rename to docs/libraries.md
diff --git a/docs/16.limits.md b/docs/limits.md
similarity index 100%
rename from docs/16.limits.md
rename to docs/limits.md
diff --git a/docs/14.benchmarking.md b/docs/load-testing.md
similarity index 100%
rename from docs/14.benchmarking.md
rename to docs/load-testing.md
diff --git a/docs/system/1.configuration.md b/docs/system/a.configuration.md
similarity index 99%
rename from docs/system/1.configuration.md
rename to docs/system/a.configuration.md
index 9fa5ea713..5fe3d2f35 100644
--- a/docs/system/1.configuration.md
+++ b/docs/system/a.configuration.md
@@ -44,7 +44,7 @@ To start the server with a configuration file, simply run `skyd --config [--from ]
+```
+
+- `--type=direct`: This specifies the kind of backup created. The `direct` type indicates that it's a simple copy of the data files and directories
+- `--to=`: This specifies where this backup is to be created
+- `--from ` *(optional)*: When this is not provided, the `backup` subcommand assumes that the current working directory is the installation directory. If you're running it from a different directory then set this option.
+
+**Example**:
+
+```sh
+skyd backup \
+ --type=direct \
+ --from=/var/lib/skytable \
+ --to=/mnt/backupnfsdrive/quick-backup-before-migration
+```
+
+:::info Backup types
+Note that in the future we may add more backup types including compressed archives or other modes. The only type of backup (specified using `--type`) is `direct` which clones the data files and directories. But you do not need to worry about this as the restore subcommand will take care of determining what kind of backup is being pointed to.
+:::
+
+### Backup protections
+
+The `backup` subcommand includes some protections to create consistent and valid backups. These include not allowing backups if the database is currently using the data files and some other parameters. If you need to override any of these parameters, then please check the help menu with `skyd backup --help`.
+
+## Restoring data
+
+To restore data from a backup, you can use the subcommand `skyd restore` as follows:
+
+```sh
+skyd restore --from= [--to ]
+```
+
+- `--from=`: Specifies the path to the backup
+- `--to ` *(optional)*: By default, it is assumed that the current directory is the installation directory. If not, set this option.
+
+**Example**:
+
+```sh
+skyd restore \
+ --from=/mnt/backupnfsdrive/quick-backup-before-migration \
+ --to=/var/lib/skytable
+```
+
+### Data restore protections
+
+The `restore` subcommand also has some safeguards in place that prevent you from accidentally restoring incorrect data. Some of these safeguards include:
+
+- **Backup has correct time signatures**
+- **Backup is compatible**
+- **Was created by the same host:** you will obviously need to override this when recovering from a crash and this should be okay to do. The reason this protection exists is in a situation where you're running a cluster and have multiple backups and accidentally restore from the wrong backup.
+
+If you need to override any of these conditions in special cases, then please check the help menu with `skyd restore --help`.
diff --git a/docs/system/operations.md b/docs/system/f.recovery.md
similarity index 77%
rename from docs/system/operations.md
rename to docs/system/f.recovery.md
index 7e5ec735d..9e2433955 100644
--- a/docs/system/operations.md
+++ b/docs/system/f.recovery.md
@@ -1,20 +1,8 @@
---
-title: Operations
+id: recovery
+title: Data recovery
---
-## Managing disk usage
-
-Over time, as you continue to use your database your database files will grow in size, as you would expect. However, sometimes database files may grow beyond an efficient size resulting in high memory usage or slowdowns. To counter this, Skytable uses internal heuristics to determine when a database file is "larger than needed" and automatically compacts them at startup.
-
-However, in some cases you may wish to perform a compaction regardless in order to reduce the file size. In order to do this you will have to run:
-
-```sh
-skyd compact
-```
-
-The server will then compact all files (even if a compaction wasn't triggered by internal heuristics) to their optimum size.
-
-## Data recovery
In the unforeseen event that a power failure or other catastrophic system failure causes the database to crash, the Skytable server will fail to start normally. Usually it will exit with a nonzero code and an error message such as "journal-corrupted." In such cases, you will need to recover the journal(s) and/or any other corrupted file(s).
diff --git a/docs/system/index.md b/docs/system/index.md
index 2205d473c..1dacdd0d7 100644
--- a/docs/system/index.md
+++ b/docs/system/index.md
@@ -8,7 +8,9 @@ In the following sections, we explore general system administration options with
Here's an overview of the different administration guides:
-- [**Configuration**](configuration): Understand how Skytable can be configured using command-line arguments, environment variables or a configuration file and what all configuration options are available
-- [**User management**](user-management): Learn about account types, permissions and how you can manage multiple users
+- [**Configuration**](configuration): Configuration modes (CLI, environment variables, configuration files) and options
+- [**User management**](user-management): Account types, permissions, creating and managing multiple users
- [**Global management**](global-management): Learn how to check system health and manage the global state of your database instances
-- [**Operations**](operations): Understand administrator operations tasks such as backups, recovery and more
+- [**Disk usage**](disk-usage): Understand disk usage and compaction
+- [**Backup and restore**](backup-and-restore): Backing up data and restoring data from backups
+- [**Data recovery**](recovery): Understanding data loss, mitigation and recovery options
diff --git a/docusaurus.config.js b/docusaurus.config.js
index 744cf1323..0fe12633d 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -163,8 +163,8 @@ module.exports = {
to: '/protocol/specification'
},
{
- from: '/system/recovery',
- to: '/system/operations#data-recovery'
+ from: '/system/operations',
+ to: '/system',
}
]
}]
diff --git a/sidebars.ts b/sidebars.ts
index 40556bde1..0469ff67f 100644
--- a/sidebars.ts
+++ b/sidebars.ts
@@ -27,7 +27,9 @@ module.exports = {
"system/configuration",
"system/user-management",
"system/global-management",
- "system/operations",
+ "system/disk-usage",
+ "system/backup-and-restore",
+ "system/recovery",
],
link: {
type: 'doc',