## Module 1: Deploying Backup & Restore and Preparing Data

### Step 1. Log into the remote Linux

+ Click [Terminal 1](ssh://monitor1-ssh#1) to open a new terminal connected to the virtual machine assigned for your hands-on lab.

### Step 2. Install `Backup & Restore`

  ```bash
  source .bash_profile
  ```

  ```bash
  tiup install br:v7.5.1
  ```

### Step 3. Extract the data

  ```bash
  unzip -u stage/tidb-admin-dataset-v2.zip -d stage/
  ```

  ```output
  Archive:  stage/tidb-admin-dataset-v2.zip
  ```

  ```bash
  ssh -i ~/.ssh/pe-class-key-${LAB:REGION_NAME}.pem ec2-user@${HOST_KV1_PRIVATE_IP} unzip -u stage/tidb-admin-dataset-v2.zip -d stage/
  ```

### Step 4. Read the file `solution-lightning-init.toml`

- The `solution-lightning-init.toml` file configures TiDB Lightning for data import. It specifies:
  - Logging level and log file location.
  - TiKV Importer settings, including incremental import, backend type, and sorted key-value directory.
  - Mydumper settings, such as the data source directory and table filters to exclude system schemas.
  - Task connection details in `[tidb]` section, including host, port, import task user, password, status port, and PD address.

  ```bash
  cat solution-lightning-init.toml
  ```

  ```output
  [lightning]
  level = "info"
  file = "tidb-lightning.log"

  [tikv-importer]
  incremental-import = true
  backend = "local"
  sorted-kv-dir = "/tmp/sorted-kv-dir"

  [mydumper]
  data-source-dir = "/home/ec2-user/stage/TiDB-Administration-exercise-data-v2/exercise-br"
  filter = ['*.*', '!mysql.*', '!sys.*', '!INFORMATION_SCHEMA.*', '!PERFORMANCE_SCHEMA.*', '!METRICS_SCHEMA.*', '!INSPECTION_SCHEMA.*']

  [tidb]
  host = "10.90.2.124"
  port = 4000
  user = "ltask"
  password = "q1w2e3R4_"
  status-port = 10080
  pd-addr = "10.90.2.84:2379"
  ```

### Step 5. Connect to the TiDB, and drop the `emp` database

  ```bash
  mysql -h ${HOST_DB1_PRIVATE_IP} --port 4000 -u root
  ```

  ```sql
  DROP DATABASE IF EXISTS emp;
  ```

  ```output
  mysql> DROP DATABASE IF EXISTS emp;
  Query OK, 0 rows affected (0.51 sec)
  ```

### Step 6. Create a user for the lightning task, if needed

+ For our import task user setting, check if user `ltask@'%'` exists.

  ```sql
  SELECT user FROM mysql.user;
  ```

  ```output
  +-------+
  | user  |
  +-------+
  | ltask |
  | root  |
  +-------+
  2 rows in set (0.00 sec)
  ```

+ If `ltask@'%'` does not exists, create `ltask@'%'` and grant all privileges.

  ```sql
  CREATE USER ltask@'%' IDENTIFIED BY 'q1w2e3R4_';
  ```

  ```output
  mysql> CREATE USER ltask@'%' IDENTIFIED BY 'q1w2e3R4_';
  Query OK, 0 rows affected (0.05 sec)
  ```

  ```sql
  GRANT ALL PRIVILEGES ON *.* TO ltask@'%';
  ```

  ```output
  mysql> GRANT ALL PRIVILEGES ON *.* TO ltask@'%';
  Query OK, 0 rows affected (0.05 sec)
  ```

  ```sql
  EXIT;
  ```

  ```output
  mysql> EXIT;
  ```

### Step 7. Import data into TiDB

  ```bash
  nohup tiup tidb-lightning:v7.5.1 -config solution-lightning-init.toml > nohup.out &
  ```

  ```output
  ...
  [1]+  Done  nohup tidb-lightning:v7.5.1 -config solution-lightning-init.toml > nohup.out
  ```

### Step 8. View the contents of the `nohup.out` file

  ```bash
  cat nohup.out
  ```

  ```output
  Verbose debug logs will be written to tidb-lightning.log

  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  # | CHECK ITEM                                                                                                             | TYPE        | PASSED |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  1 | Source csv files size is proper                                                                                        | performance | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  2 | checkpoints are valid                                                                                                  | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  3 | table schemas are valid                                                                                                | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  4 | Cluster is available                                                                                                   | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  5 | Lightning has the correct storage permission                                                                           | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  6 | sorted-kv-dir:/tmp/sorted-kv-dir and data-source-dir:/root/sample/init are in the same disk, may slow down performance | performance | false  |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  7 | local disk resources are rich, estimate sorted data size 343.6MiB, local available is 6.459GiB                         | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  8 | Cluster available is rich, available is 55.03GiB, we need 1.007GiB                                                     | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  |  9 | Cluster doesn't have too many empty regions                                                                            | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+
  | 10 | Cluster region distribution is balanced                                                                                | critical    | true   |
  +----+------------------------------------------------------------------------------------------------------------------------+-------------+--------+

  tidb lightning exit successfully
  ```