# AWS TiDB TiOperator tool
This tool helps you automatically setup TiDB cluster via TiOperator in AWS.

## Architecture
The tool will spin up the following resources for you 
- A new VPC with public and private subnets in each AZ
- A EKS cluster in the private subnets
- EKS managed node groups for tidb, tikv, pd and amdin in each AZ
- TiDB cluster in the EKS cluster via TiOperator
- Public NLBs and access to TiDB dashboard and grafana
- Private NLB and access to TiDB 
- Jump server in public subnet, with SSH access and ready to run demo scripts

# Prerequisites
- Install [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
- An AWS account
- Your AWS credentials. You can [create a new Access Key on this page](https://console.aws.amazon.com/iam/home?#/security_credentials)
- Install [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
- [Configure AWS crendential](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) in aws cli

# Simple run through
Automatically create tiup cluster in AWS Singapore region. If you want to switch to a different region, please check [Specify AWS region](#Specify-AWS-region)

## Initialize and create the TiDB cluster 
```bash
# checkout github repository 
git clone https://github.com/pingcap/tisample.git
# go to the tiup tool directory
cd tisample/tidb-demo-aws/tioperator
# run terraform commands to spin up AWS resources
terraform init
terraform apply
```
You should see similar results as below after successfully applied
```bash
Apply complete! Resources: 70 added, 0 changed, 0 destroyed.

Outputs:

jump_ssh_login = "ssh -i ~/.ssh/tidb_demo centos@18.136.105.205"
mysql_command = "mysql -h a82ad27e9a09e436590bf730f59cdc3a-397e8113b17bc78e.elb.ap-southeast-1.amazonaws.com -P 4000 -u root"
tidb_dashboard_url = "http://ac12873c84cbe4c9380beb1ac65373af-07dd3695de697951.elb.ap-southeast-1.amazonaws.com/dashboard"
tidb_grafana_url = "http://ac99ba5d446014a14ba3b4d7940b61d0-cec0ff271e6863e1.elb.ap-southeast-1.amazonaws.com:3000"
```

## Verify the installation
```bash
# login to jump server
# NOTE: you can copy paste the value of "jump_ssh_login" from the terraform apply output 
ssh -i ~/.ssh/tidb_demo centos@<jump server address>

# login to mysql console from the jump server
# NOTE: you can copy paste the value of "mysql_command" from the terraform apply output
mysql -h <tidb nlb host name> -P 4000 -u root
```
You can also access TiDB grafana and dashboard through the output URLs.


## Cleanup
```bash
# go back to the local machine, and go to the tioperator dir
# run destroy
# NOTE: if the destory command fails, please just rerun the command, for more details, please check the "Known Issues" section
terraform destroy
```

# Advanced configurations
You can configure terraform local variables to adjust tidb cluster setup.

For all [configurations supported](#Supported-Configurations), there are 2 ways to change the configuration.
1. This can be done by updating the [variables.tf](./variables.tf) file
2. Specify in terraform commands e.g.
```bash
terraform apply -var "region=us-east-1"
```
## Supported Configurations
### Specify AWS region 
- description: spin up tiup cluster in the specified AWS region
- variable name: region
- default value: ap-southeast-1 

### AMI
- description: AMI map used for EC2 instances creation
- variable name: ami_id
- default value: official centos 7.9 [AMI list](https://wiki.centos.org/Cloud/AWS)
- note: Change only when you want to specify a working AMI other than official centos 7.9

### EKS version
- description: version of the EKS cluster to setup tidb in
- variable name: eks_version
- default value: 1.21 
   
### Instance types
- description: EC2 instance types used for jump, tidb, tikv and tiflash 
- variable name: {jump,tidb,tikv,tiflash,pd,admin}_instance_type
- default value: jump=c5.large, admin=c5.large, tidb=c5.large, tikv=r5.large, tiflash=m5.large, pd=m5.large

### Instance disk size 
- description: EC2 instance EBS disk size(in G) used for jump, tidb, tikv and tiflash
- variable name: {jump,tidb,tikv,tiflash,pd,admin}_instance_disk_size
- default value: tikv=500, others=20

### TiKV IOPS 
- description: EBS IOPS for tikv node 
- variable name: tikv_ebs_iops 
- default value: 3000 

### TiKV Throughput
- description: EBS throughput for tikv node (in MBs)
- variable name: tikv_ebs_throughput 
- default value: 125  

### Number of instances 
- description: number of EC2 instances in tidb, tikv and tiflash groups in each AZ
- variable name: {tidb,tikv,tiflash,pd}_instance_number
- default value: tiflash=0, others=1 

### Number of instances to scale out 
- description: number of EC2 instances in tidb, tikv and tiflash group to scale out in each AZ 
- variable name: {tidb,tikv,tiflash}_instance_number_scaleout
- default value: 0 

# Known issue:
EKS API sometimes timeout, causing terraform apply/destory to fail
- short term solution: rerun the terraform command
- long term solution: in progress, waiting for pull request to be applied
https://github.com/hashicorp/terraform-provider-http/pull/29