# Develop Infra API ### Structure * `./api`: Infra API and SDK, designed as a shared package across projects. * Anything outside `./api`: default providers of Infra API (including `aws` and `gcp`), does not supposed to be a shared package. ### Development ```bash # unit test make unit # integration test make integration # run all tests above make test ``` ### Running specific integration tests If you want to run one specified test, there are a few options: - By modifying code: add the prefix “FIt” on the test you want to run; this marks the test as focused. Ginkgo will skip other tests and will only run the “focused” test. See [focused-specs](https://onsi.github.io/ginkgo/#focused-specs) for more detail. ```go It("Example test", func(){ Expect(true).Should(BeTrue()) }) FIt("Example focused test", func(){ Expect(true).Should(BeTrue()) }) ``` - From the command line: specify focus if you want to focus on: ```bash make integration focus="Gardener base Network controller" ``` ### Locally Run #### Create a temporary directory ```bash export INFRA_CONFIG_DIR=$(pwd)/cfg mkdir $INFRA_CONFIG_DIR ``` #### Grab the manifests ```bash ./hack/grab-manifest.sh $INFRA_CONFIG_DIR cp -r ./manifests/* $INFRA_CONFIG_DIR/manifests/ ``` #### Get gardener kubeconfig ```bash BASE_CTX=`kubectl config get-contexts --output=name | grep dev-base-us-west-2` kubectl --context "${BASE_CTX}" get secret -n garden garden-kubeconfig-for-admin --template={{'.data.kubeconfig'}} | base64 -D > $INFRA_CONFIG_DIR/garden-kubeconfig.yaml ``` #### Create infra-api config ```bash PROJECT=$(whoami) ./hack/gen-infra-config.sh ``` #### Start infra-api with your provider name in locally Get infra-api kubeconfig ```bash kubectl --context "${BASE_CTX}" get secret api-server-kubeconfig --template={{'.data.config'}} -n infra | base64 -D > $INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml KUBECONFIG=$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml kubectl config set-cluster infra --server=https://us-west-2.infra.shared.aws.tidbcloud.com:443 ``` #### Build & Run infra-api >[!NOTE] > >If you want to runing GCP cluster, you need to run [gcloud auth application-default login](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login) to make provider can find the Application Default Credentials (ADC). ```bash PROJECT="$(whoami)" AWS_DEFAULT_REGION=us-west-2 KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" kubectl create namespace ${PROJECT} KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" ENABLE_WEBHOOKS=false aws-vault exec dev -- go run cmd/infra-provider/main.go --config "$INFRA_CONFIG_DIR/config.yaml" --enable-gitops-v2 ``` _Note: if you want to enable tracing, you can set the env variable `ENABLE_TRACING` to `true`, traces will be printed in stdout. If you want to send traces to Lightstep, you need to set the env variable `LS_ACCESS_TOKEN` at the same time._ _Note: why need set `ENABLE_WEBHOOKS=false`? because the Webhook cannot be connected locally._ #### forward gitops request ``` BASH "reconciler group": "cloud.pingcap.com", "reconciler kind": "Backup", "name": "log-backup-62561-01", "namespace": "nightly-ms-43603", "error": "get gitops br config: Post \"http://config-service.gitops:8090/api/v1/backup/configuration\": dial tcp: lookup config-service.gitops: no such host" ``` some operations depend on GitOps。We can use kubectl forward. switch context to dev base dev-base-us-west-2 and run this command. ``` bash kubectl port-forward -n gitops service/config-service 8090:8090 ``` #### Create a sample network object >[!NOTE] > the association between CRD and infra-api in `cloud.pingcap.com/reconciler: ${PROJECT}` and `$INFRA_CONFIG_DIR/config.yaml` ```bash export PROJECT=$(whoami) cat << EOF > "$INFRA_CONFIG_DIR/sample-network.yaml" apiVersion: cloud.pingcap.com/v1 kind: Network metadata: labels: cloud.pingcap.com/provider-type: aws cloud.pingcap.com/reconciler: ${PROJECT} name: aws-us-west-2 namespace: ${PROJECT} spec: cidr: 10.240.192.0/18 forProvider: {} provider: aws region: us-west-2 EOF KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" kubectl apply -f $INFRA_CONFIG_DIR/sample-network.yaml ``` #### Create a sample cluster in network ```bash export PROJECT=$(whoami) cat << EOF > "$INFRA_CONFIG_DIR/sample-cluster.yaml" apiVersion: cloud.pingcap.com/v1 kind: Cluster metadata: labels: cloud.pingcap.com/provider-type: aws cloud.pingcap.com/reconciler: ${PROJECT} name: sample-aws-cluster namespace: ${PROJECT} spec: cluster: pd: instanceType: t3.medium replicas: 1 storageSize: 10Gi tidb: instanceType: t3.medium replicas: 1 tiflash: instanceType: "" replicas: 0 storageSize: "0" tikv: instanceType: t3.medium replicas: 1 storageSize: 10Gi networkName: aws-us-west-2 sqlEndpoints: public: enabled: true loadBalancerSourceRanges: - 10.0.0.0/8 EOF KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" kubectl apply -f $INFRA_CONFIG_DIR/sample-cluster.yaml ``` #### Create a sample cdc in network ```bash export PROJECT=$(whoami) cat << EOF > "$INFRA_CONFIG_DIR/sample-cdc.yaml" apiVersion: cloud.pingcap.com/v1 kind: TiCDC metadata: labels: cloud.pingcap.com/provider-type: aws cloud.pingcap.com/reconciler: ${PROJECT} name: sample-aws-cdc namespace: ${PROJECT} spec: instanceType: t3.medium replicas: 1 storageSize: 10Gi networkName: aws-us-west-2 clusterName: sample-aws-cluster EOF KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" kubectl apply -f $INFRA_CONFIG_DIR/sample-cdc.yaml ``` ## Connect to the shoot Creating a network creates a shoot. SHOOT_NAME=$(KUBECONFIG="$INFRA_CONFIG_DIR/infra-api-kubeconfig.yaml" kubectl -n "$PROJECT" describe network aws-us-west-2| grep 'Shoot Name:' | awk '{print $3}') aws-vault exec dev -- kubectl --context garden -n garden-infra-dev get secret -o json "${SHOOT_NAME}.kubeconfig" | jq -r .data.kubeconfig | base64 -d > "$INFRA_CONFIG_DIR/shoot.kubeconfig" ## E2E Test Note: e2e is maintained by QA team and is not part of normal development lifecycle since it is cost consuming. ```bash KUBECONFIG= make e2e ``` ### Parallel Specs You can customize the running concurrency of e2e test suites by passing `E2E_NODES=X` where `X` is the desired number of cases running in parallel (`X` defaults to 1): ```bash KUBECONFIG= E2E_NODES=2 make e2e ``` ### Focused Specs You can focus or skip specs by passing a regular expression: ```bash KUBECONFIG= E2E_FOCUS="aws" make e2e KUBECONFIG= E2E_SKIP="aws" make e2e ``` Note: If both `E2E_FOCUS` and `E2E_SKIP` are specified, the constraints will be **AND**ed together. ### Locally Run When you run e2e locally, you can specify "LOCAL_TEST" env to skip cases that may encounter network issues: ```bash LOCAL_TEST=y KUBECONFIG= make e2e ```