This resource is namespace scoped, refer to [Namespace](./namespace.md) for details. ### Custom Resources * [Import](#import) ### Sub Resources * [AccessConfig](#accessconfig) * [CsvFormat](#csvformat) * [CurrentTableStatus](#currenttablestatus) * [CustomConfigs](#customconfigs) * [GCS](#gcs) * [ImportList](#importlist) * [ImportSpec](#importspec) * [ImportStatus](#importstatus) * [S3](#s3) * [Source](#source) * [Storage](#storage) * [TableImportTimeRecord](#tableimporttimerecord) #### AccessConfig AccessConfig includes the information for access import/backup sources. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | roleArn | RoleArn is the ARN of the AWS role which the workload will assume. | string | false | [Back to Custom Resources](#custom-resources) #### CsvFormat | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | separator | | *string | true | | delimiter | | *string | true | | header | | *bool | true | | notNull | | *bool | true | | null | | *string | true | | backslashEscape | | *bool | true | | trimLastSeparator | | *bool | true | | headerSchemaMatch | | *bool | false | [Back to Custom Resources](#custom-resources) #### CurrentTableStatus CurrentTableStatus defines the detailed information on importing a table. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | name | Name is the table name to import. | string | true | | phase | Phase is the detailed import phase for this table. Such as \"Pre-Processing / Importing / ... \" | TableImportPhaseType | false | | size | Size is the total size of the source data of this table. | uint64 | true | | processedSize | ProcessedSize is the total size of the source data that has been processed. For local backend, this means that the source data is converted to local temp KV for fast import. For tidb backend, this means that the source data has been converted into INSERT statements and executed in the target. | uint64 | false | | completed_percent | CompletedPercent is the completed percentage for importing on this table. The value is in 0 ~ 100. Post-processing time is not included. | uint32 | true | | postImportCompletedPercent | PostImportCompletedPercent is the completed percentage after importing on this table (post-process and analyze). The value is in 0 ~ 100. | uint32 | false | | timeRecord | TimeRecord is detailed time record of the whole importing process on this table. | [TableImportTimeRecord](#tableimporttimerecord) | false | | message | Message is the error message of the table import | string | false | [Back to Custom Resources](#custom-resources) #### CustomConfigs CustomConfigs include some custom configs replacing the import configs. Since the generated CRD didn't distinguish value and pointers, the zero value of each fields should be the default config value. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | skipPrecheck | SkipPrecheck specifies whether to skip the precheck before import | bool | false | | duplicationHandlingForSQL | DuplicationHandlingForSQL specifies how the SQL is generated to handle duplicated rows | string | false | [Back to Custom Resources](#custom-resources) #### GCS | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | url | | string | true | | region | | string | true | [Back to Custom Resources](#custom-resources) #### Import Import is the Schema for the imports API | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | metadata | | metav1.ObjectMeta | false | | spec | | [ImportSpec](#importspec) | true | | status | | [ImportStatus](#importstatus) | false | [Back to Custom Resources](#custom-resources) #### ImportList ImportList contains a list of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | metadata | | metav1.ListMeta | false | | items | | [][Import](#import) | true | [Back to Custom Resources](#custom-resources) #### ImportSpec ImportSpec defines the desired state of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | clusterName | ClusterName specifies the name of the cluster where the data is imported to, the Cluster must exist in the same namespace as Import | string | true | | source | Source specifies where the import data should be retrieved and parsed | [Source](#source) | false | | storageRequest | StorageRequest specifies additional storage size for the import job to use | string | false | | dryRun | With DryRun marked as true, the import job will only run a dry run job to verify the permission to access the import source depacated | bool | false | | accessConfig | AccessConfig includes the information for access the import source. | [AccessConfig](#accessconfig) | false | | customConfigs | CustomConfigs include some custom configs replacing the import configs. | [CustomConfigs](#customconfigs) | false | | lightningConfig | LightningConfig is the config for TiDB Lightning. | string | false | | nodeRole | NodeRole is the aws role or gcp service account that the import node uses | *string | false | [Back to Custom Resources](#custom-resources) #### ImportStatus ImportStatus defines the observed state of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | phase | Phase means the overall phase of an import job. Available values are \"Prepare / Running / Complete / Failed\". | ImportConditionType | false | | isTaskCreated | IsTaskCreated means whether the task config has been posted to the Lightning server. | bool | false | | taskId | TaskId is a uuid generated in the task creating phase. | int64 | false | | taskPhase | TaskPhase is the phase of a running import task. Available values are \"Importing / Post-Processing\". | ImportTaskPhaseType | false | | collectTime | CollectTime is the UNIX timestamp when the status is collected. | int64 | false | | totalTables | TotalTables is the total table count for import. | uint32 | false | | completedTables | CompletedTables is the table count that the data has been imported, but the table has not been post-processed. | uint32 | false | | pendingTables | PendingTables is the table count that has not started importing. | uint32 | false | | totalSize | TotalSize is the total source data size. | uint64 | false | | totalSizeWithIndex | TotalSizeWithIndex is the total source data size including index size. | uint64 | false | | processedDataSize | ProcessedDataSize is the total source data size that has been processed. Processing data means the source data is converted into local temp KV storage for quick import. | uint64 | false | | currentTables | CurrentTables is the table detailed status for each importing table. | [][CurrentTableStatus](#currenttablestatus) | false | | allTableDetails | AllTableDetails is the table detailed status for each table. | [][CurrentTableStatus](#currenttablestatus) | false | | completedPercent | CompletedPercent is the overall completed percentage for importing. The value is in 0 ~ 100. Post-processing time is not included. | uint32 | false | | postImportCompletedPercent | PostImportCompletedPercent is the overall completed percentage after importing (post-process and analyze). The value is in 0 ~ 100. | uint32 | false | | totalFiles | TotalFiles is the total data source file count. | *uint32 | false | | message | Message is the error message of the import. | string | false | | awsNodeRoleArn | AWSNodeRoleARN is the ARN of the AWS node role. | string | false | [Back to Custom Resources](#custom-resources) #### S3 | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | url | | string | true | | region | | string | true | [Back to Custom Resources](#custom-resources) #### Source | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | format | Format is the data format of the source data 0: Unknown 1: SQL 2: Amazon Aurora Parquet 3: CSV 4: Raw Parquet | DataFormat | false | | filters | | []string | false | | storage | Storage specifies where the data is located | [Storage](#storage) | false | | userName | UserName is the TiDB username for Import job | string | false | | password | Password is the TiDB password for Import job | string | false | | csvFormat | CsvFormat is the format of the CSV files. | *[CsvFormat](#csvformat) | false | | objectNamePattern | ObjectNamePattern specifies objects which match the pattern to import | string | false | | targetTablePattern | TargetTablePattern specifies target db name and table name | string | false | [Back to Custom Resources](#custom-resources) #### Storage | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | s3 | | *[S3](#s3) | false | | gcs | | *[GCS](#gcs) | false | [Back to Custom Resources](#custom-resources) #### TableImportTimeRecord TableImportTimeRecord records some timestamp and durations information in importing a table. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | importStartTime | ImportStartTime is the UNIX timestamp when first detected the importing has started. | int64 | false | | importFinishTime | ImportFinishTime is the UNIX timestamp when first detected the importing has finished. | int64 | false | | postProcessStartTime | PostProcessStartTime is the UNIX timestamp when first detected the post-processing has started. | int64 | false | | postProcessFinishTime | PostProcessFinishTime is the UNIX timestamp when first detected the post-processing has finished. | int64 | false | | preProcessDuration | PreProcessDuration is the duration in seconds of the pre-process phase. | int64 | false | | importDuration | ImportDuration is the duration in seconds of the data import phase. | int64 | false | | buildIndexDuration | BuildIndexDuration is the duration in seconds of the building index phase. | int64 | false | | postProcessDuration | PostProcessDuration is the duration in seconds of the post-processing phase. Post-processing consists of duplication detection and checksum. | int64 | false | | analyzeDuration | AnalyzeDuration is the duration in seconds of the analyzing phase. | int64 | false | [Back to Custom Resources](#custom-resources)