This resource is namespace scoped, refer to [Namespace](./namespace.md) for details. ### Custom Resources * [Import](#import) ### Sub Resources * [AccessConfig](#accessconfig) * [CsvFormat](#csvformat) * [CurrentTableStatus](#currenttablestatus) * [GCS](#gcs) * [ImportList](#importlist) * [ImportSpec](#importspec) * [ImportStatus](#importstatus) * [S3](#s3) * [Source](#source) * [Storage](#storage) * [TableImportTimeRecord](#tableimporttimerecord) #### AccessConfig AccessConfig includes the information for access import/backup sources. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | roleArn | RoleArn is the ARN of the AWS role which the workload will assume. | string | false | [Back to Custom Resources](#custom-resources) #### CsvFormat | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | separator | | *string | true | | delimiter | | *string | true | | header | | *bool | true | | notNull | | *bool | true | | null | | *string | true | | backslashEscape | | *bool | true | | trimLastSeparator | | *bool | true | [Back to Custom Resources](#custom-resources) #### CurrentTableStatus CurrentTableStatus defines the detailed information on importing a table. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | name | Name is the table name to import. | string | true | | phase | Phase is the detailed import phase for this table. Such as \"Pre-Processing / Importing / ... \" | TableImportPhaseType | false | | size | Size is the total size of the source data of this table. | uint64 | true | | processedSize | ProcessedSize is the total source data size that has been processed. Processing data means the source data is converted into local temp KV storage for quick import. | uint64 | false | | completed_percent | CompletedPercent is the completed percentage for importing on this table. The value is in 0 ~ 100. Post-processing time is not included. | uint32 | true | | postImportCompletedPercent | PostImportCompletedPercent is the completed percentage after importing on this table (post-process and analyze). The value is in 0 ~ 100. | uint32 | false | | timeRecord | TimeRecord is detailed time record of the whole importing process on this table. | [TableImportTimeRecord](#tableimporttimerecord) | false | | message | Message is the error message of the table import | string | false | [Back to Custom Resources](#custom-resources) #### GCS | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | url | | string | true | | region | | string | true | [Back to Custom Resources](#custom-resources) #### Import Import is the Schema for the imports API | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | metadata | | metav1.ObjectMeta | false | | spec | | [ImportSpec](#importspec) | true | | status | | [ImportStatus](#importstatus) | false | [Back to Custom Resources](#custom-resources) #### ImportList ImportList contains a list of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | metadata | | metav1.ListMeta | false | | items | | [][Import](#import) | true | [Back to Custom Resources](#custom-resources) #### ImportSpec ImportSpec defines the desired state of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | clusterName | ClusterName specifies the name of the cluster where the data is imported to, the Cluster must exist in the same namespace as Import | string | true | | source | Source specifies where the import data should be retrieved and parsed | [Source](#source) | true | | storageRequest | StorageRequest specifies additional storage size for the import job to use | string | false | | dryRun | With DryRun marked as true, the import job will only run a dry run job to verify the permission to access the import source | bool | false | | accessConfig | AccessConfig includes the information for access the import source. | [AccessConfig](#accessconfig) | false | [Back to Custom Resources](#custom-resources) #### ImportStatus ImportStatus defines the observed state of Import | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | phase | Phase means the overall phase of an import job. Available values are \"Prepare / Running / Complete / Failed\". | ImportConditionType | false | | isTaskCreated | IsTaskCreated means whether the task config has been posted to the Lightning server. | bool | false | | taskId | TaskId is a uuid generated in the task creating phase. | int64 | false | | taskPhase | TaskPhase is the phase of a running import task. Available values are \"Importing / Post-Processing\". | ImportTaskPhaseType | false | | collectTime | CollectTime is the UNIX timestamp when the status is collected. | int64 | false | | totalTables | TotalTables is the total table count for import. | uint32 | false | | completedTables | CompletedTables is the table count that the data has been imported, but the table has not been post-processed. | uint32 | false | | pendingTables | PendingTables is the table count that has not started importing. | uint32 | false | | totalSize | TotalSize is the total source data size. | uint64 | false | | processedDataSize | ProcessedDataSize is the total source data size that has been processed. Processing data means the source data is converted into local temp KV storage for quick import. | uint64 | false | | currentTables | CurrentTables is the table detailed status for each importing table. | [][CurrentTableStatus](#currenttablestatus) | false | | allTableDetails | AllTableDetails is the table detailed status for each table. | [][CurrentTableStatus](#currenttablestatus) | false | | completedPercent | CompletedPercent is the overall completed percentage for importing. The value is in 0 ~ 100. Post-processing time is not included. | uint32 | false | | postImportCompletedPercent | PostImportCompletedPercent is the overall completed percentage after importing (post-process and analyze). The value is in 0 ~ 100. | uint32 | false | | totalFiles | TotalFiles is the total data source file count. | *uint32 | false | | message | Message is the error message of the import. | string | false | [Back to Custom Resources](#custom-resources) #### S3 | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | url | | string | true | | region | | string | true | [Back to Custom Resources](#custom-resources) #### Source | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | format | Format is the data format of the source data 0: Unknown 1: SQL 2: Amazon Aurora Parquet 3: CSV 4: Raw Parquet | DataFormat | true | | filters | | []string | false | | storage | Storage specifies where the data is located | [Storage](#storage) | true | | userName | UserName is the TiDB username for Import job | string | true | | password | Password is the TiDB password for Import job | string | true | | csvFormat | CsvFormat is the format of the CSV files. | *[CsvFormat](#csvformat) | false | | objectNamePattern | ObjectNamePattern specifies objects which match the pattern to import | string | false | | targetTablePattern | TargetTablePattern specifies target db name and table name | string | false | [Back to Custom Resources](#custom-resources) #### Storage | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | s3 | | *[S3](#s3) | false | | gcs | | *[GCS](#gcs) | false | [Back to Custom Resources](#custom-resources) #### TableImportTimeRecord TableImportTimeRecord records some timestamp and durations information in importing a table. | Field | Description | Scheme | Required | | ----- | ----------- | ------ | -------- | | importStartTime | ImportStartTime is the UNIX timestamp when first detected the importing has started. | int64 | false | | importFinishTime | ImportFinishTime is the UNIX timestamp when first detected the importing has finished. | int64 | false | | postProcessStartTime | PostProcessStartTime is the UNIX timestamp when first detected the post-processing has started. | int64 | false | | postProcessFinishTime | PostProcessFinishTime is the UNIX timestamp when first detected the post-processing has finished. | int64 | false | | preProcessDuration | PreProcessDuration is the duration in seconds of the pre-process phase. | int64 | false | | importDuration | ImportDuration is the duration in seconds of the data import phase. | int64 | false | | buildIndexDuration | BuildIndexDuration is the duration in seconds of the building index phase. | int64 | false | | postProcessDuration | PostProcessDuration is the duration in seconds of the post-processing phase. Post-processing consists of duplication detection and checksum. | int64 | false | | analyzeDuration | AnalyzeDuration is the duration in seconds of the analyzing phase. | int64 | false | [Back to Custom Resources](#custom-resources)