--- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.16.3 name: imports.cloud.pingcap.com spec: group: cloud.pingcap.com names: categories: - infra-api kind: Import listKind: ImportList plural: imports singular: import preserveUnknownFields: false scope: Namespaced versions: - additionalPrinterColumns: - jsonPath: .spec.clusterName name: Cluster type: string - jsonPath: .status.phase name: PHASE type: string - jsonPath: .metadata.creationTimestamp name: AGE type: date name: v1 schema: openAPIV3Schema: description: Import is the Schema for the imports API properties: apiVersion: description: |- APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources type: string kind: description: |- Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds type: string metadata: type: object spec: description: ImportSpec defines the desired state of Import properties: accessConfig: description: AccessConfig includes the information for access the import source. properties: roleArn: description: RoleArn is the ARN of the AWS role which the workload will assume. type: string type: object clusterName: description: |- ClusterName specifies the name of the cluster where the data is imported to, the Cluster must exist in the same namespace as Import type: string customConfigs: description: CustomConfigs include some custom configs replacing the import configs. properties: duplicationHandlingForSQL: description: DuplicationHandlingForSQL specifies how the SQL is generated to handle duplicated rows type: string skipPrecheck: description: SkipPrecheck specifies whether to skip the precheck before import type: boolean type: object dryRun: description: |- With DryRun marked as true, the import job will only run a dry run job to verify the permission to access the import source depacated type: boolean lightningConfig: description: LightningConfig is the config for TiDB Lightning. type: string nodeRole: description: NodeRole is the aws role or gcp service account that the import node uses type: string source: description: Source specifies where the import data should be retrieved and parsed properties: csvFormat: description: CsvFormat is the format of the CSV files. properties: backslashEscape: type: boolean delimiter: type: string header: type: boolean headerSchemaMatch: type: boolean notNull: type: boolean "null": type: string separator: type: string trimLastSeparator: type: boolean required: - backslashEscape - delimiter - header - notNull - "null" - separator - trimLastSeparator type: object filters: items: type: string type: array format: description: |- Format is the data format of the source data 0: Unknown 1: SQL 2: Amazon Aurora Parquet 3: CSV 4: Raw Parquet format: int32 type: integer objectNamePattern: description: ObjectNamePattern specifies objects which match the pattern to import type: string password: description: Password is the TiDB password for Import job type: string storage: description: Storage specifies where the data is located properties: gcs: properties: region: type: string url: type: string required: - region - url type: object s3: properties: region: type: string url: type: string required: - region - url type: object type: object targetTablePattern: description: TargetTablePattern specifies target db name and table name type: string userName: description: UserName is the TiDB username for Import job type: string type: object storageRequest: description: StorageRequest specifies additional storage size for the import job to use type: string required: - clusterName type: object status: description: ImportStatus defines the observed state of Import properties: allTableDetails: description: AllTableDetails is the table detailed status for each table. items: description: CurrentTableStatus defines the detailed information on importing a table. properties: completed_percent: description: |- CompletedPercent is the completed percentage for importing on this table. The value is in 0 ~ 100. Post-processing time is not included. format: int32 type: integer message: description: Message is the error message of the table import type: string name: description: Name is the table name to import. type: string phase: description: Phase is the detailed import phase for this table. Such as "Pre-Processing / Importing / ... " type: string postImportCompletedPercent: description: PostImportCompletedPercent is the completed percentage after importing on this table (post-process and analyze). The value is in 0 ~ 100. format: int32 type: integer processedSize: description: |- ProcessedSize is the total size of the source data that has been processed. For local backend, this means that the source data is converted to local temp KV for fast import. For tidb backend, this means that the source data has been converted into INSERT statements and executed in the target. format: int64 type: integer size: description: Size is the total size of the source data of this table. format: int64 type: integer timeRecord: description: TimeRecord is detailed time record of the whole importing process on this table. properties: analyzeDuration: description: AnalyzeDuration is the duration in seconds of the analyzing phase. format: int64 type: integer buildIndexDuration: description: BuildIndexDuration is the duration in seconds of the building index phase. format: int64 type: integer importDuration: description: ImportDuration is the duration in seconds of the data import phase. format: int64 type: integer importFinishTime: description: ImportFinishTime is the UNIX timestamp when first detected the importing has finished. format: int64 type: integer importStartTime: description: ImportStartTime is the UNIX timestamp when first detected the importing has started. format: int64 type: integer postProcessDuration: description: PostProcessDuration is the duration in seconds of the post-processing phase. Post-processing consists of duplication detection and checksum. format: int64 type: integer postProcessFinishTime: description: PostProcessFinishTime is the UNIX timestamp when first detected the post-processing has finished. format: int64 type: integer postProcessStartTime: description: PostProcessStartTime is the UNIX timestamp when first detected the post-processing has started. format: int64 type: integer preProcessDuration: description: PreProcessDuration is the duration in seconds of the pre-process phase. format: int64 type: integer type: object required: - completed_percent - name - size type: object type: array awsNodeRoleArn: description: AWSNodeRoleARN is the ARN of the AWS node role. type: string collectTime: description: CollectTime is the UNIX timestamp when the status is collected. format: int64 type: integer completedPercent: description: |- CompletedPercent is the overall completed percentage for importing. The value is in 0 ~ 100. Post-processing time is not included. format: int32 type: integer completedTables: description: CompletedTables is the table count that the data has been imported, but the table has not been post-processed. format: int32 type: integer currentTables: description: CurrentTables is the table detailed status for each importing table. items: description: CurrentTableStatus defines the detailed information on importing a table. properties: completed_percent: description: |- CompletedPercent is the completed percentage for importing on this table. The value is in 0 ~ 100. Post-processing time is not included. format: int32 type: integer message: description: Message is the error message of the table import type: string name: description: Name is the table name to import. type: string phase: description: Phase is the detailed import phase for this table. Such as "Pre-Processing / Importing / ... " type: string postImportCompletedPercent: description: PostImportCompletedPercent is the completed percentage after importing on this table (post-process and analyze). The value is in 0 ~ 100. format: int32 type: integer processedSize: description: |- ProcessedSize is the total size of the source data that has been processed. For local backend, this means that the source data is converted to local temp KV for fast import. For tidb backend, this means that the source data has been converted into INSERT statements and executed in the target. format: int64 type: integer size: description: Size is the total size of the source data of this table. format: int64 type: integer timeRecord: description: TimeRecord is detailed time record of the whole importing process on this table. properties: analyzeDuration: description: AnalyzeDuration is the duration in seconds of the analyzing phase. format: int64 type: integer buildIndexDuration: description: BuildIndexDuration is the duration in seconds of the building index phase. format: int64 type: integer importDuration: description: ImportDuration is the duration in seconds of the data import phase. format: int64 type: integer importFinishTime: description: ImportFinishTime is the UNIX timestamp when first detected the importing has finished. format: int64 type: integer importStartTime: description: ImportStartTime is the UNIX timestamp when first detected the importing has started. format: int64 type: integer postProcessDuration: description: PostProcessDuration is the duration in seconds of the post-processing phase. Post-processing consists of duplication detection and checksum. format: int64 type: integer postProcessFinishTime: description: PostProcessFinishTime is the UNIX timestamp when first detected the post-processing has finished. format: int64 type: integer postProcessStartTime: description: PostProcessStartTime is the UNIX timestamp when first detected the post-processing has started. format: int64 type: integer preProcessDuration: description: PreProcessDuration is the duration in seconds of the pre-process phase. format: int64 type: integer type: object required: - completed_percent - name - size type: object type: array isTaskCreated: description: IsTaskCreated means whether the task config has been posted to the Lightning server. type: boolean message: description: Message is the error message of the import. type: string pendingTables: description: PendingTables is the table count that has not started importing. format: int32 type: integer phase: description: Phase means the overall phase of an import job. Available values are "Prepare / Running / Complete / Failed". type: string postImportCompletedPercent: description: PostImportCompletedPercent is the overall completed percentage after importing (post-process and analyze). The value is in 0 ~ 100. format: int32 type: integer processedDataSize: description: ProcessedDataSize is the total source data size that has been processed. Processing data means the source data is converted into local temp KV storage for quick import. format: int64 type: integer taskId: description: TaskId is a uuid generated in the task creating phase. format: int64 type: integer taskPhase: description: TaskPhase is the phase of a running import task. Available values are "Importing / Post-Processing". type: string totalFiles: description: TotalFiles is the total data source file count. format: int32 type: integer totalSize: description: TotalSize is the total source data size. format: int64 type: integer totalTables: description: TotalTables is the total table count for import. format: int32 type: integer type: object required: - spec type: object served: true storage: true subresources: status: {}