Uploaded image for project: 'Talend DI components'
  1. Talend DI components
  2. TDI-46686

[ADLSGen2][TCK] Unable to read the delta lake table if a partition is used

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Work Item
    • Status: On Hold
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Components
    • None

    Description

      [ADLSGen2][TCK] Unable to read the delta lake table if a partition is used. 

      Suppose a Deltalake Table is created with a partition, 

      spark.sql("CREATE TABLE cars_partition_new (id STRING, name STRING, release STRING, sales STRING, outofdate STRING) USING DELTA PARTITIONED BY (outofdate) LOCATION '/mnt/test/From_Deltalake_Partition'")
      

       Then insert the source data into the DetlaLake table, and check the inserted content with Delta Reader in the ADLSGen2. The result is illustrated by, where the partition column (outofdate) is empty. 

       

      Attachments

        Activity

          People

            wwang Wei Wang
            wyuan Wei Yuan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: