Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2520

Date column read as Dynamic from File goes as Record:Schema.Type.STRING and no field.pattern passed to connector runtime class

Apply templateInsert Lucidchart DiagramXMLWordPrintable

    • Hide
          • In studio: Create a job tFileInputDelimiter->tAnyTCKOutput with one Dynamic column schema

      -Put a date value (i.e. 2023-08-09) in the file, run the job in debug mode, put a breakpoint on tAnyTCKOutputRuntime:next(Record input) method

      -You'll see that you have a String column and you don't have access to schema DateFormat like you'd have for Date column. 

      Show
      In studio: Create a job tFileInputDelimiter->tAnyTCKOutput with one Dynamic column schema -Put a date value (i.e. 2023-08-09) in the file, run the job in debug mode, put a breakpoint on tAnyTCKOutputRuntime:next(Record input) method -You'll see that you have a String column and you don't have access to schema DateFormat like you'd have for Date column. 
    • All
    • Small

      When we need to handle date column as part of Dynamic field, it's OK until it's read from the file (using tFileInputDelimited), so in DynamicMetadata all columns are String

      We could convert String to Date in connector Processor/Output but for that we should have a date pattern, which is not included for Dynamic->String columns

       

      I think for Dynamic fields we should add date pattern in every Schema.Entry metadata `props`

            emmanuel_g emmanuel gallois
            dsylaiev Dmytro Sylaiev
            Dmytro Sylaiev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: