Uploaded image for project: 'Talend DI components'
  1. Talend DI components
  2. TDI-46750

Runtime convergence: tDataprepRun to reuse tck processor

Apply templateInsert Lucidchart Diagram



      In the goal to avoid customer data to transit in Talend Cloud, the tDataprepRun from DI job (does not concern TBD here), must:

      • call Dataprep to retrieve the list of recipe of transformation (uppercase, phone number formatting, etc.)
      • not sent data to be transformed to Dataprep but instead transform them locally through tck processor that will be developed by Data-streams team and available in connectors-ee.
      • support migration from legacy tDataprepRun/DI

      Acceptance Criteria

      • No more data is sent to talend cloud
      • All dataprep functions are usable
      • Preparation Id and version can be defined with context value
      • An existing studio job with legacy tDataprepRun will be executable as is after new tck/dataprep replacement
      • Download dictionnary only if needed even if dictionary access is configured

      New: Must still work like before for Dataprep on-prem


      Topic Description DoR
      Description Is the functional requirement clearly stated? Is the user-story properly defined?
      (As a user XXX, I want to YYY in order to ZZZ & functional description)
      Jira information Is the Jira information correct? (fix version, labels, security level, Component, sprint, estimation)
      Environment Environment ready
      Support SSL
      Reachable for QA/Doc/automation(TTP, Junit)/beta testers…
      Target fixVersion/Application
      License Is license EE or SE clearly identified ?
      Technical Analysis Does the developer understand how it will be implemented?
      Do we have a solution? No current solution for Guess schema.
      lookup connectors dependencies still in discussion.
      Approved/discussed with architecture (in-team, global or security, depending on the scope)?
      Dependencies Are all dependencies linked to JIRA (link "depends on")?
      Are they done?
      Including SRE/Devops/IT
      Migrations Is migration needed? Maybe there will be some migration.
      Doc Is DOCT created and linked under the epic?
      Communication channel Is slack feat- created?With all the correct owner involved? (QA/Doc/PO/SM)
      UX Are there changes in the UI?
      Does it contain a mockup?
      Was it approved by UX? Not yet approved, should be the same as tadatapreprun.


        Issue Links



              ozhelezniak Oleksandr Zhelezniak
              rmartins Rodrigo Martins
              0 Vote for this issue
              7 Start watching this issue