Uploaded image for project: 'Talend Data Quality'
  1. Talend Data Quality
  2. TDQ-15030

[TB-2d] Ensure Studio 7.0 can execute preparation of Cloud

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Work Item
    • Status: closed
    • Major
    • Resolution: Done
    • None
    • 7.0.1
    • None
    • None
    • All
    • DQ18-FR-E
    • GreenHopper Ranking:
      0|i16meb:
    • 9223372036854775807
    • Small

    Description

      Aim

      • A job with the Spark tDataprepRun component created in the 7.0.1 studio must still be runnable without any modification (scheduled job in TAC for instance) when new versions of Dataprep and DQ dict will be deployed in the Cloud.

      some ideas

      Current idea:
      Have a new attribute Map<String,String> in DqCategoryForValidation to store additional parameters (like PII for instance).

      Tests:

      • create a job with tDataprepRun Spark version with the old version (7.0Beta) of DqCategoryForValidation
      • patch a Dataprep server with the new version of DqCategoryForValidation
      • confirm that the old job can still run

      another idea:

      • manual deserialization on tDataprepRun of the json payload of the DQ category and only instantiate what we need.

      Another idea:

      Attachments

        Issue Links

          Activity

            People

              vlesquere Vincent Le Squere
              scorreia sebastiao correia
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: