Uploaded image for project: 'Talend Data Prep'
  1. Talend Data Prep
  2. TDP-4084

Exporting to S3 fails (at least when using the Data Prep runtime)

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • All
    • All
    • Sprint 14 TDP, Sprint 17 TDP, Sprint 21 TDP, Sprint 22 TDP, Sprint 23 TDP (feb 26), Sprint 24 TDP (mar 19), Sprint 25 TDP (apr 12)
    • Small
    • Waiting on Talend Verification

    Description

      Steps to reproduce
      • Perform a fresh install on Windows using the June 23rd Installer (20170623_1246)
      • Create an S3 dataset
      • Do 1 preparation step
      • Try to export to S3 (CSV file, default record and field delimiters)
      Current behavior
      • The export fails with the so-helpful error message in the task history
      • Logs are attached (Data Prep & TCOMP)
      Expected behavior

      Something like "I can export my preparation to S3"

      Notes
      • Interestingly, I don't always get exactly the same error in the logs. I made 4 attempts (all visible in the logs):
        • First one at 8:59pm on a dataset containing exactly 10k rows
        • The next 3 ones on a dataset containing 100k rows. For each attempt, I tried with different record and field delimiters and I have slightly different errors in the logs.
        • Exporting to S3 fails as well if you select Avro or Parquet
        • Exporting to S3 fails even if the source dataset is not S3 (I tried with a local CSV file and got the same "output type not supported")
      • I've attached the source datasets (both were uploaded to S3 via the S3 management console)

      Attachments

        1. 4084-log-tcomp.json
          4 kB
        2. app.log
          279 kB
        3. components-service.log
          16 kB
        4. error_QAstack.txt
          5 kB
        5. export_s3.PNG
          export_s3.PNG
          29 kB
        6. image-2017-06-24-21-18-54-959.png
          image-2017-06-24-21-18-54-959.png
          24 kB
        7. S3 Datasets.zip
          3.42 MB
        8. tcomp_aws_fullrun_S3-to-CSV.log
          400 kB
        9. TDP-4084_exportS3fail.log
          50 kB

        Issue Links

          Activity

            People

              Unassigned Unassigned
              gvaznunes Gwendal Vaz Nunes
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d