Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2241

[Runtime convergence] : Join connector fails - No translator known

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Hide

      Reproduced on INT with RE 2022-07

      With lookup dataset =  Local conn and S3

      Don't reproduced with lookup dataset = JDBC

       

      CONNECTORS_IMG=ghcr.io/talend/connectors:1.34
LIVY_IMG=ghcr.io/talend/livy:3.19
PREVIEW_RUNNER_IMG=ghcr.io/talend/data-processing-runtime-previewrunner:2.18
COMPONENTS_IMG=ghcr.io/talend/component-server:1.45

       

       

      Reproduced on dev stack with

       

      componentServer:
      image:
        tag: "1.46.1"
    livy:
      image:
        tag: "3.20.0-SNAPSHOT-202207080907208"
    previewRunner:
      image:
        tag: "2.19.0-PR-788-SNAPSHOT-202207080907925"
    connectors:
      image:
        tag: "1.36.0-SNAPSHOT_20220722132725"
      Show
      Reproduced on INT with RE 2022-07 With lookup dataset =  Local conn and S3 Don't reproduced with lookup dataset = JDBC   CONNECTORS_IMG=ghcr.io/talend/connectors:1.34 LIVY_IMG=ghcr.io/talend/livy:3.19 PREVIEW_RUNNER_IMG=ghcr.io/talend/data-processing-runtime-previewrunner:2.18 COMPONENTS_IMG=ghcr.io/talend/component-server:1.45     Reproduced on dev stack with   componentServer:       image:         tag: "1.46.1"     livy:       image:         tag: "3.20.0-SNAPSHOT-202207080907208"     previewRunner:       image:         tag: "2.19.0-PR-788-SNAPSHOT-202207080907925"     connectors:       image:         tag: "1.36.0-SNAPSHOT_20220722132725"
    • All
    • Small

    Description

      When trying to execute a pipeline with join connector through TDP UI we have the following error 

       

      No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead

       

      Full stack trace

      [2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,960] [INFO ] o.a.s.s.DAGScheduler - ResultStage 2 (collect at BoundedDataset.java:96) failed in 1.110 s due to Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2) (stack-tdp-abe-common-engine-stack-dev-5d7f4f7c89-k54nd executor driver): org.apache.beam.sdk.util.UserCodeException: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.TalendFn$ProcessorFn$DoFnInvoker.invokeProcessElement(Unknown Source)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:228)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:187)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.DoFnRunnerWithMetrics.processElement(DoFnRunnerWithMetrics.java:65)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.SparkProcessContext$ProcCtxtIterator.computeNext(SparkProcessContext.java:140)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:45)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:513)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.convert.Wrappers$IteratorWrapper.hasNext(Wrappers.scala:32)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.MultiDoFnFunction.call(MultiDoFnFunction.java:128)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.MultiDoFnFunction.call(MultiDoFnFunction.java:63)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsToPair$1(JavaRDDLike.scala:186)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:863)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:863)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:386)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1481)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1408)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1472)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1295)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.scheduler.Task.run(Task.scala:131)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.lang.Thread.run(Thread.java:829)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:329)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.init(BeamProducerFinder.java:129)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.<init>(BeamProducerFinder.java:89)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder.find(BeamProducerFinder.java:63)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.LookupDatasetConfiguration.buildIterator(LookupDatasetConfiguration.java:44)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$0(JoinBuilder.java:47)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.init(IteratorFinder.java:36)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.search(IteratorFinder.java:43)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$1(JoinBuilder.java:50)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.Join.transform(Join.java:48)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.FunctionProcessor.onElement(FunctionProcessor.java:76)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.base.LifecycleImpl.doInvoke(LifecycleImpl.java:83)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.output.ProcessorImpl.onNext(ProcessorImpl.java:266)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.BaseProcessorFn.processElement(BaseProcessorFn.java:103)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.urnForTransform(PTransformTranslation.java:283)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:73)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Driver stacktrace:
[2022-07-05 13:16:29,963] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,963] [INFO ] o.a.s.s.DAGScheduler - Job 2 failed: collect at BoundedDataset.java:96, took 1.122083 s
[2022-07-05 13:16:29,965] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,965] [INFO ] o.t.d.b.c.r.ServerSocketAvroSchemaRegistry - Shutdown registry containing {}
[2022-07-05 13:16:29,966] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,966] [ERROR] o.t.d.s.FullRunJob$ - DATA_STREAMS_ERROR_MSG:No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,966] [ERROR] o.t.d.s.FullRunJob$ - Original Throwable:
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:104) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:92) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.beam.compiler.BeamJob.waitUntilFinish(BeamJob.java:211) ~[data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.beam.compiler.BeamJob.fullRun(BeamJob.java:180) ~[data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.runJob(FullRunJob.scala:239) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.processInBeamCompiler(FullRunJob.scala:134) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.main(FullRunJob.scala:56) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob.main(FullRunJob.scala) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:329) ~[beam-sdks-java-harness-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309) ~[beam-sdks-java-harness-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.init(BeamProducerFinder.java:129) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.<init>(BeamProducerFinder.java:89) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder.find(BeamProducerFinder.java:63) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.LookupDatasetConfiguration.buildIterator(LookupDatasetConfiguration.java:44) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$0(JoinBuilder.java:47) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.init(IteratorFinder.java:36) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.search(IteratorFinder.java:43) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$1(JoinBuilder.java:50) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.Join.transform(Join.java:48) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.FunctionProcessor.onElement(FunctionProcessor.java:76) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.base.LifecycleImpl.doInvoke(LifecycleImpl.java:83) ~[component-runtime-impl-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.output.ProcessorImpl.onNext(ProcessorImpl.java:266) ~[component-runtime-impl-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.BaseProcessorFn.processElement(BaseProcessorFn.java:103) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead

      Attachments

        Issue Links

          clones

          Bug - TCOMP-2111 [Runtime convergence] : Join connector fails in cloud environment with hybrid tck/beam connectors

          • Critical - The customer's production system is in a critical error situation. Basic functions of the supported software are not usable. The consequences can have a serious impact on the customer's business. This is the highest error category for issues arising outside of production systems.
          • Done

          Activity

            People

              Unassigned Unassigned
              asecher Arnaud Secher
              emmanuel gallois
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: