Loading...

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 1.50.1, 1.51.0
Affects Version/s: None
Component/s: api, beam
Labels:
- FR
- changelog

Steps to reproduce:
Hide

Reproduced on INT with RE 2022-07

With lookup dataset = Local conn and S3

Don't reproduced with lookup dataset = JDBC

CONNECTORS_IMG=ghcr.io/talend/connectors:1.34 LIVY_IMG=ghcr.io/talend/livy:3.19 PREVIEW_RUNNER_IMG=ghcr.io/talend/data-processing-runtime-previewrunner:2.18 COMPONENTS_IMG=ghcr.io/talend/component-server:1.45

Reproduced on dev stack with

componentServer: image: tag: "1.46.1" livy: image: tag: "3.20.0-SNAPSHOT-202207080907208" previewRunner: image: tag: "2.19.0-PR-788-SNAPSHOT-202207080907925" connectors: image: tag: "1.36.0-SNAPSHOT_20220722132725"
Show
Reproduced on INT with RE 2022-07 With lookup dataset = Local conn and S3 Don't reproduced with lookup dataset = JDBC CONNECTORS_IMG=ghcr.io/talend/connectors:1.34 LIVY_IMG=ghcr.io/talend/livy:3.19 PREVIEW_RUNNER_IMG=ghcr.io/talend/data-processing-runtime-previewrunner:2.18 COMPONENTS_IMG=ghcr.io/talend/component-server:1.45 Reproduced on dev stack with componentServer: image: tag: "1.46.1" livy: image: tag: "3.20.0-SNAPSHOT-202207080907208" previewRunner: image: tag: "2.19.0-PR-788-SNAPSHOT-202207080907925" connectors: image: tag: "1.36.0-SNAPSHOT_20220722132725"
Platform:
All
Epic Link:
TDI-46750
Project Size:
Small
Found in Release:

1.46.1

When trying to execute a pipeline with join connector through TDP UI we have the following error

No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead

Full stack trace

[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,960] [INFO ] o.a.s.s.DAGScheduler - ResultStage 2 (collect at BoundedDataset.java:96) failed in 1.110 s due to Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2) (stack-tdp-abe-common-engine-stack-dev-5d7f4f7c89-k54nd executor driver): org.apache.beam.sdk.util.UserCodeException: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.TalendFn$ProcessorFn$DoFnInvoker.invokeProcessElement(Unknown Source)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:228)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:187)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.DoFnRunnerWithMetrics.processElement(DoFnRunnerWithMetrics.java:65)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.SparkProcessContext$ProcCtxtIterator.computeNext(SparkProcessContext.java:140)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:45)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:513)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at scala.collection.convert.Wrappers$IteratorWrapper.hasNext(Wrappers.scala:32)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.MultiDoFnFunction.call(MultiDoFnFunction.java:128)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.translation.MultiDoFnFunction.call(MultiDoFnFunction.java:63)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsToPair$1(JavaRDDLike.scala:186)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:863)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:863)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,961] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:386)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1481)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1408)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1472)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1295)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.scheduler.Task.run(Task.scala:131)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.lang.Thread.run(Thread.java:829)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:329)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.init(BeamProducerFinder.java:129)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.<init>(BeamProducerFinder.java:89)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder.find(BeamProducerFinder.java:63)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.LookupDatasetConfiguration.buildIterator(LookupDatasetConfiguration.java:44)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$0(JoinBuilder.java:47)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.init(IteratorFinder.java:36)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.search(IteratorFinder.java:43)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$1(JoinBuilder.java:50)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.Join.transform(Join.java:48)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.FunctionProcessor.onElement(FunctionProcessor.java:76)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.base.LifecycleImpl.doInvoke(LifecycleImpl.java:83)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.output.ProcessorImpl.onNext(ProcessorImpl.java:266)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.BaseProcessorFn.processElement(BaseProcessorFn.java:103)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.urnForTransform(PTransformTranslation.java:283)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:73)
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout:
[2022-07-05 13:16:29,962] [INFO ] o.a.l.u.LineBufferedStream - stdout: Driver stacktrace:
[2022-07-05 13:16:29,963] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,963] [INFO ] o.a.s.s.DAGScheduler - Job 2 failed: collect at BoundedDataset.java:96, took 1.122083 s
[2022-07-05 13:16:29,965] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,965] [INFO ] o.t.d.b.c.r.ServerSocketAvroSchemaRegistry - Shutdown registry containing {}
[2022-07-05 13:16:29,966] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,966] [ERROR] o.t.d.s.FullRunJob$ - DATA_STREAMS_ERROR_MSG:No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: [2022-07-05T13:16:29,966] [ERROR] o.t.d.s.FullRunJob$ - Original Throwable:
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:71) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:104) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:92) ~[beam-runners-spark-3-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.beam.compiler.BeamJob.waitUntilFinish(BeamJob.java:211) ~[data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.beam.compiler.BeamJob.fullRun(BeamJob.java:180) ~[data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.runJob(FullRunJob.scala:239) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.processInBeamCompiler(FullRunJob.scala:134) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob$.main(FullRunJob.scala:56) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.datastreams.streamsjob.FullRunJob.main(FullRunJob.scala) [data-processing-runtime-streamsjob-2.19.0-main-SNAPSHOT.jar:2.19.0-main-SNAPSHOT]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: org.talend.sdk.component.api.exception.ComponentException: (org.apache.beam.sdk.Pipeline$PipelineExecutionException) java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:329) ~[beam-sdks-java-harness-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309) ~[beam-sdks-java-harness-2.36.0.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.init(BeamProducerFinder.java:129) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder$QueueInput.<init>(BeamProducerFinder.java:89) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.spi.BeamProducerFinder.find(BeamProducerFinder.java:63) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.LookupDatasetConfiguration.buildIterator(LookupDatasetConfiguration.java:44) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$0(JoinBuilder.java:47) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.init(IteratorFinder.java:36) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.finder.IteratorFinder.search(IteratorFinder.java:43) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.JoinBuilder.lambda$recordFinderFunction$1(JoinBuilder.java:50) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.join.Join.transform(Join.java:48) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.components.processing.functions.FunctionProcessor.onElement(FunctionProcessor.java:76) ~[?:?]
[2022-07-05 13:16:29,971] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.base.LifecycleImpl.doInvoke(LifecycleImpl.java:83) ~[component-runtime-impl-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.output.ProcessorImpl.onNext(ProcessorImpl.java:266) ~[component-runtime-impl-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout:  at org.talend.sdk.component.runtime.beam.BaseProcessorFn.processElement(BaseProcessorFn.java:103) ~[component-runtime-beam-1.45.2.jar:?]
[2022-07-05 13:16:29,972] [INFO ] o.a.l.u.LineBufferedStream - stdout: Caused by: java.lang.IllegalStateException: No translator known for org.apache.beam.repackaged.direct_java.runners.core.construction.SplittableParDo$PrimitiveBoundedRead

clones

TCOMP-2111 [Runtime convergence] : Join connector fails in cloud environment with hybrid tck/beam connectors

Done

Details

Description

Attachments

Issue Links

Activity

People

Dates