Talend DI

EBCDICInput to Database - wrong column names

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: 4.2.4, 5.0.2
  • Component/s: None
  • Labels:
    None
  • Found in Release:
  • Steps to reproduce:
    Hide

    To recreate:
    1) Take the SimpleOccurs.cpy and create a new copybook in the metadata tree
    2) right-click the created copybook and select retrieve schema
    3) in the schema view export the schema to XML

    Show
    To recreate: 1) Take the SimpleOccurs.cpy and create a new copybook in the metadata tree 2) right-click the created copybook and select retrieve schema 3) in the schema view export the schema to XML
  • Product Build:
    4.2.3
  • OTRS ID:
    2011100410000158
  • SLA Level:
    Gold
  • Zone Support:
    EMEA
  • Platform:
    All

Description

There is an issue when you try to insert the data from an EBCDIC file to a database:

  • the schema of the EBCDIC file contains an occurs-part and is stored in the repository
  • the target table does not yet exists and you want the job to create the table

In this case the database output component (tried with DB2 an Oracle) resolves the database column names from the input rows wrong.

"occurs" in a copybook means that the following part is repeated n times. See attached cpy-files for example: they are syntactically different but semantically equals.

The describes misbehavior juts happens, when you use the copybook with the occurs.

I just exported the schema of the SimpleOccurs copybook to XML. There you will see, that the "originalDbColumnName" is set wrong. So it must be an issue with the schema wizard for copybooks.

Please see attached job + copybook + screen-shot as example.

  1. NoOccurs.cpy
    11/Oct/11 12:49 AM
    0.4 kB
    Corinne Chaumont
  2. SimpleOccurs_Schema.xml
    11/Oct/11 12:49 AM
    2 kB
    Corinne Chaumont
  3. SimpleOccurs.cpy
    11/Oct/11 12:49 AM
    0.2 kB
    Corinne Chaumont

Activity

Hide
Corinne Chaumont added a comment -

from customer to justify the priority change:
"hence we have to process very large ebcdic files with 1000+ columns the impact is pretty high. A manual correction of that much column names takes a lot of time.

I think to fix this issue isn't that complicated: the field "originalDbColumnName"just shouldn't be filled anyway since a copybook is a file and not a database table (for a delimited file this field is empty too)."

Show
Corinne Chaumont added a comment - from customer to justify the priority change: "hence we have to process very large ebcdic files with 1000+ columns the impact is pretty high. A manual correction of that much column names takes a lot of time. I think to fix this issue isn't that complicated: the field "originalDbColumnName"just shouldn't be filled anyway since a copybook is a file and not a database table (for a delimited file this field is empty too)."
Hide
Corinne Chaumont added a comment -

Please consider this bug for resolution as it is important for the customer

Show
Corinne Chaumont added a comment - Please consider this bug for resolution as it is important for the customer
Hide
Nicolas Rousseau added a comment -

I don't think this really needs a patch since we can edit manually the db columns, there is nothing blocking here.

Show
Nicolas Rousseau added a comment - I don't think this really needs a patch since we can edit manually the db columns, there is nothing blocking here.
Hide
Nicolas Rousseau added a comment - - edited

for dev:

DB column names are now generated with original names from the copy book file.
Which generates:

COLUMN01
COLUMN02
COLUMN03
COLUMN04
COLUMN03
COLUMN04
COLUMN03
COLUMN04

best would be to add an index for any duplicate:

COLUMN01
COLUMN02
COLUMN03
COLUMN04
COLUMN03_1
COLUMN04_1
COLUMN03_2
COLUMN04_2

Show
Nicolas Rousseau added a comment - - edited for dev: DB column names are now generated with original names from the copy book file. Which generates: COLUMN01 COLUMN02 COLUMN03 COLUMN04 COLUMN03 COLUMN04 COLUMN03 COLUMN04 best would be to add an index for any duplicate: COLUMN01 COLUMN02 COLUMN03 COLUMN04 COLUMN03_1 COLUMN04_1 COLUMN03_2 COLUMN04_2
Hide
yunjie gao added a comment -

Test on Talend-All-r71296-V5.0.0NB:it is ok.

Show
yunjie gao added a comment - Test on Talend-All-r71296-V5.0.0NB:it is ok.
Hide
yunjie gao added a comment -

Test on Talend-All-r71296-V4.2.4NB:it is ok.

Show
yunjie gao added a comment - Test on Talend-All-r71296-V4.2.4NB:it is ok.
Hide
Corinne Chaumont added a comment -

Customer applied the patch and has the following remark:
"I have just one question which is in combination with the Bug TDI-17966 /support ticket 2011100410000131 (Copybook retrieve schema of occurs-columns). Why do use a different indexing mechanism for the "originalDbColumnName" compared to the internal column column indexing?

In my opinion the "originalDbColumnName" just shouldn't be filled for a copybook. For a delimited file it isn't filled either. What do you think?"

Show
Corinne Chaumont added a comment - Customer applied the patch and has the following remark: "I have just one question which is in combination with the Bug TDI-17966 /support ticket 2011100410000131 (Copybook retrieve schema of occurs-columns). Why do use a different indexing mechanism for the "originalDbColumnName" compared to the internal column column indexing? In my opinion the "originalDbColumnName" just shouldn't be filled for a copybook. For a delimited file it isn't filled either. What do you think?"
Hide
Nicolas Rousseau added a comment -

the "original" is only here as information yes, it's not too usefull.

There is no indexing for the original, it's just the name of the "original" column from the copybook without indexing...

Show
Nicolas Rousseau added a comment - the "original" is only here as information yes, it's not too usefull. There is no indexing for the original, it's just the name of the "original" column from the copybook without indexing...
Hide
Nicolas Rousseau added a comment - - edited

for dev > as said from support, need another modification for this to remove the value in the original db column name

Show
Nicolas Rousseau added a comment - - edited for dev > as said from support, need another modification for this to remove the value in the original db column name

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: