Unable to retrieve log files on some processes [Iway log exception resolution after pumping and iway log File Introduction]

Problem description:

An error occurred while checking the log after the Iway number is pumped. The information is as follows:

 

(FOC541) SU. central database machine error: S_REQ_NAME
05/10/2010 11:02:36 LOOPBACK

(FOC236) linked file does not have a matching key field or segment: S_REQ_NAME
05/10/2010 11:02:36 LOOPBACK BYPASSING TO END OF COMMAND

 

Solution:

Here are two suggestions assuming you are running DataMigrator 7.6.

It's possible that the FDS server isn' t running. On the Web Console go to Workspace> Configuration, Special Services> right click FDS and if Start is an option select it.

It's also possible that etllog has grown too large. On the Web Console go to procedures> datamigrator utilities> Manage log and Statistics> recreate

 

 

PS: Introduction to log files

 

Datamigrator writes logs from the scheduler and flows that are run. each record of etllog is 250 bytes, and a typical log for a flow that loads a small number of rows doesn't get any errors adds about 30 records to log file.

At that rate you shoshould be able to run over 200,000 datamigrator flows before the log files up.

So what does fill up the log?

• Scheduler messages. by the default the schedwakwakes up every sixty seconds and when it does it writes a line to the log file. if you 've changed this default to (say) Every 6 seconds, then you are getting ten times as your records from the schedds.
• Commit and check messages. By default DataMigrator issues a commit every 1000 rows. Every time this happens that adds two lines to the log like this:

  REFERENCE... at trans 1000 Commit forced:
  1000 for 1000 row (s)

  However if a table is loading a million rows, that's adding 2,000 rows to the log. but for a table that large committing every 1,000 rows may be more often than is needed. increasing the commit size (Target Properties ►commit every ___ rows) CES the number of rows written to the log and may improve throughput.

  But what if you want to keep the commit size small but just want to get rid of the messages? You can do that with the following two commands in a pre-extract stored procedure.

  Set message = offset emgsrv = OFF

  But a warning: some error messages are also disabled, so if there are problems with the flow, you may need to remove these settings to diagnose them.

• Rejected row messages. when the "key matching logic" option "include duplicates" is used with a relational database target, duplicate rows still can't be inserted into a table if there is a unique constraint. instead the Relational datababase rejects the rows. but when it does, it returns error messages which the server sends to the log. for a single row that gets a constraint violation there are about five rows written to the log. for example, from ms SQL Server:

  (FOC1400) sqlcode is 2601 (HEX: 00000A29) XOPEN: 23000
  : Microsoft OLE DBProvider for SQL Server: [23000] Cannot insert duplicat: e key rowin object 'dmrpts 'with unique index 'dmrpts '. [1, 01000] The sta: tement has been terminated. (FOC1416) execute error: DMRPTS

  If a log is filling up with these types of messages, then something is wrong. Did you forget to truncate the target table first? Do you have enough key columns specified to uniquely identify each row? Do you need to use key matching logic to update the target table?

  However if there are thousands of rows being rejected you probably dont 'need to see an error message for each and every one of them. in fact you can cause the job to fail after a certain number of rows are rejected. to do so go to Flow Properties ►execution ►stop processing after ____ DBMS errors.

  It's possible I suppose that rejected rows are an expected part of the load and you don't need to see them all. in that cases it's possible to let the job continue and just suppress the messages. to do so Include in profile or stored procedure the ilne:

  Set dbmsmsglimit = nnnn