Technical Notes

Release Notes: Verastream Integration Broker, Version 9.7
Technical Note 10041
Last Reviewed 02-Dec-2004
Applies To
Verastream Integration Broker version 9.7
Summary

This document describes important information about version 9.7 of Verastream Integration Broker. In addition to this document, please see the file vsreadme.txt, located in the root folder of your product CD for further information.

New Features and Improvements in Verastream Integration Broker version 9.7

-	Subapplication hierarchy now explicit at design time
-	Improvements to the Java VCP client library
-	Improvements to Process Integrator
-	Improvements to the Verastream Host Integrator Adapter
-	Improvements to CSX
-	Improvements to logging
-	Support for COBOL REDEFINES clause in RMS copybook files on OpenVMS, with new import wizard
-	New methods for outputting formatted XML documents
-	New Siebel Wizard option
-	Additions and improvements to RMS, RMS-Cobol, and RMS-Datatrieve adapters
-	Improved support for RAW, ROA, and WOA tables in Repository Manager
-	VSL: Determining the number of fields in a stream or struct
-	Now possible to specify PREFETCH as database object option
-	Raw dbs- and table option REPLACE_NULL_CHAR=<string>
-	Support for Windows 2003 Server has been added.
-	A column added to a table with Repository Manager is NULL by default.
-	Executing CLOSE on a service object (such as VHI or SIEBEL) automatically closes all open service object components at runtime.
-	The maximum number of columns possible in an index has increased from 8 to 16.

Subapplication hierarchy now explicit at design time

Subapplication loading information is now available at design time, with the result that Process Integrator now presents this information in a graphical manner, with all relevant objects available for drag-and-drop use. This feature enables recursive loading of modules and makes it much easier reuse parts of your composite application.

This change has been accomplished through the addition of a new VSL object: MODULE. (A module is a subapplication.)

Syntax

The properties of a MODULE object are:

• name

• fileexpr

• type

• direxpr

The MODULE statement works in a manner similar to the LOADMODULE/SUBAP statement (see VSL reference).

Example

Discussion

Each subapplication now refers to the subapplications it depends on. In VSL, this is represented in this manner:

This statement appears outside function and object definitions. Any number of MODULE statements can occur in the file. When the VSL code is read into a FLE or PIA file, this information is also stored, as part of the subapplication.

When the engine loads a subapplication into memory, the list of imported subapplications is read from the FLE file and all imported subapplications are automatically loaded as well, before START/START. This loading mechanism is applied recursively, so that if, for example, A imports B and B imports C and D, loading A will actually load A, B, C, and D.

The way to build an application in Process Integrator is to have one composite application (that contains START/START) import (directly or indirectly) all the subapplications it uses. When the engine starts to run the application, it loads the top-level subapplication, and, because of the above mechanism, all subapplications as well. The code itself no longer has to contain any LOAD/SUBAP calls.

Applications that were built with previous versions of Process Integrator use the old load/modules mechanism, which is still supported. You can convert such applications to the new MODULE statements (and vice versa) by selecting File > Load/Unload Modules in Process Integrator and then selecting Module > Conversion in the Load Modules dialog box.

Improvements to the Java VCP client library

The following improvements have been made to the Java VCP Client Library in Verastream Integration Broker version 9.7:

• You can now use the SNDIRECTORYSERVER property as an argument to the -D switch on the Java command line:

• When Java client code is generated, the descriptions are now commented in the Javadoc style format.
• You can specify an endpoint (VCP address 'tcpip:hostname:port') to bypass the directory server lookup.

Improvements to Process Integrator

The following improvements have been made to the Process Integrator tool in Verastream Integration Broker version 9.7:

• Subapplication loading information available is now available at design time, with the result that Process Integrator now presents this information in a graphical manner, with all relevant objects available for drag-and-drop use. See the "Subapplication hierarchy now explicit at design time" section for complete details.
• When database object properties are updated, Verastream now checks values entered in dialog boxes, to ensure correct syntax.
• You can now replace an object completely when you insert it or update it from a repository. This helps prevent the creation of additional fields when "Do not remove leading and trailing spaces from string fields" is changed. A new setting, "Replace existing data objects with the same name with the new definitions" has been added, both in the Composite Application Properties dialog box (Edit > Properties > Settings) and in the Process Integrator Options dialog box (Tools > Options > Application Defaults).

Improvements to the Verastream Host Integrator Adapter

The following improvements have been made to the Verastream Host Integrator Adapter in Verastream Integration Broker version 9.7:

• Support has been added for Verastream Host Integrator version 5.5. (And support for Verastream Host Integrator version 4.5 has been removed.)
• It is now possible to retrieve metadata from a Host Integrator model without connecting to the Host Integrator server.
• Function CLOSE on service object VHI now closes all open VHI components on the given VHI object.

Improvements to CSX

The following improvements have been made to the CSX load balancing feature in Verastream Integration Broker version 9.7:

• For CSX, the MYDBG logging environment variable now supports an %a file name directive for the component address.
• CSX logging has been improved.
• The CSX configuration tool now remembers window position and size between uses.
• In the CSX Configuration tool, you can now specify a fixed port number in a CSX component pool for the CSX switch without enabling the "external load balancer" alias option. This allows communication through a firewall.
• CSX configuration replication communication is the same as in version 9.6, but only selected information is written to the vscsx.cfg file—specifically, CSX records that are in a local namespace, registered on the local system (in snregtab.dat), or that include the local system in the CSX pool server or switch lists. This prevents undesired replication retention for systems that may be temporarily moved between environments.
• You can now configure the Verastream Component Directory Tool to display or hide running CSX switch processes. The option is on the View menu in the Component Directory Tool.

Support for COBOL REDEFINES clause in RMS copybook files on OpenVMS, with new import wizard

A copybook describes the record layout of an RMS data file. One of the clauses that can be used in a copybook is REDEFINES. A field or group with a REDEFINES clause redefines another field or group. The result is that part of an RMS record can be interpreted in two ways:

• As the redefined group or field
• As the group or field that redefines

Example 1

In Example 1, EXAMPLE_RECORD can be interpreted either as containing one long field, RECORD_DESCRIPTION, or as fields TIME_OF_ERROR, ERROR_CODE, ERROR_LOCATION and ERROR_DESCRIPTION, with field FILLER making up the total of 195 bytes.

Whether Verastream assumes the one field RECORD_DESCRIPTION or the group ALTERNATIVE_DESCRIPTION depends in this case on field RECORD_TYPE: LO means long description and ER refers to the error group.

Normally an RMS Cobol table is represented by just one TABLE object, where each field in the copybook corresponds to a column in the table. To represent a copybook with REDEFINES clauses, Verastream uses one top-level table that contains the fields of the copybook that are not redefined, and one dependent table for each REDEFINES clause in the copybook.

Example 2

To create all table definitions (Example 2) and to enable the use of functions like FIRST, ADD, and SELECT, Process Integrator provides a new RMS-Cobol Copybook Redefines Wizard, which is available from the Tools menu in Process Integrator.

Which redefines are valid for a particular record is not defined by the copybook, and therefore must be specified by the programmer. Usually there are values in the record that indicate which of the redefines are valid for that record and which are not (like field RECORD_TYPE in Example 1). The wizard allows you to specify a test expression for each redefine, to express the validity of that redefine.

Example 3

In Example 3, the expressions are: record_type == 'LO' and record_type == 'ER'. The wizard changes the basic functions on the main table so that data in the redefines is also read and written if it is valid. If you use these functions, you can access all data, including the redefined data, as if it were in one table. At any time, you can use the preview button in the wizard to see how the data is interpreted. Of course, if the tests are not yet correct, you may see incorrect data or get conversion errors. The wizard creates a module that you can load into your composite applications. You can also use this wizard to change such a module at a later time.

Example 4

In Example 4, a record of type 'LO' is added and logged. This record is then updated into a type 'ER' record and logged. Finally, all 'ER' records are selected. Module example is created by the RMS-Cobol Copybook Redefines Wizard. All fields of the redefine that are not valid are NULL. It is not possible to use SELECT on redefine tables.

The output from function LOG after the ADD is:

The output from function LOG after the UPD and SELECT is:

Note: All DATABASE objects generated by the RMS-Cobol Copybook Redefines Wizard automatically get option PREFETCH=1, which is appropriate if objects are to be used with Data Server. Without this setting, the Redefine tables read their data from the wrong main RMS table record. Do not change or remove this PREFETCH option.

New methods for outputting formatted XML documents

Several new methods have been added, some to the XML DOM adapter, and some to Verastream's extension to the XML DOM, to enable Verastream to output formatted XML documents, with with line returns and indents.

• In the extensions to the XML DOM interface, two new methods have been added: saveFormattedDocumentText and saveFormattedDocumentBinary. These methods work like existing methods saveDocumentText and saveDocumentBinary, respectively, except that they output formatted XML methods and provide a new argument, indentAmount (0..n), which indicates the number of spaces to indent.
• In the Verastream adapter for XML DOM, three new methods have been added: writeFormattedText, writeFormattedBinary, and writeFormattedFile. These methods work like existing methods writeText, writeBinary, and writeFile, respectively, except that they output formatted XML methods and provide a new argument, indentAmount (0..n), which indicates the number of spaces to indent.

New Siebel Wizard option

In the Siebel Integration Object (IO) Wizard and the Siebel Virtual Business Component (VBC) Wizard it is now easier to update an existing application. The "Place the Field objects in the module" check box, on the Naming pane in the IO wizard, and on the Module pane in the VBC wizard, determines whether the fields for the IO or VBC are placed in the module or in the main application. For more information, see Technical Note 10042.

Additions and improvements to RMS, RMS-Cobol, and RMS-Datatrieve adapters

Additions and improvements to the RMS adapter

The Verastream adapter for RMS is now able to read and create tables with variable-length records, which means that the adapter now supports text and byte fields. Such fields can be used without size, but only one per table, and only as the last field in the record. This allows Verastream to read the remainder of a record into one BLOB field.

To be able to create a table with variable length records, two options are available:

• VARIABLE_LENGTH=[yes|no], with no as default
• MAXIMUM_RECORD_SIZE=<n>, with 0 <n < 32232

To specify some RMS table attributes when creating a table, two further options are now available:

• EXTENSION_QUANTITY=<n>, the FDL FILE EXTENSION attribute
• ALLOCATION_QUANTITY=<n>, the FDL ALLOCATION attribute

If variable length is specified, the maximum length is used only to create a table. The actual record size depends on the size of the last BLOB field in the record. An error is given if that field value is too large to fit in the remainder of the record.

Additions and improvements to the RMS-Datatrieve adapter

The RMS Datatrieve adapter is now capable of reading and writing RMS files with records containing numeric BASIC types. The supported BASIC float types are:

• f_floating (or single)
• d_floating (or double)
• g_floating (or gfloat)
• h_floating (or hfloat)
• s_floating (or sfloat)
• t_floating (or tfloat)
• x_floating (or xfloat)

All of these types are mapped onto a Verastream float field. Also supported are:

• decimal, with VIB type fixed(m,n)
• byte, word, long, with VIB type int
• quad, with VIB type fixed(n)

Additions and improvements to the RMS-Datatrieve and RMS-Cobol adapters

The RMS Datatrieve and RMS-Cobol adapters are now able to read and create tables with variable-length records, which means that these adapters now support text and byte fields with maximum size. However only the last (BLOB) field in the record has a variable size, because the remainder of the record is read into this field. All other BLOB field values have always a size equal to the maximum.

To be able to create a table with variable length records, use VARIABLE_LENGTH=[yes|no], with no as default. The record size is calculated out of the individual column sizes.

If variable length is specified, the maximum length is used only to create a table. The actual record size depends on the size of the last BLOB field in the record. An error is given if that field value is too large to fit in the remainder of the record.

Copybooks with OCCURS and variable length records using NULL

When a copybook contains an OCCURS clause and the RMS table has variable-length records, it is not obvious whether an actual value has been read from the record. A copybook, for instance, might contain a clause 'PIC 9(02) OCCURS 4 TIMES.' This value of 4 is a maximum, so if only 2 columns are present in the record, Verastream needs to be able to detect that. In these cases the values that do not get a record value are set to NULL. This is possible because RMS-Datatrieve and RMS-Cobol do not support native NULLs.

A similar mechanism is used when a record is added or updated. Just set the values of the columns that do not need to be added to the record to NULL. Beware, however, of a 'hole'; that is, a non-NULL column between NULL columns. The NULL values before the non-NULL are stored as pseudo NULL, because they cannot be ignored.

Improvements to logging

The following changes have been made to the way logging works in Verastream Integration Broker version 9.7. Most of these improvements are described in greater detail in Technical Note 10234 titled, Enabling and Using Logging in Verastream Integration Broker.

Logging no longer enabled by NEWDBG STATUS or NEWDBGFILE STATUS

The interpretation of these VSL functions has changed in Verastream Integration Broker version 9.7:

• Executing NEWDBG STATUS no longer changes the application logging status in the engine.
• Executing NEWDBGFILE STATUS no longer enables application logging. In Verastream Integration Broker version 9.7, executing NEWDBGFILE STATUS enables the application logging status in the engine if the appl debug flag is present in the argument, and disables it otherwise.

After NEWDBGFILE has been executed (to close the existing log file and open a new log file using the supplied file name), the new complete filename is logged. In the new log file the complete original file name is logged. A complete name refers in this case to a name with log options %a, %p, %j, and so forth, resolved.

Example

After executing NEWDBGFILE, the last two lines of file log_1019_1098183170408.txt would contain:

The first line of file log_1019_1098183170488.txt would be:

Flushing the log file

When sntcpd --kill is called, the Verastream daemon is now stopped in a controlled manner with the logfiles being flushed:

sntcpd --kill <process-id>:

In the event that the daemon cannot be stopped by --kill only (no automatic detection of pid) it is also possible to specify a hexadecimal argument that will be used as process id. This same call can also be used to stop a running engine with the logfile being flushed, even if the new log control flag flush is not specified.

Two new log control flags are now provided: sync (available only on OpenVMS) and flush.

DEBUG on BASE

The new function DEBUG on BASE, for a table, field, structure, and stream, allows you to write to the current log file, regardless of whether application debug logging ('appl' log group) is enabled. Otherwise DEBUG on BASE is identical to LOG on BASE. DEBUG on BASE is helpful for applications where full debug logging generates too much activity in the log file.

New %a log file name directive support for CSX

The new %a log file name directive can be used in sn.ini [csx] MYDBG). This expands to the VCP address including namespace (with slashes replaced with underscores). This log file name directive can be helpful for identifying the specific CSX switch that is writing to the log when you are running multiple CSX pools and/or namespaces.

Improved support for RAW, ROA, and WOA tables in Repository Manager

In Repository Manager there is a new dialog box, "Import Raw/Roa Tablegroup in Repository," available by choosing "Import table definitions from Data File" from the Tools menu. Use this dialog box to specify a set of properties that determine how data files can be imported into Verastream as table groups.

To use the "Import Raw/Roa Tablegroup in Repository," dialog box, follow these steps:

1. Select an existing writeable repository from the Repository list box. This is the repository in which you want to store the table group.
2. Type the name for a new table group in the Tablegroup field.

3. Select the Table type you wish to use.
4. Select Local as the access method if you do not want to use Data Server, or Using Service to be able to browse for a service.

When you click OK, the "Read/Write Raw Data File Properties Wizard" or the "Read-Only ASCII File Properties Wizard" starts automatically, depending on your choice of table type. Both of these wizards are documented in the online help.

When you're done, you will have a RAW or ROA type table group in the repository, that you can then import into Process Integrator.

VSL: Determining the number of fields in a stream or struct

You can now use the info function with a stream or struct as the first argument and 'NFIELDS' as the second argument to return the number of fields in the specified stream or struct:

Now possible to specify PREFETCH as database object option

Although it has been possible to use property SERVICE on a database object to specify a dataserver, the prefetch option needed to be set by means of OPTIONS/RMLNET. Now OPTIONS='PREFETCH=n' is possible for every database object type.

Raw dbs- and table option REPLACE_NULL_CHAR=<string>

When a (raw) file contains NULL characters, it is impossible to use a regular expression to detect columns and/or records (column property TOREGEXP). With option REPLACE_NULL_CHAR=<string> all NULLs are replaced with the specified <string>, before the regular expressions are used. In the result, the replacement characters are still present. The complete replacement string must be unique because it is also used to reposition file contents during parsing. A non-unique string results in the file not being read correctly.

Related Technical Notes

10022	Release Notes: Verastream Integration Broker, Version 9.6
10042	Importing a Modified Siebel Integration Object into Verastream Process Integrator