Pervasive

Basics

What are some differences in Data Junction (7.55, 7.0, etc) and Cosmos (Pervasive Data Integrator)?

There are two major differences between Data Junction and Cosmos. One difference is the user interface. The source and target connections in map designer have new interfaces. There are also Lookup Wizards that allow the creation of Dynamic SQL, Incore and flat file lookups. The parser has been moved to the stand-alone application called Structured Schema Designer. This allows Structured Schema Designer to be accessed from other applications.

The other difference is the transformation repository. In Cosmos XML repositories are used instead of Microsoft Access repositories. This provides more reuse and transformations can be accesses individually.

For more information look in help file under topic "What's New in Map Designer?".

How does Map Designer work? How do I create a conversion?

See Tutorial 1 in the Help file.

The Help file can be accessed by clicking Help in either the full or trial version of Map Designer or Content Extractor.

I have an older version like 7.55, 7.0 (or 6.5, etc). Can I install the v8.x Maintenance Release to get up to date?

Maintenance Updates only update the version for which they were built. DO NOT DOWNLOAD A MAINTENANCE UPDATE FOR A DIFFERENT VERSION! If a maintenance release for a later version is installed on your system, it may destroy the current install. You will have to completely uninstall and reinstall the original program to get it working again.

When do I use Map Designer, and when do I use Extract Schema Designer (Content Extractor)? What's the difference?

Map Designer is designed to convert structured data from one format to another. It is also designed to clean and restructure the data to fit the new format. If your data is in a database of some sort and you want it converted to some other sort of database, or you want to clean and re- structure the data, use Map Designer.

Extract Schema Designer is a report reader. It is designed to mine important data out of non-structured report formats such as invoices, bank statements, emails, html docs, etc., and turn that information into a database. If your data is in a format that is not structured format, and you want to extract the data and put it into a database, use Extract Schema Designer.

I need to completely uninstall the Pervasive Data Integrator that I have, and reinstall it fresh. What is the best way to do that?

See Installation Instructions.

COM

Why won't the EngineSDK run?

Make sure that Integration Engine is installed. The EngineSDK requires Integration Engine to run.

I can't get my SDK samples to run? What can I do?

Make sure that the folder where djengine.exe is installed is in your PATH.

How do I create the Engine and Conversion objects?

To create your Engine object: Private djEngine As DJEC.Engine

To create your Conversion object: Private myConv As DJEC.Conversion

I've created my Engine object, but I still can't get the SDK samples to run.

Be sure that you have set the Engine initialization file. The application will not run if the Engine INI file is not set and referenced properly.

To do this: Set djEngine = New DJEC.Engine
djEngine.InitializationFile = "dj800.ini"

Is there an order that I should set my Engine and Conversion objects in?

Assuming you already have an Engine object and a Conversion object created, it is good practice to set the Engine object BEFORE setting the Conversion object.

To set your Conversion object: Set myConv = New DJEC.Conversion

How do I reference the Integration Engine Components Type Library?

In VB or VBA, the reference that needs to be set is for dj800ec.dll. These Com objects are more up-to-date than those from v7.0 -- dj750ec.dll. In VB, the reference is named "DJEngine 8.0 Components Type Library".

Extract Schema Designer

How does Extract Schema Designer (Content Extractor) work? How do I create a script?

Extract Schema Designer scripts are created in a point and click interface, which is based on pattern recognition. To build a script, you need to choose a file and identify patterns that can be used to find the data you want extracted.

Six simple step-by-step examples of script building with various types of report files are available in the Help file. Click Help in Extract Schema Designer and search for "Tutorial".

See also the Products section of the website.

I just upgraded. How do I move my old scripts into the new Extract Schema Designer (previously Content Extractor, Cambio)?

To move your old scripts to a newer version, now Extract Schema Designer:

1. Make a backup copy of the .mdb file. (example: extractor750.mdb or cambio70.mdb) You will find this file in the directory where Cambio is installed. Place that copy in a safe place outside of the common800 directory.
2. Install the new version of Business Integrator.
3. Paste the copy of the .mdb into the new directory.
4. When you are finished, open the Extract Schema Designer.
5. If the .mdb is an older version, a window will pop up asking if you wish to make a copy of it then update it, or simply update it. Since you have already made a copy, you can choose to simply update it.
6. Your scripts should appear in the script manager when Extract Schema Designer opens.

If you are moving from an earlier version of Cambio to Content Extractor, you may experience difficulty. In that case, call Integration tech support, (512) 231-6000, or submit a Support Request via our website.

I have extracted my data with Extract Schema Designer, but I would also like to change it. For example, combine two fields, add another field, or take out certain characters in a field. How do I do that?

Extract Schema Designer only extracts data. It has very little ability to clean, massage, or re-structure data. However, Extract Schema Designer is designed to coordinate very well with Map Designer. Map Designer can read the Extract Schema script that you created and export the extracted data into any of it's many target formats. It is also a very powerful tool for cleaning, massaging, and re-structuring your data.

I have finished my script in Extract Schema Designer. I know I am going to need Map Designer to get to my final target or to do some additional manipulation of the data. How do I coordinate the two? Do I need to export to ASCII delimited or some other format from Extract Schema Designer before I start Map Designer?

IT IS NOT NECESSARY TO RUN AN EXPORT FROM EXTRACT SCHEMA DESIGNER BEFORE USING MAP DESIGNER. In 8.x or above, it is not possible to export data from Extract Schema Designer. Extract Schema Designer and Map Designer are designed to work together as a team. If you need to manipulate your data after you have designed a Extract Schema script, or you need to export that data to a format that Extract Schema cannot reach, use Map Designer. Here's how:

1. Design your script in Extract Schema Designer and save it. You do not need to run an export.
2. Open Map Designer, and begin a New Conversion.
3. Choose Report Reader, Content Extractor or Extract Schema Designer's Connector as your Source Type. This will bring up two input boxes.
4. In the first input box, labeled Extract Schema, enter the path and file name to the script you saved in Extract Schema Designer. It should be a small file with a .cxl extension.
5. In the second input box, labeled Source File/URI, enter the path and file name to the report file you started out with in Extract Schema Designer, or click the down arrow and browse to it.
6. Click the Connect button if it does not automatically disable.
7. Check the Source Data Browser, the button on the tool bar that looks like an S inside a magnifying glass, to make certain your script is being read correctly and the data looks the same as it did in the Extract Schema browser.
   
   You can switch to Extract Schema Designer and back if you need to make any changes in your script. Click the Refresh Script button after making those changes so Map Designer will re-read the script.
8. Go to the Target Tab and proceed with the conversion normally. Massaging and manipulation is generally done on the Map Tab.

See the Extract Schema Designer and Map Designer Help files for more information.

Map Designer

I just upgraded. How do I move my old conversions into the new Pervasive Data Integrator (formerly known as Data Junction/Cosmos)?

• You can select the Upgrade Utility from the Integration Architect program group in the Start Menu.
• Indicate which Transformations you want to upgrade, select the Workspace and Repository to place the new files then click Start Upgrade.
• When you are finished, open the new version of Map Designer. Your Transformation names will appear in the Repository Explorer.

See the Upgrade Utility Help for more information.

Limitation

When you upgrade a Transformation, the version number of the map gets reset to 1.0 as if you were adding a new map to the database. You will need to change the version information in the Transformation and Map Properties.

Import Your .DJS Files Directly Into Map Designer: To import your version 7.5.5 .djs files, see Importing .DJS Files for details.

Users of Older Versions

If you are upgrading Transformations (formerly called Conversions) from a 7.5.1 or earlier version of Map Designer to this latest version, you may experience difficulty. If you are moving conversions from a 5.1 or earlier version of Pervasive Data Integrator/Pervasive Business Integrator (formerly known as Data Junction for Windows) to the latest, you may experience difficulty. In that case, call the Pervasive Help Desk at (800) 287-4383, or fill out a Support Request form on our website.

See the Upgrade Utility Help for more information.

When attempting to connect to a Btrieve file, I get the following message: "One of the library files (wbtrv32.dll) needed to run this program cannot be found (25547)"

The problem is caused by, and can be resolved by, one of the following:

1. You have an application somewhere on your workstation or LAN that uses the Btrieve engine, but Map Designer cannot find it. If this is true, you should enter the path where the Btrieve DLLs reside in your PATH Environment Variable. Then restart your workstation.
2. The Btrieve file was sent to you by someone outside your working environment, and you do not have the Btrieve engine on your workstation or LAN. If this is true, you will need to install a Btrieve engine or Pervasive PSQL. Go to the Pervasive website.

When attempting to connect to Paradox via IDAPI file, I get the following message: "Error loading library file dj800idapi.dll; one of the library files needed to run this can not be found)(25547)."

The problem can be resolved by following these procedures:

1. Make sure you have the IDAPI drivers (for the Paradox connection) properly installed and configured on your workstation. Borland BDE drivers can be downloaded here(link).
2. Make sure the path to your IDAPI32.DLL file is in the PATH Environment Variable. This is usually c:\Program Files\Borland.
3. Make sure you have an Alias set up properly in the IDAPI Configuration Utility. For details on the above, please refer to "IDAPI" in the Map Designer Help file.

Expressions

What are expressions? Why would I need them?

Expressions are tiny little scripts written in Rapid Integration Flow Language (RIFL). RIFL is based on Visual Basic. Expressions are used to accomplish things that are very specific to your conversion.

They can be used to filter out unwanted characters or complete records, combine or separate fields, perform mathematical calculations, and many other things. The user interface is designed for simplicity, so there is no way it can represent all of the functions available in the Rapid Integration Flow Language. In general, if you need to do something with your data for which there is no button or screen, use the expression language.

See Writing Expressions in the Help file of Map Designer for more information.

Where do I write expressions?

The placement of an expression is determined by its intended function.

As a general rule, expressions written to do field level data manipulation should be placed in the Target Field Expression cells on the Map Tab.

Expressions written to determine if a record gets converted or ignored should be placed in the Source or Target Filters.

Expressions written to do record level data manipulation, but do not specifically filter the records (such as Reject expressions) should be written in the first Target Field Expression cell. Or in Execute action before the Map action.

Expressions written to initialize global variables are executed only at the very beginning of a transformation.

Expressions written to be executed only once at the very end of a conversion should be placed in the Before Transformation event actions of the Transformation Properties dialog box. See Writing Expressions in the Help file of Map Designer for more information.

I need to write an expression to filter certain characters out of a field. For example, commas, dollar signs, letters, etc. Where do I put the expression? And which one should I use?

Expressions written to alter or otherwise affect fields should be placed in the Target Field Expression cells on the Map Tab. Place the expression in the Target Field Expression cell next to the name of the Target Field it will affect.

Some good functions for filtering out unwanted characters include StrReplace, GSub, and Sub.

Here are some examples:

 

GSub("[^0-9n]","", [SourceFieldName])

This will remove every character that is not a digit.

 

 

StrReplace("#","Number", [SourceFieldName])

This will replace all instances of the # symbol with the word "Number".

 

 

Sub("Doctor", "Dr", [SourceFieldName])

This will replace the first instance of the word "Doctor with the abbreviation "Dr".

See the Help file in Map Designer for more information and examples of these functions.

See also Writing Expressions in the Help file of Map Designer for more information.

Can ADO be written in RIFL?

Yes. Map Designer now implements the IDispatch interface that gives it automation capabilities. That means that you can write client programs in the integration expression language.

Also, with this interface, automation applications can be controlled from outside. Most of the OLE enabled application now exposes a set of classes. Map Designer working as a client can instantiate those objects, and call the methods and properties. To start programming using ActiveX, you should have a basic knowledge of the object model and how its components fit together. To see the object models of different software vendors see the documentation that comes with the product.

Also, with this interface, automation applications can be controlled from outside. Most of the OLE enabled application now exposes a set of classes. Map Designer working as a client can instantiate those objects, and call the methods and properties. To start programming using ActiveX, you should have a basic knowledge of the object model and how its components fit together. To see the object models of different software vendors see the documentation that comes with the product.

In Map Designer, you can only create objects (classes) that are inside Automation DLLs at run time. The CreateObject function creates an object at run time. The class of the object is passed as a String argument to CreateObject.

The following are ADO createable object classes: ADODB.Connection, ADODB.Command, ADODB.Parameter, and ADODB.Recordset.

See the topic OLE Automation/ActiveX in Map Designer Help file in for more information and examples.

Can Map Designer run Java Classes?

Yes. Rapid Integration Flow Language (RIFL) supports Java class objects from within a Transformation. The Java object can be used to call any method available to the object from within its underlying Java class. You can pass parameters in the method calls and any value returned from the method call can be used within the Transformation. Passing and returning of Java objects is also supported.

For related information, see JavaVM and JavaObject and Java Classes and RIFL Expressions.

Java

How to use multi-threading with the Java SDK?

Create an individual Conversion or Project Object in each Thread. Then run the Object within its own thread. The Java SDK defines classes for creating/running Conversion and Project Objects. Each Object can run concurrently with other object instances. The only restriction is that each of these objects should be dedicated to a single thread. The SDK does not support access to any of these objects from two or more threads concurrently. It is also possible to load the same Conversion design multiple times using multiple instances of the Conversion Object. This will allow the execution of multiple instances of the same conversion design concurrently. Typically each instance will change source and/or target connection information to prevent conflicts trying to write data to the same target file.

How to change the source and/or target connection information?

Refer to the ConnectionInfo Class in the Integration Engine – Java Interface documentation for information on available methods. A menu containing a list of available SDK documents can be found at: Start, Programs, Pervasive, Integration Architect, SDK Files. Example to change the source/target file:

Conversion myConv = new Conversion( ); 
MyConv.Load("myConv.djs"); 
// To change the source file: myConv.getSources( ).getItem(0).getConnectionInfo.setFile("new_source_file"); 
// To change the target file: myConv.getTargets( ).getItem(0).getConnectionInfo.setFile("new_target_file");

Where can I find sample Java code?

Several Java samples are included with the SDK installation. The location of these samples is: Common800\SDKs\EngineSDK\Java\Samples.

Engine

Trying to connect to the engine on Linux or Unix. Receiving error "djengine: command not be found". The engine is installed. What could be causing the error?

If calling the engine from outside of the install directory, the directory that contains the engine will have to be in the PATH or operating system will not be able to find it. Make sure the installation directory in the PATH.
Also check to see if you have execute permissions on the application.

Trying to connect to the engine on Linux or Unix. Receiving error "djengine: error while loading shared libraries: libdj800util.so: cannot open shared object file: No such file or directory". What could be causing this error?

If calling the engine from outside of the install directory, the directory that contains the shared libraries will have to be in the LD_LIBRARY_PATH or operating system will not be able to find it. Make sure the installation directory in the LD_LIBRARY_PATH.

When installing the engine what permissions are needed?

On Windows machines, make sure to be logged on as admin. You must have administrator rights.
On Linux or Unix machines, make sure to be logged on as root. Or have write permissions to the install directory.