Extracting Data with ExtractorEngine.sh

To extract data from the PPM Oracle instance and dump data to flat files, use the ExtractorEngine.sh command.

You do not need to run the ExtractorEngine.sh command on a daily basis. You can use this command together with FlowEngine.sh to execute the ETL job manually for troubleshooting.

The log file of this command is under <VDW_HOME>/logs/ ExtractorEngine.log.

The following table describes the supported parameters.

Parameter

Mandatory?

Sample Value

Description

instancename

Yes

ppm01

Should be the same when you register the PPM instance

forceinitialload

No

true/false

 

Executes initial load manually. This dumps all records from the PPM Oracle instance to flat files. The initial load duration setup is also applied, which means records that are created before the initial load will not be dumped.

pagesize

No

20000

Specifies the page size when the extractor issues SQL queries to the PPM database instance. Use this parameter only when there is critical performance downgrade on the PPM Oracle instance.

The default value is 50000. The system applies the default value if no value is specified.

parallelism

No

200

Specifies the thread pool size that the extractor has to extract data from PPM in parallel.

The default value is 20. The system will use the default value if no value is specified.

cpname

Yes; one of the three should be used

PPM

Content pack name.

Use this parameter is recommended, which can extract all flat files that the ETL job needs.

streamname

PPM_PERSON_STREAM

ETL stream name. Same as the stream name defined in the stream JSON file.

The extractor extracts all flat files needed for this stream.

entityname

PERSON

Source entity name. Same as the entity name defined in the source JSON file.

The extractor extracts the flat file of a specified source entity.

 

help

No

 

Prints out the help message.

For example, to extract data from the PPM01 instance, run the following command:

Example: sh ExtractorEngine.sh --cpname PPM --instancename PPM01