Processing ETL with FlowEngine.sh

Use the FlowEngine.sh command to process the ETL tasks. This command loads data to the Vertica database from the flat files that the extractor engine generated.

You do not need to run the FlowEngine.sh command on a daily basis. You can use this command together with ExtractorEngine.sh to execute ETL job manually for troubleshooting.

The log file of this command is located at <VDW_HOME>/logs/ FlowEngine.log.

The following table describes the supported parameters.

Parameter

Mandatory?

Sample Value

Description

instancename

Yes

ppm01

Should be the same when you register the PPM instance

streamname

Yes

PPM_PERSON_STREAM

ETL stream name

batch

Yes

1

The batch ID from the ExtractorEngine output

help

No

 

Prints out the help message

If you want to process ETL for the PPM01 instance, run the following command:

sh FlowEngine.sh --streamname PPM_PERSON_STREAM --instancename PPM01 --batch 1