Extending or Creating Entities

You can extend existing entities or create new entities for both dimension and fact tables. This section shows you how to customize the entities step by step.

For examples of how to extend or create dimension and fact entities, see Scenarios of Extending or Creating Entities.

Note: It is suggested to run the sample content packs in your test environment before extending entities. See Running Sample Content Packs for details.

Defining Content Packs

To extend or create entities, you need to define content packs by following these steps:

  1. Create two content pack folders: CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp.
  2. Create folders under these two content packs. For detailed structures, see Content Pack Structure .
  3. Under CUSTOMIZATION_TARGET.cp, do the following:

    1. Under the dwmetadata\entities directory, create and define the target entity JSON file according to the sample content packs at HP Live Network. For more information for obtaining the sample content packs, see Obtaining Sample Content Packs for Customization.

      Note the following when defining the target entity JSON file:

      • For details about the attributes in this JSON file, see Entity Attribute Descriptions.

      • The entity_name attribute should be unique because the table name is generated according to the entity name.
      • The segmented_by attribute is used for cluster segmentation. See HP Vertica Analytics Platform Version 7.0.x Documentation for details.

      • The schema attribute includes the new attributes:

        • attribute_name: The attribute name.

        • scd (for dimension tables only): Determines whether to update the record (scd1) or to insert a new record (scd2).

      • To extend OOTB entities, you need to define the *_associated_* attribute:

        • lookup_entity_name: The name of the entity that needs to be extended.
        • role_entity_name: Includes CUSTOMIZATION_ as the prefix and _EXTEND as the suffix; for example, CUSTOMIZATION_CONTACTS_EXTEND.
      • *_associated_* is required for extended entities while optional for new entities. Refer to the following table for details.

        Attribute Description

        Attribute Type

        Description

        Required for Optional for

        dimension_associated_dimension

        Used to define the reference from a dimension table to another dimension table (for example, snowflake model). For an extended dimension entity, this attribute is used to specify which entity this table is extended from.

        Extended dimension tables New dimension tables

        fact_associated_dimension

        Used to define the reference from a fact table to a dimension table.

        Extended fact tables; Use only one of the attrbutes for a single entity New fact tables

        fact_associated_fact

        Used to define the reference from a fact table to another fact table. For an extended fact entity, this attribute is used to specify which entity this table is extended from.

        (Not supported for new fact tables)
    2. Create and define cp.json according to the samples provided at HP Live Network. For more information for obtaining the sample content packs, see Obtaining Sample Content Packs for Customization.

      Note the following when defining cp.json:

      • The value of content_pack_name should be the same as defined in content_pack of the target entity.
      • cp.json in CUSTOMIZATION_TARGET.cp must have target_entities defined.

      • For detailed descriptions of the attributes in this JSON file, see Entity Attribute Descriptions

      After you complete this step, the target entity JSON file is defined.

  4. In CUSTOMIZATION_PPM.cp, do the following:

    1. Under the dwmetadata\entities directory, create and define the source entity JSON file according to the samples provided at HP Live Network. For more information, see Obtaining Sample Content Packs for Customization.

      When defining the source entity JSON file, note the following:

      • schema: Includes the attribute definitions for the source entity.

      • attribute_name: Defines the attribute name.
      • For detailed descriptions of the attributes in this JSON file, see Entity Attribute Descriptions.

      After you complete this step, the source entity JSON file is defined.

    2. Under dwmetadata\streams, create and define the stream entity JSON file according to the samples provided at HP Live Network. For more information, see Obtaining Sample Content Packs for Customization.

      When defining the stream entity JSON file, note the following:

      • content_pack: Aligns with content_pack defined in the source entity JSON file.
      • stream_name: Includes CUSTOMIZATION_ as the prefix.
      • source_entities_includes: Includes the source entity of the stream; for example, CUSTOMIZATION_KCRT_CONTACTS.
      • target_entities_includes: Includes the target entity of the stream; for example, CUSTOMIZATION_CONTACTS.
      • transforms: see ETL Step 2: SSIfor a definition.
      • post_target_transforms: Optional. If you want to process other SQLs after data is loaded to the target table, include SQLs in this attribute. For more information, see ETL Step 10: POSTTARGET.

      • For detailed descriptions of the attributes in this JSON file, see Entity Attribute Descriptions.

      After you complete this step, the stream entity JSON file is defined.

    3. Under extmetadata, create and define the extractor entity JSON file according to the samples provided at HP Live Network. For more information, see Obtaining Sample Content Packs for Customization.

      When defining the extractor entity JSON file, note the following:

      • content_pack: Aligns with content_pack defined in the stream entity and source entity.
      • entity_name: Includes CUSTOMIZATION_ as the prefix and _EXT as the suffix.
      • source_entity_name: Defines the source table name.

      • extraction_view: Selects the attributes defined in the source entity JSON file.
      • For detailed descriptions of the attributes in this JSON file, see Entity Attribute Descriptions.

      After you complete this step, the extractor entity JSON file is defined.

    4. Create and define cp.json according to the samples provided at HP Live Network. For more information, see Obtaining Sample Content Packs for Customization.

      When defining cp.json, note the following:

      • content_pack_name should be unique and align with content_pack defined in stream entity, source entity, and extractor entity.
      • cp.json in CUSTOMIZATION_PPM.cp must have the streams, source_entities, and extraction_entities attributes defined.

      • For detailed descriptions of the attributes in this JSON file, see Entity Attribute Descriptions.

Deploying Content Packs

To deploy content packs, follow these steps:

  1. Run the following command on Linux to ensure that $VDW_HOME is configured correctly:

    Echo $VDW_HOME

    If $VDW_HOME is correctly configured, the directory that you have Vertica for PPM content pack installed returns. For example,

    /VDW_HOME

    Otherwise, add $VDW_HOME as a system environment variable and point it to the Vertica for PPM content pack directory.

  2. Place CUSTOMIZATION_PPM.cp and CUSTOMIZATION_TARGET.cp under <VDW_HOME>/Content.
  3. Run the ContentManager.sh script under the <VDW_HOME>/bin directory to deploy CUSTOMIZATION_TARGET.cp:

    sh ContentManager.sh --instruction install --cpname CUSTOMIZATION_TARGET;

    You can find the following message from ContentManager.log under <VDW_HOME>/logs if the content pack is deployed successfully:

    The content of package CUSTOMIZATION_TARGET was successfully installed

    The DIM_ CUSTOMIZATION_CONTACTS table is generated in the Vertica database.

  4. Run the ContentManager.sh script under the <VDW_HOME>/bin directory to deploy CUSTOMIZATION_PPM.cp:

    sh ContentManager.sh --instruction install --cpname CUSTOMIZATION_PPM;

    You can find the following message from ContentManager.log under <VDW_HOME>/logs if the content pack is deployed successfully:

    The content of package CUSTOMIZATION_PPM was successfully installed
  5. Run the ExtractorEngine.sh script under the <VDW_HOME>/bin directory to extract data from the PPM database to flat files:

    sh ExtractorEngine.sh --streamname <Stream_Entity_Name> --instancename <PPM_Instance_Name>

    You can find the following message from ExtractorEngine.log under <VDW_HOME>/logs if the content pack is deployed successfully:

    Extractor was successfully executed. The BATCH ID is: <Batch_ID>.
    • <Stream_Entity_Name>: Should be the same as defined in stream_name of the stream entity JSON file.
    • <PPM_Instance_Name>: The PPM instance name you specified when installing the Vertica for PPM content pack.
  6. Run the FlowEngine.sh script under the <VDW_HOME>/bin directory to process ETL:

    sh FlowEngine.sh --batch <Batch_ID> --streamname <Stream_Entity_Name> --instancename <PPM_Instance_Name>

    You can find the following message from FlowEngine.log under <VDW_HOME>/logs if the content pack is deployed successfully:

    ETL process was executed successfully

    • <Batch_ID>: The batch ID that was generated in Step 5.
    • <Stream_Entity_Name>: Should be the same as defined in stream_name of the stream entity JSON file.
    • <PPM_Instance_Name>: The PPM instance name you specified when installing the Vertica for PPM content pack.
  7. Connect to the Vertica database and check whether the data has been loaded successfully:

    select * from <Target_Schema>.DIM_<Target_Entity_Name>

    Or

    select * from <Target_Schema>.FACT_<Target_Entity_Name>
    • <Target_Schema>: The name for the schema that contains target data and tables for reporting.
    • <Target_Entity_Name>: Should be the same as defined in entity_name of the target entity JSON file.

To find the new target tables in views, you need to manually update the views in the Vertica database with the following SQL queries:

CREATE OR REPLACE VIEW <vdwtarget_schema>.<customized target table>_V
AS
SELECT <OOTB target table>.*, <customized target table>.<extended field> as extended_field
FROM <vdwtarget_schema>.<CUSTOMIZED target table> right join <vdwtarget_schema>.<OOTB target table> on
<vdwtarget_schema>.<CUSTOMIZED target table>.MD_ENTERPRISE_KEY = <vdwtarget_schema>.<OOTB target table>.MD_ENTERPRISE_KEY;

Replace the following variables:

  • <vdwtarget_schema>: The schema that contains the target tables
  • <customized target table>: The target table that is created
  • <OOTB target table>: The OOTB target table that is customized
  • <extended field>: The field that is to be extended

You can find all views on specified tables in the PPM9.50Data Model Guide .

Note the following when deploying the content packs:

  • Always deploy CUSTOMIZATION_TARGET.cp before deploying CUSTOMIZATION_PPM.cp.
  • If you change the content packs after deployment, you need to run ContentManager.sh again to make the changes effective.
  • You need to create a Shell to call the Extractor Engine and Flow Engine, and run ETL on a regular basis with crontab. You also need to make sure the script can run after the vdwetljob.sh process completes. For how to create the Shell script, you can take vdwCustomizationEtlJob.sh that is included in the sample package as an example.

    For instructions on obtaining the sample package, see Obtaining Sample Content Packs for Customization.

For more information about these scripts, refer to "Administration Tasks" of the Vertica for PPM Administrator Guide for Content Pack 1.0.