Kubernetes configuration settings

This section describes how to configure settings for a Kubernetes image on the LoadRunner Enterprise server.

In this topic:

Configure Kubernetes settings

When using a Kubernetes orchestrator with Dockerized host images, you can configure the following Kubernetes settings in the kubernetes.config file:

• Select the Kubernetes image pull policy.

• Define how Kubernetes containerized load generators are accessed. You can now connect to pods (containerized load generators) via a node port or a load balancer using an external IP.

• Set timeout to get load balancer external IPs and a delay for connecting the Controller to the pods.

• Use annotations to attach non-identifying metadata to Kubernetes services.

• Disable the service account auto-mount in Kubernetes.

• Enable Kubernetes health probes.

• Disable TLS verification for requests sent to a Kubernetes cluster.

• Specify security settings for all containers in the pod.

Back to top

Configure the image pull policy

When creating the pod, you can specify the image pull policy, which guides the kubelet service on how to pull the specified image during an update. Kubernetes has three image pull policies: IfNotPresent, Always, and Never.

To configure the image pull policy:

1. Open the kubernetes.config file located in the <LRE server installation path>\dat folder.

2. Under <appsettings>, add a key named "ImagePullPolicy" with one of the following values:

   AKS	EKS
   "Pull If Not Present"	"IfNotPresent"	Kubernetes only pulls the image when it does not already exist on the node. This is the default value if no value is set, or if the pull policy key is missing from the Kubernetes configuration file.
   "Pull Always"	"Always"	Kubernetes always pull the image from the registry when updating the container.
   "Pull Never"	"Never"	Kubernetes never pulls the image.

   Copy code

   Example of configuring the Kubernetes image pull policy for EKS:

   <?xml version="1.0" encoding="utf-8" ?>
   <configuration>
     <appSettings>
       <add key="DeploymentAPIVersion" value="apps/v1"/>
       <add key="PodAPIVersion" value="v1"/>
       <add key="ServiceAPIVersion" value="v1"/>
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="ImagePullPolicy" value="Always" />
     </appSettings>
   </configuration>

3. If you are using a Kubernetes cluster with a private registry that requires authorization, you also need to perform the following:

   1. In Docker Hub, log in to your private registry by running docker login

   2. Create a secret of your private registry to pull a private image:

      Copy code

      kubectl create secret generic <secret name> \   
         --from-file=.dockerconfigjson=<path/to/.docker/config.json> \   
         --type=kubernetes.io/dockerconfigjson

   3. Keep a record of your registry secret. You need to add it when configuring the orchestrator settings in LoadRunner Enterprise Administration (Orchestration > Secret Name box). For details, see Add and configure the orchestrator.

4. Restart IIS after updating the kubernetes.config file so that the changes take effect immediately (run 'iisreset' in a command line).

Back to top

Define containerized load generator access

You can define how the Kubernetes pods, or containerized load generators, are accessed based on your cluster configuration. If you use a load balancer such as AKS, you must use the cloud provider's load balancer. If you are using Kubernetes on-premises, you can connect to pods via a node port or a load balancer using an external IP. Node means the machine which runs the pods or containers.

Example: This diagram shows the implementation of the Load balancer service with external IP flow:

To define how containerized load generators are accessed:

1. Open the kubernetes.config file in the <LRE server installation path>\dat folder.

2. Under <appsettings>, add a key named "KubernetesAccessType" with one of the following values:

   • "NodePortNodeNames". Exposes the service on an internal IP in the cluster. This is the default value if no value is set, or if the access type key is missing from the Kubernetes configuration file.

   • "NodePortExternalIPs". Exposes services to external clients via a static port on each node's IP.

   • "LoadBalancerIngress". Exposes the service to external clients via the cloud provider's load balancer. This enables provisioning load generators that are set up in different networks to the Controller. When using this access type, you can set a timeout for waiting for all Load Balancer IPs to be allocated. For details, see Set load balancer IP timeout and pod connection delay.

     The external IP is displayed in the performance test as follows:

     

   Copy code

   Example of configuring Kubernetes with "NodePortExternalIPs" access type

   <?xml version="1.0" encoding="utf-8" ?>
   <configuration>
     <appSettings>
       <add key="DeploymentAPIVersion" value="apps/v1"/>
       <add key="PodAPIVersion" value="v1"/>
       <add key="ServiceAPIVersion" value="v1"/>
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="KubernetesAccessType" value="NodePortExternalIPs"/>
     </appSettings>
   </configuration>

3. Restart IIS after updating the kubernetes.config file so that the changes take effect immediately (run 'iisreset' in a command line).

Back to top

Set load balancer IP timeout and pod connection delay

When an external traffic policy is set using the "LoadBalancerIngress" access type, you can configure the following settings:

• Add a timeout for getting all Load Balancer IPs. This is the amount of time, in minutes, that LoadRunner Enterprise waits for all Load Balancer IPs to be allocated. This is useful where numerous Docker load generators are provisioned in the test.

• Determine how much time, in seconds, the Controller waits before connecting to the pods. The default delay time is 15 seconds. We recommend increasing the value if you encounter connecting to host (Docker LGs) errors.

To set a timeout for Kubernetes to get load balancer external IPs:

1. Open the kubernetes.config file located in the <LRE server installation path>\dat folder.

2. Under <appsettings>, add a key named "WaitForIngressIPsTimeOut" and enter an integer of 0 or higher. If no value is set, the default timeout is 5 minutes.

3. To set a wait time for connecting to the pods, add a key named "ConnectToPodDelay" and enter a value in seconds.

   Copy code

   Example of timeout for load balancer external IPs and delay connecting to pod:

   <?xml version="1.0" encoding="utf-8" ?>
   <configuration>
     <appSettings>
       <add key="DeploymentAPIVersion" value="apps/v1"/>
       <add key="PodAPIVersion" value="v1"/>
       <add key="ServiceAPIVersion" value="v1"/>
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="KubernetesAccessType" value="LoadBalancerIngress"/>
       <add key="WaitForIngressIPsTimeOut" value="5"/> <!--in minutes-->
       <add key="ConnectToPodDelay" value="15"/> <!--in seconds-->
     </appSettings>
   </configuration>

4. Restart IIS after updating the kubernetes.config file so that the changes take effect immediately (run 'iisreset' in a command line).

Back to top

Add Kubernetes services annotations

You can use annotations to attach non-identifying metadata to Kubernetes services.

To define annotations in services:

1. Open the kubernetes.config file located in the <LRE server installation path>\dat folder.

2. Under <appsettings>, add a key named "Annotations" and enter a key and value using the following format:

   <add key="Annotations" value="{'key':'value'}"/> for a single annotation.

   <add key="Annotations" value="{'key1':'value1','key2':'value2'}"/> for multiple annotations.

   Copy code

   Example of defining an annotation:

   <?xml version="1.0" encoding="utf-8" ?>
   <configuration>
     <appSettings>
       <add key="DeploymentAPIVersion" value="apps/v1"/>
       <add key="PodAPIVersion" value="v1"/>
       <add key="ServiceAPIVersion" value="v1"/>
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="Annotations" value="{'service.beta.kubernetes.io/azure-load-balancer-internal':'true'}" />
      </appSettings>
   </configuration>

3. Restart IIS after updating the kubernetes.config file so that the changes take effect immediately (run 'iisreset' in a command line).

Back to top

Disable the service account auto-mount

By default, a service account is mounted to every pod. If the application doesn’t require access to the Kubernetes API, disable the service account auto-mount in Kubernetes. Disabling service auto-mount hardens cluster accessibility to the API server.

To disable the auto-mount service account token:

1. Open the kubernetes.config file located in the <LRE server installation path>\dat folder.

2. Under <appsettings>, add a key named "AutoMountServiceAccountToken" with a value of "false".

   Copy code

   Example of disabling the service auto-mount:

   <?xml version="1.0" encoding="utf-8" ?>
   <configuration>
     <appSettings>
       <add key="DeploymentAPIVersion" value="apps/v1"/>
       <add key="PodAPIVersion" value="v1"/>
       <add key="ServiceAPIVersion" value="v1"/>
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="WaitForIngressIPsTimeOut" value="5"/> <!--in minutes-->
       <add key="CreateDeploymentTimeout" value="10"/> <!--in seconds-->
       <add key="AutomountServiceAccountToken" value="false"/> <!--This is used for hardening cluster accessibility to the API server.-->
       <add key="ConnectToPodDelay" value="15"/> <!--in seconds-->
     </appSettings>
   </configuration>

Back to top

See also:

• Provision Docker hosts automatically