LLM protocol

The LLM (large language model) protocol lets you create scripts for testing LLMs, such as OpenAI, Ollama, and Gemini. This protocol allows you to develop scripts and test applications that have an embedded LLM. LLM-specific monitors and graphs show the LLM token input processing and decoding rates during a loadtest.

Create an LLM test

LLM scripts contain standard Web-HTTP/HTML functions with additional calls to custom LLM calls, such as llm_request.

Currently, only non-streaming AI is supported.

To create a script:

In VuGen, create a new script of the type LLM (Large Language Model).
Add the relevant functions as described in the Function Reference and the examples below.

Examples

The following example illustrates a script with an LLM request for OpenAI.

Copy code

web_add_header("Content-Type","application/json");
web_add_header("Authorization","<API_KEY>");
llm_request("OpenAI_Chat_Completion",
    "URL=https://api.openai.com/v1/chat/completions",
    "AIModel=openai",
    lr_eval_string("{request_body}"),
    LAST);
return 0;

The following example illustrates a script with an LLM request for Gemini.

Copy code

lr_param_sprintf ("request_body", "Body={ \"contents\":[ { \"parts\":[{\"text\": \"hello?\"}]} ] }");
web_add_header("Content-Type","application/json");
llm_request("Gemini_Chat_Completion",
     "URL=https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=<API_KEY> ",
     "AIModel=gemini",
      lr_eval_string("{request_body}"),
      LAST);