Appium simulations

You may need to test an app that uses the camera for taking a photo or scanning a barcode, or that uses authentication to secure sensitive features such as sign-on or payments. This poses a problem if the device you are testing is not physically available. Using simulations, you can test these aspects of your app without needing access to the physical device.

Note: Simulations are supported only for packaged apps. To install and launch a packaged app, set the capability installPackageApp to true.

Biometrics

You can simulate fingerprint and Face ID (iOS only) authentications in your Appium test. Use the following parameters and values to specify the data to be used for simulation.

SimulationData

Parameters Values
authResult

The result of the simulation:

  • Success

  • Failure

  • Cancel

authType

The authentication type:

Fingerprint

Note: The Fingerprint authType is also used for FaceID.

authResultDetails

The reason for the result, when simulation fails or is canceled. Leave this value empty when authResult = Success.

authResult = Failure

  • NotRecognized
    The biometric data being used does not match the registered data. For example, the owner of the device registered their biometric data, and someone else tried to unlock the device using their own data.

  • Lockout
    Too many failed attempts.
  • FingerIncomplete
    Android only: Finger moving too fast.
  • SensorDirty
    Android only: Fingerprint sensor is dirty.
  • NoFingerprintRegistered
    iOS only: No authentication method was registered on the device. For example, the owner of the device did not register their TouchID/ FaceID but the application being tested requires TouchID/ FaceID at some point.
 

authResult = Cancel

  • User
    The user canceled the authentication
  • System
    The system canceled the authentication.

action

This specifies the type of simulation to perform during the test:

Parameters Values
action

authentication

Back to top

Photo, barcode, and QR code simulation

You can simulate taking a photo, or scanning a barcode or a QR code in your Appium test. When using camera simulation, you specify the image to use instead of the actual output of the camera.

Parameters Values
contentType image
filename
Any file name, including the suffix.
uploadMedia The media should be encoded in base64.
action

The type of simulation being performed:

  • barcode

  • camera

Note: To turn off camera simulation, set uploadMedia to "0".

Back to top

Code samples

The following section provides code samples for using Digital Lab camera and authentication simulation with Appium. For a working project using Appium and Digital Lab, see the Digital Lab github repository.

Android authentication and camera simulation - Java

The sample code uses the Java-Client v8. For Java-Client v7.x, see the example in the Digital Lab github repository.

Copy code
import com.google.common.io.Resources;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import io.appium.java_client.AppiumBy;
import io.appium.java_client.TouchAction;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.options.UiAutomator2Options;
import io.appium.java_client.touch.offset.PointOption;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.RemoteWebElement;
import java.awt.*;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.util.Base64;
import java.util.HashMap;
import java.util.Objects;
public class Simulation {
    private static AndroidDriver driver;
    private static String SERVER = "https://digitallab-server:8443";
    public static void main(String[] args) throws MalformedURLException {
        UiAutomator2Options caps = new UiAutomator2Options();
        caps.setCapability("platformName", "Android");
        caps.setCapability("udid", "8BSX1EDTD");
        caps.setCapability("automationName", "UiAutomator2");
        caps.setCapability("appPackage", "com.Advantage.aShopping");
        caps.setCapability("appActivity", "com.Advantage.aShopping.SplashActivity");
        caps.setCapability("dl:oauthClientId", "oauth2-tGR1nN6ZxLQNMrKH4Hq9@opentext.com");
        caps.setCapability("dl:oauthClientSecret", "A3BNk1Tc4gbf7o2jDU5m");
        caps.setCapability("dl:tenantId", "999999999");
        //Simulations are supported only on packaged apps. The below capability instructs Digital Lab to install the packaged version of the app. Default value is false
        caps.setCapability("dl:installPackagedApp", true);
        driver = new AndroidDriver(new URL(SERVER + "/wd/hub"), caps);
        System.out.println("Digital Lab session was successfully created [Android device]");
        try{
            //Implicit wait
            driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
            driver.findElement(AppiumBy.id("imageViewMenu")).click();
            driver.findElement(AppiumBy.id("textViewMenuUser")).click();
            WebElement username = driver.findElement(AppiumBy.xpath("//*[@resource-id='com.Advantage.aShopping:id/AosEditTextLoginUserName']/android.widget.EditText[1]"));
            username.click();
            username.sendKeys("Mercury");
            WebElement password = driver.findElement(AppiumBy.xpath("//*[@resource-id='com.Advantage.aShopping:id/AosEditTextLoginPassword']/android.widget.EditText[1]"));
            password.click();
            password.sendKeys("Mercury");
            driver.findElement(AppiumBy.id("buttonLogin")).click();
            Thread.sleep(3000);
            String message = driver.findElement(AppiumBy.id("android:id/message")).getText();
            if(message.contains("Would you like to use fingerprint authentication for logging in ?")){
                driver.findElement(AppiumBy.id("android:id/button1")).click();
                //Perform biometric simulation
                System.out.println("Biometric simulation result:" + biometricSimulation("Success",""));
            }
            driver.findElement(AppiumBy.androidUIAutomator("new UiSelector().textContains(\"LAPTOPS\")")).click();
            driver.findElement(AppiumBy.androidUIAutomator("new UiSelector().textContains(\"HP CHROMEBOOK 14 G1(ENERGY STAR)\")")).click();
            int requiredSwipes = driver.findElements(AppiumBy.xpath("//*[@resource-id='com.Advantage.aShopping:id/linearLayoutImagesLocation']/android.widget.FrameLayout")).size();
            WebElement imageList = driver.findElement(AppiumBy.id("viewPagerImagesOfProducts"));
            for(int i = 1;i <= requiredSwipes;i++){
                swipeElement(((RemoteWebElement) imageList).getId(),"up", 1);
            }
            //load the photo before opening the camera
            System.out.println("Photo simulation result:" + cameraSimulation("image","cat.png","camera"));
            imageList.click();
            //The steps below are performed on the device's camera and correspond to a device with Android 12.
            //These steps may change depending on the device model/brand since the camera app might be different.
            driver.findElement(AppiumBy.accessibilityId("Take photo")).click();
            driver.findElement(AppiumBy.accessibilityId("Done")).click();
            //Barcode simulation sample usage:
            //System.out.println(cameraSimulation("image","QRCode.jpg","barcode"));
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            if (driver != null) driver.quit();
        }
    }
    /**
     * Biometric authentication simulation enable/update command.
     * @param authResult The simulate result: Failure, Success, or Cancel
     * @param authResultDetails The simulate reason when Failure/Cancel
     *     Failure: NotRecognized, Lockout, FingerIncomplete(Android only), SensorDirty(Android only), NoFingerprintRegistered(iOS only)
     *     Cancel: System, User
     * @return The result of ExecuteScript
     */
    private static String biometricSimulation(String authResult, String authResultDetails) throws InterruptedException {
        HashMap<String, Object> sensorSimulationMap = new HashMap<String, Object>();
        HashMap<String, String> simulationData = new HashMap<String, String>();
        simulationData.put("authResult", authResult);
        simulationData.put("authType", "Fingerprint");
        simulationData.put("authResultDetails", authResultDetails);

        sensorSimulationMap.put("simulationData", simulationData);
        sensorSimulationMap.put("action", "authentication");
        sensorSimulationMap.put("action", "authentication");        

        //Execute the script and convert the result to a JSON string
        String simulationResult = new Gson().toJson(driver.executeScript("mc:sensorSimulation", sensorSimulationMap));
        //Return just the message value from simulationResult
        return new Gson().fromJson(simulationResult, JsonObject.class).get("message").getAsString();
    }
    /**
     *
     * @param contentType Image
     * @param fileName Any file name
     * @param action Action: barcode, camera
     * @return
     */
    private static String cameraSimulation(String contentType, String fileName, String action) throws IOException, URISyntaxException, InterruptedException {
        Path path = Paths.get(Objects.requireNonNull(Resources.getResource("photo/" + fileName)).toURI());
        byte[] bytes = Files.readAllBytes(path);
        String encodedString = Base64.getEncoder().encodeToString(bytes);
        HashMap<String, String> sensorSimulationMap =  new HashMap<>();
        sensorSimulationMap.put("uploadMedia", encodedString);
        sensorSimulationMap.put("contentType", contentType);
        sensorSimulationMap.put("fileName", fileName);
        sensorSimulationMap.put("action", action);
        //Execute the script and convert the result to a JSON string
        String simulationResult = new Gson().toJson(driver.executeScript("mc:sensorSimulation", sensorSimulationMap));
        //Return just the message value from simulationResult
        return new Gson().fromJson(simulationResult, JsonObject.class).get("message").getAsString();
    }
    /**
     *
     * @param elementId The id of the element to be swiped
     * @param direction Swipe direction. Mandatory value. Acceptable values are: up, down, left and right (case insensitive)
     * @param percent The size of the swipe as a percentage of the swipe area size. Valid values must be float numbers in range 0..1, where 1.0 is 100%
     */
    private static void swipeElement(String elementId, String direction, double percent){
        HashMap<String, Object> swipeMap =  new HashMap<>();
        swipeMap.put("elementId", elementId);
        swipeMap.put("direction", direction);
        swipeMap.put("percent", percent);
        driver.executeScript("mobile: swipeGesture", swipeMap);
    }
}

Android authentication and camera simulation - C# .Net

Copy code
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Enums;
using OpenQA.Selenium.Appium.Android;
using OpenQA.Selenium;
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using System.Threading;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.IO;
using System.Drawing.Imaging;
using System.Drawing;
using Appium_Simulations.Resources;
using OpenQA.Selenium.Remote;

namespace Appium_Simulations
{
    public class Android
    {
        public static AndroidDriver driver;
        public static String DL_SERVER = "http://digitallab-server:8080";
        //Update the below variable to indicate whether you want to use Appium v2 or the default Appium v1
        private static Boolean useAppiumV2 = true;

        static void Main(string[] args)
        {            
            AppiumOptions appiumOptions = new AppiumOptions();

            //Digital Lab  embedded Appium capabilities                                                                                 
            appiumOptions.AddAdditionalAppiumOption("appium:udid", "8BSX1EDTD");            
            appiumOptions.AddAdditionalAppiumOption("appium:appPackage", "com.Advantage.aShopping");
            appiumOptions.AddAdditionalAppiumOption("appium:appActivity", "com.Advantage.aShopping.SplashActivity");
            appiumOptions.AddAdditionalAppiumOption("dl:oauthClientId", "oauth2-UNmB6dXe4XNzJpjY4GSg@opentext.com");
            appiumOptions.AddAdditionalAppiumOption("dl:oauthClientSecret", "3Rd1wgB0g71KBT6S2tv3");
            appiumOptions.AddAdditionalAppiumOption("dl:tenantId", "999999999");

            //Simulations are supported only on packaged apps. The below capability instructs Digital Lab to install the packaged version of the app. Default value is false
            appiumOptions.AddAdditionalAppiumOption("dl:installPackagedApp", true);
            System.Uri url = new System.Uri(string.Format("{0}/wd/hub", DL_SERVER));            

            try
            {
                driver = new AndroidDriver(url, appiumOptions);
                System.Console.WriteLine("Digital Lab session was successfully created [Android device]");

                //Implicit wait
                driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);
                driver.FindElement(MobileBy.Id("imageViewMenu")).Click();
                driver.FindElement(MobileBy.Id("textViewMenuUser")).Click();
                WebElement username = driver.FindElement(MobileBy.XPath("//*[@resource-id='com.Advantage.aShopping:id/AosEditTextLoginUserName']/android.widget.EditText[1]"));
                username.Click();
                username.SendKeys("Mercury");
                WebElement password = driver.FindElement(MobileBy.XPath("//*[@resource-id='com.Advantage.aShopping:id/AosEditTextLoginPassword']/android.widget.EditText[1]"));
                password.Click();
                password.SendKeys("Mercury");
                driver.FindElement(MobileBy.Id("buttonLogin")).Click();
                Thread.Sleep(3000);
                String message = driver.FindElement(MobileBy.Id("android:id/message")).Text;
                if (message.Contains("Would you like to use fingerprint authentication for logging in ?"))
                {
                    driver.FindElement(MobileBy.Id("android:id/button1")).Click();
                    //Perform biometric simulation
                    Console.WriteLine("Biometric simulation result:" + BiometricSimulation("Success", ""));
                }
                driver.FindElement(MobileBy.AndroidUIAutomator("new UiSelector().textContains(\"LAPTOPS\")")).Click();
                driver.FindElement(MobileBy.AndroidUIAutomator("new UiSelector().textContains(\"HP CHROMEBOOK 14 G1(ENERGY STAR)\")")).Click();
                int requiredSwipes = driver.FindElements(MobileBy.XPath("//*[@resource-id='com.Advantage.aShopping:id/linearLayoutImagesLocation']/android.widget.FrameLayout")).Count;
                WebElement imageList = driver.FindElement(MobileBy.Id("viewPagerImagesOfProducts"));
                for (int i = 1; i <= requiredSwipes; i++)
                {                                        
                    swipeElement(imageList.Location, imageList.Size, "up", 1);
                }
                //load the photo before opening the camera                
                Console.WriteLine("Photo simulation result:" + CameraSimulation(SimulationResources.cat, ImageFormat.Png, contentType: "image", fileName: "cat.png", action: "camera"));
                imageList.Click();
                //The steps below are performed on the device's camera and correspond to a device with Android 12.
                //These steps may change depending on the device model/brand since the camera app might be different.
                driver.FindElement(MobileBy.AccessibilityId("Take photo")).Click();
                driver.FindElement(MobileBy.AccessibilityId("Done")).Click();

                //Barcode simulation sample usage:
                //Console.WriteLine(CameraSimulation(SimulationResources.QRCode, ImageFormat.Png, contentType: "image", fileName: "QRCode.png", action: "barcode"));                
            }
            catch (Exception e)
            {
                Console.WriteLine(e.StackTrace);
            }
            finally
            {
                if (driver != null)
                    driver.Quit();
            }
        }

        /// <summary>
        /// Biometric authentication simulation enable/update command.
        /// </summary>
        /// <param name="authResult">The simulate result, including Failure, Success, Cancel</param>
        /// <param name="authResultDetails">
        /// The simulate reason when Failure/Cancel
        /// Failure: NotRecognized, Lockout, FingerIncomplete(Android only), SensorDirty(Android only), NoFingerprintRegistered(iOS only)
        /// Cancel: System, User
        /// </param>
        /// <returns>The result of ExecuteScript</returns>
        public static string BiometricSimulation(string authResult, string authResultDetails = "")
        {
            Dictionary<string, object> sensorSimulationMap = new Dictionary<string, object>();
            Dictionary<string, string> simulationData = new Dictionary<string, string>
            {
                { "authResult", authResult },
                { "authType", "Fingerprint" },
                { "authResultDetails", authResultDetails }
            };
            sensorSimulationMap.Add("simulationData", simulationData);
            sensorSimulationMap.Add("action", "authentication");                       

            string simulationResult = JsonConvert.SerializeObject(driver.ExecuteScript("mc:sensorSimulation", sensorSimulationMap));
            return JToken.Parse(simulationResult)["message"].ToString();
        }

        /// <summary>
        /// 
        /// </summary>
        /// <param name="picture"></param>
        /// <param name="format"></param>
        /// <param name="contentType">image</param>
        /// <param name="fileName"></param>
        /// <param name="action">Action: barcode, camera</param>
        /// <returns></returns>
        public static string CameraSimulation(Bitmap picture, ImageFormat format, string contentType, string fileName, string action)
        {
            MemoryStream ms = new MemoryStream();
            picture.Save(ms, format);
            Byte[] bytes = ms.ToArray();
            string encodeString = Convert.ToBase64String(bytes);

            Dictionary<string, string> sensorSimulationMap = new Dictionary<string, string>
            {
                { "uploadMedia", encodeString },
                { "contentType", contentType },
                { "fileName", fileName },
                { "action", action }
            };

            string simulationResult = JsonConvert.SerializeObject(driver.ExecuteScript("mc:sensorSimulation", sensorSimulationMap));
            return JToken.Parse(simulationResult)["message"].ToString();
        }


        /// <summary>
        /// Performs swipe on element
        /// </summary>        
        /// <param name="location">Object location</param>
        /// <param name="size">Object size</param>
        /// <param name="direction">Swipe direction. Mandatory value. Acceptable values are: up, down, left and right (case insensitive)</param>
        /// <param name="percent">The size of the swipe as a percentage of the swipe area size. Valid values must be float numbers in range 0..1, where 1.0 is 100%</param>

        private static void swipeElement(Point location, Size size, String direction, double percent)
        {            
            Dictionary<string, object> swipeMap = new Dictionary<string, object>
            {
                { "left", 0},
                { "top", 0},
                { "width", size.Width },
                { "height", size.Height + location.Y - 30},
                { "direction", direction },
                { "percent", percent }
            };
            
            driver.ExecuteScript("mobile: swipeGesture", swipeMap);
        }

    }
}

 

For additional code samples, see Appium integration and Appium code example - Java