Simulation load test

Description

Simulation load tests run on principle that one test case can be executed for a number of users within a duration of certain time frame. The user load can be distributed across many worker types. The distribution is random as it usually is in production. Although the load will be very close to average request per second metric.

test: load `(mandatory) property key and constant value to define the test yaml type as test load`
name: string `(mandatory) Name label of test load case`
loadtype: "simulation" `(mandatory)
details:
    setupCases: [string, string,...] `List of test case names which will be executed before main tests` [optional]
    testCase: string `(mandatory) Name of test case for load scenario that is defined under project scope`
    finalCases: [string, string,...] `List of test case names which will be executed after all test has been executed` [optional]
    environment: string `Name of default environment for load test execution` [optional]
    users: int `Number of total users in scope of which testCase will be executed`
    duration: string `(mandatory) duration of load test scenario in seconds(s), minutes(m), hours(h)`
    distribution: "random" `For now only random distribution is allowed as value`
    secondaryWorker: string `Worker type for setup and final case execution, for remote run` [optional]
    workers: `list of workers through which load test will be distributed`
        -   worker: string `Type name of worker`
            users: int `Number of users for particular worker`
        -   ...
    input-positions:
        fromList: `Dictionary which holds variables as keys and list of values for each of them`
            (varname:string): [(varvalue:string|int), varvalue, varvalue...]
            example:
            $variable1: ["var value", "var value 2", "var value 3"]
            $variable2: ["var value"]
        fromVar: `Dictionary which holds variables as keys and variables that are resolved to list during runtime`
            (varname:string): (varvalue:string|int)
            example:
            $variable1: $varList1
            $variable2: $varList2
    writeVariablesInYamlToFile: string `Path to yaml file to which to write all load test scoped (load test, setup and final test cases) variables` [optional]
    writeVariablesToFile: string `Path to ini type file to which to write all load test scoped (load test, setup and final test cases) variables` [optional]
    asserts: `List of performance type assertions` [optional]
        -   type: string ("latency"|"wait"|"requestrate"|"requestcount"|"status")
            metric: string ("max"|"mean"|"50th"|"95th"|"99th"|"total"|status code)
            operator: ("eq"|"ne"|"gt"|"lt"|"ge"|"le"|"regex") `operator values, defaults to 'le' less than or equals`
            expected: (string|int) `Latency,wait types: duration in milliseconds(ms), seconds(s) or minutes(m); requestrate and requestcount types: int; status type: int(response count)`

Example

test: load
name: loginLoadTest
loadtype: simulation
details:
  setupCases: ["Create account"]
  finalCases: ["Delete user account"]
  testCase: Login
  users: 100
  duration: 180s
  distribution: random
  workers:
    - worker: loc
      users: 100
  input-positions:
    fromList:
      $userName: ["testUser"]
      $userPassword: ["password"]
  asserts:
    - type: latency
      metric: 95th
      expected: 5s

results matching ""

    No results matching ""