Constant load test

Description

Constant load tests run on principle that all requests that are defined in the test case and set as property 'testCase' will be run in loop at a constant request rate per second set as property 'rate' for duration of time that is set as property 'duration'.

test: load `(mandatory) Property key and constant value to define the test yaml type as test load`
name: string `(mandatory) Name label of test load case
loadtype: "constant" `(mandatory)
details:
    setupCases: [string, string,...] `List of test case names which will be executed before main tests` [optional]
    testCase: string `(mandatory) Name of test case for load scenario that is defined under project scope`
    finalCases: [string, string,...] `List of test case names which will be executed after all test has been executed` [optional]
    environment: string `Name of default environment for load test execution` [optional]
    rate: int `Requests per second, constant load rate`
    duration: string `(mandatory) Duration of constant load test in seconds(s), minutes(m), hours(h) - 10s, 10m, 1h`
    workerType: string `Worker type that are setup for the load test`
    secondaryWorker: string `Worker type for setup and final case execution, for remote run` [optional]
    workers: `list of workers through which load test will be distributed` [optional]
        -   worker: string `Type name of worker`
            rate: int `Number of users for particular worker`
        -   ...
    input-positions:
        fromList: `Dictionary which holds variables as keys and list of values for each of them`
            (varname:string): [(varvalue:string|int), varvalue, varvalue...]
            example:
            $variable1: ["var value", "var value 2", "var value 3"]
            $variable2: ["var value"]
        fromVar: `Dictionary which holds variables as keys and variables that are resolved to list during runtime`
            (varname:string): (varvalue:string|int)
            example:
            $variable1: $varList1
            $variable2: $varList2
    maxIdleConnections: int `Number of maximum idle connections` [optional] `500 by default`
    asserts: `List of performance type assertions` [optional]
        -   type: string ("latency"|"wait"|"requestrate"|"requestcount"|"status")
            metric: string ("max"|"mean"|"50th"|"95th"|"99th"|"total"|status code)
            operator: ("eq"|"ne"|"gt"|"lt"|"ge"|"le"|"regex") `operator values, defaults to 'le' less than or equals`
            expected: (string|int) `Latency,wait types: duration in milliseconds(ms), seconds(s) or minutes(m); requestrate and requestcount types: int; status type: int(response count)`

Example

test: load
name: constantLoginLoadTest
loadtype: constant
details:
    testCase: Login
    environment: Staging 
    rate: 1
    duration: 60s
    workerType: loc
    maxIdleConnections: 400
    httpTimeout: 25s
    distribution: random
    asserts:
      - type: latency
        metric: 50th
        operator: lt
        expected: 5s