Rally

rps runner type in rally

Asked by Rajeev

I've modified the runner type from "constant" to "rps" and i have provided the following values in my JSON file:

"runner": {
                "type": "rps",
                "times": 10,
                "rps": 2,
                "anyOf": {
                    "start": 1,
                    "end": 12,
                    "step": 2,
                }
            },
=========================
I'm getting an error when i execute the benchmark scenario. Error is as shown below:

Reason:
 Additional properties are not allowed ('anyOf' was unexpected)

Failed validating 'additionalProperties' in schema:
    {'$schema': 'http://json-schema.org/draft-04/schema',
     'additionalProperties': False,
     'properties': {'max_concurrency': {'minimum': 1, 'type': 'integer'},
                    'max_cpu_count': {'minimum': 1, 'type': 'integer'},
                    'rps': {'exclusiveMinimum': True,
                            'minimum': 0,
                            'type': 'number'},
                    'timeout': {'type': 'number'},
                    'times': {'minimum': 1, 'type': 'integer'},
                    'type': {'type': 'string'}},
     'required': ['type', 'times', 'rps'],
     'type': 'object'}

On instance:
    {'anyOf': {'end': 12, 'start': 1, 'step': 2},
     'rps': 2,
     'times': 10,
     'type': 'rps'}`

====================================================================

I modified the json file with :
1. "properties" instead of "anyOf", but with no luck.
2. specified "step","start" and "end" within "rps" as hash-of-hash and it didnt work as well

I want to know how to specify "start","end" and "step" values for "rps" runner type?

from the plugin reference, i can see that these values can be used for rps config:

class RPSScenarioRunner(runner.ScenarioRunner):
    """Scenario runner that does the job with specified frequency.
    Every single benchmark scenario iteration is executed with specified
    frequency (runs per second) in a pool of processes. The scenario will be
    launched for a fixed number of times in total (specified in the config).
    An example of a rps scenario is booting 1 VM per second. This
    execution type is thus very helpful in understanding the maximal load that
    a certain cloud can handle.
    """

    CONFIG_SCHEMA = {
        "type": "object",
        "$schema": consts.JSON_SCHEMA,
        "properties": {
            "type": {
                "type": "string"
            },
            "times": {
                "type": "integer",
                "minimum": 1
            },
            "rps": {
                "anyOf": [
                    {
                        "type": "number",
                        "exclusiveMinimum": True,
                        "minimum": 0
                    },
                    {
                        "type": "object",
                        "properties": {
                            "start": {
                                "type": "number",
                                "minimum": 1
                            },
                            "end": {
                                "type": "number",
                                "minimum": 1
                            },
                            "step": {
                                "type": "number",
                                "minimum": 1
                            },
                            "duration": {
                                "type": "number",
                                "minimum": 1
                            }
                        },
                        "required": ["start", "end", "step"]
                    }
                ],
            },
            "timeout": {
                "type": "number",
            },
            "max_concurrency": {
                "type": "integer",
                "minimum": 1
            },
            "max_cpu_count": {
                "type": "integer",
                "minimum": 1
            }
        },
        "required": ["type", "times", "rps"],
        "additionalProperties": False
    }

Question information

Language:
English Edit question
Status:
Solved
For:
Rally Edit question
Assignee:
No assignee Edit question
Solved by:
Andrey Kurilin
Solved:
Last query:
Last reply:
Revision history for this message
Andrey Kurilin (andreykurilin) said : 		#1

Hi!
You are trying to put jsoschema format as json. It is a wrong way:) JSONSchema is designed to check input format, so it can include several use cases(with different input argument). AnyOf is a keyword for splitting such cases in JSONShhema.

In json it should look like:

"runner": {
                "type": "rps",
                "times": 10,
                "rps": {
                    "start": 1,
                    "end": 12,
                    "step": 2,
                }
            }

An example from gates - https://github.com/openstack/rally/blob/master/rally-jobs/rally.yaml#L530-L536

Revision history for this message
Rajeev (rsomayaj) said : 		#2

Hi Andrey,

I had tried the same way which you have mentioned above, but with no luck. Its expecting the "rps" to be a number. Please find below rally task console logs:

 rally task start boot-from-volume-and-delete.json
--------------------------------------------------------------------------------
 Preparing input task
--------------------------------------------------------------------------------

Input task is:
{
    "NovaServers.boot_server_from_volume_and_delete": [
        {
            "args": {
                "flavor": {
                    "name": "m1.small"
                },
                "image": {
                    "name": "cirros"
                },
                "volume_size": 1,
                "volume_type": "iscsi",
                "force_delete": false,
                "auto_assign_nic": true
            },
            "runner": {
                "type": "rps",
                "times": 5,
                "rps": {
                    "start": 1,
                    "step": 2,
                    "end": 10,
                },
            },
            "context": {
                "users": {
                    "tenants": 1,
                    "users_per_tenant": 1
                },
                "network": {
                    "start_cidr": "10.2.0.0/24",
                    "networks_per_tenant": 2
                }
            }
        }
    ]
}

Task syntax is correct :)
2016-11-28 01:49:03.946 19360 INFO rally.task.engine [-] Task 6b5da8f6-3a90-461d-80c4-4192f94a42a7 | Starting: Task validation.
2016-11-28 01:49:04.120 19360 INFO rally.task.engine [-] Task 6b5da8f6-3a90-461d-80c4-4192f94a42a7 | Starting: Task validation of scenarios names.
2016-11-28 01:49:04.123 19360 INFO rally.task.engine [-] Task 6b5da8f6-3a90-461d-80c4-4192f94a42a7 | Completed: Task validation of scenarios names.
2016-11-28 01:49:04.123 19360 INFO rally.task.engine [-] Task 6b5da8f6-3a90-461d-80c4-4192f94a42a7 | Starting: Task validation of syntax.
Task config is invalid: `Input task is invalid!

Subtask NovaServers.boot_server_from_volume_and_delete[0] has wrong configuration\Subtask configuration:
{'runner': {'rps': {'start': 1, 'step': 2, 'end': 10}, 'type': 'rps', 'times': 5}, 'args': {'force_delete': False, 'image': {'name': 'cirros'}, 'volume_type': 'iscsi', 'volume_size': 1, 'flavor': {'name': 'm1.small'}, 'auto_assign_nic': True}, 'context': {'users': {'users_per_tenant': 1, 'tenants': 1}, 'network': {'start_cidr': '10.2.0.0/24', 'networks_per_tenant': 2}}}

Reason:
 {'start': 1, 'step': 2, 'end': 10} is not of type 'number'

Failed validating 'type' in schema['properties']['rps']:
    {'exclusiveMinimum': True, 'minimum': 0, 'type': 'number'}

On instance['rps']:
    {'end': 10, 'start': 1, 'step': 2}`

===========
JSON file :

{
    "NovaServers.boot_server_from_volume_and_delete": [
        {
            "args": {
                "flavor": {
                    "name": "m1.small"
                },
                "image": {
                    "name": "cirros"
                },
                "volume_size": 1,
                "volume_type": "iscsi",
                "force_delete": false,
                "auto_assign_nic": true
            },
            "runner": {
                "type": "rps",
                "times": 5,
                "rps": {
                    "start": 1,
                    "step": 2,
                    "end": 10,
                },
            },
            "context": {
                "users": {
                    "tenants": 1,
                    "users_per_tenant": 1
                },
                "network": {
                    "start_cidr": "10.2.0.0/24",
                    "networks_per_tenant": 2
                }
            }
        }
    ]
}

-------------------
runner: {
    "type": "rps",
     "times": 10,
     "rps": 5,
},
..

When i specify the rps parameter with a number, it works. But the same scenario doesn't work with start,step and end specified.

I'm using:
nova-manage --version
2015.1.4

Let me know if I'm missing something.

Thanks,
Rajeev

Revision history for this message
Best Andrey Kurilin (andreykurilin) said : 		#3

"RPS with steps" behaviour was added to latest Rally and this feature is not included in any Rally release. I can assume that you have rally version without this feature.

Revision history for this message
Rajeev (rsomayaj) said : 		#4

Thanks Andrey Kurilin, that solved my question.