openstack-helm

vm creation goes into error - ceph backend - flavor has zero in disk

Asked by chinasubbareddy mallavarapu

once single node with ceph as backend comes up, if we try to boot vm with image size of 600MB, vms goes into error as timeout as error.

Error: Failed to perform requested operation on instance "vm2", the instance has an error status: Please try again later [Error: Build of instance df6816d2-f39b-4ed3-9175-60dd9b075f69 aborted: Volume c2ea0d78-5cb3-4a61-ad4d-c155e1f5b0e6 did not finish being created even after we waited 12 seconds or 5 attempts. And its status is error.].

Question information

Language:
English Edit question
Status:
Solved
For:
openstack-helm Edit question
Assignee:
No assignee Edit question
Solved by:
chinasubbareddy mallavarapu
Solved:
Last query:
Last reply:
Revision history for this message
chinasubbareddy mallavarapu (chinasubbareddy) said : 		#1

as per links in google we need to set these values :

block_device_allocate_retries=600
block_device_allocate_retries_interval=1

Revision history for this message
chinasubbareddy mallavarapu (chinasubbareddy) said : 		#2

error still there :

{"log":"2018-04-23 19:07:25.108 10999 INFO nova.virt.block_device [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Booting with blank volume at /dev/vda\n","stream":"stderr","time":"2018-04-23T19:07:25.112665477Z"}
{"log":"2018-04-23 19:07:36.299 10999 WARNING nova.compute.manager [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] Volume id: 1d5dc689-db6d-4a50-80e6-d891f38d5ef5 finished being created but its status is error.\n","stream":"stderr","time":"2018-04-23T19:07:36.299793876Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Instance failed block device setup\n","stream":"stderr","time":"2018-04-23T19:07:36.300544994Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Traceback (most recent call last):\n","stream":"stderr","time":"2018-04-23T19:07:36.300565108Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 1586, in _prep_block_device\n","stream":"stderr","time":"2018-04-23T19:07:36.300569929Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] wait_func=self._await_block_device_map_created)\n","stream":"stderr","time":"2018-04-23T19:07:36.300574719Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/virt/block_device.py\", line 514, in attach_block_devices\n","stream":"stderr","time":"2018-04-23T19:07:36.300579322Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] map(_log_and_attach, block_device_mapping)\n","stream":"stderr","time":"2018-04-23T19:07:36.300584124Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/virt/block_device.py\", line 512, in _log_and_attach\n","stream":"stderr","time":"2018-04-23T19:07:36.300588531Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] bdm.attach(*attach_args, **attach_kwargs)\n","stream":"stderr","time":"2018-04-23T19:07:36.300592724Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/virt/block_device.py\", line 404, in attach\n","stream":"stderr","time":"2018-04-23T19:07:36.300610695Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] self._call_wait_func(context, wait_func, volume_api, vol['id'])\n","stream":"stderr","time":"2018-04-23T19:07:36.300615038Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/virt/block_device.py\", line 363, in _call_wait_func\n","stream":"stderr","time":"2018-04-23T19:07:36.300620987Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] {'volume_id': volume_id, 'exc': exc})\n","stream":"stderr","time":"2018-04-23T19:07:36.300625318Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n","stream":"stderr","time":"2018-04-23T19:07:36.30062953Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] self.force_reraise()\n","stream":"stderr","time":"2018-04-23T19:07:36.300634016Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n","stream":"stderr","time":"2018-04-23T19:07:36.300637728Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] six.reraise(self.type_, self.value, self.tb)\n","stream":"stderr","time":"2018-04-23T19:07:36.300641624Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/virt/block_device.py\", line 353, in _call_wait_func\n","stream":"stderr","time":"2018-04-23T19:07:36.300645474Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] wait_func(context, volume_id)\n","stream":"stderr","time":"2018-04-23T19:07:36.300649994Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 1258, in _await_block_device_map_created\n","stream":"stderr","time":"2018-04-23T19:07:36.300653825Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] volume_status=volume_status)\n","stream":"stderr","time":"2018-04-23T19:07:36.300658244Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] VolumeNotCreated: Volume 1d5dc689-db6d-4a50-80e6-d891f38d5ef5 did not finish being created even after we waited 10 seconds or 10 attempts. And its status is error.\n","stream":"stderr","time":"2018-04-23T19:07:36.30066253Z"}
{"log":"2018-04-23 19:07:36.299 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] \n","stream":"stderr","time":"2018-04-23T19:07:36.300667871Z"}
{"log":"2018-04-23 19:07:36.318 10999 INFO nova.scheduler.client.report [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] Deleted allocation for instance 47d9469e-ce3e-4cf9-ba61-50111c793ff9\n","stream":"stderr","time":"2018-04-23T19:07:36.31925297Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Build of instance 47d9469e-ce3e-4cf9-ba61-50111c793ff9 aborted: Volume 1d5dc689-db6d-4a50-80e6-d891f38d5ef5 did not finish being created even after we waited 10 seconds or 10 attempts. And its status is error.\n","stream":"stderr","time":"2018-04-23T19:07:36.470524311Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Traceback (most recent call last):\n","stream":"stderr","time":"2018-04-23T19:07:36.470556219Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 1787, in _do_build_and_run_instance\n","stream":"stderr","time":"2018-04-23T19:07:36.470561276Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] filter_properties)\n","stream":"stderr","time":"2018-04-23T19:07:36.470566269Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 1947, in _build_and_run_instance\n","stream":"stderr","time":"2018-04-23T19:07:36.470570383Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] 'create.error', fault=e)\n","stream":"stderr","time":"2018-04-23T19:07:36.470574964Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 220, in __exit__\n","stream":"stderr","time":"2018-04-23T19:07:36.470578809Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] self.force_reraise()\n","stream":"stderr","time":"2018-04-23T19:07:36.470583117Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py\", line 196, in force_reraise\n","stream":"stderr","time":"2018-04-23T19:07:36.470587309Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] six.reraise(self.type_, self.value, self.tb)\n","stream":"stderr","time":"2018-04-23T19:07:36.470591758Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 1912, in _build_and_run_instance\n","stream":"stderr","time":"2018-04-23T19:07:36.470597153Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] block_device_mapping) as resources:\n","stream":"stderr","time":"2018-04-23T19:07:36.470601623Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/usr/lib/python2.7/contextlib.py\", line 17, in __enter__\n","stream":"stderr","time":"2018-04-23T19:07:36.470605871Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] return self.gen.next()\n","stream":"stderr","time":"2018-04-23T19:07:36.470610305Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] File \"/var/lib/openstack/local/lib/python2.7/site-packages/nova/compute/manager.py\", line 2075, in _build_resources\n","stream":"stderr","time":"2018-04-23T19:07:36.470614551Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] reason=e.format_message())\n","stream":"stderr","time":"2018-04-23T19:07:36.47061892Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] BuildAbortException: Build of instance 47d9469e-ce3e-4cf9-ba61-50111c793ff9 aborted: Volume 1d5dc689-db6d-4a50-80e6-d891f38d5ef5 did not finish being created even after we waited 10 seconds or 10 attempts. And its status is error.\n","stream":"stderr","time":"2018-04-23T19:07:36.470639996Z"}
{"log":"2018-04-23 19:07:36.469 10999 ERROR nova.compute.manager [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] \n","stream":"stderr","time":"2018-04-23T19:07:36.470644996Z"}
{"log":"2018-04-23 19:07:37.072 10999 INFO nova.compute.manager [req-a47100b1-5ccb-4c87-bec6-28a16d1ddf87 173ded36c69c48159cb2b3be36758384 c5b35352bb9f44b1bad807a10decf03f - - -] [instance: 47d9469e-ce3e-4cf9-ba61-50111c793ff9] Took 0.60 seconds to deallocate network for instance.\n","stream":"stderr","time":"2018-04-23T19:07:37.07289648Z"}
{"log":"2018-04-23 19:07:56.000 10999 INFO nova.compute.resource_tracker [req-01160aa3-c840-4a28-a953-f121fe81de1d - - - - -] Auditing locally available compute resources for node allceph.c.kube5s-199510.internal\n","stream":"stderr","time":"2018-04-23T19:07:56.001303447Z"}
{"log":"2018-04-23 19:07:56.148 10999 INFO nova.compute.resource_tracker [req-01160aa3-c840-4a28-a953-f121fe81de1d - - - - -] Total usable vcpus: 4, total allocated vcpus: 0\n","stream":"stderr","time":"2018-04-23T19:07:56.149000353Z"}
{"log":"2018-04-23 19:07:56.148 10999 INFO nova.compute.resource_tracker [req-01160aa3-c840-4a28-a953-f121fe81de1d - - - - -] Final resource view: name=allceph.c.kube5s-199510.internal phys_ram=15032MB used_ram=512MB phys_disk=484GB used_disk=0GB total_vcpus=4 used_vcpus=0 pci_stats=[]\n","stream":"stderr","time":"2018-04-23T19:07:56.14931533Z"}
{"log":"2018-04-23 19:07:56.206 10999 INFO nova.compute.resource_tracker [req-01160aa3-c840-4a28-a953-f121fe81de1d - - - - -] Compute_service record updated for allceph:allceph.c.kube5s-199510.internal\n","stream":"stderr","time":"2018-04-23T19:07:56.207394225Z"}

Revision history for this message
chinasubbareddy mallavarapu (chinasubbareddy) said : 		#3

In the instance launcher, when selecting Image or Image snapshot as boot source, the Create volume option is set to yes by default and size of the volume will be 1GB, that is where vm launch gets failed if image expects the disk size more than 1Gb.

this is the bug only in horizon it seems as per below :

https://bugs.launchpad.net/horizon/+bug/1678109

Revision history for this message
chinasubbareddy mallavarapu (chinasubbareddy) said : 		#4

this is the issue from openstack version , nothing to do with openstack helm so closing it ,