OpenStack Compute (nova)

Essex stable running on Xenserver: DEVICE_DETACH_REJECTED errors during stress test

Asked by Giuseppe Civitella

Hi all,

I'm seeing multiple DEVICE_DETACH_REJECTED errors in my nova-compute's logs when running a deploy's stress test.
I run a few nova-compute vms (ubuntu 10.04 on kernel 3.2.7) on some xenserver6 nodes.
The openstack's version I was running is Essex stable.
While I was able to deploy correctly a single vm on every node, if I did try to schedule about five virtual machine's deploy per node, I received errors like those reported here:

http://paste.openstack.org/show/12634/

I did rollback to essex4 running on same xenserver hosts and on same nova-compute virtual machine.
the errors vanished.

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
John Garbutt
Solved:
Last query:
Last reply:

This question was originally filed as bug #978207.

Revision history for this message
Giuseppe Civitella (gcivitella) said : 		#1

Sorry,

the problem was documented and Xen spcific.
It was something similar to the problem described here:
http://www.crucial.com.au/blog/2011/05/04/xenserver-internal-error-another-frontend-device-is-already-connect-to-this-domain/

Giuseppe

Revision history for this message
John Garbutt (johngarbutt) said : 		#2

Hi,

It would be good to get you past these problems.

What exact version of XenServer were you trying with Essex stable?
XenServer 6.0.2 (+ hotfix) has been released in the last few weeks.

Thanks,
John

Revision history for this message
Giuseppe Civitella (gcivitella) said : 		#3

Hi,

I'm running Xenserver6, I did install all the patches available.

thanks

Revision history for this message
Giuseppe Civitella (gcivitella) said : 		#4

I mean XenServer 6.0 (build 50762p) with the following updates:
XS60E001
XS60E002
XS60E003
XS60E004

Revision history for this message
John Garbutt (johngarbutt) said : 		#5

My suggestion would be to try 6.0.2

It might be worth raising the issue on the XenServer forum:
http://forums.citrix.com/category.jspa?categoryID=101

However, as you Essex 4 seems to work, it might be a race condition within OpenStack. Are you OK adding a bug for this issue?

Revision history for this message
Giuseppe Civitella (gcivitella) said : 		#6

Sorry John,

I did not explain the right way why I transformed a bug to the present question.
I found out that my problem was something happening at the Xenserver level during a stress test.
It was something already documented by the xen community.
It was not related on Openstack, neither Essex4 or Essex stable.
To get through the problem i just implemented a quick and dirty nagios plugin to check for lost vbd.

thanks a lot for your time.
Giuseppe

Revision history for this message
Best John Garbutt (johngarbutt) said : 		#7

OK, cool, I get you now.

It is certainly worth trying 6.0.2 to see if that has fixed your issue. Sorry that isn't a more useful answer!

If you still have that problem, it is well worth going through the Citrix support channels, and I may be able to help with that.

Revision history for this message
Giuseppe Civitella (gcivitella) said : 		#8

Thanks John Garbutt, that solved my question.