stack.sh failing with Table 'ovs_neutron.floatingips' doesn't exist

Asked by kedar kulkarni

We are running jenkins setup which gets triggers on new commits.
Suddenly from last two days we are facing this issue Table 'ovs_neutron.floatingips' doesn't exist. And all consecutive trigger builds are failing with this error.
The complete stacktrace is here
2014-03-04 12:40:45.238 | condition when an agent entry is 'upserted'.
2014-03-04 12:40:45.248 | INFO [alembic.migration] Running upgrade 1fcfc149aca4 -> 50e86cb2637a, nsx_mappings
2014-03-04 12:40:45.257 | INFO [alembic.migration] Running upgrade 50e86cb2637a -> ed93525fd003, bigswitch_quota
2014-03-04 12:40:45.260 | INFO [alembic.migration] Running upgrade ed93525fd003 -> 8f682276ee4, ryu plugin quota
2014-03-04 12:40:45.261 | INFO [alembic.migration] Running upgrade 8f682276ee4 -> 1421183d533f, NSX DHCP/metadata support
2014-03-04 12:40:45.264 | INFO [alembic.migration] Running upgrade 1421183d533f -> 3d3cb89d84ee, nsx_switch_mappings
2014-03-04 12:40:45.266 | INFO [alembic.migration] Running upgrade 3d3cb89d84ee -> 4ca36cfc898c, nsx_router_mappings
2014-03-04 12:40:45.268 | INFO [alembic.migration] Running upgrade 4ca36cfc898c -> 27cc183af192, ml2_vnic_type
2014-03-04 12:40:45.270 | INFO [alembic.migration] Running upgrade 27cc183af192 -> 50d5ba354c23, ml2 binding:vif_details
2014-03-04 12:40:45.272 | INFO [alembic.migration] Running upgrade 50d5ba354c23 -> 157a5d299379, ml2 binding:profile
2014-03-04 12:40:45.274 | INFO [alembic.migration] Running upgrade 157a5d299379 -> 3d2585038b95, VMware NSX rebranding
2014-03-04 12:40:45.276 | INFO [alembic.migration] Running upgrade 3d2585038b95 -> abc88c33f74f, lb stats
2014-03-04 12:40:45.278 | INFO [alembic.migration] Running upgrade abc88c33f74f -> 1b2580001654, nsx_sec_group_mapping
2014-03-04 12:40:45.280 | INFO [alembic.migration] Running upgrade 1b2580001654 -> e766b19a3bb, nuage_initial
2014-03-04 12:40:45.282 | INFO [alembic.migration] Running upgrade e766b19a3bb -> f44ab9871cd6, bsn_security_groups
2014-03-04 12:40:45.284 | INFO [alembic.migration] Running upgrade f44ab9871cd6 -> 2eeaf963a447, floatingip_status
2014-03-04 12:40:45.286 | Traceback (most recent call last):
2014-03-04 12:40:45.288 | File "/usr/local/bin/neutron-db-manage", line 10, in <module>
2014-03-04 12:40:45.290 | sys.exit(main())
2014-03-04 12:40:45.292 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 141, in main
2014-03-04 12:40:45.294 | CONF.command.func(config, CONF.command.name)
2014-03-04 12:40:45.296 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 78, in do_upgrade_downgrade
2014-03-04 12:40:45.298 | do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
2014-03-04 12:40:45.300 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 57, in do_alembic_command
2014-03-04 12:40:45.302 | getattr(alembic_command, cmd)(config, *args, **kwargs)
2014-03-04 12:40:45.304 | File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
2014-03-04 12:40:45.306 | script.run_env()
2014-03-04 12:40:45.308 | File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 199, in run_env
2014-03-04 12:40:45.310 | util.load_python_file(self.dir, 'env.py')
2014-03-04 12:40:45.312 | File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 199, in load_python_file
2014-03-04 12:40:45.314 | module = load_module(module_id, path)
2014-03-04 12:40:45.316 | File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 55, in load_module
2014-03-04 12:40:45.318 | mod = imp.load_source(module_id, path, fp)
2014-03-04 12:40:45.320 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 103, in <module>
2014-03-04 12:40:45.322 | run_migrations_online()
2014-03-04 12:40:45.324 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 87, in run_migrations_online
2014-03-04 12:40:45.326 | options=build_options())
2014-03-04 12:40:45.328 | File "<string>", line 7, in run_migrations
2014-03-04 12:40:45.330 | File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 652, in run_migrations
2014-03-04 12:40:45.332 | self.get_context().run_migrations(**kw)
2014-03-04 12:40:45.334 | File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 225, in run_migrations
2014-03-04 12:40:45.337 | change(**kw)
2014-03-04 12:40:45.339 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/2eeaf963a447_floatingip_status.py", line 48, in upgrade
2014-03-04 12:40:45.341 | nullable=True))
2014-03-04 12:40:45.343 | File "<string>", line 7, in add_column
2014-03-04 12:40:45.345 | File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 365, in add_column
2014-03-04 12:40:45.347 | schema=schema
2014-03-04 12:40:45.349 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 127, in add_column
2014-03-04 12:40:45.351 | self._exec(base.AddColumn(table_name, column, schema=schema))
2014-03-04 12:40:45.353 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
2014-03-04 12:40:45.355 | conn.execute(construct, *multiparams, **params)
2014-03-04 12:40:45.358 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
2014-03-04 12:40:45.360 | params)
2014-03-04 12:40:45.362 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
2014-03-04 12:40:45.364 | compiled
2014-03-04 12:40:45.366 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
2014-03-04 12:40:45.368 | context)
2014-03-04 12:40:45.370 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
2014-03-04 12:40:45.373 | context)
2014-03-04 12:40:45.376 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
2014-03-04 12:40:45.378 | cursor.execute(statement, parameters)
2014-03-04 12:40:45.380 | File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
2014-03-04 12:40:45.382 | self.errorhandler(self, exc, value)
2014-03-04 12:40:45.384 | File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
2014-03-04 12:40:45.386 | raise errorclass, errorvalue
2014-03-04 12:40:45.388 | sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'ovs_neutron.floatingips' doesn't exist") 'ALTER TABLE floatingips ADD COLUMN last_known_router_id VARCHAR(36)' ()
2014-03-04 12:40:45.390 | + exit_trap

Is there any core code change happened due to this is arising? what changes do i need to make in my plugin code/configuration to pass this, is anybody else seeing this?

Please help.

Question information

Language:
English Edit question
Status:
Open
For:
devstack Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Salvatore Orlando (salvatore-orlando) said :
#1

Hi,

I am the person you should be blaming for this.
This is happening because no L3 plugin is configured, as the migration says it applies to all plugin, but then some deployments are executed without a L3 plugin at all.

I am fixing that.
In the meanwhile you can consider adding the base l3 plugin to service_plugins and run devstack again.

service_plugins =neutron.services.l3_router.l3_router_plugin.L3RouterPlugin

Revision history for this message
One Convergence CI (oc-neutron-test) said :
#2

After adding the base l3 plugind to service_plugins the issue doesn't seems to resolve build failed with the same error.

2014-03-04 16:58:40.270 | + mysql -uroot -pstack123 -h127.0.0.1 -e 'DROP DATABASE IF EXISTS oc_nvsd_neutron;'
2014-03-04 16:58:40.287 | + mysql -uroot -pstack123 -h127.0.0.1 -e 'CREATE DATABASE oc_nvsd_neutron CHARACTER SET utf8;'
2014-03-04 16:58:40.300 | + /usr/local/bin/neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/oneconvergence/nvsdplugin.ini upgrade head
2014-03-04 16:58:40.714 | No handlers could be found for logger "neutron.common.legacy"
2014-03-04 16:58:41.004 | INFO [alembic.migration] Context impl MySQLImpl.
2014-03-04 16:58:41.008 | INFO [alembic.migration] Will assume non-transactional DDL.
2014-03-04 16:58:41.046 | INFO [alembic.migration] Running upgrade None -> folsom, folsom initial database
2014-03-04 16:58:41.168 | INFO [alembic.migration] Running upgrade folsom -> 2c4af419145b, l3_support
2014-03-04 16:58:41.177 | INFO [alembic.migration] Running upgrade 2c4af419145b -> 5a875d0e5c, ryu
2014-03-04 16:58:41.181 | INFO [alembic.migration] Running upgrade 5a875d0e5c -> 48b6f43f7471, DB support for service types
2014-03-04 16:58:41.188 | INFO [alembic.migration] Running upgrade 48b6f43f7471 -> 3cb5d900c5de, security_groups
2014-03-04 16:58:41.195 | INFO [alembic.migration] Running upgrade 3cb5d900c5de -> 1d76643bcec4, nvp_netbinding
2014-03-04 16:58:41.201 | INFO [alembic.migration] Running upgrade 1d76643bcec4 -> 2a6d0b51f4bb, cisco plugin cleanup
2014-03-04 16:58:41.206 | INFO [alembic.migration] Running upgrade 2a6d0b51f4bb -> 1b693c095aa3, Quota ext support added in Grizzly
2014-03-04 16:58:41.215 | INFO [alembic.migration] Running upgrade 1b693c095aa3 -> 1149d7de0cfa, initial port security
2014-03-04 16:58:41.220 | INFO [alembic.migration] Running upgrade 1149d7de0cfa -> 49332180ca96, ryu plugin update
2014-03-04 16:58:41.230 | INFO [alembic.migration] Running upgrade 49332180ca96 -> 38335592a0dc, nvp_portmap
2014-03-04 16:58:41.236 | INFO [alembic.migration] Running upgrade 38335592a0dc -> 54c2c487e913, 'DB support for load balancing service
2014-03-04 16:58:41.242 | INFO [alembic.migration] Running upgrade 54c2c487e913 -> 45680af419f9, nvp_qos
2014-03-04 16:58:41.250 | INFO [alembic.migration] Running upgrade 45680af419f9 -> 1c33fa3cd1a1, Support routing table configuration on Router
2014-03-04 16:58:41.257 | INFO [alembic.migration] Running upgrade 1c33fa3cd1a1 -> 363468ac592c, nvp_network_gw
2014-03-04 16:58:41.263 | INFO [alembic.migration] Running upgrade 363468ac592c -> 511471cc46b, Add agent management extension model support
2014-03-04 16:58:41.270 | INFO [alembic.migration] Running upgrade 511471cc46b -> 3b54bf9e29f7, NEC plugin sharednet
2014-03-04 16:58:41.276 | INFO [alembic.migration] Running upgrade 3b54bf9e29f7 -> 4692d074d587, agent scheduler
2014-03-04 16:58:41.278 | INFO [alembic.migration] Running upgrade 4692d074d587 -> 1341ed32cc1e, nvp_net_binding
2014-03-04 16:58:41.282 | INFO [alembic.migration] Running upgrade 1341ed32cc1e -> grizzly, grizzly
2014-03-04 16:58:41.291 | INFO [alembic.migration] Running upgrade grizzly -> f489cf14a79c, DB support for load balancing service (havana)
2014-03-04 16:58:41.300 | INFO [alembic.migration] Running upgrade f489cf14a79c -> 176a85fc7d79, Add portbindings db
2014-03-04 16:58:41.305 | INFO [alembic.migration] Running upgrade 176a85fc7d79 -> 32b517556ec9, remove TunnelIP model
2014-03-04 16:58:41.312 | INFO [alembic.migration] Running upgrade 32b517556ec9 -> 128e042a2b68, ext_gw_mode
2014-03-04 16:58:41.321 | INFO [alembic.migration] Running upgrade 128e042a2b68 -> 5ac71e65402c, ml2_initial
2014-03-04 16:58:41.323 | INFO [alembic.migration] Running upgrade 5ac71e65402c -> 3cbf70257c28, nvp_mac_learning
2014-03-04 16:58:41.325 | INFO [alembic.migration] Running upgrade 3cbf70257c28 -> 5918cbddab04, add tables for router rules support
2014-03-04 16:58:41.330 | INFO [alembic.migration] Running upgrade 5918cbddab04 -> 3cabb850f4a5, Table to track port to host associations
2014-03-04 16:58:41.333 | INFO [alembic.migration] Running upgrade 3cabb850f4a5 -> b7a8863760e, Remove cisco_vlan_bindings table
2014-03-04 16:58:41.341 | INFO [alembic.migration] Running upgrade b7a8863760e -> 13de305df56e, nec_add_pf_name
2014-03-04 16:58:41.344 | INFO [alembic.migration] Running upgrade 13de305df56e -> 20ae61555e95, DB Migration for ML2 GRE Type Driver
2014-03-04 16:58:41.351 | INFO [alembic.migration] Running upgrade 20ae61555e95 -> 477a4488d3f4, DB Migration for ML2 VXLAN Type Driver
2014-03-04 16:58:41.356 | INFO [alembic.migration] Running upgrade 477a4488d3f4 -> 2032abe8edac, LBaaS add status description
2014-03-04 16:58:41.363 | INFO [alembic.migration] Running upgrade 2032abe8edac -> 52c5e4a18807, LBaaS Pool scheduler
2014-03-04 16:58:41.368 | INFO [alembic.migration] Running upgrade 52c5e4a18807 -> 557edfc53098, New service types framework (service providers)
2014-03-04 16:58:41.373 | INFO [alembic.migration] Running upgrade 557edfc53098 -> e6b16a30d97, Add cisco_provider_networks table
2014-03-04 16:58:41.378 | INFO [alembic.migration] Running upgrade e6b16a30d97 -> 39cf3f799352, FWaaS Havana-2 model
2014-03-04 16:58:41.380 | INFO [alembic.migration] Running upgrade 39cf3f799352 -> 52ff27f7567a, Support for VPNaaS
2014-03-04 16:58:41.383 | INFO [alembic.migration] Running upgrade 52ff27f7567a -> 11c6e18605c8, Pool Monitor status field
2014-03-04 16:58:41.393 | INFO [alembic.migration] Running upgrade 11c6e18605c8 -> 35c7c198ddea, remove status from HealthMonitor
2014-03-04 16:58:41.413 | INFO [alembic.migration] Running upgrade 35c7c198ddea -> 263772d65691, Cisco plugin db cleanup part II
2014-03-04 16:58:41.419 | INFO [alembic.migration] Running upgrade 263772d65691 -> c88b6b5fea3, Cisco N1KV tables
2014-03-04 16:58:41.424 | INFO [alembic.migration] Running upgrade c88b6b5fea3 -> f9263d6df56, remove_dhcp_lease
2014-03-04 16:58:41.440 | INFO [alembic.migration] Running upgrade f9263d6df56 -> 569e98a8132b, metering
2014-03-04 16:58:41.455 | INFO [alembic.migration] Running upgrade 569e98a8132b -> 86cf4d88bd3, remove bigswitch port tracking table
2014-03-04 16:58:41.463 | INFO [alembic.migration] Running upgrade 86cf4d88bd3 -> 3c6e57a23db4, add multiprovider
2014-03-04 16:58:41.470 | INFO [alembic.migration] Running upgrade 3c6e57a23db4 -> 63afba73813, Add unique constraint for id column of TunnelEndpoint
2014-03-04 16:58:41.479 | INFO [alembic.migration] Running upgrade 63afba73813 -> 40dffbf4b549, nvp_dist_router
2014-03-04 16:58:41.486 | INFO [alembic.migration] Running upgrade 40dffbf4b549 -> 53bbd27ec841, Extra dhcp opts support
2014-03-04 16:58:41.489 | INFO [alembic.migration] Running upgrade 53bbd27ec841 -> 46a0efbd8f0, cisco_n1kv_multisegment_trunk
2014-03-04 16:58:41.499 | INFO [alembic.migration] Running upgrade 46a0efbd8f0 -> 2a3bae1ceb8, NEC Port Binding
2014-03-04 16:58:41.505 | INFO [alembic.migration] Running upgrade 2a3bae1ceb8 -> 14f24494ca31, DB Migration for Arista ml2 mechanism driver
2014-03-04 16:58:41.521 | INFO [alembic.migration] Running upgrade 14f24494ca31 -> 32a65f71af51, ml2 portbinding
2014-03-04 16:58:41.526 | INFO [alembic.migration] Running upgrade 32a65f71af51 -> 66a59a7f516, NEC OpenFlow Router
2014-03-04 16:58:41.529 | INFO [alembic.migration] Running upgrade 66a59a7f516 -> 51b4de912379, Cisco Nexus ML2 mechanism driver
2014-03-04 16:58:41.532 | INFO [alembic.migration] Running upgrade 51b4de912379 -> 1efb85914233, allowedaddresspairs
2014-03-04 16:58:41.535 | INFO [alembic.migration] Running upgrade 1efb85914233 -> 38fc1f6789f8, Cisco N1KV overlay support
2014-03-04 16:58:41.539 | INFO [alembic.migration] Running upgrade 38fc1f6789f8 -> 4a666eb208c2, service router
2014-03-04 16:58:41.546 | INFO [alembic.migration] Running upgrade 4a666eb208c2 -> 338d7508968c, vpnaas peer_address size increase
2014-03-04 16:58:41.552 | INFO [alembic.migration] Running upgrade 338d7508968c -> 3ed8f075e38a, nvp fwaas plugin
2014-03-04 16:58:41.560 | INFO [alembic.migration] Running upgrade 3ed8f075e38a -> 3d6fae8b70b0, nvp lbaas plugin
2014-03-04 16:58:41.566 | INFO [alembic.migration] Running upgrade 3d6fae8b70b0 -> 1064e98b7917, nec-pf-port-del
2014-03-04 16:58:41.570 | INFO [alembic.migration] Running upgrade 1064e98b7917 -> 2528ceb28230, NEC PacketFilter network_id nullable fix
2014-03-04 16:58:41.576 | INFO [alembic.migration] Running upgrade 2528ceb28230 -> 3a520dd165d0, Cisco Nexus multi-switch
2014-03-04 16:58:41.581 | INFO [alembic.migration] Running upgrade 3a520dd165d0 -> 27ef74513d33, quota_in_plumgrid_plugin
2014-03-04 16:58:41.586 | INFO [alembic.migration] Running upgrade 27ef74513d33 -> 49f5e553f61f, security_groups
2014-03-04 16:58:41.594 | INFO [alembic.migration] Running upgrade 49f5e553f61f -> havana, havana
2014-03-04 16:58:41.611 | INFO [alembic.migration] Running upgrade havana -> e197124d4b9, add unique constraint to members
2014-03-04 16:58:41.628 | INFO [alembic.migration] Running upgrade e197124d4b9 -> 1fcfc149aca4, Add a unique constraint on (agent_type, host) columns to prevent a race
2014-03-04 16:58:41.637 | condition when an agent entry is 'upserted'.
2014-03-04 16:58:41.649 | INFO [alembic.migration] Running upgrade 1fcfc149aca4 -> 50e86cb2637a, nsx_mappings
2014-03-04 16:58:41.661 | INFO [alembic.migration] Running upgrade 50e86cb2637a -> ed93525fd003, bigswitch_quota
2014-03-04 16:58:41.674 | INFO [alembic.migration] Running upgrade ed93525fd003 -> 8f682276ee4, ryu plugin quota
2014-03-04 16:58:41.688 | INFO [alembic.migration] Running upgrade 8f682276ee4 -> 1421183d533f, NSX DHCP/metadata support
2014-03-04 16:58:41.699 | INFO [alembic.migration] Running upgrade 1421183d533f -> 3d3cb89d84ee, nsx_switch_mappings
2014-03-04 16:58:41.705 | INFO [alembic.migration] Running upgrade 3d3cb89d84ee -> 4ca36cfc898c, nsx_router_mappings
2014-03-04 16:58:41.712 | INFO [alembic.migration] Running upgrade 4ca36cfc898c -> 27cc183af192, ml2_vnic_type
2014-03-04 16:58:41.714 | INFO [alembic.migration] Running upgrade 27cc183af192 -> 50d5ba354c23, ml2 binding:vif_details
2014-03-04 16:58:41.716 | INFO [alembic.migration] Running upgrade 50d5ba354c23 -> 157a5d299379, ml2 binding:profile
2014-03-04 16:58:41.718 | INFO [alembic.migration] Running upgrade 157a5d299379 -> 3d2585038b95, VMware NSX rebranding
2014-03-04 16:58:41.721 | INFO [alembic.migration] Running upgrade 3d2585038b95 -> abc88c33f74f, lb stats
2014-03-04 16:58:41.723 | INFO [alembic.migration] Running upgrade abc88c33f74f -> 1b2580001654, nsx_sec_group_mapping
2014-03-04 16:58:41.725 | INFO [alembic.migration] Running upgrade 1b2580001654 -> e766b19a3bb, nuage_initial
2014-03-04 16:58:41.727 | INFO [alembic.migration] Running upgrade e766b19a3bb -> f44ab9871cd6, bsn_security_groups
2014-03-04 16:58:41.729 | INFO [alembic.migration] Running upgrade f44ab9871cd6 -> 2eeaf963a447, floatingip_status
2014-03-04 16:58:41.731 | Traceback (most recent call last):
2014-03-04 16:58:41.733 | File "/usr/local/bin/neutron-db-manage", line 10, in <module>
2014-03-04 16:58:41.736 | sys.exit(main())
2014-03-04 16:58:41.738 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 141, in main
2014-03-04 16:58:41.740 | CONF.command.func(config, CONF.command.name)
2014-03-04 16:58:41.742 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 78, in do_upgrade_downgrade
2014-03-04 16:58:41.744 | do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
2014-03-04 16:58:41.747 | File "/opt/stack/neutron/neutron/db/migration/cli.py", line 57, in do_alembic_command
2014-03-04 16:58:41.749 | getattr(alembic_command, cmd)(config, *args, **kwargs)
2014-03-04 16:58:41.751 | File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
2014-03-04 16:58:41.753 | script.run_env()
2014-03-04 16:58:41.755 | File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 199, in run_env
2014-03-04 16:58:41.757 | util.load_python_file(self.dir, 'env.py')
2014-03-04 16:58:41.759 | File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 199, in load_python_file
2014-03-04 16:58:41.761 | module = load_module(module_id, path)
2014-03-04 16:58:41.763 | File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 55, in load_module
2014-03-04 16:58:41.766 | mod = imp.load_source(module_id, path, fp)
2014-03-04 16:58:41.768 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 103, in <module>
2014-03-04 16:58:41.770 | run_migrations_online()
2014-03-04 16:58:41.772 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 87, in run_migrations_online
2014-03-04 16:58:41.774 | options=build_options())
2014-03-04 16:58:41.776 | File "<string>", line 7, in run_migrations
2014-03-04 16:58:41.778 | File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 652, in run_migrations
2014-03-04 16:58:41.780 | self.get_context().run_migrations(**kw)
2014-03-04 16:58:41.783 | File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 225, in run_migrations
2014-03-04 16:58:41.785 | change(**kw)
2014-03-04 16:58:41.787 | File "/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/2eeaf963a447_floatingip_status.py", line 48, in upgrade
2014-03-04 16:58:41.789 | nullable=True))
2014-03-04 16:58:41.791 | File "<string>", line 7, in add_column
2014-03-04 16:58:41.794 | File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 365, in add_column
2014-03-04 16:58:41.796 | schema=schema
2014-03-04 16:58:41.798 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 127, in add_column
2014-03-04 16:58:41.800 | self._exec(base.AddColumn(table_name, column, schema=schema))
2014-03-04 16:58:41.803 | File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
2014-03-04 16:58:41.805 | conn.execute(construct, *multiparams, **params)
2014-03-04 16:58:41.807 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
2014-03-04 16:58:41.809 | params)
2014-03-04 16:58:41.811 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
2014-03-04 16:58:41.813 | compiled
2014-03-04 16:58:41.815 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
2014-03-04 16:58:41.817 | context)
2014-03-04 16:58:41.819 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
2014-03-04 16:58:41.822 | context)
2014-03-04 16:58:41.824 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
2014-03-04 16:58:41.826 | cursor.execute(statement, parameters)
2014-03-04 16:58:41.828 | File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
2014-03-04 16:58:41.830 | self.errorhandler(self, exc, value)
2014-03-04 16:58:41.833 | File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
2014-03-04 16:58:41.835 | raise errorclass, errorvalue
2014-03-04 16:58:41.838 | sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'oc_nvsd_neutron.floatingips' doesn't exist") 'ALTER TABLE floatingips ADD COLUMN last_known_router_id VARCHAR(36)' ()

Revision history for this message
Salvatore Orlando (salvatore-orlando) said :
#3

I will post a patch soon for this problem.
If that still will cause failures in your CI system, i will just revert the original patch which introduced this migration.

Apologies again for the inconvenience.

Revision history for this message
kedar kulkarni (kedar-kulkarni) said :
#4

Thanks For the response Salvatore, but the thing is we have already enabled l3 service apis in our plugin. Then it should not fail isn't it?

Revision history for this message
Salvatore Orlando (salvatore-orlando) said :
#5

Can you post the neutron.conf file generated by devstck?

The reason why I've been pondering about this fix is that I found out something not quite right on l3 migrations.
Therefore what might be happening is that in your setup l3 tables are created when neutron starts using schema autogeneration.

Revision history for this message
kedar kulkarni (kedar-kulkarni) said :
#6

Ok, So here is my neutron.conf file

[DEFAULT]
auth_strategy = keystone
allow_overlapping_ips = True
policy_file = /etc/neutron/policy.json
debug = True
verbose = True
service_plugins = neutron.services.loadbalancer.plugin.LoadBalancerPlugin,neutron.services.metering.metering_plugin.MeteringPlugin
core_plugin = neutron.plugins.oneconvergence.plugin.OneConvergencePluginV2
rabbit_password = stack123
rabbit_hosts = localhost
rpc_backend = neutron.openstack.common.rpc.impl_kombu
logging_exception_prefix = %(color)s%(asctime)s.%(msecs)03d TRACE %(name)s %(instance)s
logging_debug_format_suffix = from (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d
logging_default_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s
logging_context_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_id)s%(color)s] %(instance)s%(color)s%(message)s
state_path = /opt/stack/data/neutron
# Print more verbose output (set logging level to INFO instead of default WARNING level).
# verbose = False

# Print debugging output (set logging level to DEBUG instead of default WARNING level).
# debug = False

# Where to store Neutron state files. This directory must be writable by the
# user executing the agent.
# state_path = /var/lib/neutron

# Where to store lock files
lock_path = $state_path/lock

# log_format = %(asctime)s %(levelname)8s [%(name)s] %(message)s
# log_date_format = %Y-%m-%d %H:%M:%S

# use_syslog -> syslog
# log_file and log_dir -> log_dir/log_file
# (not log_file) and log_dir -> log_dir/{binary_name}.log
# use_stderr -> stderr
# (not user_stderr) and (not log_file) -> stdout
# publish_errors -> notification system

# use_syslog = False
# syslog_log_facility = LOG_USER

# use_stderr = True
# log_file =
# log_dir =

# publish_errors = False

# Address to bind the API server to
# bind_host = 0.0.0.0

# Port the bind the API server to
# bind_port = 9696

# Path to the extensions. Note that this can be a colon-separated list of
# paths. For example:
# api_extensions_path = extensions:/path/to/more/extensions:/even/more/extensions
# The __path__ of neutron.extensions is appended to this, so if your
# extensions are in there you don't need to specify them here
# api_extensions_path =

# (StrOpt) Neutron core plugin entrypoint to be loaded from the
# neutron.core_plugins namespace. See setup.cfg for the entrypoint names of the
# plugins included in the neutron source distribution. For compatibility with
# previous versions, the class name of a plugin can be specified instead of its
# entrypoint name.
#
# core_plugin =
# Example: core_plugin = ml2

# (ListOpt) List of service plugin entrypoints to be loaded from the
# neutron.service_plugins namespace. See setup.cfg for the entrypoint names of
# the plugins included in the neutron source distribution. For compatibility
# with previous versions, the class name of a plugin can be specified instead
# of its entrypoint name.
#
# service_plugins =
# Example: service_plugins = router,firewall,lbaas,vpnaas,metering

# Paste configuration file
# api_paste_config = api-paste.ini

# The strategy to be used for auth.
# Supported values are 'keystone'(default), 'noauth'.
# auth_strategy = keystone

# Base MAC address. The first 3 octets will remain unchanged. If the
# 4h octet is not 00, it will also be used. The others will be
# randomly generated.
# 3 octet
# base_mac = fa:16:3e:00:00:00
# 4 octet
# base_mac = fa:16:3e:4f:00:00

# Maximum amount of retries to generate a unique MAC address
# mac_generation_retries = 16

# DHCP Lease duration (in seconds)
# dhcp_lease_duration = 86400

# Allow sending resource operation notification to DHCP agent
# dhcp_agent_notification = True

# Enable or disable bulk create/update/delete operations
# allow_bulk = True
# Enable or disable pagination
# allow_pagination = False
# Enable or disable sorting
# allow_sorting = False
# Enable or disable overlapping IPs for subnets
# Attention: the following parameter MUST be set to False if Neutron is
# being used in conjunction with nova security groups
# allow_overlapping_ips = False
# Ensure that configured gateway is on subnet
# force_gateway_on_subnet = False

# RPC configuration options. Defined in rpc __init__
# The messaging module to use, defaults to kombu.
# rpc_backend = neutron.openstack.common.rpc.impl_kombu
# Size of RPC thread pool
# rpc_thread_pool_size = 64
# Size of RPC connection pool
# rpc_conn_pool_size = 30
# Seconds to wait for a response from call or multicall
# rpc_response_timeout = 60
# Seconds to wait before a cast expires (TTL). Only supported by impl_zmq.
# rpc_cast_timeout = 30
# Modules of exceptions that are permitted to be recreated
# upon receiving exception data from an rpc call.
# allowed_rpc_exception_modules = neutron.openstack.common.exception, nova.exception
# AMQP exchange to connect to if using RabbitMQ or QPID
# control_exchange = neutron

# If passed, use a fake RabbitMQ provider
# fake_rabbit = False

# Configuration options if sending notifications via kombu rpc (these are
# the defaults)
# SSL version to use (valid only if SSL enabled)
# kombu_ssl_version =
# SSL key file (valid only if SSL enabled)
# kombu_ssl_keyfile =
# SSL cert file (valid only if SSL enabled)
# kombu_ssl_certfile =
# SSL certification authority file (valid only if SSL enabled)
# kombu_ssl_ca_certs =
# IP address of the RabbitMQ installation
# rabbit_host = localhost
# Password of the RabbitMQ server
# rabbit_password = guest
# Port where RabbitMQ server is running/listening
# rabbit_port = 5672
# RabbitMQ single or HA cluster (host:port pairs i.e: host1:5672, host2:5672)
# rabbit_hosts is defaulted to '$rabbit_host:$rabbit_port'
# rabbit_hosts = localhost:5672
# User ID used for RabbitMQ connections
# rabbit_userid = guest
# Location of a virtual RabbitMQ installation.
# rabbit_virtual_host = /
# Maximum retries with trying to connect to RabbitMQ
# (the default of 0 implies an infinite retry count)
# rabbit_max_retries = 0
# RabbitMQ connection retry interval
# rabbit_retry_interval = 1
# Use HA queues in RabbitMQ (x-ha-policy: all). You need to
# wipe RabbitMQ database when changing this option. (boolean value)
# rabbit_ha_queues = false

# QPID
# rpc_backend=neutron.openstack.common.rpc.impl_qpid
# Qpid broker hostname
# qpid_hostname = localhost
# Qpid broker port
# qpid_port = 5672
# Qpid single or HA cluster (host:port pairs i.e: host1:5672, host2:5672)
# qpid_hosts is defaulted to '$qpid_hostname:$qpid_port'
# qpid_hosts = localhost:5672
# Username for qpid connection
# qpid_username = ''
# Password for qpid connection
# qpid_password = ''
# Space separated list of SASL mechanisms to use for auth
# qpid_sasl_mechanisms = ''
# Seconds between connection keepalive heartbeats
# qpid_heartbeat = 60
# Transport to use, either 'tcp' or 'ssl'
# qpid_protocol = tcp
# Disable Nagle algorithm
# qpid_tcp_nodelay = True

# ZMQ
# rpc_backend=neutron.openstack.common.rpc.impl_zmq
# ZeroMQ bind address. Should be a wildcard (*), an ethernet interface, or IP.
# The "host" option should point or resolve to this address.
# rpc_zmq_bind_address = *

# ============ Notification System Options =====================

# Notifications can be sent when network/subnet/port are created, updated or deleted.
# There are three methods of sending notifications: logging (via the
# log_file directive), rpc (via a message queue) and
# noop (no notifications sent, the default)

# Notification_driver can be defined multiple times
# Do nothing driver
# notification_driver = neutron.openstack.common.notifier.no_op_notifier
# Logging driver
# notification_driver = neutron.openstack.common.notifier.log_notifier
# RPC driver.
notification_driver = neutron.openstack.common.notifier.rpc_notifier

# default_notification_level is used to form actual topic name(s) or to set logging level
# default_notification_level = INFO

# default_publisher_id is a part of the notification payload
# host = myhost.com
# default_publisher_id = $host

# Defined in rpc_notifier, can be comma separated values.
# The actual topic names will be %s.%(default_notification_level)s
# notification_topics = notifications

# Default maximum number of items returned in a single response,
# value == infinite and value < 0 means no max limit, and value must
# be greater than 0. If the number of items requested is greater than
# pagination_max_limit, server will just return pagination_max_limit
# of number of items.
# pagination_max_limit = -1

# Maximum number of DNS nameservers per subnet
# max_dns_nameservers = 5

# Maximum number of host routes per subnet
# max_subnet_host_routes = 20

# Maximum number of fixed ips per port
# max_fixed_ips_per_port = 5

# =========== items for agent management extension =============
# Seconds to regard the agent as down; should be at least twice
# report_interval, to be sure the agent is down for good
# agent_down_time = 9
# =========== end of items for agent management extension =====

# =========== items for agent scheduler extension =============
# Driver to use for scheduling network to DHCP agent
# network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.ChanceScheduler
# Driver to use for scheduling router to a default L3 agent
# neutron.scheduler.l3_agent_scheduler.ChanceScheduler, to use L3 agent with HA
# mode the following scheduler has to be used : neutron.scheduler.l3_agent_scheduler.HARouterScheduler
# router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.ChanceScheduler
# Driver to use for scheduling a loadbalancer pool to an lbaas agent
# loadbalancer_pool_scheduler_driver = neutron.services.loadbalancer.agent_scheduler.ChanceScheduler

# Allow auto scheduling networks to DHCP agent. It will schedule non-hosted
# networks to first DHCP agent which sends get_active_networks message to
# neutron server
# network_auto_schedule = True

# Allow auto scheduling routers to L3 agent. It will schedule non-hosted
# routers to first L3 agent which sends sync_routers message to neutron server
# router_auto_schedule = True

# Number of DHCP agents scheduled to host a network. This enables redundant
# DHCP agents for configured networks.
# dhcp_agents_per_network = 1

# =========== end of items for agent scheduler extension =====

# =========== items for l3 extension ==============
# Enable the High Availibility for virtual routers. A HA router scheduler has
# to be used.
# l3_ha = False
#
# Maximum number of l3 agents which an HA router will be scheduled on. If it
# is not set the router will be scheduled on every agent.
# max_l3_agents_per_router = 2
#
# Minimum number of l3 agents which an HA router will be scheduled on. If it
# is not set the minimum will be 2.
# min_l3_agents_per_router = 2
#
# CIDR of the administrative network used by the l3 if the mode HA is enabled
# l3_ha_net_cidr = 172.31.255.0/24
# =========== end of items for l3 extension =======

# =========== WSGI parameters related to the API server ==============
# Number of separate worker processes to spawn. The default, 0, runs the
# worker thread in the current process. Greater than 0 launches that number of
# child processes as workers. The parent process manages them.
# api_workers = 0

# Sets the value of TCP_KEEPIDLE in seconds to use for each server socket when
# starting API server. Not supported on OS X.
# tcp_keepidle = 600

# Number of seconds to keep retrying to listen
# retry_until_window = 30

# Number of backlog requests to configure the socket with.
# backlog = 4096

# Max header line to accommodate large tokens
# max_header_line = 16384

# Enable SSL on the API server
# use_ssl = False

# Certificate file to use when starting API server securely
# ssl_cert_file = /path/to/certfile

# Private key file to use when starting API server securely
# ssl_key_file = /path/to/keyfile

# CA certificate file to use when starting API server securely to
# verify connecting clients. This is an optional parameter only required if
# API clients need to authenticate to the API server using SSL certificates
# signed by a trusted CA
# ssl_ca_file = /path/to/cafile
# ======== end of WSGI parameters related to the API server ==========

[quotas]
# Default driver to use for quota checks
# quota_driver = neutron.db.quota_db.DbQuotaDriver

# Resource name(s) that are supported in quota features
# quota_items = network,subnet,port

# Default number of resource allowed per tenant. A negative value means
# unlimited.
# default_quota = -1

# Number of networks allowed per tenant. A negative value means unlimited.
# quota_network = 10

# Number of subnets allowed per tenant. A negative value means unlimited.
# quota_subnet = 10

# Number of ports allowed per tenant. A negative value means unlimited.
# quota_port = 50

# Number of security groups allowed per tenant. A negative value means
# unlimited.
# quota_security_group = 10

# Number of security group rules allowed per tenant. A negative value means
# unlimited.
# quota_security_group_rule = 100

# Number of vips allowed per tenant. A negative value means unlimited.
# quota_vip = 10

# Number of pools allowed per tenant. A negative value means unlimited.
# quota_pool = 10

# Number of pool members allowed per tenant. A negative value means unlimited.
# The default is unlimited because a member is not a real resource consumer
# on Openstack. However, on back-end, a member is a resource consumer
# and that is the reason why quota is possible.
# quota_member = -1

# Number of health monitors allowed per tenant. A negative value means
# unlimited.
# The default is unlimited because a health monitor is not a real resource
# consumer on Openstack. However, on back-end, a member is a resource consumer
# and that is the reason why quota is possible.
# quota_health_monitors = -1

# Number of routers allowed per tenant. A negative value means unlimited.
# quota_router = 10

# Number of floating IPs allowed per tenant. A negative value means unlimited.
# quota_floatingip = 50

[agent]
root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
# Use "sudo neutron-rootwrap /etc/neutron/rootwrap.conf" to use the real
# root filter facility.
# Change to "sudo" to skip the filtering and just run the comand directly
# root_helper = sudo

# =========== items for agent management extension =============
# seconds between nodes reporting state to server; should be less than
# agent_down_time, best if it is half or less than agent_down_time
# report_interval = 4

# =========== end of items for agent management extension =====

[keystone_authtoken]
auth_host = 192.168.6.100
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = stack123
signing_dir = /var/cache/neutron

[database]
# This line MUST be changed to actually run the plugin.
# Example:
# connection = mysql://root:pass@127.0.0.1:3306/neutron
# Replace 127.0.0.1 above with the IP address of the database used by the
# main neutron server. (Leave it as is if the database runs on this host.)
# connection = sqlite://

# The SQLAlchemy connection string used to connect to the slave database
# slave_connection =

# Database reconnection retry times - in event connectivity is lost
# set to -1 implies an infinite retry count
# max_retries = 10

# Database reconnection interval in seconds - if the initial connection to the
# database fails
# retry_interval = 10

# Minimum number of SQL connections to keep open in a pool
# min_pool_size = 1

# Maximum number of SQL connections to keep open in a pool
# max_pool_size = 10

# Timeout in seconds before idle sql connections are reaped
# idle_timeout = 3600

# If set, use this value for max_overflow with sqlalchemy
# max_overflow = 20

# Verbosity of SQL debugging information. 0=None, 100=Everything
# connection_debug = 0

# Add python stack traces to SQL as comment strings
# connection_trace = False

# If set, use this value for pool_timeout with sqlalchemy
# pool_timeout = 10

[service_providers]
# Specify service providers (drivers) for advanced services like loadbalancer, VPN, Firewall.
# Must be in form:
# service_provider=<service_type>:<name>:<driver>[:default]
# List of allowed service types includes LOADBALANCER, FIREWALL, VPN
# Combination of <service type> and <name> must be unique; <driver> must also be unique
# This is multiline option, example for default provider:
# service_provider=LOADBALANCER:name:lbaas_plugin_driver_path:default
# example of non-default provider:
# service_provider=FIREWALL:name2:firewall_driver_path
# --- Reference implementations ---
service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default
# In order to activate Radware's lbaas driver you need to uncomment the next line.
# If you want to keep the HA Proxy as the default lbaas driver, remove the attribute default from the line below.
# Otherwise comment the HA Proxy line
#service_provider = LOADBALANCER:Radware:neutron.services.loadbalancer.drivers.radware.driver.LoadBalancerDriver:default
#uncomment the following line to make the 'netscaler' LBaaS provider available.
#service_provider=LOADBALANCER:NetScaler:neutron.services.loadbalancer.drivers.netscaler.netscaler_driver.NetScalerPluginDriver

Can you help with this problem?

Provide an answer of your own, or ask kedar kulkarni for more information if necessary.

To post a message you must log in.