OpenStack instances power off by itselfopenstack: how to run multiple glance instances?Can't connect to instance via floating IPs, not natting/forwarding properlyCannot connect to Openstack Instance via horizon, but SSH works?Openstack Grizzly Fails to Provision New VMsConnectionError: [Errno 111] Connection refused (Openstack installation through Puppet)Openstack Neutron - Connection Refused when Creating Instance - I'm Out of Ideasiscsiadm: No portal foundHTTP failure within Docker container hosted in OpenStackopenstack shared PCI between numa with SR-IOVOpenStack Hyper-V vSwitch not being selected
Multi tool use
Delta TSA-Precheck status removed
How can I answer high-school writing prompts without sounding weird and fake?
How to select certain lines (n, n+4, n+8, n+12...) from the file?
Can 'sudo apt-get remove [write]' destroy my Ubuntu?
Exception propagation: When to catch exceptions?
On studying Computer Science vs. Software Engineering to become a proficient coder
What does this quote in Small Gods refer to?
Ex-manager wants to stay in touch, I don't want to
Early arrival in Australia, early hotel check in not available
51% attack - apparently very easy? refering to CZ's "rollback btc chain" - How to make sure such corruptible scenario can never happen so easily?
Why is it so slow when assigning a concatenated string to a variable in python?
What are the ramifications of setting ARITHABORT ON for all connections in SQL Server?
What does i386 mean on macOS Mojave?
How did Thanos not realise this had happened at the end of Endgame?
How to make the table in the figure in LaTeX?
Can you book a one-way ticket to the UK on a visa?
What is the best way for a skeleton to impersonate human without using magic?
Noob at soldering, can anyone explain why my circuit won't work?
Control variables and other independent variables
Was the Highlands Ranch shooting the 115th mass shooting in the US in 2019
Is there enough time to Planar Bind a creature conjured by a 1-hour-duration spell?
Set a camera to free fall like a Rigid Body?
find not returning expected files
How can I make a vertical rule that extends to the edges of an fbox?
OpenStack instances power off by itself
openstack: how to run multiple glance instances?Can't connect to instance via floating IPs, not natting/forwarding properlyCannot connect to Openstack Instance via horizon, but SSH works?Openstack Grizzly Fails to Provision New VMsConnectionError: [Errno 111] Connection refused (Openstack installation through Puppet)Openstack Neutron - Connection Refused when Creating Instance - I'm Out of Ideasiscsiadm: No portal foundHTTP failure within Docker container hosted in OpenStackopenstack shared PCI between numa with SR-IOVOpenStack Hyper-V vSwitch not being selected
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
Already asked this over on ask.openstack.org but I figured it would get more views here too.
I am running OpenStack Mitaka on a 5 node cluster, 1 controller/network node and 4 compute nodes. I am able to create VMs and use them normally throughout my day but when I come back in the morning some instances are powered off.
I've checked to make sure there is enough disk space (OK). I've checked if the problem is on any compute node (YES). I've also checked the RAM and CPU usage (OK).
I've tried to change the handle_virt_lifecycle_events variable in nova.conf to true and false with no success as some research has led me to.
It seems like the power state from the DB to the hypervisor (KVM) being mismatched is causing this.
Any ideas?
Here's the only relevant information that I could fine from /var/log/nova/nova-compute.log:
2016-06-08 21:20:18.208 1738 INFO nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.228 1738 INFO nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.232 1738 INFO nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.298 1738 WARNING nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.314 1738 WARNING nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.334 1738 WARNING nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
kvm-virtualization libvirt openstack
add a comment |
Already asked this over on ask.openstack.org but I figured it would get more views here too.
I am running OpenStack Mitaka on a 5 node cluster, 1 controller/network node and 4 compute nodes. I am able to create VMs and use them normally throughout my day but when I come back in the morning some instances are powered off.
I've checked to make sure there is enough disk space (OK). I've checked if the problem is on any compute node (YES). I've also checked the RAM and CPU usage (OK).
I've tried to change the handle_virt_lifecycle_events variable in nova.conf to true and false with no success as some research has led me to.
It seems like the power state from the DB to the hypervisor (KVM) being mismatched is causing this.
Any ideas?
Here's the only relevant information that I could fine from /var/log/nova/nova-compute.log:
2016-06-08 21:20:18.208 1738 INFO nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.228 1738 INFO nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.232 1738 INFO nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.298 1738 WARNING nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.314 1738 WARNING nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.334 1738 WARNING nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
kvm-virtualization libvirt openstack
add a comment |
Already asked this over on ask.openstack.org but I figured it would get more views here too.
I am running OpenStack Mitaka on a 5 node cluster, 1 controller/network node and 4 compute nodes. I am able to create VMs and use them normally throughout my day but when I come back in the morning some instances are powered off.
I've checked to make sure there is enough disk space (OK). I've checked if the problem is on any compute node (YES). I've also checked the RAM and CPU usage (OK).
I've tried to change the handle_virt_lifecycle_events variable in nova.conf to true and false with no success as some research has led me to.
It seems like the power state from the DB to the hypervisor (KVM) being mismatched is causing this.
Any ideas?
Here's the only relevant information that I could fine from /var/log/nova/nova-compute.log:
2016-06-08 21:20:18.208 1738 INFO nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.228 1738 INFO nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.232 1738 INFO nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.298 1738 WARNING nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.314 1738 WARNING nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.334 1738 WARNING nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
kvm-virtualization libvirt openstack
Already asked this over on ask.openstack.org but I figured it would get more views here too.
I am running OpenStack Mitaka on a 5 node cluster, 1 controller/network node and 4 compute nodes. I am able to create VMs and use them normally throughout my day but when I come back in the morning some instances are powered off.
I've checked to make sure there is enough disk space (OK). I've checked if the problem is on any compute node (YES). I've also checked the RAM and CPU usage (OK).
I've tried to change the handle_virt_lifecycle_events variable in nova.conf to true and false with no success as some research has led me to.
It seems like the power state from the DB to the hypervisor (KVM) being mismatched is causing this.
Any ideas?
Here's the only relevant information that I could fine from /var/log/nova/nova-compute.log:
2016-06-08 21:20:18.208 1738 INFO nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.228 1738 INFO nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.232 1738 INFO nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.
2016-06-08 21:20:18.298 1738 WARNING nova.compute.manager [-] [instance: 493ad84c-1e46-410c-8fab-078c77ddea10] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.314 1738 WARNING nova.compute.manager [-] [instance: 2dcb6116-55ab-4987-a4f5-ce3e0f1edda4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
2016-06-08 21:20:18.334 1738 WARNING nova.compute.manager [-] [instance: d1d0172d-20aa-4333-b9e9-aabc064287bc] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
kvm-virtualization libvirt openstack
kvm-virtualization libvirt openstack
asked Jun 9 '16 at 18:24
Matthew SanabriaMatthew Sanabria
9219
9219
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In /etc/nova/nova.conf, set the following option to disable the synchronization
sync_power_state_interval=-1
This will prevent enforcing VM state from DB.
Quoting from here:
The synchronization task will see that the DB is out of sync and will
try to 'correct' the inconsistency by powering off the VM.
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..
– adiflinux
Nov 8 '16 at 14:47
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f783053%2fopenstack-instances-power-off-by-itself%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In /etc/nova/nova.conf, set the following option to disable the synchronization
sync_power_state_interval=-1
This will prevent enforcing VM state from DB.
Quoting from here:
The synchronization task will see that the DB is out of sync and will
try to 'correct' the inconsistency by powering off the VM.
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..
– adiflinux
Nov 8 '16 at 14:47
add a comment |
In /etc/nova/nova.conf, set the following option to disable the synchronization
sync_power_state_interval=-1
This will prevent enforcing VM state from DB.
Quoting from here:
The synchronization task will see that the DB is out of sync and will
try to 'correct' the inconsistency by powering off the VM.
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..
– adiflinux
Nov 8 '16 at 14:47
add a comment |
In /etc/nova/nova.conf, set the following option to disable the synchronization
sync_power_state_interval=-1
This will prevent enforcing VM state from DB.
Quoting from here:
The synchronization task will see that the DB is out of sync and will
try to 'correct' the inconsistency by powering off the VM.
In /etc/nova/nova.conf, set the following option to disable the synchronization
sync_power_state_interval=-1
This will prevent enforcing VM state from DB.
Quoting from here:
The synchronization task will see that the DB is out of sync and will
try to 'correct' the inconsistency by powering off the VM.
edited Nov 8 '16 at 14:02
answered Nov 8 '16 at 11:31
adiflinuxadiflinux
235
235
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..
– adiflinux
Nov 8 '16 at 14:47
add a comment |
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..
– adiflinux
Nov 8 '16 at 14:47
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
This won't stop the instance from shutting itself down, it will only stop nova from reporting its status correctly.
– Michael Hampton♦
Nov 8 '16 at 13:20
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
It made the difference at my end (Mitaka). as stated here: link
– adiflinux
Nov 8 '16 at 13:58
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
If nova shut down the VM, then yes it would help, but if the instance shut down from within the instance, as it seems happened here, then it will not.
– Michael Hampton♦
Nov 8 '16 at 13:59
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:
...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..– adiflinux
Nov 8 '16 at 14:47
@Michael Hampton - I think the error message is missleading. You'll mostly find these two lines simultaneously:
...During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. ...Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4
indicating that nova periodic task were responsible for that. Worth trying..– adiflinux
Nov 8 '16 at 14:47
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f783053%2fopenstack-instances-power-off-by-itself%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
a8yYegrOUYU o0BcDbHwXIGjSbbge H08X9e PYy5U et6KJ Jl zRw2lpZFG4eH1ltRqQxrP,03p3cI,3fElGtGjKYUZEv,chSA,M4W,P