Eucalyptus

SYSTEM mode, instances do not run

Bug #430957 reported by Neil Soman
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Eucalyptus
Fix Released
Critical
Unassigned
1.6
Fix Released
Critical
Unassigned
eucalyptus (Ubuntu)
Fix Released
Medium
Dustin Kirkland 

Bug Description

Multicluster (CLC/CC/SC Walrus/NC).

Resources are allocated, run instances not sent to the CC.

12:16:08 WARN ClusterNodeState | BEFORE ALLOCATE ============================
12:16:08 WARN ClusterNodeState | [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 WARN ClusterNodeState | DURING ALLOCATE ============================
12:16:08 WARN ClusterNodeState | TAILSET: [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 WARN ClusterNodeState | HEADSET: []
12:16:08 WARN ClusterNodeState | AFTER ALLOCATE ============================
12:16:08 WARN ClusterNodeState | [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=1}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=0}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=0}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 INFO SLAs | [ preallocate] NetworkToken{networkName='default', cluster='mycluster', vlan=10, userName='admin', name='admin-default'}
12:16:08 INFO SLAs | [ accepted] NetworkToken{networkName='default', cluster='mycluster', vlan=10, userName='admin', name='admin-default'}
12:16:08 INFO VmAdmissionControl | Taking address index: 2

Neil Soman (neilsoman)
Changed in eucalyptus:
importance: Undecided → Critical
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

I haven't confirmed, but I'm matching upstream's priority of Critical. This seems like core, broken functionality that simply must work for Eucalyptus to be useful.

:-Dustin

Changed in eucalyptus (Ubuntu):
importance: Undecided → Critical
Revision history for this message
Neil Soman (neilsoman) wrote :

This was a problem with SYSTEM mode (UEC default is supposed to be MANAGED_NOVLAN if I'm not mistaken so this shouldn't affect that). I believe this was fixed but I need to confirm that against the upstream (I will update this bug when I do).

Matt Zimmerman (mdz)
Changed in eucalyptus (Ubuntu):
status: New → Triaged
Dustin Kirkland  (kirkland)
Changed in eucalyptus (Ubuntu):
importance: Critical → High
Revision history for this message
Neil Soman (neilsoman) wrote :
Download full text (5.9 KiB)

MANAGED and MANAGED_NOVLAN work fine with revno 894.

SYSTEM mode shows the following behavior now.

The NC keeps getting run instance over and over.

cc.log

[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] RunInstances(): called
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] refresh_resources(): called
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] node=eucabuild29 mem=3728/3728 disk=117249/117249 cores=2/2
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] refresh_resources(): done
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] calling http://eucabuild29:8775/axis2/services/EucalyptusNC
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] running instance i-35E9072D with emiId emi-E750108E...
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] assigning MAC/IP: d0:0d:35:E9:07:2D/0.0.0.0/0.0.0.0/-1
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] time left for next op: 60
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] node=eucabuild29 mem=3728/3728 disk=117249/117249 cores=2/2
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] refresh_resources(): done
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] running instance i-35E9072D with emiId emi-E750108E...
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] scheduler using ROUNDROBIN policy to find next resource
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] scheduler state starting at resource 0
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] scheduler state finishing at resource 0
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] scheduler decided to run instance 'i-35E9072D' on resource 'http://eucabuild29:8775/axis2/services/EucalyptusNC'
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] assigning MAC/IP: d0:0d:35:E9:07:2D/0.0.0.0/0.0.0.0/-1
[Fri Sep 25 20:18:30 2009][019952][EUCAINFO ] client (http://eucabuild29:8775/axis2/services/EucalyptusNC) running instance: i-35E9072D emi-E750108E d0:0d:35:E9:07:2D d0:0d:35:E9:07:2D 10 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTMSCSb4VovoHjYjlm5QpmSHZi0WPe4HD5PGoN+jqPoLESJBNIM85g2DPQnEtxy3JIQk5iLpM6yqM+qsZKjnTRg9OIid2Ile/dI5nVjt6UgCcnz9KJSlzdKPhsGxsviRJS2gKJ1mkqZW3SKv1kAAyj5PQywwz5mTCfjd6BrhJhppHstFal0RQNwnpT2zgvvMtWpmgJkG3OOe2uZPQ74HOVGsrq+lcSJa5T6aWpSk8655QKCS0CYsAnMBOcBOhpfdAtv2GTiRgxtXaQgDFHfjI/GaguzesNtSyQWB5BAXlwWqYHJGpf5cYWvFyRiS1tOR0vpP4LM3JIq+vMoBwjlaD admin@eucalyptus
[Fri Sep 25 20:18:30 2009][019952][EUCAINFO ] asking for virtual hardware (mem/disk/cores): 128/2/1
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] time left for op: 60
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] call complete (pid/rc): 19952/0
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] RunInstances(): done
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] scheduler using ROUNDROBIN policy to find next resource
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] scheduler state starting at resource 0
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] scheduler state finishing at resource 0
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] scheduler decided to run instance 'i-35E9072D' on resource 'http://eucabuild29:8775/axis2/services/EucalyptusNC'
[Fri Sep 25 20:18:30 2009][019953][EUCAINFO ] client (http://eucabuild29:8775/axis2/services/EucalyptusNC) running instance: i-35E9072D emi-E750108E d0:0d:35:...

Read more...

summary: - SYSTEM mode, clc does not send runinstances.
+ SYSTEM mode, instances do not run
Revision history for this message
Neil Soman (neilsoman) wrote :

Revno 897 fixes this.

Changed in eucalyptus:
status: New → Fix Committed
Matt Zimmerman (mdz)
Changed in eucalyptus (Ubuntu):
importance: High → Medium
Daniel Nurmi (nurmi)
Changed in eucalyptus (Ubuntu):
status: Triaged → In Progress
assignee: nobody → Dustin Kirkland (kirkland)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package eucalyptus - 1.6~bzr912-0ubuntu1

---------------
eucalyptus (1.6~bzr912-0ubuntu1) karmic; urgency=low

  * Merge upstream revision 912
  * tools/eucalyptus-cc.in: resolve conflict, ensuring that we maintain
    the 'ThreadsPerChild 1' sed
  * clc/modules/www/src/main/java/edu/ucsb/eucalyptus/admin/server/ServletUtils.java:
    resolve conflict, drop our diff, as upstream has solved the smtp
    hostname issue
  * debian/patches/boot-order.patch: dropped, applies to a file we don't
    use anymore so it can be removed
  * debian/eucalyptus-walrus.eucalyptus-walrus-registration.upstart: source
    conf file and use $WALRUS_IP_ADDR variable, like the other scripts
  * This snapshot is expected to fix the following bugs:
    - LP: #388934 - apply authorization checks
    - LP: #430226 - fix display of image permissions
    - LP: #430957 - fix running of instances in SYSTEM mode
    - LP: #436276 - sc should run as eucalyptus, instead of root
    - LP: #436313 - fix sc registration through web ui
    - LP: #436407 - fix cc segfaults with apache-mpm-worker, rampart
    - LP: #436885 - fix database corruption, c3p0 deadlock on CLC
    - LP: #437014 - handle execessive CLC sockets
    - LP: #439251 - fix restart-required after autoregistration issue
    - LP: #440744 - handle external command
    - LP: #440817 - fail gracefully with volume deletion on sc
    - LP: #443125 - fix loss of admin credentials

 -- Dustin Kirkland <email address hidden> Mon, 05 Oct 2009 15:02:11 -0500

Changed in eucalyptus (Ubuntu):
status: In Progress → Fix Released
graziano obertelli (graziano.obertelli)
Changed in eucalyptus:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.