Improve SMP kernel performance on UP systems

Bug #12622 reported by Sam Williams
12
Affects Status Importance Assigned to Milestone
linux-source-2.6.15 (Ubuntu)
Fix Released
Medium
Ben Collins

Bug Description

I wanted to use the Hoary live cd over the weekend to fire up a Quad Opteron
system in one of my labs and actually run some stress testing programs.
Unfortunately when the CD booted it only saw 1 processor. Since its live I
couldn't apt-get install a more appropriate smp kernel, because even if I could
download it the reboot would kill it anyway. Not a show stopped, but something
you might want to take a look at.

Revision history for this message
Matt Zimmerman (mdz) wrote :

What would be the consequences of using an SMP kernel by default? I assume
there is a significant tradeoff for uniprocessor systems (locking overhead?) to
justify having separate kernels.

Colin, would it be possible to get an alternate kernel onto the live CD as a
second isolinux option?

Revision history for this message
Johan Walles (walles) wrote :

Bugzilla voting seems to have been disabled (bug 4443), so: "Me too".

This would enable me to verify bug 10392 before Breezy release.

I'm interested in this for 32 bit ia32 systems though. Is there any reason for
this bug to be "Hardware: AMD64" rather than "Hardware: All"? Should I open a
new bug for ia32?

Revision history for this message
Matt Zimmerman (mdz) wrote :

Passing the buck to the kernel team on this one. We raised this topic at Ubuntu
Down Under, and I asked that we measure the performance difference in running an
SMP kernel on uniprocessor hardware. If we had a single kernel for both UP and
SMP systems, we could use this kernel on the live CD. It isn't practical to
include additional kernels on the live CD because space is limited and kernels
are large.

Revision history for this message
Ben Collins (ben-collins) wrote :

I did some searching and found that an SMP kernel on a UP system has unavoidable
performance issues due to the nature of the lock operation. Some reports say
that the operation can take as many as 12 cycles from the CPU, and that almost
always it will cause a pipeline flush because of the operation ordering.

However, I did find the below thread and patch, which can be easily ported to
the current kernel:

http://www.ussg.iu.edu/hypermail/linux/kernel/0203.2/0178.html

It implements a function which is called when an SMP kernel boots on a UP
system. In this case, it does a dynamic code rewrite based on a lookup table,
which turns all lock op's into nop's. This is a vast improvement on the standard
SMP kernel when run on a UP system (likely this would also take affect if
someone boots with nosmp).

I suggest for now that we forget about doing anything related to this in breezy.
Post-breezy, it would be nice to start testing this. It should be easy to
implement the same thing for other architectures (ppc, and amd64 for example).

Revision history for this message
Ben Collins (ben-collins) wrote :

This bug has been flagged because it is old and possibly inactive. It may or may
not be fixed in the latest release (Breezy Badger 5.10). It is being marked as
"NEEDSINFO". In two weeks time, if the bug is not updated back to "NEW" and
validated against Breezy, it will be closed.

This is needed in order to help manage the current bug list for the kernel. We
would like to fix all bugs, but need users to test and help with debugging.

If this change was in error for this bug, please respond and make the
appropriate change (or email <email address hidden> if you cannot make the
change).

Thanks for your help.

Revision history for this message
Colin Watson (cjwatson) wrote :

It sounds to me as if this bug is fixed in our 2.6.15 series?

Revision history for this message
Ben Collins (ben-collins) wrote :

SMP2UP is in dapper kernel.

Revision history for this message
Johan Walles (walles) wrote :

Does that mean that I can expect the Dapper LiveCD updated on november 18
available from here...

  http://cdimage.ubuntu.com/releases/dapper/flight-1/

... to boot in SMP mode on my SMP machine?

Revision history for this message
Colin Watson (cjwatson) wrote :

(In reply to comment #8)
> Does that mean that I can expect the Dapper LiveCD updated on november 18
> available from here...
>
> http://cdimage.ubuntu.com/releases/dapper/flight-1/
>
> ... to boot in SMP mode on my SMP machine?

No, because Flight CD 1 predates the move to 2.6.15. Flight CD 2 should work
this way.

Revision history for this message
Colin Watson (cjwatson) wrote :

(In reply to comment #9)
> Flight CD 2 should work this way.

... when it's released - soon.

Revision history for this message
Ben Collins (ben-collins) wrote :

(In reply to comment #9)
> (In reply to comment #8)
> > Does that mean that I can expect the Dapper LiveCD updated on november 18
> > available from here...
> >
> > http://cdimage.ubuntu.com/releases/dapper/flight-1/
> >
> > ... to boot in SMP mode on my SMP machine?
>
> No, because Flight CD 1 predates the move to 2.6.15. Flight CD 2 should work
> this way.

That depends, if it's using -386 or -686 to boot. I suspect it's using -386,
right? Which is UP only.

The -686 and -k7 kernels have this feature though.

Revision history for this message
Colin Watson (cjwatson) wrote :

Oh, yeah, it's using -386, and we probably still don't have room for more than
one kernel on the CD (as with every previous time we've discussed that ...). Hmm.

Revision history for this message
Johan Walles (walles) wrote :

Shouldn't this bug be re-opened?

AFAICT this bug was originally about having LiveCDs use all CPUs in a machine:

"
I wanted to use the Hoary live cd over the weekend to fire up a Quad Opteron
system in one of my labs and actually run some stress testing programs.
Unfortunately when the CD booted it only saw 1 processor.
"

The LiveCDs still use only one CPU of an SMP machine.

This bug doesn't look neither RESOLVED nor FIXED to me. Reopen?

Revision history for this message
Ben Collins (ben-collins) wrote :

(In reply to comment #13)
> Shouldn't this bug be re-opened?
>
> AFAICT this bug was originally about having LiveCDs use all CPUs in a machine:
>
> "
> I wanted to use the Hoary live cd over the weekend to fire up a Quad Opteron
> system in one of my labs and actually run some stress testing programs.
> Unfortunately when the CD booted it only saw 1 processor.
> "
>
> The LiveCDs still use only one CPU of an SMP machine.
>
> This bug doesn't look neither RESOLVED nor FIXED to me. Reopen?

The purpose of the bug report changed. I only did what the title and followup
threads suggested. Should this be re-assigned to ubuntu-live, or is there
already a bug report concerning this?

Revision history for this message
Johan Walles (walles) wrote :

I couldn't find one so I just filed bug 26774 about getting SMP support on the
LiveCDs.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.