cpu idle time in /proc/stat wrong

Some specifics on your hw (mobo type, etc) would be helpful, as would dmesg output. Please attach dmesg (IOW, do not paste into comment).

Asus W5A laptop
Pentium-M 1.73GHz, i915 chipset, 1.25GB DDR2 RAM
dmesg attatched.

00] PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
[4294670.678000] PCI: Ignore bogus resource 6 [0:0] of 0000:00:02.0
[4294670.678000] PCI: Bus 2, cardbus bridge: 0000:01:03.0
[4294670.678000] IO window: 0000d000-0000d0ff
[4294670.678000] IO window: 0000d400-0000d4ff
[4294670.678000] PREFETCH window: 50000000-51ffffff
[4294670.678000] MEM window: 52000000-53ffffff
[4294670.678000] PCI: Bridge: 0000:00:1e.0
[4294670.678000] IO window: d000-dfff
[4294670.678000] MEM window: fe800000-fe8fffff
[4294670.678000] PREFETCH window: 50000000-51ffffff
[4294670.678000] PCI: Setting latency timer of device 0000:00:1e.0 to 64
[4294670.678000] ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 21 (level, low) -> IRQ 169
[4294670.678000] audit: initializing netlink socket (disabled)
[4294670.678000] audit(1139677535.676:1): initialized
[4294670.678000] VFS: Disk quotas dquot_6.5.1
[4294670.678000] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[4294670.678000] Initializing Cryptographic API
[4294670.678000] io scheduler noop registered
[4294670.678000] io scheduler anticipatory registered
[4294670.678000] io scheduler deadline registered
[4294670.678000] io scheduler cfq registered
[4294670.679000] isapnp: Scanning for PnP cards...
[4294671.036000] isapnp: No Plug & Play device found
[4294671.050000] PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
[4294671.052000] i8042.c: Detected active multiplexing controller, rev 1.1.
[4294671.053000] serio: i8042 AUX0 port at 0x60,0x64 irq 12
[4294671.054000] serio: i8042 AUX1 port at 0x60,0x64 irq 12
[4294671.054000] serio: i8042 AUX2 port at 0x60,0x64 irq 12
[4294671.054000] serio: i8042 AUX3 port at 0x60,0x64 irq 12
[4294671.054000] serio: i8042 KBD port at 0x60,0x64 irq 1
[4294671.054000] Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
[4294671.055000] RAMDISK driver initialized: 16 RAM disks of 65536K size 1024 blocksize
[4294671.055000] Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
[4294671.055000] ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
[4294671.055000] ACPI: bus type ide registered
[4294671.055000] mice: PS/2 mouse device common for all mice
[4294671.056000] EISA: Probing bus 0 at eisa.0
[4294671.056000] EISA: Detected 0 cards.
[4294671.056000] NET: Registered protocol family 2
[4294671.065000] IP route cache hash table entries: 65536 (order: 6, 262144 bytes)
[4294671.065000] TCP established hash table entries: 262144 (order: 9, 3145728 bytes)
[4294671.069000] TCP bind hash table entries: 65536 (order: 7, 786432 bytes)
[4294671.070000] TCP: Hash tables configured (established 262144 bind 65536)
[4294671.070000] TCP reno registered
[4294671.070000] TCP bic registered
[4294671.070000] NET: Registered protocol family 1
[4294671.070000] NET: Registered protocol family 8
[4294671.070000] NET: Registered protocol family 20
[4294671.070000] Using IPI No-Shortcut mode
[4294671.071000] ACPI wakeup devices:
[4294671.071000] P0P1 LAN1 CBS0 P394 ...

Well, I didn't paste that into the bug, I attatched it via the email I sent in reply, but it seems that doesn't work right yet, https://launchpad.net/products/malone/+bug/30225

I am also seeing this on a Toshiba Portege R200. Similar age machine to James', Intel Pentium M, i915 chipset, etc. I think the testing team has one of these on hand.

Subscribing Stephen Herman as he's listed on the LaptopTestingTeam wiki page as having an R200.

Same problem here on an HP Compaq nx8220. Pentium-M 2.13Ghz, i915 chipset, 1GB DDR2 RAM, ATI X600 graphics.

Interestingly playing audio seems to workaround the issue.

Same here on a Sony VAIO fj1s/w with the i915 chipset.

can't see this happen on the r200.
the battery life is just as normal....
and cpu idle timer is behaving normally.

Stephan, with what kernel?

I wasn't getting it with 2.6.12, but it is now noticable with 2.6.15-ubuntulatest. With my machine doing nothing, I am seeing usages of 11%, 44%, 22%, 66% etc. Interestingly all multiples of 11.

It's the latest dapper kernel:

Linux r200 2.6.15-15-386 #1 PREEMPT Thu Feb 9 19:41:22 UTC 2006 i686 GNU/Linux

But I must correct myself:

The cpu idle time is really increasing about 100 ticks per second, but the system is behaving properly. 2 1/2 to 3 hours battery life is normal for this r200.
Depends on what I'm doing. On Windows it's just 2 to 2 1/2 hours of working time on battery.

I can determine the high load of my cpu, because the machine itself turns on cpu fan. On Windows this fan is running 75% times more often then with Ubuntu Linux.

Only during dist-upgrades (while it's preparing and setting up the packages) I can see a higher increase of the cpu.

I think the reason you are seeing these numbers is because the HZ in dapper's 2.6.15 is 1000 where as in breezy (and probably debian 2.6.15) it is 100.

So the counts are finer grained, but the actual time is the same.

If your battery is not lasting long enough, that's a seperate issue.

So to clarify: When the system is idle the 4th value should increase by roughly 1000 a second? The proc man page certainly seems to say that this is the case.

If that is the case then there is definitely a problem, as when sitting idle the count increase by much, much less. The following two lines were taken with a 1 second sleep in between:

cpu 243374 424 57571 576399 51212 3967 1108 0

cpu 243376 424 57572 576419 51214 3967 1108 0

Battery life is an ancillary issue, as this miss reporting of idle time seems to coincide with cpu governors to stop working correctly.

To reiterate, this issue is NOT present in the Debian 2.6.15 kernel. According to some preliminary analysis this is possibly related to an Ubuntu patch for C-state handling in ACPI.

This issue was not present in 2.6.12 and is affecting things primarily related to power management in the machines (powernowd thinks the machine is not idle, therefore it ramps up the clock, therefore battery lasts less time).

I can comfirm (thanks to Davyd's tip) that this problem is indeed related to C-states.

Setting the max cpu C-state to C2 (it spends most of it's time in C3 by default) using:

echo 2 > /sys/module/processor/parameters/max_cstate

Causes the /proc/stat values start to increase by reasonable values and more importantly cpufreq_conservative actually starts to scale the cpu speed when idle.

Indeed, setting my max_cstate to C1 makes the idle clock ticks increase by 100/s and idle time seem much more reasonable.

Running the -386 kernel makes everything run ok again. BenC suggests on irc that it's a SMP issue as that's all that's really different between -386 and -686.

I am having the same issues.

Attached is my dmesg output for Dell Inspiron 6000 laptop. Brandon Hale of the Ubuntu Laptop Testing Team has the same laptop.

Same issues whether using the stock 386 or 686 kernel.

I'm going to test it out using this command:
echo 2 > /sys/module/processor/parameters/max_cstate

Further testing with the 386 kernel suggests to me that "running the -386 kernel makes everything run ok again. BenC suggests on irc that it's a SMP issue as that's all that's really different between -386 and -686."

I am now running the stock 386 kernel and it is a lot more CPU friendly:
julian@jubuntu:~$ uname -a
Linux jubuntu 2.6.15-21-386 #1 PREEMPT Fri Apr 21 16:43:33 UTC 2006 i686 GNU/Linux

So yes. I agree, it's a SMP issue. I have since reverted my /sys/module/processor/parameters/max_cstate to the previous value of 8.

Is there any reason why the 686 kernel is a SMP one?

The workaround is good for me (Pentium M 2.13 GHz) but I'd like a fix :)

Note that this workaround causes a performance hit in my cpu-intensive apps, so it's not really an acceptable solution.

> Setting the max cpu C-state to C2
> (it spends most of it's time in C3 by default) using:

> echo 2 > /sys/module/processor/parameters/max_cstate

> Causes the /proc/stat values start to increase by
> reasonable values and more importantly cpufreq_conservative
> actually starts to scale the cpu speed when idle.

On Thu, 2006-05-18 at 07:04 +0000, Pausanias wrote:
> Note that this workaround causes a performance hit in my cpu-intensive
> apps, so it's not really an acceptable solution.
>
> > Setting the max cpu C-state to C2
> > (it spends most of it's time in C3 by default) using:
>
> > echo 2 > /sys/module/processor/parameters/max_cstate
>
> > Causes the /proc/stat values start to increase by
> > reasonable values and more importantly cpufreq_conservative
> > actually starts to scale the cpu speed when idle.

Can we escalate the priority of this bug? It is a ubuntu patch (iirc)
and we don't need any help from upstream.

I can confirm that this is still a problem in the 2.6.15-23 kernel.

Marking this as "confirmed" given that so many people have corroborated it here and in duplicate bug reports. Again, this is an ubuntu-only problem (patch for C-state handling in ACPI), and it appears to affect all Pentium M CPUs running the default 686 kernel.

> Note that this workaround causes a performance hit in my cpu-intensive apps, so it's > not really an acceptable solution.

No it doesn't. C states are power saving states - limiting the maximum C state will have a marginal improvement on performance, and certainly not a hit.

Ben, this sounds like something mad like the jiffies counter not being increased in C3 and C4 states. Can you take a quick look at the code there?

I'd like to emphasize again that this bug has severe implications on some systems. I just found out that bug #50341 is actually caused by this bug. To summarize it: As long as I don't work around with max_cstate=1, I can't even scroll smoothly in an xterm (not speaking about something like firefox) and my keyboard repeat rate goes down to about 1/6 of the value with max_cstate=1.

I too would like to emphasize the importance of this issue. Performance is significantly degraded on my HP nc6230 with 686 kernel. With such a variety of systems being impacted, I hope this issue will be be changed to a higher level of importance.

This problem occurs also on Celeron-M processors (like mine 360J in HP nx6110). Performance in Dapper is severely degraded when compared to Breezy.

Output of dmesg on HP nx6110 (Celeron-M 360J)

Confirming the bug with latest Dapper and HP nx6110. Willing to assist with debugging as I am a DD and am willing to follow instructions to fix this.

Last kernel 686-26 doesn't seem to solve the problem for me.
At the moment I use 386 kernel because I think that set max_cstate=1 decreases battery life according to this web: http://www.thinkwiki.org/wiki/Talk:Problem_with_high_pitch_noises

I can confirm that this is a SMP related issue. I re-built my kernel with the only differences being that I changed my CPU to a Pentium-M, turned off SMP, and modified the ticks to 250/sec.

Performance is back to that of Breezy on my Celeron-M 1.4Ghz laptop, and I don't think its the different CPU settings, or the scheduler. Interactivity, especially while in a terminal is also much improved.

On Wed, 2006-07-12 at 23:04 +0000, Jer wrote:
> I can confirm that this is a SMP related issue. I re-built my kernel
> with the only differences being that I changed my CPU to a Pentium-M,
> turned off SMP, and modified the ticks to 250/sec.
>
> Performance is back to that of Breezy on my Celeron-M 1.4Ghz laptop, and
> I don't think its the different CPU settings, or the scheduler.
> Interactivity, especially while in a terminal is also much improved.

Basically what you did was recreate the -386 kernel.

I've put together a test program that demonstrates the
performance degradation introduced by this bug.
The attached tar ball contains information about the test program
and oprofile runs demonstrating the differences in
performance with max_cstate=1 and max_cstate=8.

I believe Ubuntu defect 50153 is a duplicate of 30557
because the symptoms and workarounds are the same.
However, 50153 is not reproducible in 2.6.15-23. I first
noticed the problem in 2.6.15-25. This is in conflict
with what some other reporters have experienced, with
30557 first appearing in 2.6.15-15.

The attached tar file contains the oprofile reports, the
test program, and a more detailed description of my tests.

This problem still occurs if I use the stock kernel.org 2.6.15 source (or the Debian source) with the Ubuntu 686 config file. I see a couple of possibilities:

1) There are two separate bugs being discussed here. This would explain several discrepancies in the bug reports, particularly why some people see serious performance degradation, and others only reduced battery life.

2) These are the same bug with different environments producing different symptoms.

Given that the bug I'm seeing (which is probably the same as the one Steve Bourg is seeing) can be reproduced using the stock kernel source code, should this issue be pushed upstream?

Well, my problem goes away if I change my config from CONFIG_HZ_250 to CONFIG_HZ_1000. The change from 1000 to 250 was done in 2.5.15-24, which corresponds nicely to when I started to see problems with sluggish performance.

  * i386/amd64: Change HZ=1000 to HZ=250. The high frequency was causing high
    power consumption on some laptops, and also some latency under certain I/O
    loads.

Even when I rebuild with CONFIG_HZ_1000, the idle time in /proc/stat continues to go up very slowly. Seems more and more likely we're talking about different problems.

Nonetheless, I'd much prefer it if the ubuntu 686 kernel didn't hose X performance on my laptop. I doubt regressing the "HZ=250" change would count as progress, but if anyone has suggestions for further research into why HZ=250 causes problems on my machine, I'd be interested in hearing them.

I'd also be interested whether anybody else reporting issues sees a change when HZ is set to 1000 instead of 250.

Neither bug, the /proc/stat issue or the lousy X performance, is reproducible in 2.6.17.5. I'm going to try to figure out which kernel rev resolved the issue(s).

Both problems are reproducible in kernel.org 2.6.15.7. Both appear to be fixed in 2.6.16.

The problem and the fix are described in these notes to LKML:

http://lkml.org/lkml/2005/12/8/304

There are some further changes upstream to deal with bugs in the first set of patches. Right now I'm running the Ubuntu 2.6.15-26 code with this set of patches, and things look good:

5a07a30c3cc4dc438494d6416ffa74008a2194b3
6eb0a0fd059598ee0d49c6283ce25cccd743e9fc
d25bf7e5fe73b5b6d2246ab0be08ae35d718456b
76b461c21468f41837283b7888d55f1c0671f719

Thanks. Patches applied for next dapper update.

Is this supposed to be fixed for 2.6.15-26-686? Because it still isn't
fixed for me (top reports ~80% CPU usage with max_cstate set to 3)

On Wed, 2006-07-19 at 16:06 +0000, Ben Collins wrote:
> Thanks. Patches applied for next dapper update.
>
> ** Changed in: linux-source-2.6.15 (Ubuntu)
> Assignee: (unassigned) => Ben Collins
> Status: Confirmed => Fix Committed
>

On Thu, 2006-07-20 at 02:56 +0000, sohail wrote:
> Is this supposed to be fixed for 2.6.15-26-686? Because it still isn't
> fixed for me (top reports ~80% CPU usage with max_cstate set to 3)

"Fix committed" means it's in our source repo. When it's marked "Fix
released", then it will mean it's uploaded and available.

Ah. I picked up a kernel update last night that was the version below so
I thought it might have been that.

Thanks!

On Thu, 2006-07-20 at 03:25 +0000, Ben Collins wrote:
> On Thu, 2006-07-20 at 02:56 +0000, sohail wrote:
> > Is this supposed to be fixed for 2.6.15-26-686? Because it still isn't
> > fixed for me (top reports ~80% CPU usage with max_cstate set to 3)
>
>
> "Fix committed" means it's in our source repo. When it's marked "Fix
> released", then it will mean it's uploaded and available.
>

Should the bug now be marked "Fix released"?

According to the changelog on the latest (2.6.15-26.46) dapper kernel:
http://changelogs.ubuntu.com/changel...6.46/changelog
It seems to be out now.

I'm now running linux-image-2.6.15-26-686, version 2.6.15-26.46 on a Pentium M 740 1.73MHz (Acer Aspire 1692WLMi).

So far, it seems fine: Without changing max_cstate, CPU rapidly throtles down to 800MHz and, when idle, CPU usage settles around 5--6%. Starting new apps, such as opening a firefox window, does not feel sluggish like it used to.

Here is a sample of data taken some 19 minutes after booting, laptop connected to AC power, with Wi-Fi on and a USB mouse plugged in:

$ cat /sys/module/processor/parameters/max_cstate
8

$ top

top - 18:56:55 up 14 min, 3 users, load average: 0.08, 0.33, 0.27
Tasks: 103 total, 1 running, 102 sleeping, 0 stopped, 0 zombie
Cpu(s): 5.3% us, 0.0% sy, 0.0% ni, 94.7% id, 0.0% wa, 0.0% hi, 0.0% si
...

$ cat /proc/acpi/processor/CPU*/power
active state: C2
max_cstate: C8
bus master activity: 04004002
states:
    C1: type[C1] promotion[C2] demotion[--] latency[000] usage[00000010]
   *C2: type[C2] promotion[C3] demotion[C1] latency[001] usage[00356747]
    C3: type[C3] promotion[C4] demotion[C2] latency[085] usage[00053269]
    C4: type[C3] promotion[--] demotion[C3] latency[185] usage[00065042]

Looks like this one is fixed. Thanks.

Dell Latitude D810 here.

This problem seems dramatically lessened but not altogether gone. Dragging windows and other video-intensive actions are marginally slower than with the -386 kernel. Also my screensavers now run at about 10% their regular speed. (ati binary driver) Yes, it's like watching a slideshow now.

I've been following this thread/problem for quite awhile, and hoped that this kernel update would solve all the problems. Looks like it's FAR better than it was, but things still aren't 'normal' vs. the -386 kernel.

I'd stick with the -386 kernel except that vmware is almost unusable with it; the -686 kernel is considerably faster.

Dell Inspiron 6000. Problem solved.

@anonym - if you've got the time and the patience, check to see
whether upstream kernels fix your problem. git-bisect can help you
narrow it down to which change fixed the bug.

Problem resolved with HP/Compaq nc6230

how do you guys apply these fixes?

my adept/synaptic managers last time downloaded updates 24 hours ago and problem is still there.

is there any way to see if this patch is implemented?

or download it manually?

is there any guide how to handle this bug fixing beyond running usual adept "fetch updates"?

I think you can check that you are using the new kernel this way:

$ cat /proc/version
Linux version 2.6.15-26-686 (buildd@terranova) (gcc version 4.0.3
(Ubuntu 4.0.3-1ubuntu5)) #1 SMP PREEMPT Thu Aug 3 03:13:28 UTC 2006

You may need to edit /boot/grub/menu.lst (CAREFULLY) or use the grub
boot menu to switch to the new kernel.

On Fri, 2006-08-04 at 13:35 +0000, João Manuel Rodrigues wrote:
> Should the bug now be marked "Fix released"?
>
> According to the changelog on the latest (2.6.15-26.46) dapper kernel:
> http://changelogs.ubuntu.com/changel...6.46/changelog
> It seems to be out now.
>

Seems to work fine for me now. Dell XPS Gen2.

Though I haven't compared it to -386 so I can't say for sure.

thanks Brian, looks like my kernel headers and image are updated (not sure about restricted modules, is that relevant?).

I am attaching little info from my laptop (DELL 600m Celeron 1.5Ghz). My ultimate problem of processor persistance in high C2 power state remains.

With 2.6.12 kernel I have frequent C3 and C4 states, with current 2.6.15-26.46 - very low.

Fixed my problem with slow graphics (Dell D810) - I had the correct binary driver installed for my kernel, but for some reason depmod -a was never run and the module wasn't loading.

After fixing that everything seems ok now.

how exactly you lowered your graphics, I don't have any fancy effects either. but kernel is in C2 for 99.9% time

I confirm anonym's problems.

I have an Acer aspire 1692WLmi, Pentium M740 with ATI X700 graphics. With the proprietary fglrx driver (version 8.25.18), hardware 3D is unavailable with -686 kernel (version 2.6.15-26.46) but works with -386 kernel. This can be verified by running glxinfo and checking the "direct rendering:" line.

However, I suspect this may be a different bug, and possibly not with the kernel but with the fglrx driver or the restricted kernel module it needs.

This probably does not belong here, but just to let you know:

With latest update of the 686 kernel (2.6.15-26.47), restricted modules (2.6.15.11-4) and proprietary ATI fglrx driver (7.0.0-8.25.18+2.6.15.11-4),
I now have hardware 3D with the 686 kernel. Problem seems to be gone.

i am still having the problem of high CPU usage while idle, movie playback choppy etc. The maxcstate patch does take care of the problem. it appears that some effort has be taken to fix this, although i doesn't work with my configuration.

asus z70va notebook pent. m. w/ fglrx driver.

is this an issue that is still being looked into, or could something else in my configuration still be causing this problem?

thanks
Arlen

this bug is still present on feisty herd2 on a sony vaio fs11m (centrimo 1.6GHz)
kernel:
2.6.20-5-generic

changing max_cstate as follows, resolve the issue:

echo 1 > /sys/module/processor/parameters/max_cstate

On 7/12/06, Ben Collins <email address hidden> wrote:
> On Wed, 2006-07-12 at 23:04 +0000, Jer wrote:
> > I can confirm that this is a SMP related issue. I re-built my kernel
> > with the only differences being that I changed my CPU to a Pentium-M,
> > turned off SMP, and modified the ticks to 250/sec.
> >
> > Performance is back to that of Breezy on my Celeron-M 1.4Ghz laptop, and
> > I don't think its the different CPU settings, or the scheduler.
> > Interactivity, especially while in a terminal is also much improved.
>
> Basically what you did was recreate the -386 kernel.
>
> --
> cpu idle time in /proc/stat wrong
> https://launchpad.net/bugs/30557
>

this bug is still present on feisty herd2 on a sony vaio fs11m (centrimo 1.6GHz)
kernel:
2.6.20-5-generic

changing max_cstate as follows, resolve the issue:

echo 1 > /sys/module/processor/parameters/max_cstate

On 10/30/06, hardyn <email address hidden> wrote:
> i am still having the problem of high CPU usage while idle, movie
> playback choppy etc. The maxcstate patch does take care of the problem.
> it appears that some effort has be taken to fix this, although i doesn't
> work with my configuration.
>
> asus z70va notebook pent. m. w/ fglrx driver.
>
> is this an issue that is still being looked into, or could something
> else in my configuration still be causing this problem?
>
> thanks
> Arlen
>
> --
> cpu idle time in /proc/stat wrong
> https://launchpad.net/bugs/30557
>

I confirm that this problem still exists in Feisty, including the latest 2.6.20-16 kernel.

I'm running on a Compal DL71 with an Intel Pentium M 740 1.7 MHz CPU. Out of the box, the system was not doing any CPU throttling, always running at max CPU clock. I had previously run Breezy 5.10 which didn't exhibit any of these symptoms.

Some Googling later, I learned to remove powernowd and how to use the cpufreq-info and cpufreq-set tools. With the "ondemand" governor, the system didn't throttle at all. With the "conservative" governor, it did throttle propertly, however KDE's power management tool (package: kde-guidance-powermanagemet) kept crashing because it doesn't know about the "conservative" governor.

Long story short, I recompiled the kernel, configuring it for single CPU (no SMP) and the Pentium M cpu, and CPU throttling with the "ondemand" governor now works well. The following articles were helpful with the kernel recompile:
https://help.ubuntu.com/community/Kernel/Compile
https://help.ubuntu.com/community/Suspend2Kernel
https://help.ubuntu.com/community/CustomRestrictedModules

I'm not an IT guy, but I am having problems with high CPU usage. Always over 90% on a fairly new install of Ubuntu running on a Pentium 4 on a Toshiba Satellite 1905-s301. What I did come to realise is that it happens only when I 'look' at the system monitor resources. I updated the preferences: graph update rate to 10 sec, Process update rate to 17sec, and File system to 29sec. Now my CPU History graph slips by at a cool 15%.

I think I am OK for now.