System runs out of memory - white screens frequently seen in web renderers

Status changed to 'Confirmed' because the bug affects multiple users.

The white screen is very likely a renderer process crash. I’ve tried to reproduce the issue on my krillin with image #2 and all updates, but so far no luck. I’ll keep trying.

The oxide rendered is being killed
[ 9484.781547] (1)[19379:scoperunner]select 'oxide-renderer' (19224), adj 300, size 7805, to kill
[ 9484.781580] (1)[19379:scoperunner]Killing 'oxide-renderer' (19224), adj 300,
[ 9484.781586] (1)[19379:scoperunner] to free 31220kB on behalf of 'scoperunner' (19379) because
[ 9484.781594] (1)[19379:scoperunner] cache 65428kB is below limit 65536kB for oom_score_adj 12
[ 9484.781601] (1)[19379:scoperunner] Free memory is -3076kB above reserved [gfp(0x200da)]
[ 9494.491841] (2)[2435:Mir/IPC]select 'webbrowser-app' (19130), adj 798, size 10527, to kill
[ 9494.491882] (2)[2435:Mir/IPC]Killing 'webbrowser-app' (19130), adj 798,
[ 9494.491889] (2)[2435:Mir/IPC] to free 42108kB on behalf of 'Mir/IPC' (2435) because
[ 9494.491895] (2)[2435:Mir/IPC] cache 65432kB is below limit 65536kB for oom_score_adj 12
[ 9494.491902] (2)[2435:Mir/IPC] Free memory is -2676kB above reserved [gfp(0x282d2)]

Confirmed from Tom’s syslog: it appears the system is running with a high memory load, and OOM killer gets rid of processes to free up memory. It happens that some of those processes are the oxide renderers, but it’s not only them that are affected.

Added a unity8 task to the report as it appears unity8-dash is eating up a lot of memory, even when idle after a reboot. Tom is seeing it consume 45% memory, while I’m seeing it at a stable 12.7%, which still feels a lot to me (if one adds to that the 8.8% of unity8, then the shell is already consuming more than 20% of the system’s available memory).

Not sure whether something can be done at the system level to mitigate the aggressiveness with which oxide-renderer is being killed (with regards to other applications).

I think there are still 2 options there. Either the OOM killer should get rid of the whole process group, so that webbrowser or webapps are not "half killed'. Or oxide should trap a signal indicating that the underlying renderer processes died, and let the client apps quit.

we did very recently fix a bug in memleak qtubuntu.
please retest ensuring an image with the branch linked against
bug 1206146

https://bugs.launchpad.net/ubuntu/+source/webbrowser-app/+bug/1303676 provides some background about the general problem, and has links to other individual bug / tasks and branches that were used to manage the problem a few months ago.

Client apps should definitely not quit if the render process quits, as that kind of defeats the purpose of having isolated render processes (if they crash, they should not take down the app)

> we did very recently fix a bug in memleak qtubuntu.
> please retest ensuring an image with the branch linked against
> bug 1206146

Unfortunately this fix isn’t even part of a 14.09-proposed image as far as I can tell.

after some discussion on IRC, this isn't a memory leak.
however, this is more like a request for memory optimization of unity8 dash/scopes etc.

according to the guys doing the testing
they see ~45% mem consumption due to unity8-dash, but then it settles to ~12% thinking its due to swap.
So is there an action there to improve this ? make it happen faster ?

12% is when idle after a reboot and without any additional scopes other than the default ones. It seems that adding custom scopes to the dash makes the memory usage grow significantly.

I've removed all scopes apart from the app scope, and rebooted my phone. unity8-dash is currently using about half the virtual memory it was previously (I had configured the Wikipedia scope, the BBC news scope, the My Music scope and the My Photos scope to appear in my scope list) upon initial reboot.

I've just updated to krillin 14.09 #3 + all updates applied. Here's unity processes from just after a reboot. Will keep an eye on it and let you know if things get worse:

https://pastebin.canonical.com/117883/

I've still got just the app scope configured in my dash.

Status changed to 'Confirmed' because the bug affects multiple users.

I think this bug should be de-duped, and re-opened.

the behaviour is back, and the fix in bug #1376165 seems to have been insufficient :-)

Recent reproduction notes (on OEM bug #1490529 )

Scrolling in webapps can cause white screen not being able to scroll anymore.

Product: Krillin bq Aquaris E4.5, E5
FW version: r24,r4
HW version: MP

Actual Result:
FB & browser screen go white after scrolling a while.

Expected Result:
Facebook should continue scrolling and not getting freeze in a white screen.

VIDEO: https://drive.google.com/a/booqreaders.com/file/d/0B1rUJ4nMTZlhZmVqM0ItZ1ZrRjQ/view?usp=sharing

Confirmation in Ubuntu-Phone mailing list: https://lists.launchpad.net/ubuntu-phone/msg15360.html

FYI, I've also seen this on arale.

The browser's render process being killed by OOM is bug 1478853

After OTA7 I have still white screens in a WebApp. Mostly when enter App again coming from another App, but also during using the App.
I use a BQ4.5

"white screens" while using the app is pretty extreme: it means that the
renderer gets killed while the app runs, as opposed to the oom killer
getting rid of a dormant app to free up some resources for the foreground
tasks.

Is there are particular webapp where you manage to reproduce that more
specifically ?

On Wed, Oct 21, 2015 at 7:27 PM, Ilonka <email address hidden> wrote:

> After OTA7 I have still white screens in a WebApp. Mostly when enter App
> again coming from another App, but also during using the App.
> I use a BQ4.5
>
> --
> You received this bug notification because you are a bug assignee.
> Matching subscriptions: oxide in Ubuntu
> https://bugs.launchpad.net/bugs/1375215
>
> Title:
> System runs out of memory - white screens frequently seen in web
> renderers
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1375215/+subscriptions
>

This bug was fixed in the package webbrowser-app - 0.23+15.10.20151022.1-0ubuntu1

---------------
webbrowser-app (0.23+15.10.20151022.1-0ubuntu1) wily; urgency=medium

  [ CI Train Bot ]
  * New rebuild forced.
  * Resync trunk.

  [ Olivier Tilloy ]
  * Add an exception to the generated apparmor profile to allow reading
    HERE’s TOS in the browser. (LP: #1507667)
  * Modify the generated apparmor profile to allow rw access to
    /dev/shm/.org.chromium.Chromium.* too. (LP: #1508054)
  * Update translation template.

  [ Ugo Riboni ]
  * Fix inability to drag the map to pan in Google maps, on desktop.
    (LP: #1503506)
  * Implement support for allowing or denying access to media input
    devices and for setting default media input devices. (LP: #1410996)
  * Refactor the BookmarksModel to be a singleton.

 -- Olivier Tilloy <email address hidden> Thu, 22 Oct 2015 15:07:49 +0000

Hi, IMO this persists in RC-proposed.
uNav has a white page when you return from another to it. uNav is generating a lot of process/second.
Thanks in advance!

@Marcos: what do you mean by "uNav is generating a lot of process/second"?

IIRC I once suggested you to monitor the 'webProcessStatus' property of the webview, and offer some user feedback when it’s killed or crashed. It doesn’t look like anything like that is implemented in uNav. You can take a look at how it’s done in the browser app: https://bazaar.launchpad.net/~phablet-team/webbrowser-app/trunk/view/head:/src/app/WebProcessMonitor.qml.

Hi Olivier :)

> @Marcos: what do you mean by "uNav is generating a lot of
> process/second"?

uNav is doing a lot of calculations by second when you're driving.
Then the CPU usage is so high.

> IIRC I once suggested you to monitor the 'webProcessStatus' property of
> the webview, and offer some user feedback when it’s killed or crashed.

We have this log from users and experimented by myself:

file:///opt/click.ubuntu.com/navigator.costales/0.59/qml/Main.qml:103:
TypeError: Type error
ubuntumirclient: Attempted to deliver an event to a non-existent
window, ignoring.

This is because the webview was killed and uNav doesn't find it.

Regenerate the webview could be a solution, but it would lost a current route.

A hug!

Hi Olivier
I'm experiencing the same problem mentioned by Marcos, from 2/3 OTAs now. In my case you don't even need to return from another app, it happens while driving.

Thank you guys

Marcos and Michele: if you can reliably reproduce the issue, could you please monitor what’s happening with the CPU and memory usage of all processes (using top in phablet-shell) when that happens?

When the web process is killed by the system because it is running out of memory (CPU usage shouldn’t be a problem), the syslog (/var/log/syslog) should contain useful information, could you please attach your syslog to this bug report just after the problem happened? Thanks!

Monitoring webProcessStatus and then doing a reload if the process is killed by the system is probably ok for most webapps, but this isn't great for a navigation app. What happens to the state of the current navigation if the render process is killed and the webview has to be reloaded?

I agree, reloading the webview (thus most probably loosing the current navigation state) wouldn’t be great. Displaying an error message would at the very least give some useful feedback to the user, as opposed to a plain white view.

Of course this is not enough, we need to understand where the app consumes so much memory that it gets killed by the system, and if that can be avoided.

Hi Olivier :)

I found a behavior: Open uNav and put in background. Open it after
8hours and uNav will have a white screen (because it hasn't that
webview anymore, I don't know why).

I saw some apps, like Telegram, are reloaded when they come from
background, but it is not a new fresh start. Appears the application
reloads to itself (?).

In uNav there is nothing for reload the webview. Am I missing an app cycle life?

Thanks in advance!!
Costales.

Marcos, what probably happened is that during those 8 hours that the app was in the background your device ran low on memory at some point, and the system decided to kill the oxide renderer process to free up some memory. The way to go in that case is to monitor webProcessStatus, as I suggested earlier: upon moving the app to the foreground, you will be notified that your webview needs reloading.

Thanks a lot Olivier for your support :) We'll try that |o/
--
Sent using Dekko from my Ubuntu device