beagled uses >250 Megs RAM when Idle

What is the output of
* beagle-info --index-info
* beagle-info --status

and what is VMSize, RSS, Shared memory footprints of beagled ?

Are you using the Thunderbird backend? (Check beagle-index-info to see if it's listed)

Also, where are you getting the 250 meg number from? Is it the VSIZE or RSS column from top/ps or something else?

The 280 MB number was taken from the "Memory" column of the System Monitor
utility.

I have rebooted to find beagled using 35 megs at first. In the last hour of
normal system usage I've watched that number climb up to 84 megs. I am
keeping a close eye on the number now to see if I can determine what makes
it climb (Crawling, firefox or other plugin usage, etc.).

I will send more details as soon as I have collected additional relevant
data.

On 2/23/07, Joe Shaw <email address hidden> wrote:
>
> Also, where are you getting the 250 meg number from? Is it the VSIZE or
> RSS column from top/ps or something else?
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

In the 3 hours and 15 minutes since I booted this morning, I've seen beagled
memory usage slowly inch it's way up to the following:

Virtual: 249.2
Resident: 193.7
Writable: 182.8
Shared: 11.3
Memory: 182.8

I haven't noticed any ties between specific apps or activities and a climb
in memory usage but, rather, a slow but steady progression. Once memory
usage goes up it never goes down again.

Past experience tells me that, on this system, memory usage will eventually
top out at around 280 megs.

I am not using Thunderbird. The output of beagle-info --index-info and
beagle-info --status follows:

-----------------------------
jonathan@laptop:~$ beagle-info --index-info
Index information:
Name: EvolutionMail
Count: 11400
Crawling: False

Name: EvolutionDataServer
Count: 227
Crawling: False

Name: KMail
Count: 0
Crawling: False

Name: Files
Count: 91068
Crawling: True

Name: GaimLog
Count: 0
Crawling: False

Name: IndexingService
Count: 0
Crawling: False

Name: Tomboy
Count: 80
Crawling: False

Name: Labyrinth
Count: 0
Crawling: False

Name: Blam
Count: 0
Crawling: False

Name: Liferea
Count: 577
Crawling: False

Name: Akregator
Count: 0
Crawling: False

Name: KonquerorHistory
Count: 17
Crawling: False

Name: KonqBookmark
Count: 0
Crawling: False

Name: KNotes
Count: 0
Crawling: False

Name: KOrganizer
Count: 0
Crawling: False

Name: KAddressBook
Count: 0
Crawling: False

Name: Kopete
Count: 0
Crawling: False

Name: Konversation
Count: 0
Crawling: False

Name: applications
Count: 293
Crawling: False

Name: documentation
Count: 3946
Crawling: False

---------------------------

jonathan@laptop:~$ beagle-info --status
Scheduler:
Count: 6383
Status: Waiting for the next trigger time

Pending Tasks:
Scheduler queue is empty.

Future Tasks:
Maintenance 0 (2/23/2007 1:45:52 PM)
Optimize FileSystemIndex
Hold until 2/23/2007 1:55:52 PM

---------------------------

In the few minutes it has taken me to write the above, memory usage has
climbed to 191.2 megs.

After 5 1/2 hours of light, intermittent usage beagled is up to 305 megs.

Virtual: 381.8
Resident: 315.6
Writable: 304.4
Shared: 11.2
Memory: 3104.4

I won't bother including the output for beagle-info as it looks pretty much
identical to what I've previously posted.

Please let me know if there is any other information that I can supply.

Thanks,

Jonathan

Correction: "Memory: 3104.4" should read "304.4"

I dont think there is a package for heap-shot for ubuntu, so there isnt much information you can provide. You can provide one valuable information though - which index source (aka backend) is causing the memory rise. You can shutdown beagle (using beagle-shutdown) and then restart as
$ beagled --backend name_of_backend
to start beagle with only that backend
From index-info, you seem to be using these backends:
Files, EvolutionMail, EvolutionDataServer, Liferea, Tomboy

You may try each of the above, one at a time to see which exact backend is causing the behaviour. e.g. use
"$ beagled --backend EvolutionMail"
to only index the EvolutionMail data.

Beagle has to work on a variety of data sources; unfortunately, these memory bahevious are absent on the tests we perform on our computers. Any help is appreciated.

Ahh one more thing, can you attach the log directory ~/.beagle/Log/ as tar-gz ? If its too large, then attach the files pointed to by current-Beagle, current-IndexHelper. Thanks

Will do and get back to you. Thanks for the help. You guys do a huge service
for all of us.

Haven't had any luck narrowing the culprits; when I run only one of the 5
backends you specified at a time the memory bug doesn't come up.

I've attached a copy of the most current log file. Looking through it, I
noticed that it complained about being unable to find EvolutionDataServer. I
did double-check and it is installed. Not sure if that's a clue.

There are additional logs that I have not included (console, exceptions,
etc.). Please let me know if they would be of use.

Cheers.

On 2/24/07, dBera <email address hidden> wrote:
>
> I dont think there is a package for heap-shot for ubuntu, so there isnt
> much information you can provide. You can provide one valuable information
> though - which index source (aka backend) is causing the memory rise. You
> can shutdown beagle (using beagle-shutdown) and then restart as
> $ beagled --backend name_of_backend
> to start beagle with only that backend
> >From index-info, you seem to be using these backends:
> Files, EvolutionMail, EvolutionDataServer, Liferea, Tomboy
>
> You may try each of the above, one at a time to see which exact backend is
> causing the behaviour. e.g. use
> "$ beagled --backend EvolutionMail"
> to only index the EvolutionMail data.
>
> Beagle has to work on a variety of data sources; unfortunately, these
> memory bahevious are absent on the tests we perform on our computers.
> Any help is appreciated.
>
> Ahh one more thing, can you attach the log directory ~/.beagle/Log/ as
> tar-gz ? If its too large, then attach the files pointed to by current-
> Beagle, current-IndexHelper. Thanks
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

The evoserver not being found could be the problem. I am still waiting for the log file.

When you run all the backends, you still get the memory problem ?

>> When you run all the backends, you still get the memory problem ?

I ran each backend separately and did not encounter the memory problem.

I attached the log file with my last message and am attaching it again with
this one. If it is not getting through for some reason, please let me know.

On 2/28/07, dBera <email address hidden> wrote:
>
> The evoserver not being found could be the problem. I am still waiting
> for the log file.
>
> When you run all the backends, you still get the memory problem ?
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

No, the attachment is somehow getting attached.

Also I meant, if you run
$ beagled
(i.e. enable all backends)
do you still get the memory spike ?
and the same question for
$ beagled --backend Files --backend EvolutionMail --backend EvolutionDataServer --backend Liferea --backend Tomboy

I was trying to attach the file when replying via email. This time I will attach the logs through the web page.

I am trying beagled without arguments now and will get back to you.

Looking through the logs I noticed that in addition to the backends I ran in earlier tests, beagled reports starting the following:

Starting backend: 'GaimLog'
Starting backend: 'IndexingService'
Starting backend: 'Tomboy'
Starting backend: 'Labyrinth'
Starting backend: 'Blam'
Starting backend: 'Liferea'
Starting backend: 'Akregator'
Starting backend: 'KonquerorHistory'
Starting backend: 'KonqBookmark'
Starting backend: 'KNotes'
Starting backend: 'KOrganizer'
Starting backend: 'KAddressBook'
Starting backend: 'Kopete'
Starting backend: 'Konversation'
Starting backend: 'applications'
Starting backend: 'documentation'

beagled, by itself, gives the same memory leak.

On 2/28/07, dBera <email address hidden> wrote:
>
> No, the attachment is somehow getting attached.
>
> Also I meant, if you run
> $ beagled
> (i.e. enable all backends)
> do you still get the memory spike ?
> and the same question for
> $ beagled --backend Files --backend EvolutionMail --backend
> EvolutionDataServer --backend Liferea --backend Tomboy
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

I would have guessed that you will get some memory problem when running with "--backend Files" or one of the Evolution backends. Since that didnot happen, some other backend is causing the problem. As you pointed out, there are a 2 other backends that have some data. Can you also test those 2 ?

$ beagled --backend KonquerorHistory
....
and
$ beagled --backend none --add-static-backend /var/cache/beagle/indexes/applications --add-static-backend /var/cache/beagle/indexes/documentation

Can we confirm its still an issue in the beta?

It APPEARS no longer to be an issue over the last week or so.

I'd feel more comfortable if I can keep an eye on it for another few days
before declaring it a squashed bug.

Thanks.

On 3/26/07, Kevin Kubasik <email address hidden> wrote:
>
> Can we confirm its still an issue in the beta?
>
> ** Changed in: beagle (Ubuntu)
> Status: Unconfirmed => Needs Info
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

I think it's safe to say that this bug can now be closed.

Not sure what fixed it or even when, but the memory leak is gone now. Thanks
for all of your efforts.

On 3/26/07, Kevin Kubasik <email address hidden> wrote:
>
> Can we confirm its still an issue in the beta?
>
> ** Changed in: beagle (Ubuntu)
> Status: Unconfirmed => Needs Info
>
> --
> beagled uses >250 Megs RAM when Idle
> https://launchpad.net/bugs/87198
>

Thanks Jonathan Michaels i have closed the bug.