Ubuntu
kdebase package

kwrite crashes when trying to open a 2.8 GB text file

Bug #620789 reported by rusivi2
4
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KDE Base
Won't Fix
Wishlist
kdebase (Ubuntu)
Opinion
Undecided
Unassigned

Bug Description

Binary package hint: kdebase

1) lsb_release -rd
Description: Ubuntu 10.04.1 LTS
Release: 10.04

2) apt-cache policy kwrite
kwrite:
  Installed: 4:4.4.2-0ubuntu2
  Candidate: 4:4.4.2-0ubuntu2
  Version table:
 *** 4:4.4.2-0ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

3) What I expected to happen is that when i try to open a 2.8 GB text file, it opens with no problems.

4) What happens instead is that when I attempt to open the 2.8 GB text file with kwrite, my Memory in System Monitor spikes to 95.6%, and my Swap gradually increase until both dramatically decrease, indicating the program crashed. I 12 GB of Swap available.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: kwrite 4:4.4.2-0ubuntu2
ProcVersionSignature: Ubuntu 2.6.32-24.39-generic 2.6.32.15+drm33.5
Uname: Linux 2.6.32-24-generic i686
Architecture: i386
Date: Thu Aug 19 19:41:52 2010
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release i386 (20100429)
ProcEnviron:
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: kdebase

Revision history for this message
In , Matthias-pospiech (matthias-pospiech) wrote :

Version: 4.2.4 (KDE 4.2.4) "release 2" (using 4.2.4 (KDE 4.2.4) "release 2", KDE:42 / openSUSE_11.0)
Compiler: gcc
OS: Linux (i686) release 2.6.25.20-0.4-pae

I have a 200 MB large sql text file. Loading it with kde3/kwrite takes a few seconds and works after that without a problem. Loading it with kde4/kwrite takes ages, causes 100% CPU and at some time is shows the first lines. If I try to edit or move anything the whole application freezes.

Revision history for this message
In , FiNeX (finex) wrote :

This is simlar to bug #145686

Revision history for this message
In , Milian Wolff (milianw) wrote :

I remember that I once pushed a change upstream which added a few new features to the MySQL highlighting file. I think I used a few RegExpes which could probably lead to the decrease in performance.

I'll have a look at it eventually.

Revision history for this message
In , Cullmann-t (cullmann-t) wrote :

*** Bug 192495 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Dhaumann (dhaumann) wrote :

@Matthias: Can you update to KDE 4.4 and then compile the development version from gitorious according to this howto:
http://gitorious.org/kate/pages/Building%20Kate

The file buffer has been rewritten completely and as Milian said there were some speedups in the highlighting. It should work again... Can you confirm?

Revision history for this message
rusivi2 (rusivi2-deactivatedaccount) wrote :
Bug Watch Updater (bug-watch-updater)
Changed in kdebase:
status: Unknown → In Progress
Revision history for this message
In , rusivi2 (rusivi2-deactivatedaccount) wrote :

This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It crashes everytime I try to open the file.

lsb_release -rd
Description: Ubuntu 10.04.1 LTS
Release: 10.04

apt-cache policy kwrite
kwrite:
  Installed: 4:4.4.2-0ubuntu2
  Candidate: 4:4.4.2-0ubuntu2
  Version table:
 *** 4:4.4.2-0ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

Downstream bug may be found at:
https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789

Revision history for this message
Myriam Schweingruber (myriam) wrote :

This is a joke, no? Do you even have enough RAM to perform this?

I don't see how somebody can have a 2.8 GB text file:

Assuming an average A4 page has roughly 3 kB of data [1], a file of 2.8 GB would then be about 1 Million of A4 pages...

[1] http://www.virtualmv.com/wiki/index.php?title=Data#How_many_bytes_on_an_A4_page.3F

Revision history for this message
In , Myriam Schweingruber (myriam) wrote :

(In reply to comment #5)
> This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It
> crashes everytime I try to open the file.
>
> lsb_release -rd
> Description: Ubuntu 10.04.1 LTS
> Release: 10.04
>
> apt-cache policy kwrite
> kwrite:
> Installed: 4:4.4.2-0ubuntu2
> Candidate: 4:4.4.2-0ubuntu2
> Version table:
> *** 4:4.4.2-0ubuntu2 0
> 500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
> 100 /var/lib/dpkg/status
>
> Downstream bug may be found at:
> https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789

This is a joke, right? Do you even have enough RAM to perform this?

I don't see how somebody can have a 2.8 GB text file:

Assuming an average A4 page has roughly 3 kB of data [1], a file of 2.8 GB would then be about 1 Million of A4 pages...

[1] http://www.virtualmv.com/wiki/index.php?title=Data#How_many_bytes_on_an_A4_page.3F

Revision history for this message
In , Andreas Pakulat (apaku) wrote :

(In reply to comment #6)
> (In reply to comment #5)
> > This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It
> > crashes everytime I try to open the file.
> >
> > lsb_release -rd
> > Description: Ubuntu 10.04.1 LTS
> > Release: 10.04
> >
> > apt-cache policy kwrite
> > kwrite:
> > Installed: 4:4.4.2-0ubuntu2
> > Candidate: 4:4.4.2-0ubuntu2
> > Version table:
> > *** 4:4.4.2-0ubuntu2 0
> > 500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
> > 100 /var/lib/dpkg/status
> >
> > Downstream bug may be found at:
> > https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789
>
> This is a joke, right? Do you even have enough RAM to perform this?

You don't need ten GB of ram to open that, with a text-editor built for being able to handle such large files. The solution is pretty easy: read the file into memory only in chunks.

> I don't see how somebody can have a 2.8 GB text file:

Database dump, Log-Files, text-file with losts binary data attached (I know lots of *nix-installers doing this). There are endless use-cases.

> Assuming an average A4 page has roughly 3 kB of data [1], a file of 2.8 GB
> would then be about 1 Million of A4 pages...

So? Nobody said he's going to print this.

Revision history for this message
Jonathan Thomas (echidnaman) wrote :

Hi there!

Thanks for reporting this bug! Your bug seems to be a problem with the KDE program itself, and not with our KDE packages. But don't worry! This issue is being tracked by the KDE developers at: http://bugs.kde.org/show_bug.cgi?id=203355
Once fixed in KDE, it will be included in Kubuntu once the KDE version the fix is in in reaches Kubuntu.

Thanks!

Changed in kdebase (Ubuntu):
status: New → Invalid
Revision history for this message
rusivi2 (rusivi2-deactivatedaccount) wrote :

Myriam:

Thank you for addressing this bug. My bugs are not jokes, otherwise I would not post them! ;) As well, the amount of RAM I have and the amount of free space I have on my hard disk drive (both more than the size of the file), is not quite the problem.

Jonathan:

Thank you for addressing this bug. When I typed ubuntu-bug kwrite this is the bug that was created. So, until this problem has been resolved upstream, it should remain an open bug. Thanks!

Changed in kdebase (Ubuntu):
status: Invalid → New
Revision history for this message
Harald Sitter (apachelogger) wrote :

Aloha!

As per policy bugs that are within upstream's code are to be resolved upstream and by the usual means to be included into Kubuntu, and since tracking them in launchpad and at bugs.kde.org does not have any advantage the launchpad reports ought to be claused in order to keep launchpad clean and help developers find issues originating within Kubuntu more easily.

So it is only in your interest if we close the bug report in launchpad, otherwise it would show up in lists that developers use to find new problems that need solving as soon as possible while the issue at hand does not qualify as such. Further more I doubt most of or developers have insight in the inner workings of kwrite or the kate editor part, so they would not be able to do anything about it anyway.

Thanks and have a good day.

Changed in kdebase (Ubuntu):
status: New → Invalid
Revision history for this message
rusivi2 (rusivi2-deactivatedaccount) wrote :

Harold:

Thank you for addressing this bug. I did review the policy before I changed the status: https://wiki.ubuntu.com/Bugs/Status and I was mistaken. Based on this being a confirmed problem upstream, it should have been marked this but policy states don't confirm your own bugs. Due to the status disagreement, I'll place in Opinion, but if you would like to place in Confirm that's great too!

Changed in kdebase (Ubuntu):
status: Invalid → Opinion
Revision history for this message
In , FiNeX (finex) wrote :

I agree with Andreas. It is not rare to find big files nowadays :-)

Usually I use vim/gedit when I've to open big files due to the kate limitations :-(

Revision history for this message
In , Cullmann-t (cullmann-t) wrote :

If somebody implements a lazy loading, will be happy.
But I don't see me doing that atm.
It was once implemented (at least in parts, but still even there everything was first converted to unicode and swapped)

Revision history for this message
In , Devin Cofer (ranguvar) wrote :

Perhaps relevant information
When opening the VMware Workstation install shell script in KWrite (you can also test, it is a free download with (also free) registration), KWrite works fine at first.
This file is several hundred MiB, with a small shell code portion at the beginning followed by raw data.

Scroll down, and once you get to the raw binary part, KWrite locks up, and quickly begins to swallow multiple GiB of memory. It completely bogged down the system, and I had to use kill(8) to stop it.

Bug Watch Updater (bug-watch-updater)
Changed in kdebase:
importance: Unknown → Medium
Revision history for this message
In , Cullmann-t (cullmann-t) wrote :

The workstation installer contains binary data at the end, you can't assume that the qt text layouter will work ok with >> 1MB lines of binary blobs.
To have a better handling of large files is a nice wish, but unlikely to happen anytime soon. Given the average use is for programming, multi-GB files are not that typical.

Revision history for this message
In , Devin Cofer (ranguvar) wrote :

What I would be most interested in learning is why KDE3 can deal with huge text files so much better than KDE4. Surely something can be done?

I agree that multi-GiB files are not of much concern, but text-format databases and the like regularly reach into the dozens or hundreds of megabytes, and should be considered.

Even if it's a solution as clunky as a checkbox upon open, or auto-detect, for "large file mode".

Bug Watch Updater (bug-watch-updater)
Changed in kdebase:
importance: Medium → Wishlist
status: In Progress → Confirmed
Revision history for this message
In , Cullmann-t (cullmann-t) wrote :

Dear user,

this wish list item is now closed, as it wasn't touched in the last two years and no contributor stepped up to implement it.

The Kate/KTextEditor team is very small and we can just try to keep up with fixing bugs. Therefore wishs that show no activity for two years or more will be closed from now on to keep at least a bit overview about 'current' wishs of the users.

If you want your feature to be implemented, please step up to provide some patch for it. If you think it is really needed, you can reopen your request, but keep in mind, if no new good arguments are made and no people get attracted to help out to implement it, it will expire in two years again.

We have a nice website kate-editor.org that provides all the information needed to contribute, please make use of it. For highlighting improvements our user manual shows how to write syntax definition files.

Greetings
Christoph

Bug Watch Updater (bug-watch-updater)
Changed in kde-baseapps:
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.