Move is Copy + Thrash, should be Move that removes all traces of the first.

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please answer these questions:

 * Is this reproducible?
 * If so, what specific steps should we take to recreate this bug?

 This will help us to find and resolve the problem.

yes - Sebastian - it is consistent.
You can make another folder than Inbox, "touch" and email and try to drop it into the new folder.
What happens is that the email ends up in the new folder - but a copy of it is in the "Deleted Items" /"Trash" folder.
This means that a duplicate has been created.

Then make a Message rule that moves messages from e.g. a named sender to the new folder - and the result will be the same on receipt: one copy in the new folder and a duplicate in Trash.

Related to priority - i have a long experience in database development - and can tell you that the bug is fundamental, and is consequential to a number of other bugs reported.
It renders all "Message rules" useless - because it results in messages being deleted.
It is critical, because nobody can use an email client that causes messages to be lost.

Make a version that processes one email account at a time - avoiding multiple streams allocate new message indexes. Then when you have verified that the message has been moved properly, delete the content by releasing the message index of the original. If the two does not compare after "move" - then delete the new index (best also notify that the move failed for some reason) and release the message index to the new - and hold on to the old index.

What happens now is that the message index of some other stream uses the same message index, causing the emails to get lost - or "deleted".

It is not a desktop bug - it is a bug in the core system.

You have to consider concurrency and you cannot make duplicates to delete them later.

the description is confusing, that way to move is a standard imap one and if you disagree you should open a bug on bugzilla.gnome.org to discuss it with the people writting the software

I have no clue to your background - not have you any clue to mine.
I have led Unix ports for a computer company and been deputy director of
R&D in the same. I have lead knowsoft systems such as Oracle ond also
contributed to Sybase. I have managed huge projects, most software.

The bug is obcious for all variants of "Move Object" - C- library:
Move(a,d,l) { char *a,*d;int l;
            memcpy(a,d,l);
            if (memcmp(a,d,l)==0) {
                    memset (a,sizeof(a),0); return OK;
            } else {
                    memset(d,sizeof(d),0);; return ERROR;
            }
}

The context is more complex, since you have to make room for
objects/emails as this goes on.You have to allocate slots first for the
incoming mail - and map this to the "Inbox". Then say a rule says that
the message is to be placed in another folder F. So the email client
will alocate a new memory slot in Inbox for the email. I would keep this
as an array of pointers to objects - and have the same array of pointers
for every "mailbox".

First the Move from Inbox to F is a copy operation - holding on to the
index in Inbox. But if a new message comes in from another stream, the
code has to decide if the slot "i" in Inbox can be used - or if this is
the "next" email - "i+1". Say that it allocates the next slot to
position i+1, and the order of things is maintained as a separate list.
Should the code re-use "i" - the "move" has to be successful or the
email is lost.

Regardless - no "Move" is performed - just copy, but where the
successful "copy" ends the completed object in Trash. Then what is te
first "Free" index in Inbox? If the "Copy" is ongoing - what is the
"Next" free slot in the Inbox?

When you make a rule e.g. "where Sender with this email address, the
email should be moved to Folder" . there is not "Delete" in the
semantics - the code may skip storage in "Inbox" - and store directly in
"Folder".

When you touch a file, press shift and drags it to another directory,
there is no file placed in a Trash directory. Similar, when a rule exist
that says that an email is to be delivered in another folder, the
"Inbox" may be replaced with another destination for first storage.

If you make a duplicate by "Copy" - you have one slot too much, and I
consider it likely that the wrong copy is deleted.When I then find error
reports of lost messages, I considered this as the most likely
inconsistency - where the wrong copy is deleted. The other error is that
the code allow multiple streams - and thus that another email can ask
for a new slot in the "Inbox" while this creation of duplicates is done
- then which copy is the correct. If you want to compare to make certain
that the "copy" was successful is impossible to conduct, since another
thread may already be using the source memory slots.

The code is usually very simple, and a simple directive as "Move" -
should be very simple and verified over and over again.It is a core
functionality, it is consistent in use and can be reproduced every time
you try to "move" anything.

It is critical, since the error causes mail to be lost.

I cannot waste time on software like this and consider replacing
"Evolution".

Kind regar...

changing back to NEW, to pick by somebody who has a clear opinion about that and wants to bring it to the sofrware writters rather than the distributor

I installed a complete utdate of Ubuntu and Evolution upgrade to 2.26.1
The bug is very obvious and consistent still.

On Fri, 2009-04-24 at 22:46 +0000, Jamie Strandboge wrote:

> ** Visibility changed to: Public

There is nothing wrong with Evolution. The behaviour is expected in an imap mail client. The "Trash" folder isn't actually a folder, but simply lists email that has the "deleted" flag set. When you move an email from Folder A to Folder B, a copy is made into Folder B and, if successful, the original mail gets the "deleted" flag set. In order to permanently erase the original message, an expunge command needs to be issued with Folder/Expunge. See RFC 3501.

In any case, this is not a security issue.

Thanks for taking the time to report this bug and helping to make Ubuntu better. We appreciate the difficulties you are facing, but this appears to be a "regular" (non-security) bug. I have unmarked it as a security issue since this bug does not show evidence of allowing attackers to cross privilege boundaries nor directly cause loss of data/privacy. Please feel free to report any other bugs you may find.

Your interpretation of the named RFC and software engineering leaves a
lot to be appreciated related to semantics.
As you will see from several other bugs, "MOVE" is not the same as
"COPY" + "DELETE_ORIGINAL".
"MOVE" is similar to "RENAME" - no new instance is to be made.
In a object oriented explanation, its an "attach" with a "detach".
With the DELETE flag set, this allows another incoming email to use the
"Trashed" slot - since the code assumes that the copy by then is in the
Trash-bin.

In any system, "moving" an object means that the number of objects are
to remain the same before and after the operation. The "cardinality" is
to be consistent, and you cannot populate with repeated "Move"
operations. Should this be unclear in the refered Request for Comment,
include this comment and change the response. This is just like your car
- if your drive (move it), you do not get a new copy created in a local
trash dump that you can collect later should you regret the trip. You
still only have one car, and moving home will not create a third car
(nor object nor email). If the Linux community is of a different
opinion, wake up to the real world.

The bug made me decide to remove Evolution from the system, because it
introduce a high risk of loosing emails which renders the software
useless.

Bottom line is that I do not care about classification - and caution
others from using Evolution.

On Sun, 2009-04-26 at 23:23 +0000, Marc Deslauriers wrote:

> There is nothing wrong with Evolution. The behaviour is expected in an
> imap mail client. The "Trash" folder isn't actually a folder, but simply
> lists email that has the "deleted" flag set. When you move an email from
> Folder A to Folder B, a copy is made into Folder B and, if successful,
> the original mail gets the "deleted" flag set. In order to permanently
> erase the original message, an expunge command needs to be issued with
> Folder/Expunge. See RFC 3501.
>
> In any case, this is not a security issue.
>
>
> ** This bug is no longer flagged as a security vulnerability

there is no reason why copy + flag should destroy any data since it will flag only after a working copy

there's an upstream bug with similar issues at http://bugzilla.gnome.org/show_bug.cgi?id=329946 ; may you please comment/discuss there where the software authors can read about it? Thanks in advance.

Upstream bug has been marked as duplicate of bug https://bugzilla.gnome.org/show_bug.cgi?id=206061

1. The error is GENERAL.
    Every user of Evolution suffer from this error. Just look at the chat's
2,Importance is HIGH
   The error causes incosistency in cardinality in the indexes used for later searches. You will skip object for search.
    In databases the error is common, and a disaster. That you rate this low just exhibit gross ignorance on algorithms for
    search and retrieval

3. I am not paid a penny, nor charge a penny for this - but to be blunt, had you attended introductory classes in computer
    programming and said the above - I would have flunked you. As consultant you would have been reassigned to MS
    Excel spreadsheet debugging.

But - as you see - it is easy to avoid - just do not use Evolution, move on the Thunderbird.
So - why not drop Evolution, let Novell debug it themselves, and drop it from the support base of Ubuntu.

I'm loathe to get involved here since Knut seems intent on flaming and not so interested in understanding, but reading the comments here I've been struck by a great attention to irrelevant details and not much clear description of the real issue.

Knut continually makes reference to a "MOVE" operation, as if such a thing existed. But it doesn't. If it did, you can be sure that Evolution would use it.

Why can't Evolution create that operation? Because Evolution is a mail CLIENT. It talks to mail SERVERS. Evolution does not implement and cannot influence the servers. The only operations that the client can perform are those that the server supports. The servers implement the IMAP standard, and the IMAP standard does not define a MOVE operation. So, no IMAP server supports a MOVE operation, and so no IMAP client can use a MOVE operation.

So good, there's no MOVE operation. The only way an IMAP client can effect a move from one folder to another is by first doing a COPY, then marking the original message as deleted, and this is what Evolution and ALL OTHER IMAP CLIENTS (yes, including Thunderbird) do. That by itself is enough to blunt some of Knut's criticisms, since no client will ever be able to avoid the creation of a new message via the COPY command.

Similarly, the IMAP standard does not provide any way to delete a single message. The ONLY way to actually cause messages to disappear out of an IMAP folder is by an EXPUNGE, which takes all the messages that have been marked as deleted and removes them (not just one, but ALL of them). This is a very expensive operation, typically, for your server (as opposed to marking a message as deleted which is typically very fast). After the expunge is done, then often the client has to re-read and re-index the messages, because its local cache of which messages are marked deleted and which aren't might be wrong. The whole thing is very labor intensive on both the client and server, as well as the network.

Evolution works the way that the IMAP standard writers intended clients to work: it hides deleted messages and leaves the decision about when to expunge them up to the user.

The problem is that many other clients don't work like this. They don't hide the deleted messages, which leads to lots of confusion about duplicated messages, and they also expunge messages "for" you. For example other IMAP clients implement the semantics of MOVE by doing a COPY, marking the original as deleted, then running EXPUNGE. For every single email you delete. This is a big load on both clients and servers, AND it means that your email that you might have deleted but not wanted expunged yet, is deleted for you.

This is a shame because if all clients worked the way the standard expected them to then they'd work together well. However, just like Evolution cannot change the way servers work, so it cannot change the way other clients work. So, or course Evolution should have an option to behave the way other clients do and have a physical trash folder and implement the COPY/delete/EXPUNGE semantics that other clients favor. You won't need this if you only ever use Evolution as a client but if you switch ...

  I will refrain from commenting on your skills.

For move: Please study available literature - such as SVID / POSIX
"Memory()". These documents are integral to the Linux definition.
Evolution is an Email client according to the SMTP protocol - and not
the IMAP. IMAP is another functionality.

If IMAP cannot do a MOVE without a heavy load on both client and server,
then the server side needs to be modified. If they do not want to do so
because it is a heavy load - it is like saying "I do not want to cross
the road, so the store has to move to my side" - let them just say so,
and produce more unreliable software. If the specification is the source
of the excessive load - then change the specifications so that they make
sense. Like you you may use a "memcpy(s.d.n)" with a subsequent
"memcmp(s.d)" instead of a simple "memmov(s,d,n)" - everything is in the
same address space (so where is the "client" and "server"?). SVID never
refers to "Expunge" not does Posix. So, get on and make the software
instead of arguing.

Regarding searches - you can rest very assured that as long as the
software does not know where to search and when to stop it, objects will
not be found. Please study chat's. If you do not know what "cardinality"
is, go back and study e.g. Don Knuth: "Fundamental Algorithms" ,
"Sorting and Searching". Nothing has changed since he wrote those
volumes. if you need further references about me - call Chris Date at
Stanford.

I have the succesful implementation of indexing in 5 industry standard
databases underneath my belt. You have done none, and claim experience.
If you want experience, get on doing it. When you outperform Oracle, MS
Sqlserver and Sybase and a couple of others - you can claim my
experience, until then, you listen and try to learn.

Unfortunately a company has introduced a generation of so called "knows
how to do it" that can receive a a certificate for the skills in
exchange for a small sum.

It all boils down to so called self-acclaimed "specialists" that claims
to know things and have read a pile of document and never coded a system
in their life. Please stay away, if the software is to be usable by
others, errors needs to be fixed, and not argued.

To a system, a bug is CRITICAL if it makes everyone leave the software
and use another package. How would Oracle survive if it contained a bug
that required everyone to use Sybase? The move is very easy from
Evolution to Thunderbird - so just drop Evolution in Ubuntu and use
Thunderbird with Lightening instead. Thuunderbird can implement Move
Copy, Expunge - and a lot of other things, so why waste your breath
arguing? RIP Evolution - your early demise was caused by because of
silliness and incompetence. What might have been, will never be.

Knut H. flottorp

On to-16.09.2010 01:39, Paul Smith wrote:
> I'm loathe to get involved here since Knut seems intent on flaming and
> not so interested in understanding, but reading the comments here I've
> been struck by a great attention to irrelevant details and not much
> clear description of the real issue.
>
> Knut continually makes reference to a "MOVE" operation, as if such a
> thing existed. B...

Ayeyiyi. Knut, please stop trying to impress everyone with your credentials and consider that you might actually not know everything about everything quite yet, regardless of how much you know about databases. I've been programming on UNIX since before POSIX even existed and I certainly understand the concepts quite well. I'll post one last message in the hopes that others reading this will understand what's going on, at least.

Since you've done a lot of DB work apparently, consider Evolution as if it were a generic open source SQL client program that can connect to lots of different types of SQL servers: Oracle, Postgres, MySQL, etc. The data is left on the server (for at least some of the protocols) and the client only provides access to the data. The client can only perform the operations that the server makes available: you can't perform Oracle-specific SQL commands on a Postgres server, etc. Suppose this client could also access databases that are local and don't have a server (sqlite databases maybe). That's a good analogy for what Evolution is. You didn't tell us what kind of server you're trying to talk to but people are assuming IMAP so I will too. If you're using something different let us know: that would be very helpful information.

First, SMTP is a mail *transfer* protocol. It defines the network protocol for two mail *servers* (e.g., sendmail etc.) to talk to each other to exchange mail. In SMTP parlance these are known as MTAs, or Mail Transfer Agents. Evolution is NOT an MTA. Evolution is a Mail User Agent (MUA). The SMTP standard has virtually nothing to say about the "protocol" that MUA's use to talk to MTA's. In short, Evolution has virtually nothing to do with with SMTP at all (Evolution has the ability to connect directly to MTA's to _send_ mail, but it cannot, and neither does any other client, use SMTP at all when _reading_ mail, as we are discussing in this bug).

Second, Evolution as an email client actually can use lots of different methods to retrieve email. It can get mail from a POP server, from an IMAP server, from a Microsoft Exchange server, or from a Novell Groupwise server. It can even read from USENET. And, of course, it can read from local mail spool files of various types. One of these types is the traditional UNIX mail spool that sendmail uses, and others are spool files left by other types of local software. Each of these uses a different protocol for getting mail.

Third, Evolution does not implement the server software, at all. If you connect to a POP server then someone else wrote that POP server. It may not even be free software at all; your ISP (that you download your email from) decides all that. Ditto for all the other ways Evolution gets email. Even the local spool files are created by other software such as sendmail, etc., that Evolution has no control over.

Fourth, for a number of those server types, including IMAP, Exchange, Groupwise, etc., Evolution does not manage your email locally. The email is left on the server and Evolution sends commands to the server to ask it to manipulate your email. This is what I mean when I say that the IMAP protocol doesn't support ...

Disregarding the "database" problem, the point of view of the submitters of the two bug that were marked as dupplicate of this one is quiet different. There problem (and mine) is that, whenever a mail is moved along, a copy _appears_ in the trash folder (both with local mail and with IMAP folders).

I understand the technical reason not the delete the message from the server/mbox file immediately. But, as it is a technocal restriction, the user need not be aware of that. Read bug 209214 to see why people dislike this way of functionning.

Adding a mechanism (on evolution's side) which hide these message from the trash folder until it is really expunged. Moreover, I believe that expunging the email folders and emptying the trash folder are two very different thing. One is a detail of the implementation and should be hided from the user, the second is a standard part of user interfaces and should act the same way as it always act (or be called differently). The current situation is very misleading.

But this is more an upstream issue.

  Well, thank you for at least taking the time.

But it is completely wasted. You do not know enough to answer. The mail
service used to be a simple Unix-to-Unix copy-file (UUCP) command. So, a
file was transferred from one Unix server/client to another Unix
server/client. The various "Exchange" servers were made to mimmick what
we invented in DS/ID - and Lotus Notes commercialised. This has nothing
to do with the email client's functionality.

I know of IMAPs shortcomings - and it bothers me that you do not. How
you implement the access to the ".sdb" files that is used by the email
client is beyond my concern, but it is not SQL or not anything related
to databases like Oracle, Ingres, Postres, or MySQL even.

Just that you consider this to be a "client/server" application exhibits
just your level of competence. I have no doubt that you have programmed
many years. I have also terminated employment of consultants that have
programmed for many years when they insist that they know the truth, and
do not want to listen.

The functional deficiency that I reported is where an object being an
email with attachments and all arrives on the SMTP protocol, and what is
done. The FTP and SMTP defines what is supposed to be done - it should
be delivered by the MTA to the MUA - where the MUA is to show it
according to certain conventions. I am not addressing these conventions
- the MTU. The MTU can do a number of things, such as stripping off
attachments, storing journal information, running rules - like as in
Documentum or OpenText's LiveLink.

These applications may have included functionality that duplicates the
message, forward it according to rules and place it in a "workflow"
defined by the EDMS. Evolution is not there, but a simple rule can
integrate with these systems. These systems are "servers" and run in
another address space and can be moved around and run on other computers.

Evolution as a software piece may be designed in modules that interact
using a published set of procedures that are linked in to its address
space. But a library function has nothing to do with "clients" and
"servers" - even on Windows. There are not "lots of different methods"
to retrieve emails, there are just one and variants of this - and this
(POP to POP3) are variants of the "telnet" protocol. IMAP is another
set of standards - but except for origin, is pretty similar.

MS Exchange is a way of storing emails in a central repository, similar
to Lotus Notes. Microsoft has made their definition of how to access
this, but this interface is like the Documentum MTU - an interface to
show the emails that are stored in the MS Exchange.

If you move an email on a server, and this produce duplicates , you file
a bug report to maker of the server. with MS Exchange, you file it with
Microsoft.

In the case where Evolution uses plain vanilla Linux files that
Evolution has made to hold the mail objects that are received - the
error goes to Evolution. If the Documentum server also has to move it,
and duplicates this effort, it is none of your concern. If MS Exchange
hates this - it is none of your business. What is your business is to
see to...

  Thanks Mathias,

It not only "appears" in the Trash folder, it allocates place somewhere,
and can be "undeleted". Now I wonder where the undeleted email will
appear, since it has no original reference. It is incorrect to place it
in the Intray and then "Move" it to where it belongs - that will produce
another duplicate and a new entrance in the Trash.

Why no tag it as "DELETED" in Trash. Then it will not show, it cannot be
"undeleted" and the number of emails remains the same.

It is a problem on both sides - because the MBOX files gets a wrong
number of messages contained.

If you search, you risk that emails are not found, because the search
engine will look for the "n" emails it has been told is in the file.
The search is not sequential, but will typically follow the received
date - and skip those marked "Deleted". You cannot "grep" the MBOX,
since that search will include the header and (encoded) attachments.

The bug may be a source of a number of other issues.
If you "compact" these - you risk loosing emails.
Since most "Move" will be while running in the rules when the emails are
received, making the inconsistency in the "Intray" only. My guess is
that they regenerate the indexing every time new mail is received to
reclaim space making this a "special" container. However moving from
other folders later on. Consider usage of 3 folders, where a message is
moved by rule from InTray to "To Be done". Then action cause the mail to
be moved to "In Work" and then to "Completed". This will cause 3
entries in Trash. Now "UnDelete" one of these and run the regular rules
on it and move it to "To Be Done" - with a duplicate in "Completed".

"Move" on an IMAP server or any other server is just an update of an
index on the server side.

I wonder how many patches have been done on the "server side"
(MBOX/IMAP) because of this.

-K

On to-16.09.2010 23:41, Mathias Kende wrote:
> Disregarding the "database" problem, the point of view of the submitters
> of the two bug that were marked as dupplicate of this one is quiet
> different. There problem (and mine) is that, whenever a mail is moved
> along, a copy _appears_ in the trash folder (both with local mail and
> with IMAP folders).
>
> I understand the technical reason not the delete the message from the
> server/mbox file immediately. But, as it is a technocal restriction, the
> user need not be aware of that. Read bug 209214 to see why people
> dislike this way of functionning.
>
> Adding a mechanism (on evolution's side) which hide these message from
> the trash folder until it is really expunged. Moreover, I believe that
> expunging the email folders and emptying the trash folder are two very
> different thing. One is a detail of the implementation and should be
> hided from the user, the second is a standard part of user interfaces
> and should act the same way as it always act (or be called differently).
> The current situation is very misleading.
>
> But this is more an upstream issue.
>

Knut: your comments reveal a relatively shallow understanding of the email environment. And they show you haven't understood the real situation with Evolution. As for discussions of how IMAP servers ought to work, you should be filing those bugs with the teams that create the IMAP servers; I'm sure they'll welcome your input.

The comments around moving files into/out of Trash shows you haven't read the various discussions of the problem (even to bugs marked as duplicates in Gnome Bugzilla). There is no "real" Trash folder in Evolution, so there is no move or copy of email from any other folder into Trash. Trash is a virtual folder: it doesn't exist physically, but instead is constructed as a user-interface artifact, containing all the messages that are marked as deleted. Thus, to delete a message (have it appear in Trash) all that has to be done is mark that message as deleted. To undelete a message, you just remove the deleted attribute and it reappears exactly where it was before... because it was never moved anywhere else. There is no copy involved with delete, no extra space consumed, etc. And similarly, there is no copy involved with undelete either.

This is why messages appear in "Trash" when you move them to another folder: as I've described there is no MOVE operation in IMAP, so a MOVE must be implemented as a COPY followed by a delete. When you mark a message deleted that message appears in the Trash folder.

I do understand Mathias's frustration, but unfortunately it's not so simple. Mathias you suggest it's just a matter of Evolution hiding these details, but that cannot be done reliably. There is only one delete flag in IMAP, so there is no way to make a distinction on the server between messages that are deleted by hand, and those that are deleted as a side-effect of a desired move operation. They are identical as far as any IMAP client is concerned.

Yes, of course, Evolution could maintain its own local list of messages that should be hidden rather than displayed in Trash because they are a side-effect of a copy/delete. However this list would be present only in that instance of Evolution. If you ever attached to your IMAP server using a different Evolution (say on a laptop or work system), or you used a webmail client or another IMAP client (maybe on your smart phone, etc.), all those extra deleted messages WOULD appear there, and you'd have even more confusion and bugs reported.

  Dear Peter Smith,

I have managed the implementation os several email clients with a larger
distribution than "Evolution".
You have some serious misunderstandings to cope with.
How "Trash" is implemented in "Evolution" is not my concern. If this is
a virtual folder, the bug is even worse.
Just implement a bit that flags that the message has been deleted and is
not to be shown, and attach it to the "freelist" of spare slots in the
MBOX. That will DELETE it. Just never show it as "recoverable".

IMAP - do not care. If they do not have a MOVE it is their problem - and
as you say you have to write a dummy "MOVE" that makes a COPY+ERASE+
EXPUNGE. Report this problems with MS Exchange to Microsoft. Attach a
Mp3 dictation of the bug you find in MS Exchange - it makes it easier
for them to read the message. Writing an IMAP server will teach you a
lesson or 10 about that side of the fence. It is very easy with a RDBMS
to implement it.

If you cannot maintain conceptual clarity on a simple software module as
Evolution, you have a huge problem.

If it cannot be done this simple, then scratch the entire Evolution
project - it is obviously not in a state that is manageable. it is
simpler to start all over without a history of patches to trace.

No - I report a problem, I get angered by ignorant response, I find in
the Bugzilla several errors that are indicative to not knowing the right
number of objects held, and I get fed up by silly responses.

Regarding your vision of the future bugs with e.g. mobile phones
attached, rest assured I know vastly more about this than you. I use
Thunderbird, and have no problems what so ever, and recommend to those
left using the Evolution spaghetti, move toi this and regain control of
your Inbox - and allow synchronisation and other functionality. And Paul
Smith is not around to waste your time.

On ma-20.09.2010 07:02, Paul Smith wrote:
> Knut: your comments reveal a relatively shallow understanding of the
> email environment. And they show you haven't understood the real
> situation with Evolution. As for discussions of how IMAP servers ought
> to work, you should be filing those bugs with the teams that create the
> IMAP servers; I'm sure they'll welcome your input.
>
> The comments around moving files into/out of Trash shows you haven't
> read the various discussions of the problem (even to bugs marked as
> duplicates in Gnome Bugzilla). There is no "real" Trash folder in
> Evolution, so there is no move or copy of email from any other folder
> into Trash. Trash is a virtual folder: it doesn't exist physically, but
> instead is constructed as a user-interface artifact, containing all the
> messages that are marked as deleted. Thus, to delete a message (have it
> appear in Trash) all that has to be done is mark that message as
> deleted. To undelete a message, you just remove the deleted attribute
> and it reappears exactly where it was before... because it was never
> moved anywhere else. There is no copy involved with delete, no extra
> space consumed, etc. And similarly, there is no copy involved with
> undelete either.
>
> This is why messages appear in "Trash" when you move them to an...

I agree with Knut. If the concept of Move to Another Folder is so hard to grasp that you think it is okay to allow a valuable email to be trashed instead (and possibly deleted forever with stuff you deem to be trash), then it seems you are impossible to convince.

I have had such a problem that I can no longer move my emails with any confidence and I am considering looking for alternatives to using Evolution. I'm going to have a good look at Thunderbird (I even like the name better!).

Developers need to look at issues from the users' side, not the semantics of the development team's internal language.

Tonight, I lost an important email, and this is now one too many.

  In a way: Good to hear, because if they loose emails ("objects" in my
terminology) its because they do not know how many "email objects" that
the containers (e.g. files) hold. A "inconsistency" is then that the
number of actual emails in a folder, including deleted ones, then
subtract the number of deleted email, differs from the number of emails
displayed.

The problem is also that they have split the user interface from the
storage "module". Had these been the same, a simple "recover" command on
a folder would allow them to list all emails marked as "deleted", and
allow you to "recover" them - until you "compact" them by copy'ing all
non-deleted email-object, according to default display order to a new
container/file - and then permanently delete the old container / file.

I assure you that qualified developers understand this and can take
input from the end user, and it is correct as you say, it is what we see
that counts - not what a committee of varying skills comes to as "correct".

Now I challenge the developers of both Evolution and Thunderbird (and
Seamonkey) to provide a email client with an SQL database that supports
it - starting with MySQL. This will open for easy extension of the email
client to document management (EDMS), workflow, CRM and regular legacy
systems - SAP and Oracle. Norsk Data made that in 1985 - known as
"NOTIS-ID". What they then need is a simple 3GL to access the database,
add fields and make queries, and suddenly, the emails can flow between
applications.

An other alternative to Evolution is Opera - from www.opera.com. This is
free, contains an email client as part of the browser - and more
history, was developed by the same community as NOTIS-ID. Opera mail has
everything Evolution has and a few more gadgets, and moves emails
safely. Since opera supports "Java widgets" it is easier to extend
functionality without even leaving the browser.

On ti-12.10.2010 13:00, Trevor Roberts wrote:
> I agree with Knut. If the concept of Move to Another Folder is so hard
> to grasp that you think it is okay to allow a valuable email to be
> trashed instead (and possibly deleted forever with stuff you deem to be
> trash), then it seems you are impossible to convince.
>
> I have had such a problem that I can no longer move my emails with any
> confidence and I am considering looking for alternatives to using
> Evolution. I'm going to have a good look at Thunderbird (I even like
> the name better!).
>
> Developers need to look at issues from the users' side, not the
> semantics of the development team's internal language.
>
> Tonight, I lost an important email, and this is now one too many.
>

It would be good if people could be explicit about what version of Evolution they're using AND (more importantly) what type of email server they're connecting to. Evolution has to interact with them differently (because the services have different capabilities). Are you using locally-delivered email to a local mailbox, as Knut seems to be? Are you connecting to an ISP mail provider using POP? Using IMAP? Are you connecting to Google mail? Are you connecting to an Exchange server? Something else? In short, please go to your email account (Edit->Preferences->Mail Accounts) and click the Recieving Mail tab and tell use what the "Server Type" setting is.

I'm only speaking about connecting to IMAP servers here; I'm also familiar with Evolution connecting to Exchange servers but that works very differently. I've not used local mail delivery or POP (which work similarly to each other) in many years.

In IMAP, there is no way that the current behavior of Evolution will lose mail in the way Knut suggests. A "move" operation is implemented as a copy and after the copy succeeds, the original is marked as deleted by setting a flag. No mail is actually removed from any folder during this operation and so no mail can be lost. In fact that is what typically generates complaints: that your trash folder has EXTRA mail in it that you didn't expect. Sometime later you will initiate an Expunge operation and that will go through your folders and remove emails that are marked as deleted. This operation is performed by the IMAP server, not Evolution (all Evolution does is send a single "expunge" command) in an atomic way so that it does not cause any mail loss (unless there's a bug in the IMAP server but there's nothing Evolution can do about that, and switching to a different mail client will not help). During Expunge, if you have an email that is marked deleted that you didn't actually want to be deleted then yes, it will be lost. But only mail marked deleted is gone: if you'd copied it to another folder and that copy is not marked for deletion, then the copy is still present.

One way people using IMAP with Evolution can get frustrated is if they tend to store things in their Trash folder; they want to use Trash as a kind of longer-term storage folder. You definitely cannot do that with Evolution; you need to choose a different folder and put things you want to save there.

FYI, Thunderbird connecting to an IMAP server has the same issues that Evolution does: they have to because an IMAP server only publishes a certain interface and all clients have to use the same interface. What Thunderbird does differently is it runs the Expunge operation for you all the time in the background. So, when you "move" mail in Thunderbird it does the same copy and mark as deleted operation that Evolution does, but then it also automatically runs Expunge so that the "extra" deleted copy is gone. When you delete mail in Thunderbird it actually "moves" (copy, mark as deleted, expunge) the email to a real folder (sometimes named Trash). Thus expunging doesn't cause that mail to go away, you have to delete it specifically from the Trash folder.

Trevor: I'm sorry ...

  Both is past tense, and none of us indicated that we interface to a
remote storage. That is your imagination.
What we use is the regular MBOX formatted files - and access the email
as regular POP accounts. You make assumptions, and refuse to count. The
result is that the software you make is not reliable.
Thunderbird does not have this error, nor Opera mail client, so why bother?

On ti-12.10.2010 19:14, Paul Smith wrote:
> It would be good if people could be explicit about what version of
> Evolution they're using AND (more importantly) what type of email server
> they're connecting to. Evolution has to interact with them differently
> (because the services have different capabilities). Are you using
> locally-delivered email to a local mailbox, as Knut seems to be? Are
> you connecting to an ISP mail provider using POP? Using IMAP? Are you
> connecting to Google mail? Are you connecting to an Exchange server?
> Something else? In short, please go to your email account
> (Edit->Preferences->Mail Accounts) and click the Recieving Mail tab and
> tell use what the "Server Type" setting is.
>
> I'm only speaking about connecting to IMAP servers here; I'm also
> familiar with Evolution connecting to Exchange servers but that works
> very differently. I've not used local mail delivery or POP (which work
> similarly to each other) in many years.
>
> In IMAP, there is no way that the current behavior of Evolution will
> lose mail in the way Knut suggests. A "move" operation is implemented
> as a copy and after the copy succeeds, the original is marked as deleted
> by setting a flag. No mail is actually removed from any folder during
> this operation and so no mail can be lost. In fact that is what
> typically generates complaints: that your trash folder has EXTRA mail in
> it that you didn't expect. Sometime later you will initiate an Expunge
> operation and that will go through your folders and remove emails that
> are marked as deleted. This operation is performed by the IMAP server,
> not Evolution (all Evolution does is send a single "expunge" command) in
> an atomic way so that it does not cause any mail loss (unless there's a
> bug in the IMAP server but there's nothing Evolution can do about that,
> and switching to a different mail client will not help). During
> Expunge, if you have an email that is marked deleted that you didn't
> actually want to be deleted then yes, it will be lost. But only mail
> marked deleted is gone: if you'd copied it to another folder and that
> copy is not marked for deletion, then the copy is still present.
>
> One way people using IMAP with Evolution can get frustrated is if they
> tend to store things in their Trash folder; they want to use Trash as a
> kind of longer-term storage folder. You definitely cannot do that with
> Evolution; you need to choose a different folder and put things you want
> to save there.
>
> FYI, Thunderbird connecting to an IMAP server has the same issues that
> Evolution does: they have to because an IMAP server only publishes a
> certain interface and all clients have to use the same interface. What
> Thunderbird does differently is it runs the Expunge operati...

Paul Smith : The problem exists with any version of evolution and I see it with both IMAP and POP server.

My problem is really not the reliability of the software, but its usability. E.g. I'm using evolution and, I suddenly press the DEL key (accidentally). And I don't which exact mail has been deleted.
That is not a problem, I have used Outlook for many years or any Desktop system and I know that the trash will contains the usual trash and just one file (i.e. one mail) that I did not delete on purpose. That is the one I want to restore.

But, with evolution (and maybe other software,, but not all IMAP clients), this won't work because this email is lost among a huge number of email that are not trash but just email that I have moved. So I can't find that email in the trash (it is there, but I can't find it).

Maybe Evolution should just use a real Trash folder that the user fill and empty manually (this is done by some IMAP client). Message deleted by the user would be _moved_ to this folder. And the content of this folder would be distinct from the list of non-expunged-email (which is a virtual folder). The IMAP DELETE operation and the delete operation performed by an user are two very different things. I consider it a bug that evolution manage them as the same thing.

  That is the way it works in Thunderbird and Opera Mail and all other
mail systems I know of.
DELETE is to MOVE to the Trash folder, and "ctrl+z" / "Undo" will MOVE
it back.
MOVE is to copy the content from source to destination, and if
successful, delete the source, if failure, remove all traces of the
destination and report the error. Move does not use the Trash - that is
an error .a.k.a. "Bug".

Thunderbird have a special command "Empty Trash" and the "Compact
Folder" to recover space held by deleted messages.
Observe that the command "Expunge" is avoided since this should be done
automatically by underlying software.

In Thunderbird there is an option allowing the "Trash" folder to be
automatically emptied on exit.

They should make a mail storage based on SQL that allows storage in a
regular SQL database, starting with MySQL.
Then they can implement their IMAP interface on this - and drop the MBOX
file system, and maintain only one source for the user interface. Using
a RDBMS to hold the emails removes the need for "Trash", since the RDBMS
is to provide a "Rollback" function limited to the size of the logs
configured.

On ti-12.10.2010 21:42, Mathias Kende wrote:
> Paul Smith : The problem exists with any version of evolution and I see
> it with both IMAP and POP server.
>
> My problem is really not the reliability of the software, but its usability. E.g. I'm using evolution and, I suddenly press the DEL key (accidentally). And I don't which exact mail has been deleted.
> That is not a problem, I have used Outlook for many years or any Desktop system and I know that the trash will contains the usual trash and just one file (i.e. one mail) that I did not delete on purpose. That is the one I want to restore.
>
> But, with evolution (and maybe other software,, but not all IMAP
> clients), this won't work because this email is lost among a huge number
> of email that are not trash but just email that I have moved. So I can't
> find that email in the trash (it is there, but I can't find it).
>
> Maybe Evolution should just use a real Trash folder that the user fill
> and empty manually (this is done by some IMAP client). Message deleted
> by the user would be _moved_ to this folder. And the content of this
> folder would be distinct from the list of non-expunged-email (which is a
> virtual folder). The IMAP DELETE operation and the delete operation
> performed by an user are two very different things. I consider it a bug
> that evolution manage them as the same thing.
>

Paul, I am using Evolution 2.28.3.

I am receiving my mail from a remote ISP via POP, and mail arrives on my local drive - there is no local server involved. The Preferences window does not provide any tab for incoming mail - it just shows my one POP account.

Your assertion that there is now way that email will be lost is incorrect. It happens - that is why we are frustrated. If it did not happen, why would we spend so much time chasing a myth when life outside the office beckons?

Sometimes the move to another folder works (most of the time it does) and sometimes it does not. When it does not, the problem seems to stick to the email object, as it refuses to handle the email object properly regardless of the number of tries. Typically, I end up with multiple copies of the email object in the Inbox or Trash or both, and still none in the destination folder.

Another thing that happens is that one cannot move an email object out of Junk directly into another folder - the object has to be moved to the Inbox first (this works okay) and then moved to the destination folder. As mentioned above, this will work most of the time but not for some objects. There is nothing outwardly different about the objects that are not being handled properly.

Hope this helps.

This bug was fixed in the package evolution - 2.32.0-0ubuntu1

---------------
evolution (2.32.0-0ubuntu1) natty; urgency=low

  * Upstream release 2.32.0
    - Empty reply quotation for HTML messages (LP: #630566, #659513)
    - "Encrypt to self" by default on newly created mail (LP: #326979)
    - Backup settings uses unhelpful yes/no dialog (LP: #572985)
    - Can't drag email addresses to Contact List Editor (LP: #282530)
    - Contact List Editor calls wrong EDestination function (LP: #229187)
    - Allow normal, non-vFolder, Trash and Junk folder (LP: #13983, #64762)
      (LP: #135485, #280325, #365270)
    - Allow change of signature hash algorithm (LP: #381290, #381295, #635937)
    - Dialog for mark-all-read always mentions subfolders (LP: #608462)
    - Evolution allows deletion of default views (LP: #498040)
    - Add checks for event->comp_data != NULL (LP: #466415, #546952)
    - Properly free unused message infos periodically (LP: #507972)
    - Calendar compressed weekend print improvement (LP: #88926)
    - Attachment bar causes drawing issues in RTL locales (LP: #545459)
    - [PST] evolution crashed with SIGSEGV (LP: #471852)
    - Swap "Save" and "Save as Draft" accelerators in composer (LP: #424416)
    - Evolution hangs when formatting message - fixes part of it (LP: #175233)
      (LP: #327775)
    - Calendar Day view All Day events print improvements (LP: #88926)
    - Crash on a changed mail filter action removal (LP: #452921)
    - Do not block UI with publish-calendar messages (LP: #594289)
    - Duplicate mnemonic in meeting window (LP: #499418)
    - Hide variable used only with HAVE_LIBNOTIFY (LP: #594289)
  * debian/patches/02_fix_missing_include_for_composer.patch: refreshed
  * debian/patches/03_lpi.patch: refreshed
  * debian/patches/12_remove_not_recommended_for_top_posting.patch: updated.
    The "recommended" text is now a separate label, so removing that object
    entirely.
  * debian/patches/89_express.patch: dropped, applied upstream.
  * debian/patches/90_disable_deprecation_warning.patch: refreshed
  * debian/patches/91_add_u1_email_translations.patch: refreshed
  * debian/patches/91_git_additional_chinese_translations.patch: dropped
  * debian/patches/91_git_fix_e_shell_ref_counting.patch: dropped
  * debian/patches/91_git_fix_untranslatable_list_editor.patch: dropped
  * debian/patches/92_git_fix_proxy_ignore_hosts.patch: dropped
  * debian/patches/92_git_only_hide_signature_on_express.patch: dropped
  * debian/control: update Depends/Build-Depends of all e-d-s parts to 2.32
  * debian/control: libgdata was split out of e-d-s, so updating Build-Depends
    to use the external libgdata library.
  * debian/control: bump libgtkhtml Build-Depends to >= 1:3.31.90
  * debian/*.install: install plugins to /usr/lib/evolution/2.32, not 2.30
  * debian/rules: remove --disable-pilot-conduits, it's not a valid configure
    switch anymore
  * debian/evolution-dev.install: no longer install libeconduit.so
  * debian/control: update Build-Depends with new and updated requirements for
    2.32: adding libpango1.0-dev, libgail-dev, updating libglib2.0-dev,
    libgtk2.0-dev, gnome-icon-theme and libunique-dev
  *...

Who cares? I have moved to Thunderbird a long time ago, cannot use
software with critical errors that is not funny to correct, and those
set to maintain the source spend more time arguing semantics than
getting the bug fixed.
So I see Novell has been acquired, and I hope "Evolution" is taken off
the Ubuntu distribution.

A critical error is an error that renders the system worthless. It is
not related to how interesting it is to fix it.

On ti-23.11.2010 14:40, Launchpad Bug Tracker wrote:
> This bug was fixed in the package evolution - 2.32.0-0ubuntu1
>
> ---------------
> evolution (2.32.0-0ubuntu1) natty;urgency=low
>
> * Upstream release 2.32.0
> - Empty reply quotation for HTML messages (LP: #630566, #659513)
> - "Encrypt to self" by default on newly created mail (LP: #326979)
> - Backup settings uses unhelpful yes/no dialog (LP: #572985)
> - Can't drag email addresses to Contact List Editor (LP: #282530)
> - Contact List Editor calls wrong EDestination function (LP: #229187)
> - Allow normal, non-vFolder, Trash and Junk folder (LP: #13983, #64762)
> (LP: #135485, #280325, #365270)
> - Allow change of signature hash algorithm (LP: #381290, #381295, #635937)
> - Dialog for mark-all-read always mentions subfolders (LP: #608462)
> - Evolution allows deletion of default views (LP: #498040)
> - Add checks for event->comp_data != NULL (LP: #466415, #546952)
> - Properly free unused message infos periodically (LP: #507972)
> - Calendar compressed weekend print improvement (LP: #88926)
> - Attachment bar causes drawing issues in RTL locales (LP: #545459)
> - [PST] evolution crashed with SIGSEGV (LP: #471852)
> - Swap "Save" and "Save as Draft" accelerators in composer (LP: #424416)
> - Evolution hangs when formatting message - fixes part of it (LP: #175233)
> (LP: #327775)
> - Calendar Day view All Day events print improvements (LP: #88926)
> - Crash on a changed mail filter action removal (LP: #452921)
> - Do not block UI with publish-calendar messages (LP: #594289)
> - Duplicate mnemonic in meeting window (LP: #499418)
> - Hide variable used only with HAVE_LIBNOTIFY (LP: #594289)
> * debian/patches/02_fix_missing_include_for_composer.patch: refreshed
> * debian/patches/03_lpi.patch: refreshed
> * debian/patches/12_remove_not_recommended_for_top_posting.patch: updated.
> The "recommended" text is now a separate label, so removing that object
> entirely.
> * debian/patches/89_express.patch: dropped, applied upstream.
> * debian/patches/90_disable_deprecation_warning.patch: refreshed
> * debian/patches/91_add_u1_email_translations.patch: refreshed
> * debian/patches/91_git_additional_chinese_translations.patch: dropped
> * debian/patches/91_git_fix_e_shell_ref_counting.patch: dropped
> * debian/patches/91_git_fix_untranslatable_list_editor.patch: dropped
> * debian/patches/92_git_fix_proxy_ignore_hosts.patch: dropped
> * debian/patches/92_git_only_hide_signature_on_express.patch: dropped
> * debian/control: update Depends/Build-Depends of all e-d-s parts to 2.32
> * debian...