Defective message stops download

I just installed Pandora (and paid, and am waiting for the registration number) on my wife's work machine. She has a problem that I hope Pandora can solve. She has been using *paid* Yahoo forever, originally with Eudora and more recently Thunderbird. During the Oauth2 fiasco (no warning or instructions given), POP stopped working, so I installed a second IMAP personality. This meant leaving mail on that server forever. I did later get POP to work and Yahoo let it download, but the message count got well over 100k. This made the POP function quite slow. Recently Yahoo changed their IMAP policy so it only shows 10k messages, a few months' worth in this case. The rest are still on Yahoo's server but only visible on the webmail page. So clearly I want to a) go back to POP, and b) delete the really old stuff.

However, Thunderbird can't do that. It no longer talks to Yahoo POP at all! It just goes into a hang, and pretends to be mid-operation for days on end. No download. And that's even when I no longer try to tell it to delete anything. So I figured a fresh new mail client that doesn't have a lot of message IDs to compare would be faster. And since we're old Eudora users, Pandora looks like a great choice.

But running it on that account, it sits for about 20 minutes, then gives an error that a message is defective, with an incorrect message size. And then Pandora STOPS. And it displays some of the oldest messages, about 90% spam to be sure. This is after about 0.5% of messages have been downloaded. And retrying it, still the same result. Going onto the Yahoo web mail and deleting all the messages around the last one downloaded, assuming one of them might be the culprit, doesn't change it. Pandora just stops when it sees a bad message.

Is there some way to get around this, and download all of the messages even if there is a bad one (or more than one) on the server?

Also, if I use IMAP to see the new messages (that is, only the latest 10,000 are visible to IMAP now), can Pandora treat them as permanent downloads, not two-way sync? Right now Yahoo has a REALLY BAD habit of deleting the messages already downloaded, because they're beyond the 10,000 limit. That's why I need POP, which does not let the server touch downloaded messages! Also, when Yahoo thinks mail is spam, and you fix it on the web mail, it goes into the regular inbox so it can be downloaded, but a few minutes later Yahoo re-marks it as spam and removes it from Thunderbird. T'bird really treats IMAP as just a viewport, and Yahoo is not a good enough server for that. Can Pandora used IMAP the way Eudora treated POP back in the Worldmail server days? Thanks!

isdnip wrote:I just installed Pandora (and paid, and am waiting for the registration number) on my wife's work machine. She has a problem that I hope Pandora can solve. She has been using *paid* Yahoo forever, originally with Eudora and more recently Thunderbird. During the Oauth2 fiasco (no warning or instructions given), POP stopped working, so I installed a second IMAP personality. This meant leaving mail on that server forever. I did later get POP to work and Yahoo let it download, but the message count got well over 100k. This made the POP function quite slow. Recently Yahoo changed their IMAP policy so it only shows 10k messages, a few months' worth in this case. The rest are still on Yahoo's server but only visible on the webmail page. So clearly I want to a) go back to POP, and b) delete the really old stuff.

However, Thunderbird can't do that. It no longer talks to Yahoo POP at all! It just goes into a hang, and pretends to be mid-operation for days on end. No download. And that's even when I no longer try to tell it to delete anything. So I figured a fresh new mail client that doesn't have a lot of message IDs to compare would be faster. And since we're old Eudora users, Pandora looks like a great choice.

But running it on that account, it sits for about 20 minutes, then gives an error that a message is defective, with an incorrect message size. And then Pandora STOPS. And it displays some of the oldest messages, about 90% spam to be sure. This is after about 0.5% of messages have been downloaded. And retrying it, still the same result. Going onto the Yahoo web mail and deleting all the messages around the last one downloaded, assuming one of them might be the culprit, doesn't change it. Pandora just stops when it sees a bad message.

Is there some way to get around this, and download all of the messages even if there is a bad one (or more than one) on the server?

Also, if I use IMAP to see the new messages (that is, only the latest 10,000 are visible to IMAP now), can Pandora treat them as permanent downloads, not two-way sync? Right now Yahoo has a REALLY BAD habit of deleting the messages already downloaded, because they're beyond the 10,000 limit. That's why I need POP, which does not let the server touch downloaded messages! Also, when Yahoo thinks mail is spam, and you fix it on the web mail, it goes into the regular inbox so it can be downloaded, but a few minutes later Yahoo re-marks it as spam and removes it from Thunderbird. T'bird really treats IMAP as just a viewport, and Yahoo is not a good enough server for that. Can Pandora used IMAP the way Eudora treated POP back in the Worldmail server days? Thanks!

I do not know of an options that would help in your case, however, here are some suggestions:

• Make sure you have the newest Pandora version installed which currently is v5.4.0.
  
• Wait until you have entered the Registration information before attempting to download messages from Yahoo.
  
• In the Yahoo Personality, in the Checking Mail section, make sure the option Leave mail on server  is not checked.This will cause downloaded messages to be deleted from the mail server when the download is complete.
  
• Pandora has support for OAuth2 Authentication on Yahoo.
  
• If Pandora does give an error message, you need to capture that information for a bug report.
  
• Downloading 100K+ messages is going to take a long time even if nothing goes wrong.
  
• Submit a detailed Bug Report - go to menu item Help, Send Feedback, and Report a Bug.
  

Good luck.

Butch

There are a handful of standalone utilities which will connect to IMAP servers and download mails.  I don't know if any of them would be able to see mails held but somehow hidden by Yahoo.

There's: "IMAPSIZE" - see: https://www.broobles.com/imapsize/
- this is no longer maintained but I've read regular mentions of it on the http://www.emaildiscussions.com/ forum, and it's what I plan to use to retrieve mails from two different providers' servers .. when I get around to it.  I think there's an installer, but you can also just download the program itself and then use a text editor to set up an XML configuration file.  There's uptodate SSL DLLs that you'd need to get from the website to replace those used in the original program.  My notes suggest that you'd need to read the FAQ page to find out things that are otherwise not obvious.

There's "IMAPSYNC" - see: https://imapsync.lamiral.info/
This DOES NOT allow you to move mail from an IMAP server to your own PC, but does allow you to move stuff from one IMAP server to another one (so you could maybe find a better server than the Yahoo one and move stuff to it).  The IMAPSYNC website does mention an "online" service which is able to move mails for you - but that means telling them how to access your mails... which I wouldn't do.

cURL - http://curl.haxx.se/
is a general purpose programming tool - and it can talk to IMAP servers.  But this would only be useful to you if you know how to write programs.

perl (and probably python and lots of other scripting languages) can also talk to IMAP servers ... but again - you'd need to be a competent programmer to write your own code to do it.

There are also some email providers who, if you open an account with them, will offer a service to suck all your email from somewhere else onto their system.  Maybe all of them do this (I don't know).  If all else fails you could email some such companies asking if their mail fetcher will cope with the Yahoo situation.  Fastmail (who handle half of my email - I don't have it all with a single company, just in case) have such a service.  See: https://www.fastmail.help/hc/en-us/articles/1500000277482-Migrate-to-Fastmail-from-Yahoo   but I do not know if that help page is uptodate with the current Yahoo situation.  Also Fastmail is a paid-for service...  On the other hand that does mean they have customer support, though it's quite slow these days (especially if one asks non-trivial things).  The email discussions forum I mentioned earlier has a sub-forum just for fastmail discussions so you can read there about what people think of them as a provider.

Here are basic specs for Yahoo POP:

User Name: [Your full e-mail address]
POP Server: pop.mail.yahoo.com
Authentication: OAuth2
TLS/SSL: Always
Port #: 995

Of course you can use IMAP in POP-fashion too.

I've been able to connect to Yahoo POP (OAuth works), but it doesn't download all of the mail, just a batch, before it stops. (The program is now registered.) It is a slow process and I can watch them in the task window but that eventually stops. And it doesn't always show any more messages, past the spot it previously reached.

Deleting them all from the server would not be good, as that would make the mail inaccessible to everything else, including mobile devices, which need it! I do want to delete from server anything over say 200 days old, but we're nowhere near that point yet. Yahoo IMAP does see the messages. I tried in T'bird to set up a filter to take "all" new messages from IMAP and copy them to the POP folder, where they couldn't be deleted. For some reason that doesn't seem to get all messages, though. Just some. Weird.

> Of course you can use IMAP in POP-fashion too.

Does Pandora do that? I mean download mail, full messages, using IMAP, and leaving them alone when they're deleted from the server. It would be nice. Most IMAP implementations, including T'bird, think that it's just a viewport on a server, so you need to leave everything on the server. Fine for a corporate system where I.T. needs to see all of your mail, but not good at all for a public ISP.

isdnip wrote:I've been able to connect to Yahoo POP (OAuth works), but it doesn't download all of the mail, just a batch, before it stops. (The program is now registered.) It is a slow process and I can watch them in the task window but that eventually stops. And it doesn't always show any more messages, past the spot it previously reached.

Deleting them all from the server would not be good, as that would make the mail inaccessible to everything else, including mobile devices, which need it! I do want to delete from server anything over say 200 days old, but we're nowhere near that point yet. Yahoo IMAP does see the messages. I tried in T'bird to set up a filter to take "all" new messages from IMAP and copy them to the POP folder, where they couldn't be deleted. For some reason that doesn't seem to get all messages, though. Just some. Weird.

>   Of course you can use IMAP in POP-fashion too.

Does Pandora do that? I mean download mail, full messages, using IMAP, and leaving them alone when they're deleted from the server.  It would be nice. Most IMAP implementations, including T'bird, think that it's just a viewport on a server, so you need to leave everything on the server. Fine for a corporate system where I.T. needs to see all of your mail, but not good at all for a public ISP.

I think 100K+ messages awaiting download by POP is going to cause problems.  POP is not designed for that many messages because the file is flat and must read sequentially to find the messages for download.  Reading such a big file sequentially is very slow.

Butch

isdnip wrote:
>   Of course you can use IMAP in POP-fashion too.

Does Pandora do that? I mean download mail, full messages, using IMAP, and leaving them alone when they're deleted from the server.  It would be nice. Most IMAP implementations, including T'bird, think that it's just a viewport on a server, so you need to leave everything on the server. Fine for a corporate system where I.T. needs to see all of your mail, but not good at all for a public ISP.

You can download messages from IMAP servers, in the same way you download messages from POP servers. That includes local storage, deletion from servers etc.
In fact you'll be using the exact same interface, with some minor differences (where POP-specific and IMAP-specific options are grayed out).

I always eyeball all incoming messages using MailWasher Pro, in shich I've set up filters to automatically move unwanted messages to the MailWasher cycle bin, and in some cases I manually send other messages to the bin. Once that 's done, I switch to Pandora, click the Check Mail button, and watch the Tasks panel -- which I've configured to pop up upon each mail check -- in order to to follow what's happening to incoming messages (parsing. filtering, etc).

I've been using Pandora for just over two years now, and have found "recently" (perhaps since Pandora 4.0 but it might be earlier) that several times a week Pandora locks up, see screenshot (A). Pandora then needs to be killed via the Windows task manager.



There seems to be a commmon pattern here, and I suspect that Pandora needs to be tweaked (if possible) to detect and handle such freezes. Certainly it should not be necessary to entirely kill Pandora via Task Manager.

I am gradually gathering the messages from the Yahoo server, after having deleted the oldest ones (more than a year and a half or so). There are two glitches. One is that the downloads always stop after a while, maybe a few thousand messages at best, typically less than a months' worth. Sometimes but not always after throwing an error box about a defective message. So it's download-and-repeat. And since messages don't display until the process is over, you don't see how far it's getting until it's done. I wonder if there's some cap in downloads, perhaps because it stores them temporarily somewhere before appending them to the open Inbox window. I wish that were done incrementally.

The other glitch is spam filtering. As a new install, it's not trained, and it puts roughly half the messages, indiscriminately, into the spam folder. In fact the "ham" is more likely to be there than the spam. I finally figured out to turn off the move-spam button but I think it still did that anyway. And since there are now tens of thousands of messages in the spam folder, moving them into the inbox is really slow -- more than a couple hundred at a time turns the Pandora window black and sometimes crashes it.

This does point to a sort of generic issue with email. With POP, the expectation is that the server doesn't keep that much on it at a time. With IMAP, the expectation is that the server keeps everything forever. So when moving from IMAP back to POP (having switched when we couldn't get Yahoo POP to work), hilarity ensues.

I may yet get there but I'm not sure.