Debian Bug report logs - #478545
[alpine] alpine does not support large (>2 GiB) mbox's

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Mike Dresser <mdresser_l@windsormachine.com>

To: submit@bugs.debian.org

Subject: [alpine] alpine does not support large (>2 GiB) mbox's

Date: Tue, 29 Apr 2008 13:59:19 -0400

Package: alpine
Severity: normal
Tags: lfs

--- Please enter the report below this line. ---

If you attempt to open a mailbox that is over 2 GiB in size, alpine will 
refuse to open it, stating 'Folder "INBOX" opened with 0 messages'  Pine 
4.64 had the same behaviour previously.

--- System information. ---
Architecture: i386
Kernel: Linux 2.6.25

Debian Release: lenny/sid
500 unstable www.debian-multimedia.org
500 unstable mirrors.kernel.org
500 unstable http.us.debian.org

--- Package information. ---
Depends (Version) | Installed
=====================================-+-====================
libc6 (>= 2.7-1) | 2.7-10
libkrb53 (>= 1.6.dfsg.2) | 1.6.dfsg.3~beta1-4
libldap-2.4-2 (>= 2.4.7) | 2.4.7-6.2
libncurses5 (>= 5.6+20071006-3) | 5.6+20080419-2
libpam0g (>= 0.99.7.1) | 0.99.7.1-6
libssl0.9.8 (>= 0.9.8f-5) | 0.9.8g-8

Message #10 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Asheesh Laroia <asheesh@asheesh.org>

To: alpine-info@u.washington.edu

Cc: 478545@bugs.debian.org

Subject: Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 13:21:35 -0700 (PDT)

I'm forwarding this Debian user's bug report to the alpine-info list.

Is this a known issue in Alpine?  I imagine it is a c-client issue; is it 
a known issue there?

The "LFS" tag in the Debian bug means that this issue relates to large 
file support.

If possible, please reply to all so that the Debian bug can pick up on 
any conversation here.

-- Asheesh.

---------- Forwarded message ----------
Date: Tue, 29 Apr 2008 13:59:19 -0400
From: Mike Dresser <mdresser_l@windsormachine.com>
Resent-From: Mike Dresser <mdresser_l@windsormachine.com>
To: submit@bugs.debian.org
Resent-To: debian-bugs-dist@lists.debian.org
Resent-cc: Asheesh Laroia <asheesh@asheesh.org>
Resent-Date: Tue, 29 Apr 2008 18:03:02 +0000
Resent-Date: Tue, 29 Apr 2008 18:03:06 +0000
Subject: Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's

Package: alpine
Severity: normal
Tags: lfs

--- Please enter the report below this line. ---

If you attempt to open a mailbox that is over 2 GiB in size, alpine will refuse 
to open it, stating 'Folder "INBOX" opened with 0 messages'  Pine 4.64 had the 
same behaviour previously.

--- System information. ---
Architecture: i386
Kernel: Linux 2.6.25

Debian Release: lenny/sid
500 unstable www.debian-multimedia.org
500 unstable mirrors.kernel.org
500 unstable http.us.debian.org

--- Package information. ---
Depends (Version) | Installed
=====================================-+-====================
libc6 (>= 2.7-1) | 2.7-10
libkrb53 (>= 1.6.dfsg.2) | 1.6.dfsg.3~beta1-4
libldap-2.4-2 (>= 2.4.7) | 2.4.7-6.2
libncurses5 (>= 5.6+20071006-3) | 5.6+20080419-2
libpam0g (>= 0.99.7.1) | 0.99.7.1-6
libssl0.9.8 (>= 0.9.8f-5) | 0.9.8g-8

Message #15 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Mark Crispin <MRC@Washington.EDU>

To: Asheesh Laroia <asheesh@asheesh.org>

Cc: alpine-info@u.washington.edu, 478545@bugs.debian.org

Subject: Re: [Alpine-info] Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 13:36:58 -0700 (Pacific Daylight Time)

The report is correct.  The c-client library makes no attempt to use the 
64bit system calls; and thus flat files are limited to 2GB.

The recommended solution for mailboxes with aggregate size greater than 
2GB is to use mix format instead of a flat file format.  Even if c-client 
were updated to use the 64bit system calls, flat files (especially in 
traditional UNIX format) do not perform well at multi-GB sizes.

If there is some compelling reason offered, I will consider 64bit support. 
However, as of right now, I am highly skeptical that the benefit would be 
worth the effort, especially since there is a superior solution for large 
mailboxes available now ("superior" since it will always work better than 
a large flat file).

On Tue, 29 Apr 2008, Asheesh Laroia wrote:
> I'm forwarding this Debian user's bug report to the alpine-info list.
>
> Is this a known issue in Alpine?  I imagine it is a c-client issue; is it a 
> known issue there?
>
> The "LFS" tag in the Debian bug means that this issue relates to large file 
> support.
>
> If possible, please reply to all so that the Debian bug can pick up on any 
> conversation here.
>
> -- Asheesh.
>
> ---------- Forwarded message ----------
> Date: Tue, 29 Apr 2008 13:59:19 -0400
> From: Mike Dresser <mdresser_l@windsormachine.com>
> Resent-From: Mike Dresser <mdresser_l@windsormachine.com>
> To: submit@bugs.debian.org
> Resent-To: debian-bugs-dist@lists.debian.org
> Resent-cc: Asheesh Laroia <asheesh@asheesh.org>
> Resent-Date: Tue, 29 Apr 2008 18:03:02 +0000
> Resent-Date: Tue, 29 Apr 2008 18:03:06 +0000
> Subject: Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's
>
> Package: alpine
> Severity: normal
> Tags: lfs
>
> --- Please enter the report below this line. ---
>
> If you attempt to open a mailbox that is over 2 GiB in size, alpine will 
> refuse to open it, stating 'Folder "INBOX" opened with 0 messages'  Pine 4.64 
> had the same behaviour previously.
>
> --- System information. ---
> Architecture: i386
> Kernel: Linux 2.6.25
>
> Debian Release: lenny/sid
> 500 unstable www.debian-multimedia.org
> 500 unstable mirrors.kernel.org
> 500 unstable http.us.debian.org
>
> --- Package information. ---
> Depends (Version) | Installed
> =====================================-+-====================
> libc6 (>= 2.7-1) | 2.7-10
> libkrb53 (>= 1.6.dfsg.2) | 1.6.dfsg.3~beta1-4
> libldap-2.4-2 (>= 2.4.7) | 2.4.7-6.2
> libncurses5 (>= 5.6+20071006-3) | 5.6+20080419-2
> libpam0g (>= 0.99.7.1) | 0.99.7.1-6
> libssl0.9.8 (>= 0.9.8f-5) | 0.9.8g-8
>
>
>
>
> _______________________________________________
> Alpine-info mailing list
> Alpine-info@u.washington.edu
> https://mailman1.u.washington.edu/mailman/listinfo/alpine-info
>

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Message #20 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Mike Dresser <mdresser_l@router.windsormachine.com>

To: 478545@bugs.debian.org

Subject: Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's

Date: Tue, 29 Apr 2008 17:08:00 -0400 (EDT)

That would explain why ipopd has the same issue (reported that as well 
today, #478549)

Fixing that would enable large mailbox downloads though, and would fix 
alpine at the same time.  This bug report and #478549 should probably be 
merged and moved to a c-client issue then as they both use c-client?

I think the large file support would be a good idea, since clients such as 
Icedove usually work just fine with very large mailboxes.  If someone were 
to use fetchmail or something similar to grab their >2gb Gmail/whatever 
account all at once to a local mbox, they would be unable to download it 
to another mail client via pop3.

Mike

Message #25 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Mark Crispin <MRC@Washington.EDU>

To: Asheesh Laroia <asheesh@asheesh.org>

Cc: 478545@bugs.debian.org, alpine-info@u.washington.edu

Subject: Re: Bug#478545: [Alpine-info] Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 15:15:56 -0700 (Pacific Daylight Time)

On Tue, 29 Apr 2008, Asheesh Laroia wrote:
> Thanks for the confirmation.

You're welcome.

>> The recommended solution for mailboxes with aggregate size greater than 2GB 
>> is to use mix format instead of a flat file format.  Even if c-client were 
>> updated to use the 64bit system calls, flat files (especially in 
>> traditional UNIX format) do not perform well at multi-GB sizes.
> That's true.

I'm glad that we are in agreement on this point.

>> If there is some compelling reason offered, I will consider 64bit support. 
>> However, as of right now, I am highly skeptical that the benefit would be 
>> worth the effort, especially since there is a superior solution for large 
>> mailboxes available now ("superior" since it will always work better than a 
>> large flat file).
> Well, on a system where the mail setup is configured by the sysadmin, one's 
> only way to read mail might be an enormous /var/mail/$USER that has exceeded 
> two gigabytes.

Fortunately, this isn't true!

Nothing prevents the user from creating an alternative-format INBOX.  The 
simplest way to create a mix-format INBOX in Alpine is to save a message 
to #driver.mix/INBOX and then exit Alpine.  When Alpine is restarted, it 
will see the mix-format INBOX and then automatically start transferring 
messages there from the spool directory.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Message #30 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Asheesh Laroia <asheesh@asheesh.org>

To: Mark Crispin <MRC@Washington.EDU>, 478545@bugs.debian.org

Cc: alpine-info@u.washington.edu

Subject: Re: Bug#478545: [Alpine-info] Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 15:11:50 -0700 (PDT)

On Tue, 29 Apr 2008, Mark Crispin wrote:

> The report is correct.  The c-client library makes no attempt to use the 
> 64bit system calls; and thus flat files are limited to 2GB.

Thanks for the confirmation.

> The recommended solution for mailboxes with aggregate size greater than 2GB 
> is to use mix format instead of a flat file format.  Even if c-client were 
> updated to use the 64bit system calls, flat files (especially in traditional 
> UNIX format) do not perform well at multi-GB sizes.

That's true.

> If there is some compelling reason offered, I will consider 64bit support. 
> However, as of right now, I am highly skeptical that the benefit would be 
> worth the effort, especially since there is a superior solution for large 
> mailboxes available now ("superior" since it will always work better than a 
> large flat file).

Well, on a system where the mail setup is configured by the sysadmin, 
one's only way to read mail might be an enormous /var/mail/$USER that has 
exceeded two gigabytes.

-- Asheesh.

-- 
Questionable day.

Ask somebody something.

Message #35 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Asheesh Laroia <asheesh@asheesh.org>

To: Mark Crispin <MRC@Washington.EDU>

Cc: 478545@bugs.debian.org, alpine-info@u.washington.edu

Subject: Re: Bug#478545: [Alpine-info] Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 15:24:02 -0700 (PDT)

On Tue, 29 Apr 2008, Mark Crispin wrote:

> Nothing prevents the user from creating an alternative-format INBOX.  The 
> simplest way to create a mix-format INBOX in Alpine is to save a message to 
> #driver.mix/INBOX and then exit Alpine.  When Alpine is restarted, it will 
> see the mix-format INBOX and then automatically start transferring messages 
> there from the spool directory.

But what if the spool is already >2GB?

-- Asheesh.

-- 
He draweth out the thread of his verbosity finer than the staple of his
argument.
		-- William Shakespeare, "Love's Labour's Lost"

Message #40 received at 478545@bugs.debian.org (full text, mbox, reply):

From: Mark Crispin <MRC@Washington.EDU>

To: Asheesh Laroia <asheesh@asheesh.org>

Cc: 478545@bugs.debian.org, alpine-info@u.washington.edu

Subject: Re: Bug#478545: [Alpine-info] Bug#478545: [alpine] alpine does not support large (>2 GiB) mbox's (fwd)

Date: Tue, 29 Apr 2008 15:30:01 -0700 (Pacific Daylight Time)

On Tue, 29 Apr 2008, Asheesh Laroia wrote:
> But what if the spool is already >2GB?

Yes, in that case, you have to do some external recovery to break the file 
into smaller chunks.  The split program is useful for this purpose.

By the way, I recommend that mixcvt be used instead of mailutil or 
Alpine's snarfing for converting a mailbox.  This is because mailutil and 
Alpine will do an atomic move of the messages from the spool file, thus 
putting them all into one mix data file.  mixcvt will divide the messages 
into multiple files.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Send a report that this bug log contains spam.