Hi there
Please note that I've only recently begun programming in Python, coming from a predominantly Perl/C/C++ background, so my depth of understanding Python is shallow.
While considering slimta as an alternative to qpsmtpd (written in Perl) for my mail systems, I couldn't help but notice the use of Pickle in this extension, for storing messages on disk.
My systems currently use qmail, behind qpsmtpd, so the one option would be to create a qmail-queue queue type for slimta to inject inbound messages into qmail's queue. The other option would be to use slimta to store and queue messages using a patched python-slimta-diskstorage, or a patched version of it, removing qmail from the picture. The latter option brings me to my concerns regarding the use of Pickle, in order to retain a semblance of the performance and reliability of qmail's queue to the slimta environment...
- Pickle is rather restrictive in terms of other programming languages supporting it, so reading queued messages using other languages might be an issue.
- I've found various references to the risks associated with using Pickle to store untrusted data, including Pickle's own documentation. Email messages contain inherently untrusted data!
- Performance. I've read several claims that Pickle is considerably slower than other serialization formats, including JSON.
- Human readability is another potential pitfall, but isn't a deal-breaker for me. Rummaging through messages in the qmail-queue, in the rare event of corruption, is possible, because they're not stored in a binary format. If slimta used a more language agnostic object storage format like MessagePack, CBOR, or JSON, other tools built in other languages would be able to access slimta on-disk queued messages.
In light of the above and the references below, I would consider MessagePack, CBOR or even JSON as better alternatives to Pickle for storing messages in slimta's on-disk queue.
http://www.benfrederickson.com/dont-pickle-your-data/
https://code.tutsplus.com/tutorials/serialization-and-deserialization-of-python-objects-part-2--cms-26184
I'm curious as to the slimta authors' experience/opinions regarding my Pickle concerns/insights.
Hi there
Please note that I've only recently begun programming in Python, coming from a predominantly Perl/C/C++ background, so my depth of understanding Python is shallow.
While considering slimta as an alternative to qpsmtpd (written in Perl) for my mail systems, I couldn't help but notice the use of Pickle in this extension, for storing messages on disk.
My systems currently use qmail, behind qpsmtpd, so the one option would be to create a qmail-queue queue type for slimta to inject inbound messages into qmail's queue. The other option would be to use slimta to store and queue messages using a patched python-slimta-diskstorage, or a patched version of it, removing qmail from the picture. The latter option brings me to my concerns regarding the use of Pickle, in order to retain a semblance of the performance and reliability of qmail's queue to the slimta environment...
In light of the above and the references below, I would consider MessagePack, CBOR or even JSON as better alternatives to Pickle for storing messages in slimta's on-disk queue.
http://www.benfrederickson.com/dont-pickle-your-data/
https://code.tutsplus.com/tutorials/serialization-and-deserialization-of-python-objects-part-2--cms-26184
I'm curious as to the slimta authors' experience/opinions regarding my Pickle concerns/insights.