Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

setting bounce address doesn't work

  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago #206559 by ViliusS
setting bounce address doesn't work was created by ViliusS
Hi,

we are trying to use different bounce and From: addresses on our surveys. We set "survey-bounces@example.net" in "Bounce email address" field and "surveys@example.net" in "Administrator email address" field on the survey. When I click "Send Invitation" the "From:" field is correctly prefilled with administrator's email address, but it seems that this also always overwrites the Return-Path: header in the survey invitation. No matter what I set in bounce field the Return-Path: header is always set the same as From: header. Is it a bug?
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago #206571 by DenisChenu
Replied by DenisChenu on topic setting bounce address doesn't work
LimeSurvey version and build number ?

Mail method used ?

Else : some server force Return-Path

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago - 3 years 5 months ago #206583 by ViliusS
Replied by ViliusS on topic setting bounce address doesn't work
Version 3.17.9+190731

Using SMTP method with Office 365 SMTP AUTH service. I did similar setup for other email sending services, like newsletter generation software, on Office 365 without issues. Also, sending email through telnet doesn't force Return-Path, so I don't think it's Office 365 problem.
Last edit: 3 years 5 months ago by ViliusS.
The topic has been locked.
  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago #206585 by holch
Replied by holch on topic setting bounce address doesn't work
Experience in this forum: if Office 365 is involved, the solution is within Office 365 in 95% of the cases. Not saying it is the same here, just saying... ;-)

I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.

The following user(s) said Thank You: DenisChenu
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago - 3 years 5 months ago #206591 by DenisChenu
Replied by DenisChenu on topic setting bounce address doesn't work
I know situation, maybe old, with office365 where i need to create a plugin to set the FROM to the user in SMTP.
Else it don't work …

See bugs.limesurvey.org/view.php?id=10529


[EDIT]
You have way to check what happen when sending SMTP email : activate mail debug to always : manual.limesurvey.org/Global_settings#Email_settings

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
Last edit: 3 years 5 months ago by DenisChenu. Reason: edit
The topic has been locked.
  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago - 3 years 5 months ago #206616 by ViliusS
Replied by ViliusS on topic setting bounce address doesn't work
Again, this is not an issue with Office 365. From: header is mandatory in all cases independant of SMTP provider, see the header table on tools.ietf.org/html/rfc5322#page-20

I have investigated SMTP logs and it seems that the issue is that LimeSurvey doesn't set Return-Path: header at all. During the transmission it only sets MAIL FROM: and From: headers. More over LimeSurvey sets the MAIL FROM: with the value set in "Bounce email address" which is incorrect.

In my opinion
MAIL FROM: should be set to survey administrator email.
Return-Path: should be set to bounce email if it exist and is different from administrator email, if not do not set the header.
And From: header should be set to what is specified during invitation process.

For more information see tools.ietf.org/html/rfc5321#page-57
Code:
It is possible for the mailbox in the return path to be different
   from the actual sender's mailbox, for example, if error responses are
   to be delivered to a special error handling mailbox rather than to
   the message sender.  When mailing lists are involved, this
   arrangement is common and useful as a means of directing errors to
   the list maintainer rather than the message originator.
Last edit: 3 years 5 months ago by ViliusS. Reason: added link to RFC5321
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago #206623 by DenisChenu
Replied by DenisChenu on topic setting bounce address doesn't work

ViliusS wrote: I have investigated SMTP logs and it seems that the issue is that LimeSurvey doesn't set Return-Path: header at all. During the transmission it only sets MAIL FROM: and From: headers. More over LimeSurvey sets the MAIL FROM: with the value set in "Bounce email address" which is incorrect.

Don't look at SMTP log please …

Code don't lye :
github.com/LimeSurvey/LimeSurvey/blob/15...lper.php#L2248-L2252
github.com/LimeSurvey/LimeSurvey/blob/15...mon_helper.php#L2307
github.com/LimeSurvey/LimeSurvey/blob/15.../PHPMailer.php#L1689

Did you activate mail debug ?

MAIL FROM set the Return-Path … it's not the From …


I check !: it's OK.

SMTP user didn't appear
From is set to Adminsitrator email of the survey
Return-Path is set to bouce email

If you look at mail debug tyou see
2020-10-14 12:30:12 CLIENT -> SERVER: MAIL FROM : bouce email
2020-10-14 12:30:12 CLIENT -> SERVER: From: admin email

Did you verify bounce email in Survey settings ?

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago - 3 years 5 months ago #206639 by ViliusS
Replied by ViliusS on topic setting bounce address doesn't work
By SMTP logs I've ment mail debug output from LimeSurvey. Both, the code you provided and the logs confirm exactly what I said. LimeSurvey sets MAIL FROM: and From: headers, but not the custom Return-Path: header. The code incorrectly assumes that Return-Path is set by the server from the specified MAIL FROM: header, which is not always the same.

MAIL FROM is NOT the same as Return-Path when we are talking about a system which needs to process bounces. In that case Return-Path must be set by the PHP code, not the SMTP server.
Last edit: 3 years 5 months ago by ViliusS.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago #206642 by DenisChenu
Replied by DenisChenu on topic setting bounce address doesn't work
I stop after this … since i'm not a phpMailer devloper, and they are better than me.

www.postmastery.com/about-the-return-path-header/

About the Return-Path header

The most common error senders make is adding a Return-Path header to their mail. Even some well known mail software does this. Probably written by programmers who copy from others rather than reading RFCs (the internet standards that specify SMTP and MIME).

Update: As of PHPMailer version 5.2.8 the Return-Path header is no longer set in created messages.


Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago #206678 by ViliusS
Replied by ViliusS on topic setting bounce address doesn't work
The author in the link cites the same RFC5321, but he incorrectly assumes that that RFC forbids to set Return-Path in the originating client. It doesn't!

To summarize:
  1. RFC5321 does not forbid to set Return-Path in the origin. It only says what SMTP server MUST DO:
    Section 4.4.

    When the delivery SMTP server makes the “final delivery” of a message, it inserts a return-path line at the beginning of the mail data. This use of return-path is required; mail systems MUST support it. The return-path line preserves the information in the from the MAIL command. Here, final delivery means the message has left the SMTP environment. Normally, this would mean it had been delivered to the destination user or an associated mail drop, but in some cases it may be further processed and transmitted by another mail system.

    and later

    It is sometimes difficult for an SMTP server to determine whether or
    not it is making final delivery since forwarding or other operations
    may occur after the message is accepted for delivery. Consequently,
    any further (forwarding, gateway, or relay) systems MAY remove the
    return path and rebuild the MAIL command as needed to ensure that
    exactly one such line appears in a delivered message
    .

    Later RFC even talks about the case where Return-Path is added by the originating client, but again it doesn't forbid it.
    Section 4.4.

    A message-originating SMTP system SHOULD NOT send a message that
    already contains a Return-path header field. SMTP servers performing
    a relay function MUST NOT inspect the message data, and especially
    not to the extent needed to determine if Return-path header fields
    are present. SMTP servers making final delivery MAY remove Return-
    path header fields before adding their own
    .

    Note "SHOULD NOT" here. As explained in RFC2119:

    SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that
    there may exist valid reasons in particular circumstances when the
    particular behavior is acceptable or even useful
    , but the full
    implications should be understood and the case carefully weighed
    before implementing any behavior described with this label.

  2. RFC5321 states that MAIL FROM: header is always the sender mailbox and nothing else.
    Section 3.3.

    The <reverse-path> portion of the first or
    only argument contains the source mailbox (between "<" and ">"
    brackets), which can be used to report errors (see Section 4.2 for a
    discussion of error reporting).

    Section 4.1.1.2.

    The reverse-path consists of the sender mailbox. Historically, that
    mailbox might optionally have been preceded by a list of hosts, but
    that behavior is now deprecated (see Appendix C). In some types of
    reporting messages for which a reply is likely to cause a mail loop
    (for example, mail delivery and non-delivery notifications), the
    reverse-path may be null (see Section 3.6).

    Note that here it states that the same sender mailbox can be used for error reporting, but this doesn't mean that, in case of a mailing list or other specialized bounce mailbox present, error reporting must always go to the MAIL FROM: address.
  3. RFC5321 clearly states that in some cases Return-Path could be completely different than the sender address and a system should not assume otherwise.
    Section 4.4.

    It is possible for the mailbox in the return path to be different
    from the actual sender's mailbox
    , for example, if error responses are
    to be delivered to a special error handling mailbox rather than to
    the message sender. When mailing lists are involved, this
    arrangement is common and useful as a means of directing errors to
    the list maintainer rather than the message originator.

    The text above implies that the final mail data will begin with a
    return path line, followed by one or more time stamp lines. These
    lines will be followed by the rest of the mail data: first the
    balance of the mail header section and then the body (RFC 5322 [4]).

    and Section 7.2.

    There is no inherent relationship between either "reverse" (from
    MAIL, SAML, etc., commands) or "forward" (RCPT) addresses in the SMTP
    transaction ("envelope") and the addresses in the header section.
    Receiving systems SHOULD NOT attempt to deduce such relationships and
    use them to alter the header section of the message for delivery.

  4. PHPMailer removed automatic setting of Return-Path header in 5.2.8 and in most cases, where a client just needs to send simple email, this is correct, but this doesn't mean that custom Return-Path header cannot be added. For example, if I would want to implement a mailing list software fully in PHP language by using PHPMailer I would need to set different MAIL FROM: and Return-Path headers otherwise it won't work. The same goes for LimeSurvey. As it has bounce processing capabilities it is not really a client anymore. It is an email processing server software and should modify Return-Path accordingly, using addCustomHeader() as an example.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 5 months ago #206685 by DenisChenu
Replied by DenisChenu on topic setting bounce address doesn't work
If you want : create a issue … i don't care …

You can hack code too, or create plugin.In 4 : you can use your own phpmailer …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • ViliusS
  • ViliusS's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 5 months ago - 3 years 5 months ago #206697 by ViliusS
Replied by ViliusS on topic setting bounce address doesn't work
I don't think that your response is appropriate and up to open source standards and spirit. Your messages doesn't answer anything or add any value to the community. If that's not the place to start the discussion regarding LimeSurvey functionality and in a process to try to make it better, fine by me.

If you don't care, I won't spend any more of my time here either.
Last edit: 3 years 5 months ago by ViliusS.
The topic has been locked.

Lime-years ahead

Online-surveys for every purse and purpose