Resend of message with repeating group emits two 10= checksum tags (parseGroup leaks trailer into bodyBytes)

[YoussefOuirini]

Summary

When quickfix-go parses a stored outbound message whose body contains a repeating group and a DataDictionary is configured, the parser leaks the original 10= trailer into Message.bodyBytes. On resend, buildWithBodyBytes writes that contaminated bodyBytes verbatim and appends a freshly computed 10= trailer on top — producing a malformed FIX message with two 10= tags. The counterparty rejects with Malformed message: Invalid checksum X (expected Y) and drops the session.

Reproduces on v0.9.10 (latest).

Trigger

All three conditions must hold:

1. DataDictionary is configured on the session (without it, parseGroup is never reached).
2. The outbound message contains a repeating group (e.g. NoPartyIDs=453 in a NewOrderSingle).
3. The counterparty sends a ResendRequest covering that message.

Root cause

message.go::parseGroup (line 308) handles repeating-group fields. Its inner loop updates mp.trailerBytes = mp.rawBytes after consuming each field (line 317). When it consumes the trailer field (10=NNN), rawBytes is now empty, so trailerBytes = "".

The safeguard at message.go:283-286:

if len(mp.msg.bodyBytes) > len(mp.trailerBytes) {
    mp.msg.bodyBytes = mp.msg.bodyBytes[:len(mp.msg.bodyBytes)-len(mp.trailerBytes)]
}

then strips zero bytes from bodyBytes, leaving the trailer embedded.

On resend, in_session.go::resendMessages calls msg.buildWithBodyBytes(msg.bodyBytes):

m.Header.write(&b)
b.Write(bodyBytes)        // contains original 10=NNN
m.Trailer.write(&b)       // writes fresh 10=NNN

The non-group path (default body case at line 258-262) is unaffected because the main loop sets trailerBytes to rawBytes after the last body field — i.e. to the buffer that still includes the upcoming trailer. The strip math is then correct.

Minimal repro

package main

import (
    "bytes"
    "fmt"
    "os"
    "reflect"
    "unsafe"

    "github.com/quickfixgo/enum"
    "github.com/quickfixgo/field"
    "github.com/quickfixgo/quickfix"
    "github.com/quickfixgo/quickfix/datadictionary"
    "github.com/quickfixgo/tag"
)

func main() {
    msg := quickfix.NewMessage()
    msg.Header.SetString(tag.BeginString, "FIX.4.4")
    msg.Header.SetString(tag.MsgType, "D")
    msg.Header.SetString(tag.SenderCompID, "S")
    msg.Header.SetString(tag.TargetCompID, "T")
    msg.Header.SetInt(tag.MsgSeqNum, 1)
    msg.Header.SetString(tag.SendingTime, "20260410-12:00:00")
    msg.Body.SetString(tag.ClOrdID, "X")
    msg.Body.SetField(tag.Side, field.NewSide(enum.Side_BUY))
    msg.Body.SetField(tag.OrdType, field.NewOrdType(enum.OrdType_LIMIT))
    msg.Body.SetString(tag.TransactTime, "20260410-12:00:00")

    parties := quickfix.NewRepeatingGroup(tag.NoPartyIDs, quickfix.GroupTemplate{
        quickfix.GroupElement(tag.PartyID),
        quickfix.GroupElement(tag.PartyIDSource),
        quickfix.GroupElement(tag.PartyRole),
    })
    p := parties.Add()
    p.SetString(tag.PartyID, "A")
    p.SetString(tag.PartyIDSource, "D")
    p.SetInt(tag.PartyRole, 1)
    msg.Body.SetGroup(parties)
    raw := []byte(msg.String())

    dictBytes, _ := os.ReadFile("FIX44.xml")
    dict, _ := datadictionary.ParseSrc(bytes.NewBuffer(dictBytes))

    parsed := quickfix.NewMessage()
    quickfix.ParseMessageWithDataDictionary(parsed, bytes.NewBuffer(raw), nil, dict)

    v := reflect.ValueOf(parsed).Elem().FieldByName("bodyBytes")
    bb := *(*[]byte)(unsafe.Pointer(v.UnsafeAddr()))

    fmt.Printf("bodyBytes ends with: %q\n", string(bb[len(bb)-10:]))
    fmt.Printf("10= tag count in bodyBytes: %d (expected 0)\n",
        bytes.Count(bb, []byte("\x0110=")))
}

Output on v0.9.10:

bodyBytes ends with: "452=1\x0110=132\x01"
10= tag count in bodyBytes: 1 (expected 0)

Real-world wire output during resend includes both trailers:
...447=D 452=1 10=129 10=152 → counterparty rejects.

Suggested fix

In parseGroup (line 308), save the pre-extract rawBytes and restore trailerBytes to that snapshot when a trailer field is detected:

for {
    mp.fieldIndex++
    mp.parsedFieldBytes = &mp.msg.fields[mp.fieldIndex]
    preExtract := mp.rawBytes
    mp.rawBytes, _ = extractField(mp.parsedFieldBytes, mp.rawBytes)
    mp.trailerBytes = mp.rawBytes

    if isGroupMember(...) {
        ...
    } else if isHeaderField(...) {
        ...
    } else if isTrailerField(...) {
        mp.msg.Body.add(dm)
        mp.msg.Trailer.add(...)
        mp.foundTrailer = true
        mp.trailerBytes = preExtract  // include trailer bytes
        break
    }
    ...
}

The main parser's safeguard at line 283-286 then strips the trailer correctly.

Related

• DataDictionary necessary for parsing repeating groups... #276 — umbrella issue for resend + repeating groups; the buildWithBodyBytes workaround that shipped is downstream from there.
• ParseMessage messes up order of repeating group tags #568, sending received "fromApp" messages messes repeating groups #608, please reopen #608 #633 — same family of round-trip-with-repeating-group bugs.

Workaround for affected users

Strip the trailing \x0110=NNN\x01 from Message.bodyBytes in Application.ToApp when PossDupFlag=true, via reflection. Requires unsafe.Pointer because bodyBytes is unexported.

Version

github.com/quickfixgo/quickfix v0.9.10