Skip to content

Fix merging of structured data elements with the same SD-ID in Rfc5424Layout - #4420

Open
MaxFreedomPollard wants to merge 2 commits into
apache:2.xfrom
MaxFreedomPollard:fix/rfc5424-merge-structured-data
Open

MaxFreedomPollard wants to merge 2 commits into
apache:2.xfrom
MaxFreedomPollard:fix/rfc5424-merge-structured-data

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown
Contributor

What breaks

Rfc5424Layout merges a StructuredDataMessage into an element that already uses the same SD-ID: the MDC element, a LoggerFields element, or an earlier message of a StructuredDataCollectionMessage. That merge path throws instead of merging:

StructuredDataMessage first = new StructuredDataMessage("Transfer", "Transfer Complete", "Audit");
first.put("Amount", "200.00");
StructuredDataMessage second = new StructuredDataMessage("Transfer", null, "Audit");
second.put("ToAccount", "123456");
LogEvent event = Log4jLogEvent.newBuilder()
        .setMessage(new StructuredDataCollectionMessage(Arrays.asList(first, second)))
        .build();
Rfc5424Layout.newBuilder().build().toSerializable(event);
// java.lang.NullPointerException
//     at org.apache.logging.log4j.core.layout.Rfc5424Layout.addStructuredData(Rfc5424Layout.java:446)

With "Transfer@32473" as the ID it throws UnsupportedOperationException instead. A single new StructuredDataMessage("mdc", ...) logged with a non-empty thread context also throws the NullPointerException, because its SD-ID matches the default MDC element.

Cause

Two problems in the merge path of log4j-core/src/main/java/org/apache/logging/log4j/core/layout/Rfc5424Layout.java:

  1. addStructuredData checks sdElements.containsKey(sdId), where sdId comes from getId(id), but then fetches the element with sdElements.get(id.toString()). For an ID without an enterprise number, getId appends the layout's enterprise number (Transfer@32473) and StructuredDataId.toString() does not (Transfer), so the lookup returns null.
  2. An element created from a StructuredDataMessage keeps the map returned by MapMessage.getData(), which is unmodifiable, so StructuredDataElement.union fails on putAll when a second message with the same SD-ID is merged into it.

Fix

Fetch the element by sdId, and let StructuredDataElement.union copy the fields into a new HashMap before adding the new ones. The copy only happens when two elements are merged. The same code is on main.

Testing

Two tests in Rfc5424LayoutTest:

  • testCollectionMergesElementsWithSameId, parameterized with Transfer and Transfer@32473, expects [Transfer@32473 Amount="200.00" ToAccount="123456"].
  • testMdcElementMergesStructuredDataWithSameId expects [mdc@32473 Amount="200.00" loginId="JohnDoe"].

I could not run Maven in my environment, so ./mvnw verify (including Spotless) and the real JUnit run have not been done. Instead I compiled Rfc5424Layout and Rfc5424LayoutTest with javac (JDK 17) against the log4j-api, log4j-core and log4j-core-test sources. JUnit and AssertJ were replaced by small stand-ins, and the Log4j2Plugins.dat was generated with PluginProcessor. I then ran every test method of Rfc5424LayoutTest through a reflection-based runner. On 2.x at 8d7d3c5, 35 of 38 invocations pass, and the 3 failures are the new ones (the two NullPointerExceptions and the UnsupportedOperationException above). With the fix, all 38 pass.

Checklist

  • Base your changes on 2.x branch if you are targeting Log4j 2; use main otherwise
  • ./mvnw verify succeeds (the build instructions): not run, see Testing
  • Non-trivial changes contain an entry file in the src/changelog/.2.x.x directory
  • Tests are provided

🤖 Generated with Claude Code

…4Layout

`Rfc5424Layout.addStructuredData` merges a `StructuredDataMessage` into an element that already has the same SD-ID, such as the MDC element, a `LoggerFields` element or an earlier message of a `StructuredDataCollectionMessage`. It found the element by the SD-ID computed with `getId`, but then fetched it with `StructuredDataId.toString()`. For an ID without an enterprise number `getId` appends the layout's enterprise number and `toString()` does not, so the lookup returned `null` and the merge threw a `NullPointerException`.

Elements created from a `StructuredDataMessage` also kept the unmodifiable map returned by `MapMessage.getData()`, so merging a second message with the same SD-ID into one threw an `UnsupportedOperationException`.

Look the element up by the computed SD-ID and copy the fields when merging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxFreedomPollard
MaxFreedomPollard force-pushed the fix/rfc5424-merge-structured-data branch from eaba60d to fbf551c Compare October 22, 2026 22:34
@ramanathan1504 ramanathan1504 added this to the 2.27.0 milestone Oct 23, 2026
type="fixed">
<issue id="4420" link="https://github.com/apache/logging-log4j2/pull/4420"/>
<description format="asciidoc">
Fix `Rfc5424Layout` throwing an exception when merging structured data elements with the same SD-ID.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description is 100 characters. A shorter line reads better in the release notes:

Suggested change
Fix `Rfc5424Layout` throwing an exception when merging structured data elements with the same SD-ID.
Fix `Rfc5424Layout` failing on two SD elements with one SD-ID.

@ramanathan1504

Copy link
Copy Markdown
Contributor

Looking up the element by sdId and copying the fields in union fixes both the NullPointerException and the UnsupportedOperationException, and each new test fails on 2.x without them.
Can we shorten the changelog description to one short line? There is a suggestion on the changelog file.

@ramanathan1504 ramanathan1504 added bug Incorrect, unexpected, or unintended behavior of existing code layouts Affects one or more Layout plugins labels Oct 23, 2026
Use the wording suggested in review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxFreedomPollard

Copy link
Copy Markdown
Contributor Author

Thanks, I've applied your suggested wording to the changelog entry.

@ramanathan1504 ramanathan1504 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ramanathan1504
ramanathan1504 enabled auto-merge (squash) October 29, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Incorrect, unexpected, or unintended behavior of existing code layouts Affects one or more Layout plugins port-to-main

Projects

Development

Successfully merging this pull request may close these issues.

2 participants