In any environment where there are lots of messages flying around, there are three critical things you need to know about any particular message:
- Did it arrive as intended?
An integration team *constantly* gets the ask, “Did you get my Message?” “Did you send my Message?” There Must be traceability to messages and whether they arrived or not, with business information attached in the log, like the Order Number of the message, along with the generated MessageID, so you can search on either.
2. Why did that error occur?
Messages fail. The question is why? In 99.x% of environments you can’t log a full message payload to any logger (except at the TRACE or DEBUG level), so you have to have a way of knowing what the payload of the message was that caused the error. (See upcoming post about message auditing) And you also need to know which message caused the error. If you are logging message ID’s then you can easily filter for log lines that contain the message ID and get the entire history of the flow and transformations.
3. Where did the message go next?
Many systems both upstream and downstream of messaging busses have the capability to take in an ID of some sort and correlate it with logs. So send the Message ID as an Outbound Property and have it in the next system’s logs too.
Message ID Logging
Generating a MessageID and logging it. You do this First thing in a flow after the inbound endpoint.
From this point forward, *EVERY LOGGER* has the #[messageID] variable in it. Thus you get traceability, and logging is awesome. Other ideas:
- Set an outbound property of messageID when you send the message somewhere. This works over most transports like JMS and HTTP. Thus the receiving system can get your messageID
- See if whatever system is sending the message can also generate and send you a messageID and use that one instead for traceability backwards. Instead of generating the messageID, you’re using an inboundProperty
- Correlate messageID with payloads in a database, so that you can lookup the message payload in a persisted way that’s more secure than a log file. (Upcoming blog post).
- If you can’t log a message to a database, consider using a separate log file (configured with log4j2.xml) that stores message payloads and messageIDs only. Consider also having it somewhere with better security than the log directory.
Here’s an example where it’s super useful to have messageIDs, first the version without MessageIDs. Notice that there is an error here. Also notice there are two requests, the second request comes in on line 12
Is the Error above part of the first request or the second? It’s probably something you can figure out, but it adds time to support cases. And we are looking for supportability by someone other than an expensive developer.
Now the version with MessageIDs. You’ll notice the Debugging log got a LOT clearer about which message failed. Notice that the next request that came in got a new messageID, and you can easily see what log messages are associated with one request.
The second piece that makes everyones apps even more supportable is storing the payload that is received at the “Got Records” log step into a message database, and associating that line with the MessageID generated. Any support person could then look in the DB, do a select on the messageID and immediately get the payload that caused the error.