If a packet is heard multiple times, rebroadcast using the highest hop limit (#5534)

* If a packet is heard multiple times, rebroadcast using the highest hop limit

Sometimes a packet will be in the TX queue waiting to be transmitted,
when it is overheard being rebroadcast by another node, with a higher
hop limit remaining. When this occurs, modify the pending packet in
the TX queue to avoid unnecessarily wasting hops.

* Reprocess instead of modifying queued packet

In order to ensure that the traceroute module works correctly, rather
than modifying the hop limnit of the existing queued version of the
packet, simply drop it ifrom the queue and reprocess the version of the
packet with the superior hop limit.

* Update protobufs submodule

* Merge upstream/develop into overheard-hoptimisation branch

Resolved conflicts in:
- src/mesh/FloodingRouter.cpp: Integrated hop limit optimization with refactored duplicate handling
- src/mesh/MeshPacketQueue.h: Kept both hop_limit_lt parameter and new find() method

* Improve method naming and code clarity

- Rename findPacket() to getPacketFromQueue() for better clarity
- Make code DRY by having find() use getPacketFromQueue() internally
- Resolves method overloading conflict with clearer naming

* If a packet is heard multiple times, rebroadcast using the highest hop limit

Sometimes a packet will be in the TX queue waiting to be transmitted,
when it is overheard being rebroadcast by another node, with a higher
hop limit remaining. When this occurs, modify the pending packet in
the TX queue to avoid unnecessarily wasting hops.

* Improve router role checking using IS_ONE_OF macro

- Replace multiple individual role checks with cleaner IS_ONE_OF macro
- Add CLIENT_BASE support as suggested in PR #7992
- Include MeshTypes.h for IS_ONE_OF macro
- Makes code more maintainable and consistent with other parts of codebase

* Apply IS_ONE_OF improvement to NextHopRouter.cpp

- Replace multiple individual role checks with cleaner IS_ONE_OF macro
- Add CLIENT_BASE support for consistency
- Include MeshTypes.h for IS_ONE_OF macro
- Matches the pattern used in FloodingRouter.cpp

* Create and apply IS_ROUTER_ROLE() macro across codebase

- Add IS_ROUTER_ROLE() macro to meshUtils.h for consistent router role checking
- Update FloodingRouter.cpp to use macro in multiple locations
- Update NextHopRouter.cpp to use macro
- Include CLIENT_BASE role support

* Core Changes:
- Add hop_limit field to PacketRecord (17B→20B due to alignment)
- Extend wasSeenRecently() with wasUpgraded parameter
- Enable router optimization without duplicate app delivery
- Handle ROUTER_LATE delayed transmission properly

Technical Details:
- Memory overhead: ~4000 bytes for 1000 records
- Prevents duplicate message delivery while enabling routing optimization
- Maintains protocol integrity for ACK/NAK handling
- Supports upgrade from hop_limit=0 to hop_limit>0 scenarios

* Delete files accdentally added for merge

* Trunk formatting

* Packets are supposed to be unsigned. Thankfully, it's only a log message.

* Upgrade all packets, not just 0 hop packets.

* Not just unsigned, but hex. Updating packet log IDs.

* Fixed order of operations issue that prevented packetrs from being removed from the queue

* Fixing some bugs after testing. Only storing the maximum hop value in PacketRecord which makes sense. Also, updating messaging to make more sense in the logs.

* Fixed flow logic about how to handle re-inserting duplicate packets. Removed IS_ROUTER_ROLE macro and replaced it with better isRebroadcaster().

* Add logic to re-run modules, but avoid re-sending to phone.

* Refactor how to process the new packet with hops. Only update nodeDB and traceRouteModule.

* - Apply changes to both FloodingRouter and NextHopRouter classes to make packets mutable for traceroute
- MESHTASTIC_EXCLUDE_TRACEROUTE guard for when we don't want traceroute

* Allow MeshPacket to be modified in-place in processUpgradePacket

* let's not make a copy where a copy is unncessary.

---------

Co-authored-by: Clive Blackledge <clive@ansible.org>
Co-authored-by: Clive Blackledge <git@ansible.org>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>

src/mesh/FloodingRouter.cpp

@@ -3,6 +3,10 @@
 #include "NodeDB.h"
 #include "configuration.h"
 #include "mesh-pb-constants.h"
+#include "meshUtils.h"
+#if !MESHTASTIC_EXCLUDE_TRACEROUTE
+#include "modules/TraceRouteModule.h"
+#endif
 
 FloodingRouter::FloodingRouter() {}
 
@@ -22,7 +26,37 @@ ErrorCode FloodingRouter::send(meshtastic_MeshPacket *p)
 
 bool FloodingRouter::shouldFilterReceived(const meshtastic_MeshPacket *p)
 {
-    if (wasSeenRecently(p)) { // Note: this will also add a recent packet record
+    bool wasUpgraded = false;
+    bool seenRecently =
+        wasSeenRecently(p, true, nullptr, nullptr, &wasUpgraded); // Updates history; returns false when an upgrade is detected
+
+    // Handle hop_limit upgrade scenario for rebroadcasters
+    // isRebroadcaster() is duplicated in perhapsRebroadcast(), but this avoids confusing log messages
+    if (wasUpgraded && isRebroadcaster() && iface && p->hop_limit > 0) {
+        // wasSeenRecently() reports false in upgrade cases so we handle replacement before the duplicate short-circuit
+        // If we overhear a duplicate copy of the packet with more hops left than the one we are waiting to
+        // rebroadcast, then remove the packet currently sitting in the TX queue and use this one instead.
+        uint8_t dropThreshold = p->hop_limit; // remove queued packets that have fewer hops remaining
+        if (iface->removePendingTXPacket(getFrom(p), p->id, dropThreshold)) {
+            LOG_DEBUG("Processing upgraded packet 0x%08x for rebroadcast with hop limit %d (dropping queued < %d)", p->id,
+                      p->hop_limit, dropThreshold);
+
+            if (nodeDB)
+                nodeDB->updateFrom(*p);
+#if !MESHTASTIC_EXCLUDE_TRACEROUTE
+            if (traceRouteModule && p->which_payload_variant == meshtastic_MeshPacket_decoded_tag &&
+                p->decoded.portnum == meshtastic_PortNum_TRACEROUTE_APP)
+                traceRouteModule->processUpgradedPacket(*p);
+#endif
+
+            perhapsRebroadcast(p);
+
+            // We already enqueued the improved copy, so make sure the incoming packet stops here.
+            return true;
+        }
+    }
+
+    if (seenRecently) {
         printPacket("Ignore dupe incoming msg", p);
         rxDupe++;
 
@@ -134,4 +168,4 @@ void FloodingRouter::sniffReceived(const meshtastic_MeshPacket *p, const meshtas
 
     // handle the packet as normal
     Router::sniffReceived(p, c);
-}
+}