Compare commits

...

102 Commits

Author SHA1 Message Date
Tom Fifield
7f565fd524 Merge branch 'develop' into sfpp 2026-01-16 11:49:28 +11:00
Tom Fifield
91dd39a651 Add sqlite depdendency (Cherry-picks from sfpp) (#9328)
* Add sqlite to build requires

* Add missed comma

* Add sqlite dev to more dockerfiles

* Alpine docker fix

* Add sqlite to build requires

* Add sqlite depdendency (Cherry-picks from sfpp)

Store and Forward Plus Plus requires sqlite to work.

This PR cherry picks the commits that added the dependency so that
this can be added, and reduce the amount of effort to review sfpp.

Authored-By: @jp-bennett

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-01-15 17:18:02 -06:00
Jonathan Bennett
5af6a48326 Merge branch 'develop' into sfpp 2026-01-15 14:03:43 -06:00
Ben Meadors
d493f5f171 Merge branch 'master' into develop 2026-01-15 08:26:09 -06:00
Jonathan Bennett
5633848d75 Tryfix nohop 2026-01-12 23:17:14 -06:00
Jonathan Bennett
8b8c1881a8 Add NoHopPorts 2026-01-11 17:33:23 -06:00
Jonathan Bennett
a1d6978626 Add portnum whitelist for testing 2026-01-10 23:27:44 -06:00
Jonathan Bennett
a67cf0f726 Add SF++ metrics logging 2026-01-10 13:34:25 -06:00
Jonathan Bennett
456fa3ddeb Merge branch 'develop' into sfpp 2026-01-09 16:30:32 -06:00
Jonathan Bennett
5adc9663b7 Handle text messages from local node 2026-01-09 16:29:54 -06:00
Jonathan Bennett
5de0654819 Logging 2026-01-08 22:38:55 -06:00
Jonathan Bennett
ab781e9f2d Don't try to send scratch messages from stratum0 2026-01-08 15:28:33 -06:00
Jonathan Bennett
595b5f19b3 Fix message hash recalc 2026-01-08 14:48:21 -06:00
Jonathan Bennett
4bb93c1ed2 Don't double add to canon scratch 2026-01-08 14:35:55 -06:00
Jonathan Bennett
5582e94009 Minor fix 2026-01-08 14:30:31 -06:00
Jonathan Bennett
e33fbca8d6 Refactor and bugfix 2026-01-08 14:07:42 -06:00
Jonathan Bennett
aca7fe9f95 Properly empty canon_scratch 2026-01-08 13:32:11 -06:00
Jonathan Bennett
5a0644cd4f Code refactor 2026-01-08 13:02:14 -06:00
Jonathan Bennett
e990198628 Merge branch 'develop' into sfpp 2026-01-08 12:07:24 -06:00
Jonathan Bennett
6c69d9e74c Disable SF++ by default in forced sim mode (When running with -s) 2026-01-08 11:57:25 -06:00
Jonathan Bennett
e03f1b5c5e Merge branch 'develop' into sfpp 2026-01-08 11:42:50 -06:00
Jonathan Bennett
f46a9dfe7b misc 2026-01-08 01:07:18 -06:00
Jonathan Bennett
9824357c50 Stratum0 fix 2026-01-08 00:46:00 -06:00
Jonathan Bennett
942f2cb3d1 Tryfix next 2026-01-08 00:32:09 -06:00
Jonathan Bennett
76beeda392 Re-add exception for empty chain 2026-01-08 00:12:49 -06:00
Jonathan Bennett
821735495a Unbork the things 2026-01-08 00:08:49 -06:00
Jonathan Bennett
9ab2ee3483 Check for valid hash during commit 2026-01-07 23:42:38 -06:00
Jonathan Bennett
4e92f7fa09 Merge branch 'develop' into sfpp 2026-01-07 23:00:16 -06:00
Jonathan Bennett
ae2a06eccd SFPP Logging misc 2026-01-07 17:27:14 -06:00
Jonathan Bennett
3ae331eb89 Add canon_scratch to SF++ 2026-01-07 12:44:22 -06:00
Jonathan Bennett
74a6c9f447 Alpine docker fix 2026-01-04 23:16:22 -06:00
Jonathan Bennett
c77709a327 Fix the other links_behind underflow 2026-01-04 23:03:47 -06:00
Jonathan Bennett
325f7d2e55 Don't process packet when decoding fails 2026-01-04 21:11:47 -06:00
Jonathan Bennett
c6fc7986f1 More sanity checks on incoming messages 2026-01-04 21:06:36 -06:00
Jonathan Bennett
8ecce1eb5c Fix for integer overflow 2026-01-04 20:56:01 -06:00
Jonathan Bennett
21c0dcaabb Merge branch 'develop' into sfpp 2026-01-04 16:55:59 -06:00
Jonathan Bennett
1b13f872db Don't double-process rebroadcast messages 2026-01-04 14:24:51 -06:00
Jonathan Bennett
8b5141ddb7 Merge branch 'develop' into sfpp 2026-01-04 13:26:41 -06:00
Jonathan Bennett
ee25a0a0e1 Actually include the counter in CANON ANNOUNCE 2026-01-03 21:40:38 -06:00
Jonathan Bennett
436f174bce Actually include the chain counter in CANON ANNOUNCE 2026-01-03 21:11:44 -06:00
Jonathan Bennett
a34cd4ca6f Better log messages 2026-01-03 21:11:22 -06:00
Jonathan Bennett
8c37669213 Check links behind when receiving a link 2026-01-03 20:59:41 -06:00
Jonathan Bennett
8a059bae23 Add option to clear a chain when it falls too far behind 2026-01-03 20:36:37 -06:00
Jonathan Bennett
1869f2108d Add sqlite dev to more dockerfiles 2026-01-03 12:41:41 -06:00
Jonathan Bennett
f5b41c2f2c Add missed comma 2026-01-03 12:38:59 -06:00
Jonathan Bennett
83c8875060 Merge branch 'develop' into sfpp 2026-01-03 12:34:44 -06:00
Jonathan Bennett
9134239faa Add sqlite to build requires 2026-01-03 12:34:10 -06:00
Jonathan Bennett
b3d1d563e9 Merge branch 'develop' into sfpp 2026-01-02 16:17:28 -06:00
Jonathan Bennett
c7f816e63f Add StoreAndForward 2026-01-02 16:13:33 -06:00
Jonathan Bennett
dd4fb6b0bc Add chain speculation from scratch 2026-01-02 14:43:18 -06:00
Jonathan Bennett
87798429fa Log rebroadcast timeouts 2026-01-02 11:48:42 -06:00
Jonathan Bennett
78baaf4484 Check for existing commit hash, etc 2026-01-01 23:25:49 -06:00
Jonathan Bennett
b3b115b6a6 Don't process own packets in SF++ 2026-01-01 22:40:53 -06:00
Jonathan Bennett
b90b5ff40e Merge branch 'develop' into sfpp 2026-01-01 22:17:04 -06:00
Jonathan Bennett
b7028fff08 Add peers table 2026-01-01 22:16:35 -06:00
Jonathan Bennett
7d6a0f20c6 Add extra check for end of chain matching 2026-01-01 22:16:23 -06:00
Jonathan Bennett
9b7384507d Update log messages to include StoreForwardpp 2026-01-01 14:00:40 -06:00
Jonathan Bennett
7d7091ef94 Add null check for p_encrypted before MQTT publish (#9136)
* Add null check for p_encrypted before MQTT publish

A user on BayMesh observed a strange crash in MQTT::onSend that seemed to be a null pointer dereference of this value.

* Trunk
2026-01-01 13:53:53 -06:00
Jonathan Bennett
baccd0c532 Add peers table, check for null 2026-01-01 13:47:50 -06:00
Jonathan Bennett
1fecdc7603 Check again for NTP in SF++ 2025-12-31 23:28:36 -06:00
Jonathan Bennett
1625fd88d7 Merge branch 'develop' into sfpp 2025-12-30 20:20:52 -06:00
Jonathan Bennett
fe22460f25 Keep your names consistent! 2025-12-30 20:13:44 -06:00
Jonathan Bennett
f634b7dd60 Don't Scratch! 2025-12-30 20:06:21 -06:00
Jonathan Bennett
f56e651787 Don't shadow variables 2025-12-30 19:10:07 -06:00
Jonathan Bennett
55af6c4726 Split message fixes 2025-12-29 23:43:26 -06:00
Jonathan Bennett
d272b28ed4 sfpp Split Messages 2025-12-29 21:57:42 -06:00
Jonathan Bennett
f8c27d1714 Working chain trimming 2025-12-29 18:50:34 -06:00
Jonathan Bennett
25383c9523 Add sfpp as core portnum 2025-12-29 11:43:13 -06:00
Jonathan Bennett
6d90b6536e More comment cleanup 2025-12-29 11:20:39 -06:00
Jonathan Bennett
0759197ab3 Merge branch 'develop' into sfpp 2025-12-29 11:18:43 -06:00
Jonathan Bennett
bbfca12d50 Cleanups 2025-12-29 11:05:21 -06:00
Jonathan Bennett
d44c3a8e1a And the enable 2025-12-29 09:43:02 -06:00
Jonathan Bennett
1cef1094a0 More sfpp config 2025-12-29 09:42:19 -06:00
Jonathan Bennett
02d4ca2983 Payload size work and misc 2025-12-29 09:35:18 -06:00
Jonathan Bennett
36e8a498f1 Wire in COMPRESSED port option pt 1 2025-12-29 09:33:29 -06:00
Jonathan Bennett
d63b583ea2 Hash size 16 2025-12-29 09:32:48 -06:00
Jonathan Bennett
14073e2c9f Merge branch 'develop' into sfpp 2025-12-29 08:28:59 -06:00
Jonathan Bennett
39a6ffc664 Merge branch 'develop' into sfpp 2025-12-28 15:45:21 -06:00
Jonathan Bennett
8be790890c Check for read-only DB 2025-12-28 15:29:22 -06:00
Jonathan Bennett
426a7c19dd Short hashes in 2025-12-28 15:29:10 -06:00
Jonathan Bennett
39c0824abb Minor tweak for behavior when chain is empty 2025-12-27 23:28:25 -06:00
Jonathan Bennett
a8a5086b6d Add more SFPP config values 2025-12-27 21:21:51 -06:00
Jonathan Bennett
428b839254 Merge branch 'develop' into sfpp 2025-12-26 22:22:53 -06:00
Jonathan Bennett
a70d350ce3 Fix getLinkFromCount() 2025-12-26 14:56:27 -06:00
Jonathan Bennett
00a3249c56 Merge branch 'develop' into sfpp 2025-12-26 10:40:03 -06:00
Jonathan Bennett
b51235d4fd Misc 2025-12-26 00:03:21 -06:00
Jonathan Bennett
d07f5be548 misc partial chain fixes 2025-12-25 23:57:21 -06:00
Jonathan Bennett
739ad0dc31 Add count handling to SFPP 2025-12-25 22:39:08 -06:00
Jonathan Bennett
e8fd5174ec Don't stash messages without a matching chain root 2025-12-25 19:38:44 -06:00
Jonathan Bennett
96726d22cd Allow UDP and API packets in S$F++ 2025-12-23 11:16:36 -06:00
Jonathan Bennett
3330d297b1 Set hop_limit and hop_start on message rebroadcasts 2025-12-23 11:12:05 -06:00
Jonathan Bennett
e9ed2c0335 Include hop_start in printPacket 2025-12-23 11:11:35 -06:00
Jonathan Bennett
3cbc5b7a8d Don't wipe scratch 2025-12-22 09:34:51 -06:00
Jonathan Bennett
20bf822a48 Add missing root_hash from scratch 2025-12-21 20:11:17 -06:00
Jonathan Bennett
14ee1ed075 shorthash on canon announce 2025-12-21 19:58:17 -06:00
Jonathan Bennett
4d48d517e0 Build tryfix next 2025-12-21 19:56:53 -06:00
Jonathan Bennett
ffdb3bc393 Misc fixes 2025-12-21 19:49:00 -06:00
Jonathan Bennett
6e83a9a0b3 unbreak all the targets 2025-12-21 19:39:44 -06:00
Jonathan Bennett
73cfa3c884 Store incoming non-canon messages in scratch 2025-12-21 18:47:05 -06:00
Jonathan Bennett
f2b6383cbb Scratch fix 2025-12-21 16:10:06 -06:00
Jonathan Bennett
28d507f043 Broadcast root hash for an empty chain 2025-12-21 14:25:12 -06:00
Jonathan Bennett
d508de9568 Add Store and Forward++ module 2025-12-20 13:05:13 -06:00
16 changed files with 2555 additions and 9 deletions

View File

@@ -20,7 +20,7 @@ ENV PIP_ROOT_USER_ACTION=ignore
RUN apt-get update && apt-get install --no-install-recommends -y \
cmake git zip libgpiod-dev libbluetooth-dev libi2c-dev \
libunistring-dev libmicrohttpd-dev libgnutls28-dev libgcrypt20-dev \
libusb-1.0-0-dev libssl-dev pkg-config && \
libusb-1.0-0-dev libssl-dev pkg-config libsqlite3-dev && \
apt-get clean && rm -rf /var/lib/apt/lists/* && \
pip install --no-cache-dir -U \
platformio==6.1.16 \

View File

@@ -14,7 +14,7 @@ RUN apt-get update && apt-get install --no-install-recommends -y \
curl wget g++ zip git ca-certificates pkg-config \
libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
libx11-dev libinput-dev libxkbcommon-x11-dev \
libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir -U platformio \
&& mkdir /tmp/firmware

View File

@@ -11,7 +11,7 @@ RUN apk --no-cache add \
bash g++ libstdc++-dev linux-headers zip git ca-certificates libbsd-dev \
libgpiod-dev yaml-cpp-dev bluez-dev \
libusb-dev i2c-tools-dev libuv-dev openssl-dev pkgconf argp-standalone \
libx11-dev libinput-dev libxkbcommon-dev \
libx11-dev libinput-dev libxkbcommon-dev sqlite-dev \
&& rm -rf /var/cache/apk/* \
&& pip install --no-cache-dir -U platformio \
&& mkdir /tmp/firmware

View File

@@ -203,6 +203,16 @@ HostMetrics:
# UserStringCommand: cat /sys/firmware/devicetree/base/serial-number # Command to execute, to send the results as the userString
StoreAndForward:
# Enabled: true # Enable Store and Forward++, true by default
# DBPath: /var/lib/meshtasticd/ # Path to the S&F++ Sqlite DB
# Stratum0: false # Specify if this node is a Stratum 0 node, the controller node.
# InitialSync: 10 # Number of messages to
# Hops: 3 # Number of hops to use for SF++ messages
# AnnounceInterval: 5 # Interval in minutes between announcing tip of chain hash
# MaxChainLength: 1000 # Maximum number of messages to store in a chain
Config:
# DisplayMode: TWOCOLOR # uncomment to force BaseUI
# DisplayMode: COLOR # uncomment to force MUI

3
debian/control vendored
View File

@@ -25,7 +25,8 @@ Build-Depends: debhelper-compat (= 13),
liborcania-dev,
libx11-dev,
libinput-dev,
libxkbcommon-x11-dev
libxkbcommon-x11-dev,
libsqlite3-dev
Standards-Version: 4.6.2
Homepage: https://github.com/meshtastic/firmware
Rules-Requires-Root: no

View File

@@ -39,6 +39,7 @@ BuildRequires: pkgconfig(bluez)
BuildRequires: pkgconfig(libusb-1.0)
BuildRequires: libi2c-devel
BuildRequires: pkgconfig(libuv)
BuildRequires: pkgconfig(sqlite3)
# Web components:
BuildRequires: pkgconfig(openssl)
BuildRequires: pkgconfig(liborcania)

View File

@@ -95,7 +95,7 @@ lib_deps =
# renovate: datasource=custom.pio depName=NonBlockingRTTTL packageName=end2endzone/library/NonBlockingRTTTL
end2endzone/NonBlockingRTTTL@1.4.0
build_flags = ${env.build_flags} -Os
build_src_filter = ${env.build_src_filter} -<platform/portduino/> -<graphics/niche/>
build_src_filter = ${env.build_src_filter} -<platform/portduino/> -<graphics/niche/> -<modules/Native/>
; Common libs for communicating over TCP/IP networks such as MQTT
[networking_base]

View File

@@ -145,6 +145,18 @@ bool NextHopRouter::perhapsRebroadcast(const meshtastic_MeshPacket *p)
tosend->hop_start -= (tosend->hop_limit - 2);
tosend->hop_limit = 2;
}
#elif ARCH_PORTDUINO
if (tosend->which_payload_variant == meshtastic_MeshPacket_decoded_tag &&
portduino_config.nohop_ports.size()) {
for (const auto &port : portduino_config.nohop_ports) {
if (port == tosend->decoded.portnum) {
LOG_DEBUG("0-hopping portnum %u", tosend->decoded.portnum);
tosend->hop_start -= tosend->hop_limit;
tosend->hop_limit = 0;
break;
}
}
}
#endif
if (p->next_hop == NO_NEXT_HOP_PREFERENCE) {

View File

@@ -326,9 +326,9 @@ uint32_t RadioInterface::getTxDelayMsecWeighted(meshtastic_MeshPacket *p)
void printPacket(const char *prefix, const meshtastic_MeshPacket *p)
{
#if defined(DEBUG_PORT) && !defined(DEBUG_MUTE)
std::string out =
DEBUG_PORT.mt_sprintf("%s (id=0x%08x fr=0x%08x to=0x%08x, transport = %u, WantAck=%d, HopLim=%d Ch=0x%x", prefix, p->id,
p->from, p->to, p->transport_mechanism, p->want_ack, p->hop_limit, p->channel);
std::string out = DEBUG_PORT.mt_sprintf(
"%s (id=0x%08x fr=0x%08x to=0x%08x, transport = %u, WantAck=%d, HopLim=%d HopStart=%d Ch=0x%x", prefix, p->id, p->from,
p->to, p->transport_mechanism, p->want_ack, p->hop_limit, p->hop_start, p->channel);
if (p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) {
auto &s = p->decoded;

View File

@@ -17,6 +17,7 @@
#endif
#include "Default.h"
#if ARCH_PORTDUINO
#include "modules/Native/StoreForwardPlusPlus.h"
#include "platform/portduino/PortduinoGlue.h"
#endif
#if ENABLE_JSON_LOGGING || ARCH_PORTDUINO
@@ -359,6 +360,12 @@ ErrorCode Router::send(meshtastic_MeshPacket *p)
abortSendAndNak(encodeResult, p);
return encodeResult; // FIXME - this isn't a valid ErrorCode
}
#if ARCH_PORTDUINO
if (p_decoded->decoded.portnum == meshtastic_PortNum_TEXT_MESSAGE_APP &&
(p->from == 0 || p->from == nodeDB->getNodeNum()) && storeForwardPlusPlusModule && portduino_config.sfpp_enabled) {
storeForwardPlusPlusModule->handleEncrypted(p_decoded, p);
}
#endif
#if !MESHTASTIC_EXCLUDE_MQTT
// Only publish to MQTT if we're the original transmitter of the packet
if (moduleConfig.mqtt.enabled && isFromUs(p) && mqtt) {
@@ -735,6 +742,22 @@ void Router::handleReceived(meshtastic_MeshPacket *p, RxSource src)
cancelSending(p->from, p->id);
skipHandle = true;
}
#if ARCH_PORTDUINO
if (portduino_config.whitelist_enabled) {
bool allowed = false;
for (const auto &port : portduino_config.whitelist_ports) {
if (port == p->decoded.portnum) {
allowed = true;
break;
}
}
if (!allowed) {
LOG_DEBUG("Dropping packet not on Portduino Whitelist");
cancelSending(p->from, p->id);
skipHandle = true;
}
}
#endif
} else {
printPacket("packet decoding failed or skipped (no PSK?)", p);
}

View File

@@ -61,6 +61,7 @@
#if ARCH_PORTDUINO
#include "input/LinuxInputImpl.h"
#include "input/SeesawRotary.h"
#include "modules/Native/StoreForwardPlusPlus.h"
#include "modules/Telemetry/HostMetrics.h"
#if !MESHTASTIC_EXCLUDE_STOREFORWARD
#include "modules/StoreForwardModule.h"
@@ -243,6 +244,11 @@ void setupModules()
#endif
#if ARCH_PORTDUINO
new HostMetricsModule();
#if SFPP_ENABLED
if (portduino_config.sfpp_enabled) {
storeForwardPlusPlusModule = new StoreForwardPlusPlusModule();
}
#endif
#endif
#if HAS_TELEMETRY
new DeviceTelemetryModule();

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,310 @@
#pragma once
#if __has_include("sqlite3.h")
#define SFPP_ENABLED 1
#include "Channels.h"
#include "ProtobufModule.h"
#include "Router.h"
#include "SinglePortModule.h"
#include "sqlite3.h"
#define SFPP_HASH_SIZE 16
#define SFPP_SHORT_HASH_SIZE 8
/**
* Store and forward ++ module
* There's an obvious need for a store-and-forward mechanism in Meshtastic.
* This module takes heavy inspiration from Git, building a chain of messages that can be synced between nodes.
* Each message is hashed, and the chain is built by hashing the previous commit hash and the current message hash.
* Nodes can request missing messages by requesting the next message after a given commit hash.
*
* The current focus is text messages, limited to the primary channel.
*
* Each chain is identified by a root hash, which is derived from the channelHash, the local nodenum, and the timestamp when
* created.
*
* Each message is also given a message hash, derived from the encrypted payload, the to, from, id.
* Notably not the timestamp, as we want these to match across nodes, even if the timestamps differ.
*
* The authoritative node for the chain will generate a commit hash for each message when adding it to the chain.
* The first message's commit hash is derived from the root hash and the message hash.
* Subsequent messages' commit hashes are derived from the previous commit hash and the current message hash.
* This allows a node to see only the last commit hash, and confirm it hasn't missed any messages.
*
* Nodes can request the next message in the chain by sending a LINK_REQUEST message with the root hash and the last known commit
* hash. Any node that has the next message can respond with a LINK_PROVIDE message containing the next message.
*
* When a satellite node sees a new text message, it stores it in a scratch database.
* These messages are periodically offered to the authoritative node for inclusion in the chain.
*
* The LINK_PROVIDE message does double-duty, sending both on-chain and off-chain messages.
* The differentiator is whether the commit hash is set or left empty.
*
* When a satellite node receives a canonical link message, it checks if it has the message in scratch.
* And evicts it when adding it to the canonical chain.
*
* This approach allows a node to know whether it has seen a given message before, or if it is new coming via SFPP.
* If new, and the timestamp is within the rebroadcast timeout, it will process that message as if it were just received from the
* mesh, allowing it to be decrypted, shown to the user, and rebroadcast.
*/
class StoreForwardPlusPlusModule : public ProtobufModule<meshtastic_StoreForwardPlusPlus>, private concurrency::OSThread
{
struct link_object {
uint32_t to;
uint32_t from;
uint32_t id;
uint32_t rx_time = 0;
ChannelHash channel_hash;
uint8_t encrypted_bytes[256] = {0};
size_t encrypted_len;
uint8_t message_hash[SFPP_HASH_SIZE] = {0};
size_t message_hash_len = 0;
uint8_t root_hash[SFPP_HASH_SIZE] = {0};
size_t root_hash_len = 0;
uint8_t commit_hash[SFPP_HASH_SIZE] = {0};
size_t commit_hash_len = 0;
uint32_t counter = 0;
std::string payload;
bool validObject = true; // set this false when a chain calulation fails, etc.
};
public:
/** Constructor
*
*/
StoreForwardPlusPlusModule();
/*
-Override the wantPacket method.
*/
virtual bool wantPacket(const meshtastic_MeshPacket *p) override
{
if (p->decoded.portnum == meshtastic_PortNum_TEXT_MESSAGE_APP ||
p->decoded.portnum == (portduino_config.sfpp_steal_port ? meshtastic_PortNum_TEXT_MESSAGE_COMPRESSED_APP
: meshtastic_PortNum_STORE_FORWARD_PLUSPLUS_APP)) {
return true;
} else {
return false;
}
}
void handleEncrypted(const meshtastic_MeshPacket *, const meshtastic_MeshPacket *);
protected:
/** Called to handle a particular incoming message
@return ProcessMessage::STOP if you've guaranteed you've handled this message and no other handlers should be considered for
it
*/
virtual ProcessMessage handleReceived(const meshtastic_MeshPacket &mp) override;
virtual bool handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshtastic_StoreForwardPlusPlus *t) override;
virtual int32_t runOnce() override;
private:
sqlite3 *ppDb;
sqlite3_stmt *chain_insert_stmt;
sqlite3_stmt *scratch_insert_stmt;
sqlite3_stmt *checkDupMessageHash;
sqlite3_stmt *checkDupCommitHash;
sqlite3_stmt *checkScratch;
sqlite3_stmt *removeScratch;
sqlite3_stmt *updatePayloadStmt;
sqlite3_stmt *getPayloadFromScratchStmt;
sqlite3_stmt *fromScratchStmt;
sqlite3_stmt *fromScratchByHashStmt;
sqlite3_stmt *getNextHashStmt;
sqlite3_stmt *getChainEndStmt;
sqlite3_stmt *getLinkStmt;
sqlite3_stmt *getLinkFromMessageHashStmt;
sqlite3_stmt *getHashFromRootStmt;
sqlite3_stmt *addRootToMappingsStmt;
sqlite3_stmt *getRootFromChannelHashStmt;
sqlite3_stmt *getFullRootHashStmt;
sqlite3_stmt *setChainCountStmt;
sqlite3_stmt *getChainCountStmt;
sqlite3_stmt *getScratchCountStmt;
sqlite3_stmt *getRootCanonScratchCountStmt;
sqlite3_stmt *pruneScratchQueueStmt;
sqlite3_stmt *trimOldestLinkStmt;
sqlite3_stmt *maybeAddPeerStmt;
sqlite3_stmt *getPeerStmt;
sqlite3_stmt *updatePeerStmt;
sqlite3_stmt *clearChainStmt;
sqlite3_stmt *canon_scratch_insert_stmt;
sqlite3_stmt *getCanonScratchCountStmt;
sqlite3_stmt *getCanonScratchStmt;
sqlite3_stmt *removeCanonScratch;
sqlite3_stmt *clearCanonScratchStmt;
// For a given Meshtastic ChannelHash, fills the root_hash buffer with a 32-byte root hash
// returns true if the root hash was found
bool getRootFromChannelHash(ChannelHash, uint8_t *);
// For a given root hash, returns the ChannelHash
// can handle partial root hashes
ChannelHash getChannelHashFromRoot(uint8_t *_root_hash, size_t);
// given a root hash and commit hash, returns the next commit hash in the chain
// can handle partial root and commit hashes, always fills the buffer with 32 bytes
// returns true if a next hash was found
bool getNextHash(uint8_t *, size_t, uint8_t *, size_t, uint8_t *);
// For a given Meshtastic ChannelHash, fills the root_hash buffer with a 32-byte root hash
// but this function will add the root hash if it is not already present
// returns hash size or 0 if not found/added
size_t getOrAddRootFromChannelHash(ChannelHash, uint8_t *);
// adds the ChannelHash and root_hash to the mappings table
void addRootToMappings(ChannelHash, uint8_t *);
// requests the next message in the chain from the mesh network
// Sends a LINK_REQUEST message
void requestNextMessage(uint8_t *, size_t, uint8_t *, size_t);
// request the message X entries from the end.
// used to bootstrap a chain, without downloading all of the history
void requestMessageCount(uint8_t *, size_t, uint32_t);
// sends a LINK_PROVIDE message broadcasting the given link object
void broadcastLink(uint8_t *, size_t);
// sends a LINK_PROVIDE message broadcasting the given link object
void broadcastLink(link_object &, bool, bool = false);
// sends a LINK_PROVIDE message broadcasting the given link object from scratch message store
bool sendFromScratch(uint8_t *);
// Adds the given link object to the canonical chain database
bool addToChain(link_object &);
// Adds an incoming text message to the scratch database
bool addToScratch(link_object &);
// sends a CANON_ANNOUNCE message, specifying the given root and commit hashes
void canonAnnounce(link_object &);
// checks if the message hash is present in the canonical chain database
bool isInDB(uint8_t *, size_t);
// checks if the commit hash is present in the canonical chain database
bool isCommitInDB(uint8_t *, size_t);
// checks if the message hash is present in the scratch database
bool isInScratch(uint8_t *, size_t);
// retrieves a link object from the scratch database
link_object getFromScratch(uint8_t *, size_t);
// removes a link object from the scratch database
void removeFromScratch(uint8_t *, size_t);
// iterate through our scratch database, and see if we can speculate a chain up to the given commit hash
bool speculateScratchChain(uint8_t *, size_t, uint8_t *, uint8_t *);
// retrieves the next link object from scratch given a root hash
link_object getNextScratchObject(uint8_t *);
// fills the payload section with the decrypted data for the given message hash
// probably not needed for production, but useful for testing
void updatePayload(uint8_t *, size_t, std::string);
// Takes the decrypted MeshPacket and the encrypted packet copy, and builds a link_object
// Generates a message hash, but does not set the commit hash
link_object ingestTextPacket(const meshtastic_MeshPacket &, const meshtastic_MeshPacket *);
// ingests a LINK_PROVIDE message and builds a link_object
// confirms the root hash and commit hash
link_object ingestLinkMessage(meshtastic_StoreForwardPlusPlus *);
// retrieves a link object from the canonical chain database given a commit hash
link_object getLink(uint8_t *, size_t);
// retrieves a link object from the canonical chain database given a message hash
link_object getLinkFromMessageHash(uint8_t *, size_t);
// puts the encrypted payload back into the queue as if it were just received
void rebroadcastLinkObject(link_object &);
// Check if an incoming link object's commit hash matches the calculated commit hash
bool checkCommitHash(link_object &lo, uint8_t *commit_hash_bytes, size_t hash_len);
// given a partial root hash, looks up the full 32-byte root hash
// returns true if found
bool lookUpFullRootHash(uint8_t *partial_root_hash, size_t partial_root_hash_len, uint8_t *full_root_hash);
// update the mappings table to set the chain count for the given root hash
void setChainCount(uint8_t *, size_t, uint32_t);
// get the chain count for the given root hash
uint32_t getChainCount(uint8_t *, size_t);
// get the scratch count for the given root hash
uint32_t getScratchCount(uint8_t *, size_t);
// get the canon scratch count for the given root hash
uint32_t getCanonScratchCount(uint8_t *, size_t);
link_object getLinkFromPositionFromTip(uint32_t, uint8_t *, size_t);
void pruneScratchQueue();
void trimOldestLink(uint8_t *, size_t);
void clearChain(uint8_t *, size_t);
void recalculateMessageHash(link_object &);
// given a link object with a payload and other fields, recalculates the message hash
// returns true if a match
bool recalculateHash(link_object &, uint8_t *, size_t, uint8_t *, size_t);
void updatePeers(const meshtastic_MeshPacket &, meshtastic_StoreForwardPlusPlus_SFPP_message_type);
void maybeMoveFromCanonScratch(uint8_t *, size_t);
void addToCanonScratch(link_object &);
link_object getfromCanonScratch(uint8_t *, size_t);
void removeFromCanonScratch(uint8_t *, size_t);
void clearCanonScratch(uint8_t *, size_t, uint32_t);
bool isInCanonScratch(uint8_t *, size_t);
void logLinkObject(link_object &);
// Track if we have a scheduled runOnce pending
// useful to not accudentally delay a scheduled runOnce
bool pendingRun = false;
// Once we have multiple chain types, we can extend this
enum chain_types {
channel_chain = 0,
};
uint32_t rebroadcastTimeout = 3600; // Messages older than this (in seconds) will not be rebroadcast
bool doing_split_send = false;
link_object split_link_out;
bool doing_split_receive = false;
link_object split_link_in;
bool did_announce_last = false;
uint32_t texts_rebroadcast = 0;
uint32_t links_speculated = 0;
uint32_t canon_announces = 0;
uint32_t links_requested = 0;
uint32_t links_provided = 0;
uint32_t links_added = 0;
uint32_t links_from_canon_scratch = 0;
uint32_t links_from_scratch = 0;
uint32_t split_links_sent = 0;
uint32_t split_links_received = 0;
uint32_t links_pruned = 0;
uint32_t scratch_timed_out = 0;
uint32_t sent_from_scratch = 0;
uint32_t received_from_scratch = 0;
};
extern StoreForwardPlusPlusModule *storeForwardPlusPlusModule;
#endif

View File

@@ -177,6 +177,7 @@ void portduinoSetup()
if (portduino_config.force_simradio == true) {
portduino_config.lora_module = use_simradio;
portduino_config.sfpp_enabled = false;
} else if (configPath != nullptr) {
if (loadConfig(configPath)) {
if (!yamlOnly)
@@ -859,6 +860,28 @@ bool loadConfig(const char *configPath)
}
}
if (yamlConfig["StoreAndForward"]) {
portduino_config.sfpp_stratum0 = (yamlConfig["StoreAndForward"]["Stratum0"]).as<bool>(false);
portduino_config.sfpp_enabled = (yamlConfig["StoreAndForward"]["Enabled"]).as<bool>(true);
portduino_config.sfpp_db_path = (yamlConfig["StoreAndForward"]["DBPath"]).as<std::string>("/var/lib/meshtasticd/");
portduino_config.sfpp_initial_sync = (yamlConfig["StoreAndForward"]["InitialSync"]).as<int>(10);
portduino_config.sfpp_hops = (yamlConfig["StoreAndForward"]["Hops"]).as<int>(3);
portduino_config.sfpp_announce_interval = (yamlConfig["StoreAndForward"]["AnnounceInterval"]).as<int>(5);
portduino_config.sfpp_max_chain = (yamlConfig["StoreAndForward"]["MaxChainLength"]).as<uint32_t>(1000);
portduino_config.sfpp_backlog_limit = (yamlConfig["StoreAndForward"]["BacklogLimit"]).as<uint32_t>(100);
portduino_config.sfpp_steal_port = (yamlConfig["StoreAndForward"]["StealPort"]).as<bool>(false);
}
if (yamlConfig["Routing"]) {
if (yamlConfig["Routing"]["WhitelistPorts"]) {
portduino_config.whitelist_ports = (yamlConfig["Routing"]["WhitelistPorts"]).as<std::vector<int>>();
if (portduino_config.whitelist_ports.size() > 0) {
portduino_config.whitelist_enabled = true;
}
}
if (yamlConfig["Routing"]["NoHopPorts"]) {
portduino_config.nohop_ports = (yamlConfig["Routing"]["NoHopPorts"]).as<std::vector<int>>();
}
}
if (yamlConfig["General"]) {
portduino_config.MaxNodes = (yamlConfig["General"]["MaxNodes"]).as<int>(200);
portduino_config.maxtophone = (yamlConfig["General"]["MaxMessageQueue"]).as<int>(100);

View File

@@ -171,6 +171,26 @@ extern struct portduino_config_struct {
int configDisplayMode = 0;
bool has_configDisplayMode = false;
// Store and Forward++
std::string sfpp_db_path = "/var/lib/meshtasticd/";
bool sfpp_stratum0 = false;
bool sfpp_enabled = true;
bool sfpp_steal_port = false;
int sfpp_initial_sync = 10;
int sfpp_hops = 3;
int sfpp_announce_interval = 5; // minutes
uint32_t sfpp_max_chain = 1000;
uint32_t sfpp_backlog_limit = 100;
// allowed root hashes
// upstream node
// Are we allowing unknown channel hashes? Does this even make sense?
// Allow DMs
// Routing
bool whitelist_enabled = false;
std::vector<int> whitelist_ports = {};
std::vector<int> nohop_ports = {};
// General
std::string mac_address = "";
bool mac_address_explicit = false;
@@ -504,6 +524,29 @@ extern struct portduino_config_struct {
out << YAML::EndMap; // Config
}
// StoreAndForward
if (sfpp_enabled) {
out << YAML::Key << "StoreAndForward" << YAML::Value << YAML::BeginMap;
out << YAML::Key << "Enabled" << YAML::Value << sfpp_enabled;
out << YAML::Key << "DBPath" << YAML::Value << sfpp_db_path;
out << YAML::Key << "Stratum0" << YAML::Value << sfpp_stratum0;
out << YAML::Key << "InitialSync" << YAML::Value << sfpp_initial_sync;
out << YAML::Key << "Hops" << YAML::Value << sfpp_hops;
out << YAML::Key << "AnnounceInterval" << YAML::Value << sfpp_announce_interval;
out << YAML::Key << "BacklogLimit" << YAML::Value << sfpp_backlog_limit;
out << YAML::Key << "MaxChainLength" << YAML::Value << sfpp_max_chain;
out << YAML::Key << "StealPort" << YAML::Value << sfpp_steal_port;
out << YAML::EndMap; // StoreAndForward
}
// Routing
if (whitelist_enabled || nohop_ports.size() > 0) {
out << YAML::Key << "Routing" << YAML::Value << YAML::BeginMap;
out << YAML::Key << "WhitelistPorts" << YAML::Value << whitelist_ports;
out << YAML::Key << "NoHopPorts" << YAML::Value << nohop_ports;
out << YAML::EndMap; // Routing
}
// General
out << YAML::Key << "General" << YAML::Value << YAML::BeginMap;
if (config_directory != "")

View File

@@ -9,7 +9,7 @@ lib_deps =
# renovate: datasource=custom.pio depName=Melopero RV3028 packageName=melopero/library/Melopero RV3028
melopero/Melopero RV3028@1.2.0
build_src_filter = ${portduino_base.build_src_filter}
build_src_filter = ${portduino_base.build_src_filter} +<modules/Native/>
[env:native]
extends = native_base
@@ -20,6 +20,7 @@ build_flags = ${native_base.build_flags}
!pkg-config --libs openssl --silence-errors || :
!pkg-config --cflags --libs sdl2 --silence-errors || :
!pkg-config --cflags --libs libbsd-overlay --silence-errors || :
!pkg-config --cflags --libs sqlite3 --silence-errors || :
[env:native-tft]
extends = native_base
@@ -46,6 +47,7 @@ build_flags = ${native_base.build_flags} -Os -lX11 -linput -lxkbcommon -ffunctio
!pkg-config --libs openssl --silence-errors || :
!pkg-config --cflags --libs sdl2 --silence-errors || :
!pkg-config --cflags --libs libbsd-overlay --silence-errors || :
!pkg-config --cflags --libs sqlite3 --silence-errors || :
build_src_filter =
${native_base.build_src_filter}
@@ -75,6 +77,7 @@ build_flags = ${native_base.build_flags} -Os -ffunction-sections -fdata-sections
!pkg-config --libs libulfius --silence-errors || :
!pkg-config --libs openssl --silence-errors || :
!pkg-config --cflags --libs libbsd-overlay --silence-errors || :
!pkg-config --cflags --libs sqlite3 --silence-errors || :
build_src_filter =
${native_base.build_src_filter}
@@ -108,6 +111,7 @@ build_flags = ${native_base.build_flags} -O0 -fsanitize=address -lX11 -linput -l
!pkg-config --libs libulfius --silence-errors || :
!pkg-config --libs openssl --silence-errors || :
!pkg-config --cflags --libs libbsd-overlay --silence-errors || :
!pkg-config --cflags --libs sqlite3 --silence-errors || :
build_src_filter = ${env:native-tft.build_src_filter}
[env:coverage]