Merge pull request #9262 from meshtastic/develop

Develop to master
2026-01-17 07:17:32 +00:00 · 2026-01-11 16:27:58 -06:00
parent 8cb8540ef6 8cf8fbb8e0
commit 3fabd57381
292 changed files with 11002 additions and 2716 deletions
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -0,0 +1,314 @@
+# Meshtastic Firmware - Copilot Instructions
+
+This document provides context and guidelines for AI assistants working with the Meshtastic firmware codebase.
+
+## Project Overview
+
+Meshtastic is an open-source LoRa mesh networking project for long-range, low-power communication without relying on internet or cellular infrastructure. The firmware enables text messaging, location sharing, and telemetry over a decentralized mesh network.
+
+### Supported Hardware Platforms
+
+- **ESP32** (ESP32, ESP32-S3, ESP32-C3) - Most common platform
+- **nRF52** (nRF52840, nRF52833) - Low power Nordic chips
+- **RP2040/RP2350** - Raspberry Pi Pico variants
+- **STM32WL** - STM32 with integrated LoRa
+- **Linux/Portduino** - Native Linux builds (Raspberry Pi, etc.)
+
+### Supported Radio Chips
+
+- **SX1262/SX1268** - Sub-GHz LoRa (868/915 MHz regions)
+- **SX1280** - 2.4 GHz LoRa
+- **LR1110/LR1120/LR1121** - Wideband radios (sub-GHz and 2.4 GHz capable, but not simultaneously)
+- **RF95** - Legacy RFM95 modules
+- **LLCC68** - Low-cost LoRa
+
+### MQTT Integration
+
+MQTT provides a bridge between Meshtastic mesh networks and the internet, enabling nodes with network connectivity to share messages with remote meshes or external services.
+
+#### Key Components
+
+- **`src/mqtt/MQTT.cpp`** - Main MQTT client singleton, handles connection and message routing
+- **`src/mqtt/ServiceEnvelope.cpp`** - Protobuf wrapper for mesh packets sent over MQTT
+- **`moduleConfig.mqtt`** - MQTT module configuration
+
+#### MQTT Topic Structure
+
+Messages are published/subscribed using a hierarchical topic format:
+
+```
+{root}/{channel_id}/{gateway_id}
+```
+
+- `root` - Configurable prefix (default: `msh`)
+- `channel_id` - Channel name/identifier
+- `gateway_id` - Node ID of the publishing gateway
+
+#### Configuration Defaults (from `Default.h`)
+
+```cpp
+#define default_mqtt_address "mqtt.meshtastic.org"
+#define default_mqtt_username "meshdev"
+#define default_mqtt_password "large4cats"
+#define default_mqtt_root "msh"
+#define default_mqtt_encryption_enabled true
+#define default_mqtt_tls_enabled false
+```
+
+#### Key Concepts
+
+- **Uplink** - Mesh packets sent TO the MQTT broker (controlled by `uplink_enabled` per channel)
+- **Downlink** - MQTT messages received and injected INTO the mesh (controlled by `downlink_enabled` per channel)
+- **Encryption** - When `encryption_enabled` is true, only encrypted packets are sent; plaintext JSON is disabled
+- **ServiceEnvelope** - Protobuf wrapper containing packet + channel_id + gateway_id for routing
+- **JSON Support** - Optional JSON encoding for integration with external systems (disabled on nRF52 by default)
+
+#### PKI Messages
+
+PKI (Public Key Infrastructure) messages have special handling:
+
+- Accepted on a special "PKI" channel
+- Allow encrypted DMs between nodes that discovered each other on downlink-enabled channels
+
+## Project Structure
+
+```
+firmware/
+├── src/                    # Main source code
+│   ├── main.cpp           # Application entry point
+│   ├── mesh/              # Core mesh networking
+│   │   ├── NodeDB.*       # Node database management
+│   │   ├── Router.*       # Packet routing
+│   │   ├── Channels.*     # Channel management
+│   │   ├── *Interface.*   # Radio interface implementations
+│   │   └── generated/     # Protobuf generated code
+│   ├── modules/           # Feature modules (Position, Telemetry, etc.)
+│   ├── gps/               # GPS handling
+│   ├── graphics/          # Display drivers and UI
+│   ├── platform/          # Platform-specific code
+│   ├── input/             # Input device handling
+│   └── concurrency/       # Threading utilities
+├── variants/              # Hardware variant definitions
+│   ├── esp32/            # ESP32 variants
+│   ├── esp32s3/          # ESP32-S3 variants
+│   ├── nrf52/            # nRF52 variants
+│   └── rp2xxx/           # RP2040/RP2350 variants
+├── protobufs/            # Protocol buffer definitions
+├── boards/               # Custom PlatformIO board definitions
+└── bin/                  # Build and utility scripts
+```
+
+## Coding Conventions
+
+### General Style
+
+- Follow existing code style - run `trunk fmt` before commits
+- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
+- Use `assert()` for invariants that should never fail
+
+### Naming Conventions
+
+- Classes: `PascalCase` (e.g., `PositionModule`, `NodeDB`)
+- Functions/Methods: `camelCase` (e.g., `sendOurPosition`, `getNodeNum`)
+- Constants/Defines: `UPPER_SNAKE_CASE` (e.g., `MAX_INTERVAL`, `ONE_DAY`)
+- Member variables: `camelCase` (e.g., `lastGpsSend`, `nodeDB`)
+- Config defines: `USERPREFS_*` for user-configurable options
+
+### Key Patterns
+
+#### Module System
+
+Modules inherit from `MeshModule` or `ProtobufModule<T>` and implement:
+
+- `handleReceivedProtobuf()` - Process incoming packets
+- `allocReply()` - Generate response packets
+- `runOnce()` - Periodic task execution (returns next run interval in ms)
+
+```cpp
+class MyModule : public ProtobufModule<meshtastic_MyMessage>
+{
+  protected:
+    virtual bool handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshtastic_MyMessage *msg) override;
+    virtual int32_t runOnce() override;
+};
+```
+
+#### Configuration Access
+
+- `config.*` - Device configuration (LoRa, position, power, etc.)
+- `moduleConfig.*` - Module-specific configuration
+- `channels.*` - Channel configuration and management
+
+#### Default Values
+
+Use the `Default` class helpers in `src/mesh/Default.h`:
+
+- `Default::getConfiguredOrDefaultMs(configured, default)` - Returns ms, using default if configured is 0
+- `Default::getConfiguredOrMinimumValue(configured, min)` - Enforces minimum values
+- `Default::getConfiguredOrDefaultMsScaled(configured, default, numNodes)` - Scales based on network size
+
+#### Thread Safety
+
+- Use `concurrency::Lock` for mutex protection
+- Radio SPI access uses `SPILock`
+- Prefer `OSThread` for background tasks
+
+### Hardware Variants
+
+Each hardware variant has:
+
+- `variant.h` - Pin definitions and hardware capabilities
+- `platformio.ini` - Build configuration
+- Optional: `pins_arduino.h`, `rfswitch.h`
+
+Key defines in variant.h:
+
+```cpp
+#define USE_SX1262          // Radio chip selection
+#define HAS_GPS 1           // Hardware capabilities
+#define LORA_CS 36          // Pin assignments
+#define SX126X_DIO1 14      // Radio-specific pins
+```
+
+### Protobuf Messages
+
+- Defined in `protobufs/meshtastic/*.proto`
+- Generated code in `src/mesh/generated/`
+- Regenerate with `bin/regen-protos.sh`
+- Message types prefixed with `meshtastic_`
+
+### Conditional Compilation
+
+```cpp
+#if !MESHTASTIC_EXCLUDE_GPS        // Feature exclusion
+#ifdef ARCH_ESP32                   // Architecture-specific
+#if defined(USE_SX1262)            // Radio-specific
+#ifdef HAS_SCREEN                   // Hardware capability
+#if USERPREFS_EVENT_MODE           // User preferences
+```
+
+## Build System
+
+Uses **PlatformIO** with custom scripts:
+
+- `bin/platformio-pre.py` - Pre-build script
+- `bin/platformio-custom.py` - Custom build logic
+
+Build commands:
+
+```bash
+pio run -e tbeam              # Build specific target
+pio run -e tbeam -t upload    # Build and upload
+pio run -e native             # Build native/Linux version
+```
+
+## Common Tasks
+
+### Adding a New Module
+
+1. Create `src/modules/MyModule.cpp` and `.h`
+2. Inherit from appropriate base class
+3. Register in `src/modules/Modules.cpp`
+4. Add protobuf messages if needed in `protobufs/`
+
+### Adding a New Hardware Variant
+
+1. Create directory under `variants/<arch>/<name>/`
+2. Add `variant.h` with pin definitions
+3. Add `platformio.ini` with build config
+4. Reference common configs with `extends`
+
+### Modifying Configuration Defaults
+
+- Check `src/mesh/Default.h` for default value defines
+- Check `src/mesh/NodeDB.cpp` for initialization logic
+- Consider `isDefaultChannel()` checks for public channel restrictions
+
+## Important Considerations
+
+### Traffic Management
+
+The mesh network has limited bandwidth. When modifying broadcast intervals:
+
+- Respect minimum intervals on default/public channels
+- Use `Default::getConfiguredOrMinimumValue()` to enforce minimums
+- Consider `numOnlineNodes` scaling for congestion control
+
+### Power Management
+
+Many devices are battery-powered:
+
+- Use `IF_ROUTER(routerVal, normalVal)` for role-based defaults
+- Check `config.power.is_power_saving` for power-saving modes
+- Implement proper `sleep()` methods in radio interfaces
+
+### Channel Security
+
+- `channels.isDefaultChannel(index)` - Check if using default/public settings
+- Default channels get stricter rate limits to prevent abuse
+- Private channels may have relaxed limits
+
+## GitHub Actions CI/CD
+
+The project uses GitHub Actions extensively for CI/CD. Key workflows are in `.github/workflows/`:
+
+### Core CI Workflows
+
+- **`main_matrix.yml`** - Main CI pipeline, runs on push to `master`/`develop` and PRs
+  - Uses `bin/generate_ci_matrix.py` to dynamically generate build targets
+  - Builds all supported hardware variants
+  - PRs build a subset (`--level pr`) for faster feedback
+
+- **`trunk_check.yml`** - Code quality checks on PRs
+  - Runs Trunk.io for linting and formatting
+  - Must pass before merge
+
+- **`tests.yml`** - End-to-end and hardware tests
+  - Runs daily on schedule
+  - Includes native tests and hardware-in-the-loop testing
+
+- **`test_native.yml`** - Native platform unit tests
+  - Runs `pio test -e native`
+
+### Release Workflows
+
+- **`release_channels.yml`** - Triggered on GitHub release publish
+  - Builds Docker images
+  - Packages for PPA (Ubuntu), OBS (openSUSE), and COPR (Fedora)
+  - Handles Alpha/Beta/Stable release channels
+
+- **`nightly.yml`** - Nightly builds from develop branch
+
+- **`docker_build.yml`** / **`docker_manifest.yml`** - Docker image builds
+
+### Build Matrix Generation
+
+The CI uses `bin/generate_ci_matrix.py` to dynamically select which targets to build:
+
+```bash
+# Generate full build matrix
+./bin/generate_ci_matrix.py all
+
+# Generate PR-level matrix (subset for faster builds)
+./bin/generate_ci_matrix.py all --level pr
+```
+
+Variants can specify their support level in `platformio.ini`:
+
+- `custom_meshtastic_support_level = 1` - Actively supported, built on every PR
+- `custom_meshtastic_support_level = 2` - Supported, built on merge to main branches
+- `board_level = extra` - Extra builds, only on full releases
+
+### Running Workflows Locally
+
+Most workflows can be triggered manually via `workflow_dispatch` for testing.
+
+## Testing
+
+- Unit tests in `test/` directory
+- Run with `pio test -e native`
+- Use `bin/test-simulator.sh` for simulation testing
+
+## Resources
+
+- [Documentation](https://meshtastic.org/docs/)
--- a/.github/workflows/build_firmware.yml
+++ b/.github/workflows/build_firmware.yml
@@ -21,7 +21,7 @@ jobs:
    # Use 'arctastic' self-hosted runner pool when building in the main repo
    runs-on: ${{ github.repository_owner == 'meshtastic' && 'arctastic' || 'ubuntu-latest' }}
    outputs:
-      artifact-id: ${{ steps.upload.outputs.artifact-id }}
+      artifact-id: ${{ steps.upload-firmware.outputs.artifact-id }}
    steps:
      - uses: actions/checkout@v6
        with:
@@ -29,23 +29,6 @@ jobs:
          ref: ${{github.event.pull_request.head.ref}}
          repository: ${{github.event.pull_request.head.repo.full_name}}

-      - name: Set OTA firmware source and target
-        if: startsWith(inputs.platform, 'esp32')
-        id: ota_dir
-        env:
-          PIO_PLATFORM: ${{ inputs.platform }}
-        run: |
-          if [ "$PIO_PLATFORM" = "esp32s3" ]; then
-            echo "src=firmware-s3.bin" >> $GITHUB_OUTPUT
-            echo "tgt=release/bleota-s3.bin" >> $GITHUB_OUTPUT
-          elif [ "$PIO_PLATFORM" = "esp32c3" ] || [ "$PIO_PLATFORM" = "esp32c6" ]; then
-            echo "src=firmware-c3.bin" >> $GITHUB_OUTPUT
-            echo "tgt=release/bleota-c3.bin" >> $GITHUB_OUTPUT
-          elif [ "$PIO_PLATFORM" = "esp32" ]; then
-            echo "src=firmware.bin" >> $GITHUB_OUTPUT
-            echo "tgt=release/bleota.bin" >> $GITHUB_OUTPUT
-          fi
-
      - name: Build ${{ inputs.platform }}
        id: build
        uses: meshtastic/gh-action-firmware@main
@@ -53,8 +36,66 @@ jobs:
          pio_platform: ${{ inputs.platform }}
          pio_env: ${{ inputs.pio_env }}
          pio_target: build
-          ota_firmware_source: ${{ steps.ota_dir.outputs.src || '' }}
-          ota_firmware_target: ${{ steps.ota_dir.outputs.tgt || '' }}
+
+      - name: ESP32 - Download Unified OTA firmware
+        # Currently only esp32 and esp32s3 use the unified ota
+        if: inputs.platform == 'esp32' || inputs.platform == 'esp32s3'
+        id: dl-ota-unified
+        env:
+          PIO_PLATFORM: ${{ inputs.platform }}
+          PIO_ENV: ${{ inputs.pio_env }}
+          OTA_URL: https://github.com/meshtastic/esp32-unified-ota/releases/latest/download/mt-${{ inputs.platform }}-ota.bin
+        working-directory: release
+        run: |
+          curl -L -o "mt-$PIO_PLATFORM-ota.bin" $OTA_URL
+
+      - name: ESP32-C* - Download BLE-Only OTA firmware
+        if: inputs.platform == 'esp32c3' || inputs.platform == 'esp32c6'
+        id: dl-ota-ble
+        env:
+          PIO_ENV: ${{ inputs.pio_env }}
+          OTA_URL: https://github.com/meshtastic/firmware-ota/releases/latest/download/firmware-c3.bin
+        working-directory: release
+        run: |
+          curl -L -o bleota-c3.bin $OTA_URL
+
+      - name: Update manifest with OTA file
+        if: inputs.platform == 'esp32' || inputs.platform == 'esp32s3' || inputs.platform == 'esp32c3' || inputs.platform == 'esp32c6'
+        working-directory: release
+        env:
+          PIO_PLATFORM: ${{ inputs.platform }}
+        run: |
+          # Determine OTA filename based on platform
+          if [[ "$PIO_PLATFORM" == "esp32" || "$PIO_PLATFORM" == "esp32s3" ]]; then
+            OTA_FILE="mt-${PIO_PLATFORM}-ota.bin"
+          else
+            OTA_FILE="bleota-c3.bin"
+          fi
+
+          # Check if OTA file exists
+          if [[ ! -f "$OTA_FILE" ]]; then
+            echo "OTA file $OTA_FILE not found, skipping manifest update"
+            exit 0
+          fi
+
+          # Calculate MD5 and size
+          if command -v md5sum &> /dev/null; then
+            OTA_MD5=$(md5sum "$OTA_FILE" | cut -d' ' -f1)
+          else
+            OTA_MD5=$(md5 -q "$OTA_FILE")
+          fi
+          OTA_SIZE=$(stat -f%z "$OTA_FILE" 2>/dev/null || stat -c%s "$OTA_FILE")
+
+          # Find and update manifest file
+          for manifest in firmware-*.mt.json; do
+            if [[ -f "$manifest" ]]; then
+              echo "Updating $manifest with $OTA_FILE (md5: $OTA_MD5, size: $OTA_SIZE)"
+              # Add OTA entry to files array if not already present
+              jq --arg name "$OTA_FILE" --arg md5 "$OTA_MD5" --argjson bytes "$OTA_SIZE" \
+                'if .files | map(select(.name == $name)) | length == 0 then .files += [{"name": $name, "md5": $md5, "bytes": $bytes}] else . end' \
+                "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
+            fi
+          done

      - name: Job summary
        env:
@@ -71,7 +112,7 @@ jobs:

      - name: Store binaries as an artifact
        uses: actions/upload-artifact@v6
-        id: upload
+        id: upload-firmware
        with:
          name: firmware-${{ inputs.platform }}-${{ inputs.pio_env }}-${{ inputs.version }}
          overwrite: true
@@ -84,3 +125,12 @@ jobs:
            release/*.zip
            release/device-*.sh
            release/device-*.bat
+
+      - name: Store manifests as an artifact
+        uses: actions/upload-artifact@v6
+        id: upload-manifest
+        with:
+          name: manifest-${{ inputs.platform }}-${{ inputs.pio_env }}-${{ inputs.version }}
+          overwrite: true
+          path: |
+            release/*.mt.json
--- a/.github/workflows/first_time_contributor.yml
+++ b/.github/workflows/first_time_contributor.yml
@@ -0,0 +1,47 @@
+name: Welcome First-Time Contributor
+
+on:
+  issues:
+    types: opened
+  pull_request_target:
+    types: opened
+
+permissions: {}
+
+jobs:
+  welcome:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write # Required to post comments and labels on issues
+      pull-requests: write # Required to post comments and labels on PRs
+    steps:
+      - uses: plbstl/first-contribution@v4
+        with:
+          labels: first-contribution
+          issue-opened-msg: |
+            ### @{fc-author}, Welcome to Meshtastic! :wave:
+
+            Thanks for opening your first issue. If it's helpful, an easy way
+            to get logs is the "Open Serial Monitor" button on the [Web Flasher](https://flasher.meshtastic.org).
+
+            If you have ideas for features, note that we often debate big ideas
+            in the [discussions tab](https://github.com/meshtastic/firmware/discussions/categories/ideas)
+            first. This tracker tends to be for ideas that have community
+            consensus and a clear implementation.
+
+            We're very active [on discord](https://discord.com/invite/meshtastic),
+            especially the \#firmware and \#alphanauts-testing channels. If you'll
+            be around for a while, we'd love to see you there!
+
+            Welcome to the community! :heart:
+
+          pr-opened-msg: |
+            ### @{fc-author}, Welcome to Meshtastic!
+
+            Thanks for opening your first pull request. We really appreciate it.
+
+            We discuss work as a team in discord, please join us in the [#firmware channel](https://discord.com/invite/meshtastic).
+            There's a big backlog of patches at the moment. If you have time,
+            please help us with some code review and testing of [other PRs](https://github.com/meshtastic/firmware/pulls)!
+
+            Welcome to the team :smile:
--- a/.github/workflows/main_matrix.yml
+++ b/.github/workflows/main_matrix.yml
@@ -8,7 +8,9 @@ on:
    branches:
      - master
      - develop
+      - pioarduino # Remove when merged // use `feature/` in the future.
      - event/*
+      - feature/*
    paths-ignore:
      - "**.md"
      - version.properties
@@ -18,7 +20,9 @@ on:
    branches:
      - master
      - develop
+      - pioarduino # Remove when merged // use `feature/` in the future.
      - event/*
+      - feature/*
    paths-ignore:
      - "**.md"
      #- "**.yml"
@@ -233,6 +237,48 @@ jobs:
          description: "Download firmware-${{matrix.arch}}-${{ needs.version.outputs.long }}.zip. This artifact will be available for 90 days from creation"
          github-token: ${{ secrets.GITHUB_TOKEN }}

+  shame:
+    if: github.repository == 'meshtastic/firmware'
+    continue-on-error: true
+    runs-on: ubuntu-latest
+    needs: [build]
+    steps:
+      - uses: actions/checkout@v6
+        if: github.event_name == 'pull_request_target'
+        with:
+          filter: blob:none # means we download all the git history but none of the commit (except ones with checkout like the head)
+          fetch-depth: 0
+      - name: Download the current manifests
+        uses: actions/download-artifact@v7
+        with:
+          path: ./manifests-new/
+          pattern: manifest-*
+          merge-multiple: true
+      - name: Upload combined manifests for later commit and global stats crunching.
+        uses: actions/upload-artifact@v6
+        id: upload-manifest
+        with:
+          name: manifests-${{ github.sha }}
+          overwrite: true
+          path: manifests-new/*.mt.json
+      - name: Find the merge base
+        if: github.event_name == 'pull_request_target'
+        run: echo "MERGE_BASE=$(git merge-base "origin/$base" "$head")" >> $GITHUB_ENV
+        env:
+          base: ${{ github.base_ref }}
+          head: ${{ github.sha }}
+      # Currently broken (for-loop through EVERY artifact -- rate limiting)
+      # - name: Download the old manifests
+      #   if: github.event_name == 'pull_request_target'
+      #   run: gh run download -R "$repo" --name "manifests-$merge_base" --dir manifest-old/
+      #   env:
+      #     GH_TOKEN: ${{ github.token }}
+      #     merge_base: ${{ env.MERGE_BASE }}
+      #     repo: ${{ github.repository }}
+      # - name: Do scan and post comment
+      #   if: github.event_name == 'pull_request_target'
+      #   run: python3 bin/shame.py ${{ github.event.pull_request.number }} manifests-old/ manifests-new/
+
  release-artifacts:
    runs-on: ubuntu-latest
    if: ${{ github.event_name == 'workflow_dispatch' && github.repository == 'meshtastic/firmware' }}
--- a/.github/workflows/update_protobufs.yml
+++ b/.github/workflows/update_protobufs.yml
@@ -16,7 +16,7 @@ jobs:
          submodules: true

      - name: Update submodule
-        if: ${{ github.ref == 'refs/heads/master' }}
+        if: ${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/develop' }}
        run: |
          git submodule update --remote protobufs