Merge pull request #166 from geeksville/usb

Changes to support USB

README.md

@@ -4,7 +4,7 @@ This is the device side code for the [meshtastic.org](https://www.meshtastic.org
 
 ![Continuous Integration](https://github.com/meshtastic/Meshtastic-esp32/workflows/Continuous%20Integration/badge.svg)
 
-Meshtastic is a project that lets you use
+Meshtastic™ is a project that lets you use
 inexpensive GPS mesh radios as an extensible, super long battery life mesh GPS communicator. These radios are great for hiking, skiing, paragliding -
 essentially any hobby where you don't have reliable internet access. Each member of your private mesh can always see the location and distance of all other
 members and any text messages sent to your group chat.
@@ -14,14 +14,16 @@ will optionally work with your phone, but no phone is required.
 
 Typical time between recharging the radios should be about eight days.
 
-This project is currently early-alpha, but if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
+This project is is currently in beta-testing - if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
 
 This software is 100% open source and developed by a group of hobbyist experimenters. No warranty is provided, if you'd like to improve it - we'd love your help. Please post in the chat.
 
 ## Supported hardware
 
 We currently support three models of radios.
+
 - TTGO T-Beam
+
   - [T-Beam V1.0 w/ NEO-M8N](https://www.aliexpress.com/item/33047631119.html) (Recommended)
   - [T-Beam V1.0 w/ NEO-6M](https://www.aliexpress.com/item/33050391850.html)
   - 3D printable cases
@@ -34,9 +36,10 @@ We currently support three models of radios.
   - [3D Printable case](https://www.thingiverse.com/thing:3125854)
 
 **Make sure to get the frequency for your country**
-  - US/JP/AU/NZ - 915MHz
-  - CN - 470MHz
-  - EU - 870MHz
+
+- US/JP/AU/NZ - 915MHz
+- CN - 470MHz
+- EU - 868MHz, 433MHz
 
 Getting a version that includes a screen is optional, but highly recommended.
 
@@ -165,12 +168,7 @@ Hard resetting via RTS pin...
 
 # Meshtastic Android app
 
-The source code for the (optional) Meshtastic Android app is [here](https://github.com/meshtastic/Meshtastic-Android).
-
-Alpha test builds available by opting into our alpha test group. See (www.meshtastic.org) for instructions.
-
-If you don't want to live on the 'bleeding edge' you can opt-in to the beta-test or use the released version:
-[![Download at https://play.google.com/store/apps/details?id=com.geeksville.mesh](https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png)](https://play.google.com/store/apps/details?id=com.geeksville.mesh&referrer=utm_source%3Dgithub%26utm_medium%3Desp32-readme%26utm_campaign%3Dmeshtastic-esp32%2520readme%26anid%3Dadmob&pcampaignid=pcampaignidMKT-Other-global-all-co-prtnr-py-PartBadge-Mar2515-1)
+The companion (optional) Meshtastic Android app is [here](https://github.com/meshtastic/Meshtastic-Android). You can also download it on Google Play.
 
 # Python API
 

bin/version.sh

@@ -1,3 +1,3 @@
 
 
-export VERSION=0.6.7
+export VERSION=0.7.4

docs/README.md

@@ -1,6 +1,6 @@
 # What is Meshtastic?
 
-Meshtastic is a project that lets you use
+Meshtastic™ is a project that lets you use
 inexpensive (\$30 ish) GPS radios as an extensible, long battery life, secure, mesh GPS communicator. These radios are great for hiking, skiing, paragliding - essentially any hobby where you don't have reliable internet access. Each member of your private mesh can always see the location and distance of all other members and any text messages sent to your group chat.
 
 The radios automatically create a mesh to forward packets as needed, so everyone in the group can receive messages from even the furthest member. The radios will optionally work with your phone, but no phone is required.
@@ -24,14 +24,14 @@ Not all of these features are fully implemented yet - see **important** disclaim
 - Very long battery life (should be about eight days with the beta software)
 - Built in GPS and [LoRa](https://en.wikipedia.org/wiki/LoRa) radio, but we manage the radio automatically for you
 - Long range - a few miles per node but each node will forward packets as needed
+- Secure - channels are encrypted by AES256 (But see important disclaimers below wrt this feature)
 - Shows direction and distance to all members of your channel
 - Directed or broadcast text messages for channel members
 - Open and extensible codebase supporting multiple hardware vendors - no lock in to one vendor
-- Communication API for bluetooth devices (such as our Android app) to use the mesh. So if you have some application that needs long range low power networking, this might work for you.
-- Eventually (within a couple of months) we should have a modified version of Signal that works with this project.
+- Communication API for bluetooth devices (such as our Android app) to use the mesh. An iOS application is in the works.  And [Meshtastic-python](https://pypi.org/project/meshtastic/) provides access from desktop computers.
 - Very easy sharing of private secured channels. Just share a special link or QR code with friends and they can join your encrypted mesh
 
-This project is currently in early alpha - if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
+This project is currently in beta testing but it is fairly stable and feature complete - if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
 
 This software is 100% open source and developed by a group of hobbyist experimenters. No warranty is provided, if you'd like to improve it - we'd love your help. Please post in the [forum](https://meshtastic.discourse.group/).
 
@@ -39,6 +39,7 @@ This software is 100% open source and developed by a group of hobbyist experimen
 
 Note: Updates are happening almost daily, only major updates are listed below. For more details see our forum.
 
+- 06/04/2020 - 0.6.7 Beta releases of both the application and the device code are released. Features are fairly solid now with a sizable number of users.
 - 04/28/2020 - 0.6.0 [Python API](https://pypi.org/project/meshtastic/) released. Makes it easy to use meshtastic devices as "zero config / just works" mesh transport adapters for other projects.
 - 04/20/2020 - 0.4.3 Pretty solid now both for the android app and the device code. Many people have donated translations and code. Probably going to call it a beta soon.
 - 03/03/2020 - 0.0.9 of the Android app and device code is released. Still an alpha but fairly functional.

docs/software/build-instructions.md

@@ -6,10 +6,11 @@ in these instructions I describe use of their command line tool.
 1. Purchase a suitable radio (see above)
 2. Install [PlatformIO](https://platformio.org/platformio-ide)
 3. Download this git repo and cd into it
-4. If you are outside the USA, edit [platformio.ini](/platformio.ini) to set the correct frequency range for your country. The line you need to change starts with "hw_version" and instructions are provided above that line. Options are provided for EU433, EU835, CN, JP and US. Pull-requests eagerly accepted for other countries.
-5. Plug the radio into your USB port
-6. Type "pio run --environment XXX -t upload" (This command will fetch dependencies, build the project and install it on the board via USB). For XXX, use the board type you have (either tbeam, heltec, ttgo-lora32-v1, ttgo-lora32-v2).
-7. Platform IO also installs a very nice VisualStudio Code based IDE, see their [tutorial](https://docs.platformio.org/en/latest/tutorials/espressif32/arduino_debugging_unit_testing.html) if you'd like to use it.
+4. Run `git submodule update --init --recursive` to pull in dependencies this project needs.
+5. If you are outside the USA, edit [platformio.ini](/platformio.ini) to set the correct frequency range for your country. The line you need to change starts with `hw_version` and instructions are provided above that line. Options are provided for `EU433`, `EU835`, `CN`, `JP` and `US` (default). Pull-requests eagerly accepted for other countries.
+6. Plug the radio into your USB port
+7. Type `pio run --environment XXX -t upload` (This command will fetch dependencies, build the project and install it on the board via USB). For XXX, use the board type you have (either `tbeam`, `heltec`, `ttgo-lora32-v1`, `ttgo-lora32-v2`).
+8. Platform IO also installs a very nice VisualStudio Code based IDE, see their [tutorial](https://docs.platformio.org/en/latest/tutorials/espressif32/arduino_debugging_unit_testing.html) if you'd like to use it.
 
 ## Decoding stack traces
 

docs/software/crypto.md

@@ -34,7 +34,4 @@ Note that for both stategies, sizes are measured in blocks and that an AES block
 
 ## Remaining todo
 
-- Make the packet numbers 32 bit
-- Confirm the packet #s are stored in flash across deep sleep (and otherwise in in RAM)
 - Have the app change the crypto key when the user generates a new channel
-- Implement for NRF52 [NRF52](https://infocenter.nordicsemi.com/topic/com.nordic.infocenter.sdk5.v15.0.0/lib_crypto_aes.html#sub_aes_ctr)

docs/software/nrf52-TODO.md

@@ -40,7 +40,6 @@ Needed to be fully functional at least at the same level of the ESP32 boards. At
 
 ## Items to be 'feature complete'
 
-- change packet numbers to be 32 bits
 - check datasheet about sx1262 temperature compensation
 - enable brownout detection and watchdog
 - stop polling for GPS characters, instead stay blocked on read in a thread
@@ -128,6 +127,7 @@ Nice ideas worth considering someday...
 - scheduleOSCallback doesn't work yet - it is way too fast (causes rapid polling of busyTx, high power draw etc...)
 - find out why we reboot while debugging - it was bluetooth/softdevice
 - make a file system implementation (preferably one that can see the files the bootloader also sees) - preferably https://github.com/adafruit/Adafruit_nRF52_Arduino/blob/master/libraries/InternalFileSytem/examples/Internal_ReadWrite/Internal_ReadWrite.ino else use https://infocenter.nordicsemi.com/topic/com.nordic.infocenter.sdk5.v15.3.0/lib_fds_usage.html?cp=7_5_0_3_55_3
+- change packet numbers to be 32 bits
 
 ```
 

docs/software/power.md

@@ -36,6 +36,10 @@ From lower to higher power consumption.
   onEntry: setBluetoothOn(true), screen.setOn(true)
   onExit: screen.setOn(false)
 
+- serial API usage (SERIAL) - Screen is on, device doesn't sleep, bluetooth off
+  onEntry: setBluetooth off, screen on
+  onExit:
+
 ## Behavior
 
 ### events that increase CPU activity
@@ -51,9 +55,11 @@ From lower to higher power consumption.
 - While in DARK/ON: If we receive EVENT_BLUETOOTH_PAIR we transition to ON and start our screen_on_secs timeout
 - While in NB/DARK/ON: If we receive EVENT_NODEDB_UPDATED we transition to ON (so the new screen can be shown)
 - While in DARK: While the phone talks to us over BLE (EVENT_CONTACT_FROM_PHONE) reset any sleep timers and stay in DARK (needed for bluetooth sw update and nice user experience if the user is reading/replying to texts)
+- while in LS/NB/DARK: if SERIAL_CONNECTED, go to serial
 
 ### events that decrease cpu activity
 
+- While in SERIAL: if SERIAL_DISCONNECTED, go to NB
 - While in ON: If PRESS event occurs, reset screen_on_secs timer and tell the screen to handle the pess
 - While in ON: If it has been more than screen_on_secs since a press, lower to DARK
 - While in DARK: If time since last contact by our phone exceeds phone_timeout_secs (15 minutes), we transition down into NB mode

gdbinit

@@ -6,8 +6,8 @@
 # mon exec SetMonModeDebug=1
 # mon exec SetMonModeVTableAddr=0x26000
 
-echo setting RTTAddr
-eval "monitor exec SetRTTAddr %p", &_SEGGER_RTT
+# echo setting RTTAddr
+# eval "monitor exec SetRTTAddr %p", &_SEGGER_RTT
 
 # the jlink debugger seems to want a pause after reset before we tell it to start running
 define restart

src/PowerFSM.cpp

@@ -104,6 +104,12 @@ static void darkEnter()
     screen.setOn(false);
 }
 
+static void serialEnter()
+{
+    setBluetoothEnable(false);
+    screen.setOn(true);
+}
+
 static void onEnter()
 {
     screen.setOn(true);
@@ -133,6 +139,7 @@ State stateSDS(sdsEnter, NULL, NULL, "SDS");
 State stateLS(lsEnter, lsIdle, lsExit, "LS");
 State stateNB(nbEnter, NULL, NULL, "NB");
 State stateDARK(darkEnter, NULL, NULL, "DARK");
+State stateSERIAL(serialEnter, NULL, NULL, "SERIAL");
 State stateBOOT(bootEnter, NULL, NULL, "BOOT");
 State stateON(onEnter, NULL, NULL, "ON");
 Fsm powerFSM(&stateBOOT);
@@ -148,7 +155,7 @@ void PowerFSM_setup()
 
     powerFSM.add_transition(&stateNB, &stateNB, EVENT_RECEIVED_PACKET, NULL, "Received packet, resetting win wake");
 
-    // Handle press events
+    // Handle press events - note: we ignore button presses when in API mode
     powerFSM.add_transition(&stateLS, &stateON, EVENT_PRESS, NULL, "Press");
     powerFSM.add_transition(&stateNB, &stateON, EVENT_PRESS, NULL, "Press");
     powerFSM.add_transition(&stateDARK, &stateON, EVENT_PRESS, NULL, "Press");
@@ -160,6 +167,7 @@ void PowerFSM_setup()
     powerFSM.add_transition(&stateNB, &stateSDS, EVENT_LOW_BATTERY, NULL, "LowBat");
     powerFSM.add_transition(&stateDARK, &stateSDS, EVENT_LOW_BATTERY, NULL, "LowBat");
     powerFSM.add_transition(&stateON, &stateSDS, EVENT_LOW_BATTERY, NULL, "LowBat");
+    powerFSM.add_transition(&stateSERIAL, &stateSDS, EVENT_LOW_BATTERY, NULL, "LowBat");
 
     powerFSM.add_transition(&stateDARK, &stateON, EVENT_BLUETOOTH_PAIR, NULL, "Bluetooth pairing");
     powerFSM.add_transition(&stateON, &stateON, EVENT_BLUETOOTH_PAIR, NULL, "Bluetooth pairing");
@@ -173,6 +181,13 @@ void PowerFSM_setup()
     powerFSM.add_transition(&stateDARK, &stateON, EVENT_RECEIVED_TEXT_MSG, NULL, "Received text");
     powerFSM.add_transition(&stateON, &stateON, EVENT_RECEIVED_TEXT_MSG, NULL, "Received text"); // restarts the sleep timer
 
+    powerFSM.add_transition(&stateLS, &stateSERIAL, EVENT_SERIAL_CONNECTED, NULL, "serial API");
+    powerFSM.add_transition(&stateNB, &stateSERIAL, EVENT_SERIAL_CONNECTED, NULL, "serial API");
+    powerFSM.add_transition(&stateDARK, &stateSERIAL, EVENT_SERIAL_CONNECTED, NULL, "serial API");
+    powerFSM.add_transition(&stateON, &stateSERIAL, EVENT_SERIAL_CONNECTED, NULL, "serial API");
+
+    powerFSM.add_transition(&stateSERIAL, &stateNB, EVENT_SERIAL_DISCONNECTED, NULL, "serial disconnect");
+
     powerFSM.add_transition(&stateDARK, &stateDARK, EVENT_CONTACT_FROM_PHONE, NULL, "Contact from phone");
 
     powerFSM.add_transition(&stateNB, &stateDARK, EVENT_PACKET_FOR_PHONE, NULL, "Packet for phone");

src/PowerFSM.h

@@ -14,6 +14,8 @@
 #define EVENT_NODEDB_UPDATED 8     // NodeDB has a big enough change that we think you should turn on the screen
 #define EVENT_CONTACT_FROM_PHONE 9 // the phone just talked to us over bluetooth
 #define EVENT_LOW_BATTERY 10 // Battery is critically low, go to sleep
+#define EVENT_SERIAL_CONNECTED 11
+#define EVENT_SERIAL_DISCONNECTED 12
 
 extern Fsm powerFSM;
 

src/SerialConsole.cpp

@@ -1,4 +1,5 @@
 #include "SerialConsole.h"
+#include "PowerFSM.h"
 #include "configuration.h"
 #include <Arduino.h>
 
@@ -26,12 +27,19 @@ void SerialConsole::init()
  */
 void SerialConsole::handleToRadio(const uint8_t *buf, size_t len)
 {
-    // Note: for the time being we could _allow_ debug printing to keep going out the console
-    // I _think_ this is okay because we currently only print debug msgs from loop() and we are only
-    // dispatching serial protobuf msgs from loop() as well.  When things are more threaded in the future this
-    // will need to change.
-    // setDestination(&noopPrint);
+    // Turn off debug serial printing once the API is activated, because other threads could print and corrupt packets
+    setDestination(&noopPrint);
     canWrite = true;
 
     StreamAPI::handleToRadio(buf, len);
 }
+
+/// Hookable to find out when connection changes
+void SerialConsole::onConnectionChanged(bool connected)
+{
+    if (connected) { // To prevent user confusion, turn off bluetooth while using the serial port api
+        powerFSM.trigger(EVENT_SERIAL_CONNECTED);
+    } else {
+        powerFSM.trigger(EVENT_SERIAL_DISCONNECTED);
+    }
+}

src/SerialConsole.h

@@ -26,6 +26,10 @@ class SerialConsole : public StreamAPI, public RedirectablePrint
             RedirectablePrint::write('\r');
         return RedirectablePrint::write(c);
     }
+
+  protected:
+    /// Hookable to find out when connection changes
+    virtual void onConnectionChanged(bool connected);
 };
 
 extern SerialConsole console;

src/esp32/BluetoothSoftwareUpdate.cpp

@@ -30,8 +30,6 @@ class TotalSizeCharacteristic : public CallbackCharacteristic
 
     void onWrite(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onWrite(c);
-
         LockGuard g(updateLock);
         // Check if there is enough to OTA Update
         uint32_t len = getValue32(c, 0);
@@ -67,8 +65,6 @@ class DataCharacteristic : public CallbackCharacteristic
 
     void onWrite(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onWrite(c);
-
         LockGuard g(updateLock);
         std::string value = c->getValue();
         uint32_t len = value.length();
@@ -92,8 +88,6 @@ class CRC32Characteristic : public CallbackCharacteristic
 
     void onWrite(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onWrite(c);
-
         LockGuard g(updateLock);
         uint32_t expectedCRC = getValue32(c, 0);
         uint32_t actualCRC = crc.finalize();

src/esp32/CallbackCharacteristic.h

@@ -1,33 +1,12 @@
 #pragma once
-#include "PowerFSM.h" // FIXME - someday I want to make this OTA thing a separate lb at at that point it can't touch this
 #include "BLECharacteristic.h"
-
-/**
- * This mixin just lets the power management state machine know the phone is still talking to us
- */
-class BLEKeepAliveCallbacks : public BLECharacteristicCallbacks
-{
-public:
-    void onRead(BLECharacteristic *c)
-    {
-        powerFSM.trigger(EVENT_CONTACT_FROM_PHONE);
-    }
-
-    void onWrite(BLECharacteristic *c)
-    {
-        powerFSM.trigger(EVENT_CONTACT_FROM_PHONE);
-    }
-};
+#include "PowerFSM.h" // FIXME - someday I want to make this OTA thing a separate lb at at that point it can't touch this
 
 /**
  * A characterstic with a set of overridable callbacks
  */
-class CallbackCharacteristic : public BLECharacteristic, public BLEKeepAliveCallbacks
+class CallbackCharacteristic : public BLECharacteristic, public BLECharacteristicCallbacks
 {
-public:
-    CallbackCharacteristic(const char *uuid, uint32_t btprops)
-        : BLECharacteristic(uuid, btprops)
-    {
-        setCallbacks(this);
-    }
+  public:
+    CallbackCharacteristic(const char *uuid, uint32_t btprops) : BLECharacteristic(uuid, btprops) { setCallbacks(this); }
 };

src/esp32/MeshBluetoothService.cpp

@@ -23,9 +23,6 @@ static CallbackCharacteristic *meshFromNumCharacteristic;
 
 BLEService *meshService;
 
-// If defined we will also support the old API
-#define SUPPORT_OLD_BLE_API
-
 class BluetoothPhoneAPI : public PhoneAPI
 {
     /**
@@ -58,17 +55,12 @@ class ProtobufCharacteristic : public CallbackCharacteristic
 
     void onRead(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onRead(c);
         size_t numbytes = pb_encode_to_bytes(trBytes, sizeof(trBytes), fields, my_struct);
         DEBUG_MSG("pbread from %s returns %d bytes\n", c->getUUID().toString().c_str(), numbytes);
         c->setValue(trBytes, numbytes);
     }
 
-    void onWrite(BLECharacteristic *c)
-    {
-        BLEKeepAliveCallbacks::onWrite(c);
-        writeToDest(c, my_struct);
-    }
+    void onWrite(BLECharacteristic *c) { writeToDest(c, my_struct); }
 
   protected:
     /// like onWrite, but we provide an different destination to write to, for use by subclasses that
@@ -83,112 +75,6 @@ class ProtobufCharacteristic : public CallbackCharacteristic
     }
 };
 
-#ifdef SUPPORT_OLD_BLE_API
-class NodeInfoCharacteristic : public BLECharacteristic, public BLEKeepAliveCallbacks
-{
-  public:
-    NodeInfoCharacteristic()
-        : BLECharacteristic("d31e02e0-c8ab-4d3f-9cc9-0b8466bdabe8",
-                            BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ)
-    {
-        setCallbacks(this);
-    }
-
-    void onRead(BLECharacteristic *c)
-    {
-        BLEKeepAliveCallbacks::onRead(c);
-
-        const NodeInfo *info = nodeDB.readNextInfo();
-
-        if (info) {
-            DEBUG_MSG("Sending nodeinfo: num=0x%x, lastseen=%u, id=%s, name=%s\n", info->num, info->position.time, info->user.id,
-                      info->user.long_name);
-            size_t numbytes = pb_encode_to_bytes(trBytes, sizeof(trBytes), NodeInfo_fields, info);
-            c->setValue(trBytes, numbytes);
-        } else {
-            c->setValue(trBytes, 0); // Send an empty response
-            DEBUG_MSG("Done sending nodeinfos\n");
-        }
-    }
-
-    void onWrite(BLECharacteristic *c)
-    {
-        BLEKeepAliveCallbacks::onWrite(c);
-        DEBUG_MSG("Reset nodeinfo read pointer\n");
-        nodeDB.resetReadPointer();
-    }
-};
-
-// wrap our protobuf version with something that forces the service to reload the config
-class RadioCharacteristic : public ProtobufCharacteristic
-{
-  public:
-    RadioCharacteristic()
-        : ProtobufCharacteristic("b56786c8-839a-44a1-b98e-a1724c4a0262",
-                                 BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ, RadioConfig_fields,
-                                 &radioConfig)
-    {
-    }
-
-    void onRead(BLECharacteristic *c)
-    {
-        DEBUG_MSG("Reading radio config, sdsecs %u\n", radioConfig.preferences.sds_secs);
-        ProtobufCharacteristic::onRead(c);
-    }
-
-    void onWrite(BLECharacteristic *c)
-    {
-        DEBUG_MSG("Writing radio config\n");
-        ProtobufCharacteristic::onWrite(c);
-        bluetoothPhoneAPI->handleSetRadio(radioConfig);
-    }
-};
-
-// wrap our protobuf version with something that forces the service to reload the owner
-class OwnerCharacteristic : public ProtobufCharacteristic
-{
-  public:
-    OwnerCharacteristic()
-        : ProtobufCharacteristic("6ff1d8b6-e2de-41e3-8c0b-8fa384f64eb6",
-                                 BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ, User_fields, &owner)
-    {
-    }
-
-    void onWrite(BLECharacteristic *c)
-    {
-        BLEKeepAliveCallbacks::onWrite(
-            c); // NOTE: We do not call the standard ProtobufCharacteristic superclass, because we want custom write behavior
-
-        static User o; // if the phone doesn't set ID we are careful to keep ours, we also always keep our macaddr
-        if (writeToDest(c, &o)) {
-            bluetoothPhoneAPI->handleSetOwner(o);
-        }
-    }
-};
-
-class MyNodeInfoCharacteristic : public ProtobufCharacteristic
-{
-  public:
-    MyNodeInfoCharacteristic()
-        : ProtobufCharacteristic("ea9f3f82-8dc4-4733-9452-1f6da28892a2", BLECharacteristic::PROPERTY_READ, MyNodeInfo_fields,
-                                 &myNodeInfo)
-    {
-    }
-
-    void onRead(BLECharacteristic *c)
-    {
-        // update gps connection state
-        myNodeInfo.has_gps = gps->isConnected;
-
-        ProtobufCharacteristic::onRead(c);
-
-        myNodeInfo.error_code = 0; // The phone just read us, so throw it away
-        myNodeInfo.error_address = 0;
-    }
-};
-
-#endif
-
 class ToRadioCharacteristic : public CallbackCharacteristic
 {
   public:
@@ -196,7 +82,6 @@ class ToRadioCharacteristic : public CallbackCharacteristic
 
     void onWrite(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onWrite(c);
         DEBUG_MSG("Got on write\n");
 
         bluetoothPhoneAPI->handleToRadio(c->getData(), c->getValue().length());
@@ -212,7 +97,6 @@ class FromRadioCharacteristic : public CallbackCharacteristic
 
     void onRead(BLECharacteristic *c)
     {
-        BLEKeepAliveCallbacks::onRead(c);
         size_t numBytes = bluetoothPhoneAPI->getFromRadio(trBytes);
 
         // Someone is going to read our value as soon as this callback returns.  So fill it with the next message in the queue
@@ -236,11 +120,7 @@ class FromNumCharacteristic : public CallbackCharacteristic
         // observe(&service.fromNumChanged);
     }
 
-    void onRead(BLECharacteristic *c)
-    {
-        BLEKeepAliveCallbacks::onRead(c);
-        DEBUG_MSG("FIXME implement fromnum read\n");
-    }
+    void onRead(BLECharacteristic *c) { DEBUG_MSG("FIXME implement fromnum read\n"); }
 };
 
 /*
@@ -263,12 +143,6 @@ BLEService *createMeshBluetoothService(BLEServer *server)
     addWithDesc(service, meshFromNumCharacteristic, "fromRadio");
     addWithDesc(service, new ToRadioCharacteristic, "toRadio");
     addWithDesc(service, new FromRadioCharacteristic, "fromNum");
-#ifdef SUPPORT_OLD_BLE_API
-    addWithDesc(service, new MyNodeInfoCharacteristic, "myNode");
-    addWithDesc(service, new RadioCharacteristic, "radio");
-    addWithDesc(service, new OwnerCharacteristic, "owner");
-    addWithDesc(service, new NodeInfoCharacteristic, "nodeinfo");
-#endif
 
     meshFromNumCharacteristic->addDescriptor(addBLEDescriptor(new BLE2902())); // Needed so clients can request notification
 

src/mesh/MeshService.cpp

@@ -86,12 +86,14 @@ void MeshService::sendOurOwner(NodeNum dest, bool wantReplies)
 const MeshPacket *MeshService::handleFromRadioUser(const MeshPacket *mp)
 {
     bool wasBroadcast = mp->to == NODENUM_BROADCAST;
-    bool isCollision = mp->from == myNodeInfo.my_node_num;
 
+    // Disable this collision testing if we use 32 bit nodenums
+    bool isCollision = (sizeof(NodeNum) == 1) && (mp->from == myNodeInfo.my_node_num);
+
+    if (isCollision) {
         // we win if we have a lower macaddr
         bool weWin = memcmp(&owner.macaddr, &mp->decoded.user.macaddr, sizeof(owner.macaddr)) < 0;
 
-    if (isCollision) {
         if (weWin) {
             DEBUG_MSG("NOTE! Received a nodenum collision and we are vetoing\n");
 

src/mesh/MeshTypes.h

@@ -6,8 +6,8 @@
 #include "mesh.pb.h"
 #include <Arduino.h>
 
-typedef uint8_t NodeNum;
-typedef uint8_t PacketId; // A packet sequence number
+typedef uint32_t NodeNum;
+typedef uint32_t PacketId; // A packet sequence number
 
 #define NODENUM_BROADCAST (sizeof(NodeNum) == 4 ? UINT32_MAX : UINT8_MAX)
 #define ERRNO_OK 0

src/mesh/NodeDB.cpp

@@ -30,7 +30,7 @@ DeviceState versions used to be defined in the .proto file but really only this
 #define here.
 */
 
-#define DEVICESTATE_CUR_VER 8
+#define DEVICESTATE_CUR_VER 9
 #define DEVICESTATE_MIN_VER DEVICESTATE_CUR_VER
 
 #ifndef NO_ESP32
@@ -114,7 +114,6 @@ void NodeDB::init()
     devicestate.has_my_node = true;
     devicestate.has_radio = true;
     devicestate.has_owner = true;
-    devicestate.has_radio = false;
     devicestate.radio.has_channel_settings = true;
     devicestate.radio.has_preferences = true;
     devicestate.node_db_count = 0;
@@ -124,8 +123,6 @@ void NodeDB::init()
 
     // default to no GPS, until one has been found by probing
     myNodeInfo.has_gps = false;
-    myNodeInfo.node_num_bits = sizeof(NodeNum) * 8;
-    myNodeInfo.packet_id_bits = sizeof(PacketId) * 8;
     myNodeInfo.message_timeout_msec = FLOOD_EXPIRE_TIME;
     myNodeInfo.min_app_version = 167;
     generatePacketId(); // FIXME - ugly way to init current_packet_id;
@@ -135,18 +132,13 @@ void NodeDB::init()
     sprintf(owner.id, "!%02x%02x%02x%02x%02x%02x", ourMacAddr[0], ourMacAddr[1], ourMacAddr[2], ourMacAddr[3], ourMacAddr[4],
             ourMacAddr[5]);
     memcpy(owner.macaddr, ourMacAddr, sizeof(owner.macaddr));
+
+    // Set default owner name
+    pickNewNodeNum(); // Note: we will repick later, just in case the settings are corrupted, but we need a valid
+    // owner.short_name now
     sprintf(owner.long_name, "Unknown %02x%02x", ourMacAddr[4], ourMacAddr[5]);
-
-    // Crummy guess at our nodenum
-    pickNewNodeNum();
-
     sprintf(owner.short_name, "?%02X", myNodeInfo.my_node_num & 0xff);
 
-    // Include our owner in the node db under our nodenum
-    NodeInfo *info = getOrCreateNode(getNodeNum());
-    info->user = owner;
-    info->has_user = true;
-
     if (!FSBegin()) // FIXME - do this in main?
     {
         DEBUG_MSG("ERROR filesystem mount Failed\n");
@@ -157,6 +149,20 @@ void NodeDB::init()
     loadFromDisk();
     // saveToDisk();
 
+    // We set node_num and packet_id _after_ loading from disk, because we always want to use the values this
+    // rom was compiled for, not what happens to be in the save file.
+    myNodeInfo.node_num_bits = sizeof(NodeNum) * 8;
+    myNodeInfo.packet_id_bits = sizeof(PacketId) * 8;
+
+    // Note! We do this after loading saved settings, so that if somehow an invalid nodenum was stored in preferences we won't
+    // keep using that nodenum forever. Crummy guess at our nodenum (but we will check against the nodedb to avoid conflicts)
+    pickNewNodeNum();
+
+    // Include our owner in the node db under our nodenum
+    NodeInfo *info = getOrCreateNode(getNodeNum());
+    info->user = owner;
+    info->has_user = true;
+
     // We set these _after_ loading from disk - because they come from the build and are more trusted than
     // what is stored in flash
     strncpy(myNodeInfo.region, optstr(HW_VERSION), sizeof(myNodeInfo.region));
@@ -176,8 +182,11 @@ void NodeDB::init()
  */
 void NodeDB::pickNewNodeNum()
 {
-    // Pick an initial nodenum based on the macaddr
-    NodeNum r = sizeof(NodeNum) == 1 ? ourMacAddr[5]
+    NodeNum r = myNodeInfo.my_node_num;
+
+    // If we don't have a nodenum at app - pick an initial nodenum based on the macaddr
+    if (r == 0)
+        r = sizeof(NodeNum) == 1 ? ourMacAddr[5]
                                  : ((ourMacAddr[2] << 24) | (ourMacAddr[3] << 16) | (ourMacAddr[4] << 8) | ourMacAddr[5]);
 
     if (r == NODENUM_BROADCAST || r < NUM_RESERVED)
@@ -247,15 +256,18 @@ void NodeDB::saveToDisk()
         if (!pb_encode(&stream, DeviceState_fields, &devicestate)) {
             DEBUG_MSG("Error: can't write protobuf %s\n", PB_GET_ERROR(&stream));
             // FIXME - report failure to phone
-        }
 
             f.close();
+        } else {
+            // Success - replace the old file
+            f.close();
 
             // brief window of risk here ;-)
             if (!FS.remove(preffile))
                 DEBUG_MSG("Warning: Can't remove old pref file\n");
             if (!FS.rename(preftmp, preffile))
                 DEBUG_MSG("Error: can't rename new pref file\n");
+        }
     } else {
         DEBUG_MSG("ERROR: can't write prefs\n"); // FIXME report to app
     }

src/mesh/PhoneAPI.cpp

@@ -1,6 +1,7 @@
 #include "PhoneAPI.h"
 #include "MeshService.h"
 #include "NodeDB.h"
+#include "PowerFSM.h"
 #include <assert.h>
 
 PhoneAPI::PhoneAPI()
@@ -14,11 +15,30 @@ void PhoneAPI::init()
     observe(&service.fromNumChanged);
 }
 
+void PhoneAPI::checkConnectionTimeout()
+{
+    if (isConnected) {
+        bool newConnected = (millis() - lastContactMsec < radioConfig.preferences.phone_timeout_secs * 1000L);
+        if (!newConnected) {
+            isConnected = false;
+            onConnectionChanged(isConnected);
+        }
+    }
+}
+
 /**
  * Handle a ToRadio protobuf
  */
 void PhoneAPI::handleToRadio(const uint8_t *buf, size_t bufLength)
 {
+    powerFSM.trigger(EVENT_CONTACT_FROM_PHONE); // As long as the phone keeps talking to us, don't let the radio go to sleep
+    lastContactMsec = millis();
+    if (!isConnected) {
+        isConnected = true;
+        onConnectionChanged(isConnected);
+    }
+    // return (lastContactMsec != 0) &&
+
     if (pb_decode_from_bytes(buf, bufLength, ToRadio_fields, &toRadioScratch)) {
         switch (toRadioScratch.which_variant) {
         case ToRadio_packet_tag: {
@@ -227,6 +247,9 @@ void PhoneAPI::handleToRadioPacket(MeshPacket *p) {}
 /// If the mesh service tells us fromNum has changed, tell the phone
 int PhoneAPI::onNotify(uint32_t newValue)
 {
+    checkConnectionTimeout(); // a handy place to check if we've heard from the phone (since the BLE version doesn't call this
+                              // from idle)
+
     if (state == STATE_SEND_PACKETS || state == STATE_LEGACY) {
         DEBUG_MSG("Telling client we have new packets %u\n", newValue);
         onNowHasData(newValue);

src/mesh/PhoneAPI.h

@@ -50,6 +50,11 @@ class PhoneAPI
     /// Use to ensure that clients don't get confused about old messages from the radio
     uint32_t config_nonce = 0;
 
+    /** the last msec we heard from the client on the other side of this link */
+    uint32_t lastContactMsec = 0;
+
+    bool isConnected = false;
+
   public:
     PhoneAPI();
 
@@ -85,6 +90,12 @@ class PhoneAPI
     /// Our fromradio packet while it is being assembled
     FromRadio fromRadioScratch;
 
+    /// Hookable to find out when connection changes
+    virtual void onConnectionChanged(bool connected) {}
+
+  /// If we haven't heard from the other side in a while then say not connected
+    void checkConnectionTimeout();
+
     /**
      * Subclasses can use this as a hook to provide custom notifications for their transport (i.e. bluetooth notifies)
      */

src/mesh/Router.cpp

@@ -56,8 +56,11 @@ PacketId generatePacketId()
 
     if (!didInit) {
         didInit = true;
-        i = random(0, numPacketId +
-                          1); // pick a random initial sequence number at boot (to prevent repeated reboots always starting at 0)
+
+        // pick a random initial sequence number at boot (to prevent repeated reboots always starting at 0)
+        // Note: we mask the high order bit to ensure that we never pass a 'negative' number to random
+        i = random(numPacketId & 0x7fffffff);
+        DEBUG_MSG("Initial packet id %u, numPacketId %u\n", i, numPacketId);
     }
 
     i++;

src/mesh/StreamAPI.cpp

@@ -9,6 +9,7 @@ void StreamAPI::loop()
 {
     writeStream();
     readStream();
+    checkConnectionTimeout();
 }
 
 /**

src/mesh/mesh.pb.c

@@ -9,7 +9,7 @@
 PB_BIND(Position, Position, AUTO)
 
 
-PB_BIND(Data, Data, 2)
+PB_BIND(Data, Data, AUTO)
 
 
 PB_BIND(User, User, AUTO)

src/mesh/mesh.pb.h

@@ -47,7 +47,7 @@ typedef struct _ChannelSettings {
     char name[12];
 } ChannelSettings;
 
-typedef PB_BYTES_ARRAY_T(251) Data_payload_t;
+typedef PB_BYTES_ARRAY_T(240) Data_payload_t;
 typedef struct _Data {
     Data_Type typ;
     Data_payload_t payload;
@@ -586,20 +586,20 @@ extern const pb_msgdesc_t ManufacturingData_msg;
 
 /* Maximum encoded size of messages (where known) */
 #define Position_size                            39
-#define Data_size                                256
+#define Data_size                                245
 #define User_size                                72
 #define RouteDiscovery_size                      88
-#define SubPacket_size                           285
-#define MeshPacket_size                          324
+#define SubPacket_size                           274
+#define MeshPacket_size                          313
 #define ChannelSettings_size                     60
 #define RadioConfig_size                         157
 #define RadioConfig_UserPreferences_size         93
 #define NodeInfo_size                            132
 #define MyNodeInfo_size                          110
-#define DeviceState_size                         15463
+#define DeviceState_size                         15100
 #define DebugString_size                         258
-#define FromRadio_size                           333
-#define ToRadio_size                             327
+#define FromRadio_size                           322
+#define ToRadio_size                             316
 /* ManufacturingData_size depends on runtime parameters */
 
 #ifdef __cplusplus

src/nrf52/UC1701Spi.cpp

@@ -34,3 +34,17 @@ class UC1701Spi : public OLEDDisplay
 
   private:
 };
+
+#include "variant.h"
+#include <UC1701.h>
+static UC1701 lcd(PIN_SPI_SCK, PIN_SPI_MOSI, ERC12864_CS, ERC12864_CD);
+
+
+void testLCD() {
+  // PCD8544-compatible displays may have a different resolution...
+  lcd.begin();
+
+  // Write a piece of text on the first line...
+  lcd.setCursor(0, 0);
+  lcd.print("Hello, World!");
+}

src/sleep.cpp

@@ -129,6 +129,7 @@ static void waitEnterSleep()
 
         if (millis() - now > 30 * 1000) { // If we wait too long just report an error and go to sleep
             recordCriticalError(ErrSleepEnterWait);
+            ESP.restart(); // FIXME - for now we just restart, need to fix bug #167
             break;
         }
     }

variants/ppr/variant.h

@@ -132,7 +132,7 @@ static const uint8_t SCK = PIN_SPI_SCK;
 #define SX1262_CS (10)
 #define SX1262_DIO1 (20)
 #define SX1262_DIO2 (26)
-#define SX1262_BUSY (18)
+#define SX1262_BUSY (31) // Supposed to be P0.18 but because of reworks, now on P0.31 (18)
 #define SX1262_RESET (17)
 // #define SX1262_ANT_SW (32 + 10)
 #define SX1262_RXEN (22)