Merge pull request #42 from geeksville/master

release 0.1.7

.clang-format

@@ -0,0 +1,6 @@
+Language: Cpp
+IndentWidth: 4
+ColumnLimit: 130
+PointerAlignment: Right
+BreakBeforeBraces: Linux
+AllowShortFunctionsOnASingleLine: Inline

.github/workflows/main.yml

@@ -1,5 +1,7 @@
 name: Continuous Integration
-on: push
+on:
+  - push
+  - pull_request
 
 jobs:
   main:

.vscode/settings.json

@@ -46,6 +46,7 @@
     },
     "cSpell.words": [
         "Meshtastic",
-        "descs"
+        "descs",
+        "protobufs"
     ]
 }

README.md

@@ -13,7 +13,7 @@ will optionally work with your phone, but no phone is required.
 
 Typical time between recharging the radios should be about eight days.
 
-This project is currently early-alpha, but if you have questions please join our chat [![Join the chat at https://gitter.im/Meshtastic/community](https://badges.gitter.im/Meshtastic/community.svg)](https://gitter.im/Meshtastic/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge).
+This project is currently early-alpha, but if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
 
 This software is 100% open source and developed by a group of hobbyist experimenters.  No warranty is provided, if you'd like to improve it - we'd love your help.  Please post in the chat.  
 
@@ -29,7 +29,7 @@ Prebuilt binaries for the supported radios is available in our [releases](https:
 
 The instructions currently require a few commmand lines, but it should be pretty straightforward.  Please post comments on our group chat if you have problems or successes.  Steps to install:
 
-1. Purchase a radio (see above) with the correct frequencies for your country (915MHz for US or JP, 470MHz for CN, 870MHz for EU).
+1. Purchase a radio (see above) with the correct frequencies for your country (915MHz for US or JP, 470MHz for CN, 433MHz and 870MHz for EU).
 2. Install "pip".  Pip is the python package manager we use to get the esptool installer app.  Instructions [here](https://www.makeuseof.com/tag/install-pip-for-python/).
 3. Run "pip install --upgrade esptool" to get esptool installed on your machine
 4. Connect your radio to your USB port
@@ -86,18 +86,17 @@ After our rate of change slows a bit, we will make beta builds available here (w
 [![Download at https://play.google.com/store/apps/details?id=com.geeksville.mesh](https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png)](https://play.google.com/store/apps/details?id=com.geeksville.mesh&referrer=utm_source%3Dgithub%26utm_medium%3Desp32-readme%26utm_campaign%3Dmeshtastic-esp32%2520readme%26anid%3Dadmob&pcampaignid=pcampaignidMKT-Other-global-all-co-prtnr-py-PartBadge-Mar2515-1)
 
 # Development
-The following sections are probably only interesting if you want to join us in developing the software.  
 
-## Power measurements
-Since one of the main goals of this project is long battery life, it is important to consider that in our software/protocol design.  Based on initial measurements it seems that the current code should run about three days between charging, and with a bit more software work (see the [TODO list](TODO.md)) a battery life of eight days should be quite doable.  Our current power measurements/model is in [this spreadsheet](https://docs.google.com/spreadsheets/d/1ft1bS3iXqFKU8SApU8ZLTq9r7QQEGESYnVgdtvdT67k/edit?usp=sharing).
+We'd love to have you join us on this merry little project.  Please see our [development documents](./docs/software/sw-design.md) and [join us in our discussion forum](https://meshtastic.discourse.group/).
 
-## Build instructions
-This project uses the simple PlatformIO build system. You can use the IDE, but for brevity
-in these instructions I describe use of their command line tool.
+# Credits
 
-1. Purchase a suitable radio (see above)
-2. Install [PlatformIO](https://platformio.org/platformio-ide)
-3. Download this git repo and cd into it
-4. Plug the radio into your USB port
-4. Type "pio run -t upload" (This command will fetch dependencies, build the project and install it on the board via USB)
-5. Platform IO also installs a very nice VisualStudio Code based IDE, see their [tutorial](https://docs.platformio.org/en/latest/tutorials/espressif32/arduino_debugging_unit_testing.html) if you'd like to use it
+This project is run by volunteers.  Past contributors include:
+
+* @astro-arphid: Added support for 433MHz radios in europe.
+* @claesg: Various documentation fixes and 3D print enclosures
+* @girtsf: So far our CI system, but soon lots of device improvements
+
+# IMPORTANT DISCLAIMERS AND FAQ
+
+For a listing of currently missing features and a FAQ click [here](docs/faq.md).

bin/build-all.sh

@@ -4,7 +4,7 @@ set -e
 
 source bin/version.sh
 
-COUNTRIES="US EU CN JP"
+COUNTRIES="US EU433 EU865 CN JP"
 # COUNTRIES=US
 
 SRCMAP=.pio/build/esp32/output.map

bin/exception_decoder.py

@@ -0,0 +1,329 @@
+#!/usr/bin/env python3
+
+"""ESP Exception Decoder
+
+github:  https://github.com/janLo/EspArduinoExceptionDecoder
+license: GPL v3
+author:  Jan Losinski
+
+Meshtastic notes:
+* original version is at: https://github.com/janLo/EspArduinoExceptionDecoder
+* version that's checked into meshtastic repo is based on: https://github.com/me21/EspArduinoExceptionDecoder
+  which adds in ESP32 Backtrace decoding.
+* this also updates the defaults to use ESP32, instead of ESP8266 and defaults to the built firmware.bin
+
+To use, copy the "Backtrace: 0x...." line to a file, e.g., backtrace.txt, then run:
+$ bin/exception_decoder.py backtrace.txt
+"""
+
+import argparse
+import re
+import subprocess
+from collections import namedtuple
+
+import sys
+
+import os
+
+EXCEPTIONS = [
+    "Illegal instruction",
+    "SYSCALL instruction",
+    "InstructionFetchError: Processor internal physical address or data error during instruction fetch",
+    "LoadStoreError: Processor internal physical address or data error during load or store",
+    "Level1Interrupt: Level-1 interrupt as indicated by set level-1 bits in the INTERRUPT register",
+    "Alloca: MOVSP instruction, if caller's registers are not in the register file",
+    "IntegerDivideByZero: QUOS, QUOU, REMS, or REMU divisor operand is zero",
+    "reserved",
+    "Privileged: Attempt to execute a privileged operation when CRING ? 0",
+    "LoadStoreAlignmentCause: Load or store to an unaligned address",
+    "reserved",
+    "reserved",
+    "InstrPIFDataError: PIF data error during instruction fetch",
+    "LoadStorePIFDataError: Synchronous PIF data error during LoadStore access",
+    "InstrPIFAddrError: PIF address error during instruction fetch",
+    "LoadStorePIFAddrError: Synchronous PIF address error during LoadStore access",
+    "InstTLBMiss: Error during Instruction TLB refill",
+    "InstTLBMultiHit: Multiple instruction TLB entries matched",
+    "InstFetchPrivilege: An instruction fetch referenced a virtual address at a ring level less than CRING",
+    "reserved",
+    "InstFetchProhibited: An instruction fetch referenced a page mapped with an attribute that does not permit instruction fetch",
+    "reserved",
+    "reserved",
+    "reserved",
+    "LoadStoreTLBMiss: Error during TLB refill for a load or store",
+    "LoadStoreTLBMultiHit: Multiple TLB entries matched for a load or store",
+    "LoadStorePrivilege: A load or store referenced a virtual address at a ring level less than CRING",
+    "reserved",
+    "LoadProhibited: A load referenced a page mapped with an attribute that does not permit loads",
+    "StoreProhibited: A store referenced a page mapped with an attribute that does not permit stores"
+]
+
+PLATFORMS = {
+    "ESP8266": "lx106",
+    "ESP32": "esp32"
+}
+
+BACKTRACE_REGEX = re.compile(r"(?:\s+(0x40[0-2](?:\d|[a-f]|[A-F]){5}):0x(?:\d|[a-f]|[A-F]){8})\b")
+EXCEPTION_REGEX = re.compile("^Exception \\((?P<exc>[0-9]*)\\):$")
+COUNTER_REGEX = re.compile('^epc1=(?P<epc1>0x[0-9a-f]+) epc2=(?P<epc2>0x[0-9a-f]+) epc3=(?P<epc3>0x[0-9a-f]+) '
+                           'excvaddr=(?P<excvaddr>0x[0-9a-f]+) depc=(?P<depc>0x[0-9a-f]+)$')
+CTX_REGEX = re.compile("^ctx: (?P<ctx>.+)$")
+POINTER_REGEX = re.compile('^sp: (?P<sp>[0-9a-f]+) end: (?P<end>[0-9a-f]+) offset: (?P<offset>[0-9a-f]+)$')
+STACK_BEGIN = '>>>stack>>>'
+STACK_END = '<<<stack<<<'
+STACK_REGEX = re.compile(
+    '^(?P<off>[0-9a-f]+):\W+(?P<c1>[0-9a-f]+) (?P<c2>[0-9a-f]+) (?P<c3>[0-9a-f]+) (?P<c4>[0-9a-f]+)(\W.*)?$')
+
+StackLine = namedtuple("StackLine", ["offset", "content"])
+
+
+class ExceptionDataParser(object):
+    def __init__(self):
+        self.exception = None
+
+        self.epc1 = None
+        self.epc2 = None
+        self.epc3 = None
+        self.excvaddr = None
+        self.depc = None
+
+        self.ctx = None
+
+        self.sp = None
+        self.end = None
+        self.offset = None
+
+        self.stack = []
+
+    def _parse_backtrace(self, line):
+        if line.startswith('Backtrace:'):
+            self.stack = [StackLine(offset=0, content=(addr,)) for addr in BACKTRACE_REGEX.findall(line)]
+            return None
+        return self._parse_backtrace
+
+    def _parse_exception(self, line):
+        match = EXCEPTION_REGEX.match(line)
+        if match is not None:
+            self.exception = int(match.group('exc'))
+            return self._parse_counters
+        return self._parse_exception
+
+    def _parse_counters(self, line):
+        match = COUNTER_REGEX.match(line)
+        if match is not None:
+            self.epc1 = match.group("epc1")
+            self.epc2 = match.group("epc2")
+            self.epc3 = match.group("epc3")
+            self.excvaddr = match.group("excvaddr")
+            self.depc = match.group("depc")
+            return self._parse_ctx
+        return self._parse_counters
+
+    def _parse_ctx(self, line):
+        match = CTX_REGEX.match(line)
+        if match is not None:
+            self.ctx = match.group("ctx")
+            return self._parse_pointers
+        return self._parse_ctx
+
+    def _parse_pointers(self, line):
+        match = POINTER_REGEX.match(line)
+        if match is not None:
+            self.sp = match.group("sp")
+            self.end = match.group("end")
+            self.offset = match.group("offset")
+            return self._parse_stack_begin
+        return self._parse_pointers
+
+    def _parse_stack_begin(self, line):
+        if line == STACK_BEGIN:
+            return self._parse_stack_line
+        return self._parse_stack_begin
+
+    def _parse_stack_line(self, line):
+        if line != STACK_END:
+            match = STACK_REGEX.match(line)
+            if match is not None:
+                self.stack.append(StackLine(offset=match.group("off"),
+                                            content=(match.group("c1"), match.group("c2"), match.group("c3"),
+                                                     match.group("c4"))))
+            return self._parse_stack_line
+        return None
+
+    def parse_file(self, file, platform, stack_only=False):
+        if platform == 'ESP32':
+            func = self._parse_backtrace
+        else:
+            func = self._parse_exception
+            if stack_only:
+                func = self._parse_stack_begin
+
+        for line in file:
+            func = func(line.strip())
+            if func is None:
+                break
+
+        if func is not None:
+            print("ERROR: Parser not complete!")
+            sys.exit(1)
+
+
+class AddressResolver(object):
+    def __init__(self, tool_path, elf_path):
+        self._tool = tool_path
+        self._elf = elf_path
+        self._address_map = {}
+
+    def _lookup(self, addresses):
+        cmd = [self._tool, "-aipfC", "-e", self._elf] + [addr for addr in addresses if addr is not None]
+
+        if sys.version_info[0] < 3:
+            output = subprocess.check_output(cmd)
+        else:
+            output = subprocess.check_output(cmd, encoding="utf-8")
+
+        line_regex = re.compile("^(?P<addr>[0-9a-fx]+): (?P<result>.+)$")
+
+        last = None
+        for line in output.splitlines():
+            line = line.strip()
+            match = line_regex.match(line)
+
+            if match is None:
+                if last is not None and line.startswith('(inlined by)'):
+                    line = line [12:].strip()
+                    self._address_map[last] += ("\n  \-> inlined by: " + line)
+                continue
+
+            if match.group("result") == '?? ??:0':
+                continue
+
+            self._address_map[match.group("addr")] = match.group("result")
+            last = match.group("addr")
+
+    def fill(self, parser):
+        addresses = [parser.epc1, parser.epc2, parser.epc3, parser.excvaddr, parser.sp, parser.end, parser.offset]
+        for line in parser.stack:
+            addresses.extend(line.content)
+
+        self._lookup(addresses)
+
+    def _sanitize_addr(self, addr):
+        if addr.startswith("0x"):
+            addr = addr[2:]
+
+        fill = "0" * (8 - len(addr))
+        return "0x" + fill + addr
+
+    def resolve_addr(self, addr):
+        out = self._sanitize_addr(addr)
+
+        if out in self._address_map:
+            out += ": " + self._address_map[out]
+
+        return out
+
+    def resolve_stack_addr(self, addr, full=True):
+        addr = self._sanitize_addr(addr)
+        if addr in self._address_map:
+            return addr + ": " + self._address_map[addr]
+
+        if full:
+            return "[DATA (0x" + addr + ")]"
+
+        return None
+
+
+def print_addr(name, value, resolver):
+    print("{}:{} {}".format(name, " " * (8 - len(name)), resolver.resolve_addr(value)))
+
+
+def print_stack_full(lines, resolver):
+    print("stack:")
+    for line in lines:
+        print(line.offset + ":")
+        for content in line.content:
+            print("  " + resolver.resolve_stack_addr(content))
+
+
+def print_stack(lines, resolver):
+    print("stack:")
+    for line in lines:
+        for content in line.content:
+            out = resolver.resolve_stack_addr(content, full=False)
+            if out is None:
+                continue
+            print(out)
+
+
+def print_result(parser, resolver, platform, full=True, stack_only=False):
+    if platform == 'ESP8266' and not stack_only:
+        print('Exception: {} ({})'.format(parser.exception, EXCEPTIONS[parser.exception]))
+
+        print("")
+        print_addr("epc1", parser.epc1, resolver)
+        print_addr("epc2", parser.epc2, resolver)
+        print_addr("epc3", parser.epc3, resolver)
+        print_addr("excvaddr", parser.excvaddr, resolver)
+        print_addr("depc", parser.depc, resolver)
+
+        print("")
+        print("ctx: " + parser.ctx)
+
+        print("")
+        print_addr("sp", parser.sp, resolver)
+        print_addr("end", parser.end, resolver)
+        print_addr("offset", parser.offset, resolver)
+
+        print("")
+    if full:
+        print_stack_full(parser.stack, resolver)
+    else:
+        print_stack(parser.stack, resolver)
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="decode ESP Stacktraces.")
+
+    parser.add_argument("-p", "--platform", help="The platform to decode from", choices=PLATFORMS.keys(),
+                        default="ESP32")
+    parser.add_argument("-t", "--tool", help="Path to the xtensa toolchain",
+                        default="~/.platformio/packages/toolchain-xtensa32/")
+    parser.add_argument("-e", "--elf", help="path to elf file",
+                        default=".pio/build/esp32/firmware.elf")
+    parser.add_argument("-f", "--full", help="Print full stack dump", action="store_true")
+    parser.add_argument("-s", "--stack_only", help="Decode only a stractrace", action="store_true")
+    parser.add_argument("file", help="The file to read the exception data from ('-' for STDIN)", default="-")
+
+    return parser.parse_args()
+
+
+if __name__ == "__main__":
+    args = parse_args()
+
+    if args.file == "-":
+        file = sys.stdin
+    else:
+        if not os.path.exists(args.file):
+            print("ERROR: file " + args.file + " not found")
+            sys.exit(1)
+        file = open(args.file, "r")
+
+    addr2line = os.path.join(os.path.abspath(os.path.expanduser(args.tool)),
+                             "bin/xtensa-" + PLATFORMS[args.platform] + "-elf-addr2line")
+    if os.name == 'nt':
+        addr2line += '.exe'
+    if not os.path.exists(addr2line):
+        print("ERROR: addr2line not found (" + addr2line + ")")
+
+    elf_file = os.path.abspath(os.path.expanduser(args.elf))
+    if not os.path.exists(elf_file):
+        print("ERROR: elf file not found (" + elf_file + ")")
+
+    parser = ExceptionDataParser()
+    resolver = AddressResolver(addr2line, elf_file)
+
+    parser.parse_file(file, args.platform, args.stack_only)
+    resolver.fill(parser)
+
+    print_result(parser, resolver, args.platform, args.full, args.stack_only)

bin/program-release-tbeam.sh

@@ -0,0 +1,6 @@
+
+set -e
+
+source bin/version.sh
+
+esptool.py --baud 921600 write_flash 0x10000 release/latest/firmware-TBEAM-US-$VERSION.bin

bin/read-system-info.sh

@@ -0,0 +1 @@
+esptool.py --baud 921600 read_flash 0x1000 0xf000 system-info.img

bin/version.sh

@@ -1,3 +1,3 @@
 
 
-export VERSION=0.1.0
+export VERSION=0.1.7

docs/README.md

@@ -30,9 +30,9 @@ Not all of these features are fully implemented yet - see **important** disclaim
 * Eventually (within a couple of months) we should have a modified version of Signal that works with this project.
 * Very easy sharing of private secured channels.  Just share a special link or QR code with friends and they can join your encrypted mesh
  
-This project is currently in early alpha - if you have questions please join our chat [![Join the chat at https://gitter.im/Meshtastic/community](https://badges.gitter.im/Meshtastic/community.svg)](https://gitter.im/Meshtastic/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge).
+This project is currently in early alpha - if you have questions please [join our discussion forum](https://meshtastic.discourse.group/).
 
-This software is 100% open source and developed by a group of hobbyist experimenters.  No warranty is provided, if you'd like to improve it - we'd love your help.  Please post in the [chat](https://gitter.im/Meshtastic/community).  
+This software is 100% open source and developed by a group of hobbyist experimenters.  No warranty is provided, if you'd like to improve it - we'd love your help.  Please post in the [forum](https://meshtastic.discourse.group/).
 
 # Updates
 
@@ -52,7 +52,7 @@ But if you want the bleeding edge app now, we'd love to have your help testing.
 
 1. Join [this Google group](https://groups.google.com/forum/#!forum/meshtastic-alpha-testers) with the account you use in Google Play.
 2. Go to this [URL](https://play.google.com/apps/testing/com.geeksville.mesh) to opt-in to the alpha test.
-3. If you encounter any problems or have questions, post in our gitter chat and we'll help.
+3. If you encounter any problems or have questions, post in our [forum](https://meshtastic.discourse.group/) and we'll help.
 
 If you'd like to help with development, the source code is [on github](https://github.com/meshtastic/Meshtastic-Android).
 
@@ -62,21 +62,12 @@ sure to buy the frequency range which is legal for your country.  For the USA, y
 
 Instructions for installing prebuilt firmware can be found [here](https://github.com/meshtastic/Meshtastic-esp32/blob/master/README.md).
 
-For a nice TTGO 3D printable case see this [design](https://www.thingiverse.com/thing:3773717) by [bsiege](https://www.thingiverse.com/bsiege).
-For a nice Heltec 3D printable case see this [design](https://www.thingiverse.com/thing:3125854) by [ornotermes](https://www.thingiverse.com/ornotermes).
+For a nice printable cases:
 
-# Disclaimers
+1. TTGO T-Beam V0 see this [design](https://www.thingiverse.com/thing:3773717) by [bsiege](https://www.thingiverse.com/bsiege).
+2. TTGO T_Beam V1 see this [design](https://www.thingiverse.com/thing:3830711) by [rwanrooy](https://www.thingiverse.com/rwanrooy) or this [remix](https://www.thingiverse.com/thing:3949330) by [8ung](https://www.thingiverse.com/8ung)
+3. Heltec Lora32 see this [design](https://www.thingiverse.com/thing:3125854) by [ornotermes](https://www.thingiverse.com/ornotermes).
 
-This project is still pretty young but moving at a pretty good pace.  Not all features are fully implemented in the current alpha builds.
-Most of these problems should be solved by the beta release:
+# IMPORTANT DISCLAIMERS AND FAQ
 
-* We don't make these devices and they haven't been tested by UL or the FCC.  If you use them you are experimenting and we can't promise they won't burn your house down ;-)
-* Encryption is turned off for now
-* A number of (straightforward) software work items have to be completed before battery life matches our measurements, currently battery life is about three days.  Join us on chat if you want the spreadsheet of power measurements/calculations.
-* The current Android GUI is pretty ugly still
-* The Android API needs to be documented better
-* The Bluetooth API needs to be documented better 
-* The mesh protocol is turned off for now, currently we only send packets one hop distant
-* No one has written an iOS app yet ;-)
-
-For more details see the [device software TODO](https://github.com/meshtastic/Meshtastic-esp32/blob/master/TODO.md) or the [Android app TODO](https://github.com/meshtastic/Meshtastic-Android/blob/master/TODO.md).
+For a listing of currently missing features and a FAQ click [here](faq.md).

docs/faq.md

@@ -0,0 +1,36 @@
+# Disclaimers
+
+This project is still pretty young but moving at a pretty good pace.  Not all features are fully implemented in the current alpha builds.
+Most of these problems should be solved by the beta release (within three months):
+
+* We don't make these devices and they haven't been tested by UL or the FCC.  If you use them you are experimenting and we can't promise they won't burn your house down ;-)
+* Encryption is turned off for now
+* A number of (straightforward) software work items have to be completed before battery life matches our measurements, currently battery life is about three days.  Join us on chat if you want the spreadsheet of power measurements/calculations.
+* The current Android GUI is slightly ugly still
+* The Android API needs to be documented better
+* The mesh protocol is turned off for now, currently we only send packets one hop distant.  The mesh feature will be turned on again [soonish](https://github.com/meshtastic/Meshtastic-esp32/issues/3).
+* No one has written an iOS app yet.  But some good souls [are talking about it](https://github.com/meshtastic/Meshtastic-esp32/issues/14) ;-)
+
+For more details see the [device software TODO](https://github.com/meshtastic/Meshtastic-esp32/blob/master/docs/software/TODO.md) or the [Android app TODO](https://github.com/meshtastic/Meshtastic-Android/blob/master/TODO.md).
+
+# FAQ
+
+If you have a question missing from this faq, please [ask in our discussion forum](https://meshtastic.discourse.group/).  And if you are feeling extra generous send in a pull-request for this faq.md with whatever we answered ;-).
+
+## Q: Which of the various supported radios should I buy?
+
+Basically you just need the radio + (optional but recommended) battery. The TBEAM is usually better because it has gps and huge battery socket. The Heltec is basically the same hardware but without the GPS (the phone provides position data to the radio in that case, so the behavior is similar - but it does burn some battery in the phone).  Also the battery for the Heltec can be smaller.  
+
+In addition to Aliexpress, (banggood.com) usually has stock and faster shipping, or Amazon.  If buying a TBEAM, make sure to buy a version that includes the OLED screen - this project doesn't absolutely require the screen, but we use it if is installed.
+
+@claesg has added links to various 3D printable cases, you can see them at (www.meshtastic.org).
+
+## Q: Do you have plans to commercialize this project
+
+Nope. though if some other person/group wanted to use this software and a more customized device we think that would be awesome (as long as they obey the GPL license).
+
+## Q: Does this project use patented algorithms? 
+
+(Kindly borrowed from the geeks at [ffmpeg](http://ffmpeg.org/legal.html))
+
+We do not know, we are not lawyers so we are not qualified to answer this. Also we have never read patents to implement any part of this, so even if we were qualified we could not answer it as we do not know what is patented. Furthermore the sheer number of software patents makes it impossible to read them all so no one (lawyer or not) could answer such a question with a definite no.  We are merely geeks experimenting on a fun and free project.

docs/mesh-proto.md

@@ -0,0 +1,20 @@
+TODO:
+* reread the radiohead mesh implementation
+* read about general mesh flooding solutions
+* reread the disaster radio protocol docs
+
+good description of batman protocol: https://www.open-mesh.org/projects/open-mesh/wiki/BATMANConcept
+
+interesting paper on lora mesh: https://portal.research.lu.se/portal/files/45735775/paper.pdf
+It seems like  DSR might be the algorithm used by RadioheadMesh.  DSR is described in https://tools.ietf.org/html/rfc4728
+https://en.wikipedia.org/wiki/Dynamic_Source_Routing
+
+broadcast solution:
+Use naive flooding at first (FIXME - do some math for a 20 node, 3 hop mesh.  A single flood will require a max of 20 messages sent)
+Then move to MPR later (http://www.olsr.org/docs/report_html/node28.html).  Use altitude and location as heursitics in selecting the MPR set
+
+compare to db sync algorithm?
+
+what about never flooding gps broadcasts.  instead only have them go one hop in the common case, but if any node X is looking at the position of Y on their gui, then send a unicast to Y asking for position update.  Y replies.
+
+If Y were to die, at least the neighbor nodes of Y would have their last known position of Y.

docs/radio-settings.md

@@ -10,6 +10,13 @@ See [this site](https://www.rfwireless-world.com/Tutorials/LoRa-channels-list.ht
 
 The maximum power allowed is +14dBM.
 
+### 433 MHz
+
+There are eight channels defined with a 0.2 MHz gap between them.
+Channel zero starts at 433.175 MHz
+
+### 870 MHz
+
 There are eight channels defined with a 0.3 MHz gap between them.
 Channel zero starts at 865.20 MHz
 

docs/software/TODO.md

@@ -7,7 +7,9 @@ being I have it set at 2 minutes to ensure enough time for a GPS lock from scrat
 
 * use gps sleep mode instead of killing its power (to allow fast position when we wake)
 * enable fast lock and low power inside the gps chip
+
 * remeasure wake time power draws now that we run CPU down at 80MHz
+* add a SF12 transmit option for _super_ long range
 
 # AXP192 tasks
 * "AXP192 interrupt is not firing, remove this temporary polling of battery state"
@@ -36,6 +38,7 @@ Items to complete before the first beta release.
 
 During the beta timeframe the following improvements 'would be nice' (and yeah - I guess some of these items count as features, but it is a hobby project ;-) )
 
+* figure out why this fixme is needed: "FIXME, disable wake due to PMU because it seems to fire all the time?"
 * If the phone doesn't read fromradio mailbox within X seconds, assume the phone is gone and we can stop queing location msgs 
 for it (because it will redownload the nodedb when it comes back)
 * Figure out why the RF95 ISR is never seeing RH_RF95_VALID_HEADER, so it is not protecting our rx packets from getting stomped on by sends
@@ -190,3 +193,11 @@ until the phone pulls those packets.  Ever so often power on bluetooth just so w
  Have state machine properly enter deep sleep based on loss of mesh and phone comms.
  Default to enter deep sleep if no LORA received for two hours (indicates user has probably left the mesh).
 * (fixed I think) text messages are not showing on local screen if screen was on
+* add links to todos
+* link to the kanban page
+* add a getting started page
+* finish mesh alg reeval
+* ublox gps parsing seems a little buggy (we shouldn't be sending out read solution commands, the device is already broadcasting them)
+* turn on gps https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library/blob/master/examples/Example18_PowerSaveMode/Example18_PowerSaveMode.ino
+* switch gps to 38400 baud https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library/blob/master/examples/Example11_ResetModule/Example2_FactoryDefaultsviaSerial/Example2_FactoryDefaultsviaSerial.ino
+* Use Neo-M8M API to put it in sleep mode

docs/software/bluetooth-api.md

@@ -0,0 +1,110 @@
+# Bluetooth API
+
+The Bluetooth API is design to have only a few characteristics and most polymorphism comes from the flexible set of Google Protocol Buffers which are sent over the wire.  We use protocol buffers extensively both for the bluetooth API and for packets inside the mesh or when providing packets to other applications on the phone.
+
+## A note on MTU sizes
+
+This device will work with any MTU size, but it is highly recommended that you call your phone's "setMTU function to increase MTU to 512 bytes" as soon as you connect to a service.  This will dramatically improve performance when reading/writing packets.
+
+## MeshBluetoothService 
+
+This is the main bluetooth service for the device and provides the API your app should use to get information about the mesh, send packets or provision the radio.  
+
+For a reference implementation of a client that uses this service see [RadioInterfaceService](https://github.com/meshtastic/Meshtastic-Android/blob/master/app/src/main/java/com/geeksville/mesh/service/RadioInterfaceService.kt).  Typical flow when 
+a phone connects to the device should be the following:
+
+* SetMTU size to 512
+* Read a RadioConfig from "radio" - used to get the channel and radio settings
+* Read (and write if incorrect) a User from "user" - to get the username for this node
+* Read a MyNodeInfo from "mynode" to get information about this local device
+* Write an empty record to "nodeinfo" to restart the nodeinfo reading state machine
+* Read from "nodeinfo" until it returns empty to build the phone's copy of the current NodeDB for the mesh
+* Read from "fromradio" until it returns empty to get any messages that arrived for this node while the phone was away
+* Subscribe to notify on "fromnum" to get notified whenever the device has a new received packet
+* Read that new packet from "fromradio"
+* Whenever the phone has a packet to send write to "toradio"
+
+For definitions (and documentation) on FromRadio, ToRadio, MyNodeInfo, NodeInfo and User  protocol buffers see [mesh.proto](https://github.com/meshtastic/Meshtastic-protobufs/blob/master/mesh.proto)
+
+UUID for the service: 6ba1b218-15a8-461f-9fa8-5dcae273eafd
+
+Each characteristic is listed as follows:
+
+UUID
+Properties
+Description (including human readable name)
+
+8ba2bcc2-ee02-4a55-a531-c525c5e454d5
+read
+fromradio - contains a newly received FromRadio packet destined towards the phone (up to MAXPACKET bytes per packet).
+After reading the esp32 will put the next packet in this mailbox.  If the FIFO is empty it will put an empty packet in this
+mailbox.
+
+f75c76d2-129e-4dad-a1dd-7866124401e7
+write
+toradio - write ToRadio protobufs to this characteristic to send them (up to MAXPACKET len)
+
+ed9da18c-a800-4f66-a670-aa7547e34453
+read,notify,write
+fromnum - the current packet # in the message waiting inside fromradio, if the phone sees this notify it should read messages
+until it catches up with this number.
+
+The phone can write to this register to go backwards up to FIXME packets, to handle the rare case of a fromradio packet was dropped after the esp32 callback was called, but before it arrives at the phone.  If the phone writes to this register the esp32 will discard older packets and put the next packet >= fromnum in fromradio.
+When the esp32 advances fromnum, it will delay doing the notify by 100ms, in the hopes that the notify will never actally need to be sent if the phone is already pulling from fromradio.
+
+Note: that if the phone ever sees this number decrease, it means the esp32 has rebooted.
+
+ea9f3f82-8dc4-4733-9452-1f6da28892a2
+read
+mynode - read this to access a MyNodeInfo protobuf
+
+d31e02e0-c8ab-4d3f-9cc9-0b8466bdabe8
+read, write
+nodeinfo - read this to get a series of NodeInfos (ending with a null empty record), write to this to restart the read statemachine that returns all the node infos
+
+b56786c8-839a-44a1-b98e-a1724c4a0262
+read,write
+radio - read/write this to access a RadioConfig protobuf
+
+6ff1d8b6-e2de-41e3-8c0b-8fa384f64eb6
+read,write
+owner - read/write this to access a User protobuf
+
+Re: queue management
+Not all messages are kept in the fromradio queue (filtered based on SubPacket):
+* only the most recent Position and User messages for a particular node are kept
+* all Data SubPackets are kept
+* No WantNodeNum / DenyNodeNum messages are kept
+A variable keepAllPackets, if set to true will suppress this behavior and instead keep everything for forwarding to the phone (for debugging)
+
+
+## Other bluetooth services
+
+This document focuses on the core mesh service, but it is worth noting that the following other Bluetooth services are also
+provided by the device.
+
+### BluetoothSoftwareUpdate
+
+The software update service.  For a sample function that performs a software update using this API see [startUpdate](https://github.com/meshtastic/Meshtastic-Android/blob/master/app/src/main/java/com/geeksville/mesh/service/SoftwareUpdateService.kt).
+
+SoftwareUpdateService UUID cb0b9a0b-a84c-4c0d-bdbb-442e3144ee30
+
+Characteristics
+
+| UUID                                 | properties       | description|
+|--------------------------------------|------------------|------------|
+| e74dd9c0-a301-4a6f-95a1-f0e1dbea8e1e | write,read       | total image size, 32 bit, write this first, then read read back to see if it was acceptable (0 mean not accepted) |
+| e272ebac-d463-4b98-bc84-5cc1a39ee517 | write            | data, variable sized, recommended 512 bytes, write one for each block of file |
+| 4826129c-c22a-43a3-b066-ce8f0d5bacc6 | write            | crc32, write last - writing this will complete the OTA operation, now you can read result |
+| 5e134862-7411-4424-ac4a-210937432c77 | read,notify      | result code, readable but will notify when the OTA operation completes |
+| GATT_UUID_SW_VERSION_STR/0x2a28 | read | We also implement these standard GATT entries because SW update probably needs them: |
+| GATT_UUID_MANU_NAME/0x2a29 | read | |
+| GATT_UUID_HW_VERSION_STR/0x2a27 | read | |
+
+### DeviceInformationService
+
+Implements the standard BLE contract for this service (has software version, hardware model, serial number, etc...)
+
+### BatteryLevelService
+
+Implements the standard BLE contract service, provides battery level in a way that most client devices should automatically understand (i.e. it should show in the bluetooth devices screen automatically)

docs/software/build-instructions.md

@@ -0,0 +1,25 @@
+# Build instructions
+
+This project uses the simple PlatformIO build system. You can use the IDE, but for brevity
+in these instructions I describe use of their command line tool.
+
+1. Purchase a suitable radio (see above)
+2. Install [PlatformIO](https://platformio.org/platformio-ide)
+3. Download this git repo and cd into it
+4. Edit configuration.h and comment out *one* of the following two lines (depending on which board you are using):
+```
+// #define T_BEAM_V10  
+#define HELTEC_LORA32
+```
+5. Plug the radio into your USB port
+6. Type "pio run -t upload" (This command will fetch dependencies, build the project and install it on the board via USB)
+7. Platform IO also installs a very nice VisualStudio Code based IDE, see their [tutorial](https://docs.platformio.org/en/latest/tutorials/espressif32/arduino_debugging_unit_testing.html) if you'd like to use it
+
+
+## Decoding stack traces
+
+If you get a crash, you can decode the addresses from the `Backtrace:` line:
+1. Save the `Backtrace: 0x....` line to a file, e.g., `backtrace.txt`.
+2. Run `bin/exception_decoder.py backtrace.txt` (this uses symbols from the
+   last `firmware.elf`, so you must be running the same binary that's still in
+   your `.pio/build` directory).

docs/software/mesh-alg.md

@@ -0,0 +1,72 @@
+# Mesh broadcast algorithm
+
+FIXME - instead look for standard solutions.  this approach seems really suboptimal, because too many nodes will try to rebroast.  If
+all else fails could always use the stock Radiohead solution - though super inefficient.
+
+great source of papers and class notes: http://www.cs.jhu.edu/~cs647/
+
+TODO:
+* DONE reread the radiohead mesh implementation - hop to hop acknoledgement seems VERY expensive but otherwise it seems like DSR
+* DONE read about mesh routing solutions (DSR and AODV)
+* DONE read about general mesh flooding solutions (naive, MPR, geo assisted)
+* DONE reread the disaster radio protocol docs - seems based on Babel (which is AODVish)
+* possibly dash7? https://www.slideshare.net/MaartenWeyn1/dash7-alliance-protocol-technical-presentation  https://github.com/MOSAIC-LoPoW/dash7-ap-open-source-stack
+* update duty cycle spreadsheet for our typical usecase
+* generalize naive flooding on top of radiohead or disaster.radio? (and fix radiohead to use my new driver)
+
+good description of batman protocol: https://www.open-mesh.org/projects/open-mesh/wiki/BATMANConcept
+
+interesting paper on lora mesh: https://portal.research.lu.se/portal/files/45735775/paper.pdf
+It seems like  DSR might be the algorithm used by RadioheadMesh.  DSR is described in https://tools.ietf.org/html/rfc4728
+https://en.wikipedia.org/wiki/Dynamic_Source_Routing
+
+broadcast solution:
+Use naive flooding at first (FIXME - do some math for a 20 node, 3 hop mesh.  A single flood will require a max of 20 messages sent)
+Then move to MPR later (http://www.olsr.org/docs/report_html/node28.html).  Use altitude and location as heursitics in selecting the MPR set
+
+compare to db sync algorithm?
+
+what about never flooding gps broadcasts.  instead only have them go one hop in the common case, but if any node X is looking at the position of Y on their gui, then send a unicast to Y asking for position update.  Y replies.
+
+If Y were to die, at least the neighbor nodes of Y would have their last known position of Y.
+
+## approach 1
+
+* send all broadcasts with a TTL
+* periodically(?) do a survey to find the max TTL that is needed to fully cover the current network.
+* to do a study first send a broadcast (maybe our current initial user announcement?) with TTL set to one (so therefore no one will rebroadcast our request)
+* survey replies are sent unicast back to us (and intervening nodes will need to keep the route table that they have built up based on past packets)
+* count the number of replies to this TTL 1 attempt.  That is the number of nodes we can reach without any rebroadcasts
+* repeat the study with a TTL of 2 and then 3.  stop once the # of replies stops going up.
+* it is important for any node to do listen before talk to prevent stomping on other rebroadcasters...
+* For these little networks I bet a max TTL would never be higher than 3?
+
+## approach 2
+
+* send a TTL1 broadcast, the replies let us build a list of the nodes (stored as a bitvector?) that we can see (and their rssis)
+* we then broadcast out that bitvector (also TTL1) asking "can any of ya'll (even indirectly) see anyone else?"
+* if a node can see someone I missed (and they are the best person to see that node), they reply (unidirectionally) with the missing nodes and their rssis (other nodes might sniff (and update their db) based on this reply but they don't have to)
+* given that the max number of nodes in this mesh will be like 20 (for normal cases), I bet globally updating this db of "nodenums and who has the best rssi for packets from that node" would be useful
+* once the global DB is shared, when a node wants to broadcast, it just sends out its broadcast . the first level receivers then make a decision "am I the best to rebroadcast to someone who likely missed this packet?" if so, rebroadcast
+
+## approach 3
+
+* when a node X wants to know other nodes positions, it broadcasts its position with want_replies=true.  Then each of the nodes that received that request broadcast their replies (possibly by using special timeslots?)
+* all nodes constantly update their local db based on replies they witnessed.
+* after 10s (or whatever) if node Y notices that it didn't hear a reply from node Z (that Y has heard from recently ) to that initial request, that means Z never heard the request from X.  Node Y will reply to X on Z's behalf.
+* could this work for more than one hop?  Is more than one hop needed?  Could it work for sending messages (i.e. for a msg sent to Z with want-reply set). 
+
+## approach 4
+
+look into the literature for this idea specifically.
+
+* don't view it as a mesh protocol as much as a "distributed db unification problem".  When nodes talk to nearby nodes they work together
+to update their nodedbs.  Each nodedb would have a last change date and any new changes that only one node has would get passed to the 
+other node.  This would nicely allow distant nodes to propogate their position to all other nodes (eventually).
+* handle group messages the same way, there would be a table of messages and time of creation.
+* when a node has a new position or message to send out, it does a broadcast.  All the adjacent nodes update their db instantly (this handles 90% of messages I'll bet).  
+* Occasionally a node might broadcast saying "anyone have anything newer than time X?"  If someone does, they send the diffs since that date.
+* essentially everything in this variant becomes broadcasts of "request db updates for >time X - for _all_ or for a particular nodenum" and nodes sending (either due to request or because they changed state) "here's a set of db updates".  Every node is constantly trying to
+build the most recent version of reality, and if some nodes are too far, then nodes closer in will eventually forward their changes to the distributed db.
+* construct non ambigious rules for who broadcasts to request db updates.  ideally the algorithm should nicely realize node X can see most other nodes, so they should just listen to all those nodes and minimize the # of broadcasts. the distributed picture of nodes rssi could be useful here?
+* possibly view the BLE protocol to the radio the same way - just a process of reconverging the node/msgdb database.

docs/software/power.md

@@ -1,6 +1,10 @@
-This is a mini design doc for various core behaviors...
+# Power Management State Machine
 
-# Rules for sleep
+i.e. sleep behavior
+
+## Power measurements
+
+Since one of the main goals of this project is long battery life, it is important to consider that in our software/protocol design.  Based on initial measurements it seems that the current code should run about three days between charging, and with a bit more software work (see the [TODO list](TODO.md)) a battery life of eight days should be quite doable.  Our current power measurements/model is in [this spreadsheet](https://docs.google.com/spreadsheets/d/1ft1bS3iXqFKU8SApU8ZLTq9r7QQEGESYnVgdtvdT67k/edit?usp=sharing).
 
 ## States
 
@@ -53,7 +57,8 @@ off during light sleep, but there is a TODO item to fix this.
 * While in ON: If it has been more than screen_on_secs since a press, lower to DARK
 * While in DARK: If time since last contact by our phone exceeds phone_timeout_secs (15 minutes), we transition down into NB mode
 * While in DARK or NB: If nothing above is forcing us to stay in a higher mode (wait_bluetooth_secs, min_wake_secs) we will lower down to LS state
-* While in LS: If either phone_sds_timeout_secs (default 2 hr) or mesh_sds_timeout_secs (default 2 hr) are exceeded we will lower into SDS mode for sds_secs (default 1 yr) (or a button press).  
+* While in LS: If either phone_sds_timeout_secs (default 2 hr) or mesh_sds_timeout_secs (default 2 hr) are exceeded we will lower into SDS mode for sds_secs (default 1 yr) (or a button press).  (Note: phone_sds_timeout_secs is currently disabled for now, because most users 
+are using without a phone)
 * Any time we enter LS mode: We stay in that until an interrupt, button press or other state transition.  Every ls_secs (default 1 hr) and let the arduino loop() run one iteration (FIXME, not sure if we need this at all), and then immediately reenter lightsleep mode on the CPU.
 
 TODO: Eventually these scheduled intervals should be synchronized to the GPS clock, so that we can consider leaving the lora receiver off to save even more power.
@@ -78,49 +83,3 @@ General ideas to hit the power draws our spreadsheet predicts.  Do the easy ones
 * see section 7.3 of https://cdn.sparkfun.com/assets/learn_tutorials/8/0/4/RFM95_96_97_98W.pdf and have hope radio wake only when a valid packet is received.  Possibly even wake the ESP32 from deep sleep via GPIO.
 * never enter deep sleep while connected to USB power (but still go to other low power modes)
 * when main cpu is idle (in loop), turn cpu clock rate down and/or activate special sleep modes.  We want almost everything shutdown until it gets an interrupt.
-
-# Mesh broadcast algoritm
-
-FIXME - instead look for standard solutions.  this approach seems really suboptimal, because too many nodes will try to rebroast.  If
-all else fails could always use the stock radiohead solution - though super inefficent.
-
-## approach 1
-
-* send all broadcasts with a TTL
-* periodically(?) do a survey to find the max TTL that is needed to fully cover the current network.
-* to do a study first send a broadcast (maybe our current initial user announcement?) with TTL set to one (so therefore no one will rebroadcast our request)
-* survey replies are sent unicast back to us (and intervening nodes will need to keep the route table that they have built up based on past packets)
-* count the number of replies to this TTL 1 attempt.  That is the number of nodes we can reach without any rebroadcasts
-* repeat the study with a TTL of 2 and then 3.  stop once the # of replies stops going up.
-* it is important for any node to do listen before talk to prevent stomping on other rebroadcasters...
-* For these little networks I bet a max TTL would never be higher than 3?
-
-## approach 2
-
-* send a TTL1 broadcast, the replies let us build a list of the nodes (stored as a bitvector?) that we can see (and their rssis)
-* we then broadcast out that bitvector (also TTL1) asking "can any of ya'll (even indirectly) see anyone else?"
-* if a node can see someone I missed (and they are the best person to see that node), they reply (unidirectionally) with the missing nodes and their rssis (other nodes might sniff (and update their db) based on this reply but they don't have to)
-* given that the max number of nodes in this mesh will be like 20 (for normal cases), I bet globally updating this db of "nodenums and who has the best rssi for packets from that node" would be useful
-* once the global DB is shared, when a node wants to broadcast, it just sends out its broadcast . the first level receivers then make a decision "am I the best to rebroadcast to someone who likely missed this packet?" if so, rebroadcast
-
-## approach 3
-
-* when a node X wants to know other nodes positions, it broadcasts its position with want_replies=true.  Then each of the nodes that received that request broadcast their replies (possibly by using special timeslots?)
-* all nodes constantly update their local db based on replies they witnessed.
-* after 10s (or whatever) if node Y notices that it didn't hear a reply from node Z (that Y has heard from recently ) to that initial request, that means Z never heard the request from X.  Node Y will reply to X on Z's behalf.
-* could this work for more than one hop?  Is more than one hop needed?  Could it work for sending messages (i.e. for a msg sent to Z with want-reply set). 
-
-## approach 4
-
-look into the literature for this idea specifically.
-
-* don't view it as a mesh protocol as much as a "distributed db unification problem".  When nodes talk to nearby nodes they work together
-to update their nodedbs.  Each nodedb would have a last change date and any new changes that only one node has would get passed to the 
-other node.  This would nicely allow distant nodes to propogate their position to all other nodes (eventually).
-* handle group messages the same way, there would be a table of messages and time of creation.
-* when a node has a new position or message to send out, it does a broadcast.  All the adjacent nodes update their db instantly (this handles 90% of messages I'll bet).  
-* Occasionally a node might broadcast saying "anyone have anything newer than time X?"  If someone does, they send the diffs since that date.
-* essentially everything in this variant becomes broadcasts of "request db updates for >time X - for _all_ or for a particular nodenum" and nodes sending (either due to request or because they changed state) "here's a set of db updates".  Every node is constantly trying to
-build the most recent version of reality, and if some nodes are too far, then nodes closer in will eventually forward their changes to the distributed db.
-* construct non ambigious rules for who broadcasts to request db updates.  ideally the algorithm should nicely realize node X can see most other nodes, so they should just listen to all those nodes and minimize the # of broadcasts. the distributed picture of nodes rssi could be useful here?
-* possibly view the BLE protocol to the radio the same way - just a process of reconverging the node/msgdb database.

docs/software/sw-design.md

@@ -0,0 +1,9 @@
+This is a mini design doc for developing the meshtastic software.
+
+* [Build instructions](build-instructions.md)
+* [TODO](TODO.md) - read this if you are looking for things to do (or curious about currently missing features)
+* Our [project board](https://github.com/orgs/meshtastic/projects/1) - shows what things we are currently working on and remaining work items for the current release.
+* [Power Management](power.md)
+* [Mesh algorithm](mesh-alg.md)
+* [Bluetooth API](bluetooth-api.md) and porting guide for new clients (iOS, python, etc...)
+* TODO: how to port the device code to a new device.

lib/BluetoothOTA/src/BluetoothSoftwareUpdate.cpp

@@ -117,21 +117,7 @@ void bluetoothRebootCheck()
 }
 
 /*
-SoftwareUpdateService UUID cb0b9a0b-a84c-4c0d-bdbb-442e3144ee30
-
-Characteristics
-
-UUID                                 properties          description
-e74dd9c0-a301-4a6f-95a1-f0e1dbea8e1e write|read          total image size, 32 bit, write this first, then read read back to see if it was acceptable (0 mean not accepted)
-e272ebac-d463-4b98-bc84-5cc1a39ee517 write               data, variable sized, recommended 512 bytes, write one for each block of file
-4826129c-c22a-43a3-b066-ce8f0d5bacc6 write               crc32, write last - writing this will complete the OTA operation, now you can read result
-5e134862-7411-4424-ac4a-210937432c77 read|notify         result code, readable but will notify when the OTA operation completes
-
-We also implement the following standard GATT entries because SW update probably needs them:
-
-ESP_GATT_UUID_SW_VERSION_STR/0x2a28
-ESP_GATT_UUID_MANU_NAME/0x2a29
-ESP_GATT_UUID_HW_VERSION_STR/0x2a27
+See bluetooth-api.md
 
  */
 BLEService *createUpdateService(BLEServer *server, std::string hwVendor, std::string swVersion, std::string hwVersion)

platformio.ini

@@ -59,14 +59,14 @@ debug_init_break = tbreak setup
 ; names.
 lib_deps =
   https://github.com/meshtastic/RadioHead.git
-  1655 ; TinyGPSPlus
   https://github.com/meshtastic/esp8266-oled-ssd1306.git ; ESP8266_SSD1306 
   AXP202X_Library
   SPI
-  1260 ; OneButton
+  ; 1260 ; OneButton - not used yet
   1202 ; CRC32, explicitly needed because dependency is missing in the ble ota update lib
   Wire ; explicitly needed here because the AXP202 library forgets to add it
   https://github.com/meshtastic/arduino-fsm.git 
+  https://github.com/meshtastic/SparkFun_Ublox_Arduino_Library.git
 
 ;[env:tbeam] 
 ;board = ttgo-t-beam

release/latest/curfirmwareversion.xml

@@ -5,5 +5,5 @@ release.  It is used by the android app for forcing software updates.  Do not ed
 Generated by bin/buildall.sh -->
 
 <resources>
-    <string name="cur_firmware_version">0.0.9</string>
+    <string name="cur_firmware_version">0.1.7</string>
 </resources>

src/CustomRF95.cpp

@@ -21,6 +21,7 @@ CustomRF95::CustomRF95(MemoryPool<MeshPacket> &_pool, PointerQueue<MeshPacket> &
 bool CustomRF95::canSleep()
 {
     // We allow initializing mode, because sometimes while testing we don't ever call init() to turn on the hardware
+    DEBUG_MSG("canSleep, mode=%d, isRx=%d, txEmpty=%d, txGood=%d\n", _mode, _isReceiving, txQueue.isEmpty(), _txGood);
     return (_mode == RHModeInitialising || _mode == RHModeIdle || _mode == RHModeRx) && !_isReceiving && txQueue.isEmpty(); 
 }
 
@@ -125,8 +126,7 @@ void CustomRF95::handleInterrupt()
                 // parsing was successful, queue for our recipient
                 mp->has_payload = true;
 
-                int res = rxDest.enqueueFromISR(mp, &higherPriWoken); // NOWAIT - fixme, if queue is full, delete older messages
-                assert(res == pdTRUE);
+                assert(rxDest.enqueueFromISR(mp, &higherPriWoken)); // NOWAIT - fixme, if queue is full, delete older messages
             }
 
             clearRxBuf(); // This message accepted and cleared
@@ -184,4 +184,4 @@ void CustomRF95::startSend(MeshPacket *txp)
 
     int res = RH_RF95::send(radiobuf, numbytes);
     assert(res);
-}
+}

src/GPS.cpp

@@ -4,16 +4,18 @@
 #include <sys/time.h>
 #include "configuration.h"
 
-// stuff that really should be in in the instance instead...
 HardwareSerial _serial_gps(GPS_SERIAL_NUM);
-uint32_t timeStartMsec;  // Once we have a GPS lock, this is where we hold the initial msec clock that corresponds to that time
-uint64_t zeroOffsetSecs; // GPS based time in secs since 1970 - only updated once on initial lock
 
 RTC_DATA_ATTR bool timeSetFromGPS; // We only reset our time once per _boot_ after that point just run from the internal clock (even across sleeps)
 
 GPS gps;
-bool hasValidLocation; // default to false, until we complete our first read
-bool wantNewLocation = true;
+
+// stuff that really should be in in the instance instead...
+static uint32_t timeStartMsec;  // Once we have a GPS lock, this is where we hold the initial msec clock that corresponds to that time
+static uint64_t zeroOffsetSecs; // GPS based time in secs since 1970 - only updated once on initial lock
+
+static bool hasValidLocation; // default to false, until we complete our first read
+static bool wantNewLocation = true;
 
 GPS::GPS() : PeriodicTask()
 {
@@ -21,10 +23,64 @@ GPS::GPS() : PeriodicTask()
 
 void GPS::setup()
 {
-    readFromRTC();
+    readFromRTC(); // read the main CPU RTC at first
 
 #ifdef GPS_RX_PIN
     _serial_gps.begin(GPS_BAUDRATE, SERIAL_8N1, GPS_RX_PIN, GPS_TX_PIN);
+    // _serial_gps.setRxBufferSize(1024); // the default is 256
+    // ublox.enableDebugging(Serial);
+
+    // note: the lib's implementation has the wrong docs for what the return val is
+    // it is not a bool, it returns zero for success
+    isConnected = ublox.begin(_serial_gps);
+
+    // try a second time, the ublox lib serial parsing is buggy?
+    // if(!isConnected) isConnected = ublox.begin(_serial_gps);
+
+    if (isConnected)
+    {
+        DEBUG_MSG("Connected to GPS successfully, TXpin=%d\n", GPS_TX_PIN);
+
+        bool factoryReset = false;
+        bool ok;
+        if (factoryReset)
+        {
+            // It is useful to force back into factory defaults (9600baud, NEMA to test the behavior of boards that don't have GPS_TX connected)
+            ublox.factoryReset();
+            delay(2000);
+            isConnected = ublox.begin(_serial_gps);
+            DEBUG_MSG("Factory reset success=%d\n", isConnected);
+            if (isConnected)
+            {
+                ublox.assumeAutoPVT(true, true); // Just parse NEMA for now
+            }
+        }
+        else
+        {
+            ok = ublox.setUART1Output(COM_TYPE_UBX, 500); // Use native API
+            assert(ok);
+            ok = ublox.setNavigationFrequency(1, 500); //Produce 4x/sec to keep the amount of time we stall in getPVT low
+            assert(ok);
+            //ok = ublox.setAutoPVT(false); // Not implemented on NEO-6M
+            //assert(ok);
+            //ok = ublox.setDynamicModel(DYN_MODEL_BIKE); // probably PEDESTRIAN but just in case assume bike speeds
+            //assert(ok);
+            ok = ublox.powerSaveMode(); //use power save mode
+            assert(ok);
+        }
+        ok = ublox.saveConfiguration(2000);
+        assert(ok);
+    }
+    else
+    {
+        // Some boards might have only the TX line from the GPS connected, in that case, we can't configure it at all.  Just
+        // assume NEMA at 9600 baud.
+        DEBUG_MSG("ERROR: No bidirectional GPS found, hoping that it still might work\n");
+
+        // tell lib, we are expecting the module to send PVT messages by itself to our Rx pin
+        // you can set second parameter to "false" if you want to control the parsing and eviction of the data (need to call checkUblox cyclically)
+        ublox.assumeAutoPVT(true, true);
+    }
 #endif
 }
 
@@ -75,70 +131,81 @@ uint32_t GPS::getValidTime()
 /// Returns true if we think the board can enter deep or light sleep now (we might be trying to get a GPS lock)
 bool GPS::canSleep()
 {
-    return !wantNewLocation;
+    return true; // we leave GPS on during sleep now, so sleep is okay !wantNewLocation;
 }
 
 /// Prepare the GPS for the cpu entering deep or light sleep, expect to be gone for at least 100s of msecs
 void GPS::prepareSleep()
 {
-    // discard all rx serial bytes so we don't try to parse them when we come back
-    while (_serial_gps.available())
-    {
-        _serial_gps.read();
-    }
-
-    // make the parser bail on whatever it was parsing
-    encode('\n');
+    if (isConnected)
+        ublox.powerOff();
 }
 
 void GPS::doTask()
 {
 #ifdef GPS_RX_PIN
-    // Consume all characters that have arrived
-
-    while (_serial_gps.available())
+    if (isConnected)
     {
-        encode(_serial_gps.read());
-        // DEBUG_MSG("Got GPS response\n");
-    }
+        // Consume all characters that have arrived
 
-    if (!timeSetFromGPS && time.isValid() && date.isValid())
-    {
-        struct timeval tv;
+        // getPVT automatically calls checkUblox
+        ublox.checkUblox(); //See if new data is available. Process bytes as they come in.
 
-        DEBUG_MSG("Got time from GPS\n");
+        // DEBUG_MSG("sec %d\n", ublox.getSecond());
+        // DEBUG_MSG("lat %d\n", ublox.getLatitude());
 
-        /* Convert to unix time 
-        The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT), not counting leap seconds (in ISO 8601: 1970-01-01T00:00:00Z). 
-        */
-        struct tm t;
-        t.tm_sec = time.second();
-        t.tm_min = time.minute();
-        t.tm_hour = time.hour();
-        t.tm_mday = date.day();
-        t.tm_mon = date.month() - 1;
-        t.tm_year = date.year() - 1900;
-        t.tm_isdst = false;
-        time_t res = mktime(&t);
-        tv.tv_sec = res;
-        tv.tv_usec = 0; // time.centisecond() * (10 / 1000);
+        // If we don't have a fix (a quick check), don't try waiting for a solution)
+        uint8_t fixtype = ublox.getFixType();
+        DEBUG_MSG("fix type %d\n", fixtype);
 
-        perhapsSetRTC(&tv);
+        // any fix that has time
+        if ((fixtype >= 2 && fixtype <= 5) && !timeSetFromGPS && ublox.getT())
+        {
+            struct timeval tv;
+
+            /* Convert to unix time 
+    The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT), not counting leap seconds (in ISO 8601: 1970-01-01T00:00:00Z). 
+    */
+            struct tm t;
+            t.tm_sec = ublox.getSecond();
+            t.tm_min = ublox.getMinute();
+            t.tm_hour = ublox.getHour();
+            t.tm_mday = ublox.getDay();
+            t.tm_mon = ublox.getMonth() - 1;
+            t.tm_year = ublox.getYear() - 1900;
+            t.tm_isdst = false;
+            time_t res = mktime(&t);
+            tv.tv_sec = res;
+            tv.tv_usec = 0; // time.centisecond() * (10 / 1000);
+
+            DEBUG_MSG("Got time from GPS month=%d, year=%d, unixtime=%ld\n", t.tm_mon, t.tm_year, tv.tv_sec);
+
+            perhapsSetRTC(&tv);
+        }
+
+        if ((fixtype >= 3 && fixtype <= 4) && ublox.getP()) // rd fixes only
+        {
+            // we only notify if position has changed
+            latitude = ublox.getLatitude() * 1e-7;
+            longitude = ublox.getLongitude() * 1e-7;
+            altitude = ublox.getAltitude() / 1000; // in mm convert to meters
+            DEBUG_MSG("new gps pos lat=%f, lon=%f, alt=%d\n", latitude, longitude, altitude);
+
+            hasValidLocation = (latitude != 0) || (longitude != 0); // bogus lat lon is reported as 0,0
+            if (hasValidLocation)
+            {
+                wantNewLocation = false;
+                notifyObservers();
+                //ublox.powerOff();
+            }
+        }
+        else // we didn't get a location update, go back to sleep and hope the characters show up
+            wantNewLocation = true;
     }
 #endif
 
-    if (location.isValid() && location.isUpdated())
-    { // we only notify if position has changed
-        // DEBUG_MSG("new gps pos\n");
-        hasValidLocation = true;
-        wantNewLocation = false;
-        notifyObservers();
-    }
-    else // we didn't get a location update, go back to sleep and hope the characters show up
-        wantNewLocation = true;
-
-    // Once we have sent a location once we only poll the GPS rarely, otherwise check back every 100ms until we have something over the serial
-    setPeriod(hasValidLocation && !wantNewLocation ? 30 * 1000 : 100);
+    // Once we have sent a location once we only poll the GPS rarely, otherwise check back every 1s until we have something over the serial
+    setPeriod(hasValidLocation && !wantNewLocation ? 30 * 1000 : 10 * 1000);
 }
 
 void GPS::startLock()
@@ -147,11 +214,3 @@ void GPS::startLock()
     wantNewLocation = true;
     setPeriod(1);
 }
-
-String GPS::getTimeStr()
-{
-    static char t[12]; // used to sprintf for Serial output
-
-    snprintf(t, sizeof(t), "%02d:%02d:%02d", time.hour(), time.minute(), time.second());
-    return t;
-}

src/GPS.h

@@ -1,18 +1,24 @@
 #pragma once
 
-#include <TinyGPS++.h>
 #include "PeriodicTask.h"
 #include "Observer.h"
 #include "sys/time.h"
+#include "SparkFun_Ublox_Arduino_Library.h"
 
 /**
  * A gps class that only reads from the GPS periodically (and FIXME - eventually keeps the gps powered down except when reading)
  * 
  * When new data is available it will notify observers.
  */
-class GPS : public PeriodicTask, public Observable, public TinyGPSPlus
+class GPS : public PeriodicTask, public Observable
 {
+    SFE_UBLOX_GPS ublox;
+
 public:
+    double latitude, longitude;
+    uint32_t altitude;
+    bool isConnected; // Do we have a GPS we are talking to
+    
     GPS();
 
     /// Return time since 1970 in secs.  Until we have a GPS lock we will be returning time based at zero
@@ -21,8 +27,6 @@ public:
     /// Return time since 1970 in secs.  If we don't have a GPS lock return zero
     uint32_t getValidTime();
 
-    String getTimeStr();
-
     void setup();
 
     virtual void loop();
@@ -46,3 +50,4 @@ private:
 };
 
 extern GPS gps;
+

src/MemoryPool.h

@@ -66,16 +66,14 @@ public:
     /// Return a buffer for use by others
     void release(T *p)
     {
-        int res = dead.enqueue(p, 0);
-        assert(res == pdTRUE);
+        assert(dead.enqueue(p, 0));
         assert(p >= buf && (p - buf) < maxElements); // sanity check to make sure a programmer didn't free something that didn't come from this pool
     }
 
     /// Return a buffer from an ISR, if higherPriWoken is set to true you have some work to do ;-)
     void releaseFromISR(T *p, BaseType_t *higherPriWoken)
     {
-        int res = dead.enqueueFromISR(p, higherPriWoken);
-        assert(res == pdTRUE);
+        assert(dead.enqueueFromISR(p, higherPriWoken));
         assert(p >= buf && (p - buf) < maxElements); // sanity check to make sure a programmer didn't free something that didn't come from this pool
     }
 };

src/MeshBluetoothService.cpp

@@ -238,60 +238,7 @@ void bluetoothNotifyFromNum(uint32_t newValue)
 BLEService *meshService;
 
 /*
-MeshBluetoothService UUID 6ba1b218-15a8-461f-9fa8-5dcae273eafd
-
-FIXME - notify vs indication for fromradio output.  Using notify for now, not sure if that is best
-FIXME - in the esp32 mesh managment code, occasionally mirror the current net db to flash, so that if we reboot we still have a good guess of users who are out there.
-FIXME - make sure this protocol is guaranteed robust and won't drop packets
-
-"According to the BLE specification the notification length can be max ATT_MTU - 3. The 3 bytes subtracted is the 3-byte header(OP-code (operation, 1 byte) and the attribute handle (2 bytes)).
-In BLE 4.1 the ATT_MTU is 23 bytes (20 bytes for payload), but in BLE 4.2 the ATT_MTU can be negotiated up to 247 bytes."
-
-MAXPACKET is 256? look into what the lora lib uses. FIXME
-
-Characteristics:
-UUID                                 
-properties          
-description
-
-8ba2bcc2-ee02-4a55-a531-c525c5e454d5                                 
-read                
-fromradio - contains a newly received packet destined towards the phone (up to MAXPACKET bytes? per packet).
-After reading the esp32 will put the next packet in this mailbox.  If the FIFO is empty it will put an empty packet in this
-mailbox.
-
-f75c76d2-129e-4dad-a1dd-7866124401e7                             
-write               
-toradio - write ToRadio protobufs to this charstic to send them (up to MAXPACKET len)
-
-ed9da18c-a800-4f66-a670-aa7547e34453                                  
-read|notify|write         
-fromnum - the current packet # in the message waiting inside fromradio, if the phone sees this notify it should read messages
-until it catches up with this number.
-  The phone can write to this register to go backwards up to FIXME packets, to handle the rare case of a fromradio packet was dropped after the esp32 
-callback was called, but before it arrives at the phone.  If the phone writes to this register the esp32 will discard older packets and put the next packet >= fromnum in fromradio.
-When the esp32 advances fromnum, it will delay doing the notify by 100ms, in the hopes that the notify will never actally need to be sent if the phone is already pulling from fromradio.
-  Note: that if the phone ever sees this number decrease, it means the esp32 has rebooted.
-
-meshMyNodeCharacteristic("ea9f3f82-8dc4-4733-9452-1f6da28892a2", BLECharacteristic::PROPERTY_READ)
-mynode - read this to access a MyNodeInfo protobuf
-
-meshNodeInfoCharacteristic("d31e02e0-c8ab-4d3f-9cc9-0b8466bdabe8", BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ),
-nodeinfo - read this to get a series of node infos (ending with a null empty record), write to this to restart the read statemachine that returns all the node infos
-
-meshRadioCharacteristic("b56786c8-839a-44a1-b98e-a1724c4a0262", BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ),
-radio - read/write this to access a RadioConfig protobuf
-
-meshOwnerCharacteristic("6ff1d8b6-e2de-41e3-8c0b-8fa384f64eb6", BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ)
-owner - read/write this to access a User protobuf
-
-Re: queue management
-Not all messages are kept in the fromradio queue (filtered based on SubPacket):
-* only the most recent Position and User messages for a particular node are kept
-* all Data SubPackets are kept
-* No WantNodeNum / DenyNodeNum messages are kept
-A variable keepAllPackets, if set to true will suppress this behavior and instead keep everything for forwarding to the phone (for debugging)
-
+See bluetooth-api.md for documentation.
  */
 BLEService *createMeshBluetoothService(BLEServer *server)
 {

src/MeshRadio.cpp

@@ -9,8 +9,6 @@
 #include "configuration.h"
 #include "NodeDB.h"
 
-#define DEFAULT_CHANNEL_NUM 3 // we randomly pick one
-
 /// 16 bytes of random PSK for our _public_ default channel that all devices power up on
 static const uint8_t defaultpsk[] = {0xd4, 0xf1, 0xbb, 0x3a, 0x20, 0x29, 0x07, 0x59, 0xf0, 0xbc, 0xff, 0xab, 0xcf, 0x4e, 0x69, 0xbf};
 
@@ -39,7 +37,6 @@ MeshRadio::MeshRadio(MemoryPool<MeshPacket> &_pool, PointerQueue<MeshPacket> &_r
   channelSettings.modem_config = ChannelSettings_ModemConfig_Bw125Cr48Sf4096; // slow and long range
 
   channelSettings.tx_power = 23;
-  channelSettings.channel_num = DEFAULT_CHANNEL_NUM;
   memcpy(&channelSettings.psk, &defaultpsk, sizeof(channelSettings.psk));
   strcpy(channelSettings.name, "Default");
   // Can't print strings this early - serial not setup yet
@@ -81,6 +78,22 @@ bool MeshRadio::init()
   return true;
 }
 
+/** hash a string into an integer
+ * 
+ * djb2 by Dan Bernstein.
+ * http://www.cse.yorku.ca/~oz/hash.html
+ */
+unsigned long hash(char *str)
+{
+  unsigned long hash = 5381;
+  int c;
+
+  while ((c = *str++) != 0)
+    hash = ((hash << 5) + hash) + (unsigned char)c; /* hash * 33 + c */
+
+  return hash;
+}
+
 void MeshRadio::reloadConfig()
 {
   rf95.setModeIdle(); // Need to be idle before doing init
@@ -91,10 +104,9 @@ void MeshRadio::reloadConfig()
                                                                                  //    setModemConfig(Bw125Cr48Sf4096); // slow and reliable?
   // rf95.setPreambleLength(8);           // Default is 8
 
-  assert(channelSettings.channel_num < NUM_CHANNELS); // If the phone tries to tell us to use an illegal channel then panic
-
   // Defaults after init are 434.0MHz, modulation GFSK_Rb250Fd250, +13dbM
-  float center_freq = CH0 + CH_SPACING * channelSettings.channel_num;
+  int channel_num = hash(channelSettings.name) % NUM_CHANNELS;
+  float center_freq = CH0 + CH_SPACING * channel_num;
   if (!rf95.setFrequency(center_freq))
   {
     DEBUG_MSG("setFrequency failed\n");
@@ -109,7 +121,7 @@ void MeshRadio::reloadConfig()
   // FIXME - can we do this?  It seems to be in the Heltec board.
   rf95.setTxPower(channelSettings.tx_power, false);
 
-  DEBUG_MSG("Set radio: name=%s. config=%u, ch=%d, txpower=%d\n", channelSettings.name, channelSettings.modem_config, channelSettings.channel_num, channelSettings.tx_power);
+  DEBUG_MSG("Set radio: name=%s. config=%u, ch=%d, txpower=%d\n", channelSettings.name, channelSettings.modem_config, channel_num, channelSettings.tx_power);
 
   // Done with init tell radio to start receiving
   rf95.setModeRx();

src/MeshRadio.h

@@ -13,17 +13,22 @@
 #define CH_SPACING_US   2.16f // MHz
 #define NUM_CHANNELS_US 13
 
-// EU channel settings
-#define CH0_EU          865.2f // MHz
-#define CH_SPACING_EU   0.3f // MHz
-#define NUM_CHANNELS_EU 10
+// EU433 channel settings
+#define CH0_EU433          433.175f // MHz
+#define CH_SPACING_EU433   0.2f // MHz
+#define NUM_CHANNELS_EU433 8
+
+// EU865 channel settings
+#define CH0_EU865          865.2f // MHz
+#define CH_SPACING_EU865   0.3f // MHz
+#define NUM_CHANNELS_EU865 10
 
 // CN channel settings
 #define CH0_CN          470.0f // MHz
 #define CH_SPACING_CN   2.0f // MHz FIXME, this is just a guess for 470-510
 #define NUM_CHANNELS_CN 20
 
-// CN channel settings
+// JP channel settings
 #define CH0_JP          920.0f // MHz
 #define CH_SPACING_JP   0.5f // MHz FIXME, this is just a guess for 920-925
 #define NUM_CHANNELS_JP 10
@@ -33,10 +38,14 @@
 #define CH0 CH0_US
 #define CH_SPACING CH_SPACING_US
 #define NUM_CHANNELS NUM_CHANNELS_US
-#elif defined(HW_VERSION_EU)
-#define CH0 CH0_EU
-#define CH_SPACING CH_SPACING_EU
-#define NUM_CHANNELS NUM_CHANNELS_EU
+#elif defined(HW_VERSION_EU433)
+#define CH0 CH0_EU433
+#define CH_SPACING CH_SPACING_EU433
+#define NUM_CHANNELS NUM_CHANNELS_EU433
+#elif defined(HW_VERSION_EU865)
+#define CH0 CH0_EU865
+#define CH_SPACING CH_SPACING_EU865
+#define NUM_CHANNELS NUM_CHANNELS_EU865
 #elif defined(HW_VERSION_CN)
 #define CH0 CH0_CN
 #define CH_SPACING CH_SPACING_CN

src/MeshService.cpp

@@ -173,7 +173,7 @@ void MeshService::handleFromRadio(MeshPacket *mp)
             if (d)
                 releaseToPool(d);
         }
-        assert(toPhoneQueue.enqueue(mp, 0) == pdTRUE); // FIXME, instead of failing for full queue, delete the oldest mssages
+        assert(toPhoneQueue.enqueue(mp, 0)); // FIXME, instead of failing for full queue, delete the oldest mssages
 
         if (mp->payload.want_response)
             sendNetworkPing(mp->from);
@@ -255,6 +255,9 @@ void MeshService::handleToRadio(std::string s)
             break;
         }
     }
+    else {
+        DEBUG_MSG("Error: ignoring malformed toradio\n");
+    }
 }
 
 void MeshService::sendToMesh(MeshPacket *p)
@@ -326,11 +329,13 @@ void MeshService::onGPSChanged()
     MeshPacket *p = allocForSending();
     p->payload.which_variant = SubPacket_position_tag;
     Position &pos = p->payload.variant.position;
+#if 0
     if (gps.altitude.isValid())
         pos.altitude = gps.altitude.meters();
     pos.latitude = gps.location.lat();
     pos.longitude = gps.location.lng();
     pos.time = gps.getValidTime();
+#endif
 
     // We limit our GPS broadcasts to a max rate
     static uint32_t lastGpsSend;

src/NodeDB.cpp

@@ -26,7 +26,7 @@ DeviceState versions used to be defined in the .proto file but really only this
 #define here.
 */
 
-#define DEVICESTATE_CUR_VER 1
+#define DEVICESTATE_CUR_VER 2
 #define DEVICESTATE_MIN_VER DEVICESTATE_CUR_VER
 
 #define FS SPIFFS

src/PeriodicTask.cpp

@@ -1,23 +1,25 @@
 #include "PeriodicTask.h"
 #include "Periodic.h"
 
-PeriodicTask::PeriodicTask(uint32_t initialPeriod) : period(initialPeriod)
-{
-}
+PeriodicTask::PeriodicTask(uint32_t initialPeriod) : period(initialPeriod) {}
 
 /// call this from loop
 void PeriodicTask::loop()
 {
-    uint32_t now = millis();
-    if (period && (now - lastMsec) >= period)
     {
+        meshtastic::LockGuard lg(&lock);
+        uint32_t now = millis();
+        if (!period || (now - lastMsec) < period) {
+            return;
+        }
         lastMsec = now;
-        doTask();
     }
+    // Release the lock in case the task wants to change the period.
+    doTask();
 }
 
 void Periodic::doTask()
 {
     uint32_t p = callback();
     setPeriod(p);
-}
+}

src/PeriodicTask.h

@@ -1,11 +1,12 @@
 #pragma once
 
-#include <Arduino.h>
-#include "configuration.h"
+#include <cstdint>
+
+#include "lock.h"
 
 /**
  * A base class for tasks that want their doTask() method invoked periodically
- * 
+ *
  * FIXME: currently just syntatic sugar for polling in loop (you must call .loop), but eventually
  * generalize with the freertos scheduler so we can save lots of power by having everything either in
  * something like this or triggered off of an irq.
@@ -15,9 +16,10 @@ class PeriodicTask
     uint32_t lastMsec = 0;
     uint32_t period = 1; // call soon after creation
 
-public:
-    uint32_t periodMsec;
+    // Protects the above variables.
+    meshtastic::Lock lock;
 
+  public:
     virtual ~PeriodicTask() {}
 
     PeriodicTask(uint32_t initialPeriod = 1);
@@ -26,8 +28,12 @@ public:
     virtual void loop();
 
     /// Set a new period in msecs (can be called from doTask or elsewhere and the scheduler will cope)
-    void setPeriod(uint32_t p) { period = p; }
+    void setPeriod(uint32_t p)
+    {
+        meshtastic::LockGuard lg(&lock);
+        period = p;
+    }
 
-protected: 
+  protected:
     virtual void doTask() = 0;
 };

src/PointerQueue.h

@@ -18,7 +18,7 @@ public:
     {
         T *p;
 
-        return this->dequeue(&p, maxWait) == pdTRUE ? p : NULL;
+        return this->dequeue(&p, maxWait) ? p : nullptr;
     }
 
     // returns a ptr or null if the queue was empty
@@ -26,6 +26,6 @@ public:
     {
         T *p;
 
-        return this->dequeueFromISR(&p, higherPriWoken) == pdTRUE ? p : NULL;
+        return this->dequeueFromISR(&p, higherPriWoken) ? p : nullptr;
     }
 };

src/PowerFSM.cpp

@@ -25,19 +25,24 @@ static void sdsEnter()
 
 static void lsEnter()
 {
+    DEBUG_MSG("lsEnter begin\n");
     screen.setOn(false);
-    
+
     while (!service.radio.rf95.canSleep())
         delay(10); // Kinda yucky - wait until radio says say we can shutdown (finished in process sends/receives)
 
     gps.prepareSleep(); // abandon in-process parsing
 
-    if (!isUSBPowered)      // FIXME - temp hack until we can put gps in sleep mode, if we have AC when we go to sleep then leave GPS on
-        setGPSPower(false); // kill GPS power
+    //if (!isUSBPowered)      // FIXME - temp hack until we can put gps in sleep mode, if we have AC when we go to sleep then leave GPS on
+    //    setGPSPower(false); // kill GPS power
+
+    DEBUG_MSG("lsEnter end\n");
 }
 
 static void lsIdle()
 {
+    DEBUG_MSG("lsIdle begin ls_secs=%u\n", radioConfig.preferences.ls_secs);
+
     uint32_t secsSlept = 0;
     esp_sleep_source_t wakeCause = ESP_SLEEP_WAKEUP_UNDEFINED;
     bool reached_ls_secs = false;
@@ -69,6 +74,8 @@ static void lsIdle()
     }
     else
     {
+        DEBUG_MSG("wakeCause %d\n", wakeCause);
+
         // Regardless of why we woke just transition to NB (and that state will handle stuff like IRQs etc)
         powerFSM.trigger(EVENT_WAKE_TIMER);
     }
@@ -76,7 +83,7 @@ static void lsIdle()
 
 static void lsExit()
 {
-    setGPSPower(true); // restore GPS power
+    // setGPSPower(true); // restore GPS power
     gps.startLock();
 }
 
@@ -158,7 +165,8 @@ void PowerFSM_setup()
     powerFSM.add_timed_transition(&stateDARK, &stateLS, radioConfig.preferences.wait_bluetooth_secs * 1000, NULL, "Bluetooth timeout");
 
     powerFSM.add_timed_transition(&stateLS, &stateSDS, radioConfig.preferences.mesh_sds_timeout_secs * 1000, NULL, "mesh timeout");
-    powerFSM.add_timed_transition(&stateLS, &stateSDS, radioConfig.preferences.phone_sds_timeout_sec * 1000, NULL, "phone timeout");
+    // removing for now, because some users don't even have phones
+    // powerFSM.add_timed_transition(&stateLS, &stateSDS, radioConfig.preferences.phone_sds_timeout_sec * 1000, NULL, "phone timeout");
 
     powerFSM.run_machine(); // run one interation of the state machine, so we run our on enter tasks for the initial DARK state
 }

src/TypedQueue.h

@@ -1,18 +1,22 @@
 #pragma once
 
-#include <Arduino.h>
-#include <assert.h>
+#include <cassert>
+#include <type_traits>
+
+#include <freertos/FreeRTOS.h>
+#include <freertos/queue.h>
 
 /**
- * A wrapper for freertos queues.  Note: each element object must be quite small, so T should be only
- * pointer types or ints
+ * A wrapper for freertos queues.  Note: each element object should be small
+ * and POD (Plain Old Data type) as elements are memcpied by value.
  */
 template <class T>
 class TypedQueue
 {
+    static_assert(std::is_pod<T>::value, "T must be pod");
     QueueHandle_t h;
 
-public:
+  public:
     TypedQueue(int maxElements)
     {
         h = xQueueCreate(maxElements, sizeof(T));
@@ -34,24 +38,22 @@ public:
         return uxQueueMessagesWaiting(h) == 0;
     }
 
-    // pdTRUE for success else failure
-    BaseType_t enqueue(T x, TickType_t maxWait = portMAX_DELAY)
+    bool enqueue(T x, TickType_t maxWait = portMAX_DELAY)
     {
-        return xQueueSendToBack(h, &x, maxWait);
+        return xQueueSendToBack(h, &x, maxWait) == pdTRUE;
     }
 
-    BaseType_t enqueueFromISR(T x, BaseType_t *higherPriWoken)
+    bool enqueueFromISR(T x, BaseType_t *higherPriWoken)
     {
-        return xQueueSendToBackFromISR(h, &x, higherPriWoken);
+        return xQueueSendToBackFromISR(h, &x, higherPriWoken) == pdTRUE;
     }
 
-    // pdTRUE for success else failure
-    BaseType_t dequeue(T *p, TickType_t maxWait = portMAX_DELAY)
+    bool dequeue(T *p, TickType_t maxWait = portMAX_DELAY)
     {
-        return xQueueReceive(h, p, maxWait);
+        return xQueueReceive(h, p, maxWait) == pdTRUE;
     }
 
-    BaseType_t dequeueFromISR(T *p, BaseType_t *higherPriWoken)
+    bool dequeueFromISR(T *p, BaseType_t *higherPriWoken)
     {
         return xQueueReceiveFromISR(h, p, higherPriWoken);
     }

src/configuration.h

@@ -84,14 +84,12 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define GPS_SERIAL_NUM 1
 #define GPS_BAUDRATE 9600
 
-#if defined(T_BEAM_V10)
 #define GPS_RX_PIN 34
 #ifdef USE_JTAG
 #define GPS_TX_PIN -1
 #else
 #define GPS_TX_PIN 12
 #endif
-#endif
 
 // -----------------------------------------------------------------------------
 // LoRa SPI
@@ -108,6 +106,8 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 #define BICOLOR_DISPLAY // we have yellow at the top 16 lines
 
+// #define BUTTON_NEED_PULLUP // if set we need to turn on the internal CPU pullup during sleep
+
 #define I2C_SDA 21
 #define I2C_SCL 22
 

src/debug.cpp

@@ -0,0 +1,20 @@
+#include "debug.h"
+
+#include <cstdint>
+
+#include <freertos/FreeRTOS.h>
+#include <freertos/task.h>
+
+#include "configuration.h"
+
+namespace meshtastic
+{
+
+void printThreadInfo(const char *extra)
+{
+    uint32_t taskHandle = reinterpret_cast<uint32_t>(xTaskGetCurrentTaskHandle());
+    DEBUG_MSG("printThreadInfo(%s) task: %" PRIx32 " core id: %u min free stack: %u\n", extra, taskHandle, xPortGetCoreID(),
+              uxTaskGetStackHighWaterMark(nullptr));
+}
+
+} // namespace meshtastic

src/debug.h

@@ -0,0 +1,10 @@
+#pragma once
+
+namespace meshtastic
+{
+
+/// Dumps out which core we are running on, and min level of remaining stack
+/// seen.
+void printThreadInfo(const char *extra);
+
+} // namespace meshtastic

src/lock.cpp

@@ -0,0 +1,35 @@
+#include "lock.h"
+
+#include <cassert>
+
+namespace meshtastic
+{
+
+Lock::Lock()
+{
+    handle = xSemaphoreCreateBinary();
+    assert(handle);
+    assert(xSemaphoreGive(handle));
+}
+
+void Lock::lock()
+{
+    assert(xSemaphoreTake(handle, portMAX_DELAY));
+}
+
+void Lock::unlock()
+{
+    assert(xSemaphoreGive(handle));
+}
+
+LockGuard::LockGuard(Lock *lock) : lock(lock)
+{
+    lock->lock();
+}
+
+LockGuard::~LockGuard()
+{
+    lock->unlock();
+}
+
+} // namespace meshtastic

src/lock.h

@@ -0,0 +1,46 @@
+#pragma once
+
+#include <freertos/FreeRTOS.h>
+#include <freertos/semphr.h>
+
+namespace meshtastic
+{
+
+// Simple wrapper around FreeRTOS API for implementing a mutex lock.
+class Lock
+{
+public:
+  Lock();
+
+  Lock(const Lock &) = delete;
+  Lock &operator=(const Lock &) = delete;
+
+  /// Locks the lock.
+  //
+  // Must not be called from an ISR.
+  void lock();
+
+  // Unlocks the lock.
+  //
+  // Must not be called from an ISR.
+  void unlock();
+
+private:
+  SemaphoreHandle_t handle;
+};
+
+// RAII lock guard.
+class LockGuard
+{
+public:
+  LockGuard(Lock *lock);
+  ~LockGuard();
+
+  LockGuard(const LockGuard &) = delete;
+  LockGuard &operator=(const LockGuard &) = delete;
+
+private:
+  Lock *lock;
+};
+
+} // namespace meshtastic

src/main.cpp

@@ -24,7 +24,6 @@
 #include "configuration.h"
 #include "rom/rtc.h"
 #include <driver/rtc_io.h>
-#include <TinyGPS++.h>
 #include <Wire.h>
 #include "BluetoothUtil.h"
 #include "MeshBluetoothService.h"
@@ -341,6 +340,9 @@ void loop()
   // axpDebugOutput.loop();
   loopBLE();
 
+  // for debug printing
+  // service.radio.rf95.canSleep();
+
 #ifdef T_BEAM_V10
   if (axp192_found)
   {
@@ -415,4 +417,4 @@ void loop()
   msecstosleep = 10;
 
   delay(msecstosleep);
-}
+}

src/mesh-pb-constants.cpp

@@ -30,7 +30,7 @@ bool pb_decode_from_bytes(const uint8_t *srcbuf, size_t srcbufsize, const pb_msg
     pb_istream_t stream = pb_istream_from_buffer(srcbuf, srcbufsize);
     if (!pb_decode(&stream, fields, dest_struct))
     {
-        DEBUG_MSG("Error: can't decode protobuf %s\n", PB_GET_ERROR(&stream));
+        DEBUG_MSG("Error: can't decode protobuf %s, pb_msgdesc 0x%p\n", PB_GET_ERROR(&stream), fields);
         return false;
     }
     else

src/mesh.pb.h

@@ -34,7 +34,6 @@ typedef enum _ChannelSettings_ModemConfig {
 /* Struct definitions */
 typedef struct _ChannelSettings {
     int32_t tx_power;
-    uint32_t channel_num;
     ChannelSettings_ModemConfig modem_config;
     pb_byte_t psk[16];
     char name[12];
@@ -173,7 +172,7 @@ typedef struct _ToRadio {
 #define User_init_default                        {"", "", "", {0}}
 #define SubPacket_init_default                   {0, {Position_init_default}, 0}
 #define MeshPacket_init_default                  {0, 0, false, SubPacket_init_default, 0}
-#define ChannelSettings_init_default             {0, 0, _ChannelSettings_ModemConfig_MIN, {0}, ""}
+#define ChannelSettings_init_default             {0, _ChannelSettings_ModemConfig_MIN, {0}, ""}
 #define RadioConfig_init_default                 {false, RadioConfig_UserPreferences_init_default, false, ChannelSettings_init_default}
 #define RadioConfig_UserPreferences_init_default {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
 #define NodeInfo_init_default                    {0, false, User_init_default, false, Position_init_default, 0, 0}
@@ -186,7 +185,7 @@ typedef struct _ToRadio {
 #define User_init_zero                           {"", "", "", {0}}
 #define SubPacket_init_zero                      {0, {Position_init_zero}, 0}
 #define MeshPacket_init_zero                     {0, 0, false, SubPacket_init_zero, 0}
-#define ChannelSettings_init_zero                {0, 0, _ChannelSettings_ModemConfig_MIN, {0}, ""}
+#define ChannelSettings_init_zero                {0, _ChannelSettings_ModemConfig_MIN, {0}, ""}
 #define RadioConfig_init_zero                    {false, RadioConfig_UserPreferences_init_zero, false, ChannelSettings_init_zero}
 #define RadioConfig_UserPreferences_init_zero    {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
 #define NodeInfo_init_zero                       {0, false, User_init_zero, false, Position_init_zero, 0, 0}
@@ -197,7 +196,6 @@ typedef struct _ToRadio {
 
 /* Field tags (for use in manual encoding/decoding) */
 #define ChannelSettings_tx_power_tag             1
-#define ChannelSettings_channel_num_tag          2
 #define ChannelSettings_modem_config_tag         3
 #define ChannelSettings_psk_tag                  4
 #define ChannelSettings_name_tag                 5
@@ -303,7 +301,6 @@ X(a, STATIC,   SINGULAR, UINT32,   rx_time,           4)
 
 #define ChannelSettings_FIELDLIST(X, a) \
 X(a, STATIC,   SINGULAR, INT32,    tx_power,          1) \
-X(a, STATIC,   SINGULAR, UINT32,   channel_num,       2) \
 X(a, STATIC,   SINGULAR, UENUM,    modem_config,      3) \
 X(a, STATIC,   SINGULAR, FIXED_LENGTH_BYTES, psk,               4) \
 X(a, STATIC,   SINGULAR, STRING,   name,              5)
@@ -421,12 +418,12 @@ extern const pb_msgdesc_t ToRadio_msg;
 #define User_size                                72
 #define SubPacket_size                           261
 #define MeshPacket_size                          292
-#define ChannelSettings_size                     50
-#define RadioConfig_size                         126
+#define ChannelSettings_size                     44
+#define RadioConfig_size                         120
 #define RadioConfig_UserPreferences_size         72
 #define NodeInfo_size                            155
 #define MyNodeInfo_size                          63
-#define DeviceState_size                         15064
+#define DeviceState_size                         15058
 #define FromRadio_size                           301
 #define ToRadio_size                             295
 

src/screen.cpp

@@ -31,6 +31,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #include "mesh-pb-constants.h"
 #include "NodeDB.h"
 #include "main.h"
+#include "configuration.h"
 
 #define FONT_HEIGHT 14 // actually 13 for "ariel 10" but want a little extra space
 #define FONT_HEIGHT_16 (ArialMT_Plain_16[1] + 1)
@@ -592,6 +593,7 @@ void Screen::setup()
     // dispdev.setFont(Custom_ArialMT_Plain_10);
 
     ui.disableAutoTransition(); // we now require presses
+    ui.update(); // force an immediate draw of the bootscreen, because on some ssd1306 clones, the first draw command is discarded
 #endif
 }
 

src/sleep.cpp

@@ -1,7 +1,6 @@
 #include "configuration.h"
 #include "rom/rtc.h"
 #include <driver/rtc_io.h>
-#include <TinyGPS++.h>
 #include <Wire.h>
 #include "BluetoothUtil.h"
 #include "MeshBluetoothService.h"
@@ -19,7 +18,7 @@
 #ifdef T_BEAM_V10
 #include "axp20x.h"
 extern AXP20X_Class axp;
-#endif 
+#endif
 
 // deep sleep support
 RTC_DATA_ATTR int bootCount = 0;
@@ -28,8 +27,6 @@ esp_sleep_source_t wakeCause; // the reason we booted this time
 #define xstr(s) str(s)
 #define str(s) #s
 
-
-
 // -----------------------------------------------------------------------------
 // Application
 // -----------------------------------------------------------------------------
@@ -63,7 +60,7 @@ void setLed(bool ledOn)
 
 void setGPSPower(bool on)
 {
-    DEBUG_MSG("Setting GPS power=%d\n", on);
+  DEBUG_MSG("Setting GPS power=%d\n", on);
 
 #ifdef T_BEAM_V10
   if (axp192_found)
@@ -84,9 +81,24 @@ void initDeepSleep()
         wakeButtons = ((uint64_t)1) << buttons.gpios[0];
     */
 
-  DEBUG_MSG("booted, wake cause %d (boot count %d)\n", wakeCause, bootCount);
-}
+  // If we booted because our timer ran out or the user pressed reset, send those as fake events
+  const char *reason = "reset"; // our best guess
+  RESET_REASON hwReason = rtc_get_reset_reason(0);
 
+  if (hwReason == RTCWDT_BROWN_OUT_RESET)
+    reason = "brownout";
+
+  if (hwReason == TG0WDT_SYS_RESET)
+    reason = "taskWatchdog";
+
+  if (hwReason == TG1WDT_SYS_RESET)
+    reason = "intWatchdog";
+
+  if (wakeCause == ESP_SLEEP_WAKEUP_TIMER)
+    reason = "timeout";
+
+  DEBUG_MSG("booted, wake cause %d (boot count %d), reset_reason=%s\n", wakeCause, bootCount, reason);
+}
 
 void doDeepSleep(uint64_t msecToWake)
 {
@@ -164,6 +176,10 @@ void doDeepSleep(uint64_t msecToWake)
   // Only GPIOs which are have RTC functionality can be used in this bit map: 0,2,4,12-15,25-27,32-39.
   uint64_t gpioMask = (1ULL << BUTTON_PIN);
 
+#ifdef BUTTON_NEED_PULLUP
+  gpio_pullup_en((gpio_num_t) BUTTON_PIN);
+#endif
+
   // Not needed because both of the current boards have external pullups
   // FIXME change polarity in hw so we can wake on ANY_HIGH instead - that would allow us to use all three buttons (instead of just the first)
   // gpio_pullup_en((gpio_num_t)BUTTON_PIN);
@@ -175,9 +191,6 @@ void doDeepSleep(uint64_t msecToWake)
   esp_deep_sleep_start();                              // TBD mA sleep current (battery)
 }
 
-
-
-
 /**
  * enter light sleep (preserves ram but stops everything about CPU).
  * 
@@ -188,7 +201,7 @@ esp_sleep_wakeup_cause_t doLightSleep(uint64_t sleepMsec) // FIXME, use a more r
   //DEBUG_MSG("Enter light sleep\n");
   uint64_t sleepUsec = sleepMsec * 1000LL;
 
-  Serial.flush(); // send all our characters before we stop cpu clock
+  Serial.flush();            // send all our characters before we stop cpu clock
   setBluetoothEnable(false); // has to be off before calling light sleep
 
   // NOTE! ESP docs say we must disable bluetooth and wifi before light sleep
@@ -196,15 +209,20 @@ esp_sleep_wakeup_cause_t doLightSleep(uint64_t sleepMsec) // FIXME, use a more r
   // We want RTC peripherals to stay on
   esp_sleep_pd_config(ESP_PD_DOMAIN_RTC_PERIPH, ESP_PD_OPTION_ON);
 
+#ifdef BUTTON_NEED_PULLUP
+  gpio_pullup_en((gpio_num_t) BUTTON_PIN);
+#endif
+
   gpio_wakeup_enable((gpio_num_t)BUTTON_PIN, GPIO_INTR_LOW_LEVEL); // when user presses, this button goes low
   gpio_wakeup_enable((gpio_num_t)DIO0_GPIO, GPIO_INTR_HIGH_LEVEL); // RF95 interrupt, active high
 #ifdef PMU_IRQ
-  gpio_wakeup_enable((gpio_num_t)PMU_IRQ, GPIO_INTR_HIGH_LEVEL); // pmu irq
+  // FIXME, disable wake due to PMU because it seems to fire all the time?
+  // gpio_wakeup_enable((gpio_num_t)PMU_IRQ, GPIO_INTR_HIGH_LEVEL); // pmu irq
 #endif
   assert(esp_sleep_enable_gpio_wakeup() == ESP_OK);
   assert(esp_sleep_enable_timer_wakeup(sleepUsec) == ESP_OK);
   assert(esp_light_sleep_start() == ESP_OK);
-  //DEBUG_MSG("Exit light sleep\n");
+  //DEBUG_MSG("Exit light sleep b=%d, rf95=%d, pmu=%d\n", digitalRead(BUTTON_PIN), digitalRead(DIO0_GPIO), digitalRead(PMU_IRQ));
   return esp_sleep_get_wakeup_cause();
 }
 
@@ -227,4 +245,4 @@ void enableModemSleep()
   config.light_sleep_enable = false;
   DEBUG_MSG("Sleep request result %x\n", esp_pm_configure(&config));
 }
-#endif
+#endif