MQTT at least talks to server, works in native and esp32

docs/software/TODO.md

@@ -53,11 +53,22 @@ You probably don't care about this section - skip to the next one.
 
 ## MQTT
 
+* DONE have sim provide a fake wifi connection status saying connected
+* DONE don't start MQTT if we don't have wifi connected
 * mqtt.meshtastic.org should have VERY basic auth at launch (to prevent abuse)
-* use MQTT for simulator mesh network
+* make a GlobalChat channel as an initial test (with a well known AES128 key), figure out how globally unique IDs work
+* Give place in android app for users to select which channel they are sending on (and which channels they are watching)
+* attempt reconnect to server if internet connectivity changes
+* don't bother contacting server if we don't have any uplink/downlink channels
+* test on ESP32
+* no need for python gateway to web initially: because both the simulator and ESP32 can talk wifi directly
+* if simmesh_name is set in preferences, create the MQTTSimInterface using that as the global channel_id
+* figure out how to use MQTT for simulator mesh network, use a special simmesh_name global channel_id? (because this would allow all nodes in simnet_xxx to subscribe only to those packets)
 * do initial development inside of portduino
 * do as much possible on the device side (so we can eventually just have ESP32 talk directly to server)
+* add mqtt_server to radio prefs
 * eventually add a MQTTPacket on the ToRadio & FromRadio links
+* LATER: an android gateway would be useful
 
 ## Multichannel support
 

docs/software/mqtt.md

@@ -12,7 +12,9 @@
       - [NODEID](#nodeid)
       - [USERID](#userid)
       - [CHANNELID](#channelid)
+      - [PORTID](#portid)
     - [Gateway nodes](#gateway-nodes)
+    - [MQTTSimInterface](#mqttsiminterface)
     - [Optional web services](#optional-web-services)
       - [Public MQTT broker service](#public-mqtt-broker-service)
       - [Riot.im messaging bridge](#riotim-messaging-bridge)
@@ -68,17 +70,19 @@ Any gateway-device will contact the MQTT broker.
 
 ### Topics
 
-The "mesh/crypt/CHANNELID/NODEID/PORTID" [topic](https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/) will be used for messages sent from/to a mesh.
+* The "/mesh/crypt/CHANNELID/NODEID/PORTID" [topic](https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/) will be used for messages sent from/to a mesh.
 
 Gateway nodes will foward any MeshPacket from a local mesh channel with uplink_enabled.  The packet (encapsulated in a ServiceEnvelope) will remain encrypted with the key for the specified channel.  
 
-For any channels in the local node with downlink_enabled, the gateway node will forward packets from MQTT to the local mesh.  It will do this by subscribing to mesh/crypt/CHANNELID/# and forwarding relevant packets.
+For any channels in the gateway node with downlink_enabled, the gateway node will forward packets from MQTT to the local mesh.  It will do this by subscribing to mesh/crypt/CHANNELID/# and forwarding relevant packets.
 
 If the channelid 'well known'/public it could be decrypted by a web service (if the web service was provided with the associated channel key), in which case it will be decrypted by a web service and appear at "mesh/clear/CHANNELID/NODEID/PORTID". Note: This is not in the initial deliverable. 
 
-FIXME, discuss how text message global mirroring could scale (or not)
+FIXME, consider how text message global mirroring could scale (or not)
 FIXME, possibly don't global mirror text messages - instead rely on matrix/riot? 
-FIXME, discuss possible attacks by griefers and how they can be prvented
+FIXME, consider possible attacks by griefers and how they can be prvented
+
+* The "/mstat/NODEID" topic contains a simple string showing connection status of nodes.  We rely on the MQTT feature for automatically publishing special failrue messages to this topic when the device disconnects.
 
 #### Service Envelope
 
@@ -88,7 +92,7 @@ ServiceEnvelope will include the message, and full information about arrival tim
 
 #### NODEID
 
-The unique ID for a node. A hex string that starts with a ! symbol.
+The unique ID for a node. A 8 byte (16 character) hex string that starts with a ! symbol.
 
 #### USERID
 
@@ -96,7 +100,15 @@ A user ID string. This string is either a user ID if known or a nodeid to simply
 
 #### CHANNELID
 
-FIXME, figure out how channelids work
+For the time being we simply use the local "channel name" - which is not quite good enough.
+
+FIXME, figure out how channelids work in more detail.  They should generally be globally unique, but this is not a requirement.  If someone accidentially (or maliciously) sends data using a channel ID they do not 'own' they will still lacking a valid AES256 encryption, so it will be ignored by others.
+
+idea to be pondered: When the user clicks to enable uplink/downlink check the name they entered and 'claim' it on the server?
+
+#### PORTID
+
+Portid is used to descriminated between different packet types which are sent over a channel.  As used here it is an integer typically (but not necessarily) chosen from portnums.proto.
 
 ### Gateway nodes
 
@@ -107,6 +119,17 @@ Gateway nodes (via code running in the phone) will contain two tables to whiteli
 Since multiple gateway nodes might be connected to a single mesh, it is possible that duplicate messages will be published on any particular topic.  Therefore subscribers to these topics should
 deduplicate if needed by using the packet ID of each message.
 
+
+### MQTTSimInterface
+
+This is a bit orthogonal from the main MQTT feature set, but a special simulated LoRa interface called MQTTSimInterface uses the
+MQTT messaging infrastructure to send "LoRa" packets between simulated nodes running on Linux.  This allows us to test radio topologies and code without having to use real hardware.
+
+This service uses the standard mesh/crypt/... topic, but it picks a special CHANNEL_ID.  That CHANNEL_ID is typcially of the form "simmesh_xxx".  
+
+FIXME: Figure out how to secure the creation and use of well known CHANNEL_IDs.
+
+
 ### Optional web services
 
 #### Public MQTT broker service