Fixes for #4395: nrf52 flash filesystem reliability (#4406)

* bug #4184: fix config file loss due to filesystem write errors
* Use SafeFile for atomic file writing (with xor checksum readback)
* Write db.proto last because it could be the largest file on the FS (and less critical)
* Don't keep a tmp file around while writing db.proto (because too big to fit two files in the filesystem)
* generate a new critial fault if we encounter errors writing to flash
either CriticalErrorCode_FLASH_CORRUPTION_RECOVERABLE or CriticalErrorCode_FLASH_CORRUPTION_UNRECOVERABLE
(depending on if the second write attempt worked)
* reformat the filesystem if we detect it is corrupted (then rewrite our config files) (only on nrf52 - not sure
yet if we should bother on ESP32)
* If we have to format the FS, make sure to preserve the oem.proto if it exists

* add logLegacy() so old C code in libs can log via our logging

* move filesList() to a better location (used only in developer builds)

* Reformat with "trunk fmt" to match our coding conventions

* for #4395: don't use .exists() to before attempting file open
If a LFS filesystem is corrupted, .exists() can fail when a mere .open()
attempt would have succeeded.  Therefore better to do the .open() in hopes that
we can read the file (in case we need to reformat to fix the FS).
(Seen and confirmed in stress testing)

* for #4395 more fixes, see below for details:
* check for LFS assertion failures during file operations (needs customized lfs_util.h to provide suitable hooks)
* Remove fsCheck() because checking filesystem by writing to it is very high risk, it makes likelyhood that we will
be able to read the config protobufs quite low.
* Update the LFS inside of adafruitnrf52 to 1.7.2 (from their old 1.6.1) to get the following fix:
97d8d5e96a

* use disable_adafruit_usb.py now that we are (temporarily?) using a forked adafruit lib
We need to reach inside the adafruit project and turn off USE_TINYUSB, just doing that
from platformio.ini is no longer sufficient.

Tested on a wio-sdk-wm1110 board (which is the only board that had this problem)

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
This commit is contained in:
geeksville
2024-08-13 04:45:39 -07:00
committed by GitHub
parent 6e8300287b
commit 62a0321c7d
11 changed files with 285 additions and 155 deletions

View File

@@ -18,10 +18,7 @@ import subprocess
import sys
from platformio.project.exception import PlatformioException
from platformio.public import (
DeviceMonitorFilterBase,
load_build_metadata,
)
from platformio.public import DeviceMonitorFilterBase, load_build_metadata
# By design, __init__ is called inside miniterm and we can't pass context to it.
# pylint: disable=attribute-defined-outside-init
@@ -32,7 +29,7 @@ IS_WINDOWS = sys.platform.startswith("win")
class Esp32C3ExceptionDecoder(DeviceMonitorFilterBase):
NAME = "esp32_c3_exception_decoder"
PCADDR_PATTERN = re.compile(r'0x4[0-9a-f]{7}', re.IGNORECASE)
PCADDR_PATTERN = re.compile(r"0x4[0-9a-f]{7}", re.IGNORECASE)
def __call__(self):
self.buffer = ""
@@ -75,14 +72,14 @@ See https://docs.platformio.org/page/projectconf/build_configurations.html
% self.__class__.__name__
)
return False
if not os.path.isfile(self.addr2line_path):
sys.stderr.write(
"%s: disabling, addr2line at %s does not exist\n"
% (self.__class__.__name__, self.addr2line_path)
)
return False
return True
except PlatformioException as e:
sys.stderr.write(
@@ -117,7 +114,7 @@ See https://docs.platformio.org/page/projectconf/build_configurations.html
trace = self.get_backtrace(m)
if len(trace) != "":
text = text[: last] + trace + text[last :]
text = text[:last] + trace + text[last:]
last += len(trace)
return text
@@ -125,14 +122,10 @@ See https://docs.platformio.org/page/projectconf/build_configurations.html
def get_backtrace(self, match):
trace = "\n"
enc = "mbcs" if IS_WINDOWS else "utf-8"
args = [self.addr2line_path, u"-fipC", u"-e", self.firmware_path]
args = [self.addr2line_path, "-fipC", "-e", self.firmware_path]
try:
addr = match.group()
output = (
subprocess.check_output(args + [addr])
.decode(enc)
.strip()
)
output = subprocess.check_output(args + [addr]).decode(enc).strip()
output = output.replace(
"\n", "\n "
) # newlines happen with inlined methods