Skip to content

Commit 053865f

Browse files
author
Memfault Inc
committed
Memfault Firmware SDK 1.17.0 (Build 11353)
1 parent 7cc391d commit 053865f

File tree

51 files changed

+699
-170
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+699
-170
lines changed

CHANGELOG.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,97 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to
77
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
88

9+
## [1.17.0] - 2024-11-14
10+
11+
### 📈 Added
12+
13+
- General:
14+
15+
- Add parsing of an optional log message argument for the `test_trace` command
16+
in the [core demo cli](components/demo). This argument will be inserted as a
17+
custom log message with the trace event, which can be useful for testing the
18+
insertion of custom log messages into trace events at runtime.
19+
20+
- Add the following built-in metrics, enabled by default (and quota-exempt):
21+
22+
- `MemfaultSDKMetric_log_recorded_lines` : the total number of log lines
23+
written into the Memfault log buffer during the heartbeat interval
24+
- `MemfaultSDKMetric_log_dropped_lines` : the total number of log lines
25+
dropped (overwritten or not recorded) due to buffer exhaustion during the
26+
heartbeat interval
27+
28+
For example, if the buffer is sufficiently large to store 10 logs, and 15
29+
logs are written:
30+
31+
- `MemfaultSDKMetric_log_recorded_lines` will be 15
32+
- `MemfaultSDKMetric_log_dropped_lines` will be 5
33+
34+
If 5 more logs are written:
35+
36+
- `MemfaultSDKMetric_log_recorded_lines` will be 20
37+
- `MemfaultSDKMetric_log_dropped_lines` will be 10
38+
39+
- Cosmetic updates to the `examples/freertos/` app log format, including
40+
applying ANSI color codes based on log level.
41+
42+
```bash
43+
# Before:
44+
mflt> test_log
45+
Raw log!
46+
2024-11-14T17:01:12Z|4284 I Info log!
47+
2024-11-14T17:01:12Z|4284 W Warning log!
48+
2024-11-14T17:01:12Z|4284 E Error log!
49+
50+
# After:
51+
mflt> test_log
52+
Raw log!
53+
MFLT:[INFO] Info log!
54+
MFLT:[WARN] Warning log!
55+
MFLT:[ERRO] Error log!
56+
```
57+
58+
- ESP-IDF:
59+
60+
- Enable NTP time synchronization by default, controlled with the Kconfig
61+
option `CONFIG_MEMFAULT_NTP_SYNC`. After NTP synchronization, events
62+
(heartbeats and trace events) will be timestamped with the current device
63+
time.
64+
65+
- Add a `test_trace` command to the ESP-IDF demo cli to capture an example
66+
trace event. This behaves the same as the `test_trace` command in the
67+
[core demo cli](components/demo).
68+
- Mark coredumps with the `Software Watchdog` crash reason if the IWDT
69+
triggered the fault, instead of marking them as `Hard Fault`.
70+
71+
- Add OTA update check-ins to the HTTP client's periodic upload, controlled
72+
with the Kconfig option `CONFIG_MEMFAULT_HTTP_PERIODIC_UPLOAD_OTA`. When
73+
enabled, the system will be restarted after downloading the update. To
74+
customize this behavior, enable
75+
`CONFIG_MEMFAULT_HTTP_PERIODIC_UPLOAD_OTA_CUSTOM_CBS`, and implement the
76+
global callback struct `g_memfault_ota_update_handler`. This feature is
77+
disabled by default.
78+
79+
### 🐛 Fixed
80+
81+
- General:
82+
83+
- Add missing type promotion rules for variants of `char` pointers when
84+
encoding compact logs. Previously, these types that should promote to a
85+
string in the compact logging encoding fell into the default case of `int`,
86+
causing compact logging decode to fail when processed by the Memfault
87+
backend.
88+
89+
- ESP-IDF:
90+
91+
- Fix an issue when using compact logs with
92+
`CONFIG_MEMFAULT_USE_MEMFAULT_BUILD_ID=y` (default). The command would
93+
always run (was always out-of-date) when any `idf.py [build|flash]` command
94+
is run, even if the original .elf file did not change. This caused the
95+
`log_fmt` section (used for decoding compact logs) to be removed from the
96+
`/memfault-esp32-demo-app.elf.memfault_log_fmt` file, which causes Memfault
97+
Compact Logs to fail to decode. The command is fixed to only run when the
98+
.elf file changes.
99+
9100
## [1.16.0] - 2024-10-24
10101
11102
### 🔥 Removed
@@ -58,12 +149,18 @@ earlier versions!
58149
`memfault_reboot_tracking_load()` / `memfault_reboot_tracking_save()`
59150
functions to demonstrate the functionality.
60151
152+
- Added a Kconfig flag, `CONFIG_MEMFAULT_METRICS_LOGS_ENABLE`, to control the
153+
new log lines metrics.
154+
61155
- ESP-IDF:
62156
63157
- New Kconfig setting, `CONFIG_MEMFAULT_ENABLE_REBOOT_DIAG_DUMP`, to print the
64158
ESP-IDF reboot reason code on system boot, for debugging purposes. This
65159
feature is disabled by default.
66160
161+
- Added a Kconfig flag, `CONFIG_MEMFAULT_METRICS_LOGS_ENABLE`, to control the
162+
new log lines metrics.
163+
67164
### 🛠️ Changed
68165
69166
- General:

VERSION

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
BUILD ID: 10952
2-
GIT COMMIT: c690f5abc6
3-
VERSION: 1.16.0
1+
BUILD ID: 11353
2+
GIT COMMIT: 767ed66f9e
3+
VERSION: 1.17.0

components/core/src/memfault_log.c

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,11 @@ typedef struct {
4646
// Can be changed with memfault_log_set_min_save_level()
4747
eMemfaultPlatformLogLevel min_log_level;
4848
uint32_t log_read_offset;
49-
// The number of messages that were flushed without ever being read. If memfault_log_read() is
50-
// not used by a platform, this will be equivalent to the number of messages logged since boot
51-
// that are no longer in the log buffer.
49+
// The total number of messages that were not recorded into the buffer, either
50+
// due to lack of space, or if the buffer was locked by the data source.
5251
uint32_t dropped_msg_count;
52+
// The total number of messages that were recorded into the buffer
53+
uint32_t recorded_msg_count;
5354
sMfltCircularBuffer circ_buffer;
5455
// When initialized we keep track of the user provided storage buffer and crc the location +
5556
// size. When the system crashes we can check to see if this info has been corrupted in any way
@@ -67,6 +68,14 @@ static uint16_t prv_compute_log_region_crc16(void) {
6768
offsetof(sMfltLogStorageRegionInfo, crc16));
6869
}
6970

71+
uint32_t memfault_log_get_dropped_count(void) {
72+
return s_memfault_ram_logger.dropped_msg_count;
73+
}
74+
75+
uint32_t memfault_log_get_recorded_count(void) {
76+
return s_memfault_ram_logger.recorded_msg_count;
77+
}
78+
7079
bool memfault_log_get_regions(sMemfaultLogRegions *regions) {
7180
if (!s_memfault_ram_logger.enabled) {
7281
return false;
@@ -129,11 +138,17 @@ static bool prv_try_free_space(sMfltCircularBuffer *circ_bufp, int bytes_needed)
129138

130139
// Expire oldest logs until there is enough room available
131140
while (tot_read_space != 0) {
141+
// Log lines are stored as 2 entries in the circular buffer:
142+
// 1. sMfltRamLogEntry header
143+
// 2. log message (compact or formatted)
144+
// When freeing space, clear both the header and the message
132145
sMfltRamLogEntry curr_entry = { 0 };
133146
memfault_circular_buffer_read(circ_bufp, 0, &curr_entry, sizeof(curr_entry));
134147
const size_t space_to_free = curr_entry.len + sizeof(curr_entry);
135148

136-
if ((curr_entry.hdr & MEMFAULT_LOG_HDR_READ_MASK) != 0) {
149+
if ((curr_entry.hdr & (MEMFAULT_LOG_HDR_READ_MASK | MEMFAULT_LOG_HDR_SENT_MASK)) != 0) {
150+
// note: log_read_offset can safely wrap around, circular buffer API
151+
// handles the modulo arithmetic
137152
s_memfault_ram_logger.log_read_offset -= space_to_free;
138153
} else {
139154
// We are removing a message that was not read via memfault_log_read().
@@ -220,14 +235,18 @@ static bool prv_read_log_iter_callback(sMfltLogIterator *iter) {
220235
}
221236

222237
static bool prv_read_log(sMemfaultLog *log) {
223-
if (s_memfault_ram_logger.dropped_msg_count) {
224-
log->level = kMemfaultPlatformLogLevel_Warning;
225-
const int rv = snprintf(log->msg, sizeof(log->msg), "... %d messages dropped ...",
226-
(int)s_memfault_ram_logger.dropped_msg_count);
227-
log->msg_len = (rv <= 0) ? 0 : MEMFAULT_MIN((uint32_t)rv, sizeof(log->msg) - 1);
228-
log->type = kMemfaultLogRecordType_Preformatted;
229-
s_memfault_ram_logger.dropped_msg_count = 0;
230-
return true;
238+
static uint32_t s_last_dropped_count = 0;
239+
240+
if (s_last_dropped_count != s_memfault_ram_logger.dropped_msg_count) {
241+
s_last_dropped_count = s_memfault_ram_logger.dropped_msg_count;
242+
if (s_last_dropped_count > 0) {
243+
log->level = kMemfaultPlatformLogLevel_Warning;
244+
const int rv = snprintf(log->msg, sizeof(log->msg), "... %d messages dropped ...",
245+
(int)s_memfault_ram_logger.dropped_msg_count);
246+
log->msg_len = (rv <= 0) ? 0 : MEMFAULT_MIN((uint32_t)rv, sizeof(log->msg) - 1);
247+
log->type = kMemfaultLogRecordType_Preformatted;
248+
return true;
249+
}
231250
}
232251

233252
sMfltReadLogCtx user_ctx = { .log = log };
@@ -393,13 +412,16 @@ static void prv_log_save(eMemfaultPlatformLogLevel level, const void *log, size_
393412
sMfltCircularBuffer *circ_bufp = &s_memfault_ram_logger.circ_buffer;
394413
const bool space_free = prv_try_free_space(circ_bufp, (int)bytes_needed);
395414
if (space_free) {
415+
s_memfault_ram_logger.recorded_msg_count++;
396416
sMfltRamLogEntry entry = {
397417
.len = (uint8_t)truncated_log_len,
398418
.hdr = prv_build_header(level, log_type),
399419
};
400420
memfault_circular_buffer_write(circ_bufp, &entry, sizeof(entry));
401421
memfault_circular_buffer_write(circ_bufp, log, truncated_log_len);
402422
log_written = true;
423+
} else {
424+
s_memfault_ram_logger.dropped_msg_count++;
403425
}
404426
}
405427
if (should_lock) {

components/core/src/memfault_self_test.c

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -352,31 +352,31 @@ MEMFAULT_NO_OPT static uint32_t prv_get_time_since_boot_test(void) {
352352
#define MEMFAULT_SELF_TEST_TIMESTAMP_ANCHOR (1706486400)
353353

354354
static uint32_t prv_platform_time_get_current_test(void) {
355-
sMemfaultCurrentTime time = {
355+
sMemfaultCurrentTime mflt_time = {
356356
.type = kMemfaultCurrentTimeType_Unknown,
357357
.info = {
358358
.unix_timestamp_secs = 0,
359359
},
360360
};
361-
bool result = memfault_platform_time_get_current(&time);
361+
bool result = memfault_platform_time_get_current(&mflt_time);
362362
if (!result) {
363363
MEMFAULT_LOG_ERROR("Current timestamp could not be recovered");
364364
return (1 << 2);
365365
}
366366

367-
if (time.type != kMemfaultCurrentTimeType_UnixEpochTimeSec) {
368-
MEMFAULT_LOG_ERROR("Invalid time type returned: %u", time.type);
367+
if (mflt_time.type != kMemfaultCurrentTimeType_UnixEpochTimeSec) {
368+
MEMFAULT_LOG_ERROR("Invalid time type returned: %u", mflt_time.type);
369369
return (1 << 3);
370370
}
371371

372-
if (time.info.unix_timestamp_secs < MEMFAULT_SELF_TEST_TIMESTAMP_ANCHOR) {
372+
if (mflt_time.info.unix_timestamp_secs < MEMFAULT_SELF_TEST_TIMESTAMP_ANCHOR) {
373373
MEMFAULT_LOG_ERROR("Timestamp too far in the past: %" PRIu32,
374-
(uint32_t)time.info.unix_timestamp_secs);
374+
(uint32_t)mflt_time.info.unix_timestamp_secs);
375375
return (1 << 4);
376376
}
377377

378378
MEMFAULT_LOG_INFO("Verify received timestamp for accuracy. Timestamp received %" PRIu32,
379-
(uint32_t)time.info.unix_timestamp_secs);
379+
(uint32_t)mflt_time.info.unix_timestamp_secs);
380380
return 0;
381381
}
382382

components/demo/src/memfault_demo_cli_trace_event.c

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,16 @@
77
//! CLI command that exercises the MEMFAULT_TRACE_EVENT API, capturing a
88
//! Trace Event with the error reason set to "MemfaultDemoCli_Error".
99

10-
#include "memfault/core/compiler.h"
11-
#include "memfault/core/debug_log.h"
1210
#include "memfault/core/trace_event.h"
1311
#include "memfault/demo/cli.h"
1412

15-
int memfault_demo_cli_cmd_trace_event_capture(int argc, MEMFAULT_UNUSED char *argv[]) {
13+
int memfault_demo_cli_cmd_trace_event_capture(int argc, char *argv[]) {
1614
// For more information on user-defined error reasons, see
1715
// the MEMFAULT_TRACE_REASON_DEFINE macro in trace_reason_user.h .
18-
MEMFAULT_TRACE_EVENT_WITH_LOG(MemfaultCli_Test, "Example Trace Event. Num Args %d", argc);
16+
if (argc < 2) {
17+
MEMFAULT_TRACE_EVENT_WITH_LOG(MemfaultCli_Test, "Example Trace Event. Num Args %d", argc);
18+
} else {
19+
MEMFAULT_TRACE_EVENT_WITH_LOG(MemfaultCli_Test, "%s", argv[1]);
20+
}
1921
return 0;
2022
}

components/include/memfault/core/compact_log_helpers.h

Lines changed: 72 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,38 @@
3939

4040
#ifdef __cplusplus
4141

42-
// C++ implementation of the type promotion logic
43-
//
44-
// Note: the C++ implementation requires the use of C++11 features and the GNU
45-
// "##" variadic arg extension. Memfault Compact Logs in C++ require the
46-
// compiler flag '--std=gnu++-11' or newer.
42+
//! C++ type promotion logic
43+
//!
44+
//! This code defines the MemfaultLogArgPromotionType template structure, responsible for
45+
//! determining the promotion type of different argument types passed to Memfault's logging
46+
//! functions. This is important because certain types may need to be promoted to larger or
47+
//! different types, ensuring consistent handling in logging The code uses C++11 features,
48+
//! including SFINAE (Substitution Failure Is Not An Error) with std::enable_if, and the GNU ## va
49+
//! args extension. It assumes the --std=gnu++11 compiler flag or newer.
50+
//!
51+
//! Structure definition:
52+
//! 1. Default promotion to int64:
53+
//! - By default, if no other promotion rule applies, MemfaultLogArgPromotionType promotes the
54+
//! type to
55+
//! MEMFAULT_LOG_ARG_PROMOTED_TO_INT64, indicating that any unhandled type defaults to a
56+
//! 64-bit integer.
57+
//!
58+
//! 2. Conditional promotion to int32:
59+
//! - If the size of the type T is less than or equal to 4 bytes (e.g., smaller integer types),
60+
//! MemfaultLogArgPromotionType promotes the type to MEMFAULT_LOG_ARG_PROMOTED_TO_INT32,
61+
//! allowing consistent handling for smaller integers.
62+
//!
63+
//! 3. Specialized promotions:
64+
//! - Specific types receive unique promotion constants:
65+
//! - Floating-point types (float, double, long double): Promoted to
66+
//! MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE.
67+
//! - String types (char* variants): Promoted to MEMFAULT_LOG_ARG_PROMOTED_TO_STR.
68+
//!
69+
//! Macro definition:
70+
//! - _MEMFAULT_LOG_ARG_PROMOTION_TYPE(arg): This macro determines the promotion type of an
71+
//! argument.
72+
//! The (arg) + 0 operation is used to enforce integer promotion for non-integer types, helping
73+
//! the compiler resolve the appropriate type for MemfaultLogArgPromotionType.
4774

4875
#include <type_traits>
4976

@@ -78,22 +105,43 @@ template <>
78105
struct MemfaultLogArgPromotionType<const char *>
79106
: std::integral_constant<int, MEMFAULT_LOG_ARG_PROMOTED_TO_STR> { };
80107

81-
// When expressing the final type via the template parameter expansion, operate
82-
// on (arg) + 0 to force integer promotion
108+
template <>
109+
struct MemfaultLogArgPromotionType<signed char *>
110+
: std::integral_constant<int, MEMFAULT_LOG_ARG_PROMOTED_TO_STR> { };
111+
112+
template <>
113+
struct MemfaultLogArgPromotionType<const signed char *>
114+
: std::integral_constant<int, MEMFAULT_LOG_ARG_PROMOTED_TO_STR> { };
115+
116+
template <>
117+
struct MemfaultLogArgPromotionType<unsigned char *>
118+
: std::integral_constant<int, MEMFAULT_LOG_ARG_PROMOTED_TO_STR> { };
119+
120+
template <>
121+
struct MemfaultLogArgPromotionType<const unsigned char *>
122+
: std::integral_constant<int, MEMFAULT_LOG_ARG_PROMOTED_TO_STR> { };
123+
83124
#define _MEMFAULT_LOG_ARG_PROMOTION_TYPE(arg) \
84125
MemfaultLogArgPromotionType<decltype((arg) + 0)>::value
85126

86127
#else // C Code implementation
87128

88-
//! Preprocessor macro to encode promotion type info about each va_arg in a uint32_t
129+
//! C type promotion logic
89130
//!
90131
//! Utilizes the rules around "default argument promotion" (specifically for va_args) defined in
91132
//! the C specification (http://www.iso-9899.info/wiki/The_Standard) to encode information about
92133
//! the width of arguments. (For more details see "6.5.2.2 Function calls" in the C11 Spec).
93134
//!
94-
//! In short,
95-
//! - floats are always promoted to doubles
96-
//! - any other type < sizeof(int) is promoted to the width of an int
135+
//! Promotion Rules:
136+
//! - Floating-point types (float, double, long double): Promoted to
137+
//! MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE.
138+
//! - String types (char* variants): Promoted to MEMFAULT_LOG_ARG_PROMOTED_TO_STR.
139+
//! - Default case:
140+
//! - If the size of the argument is less than or equal to the size of an int, the type is
141+
//! promoted to
142+
//! MEMFAULT_LOG_ARG_PROMOTED_TO_INT32.
143+
//! - Otherwise, it defaults to MEMFAULT_LOG_ARG_PROMOTED_TO_INT64, for larger integer types.
144+
//! - Notably, bool and _BitInt(N) are safely promoted to int by falling into this case.
97145
//!
98146
//! NOTE 1: We use a special type for "strings" (i.e char *) so we know to encode the value
99147
//! pointed to (the actual NUL terminated string) in this situation rather than the pointer
@@ -118,15 +166,19 @@ struct MemfaultLogArgPromotionType<const char *>
118166

119167
// clang-format off
120168

121-
#define _MEMFAULT_LOG_ARG_PROMOTION_TYPE(arg) \
122-
_Generic((arg) + 0, \
123-
float: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
124-
double: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
125-
long double: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
126-
char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
127-
const char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
128-
default: sizeof((arg) + 0) <= sizeof(int) ? \
129-
MEMFAULT_LOG_ARG_PROMOTED_TO_INT32 : MEMFAULT_LOG_ARG_PROMOTED_TO_INT64)
169+
#define _MEMFAULT_LOG_ARG_PROMOTION_TYPE(arg) \
170+
_Generic((arg) + 0, \
171+
float: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
172+
double: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
173+
long double: MEMFAULT_LOG_ARG_PROMOTED_TO_DOUBLE, \
174+
char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
175+
const char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
176+
signed char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
177+
const signed char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
178+
unsigned char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
179+
const unsigned char*: MEMFAULT_LOG_ARG_PROMOTED_TO_STR, \
180+
default: sizeof((arg) + 0) <= sizeof(int) ? \
181+
MEMFAULT_LOG_ARG_PROMOTED_TO_INT32 : MEMFAULT_LOG_ARG_PROMOTED_TO_INT64)
130182

131183
// clang-format on
132184

0 commit comments

Comments
 (0)