flashing DFuse-Files (.dfu) not working in 0.10

I stepped through it in gdb and found that all the way through dfu_load_file(), everything seems fine, but as soon as that function exits, the dfu_file struct from size.prefix onwards gets magically shifted over by four bytes. In other words, immediately before the end of dfu_load_file(), the struct is correct, and immediately afterwards, it is somehow incorrect.

(gdb) n
347             if ((res || file->size.prefix == 0) && check_prefix == NEEDS_PREFIX)
(gdb) print *file
$11 = {
  name = 0x21399291c50 "D:/Git/qmk_firmware_develop/testboard_proton_c_is31fl3731.bin",
  firmware = 0x21399293f00 "",
  size = {
    total = 28556,
    prefix = 0,
    suffix = 16
  },
  lmdfu_address = 0,
  prefix_type = 0,
  dwCRC = 1167771274, // 459A C68A
  bcdDFU = 57105, // DF11
  idVendor = 65535, // FFFF
  idProduct = 65535, // FFFF
  bcdDevice = 65535 // FFFF
}
(gdb) n
main (argc=9, argv=0x21399291900) at main.c:385
385                     if (match_vendor < 0 && file.idVendor != 0xffff) {
(gdb) print file
$12 = {
  name = 0x21399291c50 "D:/Git/qmk_firmware_develop/testboard_proton_c_is31fl3731.bin",
  firmware = 0x21399293f00 "",
  size = {
    total = 28556,
    prefix = 0,
    suffix = 0
  },
  lmdfu_address = 16,
  prefix_type = 0,
  dwCRC = 0,
  bcdDFU = 50826, // C68A
  idVendor = 17818, // 459A
  idProduct = 57105, // DF11
  bcdDevice = 65535 // FFFF
}

I believe this has something to do with commit 4a18c61, in which the type of size.total was changed from int to off_t. Reverting it seems to resolve the issue.

Thanks for digging deeper into this. It looks like a mismatch between the struct types which shouldn't happen because it is the same, defined in dfu_file.h. Are you sure you are rebuilding everything from scratch and that there is no stale main.o or something? What platform is this (full details please), and what build environment do you use?

In gdb you can use ptype /o file inside both main() and dfu_load_file() to see the field sizes and if there is a difference. Note that I am not sure if gdb will show a difference if there is a mismatch. Maybe info scope main and info scope dfu_load_file tells something but in one case "file" is a pointer and not the struct itself.

Indeed, ptype /o file shows size.total at 4 bytes before dfu_load_file(), and 8 bytes once I step in.

I'm on Windows 10, building inside MSYS2 with GCC 10.2.0, simply following the instructions in the INSTALL file. Definitely no stale object files.

The bug does not appear to be present in the macOS (Homebrew) build, though I'm not sure what version it was built with.

I figured it out! The issue is that main.c does not include config.h, because (at least in mingw) _FILE_OFFSET_BITS is defined to 64. Oddly though, now off_t is only 4 bytes wide instead of the 8 that I would expect from the above define... but at least they match now.

Actually, looks like config.h is included through portable.h, but it's included too late for the _FILE_OFFSET_BITS define to make any difference. Simply moving it up gives an 8 byte off_t in both scopes now:

diff --git a/src/main.c b/src/main.c
index f204a40..f8f7d88 100644
--- a/src/main.c
+++ b/src/main.c
@@ -25,6 +25,8 @@
  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
  */

+#include "portable.h"
+
 #include <stdlib.h>
 #include <stdio.h>
 #include <string.h>
@@ -34,7 +36,6 @@
 #include <fcntl.h>
 #include <limits.h>

-#include "portable.h"
 #include "dfu.h"
 #include "dfu_file.h"
 #include "dfu_load.h"

Ah, great! Right, this is the newly introduced _FILE_OFFSET_BITS fun which affects everywhere off_t end up being used, it all makes sense now. portable.h is getting a bit tricky because most of it should be evaluated early (only based on config.h) but for instance the O_BINARY part should probably only be checked after system headers. I think we need to properly add the early ifdef HAVE_CONFIG stanza to practically all files because they end up including dfu_file.h one way or another.

Should be fixed in master now, with commit ff17547.

Thanks again for chasing this down.