Weird issue with grub2-mkconfig on Rocky Linux

I am working on a new base image and ran into a weird issue with grub2-mkconfig on Rocky Linux where it wouldn't end up putting anything in the GRUB config file. It drove me crazy - I was doing everything correctly - but it still didn't work. So I edited the grub2-mkconfig script to add some debugging:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
echo "Looping over files in ${grub_mkconfig_dir} ..." >&2

for i in "${grub_mkconfig_dir}"/* ; do
case "$i" in
# emacsen backup files. FIXME: support other editors
*~) ;;
# emacsen autosave files. FIXME: support other editors
*/\#*\#) ;;
# rpm config files of yore.
*.rpmsave|*.rpmnew|*.rpmorig) ;;
*)
if grub_file_is_not_garbage "$i" && test -x "$i" ; then
echo
echo "### BEGIN $i ###"
"$i"
echo "### END $i ###"
else
echo "$i is garbage or not executable" >&2 # debugging added
fi
;;
esac
done

Then, I ran the script again and saw this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[root@ip-192-168-254-80 /]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Looping over files in /etc/grub.d ...
/etc/grub.d/00_header is garbage or not executable
/etc/grub.d/00_tuned is garbage or not executable
/etc/grub.d/01_users is garbage or not executable
/etc/grub.d/08_fallback_counting is garbage or not executable
/etc/grub.d/10_linux is garbage or not executable
/etc/grub.d/10_reset_boot_success is garbage or not executable
/etc/grub.d/12_menu_auto_hide is garbage or not executable
/etc/grub.d/20_linux_xen is garbage or not executable
/etc/grub.d/20_ppc_terminfo is garbage or not executable
/etc/grub.d/30_os-prober is garbage or not executable
/etc/grub.d/30_uefi-firmware is garbage or not executable
/etc/grub.d/35_fwupd is garbage or not executable
/etc/grub.d/40_custom is garbage or not executable
/etc/grub.d/41_custom is garbage or not executable
/etc/grub.d/README is garbage or not executable
Script `/boot/grub2/grub.cfg.new' contains no commands and will do nothing
Syntax errors are detected in generated GRUB config file.
Ensure that there are no errors in /etc/default/grub
and /etc/grub.d/* files or please file a bug report with
/boot/grub2/grub.cfg.new file attached.

Why is each file garbage or not executable?? So I poked around with stat and test -x and noticed that every one of those scripts was mode 0755 but not executable. That's weird, right?!

This is where it becomes a LFMF: I put /etc in its own ZFS dataset that I set -o exec=off on. It turns out that GRUB and several other things require the ability to execute files on the /etc filesystem. Oops! Fixing this made grub2-mkconfig work as expected.