Getting 'permission denied' log messages from multiple services after kernel upgrade from 6.11 to 6.12

braveheartleo · November 20, 2024, 11:32pm

The recent kernel upgrade from 6.11 to 6.12 caused several services to throw ‘permission denied’ log messages. These messages didn’t appear from before the upgrade.

bpftune.service:

Nov 21 05:00:32 L5400C bpftune[861]: could not load skeleton: Permission denied
Nov 21 05:00:33 L5400C bpftune[861]: could not load skeleton: Permission denied
Nov 21 05:00:33 L5400C bpftune[861]: error initializing '/usr/lib/bpftune//netns_tuner.so: Permission denied

irqbalance.service:

Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 136 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 136 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 134 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 134 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 139 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 139 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 137 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 137 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 135 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 135 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 133 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 133 affinity is now unmanaged
Nov 21 05:00:40 L5400C irqbalance[779]: Cannot change IRQ 138 affinity: Permission denied
Nov 21 05:00:40 L5400C irqbalance[779]: IRQ 138 affinity is now unmanaged

System:

           .-------------------------:                    ianp@L5400C
          .+=========================.                    -----------
         :++===++==================-       :++-           OS: CachyOS Linux x86_64
        :*++====+++++=============-        .==:           Host: Sarien (rev17)
       -*+++=====+***++==========:                        Kernel: Linux 6.12.0-2-cachyos
      =*++++========------------:                         Uptime: 38 seconds
     =*+++++=====-                     ...                Packages: 1025 (pacman)
   .+*+++++=-===:                    .=+++=:              Shell: fish 3.7.1
  :++++=====-==:                     -*****+              Display (LGD068B): 1920x1080 @ 60 Hz in 14" [Built-in]
 :++========-=.                      .=+**+.              DE: Cinnamon 6.2.9
.+==========-.                          .                 WM: Muffin (X11)
 :+++++++====-                                .--==-.     WM Theme: Mint-Y-Dark-Grey (Adwaita)
  :++==========.                             :+++++++:    Theme: Fusion [Qt], Mint-Y-Dark-Grey [GTK2/3/4]
   .-===========.                            =*****+*+    Icons: Mint-Y-Grey [Qt], Mint-Y-Grey [GTK2/3/4]
    .-===========:                           .+*****+:    Font: Cantarell (11pt) [Qt], Cantarell (11pt) [GTK2/3/4]
      -=======++++:::::::::::::::::::::::::-:  .---:      Cursor: Bibata-Modern-Classic (24px)
       :======++++====+++******************=.             Terminal: GNOME Terminal 3.54.1
        :=====+++==========++++++++++++++*-               Terminal Font: Source Code Pro (11pt)
         .====++==============++++++++++*-                CPU: Intel(R) Core(TM) i5-8265U (8) @ 3.90 GHz
          .===+==================+++++++:                 GPU: Intel UHD Graphics 620 @ 1.10 GHz [Integrated]
           .-=======================+++:                  Memory: 2.58 GiB / 31.22 GiB (8%)
             ..........................                   Swap: 0 B / 31.22 GiB (0%)
                                                          Disk (/): 25.72 GiB / 120.00 GiB (21%) - btrfs
                                                          Disk (/mnt/datastor): 179.72 GiB / 755.58 GiB (24%) - ext4
                                                          Local IP (wlan0): 192.168.8.6/24
                                                          Battery (DELL 27W580C): 79% [AC Connected]
                                                          Locale: en_PH.UTF-8

What could be the issue here that’s causing these messages to appear in the new 6.12?

naim · November 21, 2024, 2:17am

I think that’s pretty normal for bpftune, unless 6.11 doesn’t have it. IDK about irqbalance, probably needs updating.

braveheartleo · November 21, 2024, 3:13am

This is bpftune.service under Linux version 6.11.8-2-cachyos

Nov 20 07:20:55 L5400C systemd[1]: Started BPF-based auto-tuning of system parameters.
Nov 20 07:20:55 L5400C bpftune[855]: bpftune works fully
Nov 20 07:20:55 L5400C bpftune[855]: bpftune supports per-netns policy (via netns cookie)
Nov 20 07:20:57 L5400C bpftune[855]: could not load skeleton: No such process
Nov 20 07:20:57 L5400C bpftune[855]: could not load skeleton: No such process
Nov 20 07:52:29 L5400C bpftune[855]: user (pid 3558) modified sysctl 'ipv4/tcp_rmem' that tuner 'tcp_buffer' uses; disabling 'tcp_buffer' for namespace cookie 1
Nov 20 07:56:16 L5400C systemd[1]: Stopping BPF-based auto-tuning of system parameters...
Nov 20 07:56:17 L5400C bpftune[855]: Summary: scenario 'specify TCP congestion control algorithm' occurred 32 times for tunable 'TCP congestion control' in global ns. To optimize TCP performance, a TCP congesti>
Nov 20 07:56:17 L5400C systemd[1]: bpftune.service: Deactivated successfully.
Nov 20 07:56:17 L5400C systemd[1]: Stopped BPF-based auto-tuning of system parameters.
Nov 20 07:56:17 L5400C systemd[1]: bpftune.service: Consumed 2.990s CPU time, 88.6M memory peak.

This is how it looks under Linux version 6.12.0-2-cachyos

Nov 21 10:39:57 L5400C systemd[1]: Started BPF-based auto-tuning of system parameters.
Nov 21 10:39:57 L5400C bpftune[892]: bpftune works fully
Nov 21 10:39:57 L5400C bpftune[892]: bpftune supports per-netns policy (via netns cookie)
Nov 21 10:39:59 L5400C bpftune[892]: could not load skeleton: Permission denied
Nov 21 10:40:00 L5400C bpftune[892]: could not load skeleton: Permission denied
Nov 21 10:40:00 L5400C bpftune[892]: error initializing '/usr/lib/bpftune//netns_tuner.so: Permission denied
Nov 21 10:40:00 L5400C bpftune[892]: could not load skeleton: No such process
Nov 21 10:40:02 L5400C bpftune[892]: could not load skeleton: No such process

The three lines from above are the new log messages under 6.12 that weren’t present under 6.11.

naim · November 21, 2024, 4:08am

For bpftune, this has been fixed upstream and I have opened a PR that fixes the PKGBUILD bpftune-git: Rework PKGBUILD by 1Naim · Pull Request #434 · CachyOS/CachyOS-PKGBUILDS · GitHub

Nov 21 12:04:15 varvalian systemd[1]: Started BPF-based auto-tuning of system parameters.
Nov 21 12:04:15 varvalian bpftune[42129]: bpftune works fully
Nov 21 12:04:15 varvalian bpftune[42129]: bpftune supports per-netns policy (via netns cookie)
Nov 21 12:04:17 varvalian bpftune[42129]: tuner 'route_table' failed to load, tracing target was not found; this can occur for unstable tracing targets like kernel functions.
Nov 21 12:04:18 varvalian bpftune[42129]: tuner 'tcp_buffer' failed to load, tracing target was not found; this can occur for unstable tracing targets like kernel functions.
Nov 21 12:05:30 varvalian bpftune[42129]: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput
Nov 21 12:05:30 varvalian bpftune[42129]: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 19200000) -> (4096 131072 24000000)
Nov 21 12:05:30 varvalian bpftune[42129]: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput
Nov 21 12:05:30 varvalian bpftune[42129]: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 24000000) -> (4096 131072 30000000)```

braveheartleo · November 21, 2024, 4:32am

Thanks for the heads-up on bpftune. Now remains the permission issue with irqbalance.

naim · November 21, 2024, 7:23am

Try building from git AUR (en) - irqbalance-git. Last tagged release was 8 months ago.

braveheartleo · November 21, 2024, 8:50am

That irqbalance-git is older (1.9.2.r22.g4a60b1a-1) than the one installed (1.9.4-2).

naim · November 21, 2024, 11:28am

It’s a git package. Version will update when you build it.

braveheartleo · November 21, 2024, 11:37am

Didn’t know that, thanks for letting me know. I have installed irqbalance-git (1.9.4.r63.gb4b6f19-1) and still get the same ‘permission denied’ messages:

Nov 21 19:35:10 L5400C systemd[1]: Started irqbalance daemon.
Nov 21 19:35:11 L5400C irqbalance[786]: thermal: failed to receive messages.
Nov 21 19:35:11 L5400C irqbalance[786]: Failed to initialize thermal events.
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 136 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 136 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 134 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 134 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 139 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 139 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 137 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 137 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 135 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 135 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 133 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 133 affinity is now unmanaged
Nov 21 19:35:20 L5400C irqbalance[786]: Cannot change IRQ 138 affinity: Permission denied
Nov 21 19:35:20 L5400C irqbalance[786]: IRQ 138 affinity is now unmanaged

damentz · December 16, 2024, 4:37pm

Looks like this is being reported in multiple locations, it’s an upstream kernel regression: