Linux-6-13-rc3 and sched-ext doesnt works

General
1

Hi,

when u compile the rc3 cachy kernel and reboot and looking at the Boot Messages u see maybe too “Failed to start Start scx_scheduler” if u have boot messages enabled in grub with no splash

Than i manually tryd to activate with systemctl start scx.service -no error

Than checked with systemctl status scx with this Result:

❯ systemctl status scx
× scx.service - Start scx_scheduler
Loaded: loaded (/usr/lib/systemd/system/scx.service; enabled; preset: disabled)
Active: failed (Result: exit-code) since Sat 2024-12-21 00:48:13 CET; 3min 2s ago
Duration: 66ms
Invocation: 4e6d688771b540c58c7f17895b3ae57c
Process: 3062 ExecStart=/bin/bash -c exec ${SCX_SCHEDULER_OVERRIDE:-$SCX_SCHEDULER} ${SCX_FLAGS_OVERRIDE:-$SCX_FLAGS} (code=exited, status=1/FAILURE)
Main PID: 3062 (code=exited, status=1/FAILURE)

tryed with the newest cachy scx-scheds 1.0.8-1 and after with the scx-scheds-git

2

What does this command give you: dbus-send --system --print-reply --dest=org.scx.Loader /org/scx/Loader org.freedesktop.DBus.Properties.Get string:org.scx.Loader string:CurrentScheduler

3

❯ dbus-send --system --print-reply --dest=org.scx.Loader /org/scx/Loader org.freedesktop.DBus.Properties.Get string:org.scx.Loader string:CurrentScheduler
method return time=1734748716.450712 sender=:1.5 → destination=:1.111 serial=7 reply_serial=2
variant string “unknown”

4

here the output of the same both commands on the default cachy kernel 6.12.6

❯ dbus-send --system --print-reply --dest=org.scx.Loader /org/scx/Loader org.freedesktop.DBus.Properties.Get string:org.scx.Loader string:CurrentScheduler
method return time=1734749085.185777 sender=:1.6 → destination=:1.61 serial=7 reply_serial=2
variant string “unknown”
❯ systemctl status scx
● scx.service - Start scx_scheduler
Loaded: loaded (/usr/lib/systemd/system/scx.service; enabled; preset: disabled)
Active: active (running) since Sat 2024-12-21 03:44:19 CET; 50s ago
Invocation: b8da4134052a4eb1b64174bc2c15e783
Main PID: 960 (scx_bpfland)
Tasks: 4 (limit: 9253)
Memory: 14.1M (peak: 19.2M)
CPU: 145ms
CGroup: /system.slice/scx.service
└─960 scx_bpfland

5

Works fine for me. What’s the output of sudo scx_bpfland?

6

on rc3 this:

❯ sudo scx_bpfland
04:23:52 [INFO] scx_bpfland 1.0.8-ge8e68e8e-dirty x86_64-unknown-linux-gnu SMT on
Error: Failed to load BPF program

Caused by:
Permission denied (os error 13)

on cachy 6.12.6 normal output except last line

❯ sudo scx_bpfland
[sudo] Mot de passe de franck :
04:45:33 [INFO] scx_bpfland 1.0.8-ge8e68e8e-dirty x86_64-unknown-linux-gnu SMT on
04:45:33 [INFO] primary CPU domain = 0xfff
04:45:33 [INFO] cpufreq performance level: auto
04:45:33 [INFO] L2 cache ID 0: sibling CPUs: [0, 6]
04:45:33 [INFO] L2 cache ID 1: sibling CPUs: [1, 7]
04:45:33 [INFO] L2 cache ID 2: sibling CPUs: [2, 8]
04:45:33 [INFO] L2 cache ID 4: sibling CPUs: [3, 9]
04:45:33 [INFO] L2 cache ID 5: sibling CPUs: [4, 10]
04:45:33 [INFO] L2 cache ID 6: sibling CPUs: [5, 11]
04:45:33 [INFO] L3 cache ID 0: sibling CPUs: [0, 6, 1, 7, 2, 8]
04:45:33 [INFO] L3 cache ID 1: sibling CPUs: [4, 10, 5, 11, 3, 9]
Error: another sched_ext scheduler is already running

7

So I assume this is the rc3 kernel from the repo? Since rc3 is only in the znver4 repos at the moment. IDK why it’s not in the v3 and v4 repos.

8

hi there,

but if so, i installed on v3 the cachyos and never changed the sourses - yets its from the cachy rc repos di installed and compiled the kernnel - rc3

9

and another tinng, whi change to lavd and doing " sudo scx_lavd" it results in

❯ sudo scx_lavd
[sudo] Mot de passe de franck :
07:34:43 [INFO] Autopilot mode is enabled by default.
Error: another sched_ext scheduler is already running

what is where the other scheduler ???
like when u do sudo scx_bpfland

the output also claiming that another scheduler is running …

10

No rc3 in znver4 repo here…

Bildschirmfoto vom 2024-12-21 11-33-40

1 Like
11

i suggest stop the service by sudo systemctl stop scx.service and restart it as example sudo scx_bpfland -v

edit
please use code tags

12 
❯ sudo systemctl stop scx.service
❯ sudo scx_bpfland -v
23:39:45 [INFO] scx_bpfland 1.0.8-ge8e68e8e-dirty x86_64-unknown-linux-gnu SMT on
libbpf: loaded kernel BTF from '/sys/kernel/btf/vmlinux'
libbpf: loading object 'bpf_bpf' from buffer
libbpf: elf: section(2) .symtab, size 6168, link 1, flags 0, type=2
libbpf: elf: section(3) .text, size 2232, link 0, flags 6, type=1
libbpf: sec '.text': found program 'task_set_domain' at insn offset 25 (200 bytes), code size 74 insns (592 bytes)
libbpf: sec '.text': found program 'task_deadline' at insn offset 99 (792 bytes), code size 180 insns (1440 bytes)
libbpf: sec '.text': found program 'try_lookup_cpu_ctx' at insn offset 0 (0 bytes), code size 9 insns (72 bytes)
libbpf: sec '.text': found program 'try_lookup_task_ctx' at insn offset 9 (72 bytes), code size 7 insns (56 bytes)
libbpf: sec '.text': found program 'get_nr_online_cpus' at insn offset 16 (128 bytes), code size 9 insns (72 bytes)
libbpf: elf: section(4) struct_ops/bpfland_select_cpu, size 1824, link 0, flags 6, type=1
.
.
.

libbpf: map 'cpu_ctx_stor': created successfully, fd=5
libbpf: map 'task_ctx_stor': created successfully, fd=6
libbpf: map 'bpf_bpf.rodata': created successfully, fd=7
libbpf: map '.data.BPFLAND': created successfully, fd=8
libbpf: map 'bpf_bpf.bss': created successfully, fd=9
libbpf: map '.data.uei_dump': created successfully, fd=10
libbpf: map 'bpf_bpf.data': created successfully, fd=11
libbpf: map 'bpfland_ops': created successfully, fd=12
libbpf: prog 'bpfland_select_cpu': BPF program load failed: Permission denied
libbpf: prog 'bpfland_select_cpu': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; s32 BPF_STRUCT_OPS(bpfland_select_cpu, struct task_struct *p, @ main.bpf.c:784
0: (79) r9 = *(u64 *)(r1 +16)         ; R1=ctx() R9_w=scalar()
1: (79) r6 = *(u64 *)(r1 +8)          ; R1=ctx() R6_w=scalar()
2: (79) r7 = *(u64 *)(r1 +0)
func 'select_cpu' arg0 has btf_id 142 type STRUCT 'task_struct'
3: R1=ctx() R7_w=trusted_ptr_task_struct()
; struct task_struct *current = (void *)bpf_get_current_task_btf(); @ main.bpf.c:560
3: (85) call bpf_get_current_task_btf#158     ; R0_w=scalar()
4: (7b) *(u64 *)(r10 -24) = r0        ; R0_w=scalar(id=1) R10=fp0 fp-24_w=scalar(id=1)
; return bpf_task_storage_get(&task_ctx_stor, @ main.bpf.c:200
5: (18) r1 = 0xffff929d46be3a00       ; R1_w=map_ptr(map=task_ctx_stor,ks=4,vs=120)
7: (7b) *(u64 *)(r10 -16) = r7        ; R7_w=trusted_ptr_task_struct() R10=fp0 fp-16_w=trusted_ptr_task_struct()
8: (bf) r2 = r7                       ; R2_w=trusted_ptr_task_struct() R7_w=trusted_ptr_task_struct()
9: (b7) r3 = 0                        ; R3_w=0
10: (b7) r4 = 0                       ; R4_w=0
11: (85) call bpf_task_storage_get#156        ; R0=map_value_or_null(id=2,map=task_ctx_stor,ks=4,vs=120)
12: (bf) r7 = r0                      ; R0=map_value_or_null(id=2,map=task_ctx_stor,ks=4,vs=120) R7_w=map_value_or_null(id=2,map=task_ctx_stor,ks=4,vs=120)
13: (b4) w0 = -2                      ; R0_w=0xfffffffe
; if (!tctx) @ main.bpf.c:568
14: (15) if r7 == 0x0 goto pc+64      ; R7_w=map_value(map=task_ctx_stor,ks=4,vs=120)
15: (b4) w0 = -22                     ; R0_w=0xffffffea
; primary = cast_mask(primary_cpumask); @ main.bpf.c:571
16: (18) r1 = 0xffff929d46be1b20      ; R1_w=map_value(map=.data.BPFLAND,ks=4,vs=8)
18: (79) r8 = *(u64 *)(r1 +0)         ; R1_w=map_value(map=.data.BPFLAND,ks=4,vs=8) R8_w=rcu_ptr_or_null_bpf_cpumask(id=3)
; if (!primary) @ main.bpf.c:572
19: (15) if r8 == 0x0 goto pc+59      ; R8_w=rcu_ptr_bpf_cpumask()
; idle_smtmask = scx_bpf_get_idle_smtmask(); @ main.bpf.c:578
20: (85) call scx_bpf_get_idle_smtmask#92527          ; R0=ptr_cpumask(ref_obj_id=4) refs=4
21: (7b) *(u64 *)(r10 -40) = r0       ; R0=ptr_cpumask(ref_obj_id=4) R10=fp0 fp-40_w=ptr_cpumask(ref_obj_id=4) refs=4
; idle_cpumask = scx_bpf_get_idle_cpumask(); @ main.bpf.c:579
22: (85) call scx_bpf_get_idle_cpumask#92526          ; R0_w=ptr_cpumask(ref_obj_id=5) refs=4,5
23: (7b) *(u64 *)(r10 -32) = r0       ; R0_w=ptr_cpumask(ref_obj_id=5) R10=fp0 fp-32_w=ptr_cpumask(ref_obj_id=5) refs=4,5
; p_mask = cast_mask(tctx->cpumask); @ main.bpf.c:584
24: (79) r2 = *(u64 *)(r7 +0)         ; R2_w=rcu_ptr_or_null_bpf_cpumask(id=6) R7=map_value(map=task_ctx_stor,ks=4,vs=120) refs=4,5
; if (!p_mask) { @ main.bpf.c:585
25: (55) if r2 != 0x0 goto pc+7 33: R0_w=ptr_cpumask(ref_obj_id=5) R2_w=rcu_ptr_bpf_cpumask() R6=scalar() R7=map_value(map=task_ctx_stor,ks=4,vs=120) R8=rcu_ptr_bpf_cpumask() R9=scalar() R10=fp0 fp-16=trusted_ptr_task_struct() fp-24=scalar(id=1) fp-32_w=ptr_cpumask(ref_obj_id=5) fp-40_w=ptr_cpumask(ref_obj_id=4) refs=4,5
; l2_mask = cast_mask(tctx->l2_cpumask); @ main.bpf.c:591
33: (79) r3 = *(u64 *)(r7 +8)         ; R3_w=rcu_ptr_or_null_bpf_cpumask(id=7) R7=map_value(map=task_ctx_stor,ks=4,vs=120) refs=4,5
; if (!l2_mask) { @ main.bpf.c:592
34: (55) if r3 != 0x0 goto pc+7 42: R0=ptr_cpumask(ref_obj_id=5) R2=rcu_ptr_bpf_cpumask() R3=rcu_ptr_bpf_cpumask() R6=scalar() R7=map_value(map=task_ctx_stor,ks=4,vs=120) R8=rcu_ptr_bpf_cpumask() R9=scalar() R10=fp0 fp-16=trusted_ptr_task_struct() fp-24=scalar(id=1) fp-32=ptr_cpumask(ref_obj_id=5) fp-40=ptr_cpumask(ref_obj_id=4) refs=4,5
; l3_mask = cast_mask(tctx->l3_cpumask); @ main.bpf.c:598
42: (79) r1 = *(u64 *)(r7 +16)        ; R1_w=rcu_ptr_or_null_bpf_cpumask(id=8) R7=map_value(map=task_ctx_stor,ks=4,vs=120) refs=4,5
43: (7b) *(u64 *)(r10 -48) = r1       ; R1_w=rcu_ptr_or_null_bpf_cpumask(id=8) R10=fp0 fp-48_w=rcu_ptr_or_null_bpf_cpumask(id=8) refs=4,5
; if (!l3_mask) { @ main.bpf.c:599
44: (55) if r1 != 0x0 goto pc+35 80: R0=ptr_cpumask(ref_obj_id=5) R1_w=rcu_ptr_bpf_cpumask() R2=rcu_ptr_bpf_cpumask() R3=rcu_ptr_bpf_cpumask() R6=scalar() R7=map_value(map=task_ctx_stor,ks=4,vs=120) R8=rcu_ptr_bpf_cpumask() R9=scalar() R10=fp0 fp-16=trusted_ptr_task_struct() fp-24=scalar(id=1) fp-32=ptr_cpumask(ref_obj_id=5) fp-40=ptr_cpumask(ref_obj_id=4) fp-48_w=rcu_ptr_bpf_cpumask() refs=4,5
; s32 BPF_STRUCT_OPS(bpfland_select_cpu, struct task_struct *p, @ main.bpf.c:784
80: (7b) *(u64 *)(r10 -64) = r3       ; R3=rcu_ptr_bpf_cpumask() R10=fp0 fp-64_w=rcu_ptr_bpf_cpumask() refs=4,5
81: (7b) *(u64 *)(r10 -72) = r2       ; R2=rcu_ptr_bpf_cpumask() R10=fp0 fp-72_w=rcu_ptr_bpf_cpumask() refs=4,5
; cpu = bpf_get_smp_processor_id(); @ main.bpf.c:610
82: (85) call bpf_get_smp_processor_id#8      ; R0=scalar(smin=smin32=0,smax=umax=smax32=umax32=11,var_off=(0x0; 0xf)) refs=4,5
83: (63) *(u32 *)(r10 -56) = r0       ; R0=scalar(id=9,smin=smin32=0,smax=umax=smax32=umax32=11,var_off=(0x0; 0xf)) R10=fp0 fp-56=????scalar(id=9,smin=smin32=0,smax=umax=smax32=umax32=11,var_off=(0x0; 0xf)) refs=4,5
; if (wake_flags & SCX_WAKE_SYNC) @ main.bpf.c:529
84: (57) r9 &= 16                     ; R9_w=scalar(smin=smin32=0,smax=umax=smax32=umax32=16,var_off=(0x0; 0x10)) refs=4,5
85: (55) if r9 != 0x0 goto pc+9       ; R9_w=0 refs=4,5
; return p->flags & PF_KTHREAD; @ main.bpf.c:220
86: (79) r1 = *(u64 *)(r10 -24)       ; R1_w=scalar(id=1) R10=fp0 fp-24=scalar(id=1) refs=4,5
87: (61) r1 = *(u32 *)(r1 +44)
R1 invalid mem access 'scalar'
processed 85 insns (limit 1000000) max_states_per_insn 2 total_states 8 peak_states 8 mark_read 4
-- END PROG LOAD LOG --
libbpf: prog 'bpfland_select_cpu': failed to load: -13
libbpf: failed to load object 'bpf_bpf'
libbpf: failed to load BPF skeleton 'bpf_bpf': -13
Error: Failed to load BPF program

Caused by:
    Permission denied (os error 13)
13

also tryed with sudo scx_lavd -v

❯ sudo scx_lavd -v
[sudo] Mot de passe de franck : 
23:53:33 [INFO] Autopilot mode is enabled by default.
23:53:33 [DEBUG] Opts {
    autopilot: true,
    autopower: false,
    performance: false,
    powersave: false,
    balanced: false,
    slice_max_us: 3000,
    slice_min_us: 300,
    no_core_compaction: false,
    prefer_smt_core: false,
    prefer_little_core: false,
    no_prefer_turbo_core: false,
    no_freq_scaling: false,
    stats: None,
    monitor: None,
    monitor_sched_samples: None,
    verbose: 1,
    version: false,
    help_stats: false,
}
libbpf: loaded kernel BTF from '/sys/kernel/btf/vmlinux'
libbpf: loading object 'bpf_bpf' from buffer
libbpf: elf: section(2) .symtab, size 18984, link 1, flags 0, type=2
libbpf: elf: section(3) .text, size 20416, link 0, flags 6, type=1
libbpf: sec '.text': found program 'do_set_power_profile' at insn offset 0 (0 bytes), code size 123 insns (984 bytes)
libbpf: sec '.text': found program 'pick_idle_cpu' at insn offset 123 (984 bytes), code size 517 insns (4136 bytes)
libbpf: sec '.text': found program 'calc_lat_cri' at insn offset 640 (5120 bytes), code size 238 insns (1904 bytes)
libbpf: sec '.text': found program 'update_sys_stat' at insn offset 878 (7024 bytes), code size 1569 insns (12552 bytes)
libbpf: sec '.text': found program 'set_on_core_type' at insn offset 2447 (19576 bytes), code size 88 insns (704 bytes)
libbpf: sec '.text': found program 'update_timer_cb' at insn offset 2535 (20280 bytes), code size 17 insns (136 bytes)
libbpf: elf: section(4) syscall, size 32, link 0, flags 6, type=1
libbpf: sec 'syscall': found program 'set_power_profile' at insn offset 0 (0 bytes), code size 4 insns (32 bytes)
libbpf: elf: section(5) fexit/__futex_wait, size 336, link 0, flags 6, type=1
libbpf: sec 'fexit/__futex_wait': found program 'fexit___futex_wait' at insn offset 0 (0 bytes), code size 42 insns (336 bytes)
libbpf: elf: section(6) fexit/futex_wait_multiple, size 344, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_wait_multiple': found program 'fexit_futex_wait_multiple' at insn offset 0 (0 bytes), code size 43 insns (344 bytes)
libbpf: elf: section(7) fexit/futex_wait_requeue_pi, size 336, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_wait_requeue_pi': found program 'fexit_futex_wait_requeue_pi' at insn offset 0 (0 bytes), code size 42 insns (336 bytes)
libbpf: elf: section(8) fexit/futex_wake, size 384, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_wake': found program 'fexit_futex_wake' at insn offset 0 (0 bytes), code size 48 insns (384 bytes)
libbpf: elf: section(9) fexit/futex_wake_op, size 384, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_wake_op': found program 'fexit_futex_wake_op' at insn offset 0 (0 bytes), code size 48 insns (384 bytes)
libbpf: elf: section(10) fexit/futex_lock_pi, size 336, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_lock_pi': found program 'fexit_futex_lock_pi' at insn offset 0 (0 bytes), code size 42 insns (336 bytes)
libbpf: elf: section(11) fexit/futex_unlock_pi, size 360, link 0, flags 6, type=1
libbpf: sec 'fexit/futex_unlock_pi': found program 'fexit_futex_unlock_pi' at insn offset 0 (0 bytes), code size 45 insns (360 bytes)
libbpf: elf: section(12) struct_ops/lavd_select_cpu, size 872, link 0, flags 6, type=1
libbpf: sec 'struct_ops/lavd_select_cpu': found program 'lavd_select_cpu' at insn offset 0 (0 bytes), code size 109 insns (872 bytes)
.
.
.
; reinit_cpumask_for_performance = true; @ power.bpf.c:255
55: (18) r1 = 0xffffa72b22d9e108      ; frame1: R1_w=map_value(map=bpf_bpf.bss,ks=4,vs=4536,off=264)
57: (73) *(u8 *)(r1 +0) = r2          ; frame1: R1_w=map_value(map=bpf_bpf.bss,ks=4,vs=4536,off=264) R2_w=1
; debugln("Set the scheduler's power profile to performance mode: %d", util); @ power.bpf.c:256
58: (18) r1 = 0xffffa72b22f87128      ; frame1: R1_w=map_value(map=bpf_bpf.rodata,ks=4,vs=3675,off=296)
60: (71) r1 = *(u8 *)(r1 +0)          ; frame1: R1=1
61: (16) if w1 == 0x0 goto pc+64      ; frame1: R1=1
62: (18) r1 = 0xffffa72b22f87d30      ; frame1: R1_w=map_value(map=bpf_bpf.rodata,ks=4,vs=3675,off=3376)
64: (b4) w2 = 66                      ; frame1: R2_w=66
65: (18) r3 = 0xffff929d0af3df20      ; frame1: R3_w=map_value(map=.rodata.str1.1,ks=4,vs=63)
67: (b4) w4 = 256                     ; frame1: R4_w=256
68: (05) goto pc+54
;  @ power.bpf.c:0
123: (bc) w5 = w6                     ; frame1: R5_w=0 R6=0
124: (85) call bpf_trace_printk#6
program of this type cannot use helper bpf_trace_printk#6
processed 54 insns (limit 1000000) max_states_per_insn 0 total_states 4 peak_states 4 mark_read 4
-- END PROG LOAD LOG --
libbpf: prog 'set_power_profile': failed to load: -22
libbpf: failed to load object 'bpf_bpf'
libbpf: failed to load BPF skeleton 'bpf_bpf': -22
Error: Failed to load BPF program

Caused by:
    Invalid argument (os error 22)
14

Please send me your kernel config.

zcat /proc/config.gz | paste-cachyos

15

❯ zcat /proc/config.gz | paste-cachyos
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 195k 100 10 100 195k 7 146k 0:00:01 0:00:01 --:–:-- 146k

16

Tryed with rc4 today … same behaviour, same error messages and the scx git is 28 commits behind, this is my hope when it gets updated some day

17

You don’t have CONFIG_FTRACE set. That’s why scx isn’t working.

18

merci, that really was the missing link , working now formidable