How to change compression-type to `lz4`?

Issues & Assistance
41

I wouldn’t assume that those benchmarks apply to anything beyond the situation tested. There is no “definitive” without testing your test case. With a lot of storage bandwidth lz4 will be faster, if that’s your driving concern.

In general zstd will use more compute to compress compressible data more. It has a very favorable performance profile. If you have a surplus of bandwidth vs CPU and don’t care about file size lz4 is probably faster. If your data is largely incompressible lz4’s abort is slightly more efficient I think, but not that you’d really notice. If you have surplus cpu and disk bandwidth is constrained zstd is likely faster.

But they’re both “fast enough” and close enough that you should probably do your own tests.

All except systemd default to lz4on install from what I seen. And you can always change bootloaders after install if you wish.

Presumably just efi booting an efi boot stub kernel directly would be faster, to the extent that it matters.

Just follow any human-written documentation on zfs send and receive. Migrating Data With ZFS Send and Receive - Stephen Foskett, Pack Rat seems fine, for example, as a starting point.

Using his pool names something like

zfs send -R gang/scooby@migrate | zfs recv -F -o compression=lz4 gang/scrappy

zfs rollback gang/scrappy@migrate

Seems like it would work.

Or if you boot from a livecd and aren’t using the filesystem you can skip the snapshot stuff like in oracle’s docs.

You’ll need to either point your boot manager at your new dataset or send it back to rewrite the old (which you’d need to boot from something other than your system to do - either liveusb or sonething like zfsbootmenu)

Don’t send raw (-w). Probably don’t use -c (compress) on the send. Use -o compression=lz4 on the receive.

I’d also read/reference the docs. It’s quicker and imo far more useful than a video about someone else’s benchmark.

42

In short I know the above is long. What I’d personally do if I wanted to test would be just boot into zfsbootmenu, go to the recovery shell, send/receive my root dataset to a clone with different compression, and boot it.

43

And if you really just want to do it in place this likeky works. I’d snapshot first though, and delete the snapshot upon confirmation I like the results.

Or just wait for the upcoming zfs rewrite command.

44

Thanks @mattsteg for all your correct posts, I feel very foolish or miss-lead about now.

I really need to stop conversing with Debian Devs, now I know why they don’t post in regular user forums. :rofl:

If anyone needs me to clean up the bad posting on my behalf let me know, I hate bad info. :frowning:

45

Let’s be clear - a lot of this is just Calamares being weird and a bit opaque in what it’s doing behind the scenes. The parameters that you suggested for pool creation are good ones! (and used by the cachy installer, just in a hidden manner). The use of zstd for systemd-boot only is…odd. It’s definitely not at all what I expected to find when digging into this!

46

Also to be very clear, I just did another install on ZFS and Limine and Indeed got LZ4 this go around.

Even with the fore mentioned edits. :wink:

I have no idea now how old my install was with limine, with zfs but it certainly was zstd much to my surprise, however I pound on this install as a testing bed for my server that I just don’t play with (K.I.S.S)

47

Thanks @mattsteg, this a real good answer.

:one: Installed COS in VMM choosing limine:

sudo zfs get all | grep compression
zpcachyos                    compression           on                     default
zpcachyos/ROOT               compression           lz4                    local
zpcachyos/ROOT/cos           compression           lz4                    local
zpcachyos/ROOT/cos/home      compression           lz4                    local
zpcachyos/ROOT/cos/root      compression           lz4                    local
zpcachyos/ROOT/cos/varcache  compression           lz4                    local
zpcachyos/ROOT/cos/varlog    compression           lz4                    local

:+1:

  • No initramfs compression in /etc/mkinitcpio.conf
  • No zram
    :-1:
    Maybe will behave different on real hardware❓

:two: send & recv appear to complicated for me, to many links like nobody know exactly what to do. i need, if at all, reliable commands with explanations, e.g.:

zfs send -R zpcachyos/ROOT@migrate | zfs recv -F -o compression=lz4 zpcachyos/GOOT
  • make a boot-entry, e.g. create /boot/loader/entries/linux-cachyos-lz4-server.conf with following content:
title Linux Cachyos Server LZ4
options zfs=zpcachyos/GOOT/cos/root rw zswap.enabled=0 nowatchdog splash
linux /vmlinuz-linux-cachyos-server
initrd /initramfs-linux-cachyos-server.img
  • Boot the new entry, if it boot correctly & all files (new & old) are lz4-compressed…
  • than overwrite @migrate? with:
zfs send -R zpcachyos/GOOT@migrate | zfs recv -F -o compression=lz4 zpcachyos/ROOT
  • boot the old entry options zfs=zpcachyos/ROOT…:red_question_mark:
  • & if boot correctly, how to delete @migrate again❓
    • Note: i need the original name ROOT in order COS can execute updates correctly❗️

:three: Bootloader:

  • i can stay with limine but…
  • i’m interested on “ZBM” too, but i have no idea about it & i’m scared without a GUI. Besides less information/videos about setup, configuration.
  • Therefore i’m interested on a bootloader that permit me to change/modify the text in UEFI/BIOS of motherboard (MB), listen, i have an NVMe-HBA with four 2-TB NVMe on it, 2 of them have an OS on it & if i want change the boot-succession in MB/UEFI, there is only “Linux+Kernel_Version”. Is there the possibility to realize my intent❓

:four: Alternative:

  • Installing latest COS on the other NVMe including all packages already installed in the actual COS, this action (install all packages) is to be done after installation is completed.
  • Possibility to edit the packages-list, e.g. removing systemd-boot. Have you an idea how to do it❓

:five: ZFS ‘snapshot’:

  • how to make bootable 'ZFS-snapshots` &
  • which bootloader is able to boot mentioned snapshots & how to configure this bootloader in order to succeed with this task❓

Thanks again & in advance for you interest, efforts & support.

BR tony

48

zfs send sends a filesystem’s data Zfs receive receives a filesystem’s data.

If you want to do something with them that no ome else is particularly interested in doing you should honestly just invest the time in learning them. No one is likely to suggest “reliable commands” that do exactly what you want. The best you’ll get is a framework that would do the job, but requires reading the docs to do.

You can just install whatever you want. Switching is easy.

The core appeal of ZBM is that it’s a mini linux system with up to date ZFS suppprt and command line tools, plus offers a nice interface to manage boot environments. It’s not really a great fit if you’re scared without a GUI imo.

Just uninstall it, if you want. But I thought you liked it and thought it was best?

There’s a pacman hook in aur that automates snapshot creation and I use ZBM to manage/boot them. The default cachy partitioning scheme isn’t ideal for this.

I can’t emphasize enough that “videos” are mostly a terrible source of information about this stuff for anything except taking 10x longer to copy someone else’s setup than necessary, with more room to screw up and less room to actually learn what you’re doing.

Videos exist for clout and monetization, because it’s nice to see something working, and because they’re low-effort.

49

send, recv, test-disk, ‘snapshots’, etc., all very important things that are bad or not explained at all where the “user” can make only mistakes. You are totally wrong on this point! All these applications should have a GUI in order to help accomplish the tasks without errors, we are not more in the 90thies.

Again, that sound like: “Read a week long what you can find & maybe you understand & can execute the task.” No package-name, no instructions, nothings. i partition my DCs (data-carriers) by myself: 4 GiB EFI, 1850 GiB COS, the rest (to 1863 GiB) ca. 9 GiB is empty. i would also not use the entire DC for True-NAS-Scale, that’s terrible idea that soon or later end with data-loss. Therefore, no explanation why COS-partitioning is not good enough, no information, nothing.

Apparently you want to prolong this discussions at infinite, i’m searching for working solution not for discussion without a conclusion. If i don’t know something, i say it too & i’m not ashamed to say it.

Conclusions:

  • COS can apply LZ4 -algorithm if Limine is used as boot-loader, rEFInd & GRUB are not liked by me. Former installation were complete with compressed initramfs & zram, last install done on VMM didn’t.
  • send & receive can be done, but is not yet clear how to do it exactly, forum-help as well AI cannot contribute enough to clarification. Even links to wikis or other forums were not committed or are not present at all or is a ton of general information & this will take to much time to study.
  • “Bootloader-s”: The COS wiki tell how to configure them, but not how to replace them, high risk to get many entry in efibootmgr or to mess it up. Also changing the title/label, e.g. “COS-N1”, “COS-N2” is not easy or up-to impossible because is not known what the system label the OS by next update.
  • ZBM is still a black-box, a “mini-OS”? appearing to be complicated & difficult like GRUB.
  • All those bootloaders could at least use the machine/computer-name typed during the installation to individuate exactly on which DC the OS is installed, but everywhere appear only by systemd-boot: Linux Boot Manager (,0x800,0x800000) or UEFI OS (,0x100,0x100000) & by limine appear Limine (,0x800,0x800000). UEFI OS is also what appear in MB-UEFI, very informative & talented description, a masterpiece!
  • Due the fact that most probably i have to install COS on the second NVMe, were very important to know how to collect all the names of installed packages & removing some like systemd-boot, but even this’snt daily business or is rarely done & hence are no information available.

Finally:

  • Due error or intentional decision, no LZ4 compression choosing zfs & standard bootloader (systemd-boot).
  • Even if COS works perfectly, is much easier to re-install it with limine as to send & receive the data-set (ROOT). To much time already consumed to investigate on the possibility to “convert” the data-set.
50

If you want a GUI for ZFS run TrueNAS or one of the other GUIs that are floating around out there.

If command line operations are too complex…I’m honestly not sure that running an out-of-tree filesystem is the greatest fit, and certainly not with a boot manager like ZBM that isn’t fully integrated into cachy and requires (easy, but they do exist) manual tweaks to work well.

That’s…kind of the point. If you’re not somewhat comfortable with the tools and commands and their documentation…you honestly should use something else. If you can’t find a package and install it then you really shouldn’t be running a configuration that isn’t officially supported!

If you use TrueNAS as-designed you have as close to zero risk of data loss as is realistically possible. it’s pretty close to bulletproof - I have one pool that have been around for 15ish years throughout multiple hardware failures, for example. Dedicating a drive for OS-only is actually best-practice if you really care about your data.

If you can’t figure out how to operate well-documented CLI interfaces, you’re really not in a position to make claims about the data integrity practices of an organization that’s developed/funded a significant portion of ZFS development.

Because it puts your kernel on a different, non-snapshotted filesystem which breaks ZBM boot environment snapshots and also puts the kernel and initramfs outside of encryption.

ehhh…that’s not really apparent tbh.

You can use whatever bootloader that you want.

They’re well-documented, general-purpose commands with dozens upon dozens of examples of use out there, that you can use nondestructively and tweak to get your exact desired outcome. I gave you the commands that you’d need and examples of replicating using send/receive. They’re just untested because I have zero interest in doing what they do.

There’s also a reason that no one has written a step-by-step guide to do exactly what you want…it’s a combination of “very simple” and “not normally useful or desirable”.

And quite frankly if you can’t understand documentation you should VERY MUCH NOT BE USING AI here - because AI literally knows nothing, globs together likely-looking token combos, and can easily hallucinate things that will destroy your system. It’s just dangerous.

This is actually all quite well-documented, but you would need to read the docs which you seem incredibly unwilling to do.

If you feel compelled to do all of these things that you can’t find anyone else doing, and that you also don’t understand, it might be worth reflecting on whether they are actually good and useful things to do. This goes triple since you also don’t seem to care about learning how and why to do things either.

And again you could always just run the rebalance script that I linked which should do what you want - although again I haven’t tested it because I have no need for it. I’d prrobably run it on individual directories rather than on / from a live ISO.

51

Is not a question to be willing, it’s a question of understandable written explanations/wiki.
This mentality to blame always the $USER instead to make something better or explain something not completely clear, i know it already, it come from some arrogant Archlinux-forum-moderators that classify the rest of humanity as dumb, unwilling to learn, lazy, etc..

This mentality to offend people between the lines is very known in some circle & show only the true character of people writing a lot, meaning nothing, making all more difficult as it’s & not helping at all.

Fact is:

  • COS use lz4 with zfs if limine is chosen/used. For sure is grub with zfs a very bad idea, ZBM is still an “obscure- or black-box” &| not yet supported in COS & rEFInd is just a little-bit better than GRUB hence not interesting to test. i’m sure there should be a not yet known reason to not use systemd-boot with lz4 because i trust blindly the COS-coder.
  • The post-installation conversion/change of compression algorithm is not a proper solution because only the new files will be changed, the older files maintain the former comp.-alg..

This’s the end of this thread concluded with “partial” change of compression algorithm.

52

On the contrary. It’s quite well-documented what it is and how it works.

Not really. I think the cachy folks generally see zstd as the right balance and just didn’t bother changing the others.

You haven’t posted a single performance evaluation of your situation. In many ways you’re not really qualified to say what is and isn’t proper.

Why would anyone waste their time handholding through a task that’s generally not useful (especially with upcoming zfs updates)? You’re not entitled to have the world serve you bespoke solutions to your imagined problems.

It’s insane to expect people to lay out step by step, tested tutorials for things that that they don’t have and need or desire to do themselves - or even advise against without a higher level of knowledge.

Sometimes the $USER is the problem though.

This (from a different thread) really struck me. If you’re constantly breaking multiple different OSes maybe the common denominator is you.