cross-posted from: https://lemmy.ml/post/36794370

Hello everyone :)

My Linux learning and Homelab setup is going smoothly and after a long period of stagnation, I’m in a new learning curve :D ! I just learned the magic of hard links and implemented them with bind mounts (yeah hard links only work on the same file system :P) to my Qbittorrent scripting to automagically move them as hard links to a bind mount accessible to Sonarr, Radarr, Jellyfin… and move, rename and do all other things without even touching the original files: PUR MAGIC !

Everything is a file? Naah, everything is a hard link ! (Or inode? xD)

While I’m overjoyed I learned and have a better understanding of files, hard links, soft links, file system, docker, web, all kind of things related to IT… I’m getting kinda overwhelmed of what’s happening on my system !!

  • I have a dozen docker compose on my server, all behind traefik resolved by my piHole DNS on a raspberry pi, some have a custom image made by myself for certificate purpose or some manual changes I added.

  • I have some .config files in my ~/ to improve my micro experience and make it more integrated over SSH with my Mac and Desktop. If also have some config files in my /root directory for my miniCA for all my service with my personal local domain name and other config files like /etc/bash_aliases and some changes in /etc/bash.bashrc

  • I have some python .venv for learning and scripting with python

  • I have a complex and long bash script backup for all my docker containers to back up my volumes, config files and media files separately

  • Installed some useful and needed packages like resolvconf, wireguard, samba…

  • I have a few samba shares, all accessible in my LAN with sometimes some exotic configuration for Mac integration

  • My docker containers have some bind mounts shared volumes

[…]

And now I also have some hard links lying around in the mix ! So I have to say it out loud: I’m OVERWHELMED !

Yes, I do keep some notes in Obsidian and also have a self-hosted Forgejo to keep my notes updated and have some kind of version control of changes in my scripts, but I do feel like I’m not sure anymore what I have… Not to say I didn’t mention all the other stuff related to my phone (baikal, ntfy…) or how to keep everything updated (WUD does the trick for containers :)) and tidy…

I guess I’m looking for something magic, something that could In ONE blink give me what’s doing what and where? And changes my life for an ever-growing IT space ? Preference something visual…?

I hope to hear from you guys on what I can do to take away this feel of being lost and not being able to fully track my systems and my LAN !

Thank you !

  • atzanteol@sh.itjust.worksEnglish
    3·
    3 days ago

    1. Simplify. You’ve made a system more complex than it needs to be. Copy files rather than working around hard link limitations. Use off-the-shelf backup solutions. Have a single file server with a known location for all your mounts. Etc.

    2. Automate. Not with bespoke bash scripts but with tools like ansible and terraform. These tools are built to help manage infrastructure and configurations.

    Everything is a file? Naah, everything is a hard link ! (Or inode? xD)

    Hard links are files…

    • N0x0n@lemmy.mlOP
      1·
      3 days ago

      1. Yeah I guess I have made it more complex. But what hard links allow me to achieve is to save disk space while still working on the file without changing the original one. Imagine I have to copy my files to sonarr, have a copy in my torrent directory and also a backup on an external disk… That’s alot of space ! This may be a simpler solution, but only If you have money to spare on disk space. Yes, it’s “cheap” when you have a bank account and money lying around, but that’s not the case for everyone :/. I think the ARR* stack works similarly and works with hard links !

      2. Oh… never heard of Terraform, will have a look, thanks for the pointer !

      Hard links are files…

      I guess so, but files are just links pointing to inodes ? :) Sorry If I’m wrong here, and please give me the proper knowledge If you are willing to share :)

      Edit: After some reading, I think Ansible seems a better fitting. Terraform is more for creating infrastructures while Ansible to manage them and configure them?

      • atzanteol@sh.itjust.worksEnglish
        1·
        3 days ago

        I’m not sure how the *arr stuff works but hard links don’t let you “edit a file while preserving the original” - they let you have mulltiple paths to the same file.

        $ echo "hello" > file1
$ ln file1 file2
$ echo "world" > file2
$ cat file1 file2
world
world

        Does *arr have some sort of copy-on-write behavior? Some modern file-systems have de-duping behavior and copy-on-write built in that you may be able to save some space with.

        But the point of topic 1 was to simplify. You can keep doing your hardlink stuff but standardize it and simplify setup/configuration. If you always do things in the same way it’s less complicated to keep track of and fix.

        You’ve understood the difference in terraform/ansible, and yeah terraform is probably not going to be as helpful. Ansible would be much more likely to help. It can seem burdensome to have to write configuration files for things at first, but it forces you to do things in a way that is standardized and repeatable.

  • non_burglar@lemmy.world
    1·
    3 days ago

    Do bind mounts not allow you to manipulate the same data in one directory from multiple containers? That’s what I do, no fussing with hard links.