Linux - for the beginners - and also for an experienced user

Below is a structured “Linux terminal + distro + build-from-source” crash course that goes from basics to fairly advanced workflows. I’ll keep it practical: commands, what they do, and a sane end-to-end process for building software, kernels, and embedded systems.


1) Terminal basics (core commands you’ll use daily)

Navigation

  • pwd — print working directory

  • ls, ls -la, ls -lh — list files (all / long / human sizes)

  • cd /path, cd .., cd -, cd ~ — change directory (back / previous / home)

File operations

  • cp file dest, cp -r dir dest — copy (recursive)

  • mv old new — move/rename

  • rm file, rm -r dir, rm -rf dir — remove (careful with -rf)

  • mkdir dir, mkdir -p a/b/c — create directories

  • touch file — create empty file / update timestamp

  • ln file linkname — hard link

  • ln -s target linkname — symlink

Viewing files

  • cat file — dump to stdout

  • less file — scroll (recommended)

  • head -n 50 file, tail -n 50 file, tail -f logfile — start/end/follow

  • nl -ba file — show with line numbers

Permissions & ownership

  • chmod 644 file, chmod +x script.sh — permissions

  • chown user:group file — ownership (often needs sudo)

  • umask — default permission mask

Finding things

  • find . -name "*.c" -type f

  • locate something (needs updatedb database)

  • grep -R "needle" . — search text recursively

  • ripgrep (rg) is a faster “grep” replacement if installed.

Pipes and redirection (the “terminal superpower”)

  • command > file overwrite output

  • command >> file append output

  • command < file use file as stdin

  • command1 | command2 pipe stdout into another command

  • command 2> err.txt redirect stderr

  • command >out.txt 2>&1 redirect both

Helpful shell built-ins

  • history — previous commands

  • !! — repeat last command

  • !123 — run history entry 123

  • alias ll='ls -la' — shortcuts (put into ~/.bashrc or ~/.zshrc)

  • Tab completion + Ctrl-R reverse search = huge productivity boost.


2) More advanced terminal skills (what makes you “power user”)

Process & system inspection

  • ps aux, ps -ef — process list

  • top / htop — interactive process viewer

  • pgrep name, pkill name, kill -9 PID (use -9 only when needed)

  • free -h — memory

  • df -h — filesystem usage

  • du -sh * — per-folder sizes

  • uname -a — kernel/system info

  • dmesg -T | less — kernel ring buffer (boot/hardware messages)

Networking basics

  • ip a (addresses), ip r (routes)

  • ss -tulpn — listening sockets (modern replacement for netstat)

  • ping host, traceroute host, mtr host

  • curl -I https://example.com, wget URL

  • dig example.com, nslookup

Archiving / backups / sync

  • tar -czf backup.tar.gz dir/ (create)

  • tar -xzf backup.tar.gz (extract)

  • rsync -avh --progress src/ dest/ (sync)

  • rsync -avh --delete src/ dest/ (mirror exactly — careful)

Text processing toolkit

  • awk, sed, cut, tr, sort, uniq, wc
    Examples:

  • cut -d: -f1 /etc/passwd | sort | uniq

  • journalctl -u ssh -S today | grep Failed

System logs / services (systemd)

  • systemctl status service

  • systemctl start|stop|restart service

  • systemctl enable service (start at boot)

  • journalctl -xe (recent errors)

  • journalctl -u nginx --since "2026-02-10"


3) Best terminal apps (popular, practical)

Terminal emulators

  • Kitty — very fast, great font rendering, GPU accelerated.

  • Alacritty — minimal, fast.

  • GNOME Terminal / Konsole — stable defaults (Ubuntu/Fedora vs KDE).

  • WezTerm — powerful, configurable (Lua), good cross-platform.

Multiplexer (one terminal becomes many)

  • tmux — sessions, panes, detach/reattach (essential on servers).

    • Start: tmux

    • New window: Ctrl-b c

    • Split: Ctrl-b % (vertical), Ctrl-b " (horizontal)

    • Detach: Ctrl-b d

    • Reattach: tmux attach

CLI “best-in-class” utilities (worth installing)

  • fzf (fuzzy finder), ripgrep (rg), fd (find replacement)

  • bat (cat with syntax highlighting), eza (ls replacement)

  • tldr (examples for commands), jq (JSON), yq (YAML)

  • btop (system monitor), ncdu (disk usage UI)

  • sshfs (mount remote filesystem over SSH)


4) Scripting basics (Bash)

Minimal script template

#!/usr/bin/env bash set -euo pipefail echo "Hello" name="${1:-world}" echo "Hi, $name"

Run:

  • chmod +x script.sh

  • ./script.sh Alice

Key ideas:

  • set -e stops on error

  • -u errors on unset vars

  • pipefail makes pipelines fail if any part fails

Variables, loops, conditionals

  • if [ -f file ]; then ...; fi

  • for f in *.txt; do echo "$f"; done

  • while read -r line; do ...; done < file

Scheduling / automation

  • cron (classic) or systemd timers (modern)

  • One-shot: at 22:00 (if installed)


5) File management workflows in terminal

Reliable “move/copy large trees”

  • Prefer rsync:

    • rsync -a --info=progress2 src/ dest/

Checking what is taking space

  • du -sh ./* | sort -h

  • ncdu for interactive browsing

Permissions sanity checks

  • ls -l to inspect

  • For scripts not running: chmod +x file

  • For “permission denied” writing: check ownership ls -l, fix with sudo chown


6) SSH vs Telnet (and commands)

SSH (secure; always use)

Common:

  • ssh user@host

  • ssh -p 2222 user@host

  • ssh -i ~/.ssh/id_ed25519 user@host

  • Port forwarding:

    • Local: ssh -L 8080:localhost:80 user@host

    • Remote: ssh -R 2222:localhost:22 user@host

  • Copy files:

    • scp file user@host:/path/

    • Better: rsync -av file user@host:/path/

Keys:

  • Generate: ssh-keygen -t ed25519

  • Copy public key: ssh-copy-id user@host

Telnet (insecure; legacy/testing)

  • telnet host 23

  • Sometimes used to test a TCP port:

    • telnet host 80
      But modern alternative is:

  • nc -vz host 80 (netcat) or curl.


7) Packing/unpacking commands (archives)

  • .tar: tar -cf a.tar dir/ (create), tar -xf a.tar (extract)

  • .tar.gz: tar -czf a.tar.gz dir/, tar -xzf a.tar.gz

  • .tar.xz: tar -cJf a.tar.xz dir/, tar -xJf a.tar.xz

  • .zip: zip -r a.zip dir/, unzip a.zip

  • .7z: 7z a a.7z dir/, 7z x a.7z

Rule of thumb: if you see tar + compression suffix, tar can usually handle it.


8) Git from terminal (core flow)

Setup

  • git config --global user.name "Name"

  • git config --global user.email "email@example.com"

  • git config --global init.defaultBranch main

Everyday commands

  • git init

  • git clone URL

  • git status

  • git add -A

  • git commit -m "message"

  • git log --oneline --graph --decorate --all

  • git diff, git diff --staged

Branching

  • git branch

  • git switch -c feature

  • git switch main

  • git merge feature

Remotes

  • git remote -v

  • git pull --rebase

  • git push -u origin feature

Saving work temporarily

  • git stash push -m "wip"

  • git stash pop


9) Adding software sources (repositories) by distro family

Debian / Ubuntu / Mint (APT)

  • Sources live in /etc/apt/sources.list and /etc/apt/sources.list.d/*.list

  • Update: sudo apt update

  • Upgrade: sudo apt upgrade

  • Add repo (modern best practice): create a .list file + install signing key into /etc/apt/keyrings/

    • (Exact commands depend on the repo; many projects document their keyring method.)

Fedora (DNF)

  • Repos in /etc/yum.repos.d/*.repo

  • Add repo: sudo dnf config-manager addrepo --from-repofile=URL (if plugin installed)

  • Update: sudo dnf upgrade

Arch (pacman)

  • Repos configured in /etc/pacman.conf

  • Update: sudo pacman -Syu

  • AUR is community packages (typically via helpers like yay).

Gentoo (Portage)

  • Uses “ebuilds” and overlays, not “repos” in the same way.

  • Sync: emerge --sync

  • Add overlays via eselect repository (common approach).


10) Makefile + compiling an app/package (the “whole process”)

The classic build pipeline

  1. Install build tools & dependencies

  • Debian/Ubuntu: sudo apt install build-essential pkg-config

  • Fedora: sudo dnf groupinstall "Development Tools"

  • Arch: sudo pacman -S base-devel

  • Plus project deps (e.g., libssl-dev, zlib1g-dev, etc.)

  1. Get the source

  • git clone ... or download .tar.gz and extract.

  1. Read instructions

  • README, INSTALL, CONTRIBUTING

  • Look for: ./configure, cmake, meson, or just make.

  1. Configure (if needed)
    Common systems:

  • Autotools: ./configure --prefix=/usr

  • CMake: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release

  • Meson: meson setup build

  1. Build

  • make -j"$(nproc)"
    or:

  • cmake --build build -j"$(nproc)"

  1. Test (if available)

  • make test / ctest / meson test

  1. Install

  • sudo make install
    or:

  • sudo cmake --install build

What a Makefile is doing (conceptually)

  • Defines targets (like all, clean, install)

  • Defines dependencies between files (rebuild only what changed)

  • Calls compiler commands (gcc, clang) with flags.

Useful make targets:

  • make (default target, often all)

  • make clean

  • make install

  • make uninstall (sometimes)

  • make help (sometimes)


11) Distros: features + “who they’re for”

Debian

  • Conservative/stable, huge repositories, foundation for many others.

  • Great for servers and stable desktops.

Ubuntu / Kubuntu

  • Ubuntu: user-friendly, strong hardware support, big community, regular releases + LTS.

  • Kubuntu: Ubuntu base with KDE Plasma desktop.

Linux Mint

  • Based on Ubuntu (and also has a Debian Edition).

  • Very beginner-friendly, “just works” desktop focus (Cinnamon is popular).

Fedora

  • Upstream-ish, ships newer tech quickly, strong defaults, close to Red Hat ecosystem.

  • Great dev workstation, good Wayland/SELinux integration.

Arch Linux

  • Rolling release, minimal base, you build your system.

  • Pacman + AUR = massive software availability.

  • Best if you like learning and customizing.

Gentoo

  • Source-based (compile packages), extreme customization (USE flags).

  • Great learning distro; can be time-consuming.

Other notable distros (quick hits)

  • openSUSE (Leap/Tumbleweed): strong tooling (YaST), Leap stable / Tumbleweed rolling.

  • Manjaro/EndeavourOS: easier Arch-style experience.

  • Alpine: tiny, security-focused, musl libc, common in containers.

  • NixOS: declarative config, reproducible systems, unique package manager model.

  • Kali: security testing (not a daily driver for most).

  • Raspberry Pi OS: tuned for Pi ecosystem.


12) Linux facts: GNU/Linux vs Linux, and “Unix-like”

Linux vs GNU/Linux

  • Linux (Linus Torvalds): the kernel (core of OS: process scheduling, memory, drivers, filesystems).

  • GNU (Richard Stallman / FSF): major userspace tools and libraries (compiler toolchain, core utilities, shells, etc.).

  • Many “Linux” systems are actually Linux kernel + GNU tools → “GNU/Linux”.

  • Some systems use Linux kernel without GNU (e.g., Android uses Linux kernel but different userspace).

Unix-like but not “official Unix”

  • “UNIX®” is a trademark; “official Unix” requires certification (Single UNIX Specification).

  • Linux behaves like Unix in design and interfaces (POSIX-ish), but is not typically branded/certified as UNIX.


13) Kernel: why C is the main language (and what else exists)

  • The kernel is mostly C because it balances:

    • low-level control (memory, hardware),

    • performance,

    • portability,

    • and maintainability at large scale.

  • There is also assembly in arch-specific parts.

  • Increasingly, Rust is being added for some kernel components (memory safety), but C remains dominant.


14) Compile a Linux kernel yourself (from kernel.org) — end-to-end

High-level flow:

  1. Get kernel source

  2. Configure

  3. Build

  4. Install modules

  5. Install kernel + initramfs

  6. Update bootloader

  7. Reboot and select new kernel

Typical commands (example pattern; exact steps vary by distro/bootloader):

1) Prereqs

You’ll need compiler + tools:

  • gcc / clang, make, binutils

  • bc, bison, flex

  • openssl headers, elfutils, etc. (depends)

2) Download + verify + extract

  • Download from kernel.org (tarball)

  • Extract:

    • tar -xf linux-x.y.z.tar.xz

    • cd linux-x.y.z

3) Configure

Start from a known config:

  • Use your current kernel config (often available):

    • zcat /proc/config.gz > .config (if enabled)

    • or copy from /boot/config-*
      Then:

  • make oldconfig (update config with prompts)
    Or use a UI:

  • make menuconfig (ncurses UI)

  • make nconfig / make xconfig (if available)

4) Build

  • make -j"$(nproc)"

5) Modules + install

  • sudo make modules_install

  • sudo make install
    This commonly places:

  • kernel image in /boot/

  • System.map, config, etc.

6) Initramfs + bootloader

  • Many distros generate initramfs via:

    • update-initramfs (Debian/Ubuntu family)

    • dracut (Fedora)

  • Bootloader update:

    • update-grub (many Debian/Ubuntu setups)

    • or grub-mkconfig -o ... depending on layout

7) Reboot

  • uname -r after boot to confirm.

Safety tip: keep at least one known-good kernel entry in your bootloader so you can recover.


15) Embedded Linux kernel “all by yourself” (cross-compile approach)

Embedded work usually adds:

  • a target architecture (ARM, RISC-V, etc.)

  • a cross-compiler toolchain

  • a root filesystem (BusyBox, systemd, Buildroot, Yocto, etc.)

  • a device tree (DTB) for hardware description (common on ARM)

Typical embedded kernel build steps

  1. Get a cross toolchain

  • Example naming: aarch64-linux-gnu-gcc or arm-linux-gnueabihf-gcc

  1. Configure for target

  • Set environment:

    • export ARCH=arm64 (example)

    • export CROSS_COMPILE=aarch64-linux-gnu-

  • Pick a default config:

    • make defconfig

    • or a board/vendor defconfig: make <board>_defconfig

  1. Customize

  • make menuconfig (still works for cross builds)

  1. Build kernel + dtbs

  • make -j"$(nproc)" Image modules dtbs

  1. Install modules into a staging rootfs

  • make modules_install INSTALL_MOD_PATH=/path/to/rootfs

  1. Boot artifacts

  • Kernel Image (Image or zImage)

  • DTB (.dtb)

  • Rootfs (ext4 image, initramfs, etc.)

  • Bootloader (often U-Boot) config to load those

Practical advice

If your goal is “learn + succeed quickly”:

  • Use Buildroot first (simpler), then Yocto for industrial workflows.
    If your goal is “maximum control/education”:

  • Manual toolchain + BusyBox rootfs + manual kernel build is great learning, but slower.


make menuconfig is not just “a config menu” — it’s basically the control panel for the entire Linux kernel.

What is make menuconfig?

Inside the kernel source tree:

make menuconfig

It launches an ncurses-based UI that edits:

.config

That file determines:

  • What gets compiled

  • Built-in vs module

  • CPU architecture

  • Filesystems

  • Drivers

  • Security

  • Debug features


🎛 Option Types Inside menuconfig

Each option can be:

SymbolMeaning
[*]Built into kernel
[M]Build as module
[ ]Disabled

Modules = loadable via modprobe.

Built-in = part of kernel image.


Main Sections You’ll See

This is roughly what appears at top level.


1️⃣ General Setup

Controls:

  • Kernel version info

  • Local version string

  • Initramfs support

  • Control groups (cgroups)

  • Namespaces

  • Sysctl support

  • Audit system

  • System V IPC

  • Kernel compression type

Very important:

  • Enable cgroups for containers

  • Enable namespaces for Docker

  • Enable initramfs if needed


2️⃣ Processor Type and Features

Critical section.

Options include:

  • CPU family (Intel, AMD, ARM, RISC-V)

  • SMP (multi-core support)

  • Preemption model:

    • No preemption

    • Voluntary

    • Fully preemptible

  • NUMA support

  • Hyperthreading

  • Tickless kernel

  • Hugepages

  • Microcode support

Embedded systems often disable lots here.


3️⃣ Power Management

  • ACPI

  • Suspend / hibernate

  • CPU frequency scaling

  • Laptop battery features

Server kernels sometimes disable heavy power features.


4️⃣ Security Options

  • SELinux

  • AppArmor

  • TOMOYO

  • Landlock

  • Yama

  • Stack protector

  • Kernel address randomization (KASLR)

  • Hardened usercopy

  • Module signature verification

Production systems enable most.

Embedded systems often simplify.


5️⃣ Loadable Module Support

  • Enable module loading

  • Force module signing

  • Module versioning

Disable modules for ultra-minimal embedded builds.


6️⃣ Networking Support

Massive section.

Includes:

  • TCP/IP stack

  • IPv4

  • IPv6

  • Netfilter (iptables / nftables)

  • QoS

  • Bridging

  • VLAN

  • Tunnels

  • WireGuard

  • Bluetooth

  • WiFi stack (mac80211)

  • CAN bus

Containers require:

  • Namespaces

  • Netfilter

  • Virtual Ethernet devices


7️⃣ Device Drivers

The largest section.

Categories:

  • Generic Driver Options

  • Firmware loading

  • PCI

  • USB

  • I2C

  • SPI

  • SATA

  • NVMe

  • SCSI

  • RAID

  • Network card drivers

  • GPU drivers

  • Sound drivers

  • Input devices

  • HID

  • Serial ports

  • MMC/SD

This section determines whether your hardware works.


8️⃣ File Systems

Includes:

  • ext4

  • XFS

  • Btrfs

  • F2FS

  • NTFS

  • FAT

  • ISO9660

  • NFS

  • CIFS

  • FUSE

  • OverlayFS

Root filesystem must be built-in.

Network filesystems often modules.


9️⃣ Kernel Hacking

For developers.

  • Debug symbols

  • KGDB

  • Tracing

  • Ftrace

  • Kprobes

  • Lock debugging

  • Memory debugging

  • Address sanitizer

Disable for production.


10️⃣ Virtualization

  • KVM

  • Xen

  • Hyper-V

  • VirtIO drivers

  • Paravirtualization

Required for VM hosts.


11️⃣ Crypto API

  • AES

  • SHA

  • RSA

  • ECC

  • Hardware acceleration

Needed for:

  • Disk encryption

  • VPN

  • TLS


 Embedded Kernel Strategy

Typical embedded config:

  • Disable modules

  • Disable sound

  • Disable GPU

  • Disable Bluetooth

  • Disable debug

  • Keep only required filesystem

  • Enable minimal networking

  • Strip everything else

Result: small kernel.


Where All Options Come From

Each driver and feature defines:

config FEATURE_NAME bool "Feature description"

Inside:

Kconfig files

All menuconfig is doing is presenting combined Kconfig tree.


Useful Commands

Start from current running kernel config:

zcat /proc/config.gz > .config

Or from distro config:

cp /boot/config-$(uname -r) .config make oldconfig

List differences:

make savedefconfig

Search for option inside menuconfig:

Press:

/

Then type keyword.


Important Kernel Concepts

Built-in vs Module

Root disk drivers should be built-in.
Optional hardware can be modules.


Dependency System

If option is greyed out:

  • It depends on something else.

  • Press / to see dependencies.


Saving

After exit:

.config

gets updated.

Then build:

make -j$(nproc)

Brutal Truth

There is no human who understands every option.

Kernel config depends on:

  • Architecture

  • Hardware

  • Use case

  • Security model

  • Performance goals

Linux follows something called the Filesystem Hierarchy Standard (FHS).
Everything starts from:

/

That’s the root directory — the top of the tree.

Let’s walk through it properly.

The Linux Filesystem Tree





Everything hangs under /.

Not like Windows (C:\, D:\).
Linux has one unified tree.

Even other disks get mounted somewhere inside /.


Core Root Directories Explained


/ — Root

This is not “root user”.
This is the top of the filesystem.

All other directories live under it.


/home

User home directories:

/home/alice /home/bob

Contains:

  • Documents

  • Downloads

  • Desktop

  • Config files (~/.config)

  • SSH keys

Safe place for personal data.


/root

Home directory for the root user.

Not the same as /.

Only root should access it.


/etc

System configuration files.

Examples:

/etc/passwd /etc/shadow /etc/hosts /etc/fstab /etc/ssh/sshd_config

Think:

"Editable Text Configuration"

No binaries here — mostly config files.


/proc

Virtual filesystem.

Not real files on disk.

Shows kernel & process info.

Examples:

/proc/cpuinfo /proc/meminfo /proc/<pid>/

Kernel generates this live.


/sys

Another virtual filesystem.

Exposes:

  • Hardware info

  • Device tree

  • Kernel objects

Used heavily in embedded systems.


/dev

Device files.

Everything is a file in Linux — even hardware.

Examples:

/dev/sda /dev/sda1 /dev/null /dev/tty /dev/random

Kernel creates these dynamically.


/usr

This confuses many people.

/usr contains:

  • User binaries

  • Libraries

  • Shared read-only data

Examples:

/usr/bin /usr/lib /usr/share /usr/include

Most installed programs live here.


/bin

Essential command binaries needed for boot & rescue.

Examples:

/bin/ls /bin/cp /bin/sh

On modern systems, /bin often links to /usr/bin.


/sbin

System binaries for root/admin use.

Examples:

/sbin/fsck /sbin/reboot /sbin/ip

Often merged with /usr/sbin nowadays.


/lib

Essential shared libraries required by /bin and /sbin.

Examples:

/lib/x86_64-linux-gnu/

Modern systems use /usr/lib mostly.


/var

Variable data.

Things that change frequently:

/var/log /var/cache /var/spool /var/lib/mysql

Logs live here.

Databases live here.

Web server logs live here.


/tmp

Temporary files.

Often cleared at reboot.

World-writable, but sticky bit enabled.


/opt

Optional third-party software.

Example:

/opt/google/chrome

Enterprise apps often installed here.


/boot

Boot-related files:

  • Kernel image

  • initramfs

  • GRUB files

If this breaks → system won’t boot.


🔌 /mnt

Temporary mount point for manual mounts.

Example:

mount /dev/sdb1 /mnt

/media

Auto-mounted devices.

USB drives appear here:

/media/alice/USB

Important Concept: Mount Points

In Linux:

If you plug in a disk:

It gets mounted somewhere under /.

Example:

/dev/sdb1 → /mnt/backup

So even different drives appear inside the same tree.


Virtual vs Real Filesystems

DirectoryType
/homeReal disk
/etcReal disk
/varReal disk
/procVirtual (kernel)
/sysVirtual
/devVirtual device nodes

Embedded Linux Differences

In embedded systems:

Often minimal tree:

/ ├── bin ├── dev ├── etc ├── proc ├── sys ├── lib ├── sbin

No /home sometimes.

Small rootfs.


Why Linux Is Designed This Way

Unix philosophy:

  • Separate static data from variable

  • Separate configs from binaries

  • Everything is a file

  • Predictable layout

Once you learn it, you can SSH into any Linux machine and instantly understand it.


Quick Memory Trick

DirectoryPurpose
/etcConfig
/homeUsers
/varLogs & changing data
/usrPrograms
/binEssential commands
/devDevices
/procKernel info
/bootBoot files

What You Should Never Do

Never randomly delete:

/lib /bin /sbin /usr

System = gone.

Full Linux boot chain

1️⃣ Firmware (BIOS / UEFI)

When you power on:

CPU executes firmware stored on motherboard.

Two types:

  • BIOS (legacy)

  • UEFI (modern standard)

Firmware does:

  • Hardware initialization

  • Memory detection

  • CPU setup

  • Disk detection

  • Finds bootable device

  • Loads bootloader

With UEFI:

  • Looks for .efi file on EFI System Partition (ESP)

  • Typically:

    /EFI/BOOT/bootx64.efi

2️⃣ Bootloader (GRUB, systemd-boot, etc.)




Bootloader’s job:

  • Load Linux kernel into memory

  • Load initramfs

  • Pass kernel parameters

Most common:

  • GRUB

  • systemd-boot

  • U-Boot (embedded)

  • LILO (old school)

GRUB reads config:

/boot/grub/grub.cfg

Then loads:

vmlinuz-linux initramfs.img

3️⃣ Linux Kernel

Kernel is now executing.

It does:

  • Setup memory management

  • Initialize CPU scheduler

  • Initialize drivers (built-in)

  • Mount initial root filesystem (temporary)

  • Start PID 1

But here’s the catch:

The kernel cannot directly mount your real root filesystem yet if drivers are modular.

That’s where initramfs comes in.


4️⃣ initramfs (Initial RAM Filesystem)



Initramfs is:

  • A compressed mini root filesystem

  • Loaded into RAM

  • Contains minimal tools + drivers

Why needed?

Because:

  • Root disk might be LVM

  • Or encrypted (LUKS)

  • Or RAID

  • Or on NVMe requiring module

Kernel unpacks initramfs into memory.

Then:

  1. Loads required drivers

  2. Unlocks encrypted disk (if needed)

  3. Mounts real root filesystem

  4. Switches root (switch_root or pivot_root)

  5. Executes real /sbin/init

Then initramfs disappears.


5️⃣ Userspace (Init System Starts)

Now real root filesystem is mounted.

Kernel launches:

/sbin/init

In modern systems:

systemd

PID 1 is born.

You can check:

ps -p 1

systemd then:

  • Mounts remaining filesystems

  • Starts services

  • Sets hostname

  • Brings up networking

  • Starts login manager

  • Launches desktop

Now you're in userspace.


Full Chain Overview

Here’s the full flow:

Power OnFirmware (BIOS/UEFI)Bootloader (GRUB)Linux Kernelinitramfsswitch_rootsystemd (PID 1)ServicesLogin / GUI

Special Cases

 Encrypted Disk (LUKS)

During initramfs stage:

  • Prompts for passphrase

  • Unlocks disk

  • Continues boot


Virtual Machines

Firmware may be:

  • SeaBIOS

  • OVMF (UEFI for VMs)

But process is same.


Embedded Systems

Often simplified:

Boot ROM ↓ U-Boot ↓ Kernel ↓ initramfs ↓ BusyBox init

Smaller, minimal.


Why This Design?

Because:

  • Kernel must stay generic

  • Early hardware detection needed

  • Root disk setup may be complex

  • Modular drivers are flexible

Initramfs acts like:

“Boot-time toolbox”


Where Things Usually Break

Boot problems often occur in:

  • GRUB config

  • Missing initramfs

  • Wrong root= parameter

  • Missing disk driver

  • Corrupt filesystem


Useful Debug Tricks

At GRUB:
Press e to edit kernel parameters.

Add:

systemd.unit=rescue.target

Or:

init=/bin/bash

For debugging.


Deep Internal Concepts

  • pivot_root() system call

  • switch_root utility

  • Kernel cmdline parameters

  • Early userspace vs late userspace

  • initramfs vs initrd (older method)


Modern Twist: Secure Boot

With UEFI Secure Boot:

Chain becomes:

Firmware verifies bootloader Bootloader verifies kernel Kernel verifies modules

Cryptographic chain of trust.


Big Picture

Each layer hands control to next.

Each layer becomes progressively:

  • More abstract

  • More feature-rich

  • Less hardware-focused

Until you're fully in userspace.

What Is a Package Manager?

It handles:

  • Installing software

  • Resolving dependencies

  • Updating system

  • Removing packages

  • Verifying signatures

  • Managing repositories

Instead of compiling everything manually.


🟢 Debian / Ubuntu / Linux Mint

APT + dpkg (.deb packages)

Package format:

.deb

Core tools:

apt apt-get dpkg

Examples:

sudo apt update sudo apt upgrade sudo apt install nginx sudo apt remove nginx

Strengths:

  • Massive repositories

  • Stable LTS versions

  • Excellent dependency handling

Weakness:

  • Stable releases often ship older versions

Used by:

  • Debian

  • Ubuntu

  • Linux Mint

  • Kali


Fedora / RHEL / Rocky / Alma

DNF / YUM (.rpm packages)

Package format:

.rpm

Core tool:

dnf

Examples:

sudo dnf install nginx sudo dnf update sudo dnf remove nginx

Strengths:

  • Modern tooling

  • Strong enterprise support

  • Good dependency resolution

  • SELinux integration

Used by:

  • Fedora

  • RHEL

  • Rocky Linux

  • AlmaLinux


Arch Linux

pacman (.pkg.tar.zst)

Package format:

.pkg.tar.zst

Command:

sudo pacman -Syu sudo pacman -S firefox

Strengths:

  • Very fast

  • Simple design

  • Rolling release

  • AUR (huge community repository)

Weakness:

  • Not beginner-friendly

  • Updates require attention

Used by:

  • Arch

  • Manjaro

  • EndeavourOS


Gentoo

Portage (Source-based)

Core tool:

emerge

Example:

sudo emerge --ask nginx

What makes it different:

  • Compiles from source

  • USE flags customize features

  • Highly optimized builds

Strength:

  • Extreme control

  • Fine-grained customization

Weakness:

  • Slow installs

  • Requires knowledge


openSUSE

Zypper (.rpm packages)

Command:

sudo zypper install nginx

Strength:

  • Very powerful dependency solver

  • Enterprise stability (SUSE Linux Enterprise)

 NixOS

Nix Package Manager (Declarative)

Totally different philosophy.

You define system in:

/etc/nixos/configuration.nix

Example:

nixos-rebuild switch

Strength:

  • Reproducible systems

  • Atomic upgrades

  • Rollbacks built-in

Mind-bending but powerful.


Alpine Linux

apk

Used heavily in:

  • Containers

  • Embedded systems

Command:

apk add nginx

Small, musl-based, lightweight.


Universal Package Managers (Cross-Distro)

These work on many distros:

Snap (Canonical)

snap install vlc

Flatpak

flatpak install flathub org.videolan.VLC

AppImage

Just download and run.

These are sandboxed and self-contained.


Quick Comparison

DistroPackage ManagerStyle
Debian/UbuntuaptStable, huge repos
Fedora/RHELdnfEnterprise, modern
ArchpacmanRolling, minimal
GentooemergeSource-based
openSUSEzypperEnterprise + dev
NixOSnixDeclarative
AlpineapkLightweight

Philosophical Differences

Stable distros:

  • Older packages

  • Fewer breakages

  • Backports security fixes

Rolling distros:

  • Latest versions

  • Faster innovation

  • More risk

Source-based:

  • Maximum customization

  • Time cost

Declarative (Nix):

  • Entire OS as code

  • Reproducibility


Real Talk

Your package manager affects:

  • System philosophy

  • Update model

  • Stability

  • Server reliability

  • Developer workflow

It’s one of the biggest dividing lines in Linux culture.



Users, groups, and sudo — who can do what, and how.

Users in Linux

Linux is multi-user by design.

Every process runs as a user.

Check current user:

whoami

See logged in users:

who

List users:

cat /etc/passwd

Important system files:

/etc/passwd → user accounts /etc/shadow → encrypted passwords /etc/groupgroups

What is Root?

root = superuser.

UID 0.

Root can:

  • Read/write any file

  • Kill any process

  • Change system configs

  • Load kernel modules

Switch to root (if allowed):

su -

But modern systems prefer sudo.


What is sudo?

sudo = “superuser do”

It allows a regular user to run commands as root.

Example:

sudo apt update

Instead of logging in as root, you:

  • Stay normal user

  • Temporarily elevate privileges

Safer.


How sudo Works

  1. You run sudo command

  2. System checks /etc/sudoers

  3. Verifies you’re allowed

  4. Prompts for your password

  5. Executes command as root


Groups Explained

Users belong to groups.

Check your groups:

groups

Example output:

alice sudo docker audio video

Groups control access to:

  • Devices

  • Services

  • Directories

  • Permissions


Creating Users

Create user:

sudo useradd -m alice

Set password:

sudo passwd alice

More friendly version (Debian/Ubuntu):

sudo adduser alice

Delete user:

sudo userdel -r alice

Creating Groups

Create group:

sudo groupadd developers

Add user to group:

sudo usermod -aG developers alice

Important:
Use -aG
If you omit -a, you overwrite existing groups.


Giving sudo Access

On Ubuntu/Debian:

Users in group:

sudo

have sudo access.

Add user to sudo group:

sudo usermod -aG sudo alice

On RHEL/Fedora:
Group is:

wheel

Add:

sudo usermod -aG wheel alice

/etc/sudoers File

Never edit directly with nano.

Always use:

sudo visudo

Example rule:

alice ALL=(ALL) ALL

Meaning:
User alice can run any command as any user on any host.

More restricted example:

alice ALL=(ALL) /usr/bin/systemctl

Meaning:
Alice can only run systemctl with sudo.


File Permissions + Users

Permissions depend on:

owner group others

Check:

ls -l file

Example:

-rw-r----- 1 alice developers file.txt

Meaning:

  • Owner: alice

  • Group: developers

  • Others: none


UID and GID

Each user has:

  • UID (User ID)

  • GID (Group ID)

Check:

id alice

Root always:

UID = 0

System users often:

UID < 1000

Regular users usually:

UID1000

Special Service Users

Linux creates system users like:

mysql www-data nobody daemon

These run services with limited privileges.

Example:
Apache runs as:

www-data

Limits damage if hacked.


Common sudo Commands

Run shell as root:

sudo -i

Run as another user:

sudo -u alice command

Edit root file safely:

sudo nano /etc/hosts

Why sudo Is Better Than Root Login

Root login:

  • Risky

  • No accountability

  • Easier to break system

sudo:

  • Logs usage

  • Per-command privilege

  • Can restrict commands

  • More secure


Default Important Groups

GroupPurpose
sudo / wheelAdmin privileges
dockerControl Docker
audioAccess sound devices
videoAccess GPU devices
plugdevUSB access
netdevNetwork devices

Dangerous Mistakes

Never:

chmod -R 777 /

Never:
Remove yourself from sudo group.

If you do:
You lose admin access.


Real World Use Case

Multiple users on server:

  • alice → developer

  • bob → database admin

  • www-data → web service

  • root → system control

Groups separate responsibilities.


Extra Security Concepts

Advanced setups use:

  • sudo command restrictions

  • sudo timeouts

  • PAM rules

  • SELinux policies

  • Polkit


Quick Demo Scenario

Create dev team:

sudo groupadd devteam sudo useradd -m alice sudo useradd -m bob sudo usermod -aG devteam alice sudo usermod -aG devteam bob

Create shared directory:

sudo mkdir /project sudo chown :devteam /project sudo chmod 770 /project

Now only devteam can access.


Big Picture

Users → Identity
Groups → Role
sudo → Controlled privilege escalation

Linux security model is layered:

DAC (permissions) + Groups + sudo + MAC (SELinux/AppArmor)


Beginner-Friendly Editors

Nano




Command:

nano filename.txt

Why people love it:

  • Extremely simple

  • Key shortcuts shown at bottom (^X = Ctrl+X)

  • No weird modes

  • Perfect for quick edits in /etc or config files

Best for:

  • Beginners

  • Quick server edits

  • Emergency SSH sessions

Downside:

  • Limited advanced features

  • Not great for large projects


Vi / Vim




Vim (Vi Improved)

Command:

vim filename.txt

Core idea: Modal editor

  • Normal mode (navigation)

  • Insert mode (i)

  • Command mode (:)

Why people worship it:

  • Insanely powerful

  • Blazing fast

  • Works everywhere

  • Huge plugin ecosystem

  • Keyboard-driven productivity monster

Basic survival guide:

i → enter insert mode Esc → back to normal mode :w → save :q → quit :wq → save + quit :q! → quit without saving

Best for:

  • Developers

  • Sysadmins

  • Large codebases

  • People who love keyboard flow

Learning curve: Steep.
Payoff: Huge.


Neovim (Modern Vim)




Command:

nvim filename.txt

Basically Vim rebuilt with:

  • Better plugin system

  • Built-in LSP support

  • Lua configuration

  • Async support

If you want a “modern dev setup in terminal”, this is king.


Emacs




Emacs

Command:

emacs filename.txt

Emacs isn’t just an editor. It’s basically:

  • Editor

  • IDE

  • Email client

  • Task manager

  • Lisp machine

  • Operating system meme 😄

Why people love it:

  • Extremely customizable

  • Org-mode (legendary productivity tool)

  • Lisp extensibility

Why people don’t:

  • Heavy

  • Keybindings are… different

  • Can feel overwhelming


Micro (Modern Nano Alternative)



Command:

micro filename.txt

Think:

Nano + modern UX + mouse support + syntax highlighting

Very beginner-friendly but feels modern.


Advanced / Power Editors



Helix

Modern modal editor inspired by Vim but:

  • Built-in LSP

  • Multiple cursors

  • Better defaults

  • Written in Rust

Command:

hx filename.txt

Very promising “next-gen Vim”.

What Are Linux Init Scripts?

Init scripts are the instructions that start your system’s services during boot.

They tell Linux things like:

  • Start networking

  • Mount filesystems

  • Start SSH

  • Launch display manager

  • Run daemons

They are part of the init system.


Step-by-Step: What Happens at Boot?

1️⃣ Firmware

  • BIOS or UEFI starts first.

  • It loads the bootloader (e.g. GRUB).

2️⃣ Bootloader

  • Loads the Linux kernel into memory.

  • Loads initramfs (temporary root filesystem).

3️⃣ Kernel

  • Initializes hardware.

  • Mounts root filesystem.

  • Then it starts PID 1.

👉 PID 1 = the init system.

You can check:

ps -p 1 -o comm=

What Is “init”?

Originally, Linux used:

SysVinit

Classic Unix-style system.

  • PID 1 = /sbin/init

  • Uses runlevels

  • Scripts live in:

    /etc/init.d/
  • Runlevel directories:

    /etc/rc0.d/ /etc/rc1.d/ ... /etc/rc6.d/

Runlevels

LevelMeaning
0Halt
1Single user
3Multi-user (no GUI)
5Multi-user + GUI
6Reboot

Scripts look like:

#!/bin/sh case "$1" in start) echo "Starting service" ;; stop) echo "Stopping service" ;; restart) ;; esac

They’re just shell scripts.


systemd (Modern Standard)




Most modern distros use systemd:

  • Ubuntu

  • Debian

  • Fedora

  • Arch

  • Mint

  • openSUSE

  • etc.

Instead of runlevels, it uses targets.

Example:

  • multi-user.target

  • graphical.target

Services are defined in unit files, not shell scripts.

Location:

/usr/lib/systemd/system/

and

/etc/systemd/system/

Example unit file:

[Unit] Description=My Service [Service] ExecStart=/usr/bin/myprogram Restart=always [Install] WantedBy=multi-user.target

Commands:

systemctl start ssh systemctl stop ssh systemctl enable ssh systemctl status ssh

Other Init Systems (Less Common Today)

  • OpenRC (used by Gentoo, Alpine)

  • runit

  • s6

  • Upstart (old Ubuntu, mostly replaced)

Example: Gentoo uses OpenRC:

rc-service nginx start rc-update add nginx default

What Is initramfs?

Before the real root filesystem loads, Linux runs a temporary mini-root filesystem.

It:

  • Loads drivers

  • Detects disks

  • Mounts root

  • Switches to real root

This early boot logic also runs “init”.


Why PID 1 Is Special

PID 1:

  • Never dies

  • Reaps zombie processes

  • Controls system shutdown

  • Manages services

If PID 1 crashes → system panic.


Concept Summary

ComponentRole
KernelHardware + core OS
PID 1First userspace process
init systemStarts services
init scripts / unit filesInstructions for services

Why systemd caused drama

Some people prefer classic Unix philosophy:

  • Small tools

  • Simple shell scripts

  • Modular design

Others like systemd because:

  • Faster parallel boot

  • Dependency tracking

  • Logging via journald

  • Better service supervision

This debate is almost philosophical.

Ahhh yes — the classic “my machine has 2–4 GB RAM and it’s crying” situation 
Swap to the rescue.

Let’s do this clean, safe, and modern.


What Is Swap?

Swap = disk space used as virtual memory when RAM is full.

⚠️ Important:

  • It is much slower than RAM.

  • It prevents crashes.

  • It does NOT make your system magically faster.

  • On SSD it’s okay. On HDD it’s slower but still useful.


First: Check if You Already Have Swap

free -h

or

swapon --show

If you see no swap → we create one.


Method 1: Create a Swap File (Recommended)

This works on:

  • Ubuntu

  • Debian

  • Fedora

  • Arch

  • Mint

  • Basically all modern distros


✅ Step 1 — Create the file

Example: 4GB swap file

sudo fallocate -l 4G /swapfile

If fallocate fails, use:

sudo dd if=/dev/zero of=/swapfile bs=1M count=4096 status=progress

✅ Step 2 — Secure permissions

sudo chmod 600 /swapfile

Very important — swap must not be world-readable.


✅ Step 3 — Make it swap

sudo mkswap /swapfile

✅ Step 4 — Enable it

sudo swapon /swapfile

Check:

free -h

You should now see swap available.


✅ Step 5 — Make It Permanent

Add this line to /etc/fstab:

/swapfile none swap sw 0 0

Edit safely:

sudo nano /etc/fstab

Reboot to confirm it survives.


How Much Swap Should You Use?

RAMRecommended Swap
2GB2–4GB
4GB2–8GB
8GB+2–4GB usually enough
Hibernate needed?Swap ≥ RAM

If you plan to use hibernation, swap must be at least equal to RAM.


Adjust Swappiness (Important for Low-End Machines)

Swappiness = how aggressively Linux uses swap.

Default is usually 60.

Check:

cat /proc/sys/vm/swappiness

For low-RAM desktop, try:

sudo sysctl vm.swappiness=10

To make permanent:
Add to /etc/sysctl.conf:

vm.swappiness=10

Lower value → prefers RAM
Higher value → uses swap earlier

For slow HDD machines, 10–20 is good.


Bonus: Use zram (Better for Low-End Systems)

Instead of disk swap, use compressed RAM swap.

Especially good for:

  • 2–4GB RAM machines

  • Laptops

  • Old hardware

Install (example for Ubuntu/Debian):

sudo apt install zram-tools

Enable:

sudo systemctl enable zramswap

What it does:

  • Creates compressed swap in RAM

  • Faster than disk

  • Saves memory

Modern Fedora/Arch often enable zram by default.


SSD Wear?

Modern SSDs handle swap fine.
Don’t worry unless you’re writing massive amounts constantly.


Advanced: Check Swap Usage Live

watch -n1 free -h

Or:

htop

Swap File vs Swap Partition

Swap FileSwap Partition
Easy to resizeFixed size
FlexibleSlightly more “traditional”
Recommended todayUsed in older setups

Swap file is totally fine in 2026.


Real Advice for Low-End Machines

If machine feels slow:

  • Add swap

  • Lower swappiness

  • Use lightweight desktop (XFCE, LXQt)

  • Disable heavy startup services

  • Use zram

  • Consider disabling animations

 

Create a Network Block Device (NBD)
Host it on one machine
Access it over your local network (router LAN)

Let’s build this cleanly and safely.


What is NBD?

NBD (Network Block Device) lets you export a block device (or disk image file) from one machine and use it like a local disk on another machine.

To the client, it looks like:

/dev/nbd0

You can:

  • format it

  • mount it

  • use it like a real disk

  • even put LVM on it

But remember:
It depends entirely on the network connection.


Setup Overview

We’ll have:

Server (Host machine)

  • Exports disk image via NBD

  • Runs nbd-server

Client (Other LAN machine)

  • Connects via nbd-client

  • Mounts /dev/nbd0

Router just handles LAN routing — nothing special needed unless you want access from outside.


Step 1 — Install NBD

On both machines:

Debian/Ubuntu/Mint:

sudo apt install nbd-server nbd-client

Fedora:

sudo dnf install nbd nbd-client

Arch:

sudo pacman -S nbd

Step 2 — Create a Block Device to Export (Server)

You can export:

  • A real partition (/dev/sdb1)

  • Or a disk image file (safer for testing)

Let’s create a 10GB image:

sudo fallocate -l 10G /srv/nbd-disk.img

Format it:

sudo mkfs.ext4 /srv/nbd-disk.img

Step 3 — Configure NBD Server

Edit config:

sudo nano /etc/nbd-server/config

Add:

[nbdshare] exportname = /srv/nbd-disk.img

Start service:

sudo systemctl restart nbd-server sudo systemctl enable nbd-server

Check:

sudo ss -tulpn | grep 10809

Default NBD port = 10809


Step 4 — Connect From Client

Load NBD kernel module:

sudo modprobe nbd max_part=8

Connect:

sudo nbd-client SERVER_IP 10809 /dev/nbd0

Example:

sudo nbd-client 192.168.1.100 10809 /dev/nbd0

Now check:

lsblk

You should see /dev/nbd0


Step 5 — Mount It

sudo mount /dev/nbd0 /mnt

Boom. You now have remote storage mounted.


Disconnect Properly

On client:

sudo umount /mnt sudo nbd-client -d /dev/nbd0

Security (Important)

By default NBD:

  • Has no encryption

  • Has minimal authentication

For LAN use at home → usually fine.

For better security:

  • Run it over SSH tunnel:

On client:

ssh -L 10809:localhost:10809 user@server

Then connect to:

sudo nbd-client localhost 10809 /dev/nbd0

Now traffic is encrypted.


Router Setup (LAN Hosting)

If both machines are on:

192.168.x.x

You need:

  • Nothing special.

  • Just ensure firewall allows port 10809.

On server (if using UFW):

sudo ufw allow 10809/tcp

If accessing from outside your LAN:

  • Forward port 10809 in router

  • BUT: Not recommended without SSH/VPN

Better:

  • Use WireGuard VPN

  • Or SSH tunnel


When To Use NBD vs Alternatives?

ToolUse Case
NBDRaw block device access
NFSFile sharing
SambaWindows-style file sharing
iSCSIEnterprise block storage
SSHFSSimple remote mount
GlusterFSDistributed storage

NBD = simple raw block over TCP.


Performance Reality

NBD is:

  • Fast on Gigabit LAN

  • Slow on WiFi

  • Sensitive to network drops

If connection drops while mounted → filesystem corruption risk.


Advanced Ideas

You can:

  • Put LVM on NBD

  • Use it for VM storage

  • Use it for diskless clients

  • Combine with RAID

  • Export real physical disk (/dev/sdb)


Pro Mode: Export Real Partition

Server:

exportname = /dev/sdb1

Now client sees the actual physical disk over network.

Be careful:
Only one machine should write to it.


Ooooh okay 😎 now we’re in proper “Linux wizard lab” territory.

You want to:

  • use chroot

  • mount another filesystem

  • use loopback devices

  • basically simulate another Linux environment inside your system

Let’s break this clean and powerful.


What is chroot?

chroot = change root

It changes the apparent root directory / for a process.

Inside chroot:

/ → becomes whatever directory you choose

It’s like entering another Linux installation without rebooting.

Used for:

  • Fixing broken systems

  • Building minimal systems

  • Embedded dev

  • Packaging

  • Rescue environments


What is a Loopback Mount?

A loopback device lets you mount a file as if it were a disk.

Example:

disk.img /dev/loop0 mounted as filesystem

Very useful for:

  • Disk images

  • Embedded systems

  • Testing

  • ISO files

  • VM images


Let’s Build a Mini Linux Root in a File

We’ll:

  1. Create a disk image

  2. Format it

  3. Mount it

  4. Install minimal system

  5. Enter with chroot


Step 1 — Create Disk Image

Example: 2GB image

fallocate -l 2G rootfs.img

Alternative:

dd if=/dev/zero of=rootfs.img bs=1M count=2048 status=progress

Step 2 — Attach to Loop Device

sudo losetup -fP rootfs.img

Check:

losetup -a

You’ll see something like:

/dev/loop0

Step 3 — Format It

sudo mkfs.ext4 /dev/loop0

Step 4 — Mount It

sudo mkdir /mnt/testroot sudo mount /dev/loop0 /mnt/testroot

Now /mnt/testroot behaves like a real disk.


Step 5 — Install a Minimal System Inside

On Debian/Ubuntu:

sudo debootstrap stable /mnt/testroot http://deb.debian.org/debian/

If Ubuntu:

sudo debootstrap jammy /mnt/testroot http://archive.ubuntu.com/ubuntu/

Now you have a minimal root filesystem.


Step 6 — Prepare for chroot (Important!)

Before entering chroot, you must mount special filesystems:

sudo mount --bind /dev /mnt/testroot/dev sudo mount --bind /proc /mnt/testroot/proc sudo mount --bind /sys /mnt/testroot/sys sudo mount --bind /run /mnt/testroot/run

Optional but helpful:

sudo cp /etc/resolv.conf /mnt/testroot/etc/

Step 7 — Enter chroot

sudo chroot /mnt/testroot /bin/bash

Now:

root@machine:/#

You are inside the new root filesystem.


What Can You Do Inside?

  • Install packages

  • Configure system

  • Build software

  • Create custom embedded root

  • Fix broken installs

  • Compile kernels

  • Setup bootloader


Leaving chroot

Just:

exit

Then clean up:

sudo umount /mnt/testroot/dev sudo umount /mnt/testroot/proc sudo umount /mnt/testroot/sys sudo umount /mnt/testroot/run sudo umount /mnt/testroot sudo losetup -d /dev/loop0

Mount Filesystem Directly with Loop Option (Simpler Way)

Instead of manual losetup, you can:

sudo mount -o loop rootfs.img /mnt/testroot

Linux automatically assigns loop device.


Advanced Use Cases

1️⃣ Repair Broken System from Live USB

mount /dev/sda2 /mnt mount --bind /dev /mnt/dev mount --bind /proc /mnt/proc mount --bind /sys /mnt/sys chroot /mnt

Then:

grub-install update-grub apt install --reinstall linux-image

2️⃣ Build Custom Embedded RootFS

  • Create rootfs

  • Install BusyBox

  • Add minimal /etc/inittab

  • Add init script

  • Package into initramfs


3️⃣ Cross Architecture chroot (ARM on x86)

Install qemu-user-static:

sudo apt install qemu-user-static

Copy emulator binary into chroot:

/usr/bin/qemu-aarch64-static

Then you can chroot into ARM rootfs on x86.

Magic.


Important Differences

ToolIsolation Level
chrootBasic filesystem isolation
systemd-nspawnLightweight container
DockerFull container
KVMFull virtualization

chroot is NOT secure sandboxing.


Pro Tips

  • Always bind /dev, /proc, /sys

  • If networking fails → check /etc/resolv.conf

  • If “device busy” on unmount:

    lsof +D /mnt/testroot

How Linux Permissions Work

Every file has:

rwxr-x---

Break it into 3 groups:

[ owner ][ group ][ others ]

Each group has:

  • r = read (4)

  • w = write (2)

  • x = execute (1)

You can see permissions with:

ls -l

Example:

-rwxr-xr--

Meaning:

  • Owner: read + write + execute

  • Group: read + execute

  • Others: read only


Numeric (Octal) chmod — The Most Common Way

You add the numbers:

PermissionValue
r4
w2
x1

Examples:

CodeMeaning
74+2+1 = rwx
64+2 = rw-
54+1 = r-x
4r--

Common chmod Examples

Make script executable

chmod +x script.sh

Set full permissions for owner, read/execute for others

chmod 755 file

= rwxr-xr-x

Private file (like SSH key)

chmod 600 id_rsa

= rw-------

Owner full, group full, others none

chmod 770 file

Symbolic chmod (More Precise Control)

Structure:

chmod [who][operator][permissions] file

Who:

  • u = user (owner)

  • g = group

  • o = others

  • a = all

Operators:

  • + add

  • - remove

  • = set exactly

Examples:

Add execute for owner

chmod u+x file

Remove write from group

chmod g-w file

Give everyone read

chmod a+r file

Set exact permission for others

chmod o=rx file

Directories Are Different

Important:

For directories:

  • r = list files

  • w = create/delete files

  • x = enter directory

If directory lacks x, you cannot cd into it even if you can see it.


Recursive chmod

Change permissions for everything inside a directory:

chmod -R 755 folder/

Be careful with -R — you can break systems fast.


Change Owner vs Permissions

Permissions ≠ ownership.

To change owner:

sudo chown user:group file

Example:

sudo chown alice:developers project.c

Special Permissions (Advanced)

SUID (Set User ID)

Runs executable as file owner.

chmod u+s file

Shows as:

-rwsr-xr-x

Used for:

  • /usr/bin/passwd


SGID

chmod g+s dir

On directories:

  • New files inherit group


Sticky Bit

Used on shared directories like /tmp

chmod +t dir

Only file owner can delete their files.

Shows as:

drwxrwxrwt

Quick Cheat Sheet

GoalCommand
Make script executablechmod +x script.sh
Private filechmod 600 file
Web folder typicalchmod 755 folder
Remove write for otherschmod o-w file
Give group writechmod g+w file

Real-World Examples

Fix “Permission denied” when running script

chmod +x script.sh

SSH private key error:

Permissions are too open

Fix:

chmod 600 ~/.ssh/id_rsa

Web server needs folder access:

chmod 755 /var/www/html

What NOT To Do

Never casually run:

chmod -R 777 /

That’s chaos mode.

777 means:

  • Anyone can read, write, execute

Bad for security.

Window managers vs full desktop environments
X11 vs Wayland
Tiling vs floating
Minimal vs feature-rich

Let’s untangle it cleanly.


Big Picture First

Linux graphics stack (simplified):

Kernel (DRM, GPU drivers)Display Server (X11 or Wayland)Window Manager / CompositorDesktop EnvironmentApplications

Not every system has all layers. That’s where the differences start.


What Is X (X11)?









X11 (Xorg):

  • Old but powerful display server

  • Network-transparent (apps can render remotely)

  • Modular design

  • Used since the 80s

It handles:

  • Drawing windows

  • Input (keyboard/mouse)

  • Display communication

But it does NOT manage windows itself.

That’s the job of a window manager.


What Is a Window Manager (WM)?

A window manager controls:

  • Window borders

  • Moving windows

  • Resizing

  • Tiling

  • Stacking

  • Focus behavior

Lightweight. No full desktop tools.


Types of Window Managers

 Floating Window Managers

Traditional desktop style.

Examples:

  • Openbox

  • Fluxbox

  • IceWM

  • FVWM

Windows overlap and move freely.


Tiling Window Managers










Auto-arrange windows without overlap.

Examples:

  • i3

  • bspwm

  • awesome

  • dwm

  • xmonad

Keyboard-driven.
Very popular among developers.


Dynamic / Hybrid

  • Can tile and float

  • Example: AwesomeWM


What Is a Desktop Environment (DE)?

A Desktop Environment includes:

  • Window manager

  • Panel

  • File manager

  • Settings tools

  • Notification system

  • Session manager

  • Default apps

  • Display manager

It’s a complete ecosystem.


Major Desktop Environments


GNOME









  • Modern

  • Minimalist

  • Wayland-first

  • Strong ecosystem

  • Used by Fedora default

Heavy but polished.


KDE Plasma







  • Highly customizable

  • Traditional layout

  • Very feature-rich

  • Lightweight compared to GNOME (modern versions)

Used by Kubuntu, openSUSE.


XFCE









  • Lightweight

  • Stable

  • Traditional layout

  • Great for older machines


LXQt

  • Very lightweight

  • Qt-based

  • Minimal

Good for low-end systems.


Cinnamon

  • Traditional desktop

  • Linux Mint default

  • User-friendly


Wayland (The Future)

Wayland replaces X11 in many systems.

Benefits:

  • Better security

  • Better performance

  • No legacy complexity

  • Simpler architecture

In Wayland:

  • Compositor = window manager

  • No separate X server

GNOME and KDE default to Wayland now.


X11 vs Wayland

FeatureX11Wayland
AgeOldModern
Network transparencyYesNo (by default)
SimplicityComplexCleaner
SecurityWeakerStronger
Remote appsBuilt-inNeeds tools

X11 still widely used.
Wayland is growing fast.


Window Manager vs Desktop Environment

FeatureWMDE
Memory useVery lowHigher
CustomizationManualBuilt-in
GUI toolsMinimalFull suite
Best forPower usersEveryday users

Example Configurations

Minimal hacker setup:

Arch + i3 + Alacritty

Traditional desktop:

Debian + XFCE

Modern polished:

Fedora + GNOME

Customizable powerhouse:

KDE Plasma

Why People Love Tiling WMs

  • Keyboard only workflow

  • No mouse needed

  • Super fast productivity

  • Less distraction

  • Feels like “terminal for windows”


Performance Considerations

Low RAM machine?

  • XFCE

  • LXQt

  • i3

High-end machine?

  • KDE Plasma

  • GNOME


Advanced Topics

  • Compositors (Picom)

  • Xmonad in Haskell

  • Hyprland (Wayland tiling)

  • sway (Wayland i3 clone)

  • Display managers (GDM, SDDM, LightDM)

  • Xmonad + xmobar setups

  • GPU acceleration stack


Real Talk

Most beginners:
Start with GNOME or KDE.

Power users:
Move to tiling WMs.

Embedded systems:
Often no GUI at all.

Servers:
No graphical environment.

What Is LAMP?




LAMP =

  • Linux

  • Apache

  • MySQL (now usually MariaDB)

  • PHP

It’s a full web server stack.

Client (browser) → Apache → PHP → MariaDB → back to browser


What Each Part Does

Linux

The OS. Handles:

  • Networking

  • File permissions

  • Services

  • Security

Apache (Web Server)

Apache:

  • Listens on port 80 (HTTP)

  • Listens on port 443 (HTTPS)

  • Serves static files

  • Passes .php files to PHP engine

It’s one of the most used web servers historically (though Nginx is huge now too).


PHP (Server-Side Language)

PHP:

  • Executes code on the server

  • Generates HTML dynamically

  • Talks to database

  • Handles forms, sessions, login systems

Example flow:

<?php echo "Hello World"; ?>

Apache processes it → sends output to browser.


MariaDB (Database)

MariaDB is a drop-in replacement for MySQL.

MySQL was bought by Oracle.
MariaDB was forked by original MySQL developers.

MariaDB:

  • Stores data (users, posts, products)

  • Handles SQL queries

  • Runs on port 3306

  • Supports transactions, indexing, replication

Example query:

SELECT * FROM users WHERE id=1;

Installing Old-School LAMP (Ubuntu/Debian Example)

1️⃣ Install Apache

sudo apt update sudo apt install apache2

Check:

http://localhost

Service:

sudo systemctl status apache2

2️⃣ Install MariaDB

sudo apt install mariadb-server

Secure it:

sudo mysql_secure_installation

Start:

sudo systemctl enable mariadb

Test login:

sudo mysql

3️⃣ Install PHP

sudo apt install php libapache2-mod-php php-mysql

Restart Apache:

sudo systemctl restart apache2

Test PHP:

Create file:

sudo nano /var/www/html/info.php

Add:

<?php phpinfo(); ?>

Visit:

http://localhost/info.php

If you see PHP info page → working.


What MariaDB Actually Does

MariaDB:

  • Stores relational data in tables

  • Handles SQL queries

  • Manages indexes

  • Handles concurrency

  • Enforces constraints

  • Can replicate to other servers

  • Can do clustering (Galera)

In LAMP, PHP talks to MariaDB via:

mysqli_connect()

or PDO.


PHP Security Risks (Important)

PHP itself isn’t “bad” — but badly written PHP is dangerous.

Common vulnerabilities:

1️⃣ SQL Injection

Bad:

$query = "SELECT * FROM users WHERE name = '$name'";

Attacker enters:

' OR 1=1 --

Fix: Use prepared statements.


2️⃣ File Upload Exploits

If you allow:

move_uploaded_file()

Without validation → attackers upload webshell.


3️⃣ Remote Code Execution

Using:

eval() exec() shell_exec()

Very dangerous.


4️⃣ XSS (Cross-Site Scripting)

Not escaping user input.

Fix:

htmlspecialchars()

5️⃣ Old PHP Versions

PHP 5.x = disaster zone
Always use supported versions (PHP 8+)


Basic LAMP Security Hardening

Firewall

sudo ufw allow 'Apache Full'

Disable directory listing

In Apache config:

Options -Indexes

Remove info.php after testing

Secure MariaDB

  • Strong root password

  • Remove anonymous users

  • Disable remote root login

File permissions

Web root:

/var/www/html

Typically:

chmod 755 directories chmod 644 files

Never 777.


Apache Structure (Important to Understand)

Main config:

/etc/apache2/apache2.conf

Virtual hosts:

/etc/apache2/sites-available/

Enable site:

sudo a2ensite mysite.conf sudo systemctl reload apache2

Modules:

sudo a2enmod rewrite

Why Apache Was King

  • .htaccess support

  • mod_php integration

  • Shared hosting friendly

  • Huge ecosystem

  • WordPress built on it

Today:

  • Nginx + PHP-FPM is more common in performance setups

  • Apache still extremely common


Apache vs Nginx

FeatureApacheNginx
Easy configMedium
PerformanceGoodVery good
.htaccessYesNo
Market historyHugeGrowing

Real-World Advice

If you're building LAMP for:

Learning → Perfect.

Home lab → Good.

Production server → Add:

  • HTTPS (Let's Encrypt)

  • Fail2ban

  • Firewall

  • Disable root SSH login

  • Regular updates

What Is SELinux?

SELinux = Security-Enhanced Linux

It’s a Mandatory Access Control (MAC) system built into the Linux kernel.

Originally developed by the NSA (yes, really), later integrated into the Linux kernel.

It adds an extra security layer on top of normal Unix permissions.


Normal Linux Security (DAC)

Linux normally uses:

  • Owner

  • Group

  • Permissions (rwx)

This is called:

DAC — Discretionary Access Control

Example:
If a process runs as root → it can usually access anything.


SELinux Adds MAC (Mandatory Access Control)

Even if a process runs as root:

SELinux can still say:

NO.

It controls:

  • What processes can access

  • Which files

  • Which ports

  • Which system calls

  • Which other processes

Based on policies.


How SELinux Works (Conceptually)

Every object has a security context:

Files:

system_u:object_r:httpd_sys_content_t:s0

Processes:

system_u:system_r:httpd_t:s0

Format:

user:role:type:level

The important part is usually the type.

Example:

  • httpd_t → Apache process

  • httpd_sys_content_t → Web content

Policy defines:

httpd_t can read httpd_sys_content_t

But NOT:

httpd_t cannot read /etc/shadow

Even as root.


SELinux Modes

Check mode:

getenforce

Modes:

ModeMeaning
EnforcingBlocking violations
PermissiveLogs but allows
DisabledOff

Temporarily set:

sudo setenforce 0

Persistent change in:

/etc/selinux/config

Common SELinux Commands

Check context:

ls -Z

Change context:

chcon -t httpd_sys_content_t file

Restore default context:

restorecon -Rv /var/www/html

Check port context:

semanage port -l

Allow Apache on custom port:

sudo semanage port -a -t http_port_t -p tcp 8080

Example: Apache Blocked by SELinux

Apache running.
File permissions correct.
But still 403 error.

Why?

Because file label is wrong.

Fix:

restorecon -Rv /var/www/html

This is classic.


Where Is SELinux Used Most?

  • Fedora

  • RHEL

  • CentOS

  • AlmaLinux

  • Rocky Linux

  • Android (yes, Android uses SELinux heavily)

Ubuntu uses AppArmor instead (by default).


Why Some People Disable SELinux

Because:

  • Misconfigured services break

  • Hard to understand

  • Complex policies

But in production environments:

  • It’s extremely powerful

  • Prevents privilege escalation

  • Limits damage of exploits


What Is Backporting?

Now switching to second topic.

Backport = taking a feature or fix from a newer version and applying it to an older version.

Example:

Kernel 6.8 has security fix.
You’re running 5.15 LTS.

Instead of upgrading whole kernel:
→ Distro backports the fix into 5.15.

Version number stays 5.15
But security patch exists inside.


Why Backports Matter

Enterprise distros:

  • RHEL

  • Debian Stable

  • Ubuntu LTS

Prefer:

  • Stability

  • Long-term maintenance

Instead of upgrading to new kernel:
They patch old kernel with:

  • Security fixes

  • Critical bug fixes

  • Sometimes features

This keeps:

Stable API Stable ABI Stable userland

Example of Kernel Backport

You run:

uname -r

It says:

5.14.0

But internally:

  • Contains patches from 6.x

  • Includes modern driver fixes

  • Includes CVE fixes

That’s backporting.


SELinux Backports

Security patches to SELinux:

  • Policy fixes

  • Kernel LSM fixes

  • CVE fixes

Are often backported to LTS kernels.


Why Backport Instead of Upgrade?

Upgrading kernel:

  • Can break drivers

  • Can break ABI

  • Can break enterprise apps

Backporting:

  • Keeps system stable

  • Fixes security issues

  • Maintains compatibility


Real-World Example

Heartbleed bug:

  • Fix backported to old OpenSSL versions.

  • No major version change required.

Same for:

  • Kernel privilege escalation bugs.

  • Filesystem corruption fixes.


Relationship Between SELinux & Backporting

Enterprise Linux systems:

  • Use SELinux enforcing.

  • Backport security fixes.

  • Maintain stable kernel base.

  • Guarantee long-term support.

That’s why:
RHEL systems can run for 10+ years.


Big Picture

SELinux = runtime security enforcement layer.

Backporting = distribution maintenance strategy.

One protects system during execution.
The other protects system across time.


Interesting Fact

Android heavily uses SELinux enforcing mode.

Most desktop users don’t realize:
SELinux is protecting billions of devices daily.

What Are “Cloud-Based Linux Apps”?

In simple terms:

Apps that run on Linux servers in data centers instead of your local machine.

Instead of:

You → local program

It becomes:

You → browser/app → cloud server (Linux) → database → API → response

Almost all modern cloud infrastructure runs on Linux.

Examples:

  • Web apps (WordPress, SaaS tools)

  • Containers (Docker/Kubernetes workloads)

  • AI services

  • Databases

  • Storage systems

  • Serverless functions


Canonical (Ubuntu) and Cloud









Let’s break this properly:

  • What are Linux cloud-based apps?

  • What does Canonical (Ubuntu company) actually do in the cloud?

  • And where is this whole thing heading?


What Are “Cloud-Based Linux Apps”?

In simple terms:

Apps that run on Linux servers in data centers instead of your local machine.

Instead of:

You → local program

It becomes:

You → browser/app → cloud server (Linux) → database → API → response

Almost all modern cloud infrastructure runs on Linux.

Examples:

  • Web apps (WordPress, SaaS tools)

  • Containers (Docker/Kubernetes workloads)

  • AI services

  • Databases

  • Storage systems

  • Serverless functions


🏢 Canonical (Ubuntu) and Cloud

Their focus today is heavily cloud + enterprise.

1️⃣ Ubuntu Cloud Images

Ubuntu is one of the most used OS images on:

  • AWS

  • Google Cloud

  • Microsoft Azure

  • Oracle Cloud

You can launch a cloud VM and choose:

Ubuntu Server LTS

That’s Canonical’s biggest presence in cloud.


2️⃣ Ubuntu Pro

Enterprise support:

  • Security patches

  • Extended maintenance

  • Kernel livepatching

  • Compliance certifications

Companies pay for this.


3️⃣ Kubernetes & Containers

Canonical provides:

  • MicroK8s

  • Charmed Kubernetes

  • Juju orchestration

  • MAAS (bare metal provisioning)

Cloud-native world = containers + orchestration.

Ubuntu is optimized for that.


4️⃣ Snap Packages

Snaps are container-like packages.

Canonical promotes Snap as:

  • Cloud-friendly

  • Sandboxed

  • Self-contained apps

  • Easy updates

Example:

snap install nextcloud

Some love it. Some… debate it loudly 😄


What Cloud Apps Actually Use

Most cloud Linux stacks today use:

  • Docker / OCI containers

  • Kubernetes

  • CI/CD pipelines

  • Infrastructure as Code (Terraform, Ansible)

  • Observability (Prometheus, Grafana)

  • API microservices

LAMP still exists, but now usually inside containers.


Modern Linux Cloud Architecture

Instead of:

Apache + PHP + MySQL on one server

Now it's more like:

Load Balancer ↓ Kubernetes Cluster ↓ Multiple containerized services ↓ Managed database

Linux is underneath all of it.


What Does the Future Hold?

Let’s talk realistically.

1️⃣ Linux Will Stay Dominant in Cloud

Over 90% of public cloud workloads run on Linux.

That’s not changing.


2️⃣ Containers Everywhere

Traditional servers → fading.

Future:

  • Containers

  • Immutable infrastructure

  • Declarative deployments


3️⃣ More Security Focus

Expect:

  • More sandboxing

  • More kernel hardening

  • More zero-trust architecture

  • eBPF monitoring


4️⃣ Rust in the Kernel (Slowly)

Linux kernel is mostly C.

But:

  • Rust is entering kernel space.

  • Memory safety is a big future theme.


5️⃣ Serverless & Edge Computing

Apps won’t always run in big data centers.

Future trends:

  • Edge devices running Linux

  • IoT clusters

  • Micro-clouds

  • Kubernetes on tiny systems

Ubuntu Core is Canonical’s IoT/embedded cloud OS.


6️⃣ AI + Linux

AI infrastructure:

  • GPUs

  • CUDA

  • Tensor frameworks

  • Data pipelines

All heavily Linux-based.

Cloud AI workloads = Linux everywhere.


Is Traditional Hosting Dead?

No — but it’s evolving.

Old model:

SSH into server Install Apache Edit config

New model:

Push code to Git CI builds container Auto-deploy to Kubernetes

Way more automation.


Big Picture

Linux has shifted from:

  • “Desktop alternative OS”

To:

  • “Backbone of the internet”

Cloud computing basically runs on Linux.


If You’re Thinking Career-Wise

The most valuable Linux skills now are:

  • Docker

  • Kubernetes

  • Bash scripting

  • Git

  • Networking

  • Cloud provider basics

  • Infrastructure as Code

  • Security fundamentals


Real Talk

The future is less about:

manually configuring servers

And more about:

automating infrastructure

But understanding old-school LAMP, chroot, permissions, etc. gives you deeper knowledge than most “cloud-only” engineers.

That foundation matters.

How Linux Servers and DNS Work Together


Most of the internet runs on Linux.
Most DNS servers run on Linux.

Most web servers run on Linux.

So here’s the flow when you type:

https://example.com

Step 1 — DNS Lookup (Phonebook of the Internet)

Your browser asks:

“What IP address is example.com?”

That request goes to a DNS resolver.

Common DNS software running on Linux:

  • BIND

  • Unbound

  • PowerDNS

  • Knot DNS

The DNS system translates:

example.com → 93.184.216.34

Step 2 — Connect to Linux Web Server

Once your system has the IP address:

  • It opens a TCP connection (port 80 or 443)

  • It talks to a web server (Apache, Nginx, etc.)

  • That server is very often running Linux

Then:

  • PHP runs

  • Database queried

  • Response sent back

The DNS Hierarchy (Very Important)






DNS is hierarchical:

1️⃣ Root Servers
2️⃣ TLD Servers (.com, .org, .net, etc.)
3️⃣ Authoritative Name Servers
4️⃣ Your DNS Resolver

Let’s walk through a full lookup.


Full DNS Resolution Process

You type:

www.example.com
  1. Your system asks local resolver (ISP or 8.8.8.8).

  2. Resolver asks root server:

    Who handles ".com"?

  3. Root server replies:

    Ask the .com TLD servers.

  4. Resolver asks .com TLD:

    Who handles example.com?

  5. TLD replies:

    Ask ns1.example.com

  6. Resolver asks authoritative server:

    What is www.example.com?

  7. It replies with IP address.

  8. Resolver caches answer.

  9. Browser connects to Linux web server.

All this usually happens in milliseconds.


Why Linux Matters Here

Linux runs:

  • DNS root servers

  • TLD servers

  • Authoritative servers

  • Recursive resolvers

  • Web servers

  • CDN edge nodes

It’s the backbone OS of internet infrastructure.


Can DNS Crash the Entire Internet?

Short answer:

Yes — but it’s very hard.

Let’s go deeper.


Scenario 1 — DNS Outage at Major Provider

If Cloudflare DNS or Google DNS goes down:

  • Millions of sites become unreachable

  • But internet itself still works

We’ve seen this happen.


Scenario 2 — Root DNS Servers Fail

There are 13 logical root server clusters:

  • A.root-servers.net

  • B.root-servers.net

  • etc.

They are:

  • Distributed globally

  • Anycast

  • Redundant

  • Run by different organizations

If some fail → system keeps working.
If all fail → new domain lookups stop working.

BUT:

  • Cached DNS entries would still work.

  • Existing connections would continue.

So internet wouldn’t instantly die — but new lookups would fail.


Scenario 3 — Massive DNS Attack

DNS is vulnerable to:

  • DDoS

  • BGP hijacking

  • Misconfiguration

  • DNS cache poisoning

Example:
If someone hijacks a TLD server → chaos for that TLD.

But full global collapse? Extremely unlikely.

The system is designed to be resilient.


Why DNS Is Critical

Without DNS:

You’d have to type:

https://142.250.74.206

instead of:

google.com

Technically internet still works without DNS.
But humans would struggle massively.


Real Example of DNS Breaking Things

In 2016:

  • Dyn DNS was attacked.

  • Twitter, Netflix, GitHub went down.

  • Huge outage.

Internet didn’t collapse.
But it felt broken.


Could Internet Crash Worldwide?

True worldwide collapse would require:

  • Root DNS system failure

  • Major backbone BGP failure

  • Global power failures

  • Massive routing misconfiguration

Possible?
Theoretically yes.

Likely?
Extremely low.

Internet was designed to survive nuclear war-level disruptions.


DNS Is Actually More Fragile Than You Think

The system works because:

  • Trust exists between operators

  • BGP routing works correctly

  • No large-scale malicious coordination happens

But misconfiguration has caused near-global incidents before.


DNSSEC

To prevent tampering:

  • DNSSEC signs DNS records cryptographically.

  • Prevents spoofing.

  • Increasingly important.


What Would Happen If DNS Stopped for 24 Hours?

  • Banking stops

  • Cloud apps fail

  • Email breaks

  • APIs fail

  • CDNs stop resolving

  • Most modern services stop functioning

Society would feel it quickly.


Big Picture

Linux servers:

  • Host websites

  • Run DNS infrastructure

  • Serve cloud apps

  • Power data centers

DNS:

  • Connects names to IPs

  • Makes internet usable

  • Is decentralized but interdependent

Internet:

  • Highly redundant

  • Extremely distributed

  • Surprisingly resilient

How do old online games run on private servers?
What’s DNS doing there?
And is Linux used in those servers?

How Old Online Games Originally Worked

Typical legacy online game (2000–2012 era) worked like this:

Game Client (your PC) ↓ Login Server ↓ Game Server (world server) ↓ Database Server

When you launched the game, it:

  1. Contacted a domain name (like login.gamecompany.com)

  2. DNS resolved that name to an IP

  3. Connected via TCP or UDP

  4. Authenticated with central servers

  5. Joined a game world

All centralized.


What Happens When Official Servers Shut Down?

When company shuts servers:

  • Domain still exists or disappears

  • Login server stops responding

  • Game becomes “dead”

But the client still contains:

  • Network protocol logic

  • Server address

  • Encryption methods

  • Sometimes even fallback IPs

That’s where private servers come in.


How Private Servers Work

There are two main approaches.


1️⃣ DNS Redirection Trick

The client expects:

login.originalgame.com

Private server operator:

  • Creates their own server

  • Sets up custom DNS

  • Points that domain to their own IP

Ways this is done:

Method A — Local hosts file

On your PC:

C:\Windows\System32\drivers\etc\hosts

Or on Linux:

/etc/hosts

Add:

123.45.67.89 login.originalgame.com

Now your PC resolves that domain to private server IP.

No global DNS change needed.


Method B — Custom DNS server

Private server gives you a DNS resolver IP.

You change your DNS to that server.

It answers:

login.originalgame.com → private server IP

Everything else resolves normally.


Method C — Patch the game client

Some private servers:

  • Modify the client executable

  • Replace hardcoded IP address

  • Change server address inside config file

More direct approach.


What About the Game Server Itself?

Private servers usually:

  • Reverse engineered protocol

  • Rebuilt server software

  • Or leaked server binaries

The server must:

  • Emulate login/auth server

  • Emulate world server

  • Handle packets correctly

  • Match client version


Is Linux Used?

Yes. Very often.

Reasons:

  • Stable

  • Cheap hosting

  • Easy networking

  • Good performance

  • Low memory overhead

  • Good database support (MariaDB, PostgreSQL)

Many private servers run on:

  • Ubuntu Server

  • Debian

  • CentOS / AlmaLinux

  • Sometimes FreeBSD


Database Layer

Most legacy MMORPG private servers use:

  • MySQL / MariaDB

  • SQLite (small games)

  • PostgreSQL (less common in old games)

Stores:

  • Accounts

  • Character data

  • Inventory

  • World state


Network Layer

Old games usually used:

  • TCP for login

  • UDP for gameplay

  • Custom binary protocols

Private server must perfectly mimic packet structure.

Even tiny protocol mismatch = client disconnect.


Example Architecture of a Private Server

Linux VPS ├── Login server daemon ├── World server daemon ├── Database (MariaDB) └── DNS redirection (optional)

Sometimes separated across multiple machines.


Why DNS Matters

DNS redirection allows:

  • No client modification (in some cases)

  • Transparent rerouting

  • Simpler onboarding for players

It’s basically tricking the client into thinking it's talking to original servers.


Legal Reality

Important distinction:

  • Reverse engineering for interoperability → sometimes legal.

  • Using leaked server code → usually illegal.

  • Hosting copyrighted content → often illegal.

Many private servers exist in legal gray zones.


Can Big Companies Stop Them?

Yes, by:

  • Legal takedown

  • Seizing domains

  • DMCA complaints

  • Blocking via hosting providers

But technically?
It’s just:

  • DNS + TCP/IP + custom protocol.

Hard to stop at pure network level.


Why Linux Is Perfect for This

Because:

  • Efficient networking stack

  • epoll for high concurrency

  • Easy firewall control (iptables/nftables)

  • Easy daemon management

  • Strong database support

  • Cheap VPS hosting

Linux dominates game server hosting in general.


Modern Equivalent

Same thing happens today with:

  • WoW private servers

  • Old MMORPGs

  • LAN emulated console servers

  • Decommissioned online-only games

The principle is identical.


Could DNS Crash a Private Game?

Yes — if:

  • Domain expires

  • DNS misconfigured

  • DDoS on DNS provider

  • DNS record poisoned

But players could still connect via direct IP if client allows.


Big Insight

At core, it’s just:

Client expects domain → DNS maps to IP → IP runs compatible server software.

That’s it.

Everything else is protocol emulation.


What Is a Linux Sysadmin?

A Linux System Administrator is the person responsible for:

  • Keeping Linux systems running

  • Securing them

  • Updating them

  • Fixing them when they break

  • Making sure services stay online

If Linux is the engine,
the sysadmin is the mechanic + pilot + firefighter.


What Does a Sysadmin Actually Do?

It depends on environment, but generally:


1️⃣ Server Setup

  • Install Linux (Ubuntu Server, RHEL, Debian, etc.)

  • Partition disks

  • Configure LVM

  • Setup users and SSH

  • Configure firewall

  • Harden system


2️⃣ Security

  • Manage users/groups

  • Configure sudo access

  • Apply updates & patches

  • Configure SELinux/AppArmor

  • Setup Fail2ban

  • Monitor logs

  • Manage SSH keys

Security is 50% of the job.


3️⃣ Networking

  • Static IP configuration

  • DNS setup

  • NTP sync

  • Firewall rules (iptables/nftables)

  • VPN setup (WireGuard, OpenVPN)

  • Load balancers

  • Reverse proxies (Nginx, HAProxy)


4️⃣ Services & Applications

Typical services:

  • Apache / Nginx

  • MariaDB / PostgreSQL

  • Docker

  • Kubernetes

  • Samba / NFS

  • Email servers

  • DNS servers

Sysadmin ensures:

  • They start at boot

  • They don’t crash

  • They perform well


5️⃣ Monitoring & Logs

Tools used:

  • journalctl

  • /var/log/

  • top / htop

  • iostat / vmstat

  • Prometheus / Grafana

  • Nagios / Zabbix

If a server goes down at 3AM:
Sysadmin is the one debugging it.


6️⃣ Backups

Very important.

  • rsync

  • Restic

  • BorgBackup

  • Snapshot systems (LVM, Btrfs)

  • Offsite backups

Golden rule:

If it’s not backed up, it doesn’t exist.


7️⃣ Automation

Modern sysadmins don’t manually configure 100 servers.

They use:

  • Bash scripting

  • Ansible

  • Terraform

  • CI/CD pipelines

  • Infrastructure as Code

Automation is huge in modern sysadmin work.


Types of Linux Sysadmins


Enterprise Sysadmin

  • RHEL-based systems

  • Strong security policies

  • SELinux enforcing

  • Formal change management


Cloud Sysadmin / DevOps

  • AWS / Azure / GCP

  • Docker / Kubernetes

  • Infrastructure as Code

  • CI/CD

  • Load balancing

More automation heavy.


Lab / Home Sysadmin

  • Self-hosted services

  • Virtual machines

  • NAS

  • Media servers

  • Learning environment


Embedded Sysadmin

  • IoT devices

  • Custom kernels

  • Minimal rootfs

  • System hardening


Core Skills of a Good Sysadmin

  1. Understand how Linux boots

  2. Understand filesystems

  3. Understand networking basics

  4. Understand permissions & sudo

  5. Understand systemd

  6. Know how to debug logs

  7. Stay calm during outage

That last one is underrated.


A Day in the Life

Morning:

  • Check monitoring alerts

  • Apply security updates

  • Review logs

Midday:

  • Deploy new service

  • Troubleshoot slow database

  • Fix permission issue

Evening:

  • Backup verification

  • Script automation improvements

3AM:

  • “Production server is down.”


Essential Commands Every Sysadmin Uses

top htop journalctl -xe systemctl status ss -tulpn ip a df -h du -sh free -h ps aux kill chmod chown usermod

And many more.


Sysadmin Mindset

  • Always assume hardware will fail.

  • Always assume users will break things.

  • Always assume security matters.

  • Always test backups.

  • Always document changes.


Old-School vs Modern Sysadmin

Old-school:

  • Manually configure server via SSH.

  • Edit config files by hand.

Modern:

  • Git repo with server config.

  • Automated deployment.

  • Immutable infrastructure.

  • Containers.


Difference Between Sysadmin & DevOps

Traditional sysadmin:

  • Maintains servers

DevOps:

  • Blends development + operations

  • Automates deployments

  • Uses CI/CD

  • Works closely with developers

But foundations are the same.


Why Linux Dominates Sysadmin World

Because:

  • Stability

  • Flexibility

  • Remote management

  • Scriptability

  • Open-source transparency

  • Server-friendly design


Reality Check

Being a Linux sysadmin is:

  • Deep knowledge

  • Constant learning

  • Real responsibility

  • Sometimes stressful

  • Very rewarding


Android is Linux (Technically)

Android uses the Linux kernel.

But:

AndroidTraditional GNU/Linux
Linux kernelLinux kernel
Android userspace (Bionic libc)GNU userspace (glibc, coreutils)
Java/Kotlin app frameworkNative Linux apps
Custom runtimeStandard POSIX environment

Android ≠ GNU/Linux phone.

It’s Linux kernel + Google stack.


Real Linux Phones

Let’s look at the main ones.

Linux-Based Phones (Non-Android)

PinePhone

  • ARM-based

  • Open hardware philosophy

  • Runs:

    • postmarketOS

    • Manjaro ARM

    • Ubuntu Touch

  • Replaceable components

  • Dev-focused

Downside:

  • Performance modest

  • Camera meh

  • Not daily-driver for most people


Librem 5 (by Purism)

  • Privacy-focused

  • Hardware kill switches

  • Runs PureOS (Debian-based)

  • No Android components

Downside:

  • Expensive

  • Battery life not great

  • Performance average


Ubuntu Touch

Maintained by UBports.

Runs on:

  • Some older Android devices

  • PinePhone

  • Fairphone models

Goal:

  • Convergence (phone becomes desktop when docked)


postmarketOS

Alpine-based.
Goal:

Make phones supported for 10+ years.

Very hacker-friendly.


Why We Don’t All Use Linux Phones

Now here’s the real talk.

1️⃣ Drivers

Phone hardware:

  • Modems

  • Cameras

  • GPUs

  • Sensors

Most drivers are:

  • Proprietary

  • Designed for Android stack

Linux desktop-style drivers often don’t exist.


2️⃣ Apps

No:

  • Native WhatsApp

  • Native Instagram

  • Native banking apps

  • Native Uber

Web apps exist — but limited.


3️⃣ Optimization

Android is heavily optimized for:

  • Battery

  • Background tasks

  • Radio management

  • Power saving

Pure Linux phones struggle here.


4️⃣ Ecosystem Lock-in

App stores.
Google services.
iOS ecosystem.

Most users don’t care about kernel philosophy.


But… Why Linux Phones Are Interesting

They give you:

  • Real shell access

  • Real system control

  • Package managers

  • SSH

  • Native GCC

  • Kernel hacking

  • No Google spyware

It’s basically a pocket Linux PC.


What a Linux Phone Looks Like Internally

Boot process:

Bootloader → Linux kernel → init → systemd → Wayland → mobile shell

Instead of:

Android framework → ART runtime → Java layer

Could Linux Phones Replace Android?

Technically: yes.
Practically: not soon.

Why?

Android:

  • Has billions of devices

  • Has full app ecosystem

  • Is deeply integrated with carriers

Linux phones:

  • Small dev community

  • Limited hardware

  • Niche market


Future Possibilities

Interesting trends:

1️⃣ Convergence devices

Phone → dock → desktop

2️⃣ More open hardware

RISC-V phones someday?

3️⃣ Post-Android forks

De-googled Android (GrapheneOS, /e/OS)

4️⃣ AI-powered edge devices

Linux-based edge computing phones


If You Really Want Linux On a Phone

Best current path:

  • PinePhone (learning/hobby)

  • Flash postmarketOS

  • Use SSH into it

  • Treat it like portable lab

Or:

  • Flash de-Googled Android (GrapheneOS)

  • Keep Android app compatibility


Brutal Honesty

Linux phones today are:

  • Great hacker toys

  • Not mainstream replacements

But they are fascinating from:

  • Embedded Linux

  • Kernel

  • Mobile hardware

  • Privacy engineering

IBM and Linux — Big Picture

Back in 2000, IBM made a huge move:

They announced a $1 billion investment in Linux.

That was massive at the time.

Since then:

  • IBM became one of the largest corporate contributors to the Linux kernel.

  • They support Linux on enterprise servers.

  • They built cloud infrastructure around Linux.

  • They bought Red Hat (huge moment).


🐧 IBM Uses Linux Where?






1️⃣ Enterprise Servers

IBM runs Linux on:

  • IBM Power systems

  • IBM Z mainframes

  • Enterprise cloud hardware

Linux runs natively on:

  • IBM POWER CPUs

  • IBM Z (mainframe architecture)

  • x86 servers

Yes — even mainframes run Linux.


2️⃣ IBM Z (Mainframes) Running Linux

IBM Z systems can:

  • Run thousands of Linux virtual machines

  • Use KVM on s390x architecture

  • Handle banks, governments, airlines

Linux is a first-class citizen there.


3️⃣ IBM Cloud

IBM Cloud runs heavily on:

  • Linux

  • Kubernetes

  • Red Hat OpenShift

Modern cloud stack = Linux everywhere.


Red Hat Acquisition

In 2019:

IBM bought:

Red Hat

For ~$34 billion.

Red Hat is behind:

  • RHEL (Red Hat Enterprise Linux)

  • OpenShift

  • Ansible

  • Many upstream kernel contributions

So now IBM owns one of the biggest enterprise Linux ecosystems.


Does IBM Contribute to the Kernel?

Yes. A lot.

IBM engineers contribute to:

  • s390 architecture support (mainframe)

  • POWER architecture

  • Virtualization

  • Filesystems

  • Networking

  • Security

  • Performance optimization

IBM is regularly in top corporate contributors list.


IBM + Linux = Enterprise Stability

IBM loves Linux because:

  • It’s open

  • It scales

  • It runs on multiple architectures

  • It reduces vendor lock-in

  • It supports enterprise workloads


Linux on IBM POWER

IBM POWER CPUs:

  • Compete with x86 in enterprise

  • Used in supercomputers

  • Support Linux natively

Example:
POWER9 + Linux was used in:

  • Summit supercomputer (at Oak Ridge)


Banking + Linux

Many banks run:

  • Linux on IBM Z

  • RHEL on enterprise hardware

  • Virtualized Linux instances

Most people don’t see it —
But global finance heavily relies on Linux on IBM hardware.


Why IBM Didn’t Just Use Windows?

Because:

  • Linux is customizable

  • Linux supports multiple CPU architectures

  • Linux is stable long-term

  • Enterprise kernel support is strong

  • IBM can contribute upstream instead of depending on a closed OS


Interesting Historical Fact

In early 2000s:

Linux went from:
“Hobbyist OS”

To:
“Enterprise-class platform”

And IBM played a big role in that shift.

That gave Linux corporate legitimacy.


IBM + Linux in Supercomputers

Most top supercomputers run Linux.

IBM-built systems:

  • Often run custom Linux builds.

  • Sometimes heavily modified kernels.

Linux dominates HPC (high-performance computing).


So Is Linux “Corporate” Now?

Yes and no.

Linux kernel:

  • Open-source

  • Community-driven

But:

  • Many contributors are paid engineers

  • Big companies (IBM, Intel, Google, Meta, Red Hat) contribute heavily

Linux is both:

  • Community project

  • Corporate infrastructure backbone


Quick Summary

IBM uses Linux in:

  • Mainframes

  • Enterprise servers

  • Cloud infrastructure

  • Supercomputers

  • Banking systems

  • Red Hat ecosystem

IBM contributes directly to:

  • Kernel development

  • Architecture support

  • Virtualization

  • Enterprise features


Tux is the official Linux mascot, created by Larry Ewing in 1996 (using GIMP!).
He’s not just a logo — he’s been reimagined for tons of Linux occasions over the years.







The fun side of Linux culture — the GNU vs Linux memes, and some classic Linus Torvalds quotes

GNU/Linux Joke #1 — The Naming War

“It’s not Linux. It’s GNU/Linux.”

Classic counter-response:

“Sure, and my car runs on gasoline, but I don’t call it Gasoline/Car.”

This debate has been running since the 90s and will probably outlive humanity.


GNU Stallman Meme

Richard Stallman famously says:

“I’d just like to interject for a moment. What you’re referring to as Linux is in fact GNU/Linux…”

That entire speech became legendary meme material.

The internet version expands into a full essay explaining that:

  • Linux = kernel

  • GNU = userspace tools

  • Together = GNU/Linux

It’s copy-pasted everywhere as satire now.


Tux Philosophy Joke

Why is the Linux mascot a penguin?

Because Linus once said he liked penguins — and that’s basically how open-source governance works sometimes 😄


Linus Torvalds Quotes

Linus is known for being blunt. Very blunt.


🔥 On simplicity:

“Talk is cheap. Show me the code.”

Short. Brutal. Accurate.


🔥 On programming:

“Bad programmers worry about the code. Good programmers worry about data structures and their relationships.”

Deep kernel wisdom.


🔥 On open source:

“Given enough eyeballs, all bugs are shallow.”

This is actually known as Linus’s Law.


🔥 On Microsoft (classic 90s Linus):

“If Microsoft ever does applications for Linux it means I’ve won.”

History did interesting things there 😄


🔥 On perfection:

“Software is like sex: it’s better when it’s free.”

That one became legendary.


🔥 On being a dictator:

“I’m an egotistical bastard, and I name all my projects after myself.”

(Linux = Linus + Unix)

At least he’s self-aware.


GNU vs Linux Humor

Meme Logic:

GNU:
“We built the tools!”

Linux:
“I built the kernel.”

Users:
“Can I just run my server please?”


Classic nerd joke:

Why do GNU users prefer dark mode?

Because light attracts proprietary software.


Kernel Dev Humor

Why do kernel developers hate race conditions?

Because they always lose.


Why is there no Windows mascot like Tux?

Because a blue screen doesn’t cuddle well.


Terminal Joke

How many Linux users does it take to change a light bulb?

None.

They’ll just recompile the kernel to improve darkness handling.


Sysadmin Humor

User:
“My computer isn’t working.”

Admin:
“Have you tried turning it off and on again?”

Linux admin version:

“Have you checked dmesg?”


GNU Philosophical Humor

GNU philosophy:

Freedom > convenience.

Linux pragmatism:

It works. Ship it.


Subtle Cultural Difference

Stallman:

  • Focused on freedom

  • Ideological

  • Ethical software purity

Torvalds:

  • Focused on engineering

  • Pragmatic

  • Performance and practicality

That tension shaped modern Linux.


Modern Linus Quote (More Mature Era)

After cooling down publicly:

“I’ve been working on improving my behavior.”

Even Linus evolved — like the kernel.


Final Classic

What does GNU stand for?

“GNU’s Not Unix.”

Recursive acronym.

Very hacker culture.


Kommentit

Tämän blogin suosituimmat tekstit

THE WENDOL WINTER - THE LAW OF TEETH - EATERS OF HISTORY - prequel trilogy

The Wizard of Oz (1939) - Kattava tietopaketti

Raspberry Pi hobbyist projects

Clerks (1994) - legendaarisen elokuvan juttua

Kalevala: Kullervon Tarina (2026) - Katsottu Teatterissa

Working Title: COLD LAKE OF FIRE - A John Woo style action - Setting in Tampere Finland

Kevin Smith - Tietoa ja fanitusta

FPGA technology presentation

Nuremberg (2025) - elokuvan pohdintaa