Dom Corriveau

Offgrid internet-in-a-box project - Part three

Posted on Jan 22, 2025

This is part three of a series of posts detailing how I am building (for fun, no other reason) a laptop for being offline and offgrid for an entire year.

Part one is about the rules of this project and the hardware I chose.

Part two is about what distro I chose.

This post is about what apps I have installed on the laptop and then how I have create a large archive of apps that will make installing any other application I may need possible while offline. It includes anything from the apt repos, flatpaks, Docker containers, and Android apps for my mobile device.

Base packages

At a minimum, I need to have aptly, flatpak, and docker installed so I can access the archived apps, packages, and containers I have on the connected drive to the laptop.

In a later post I will detail what data I archived for this project, including all the data for the archived apps I in this post. Needless to say, I need some apps right out of the gate for this project. For example, I need Obsidian as that is where all my notes are. I also need a video player, ebook reader, and a decent audio player for music, podcasts, and audiobooks.

I have a core list of command line applications I like to use and those don’t take up much room.

Since the device I’m using has limited storage space on the OS partition, I have a 128 GB m.2 SSD connected for extra space. This makes it so I can use AppImages for key apps and then save them on the mounted SSD and reduce the storage room needed on /root and /home. I have also done the same with run-in-place binaries and then added the directory to my $PATH. I don’t need much installed in the beginning since I will be carrying a huge archive of other apps I can install at any time.

This is what I have installed (minus some core utils like git):

Base

  • helix
  • btop
  • yazi
  • chezmoi & pull my dotfiles
  • aptly
  • flatpak + flathub configured
  • Luanti + VoxeLibre & Asuna games
  • Retroarch
  • KeePassXC
  • cotp
  • Docker & Podman
  • Cryptomator, Veracrypt, and Vaults
  • Obsidian
  • yt-dlp, fdroidcl, and spotdl.
  • Calibre
  • cmus
  • VLC
  • LocalSend

Containers

  • Kiwix
  • ArchiveBox
  • Dufs

Below are more notes on how I am carrying and using archived applications and packages so essentially anything is available to me in my one year of offline living.

aptly

For a long time I’ve wanted to run my own. LAN-only, mirror of the apt repo and aptly makes that possible. After using aptly, it works perfectly for this situation (and as a data hoarder and prepper).

Basically, aptly lets you run a mirror of a repository and let’s you customize it. I believe this tool is mainly focused on enterprise situations where teams are tightly managing what can be installed on machines (for security) and developing their own custom packages that they are not sharing publicly that you can then publish as a Debian repo and configure workstations to only use the aptly repo.

But, you can totally use it to mirror apt for Debian desktops, including contrib and non-free while also adding binaries that aren’t in apt and adding them to your own custom repo.

I’ll be honest, setting up aptly is a pain in the ass and my notes are long. I’m not going to share them in this post, but ping me if you want them. I have setup both an amd64 and an arm64 repo and save to the portable disk that is part of this build. I also save it to my NAS, cause, you know, data hoarder for life. When it is said and done, it takes up about 100 GB of storage.

Using aptly offline

On the offgrid machine, all we need to do is import the key into the system, start up the server, and edit sources.list to point to the localhost.

Generating the key is one of the first steps to configuring aptly, which is part of that long setup notes I’m not sharing today. This means it is important to have my_key.pub on the portable drive with all the offline data that I am using in this build.

I did the aptly setup on a different machine and then transferred to the portable drive. Doing it this way means I do not need to go through the entire setup process again, it is a portable setup and all I need to do is edit the aptly config file to where the archive is and then start the server.

Last, make a copy of the existing sources.list file in /etc/apt and then create a new one with just this in it:

deb http://localhost:8080/ bookworm main

Now we can install anything that is in the repo, which for me is literally everything in the apt repo.

Keeping up to date

In my scenario, I am archiving once and then going offgrid for an entire year. Yet, it is possible to update an existing aptly archive without having to download the entire repo.

I’m not going to layout those steps here, again the notes are long. But, I wanted to mention it is possible for anyone who is considering adding it to their own hoard.

Flatpaks

Installing GUI apps completely offline is tricky. Installing a package could have dependencies, which aptly might be able to fulfill. However, not a lot of apps have package builds or even releases on their git. The best way to have a collection of GUI desktop apps is to take advantage of the containerized technologies we have now.

In my testing, flatpaks are the easiest path forward. I’d love for everything to be an AppImage. There just aren’t enough out there. I adore AppImages and I wish more developers would release their apps this way. It seems that flatpak has won.

For what its worth, I don’t archive flatpaks just for this offgrid build. I keep an archive of them on my NAS, too. I am a ride or die data hoarder and having flatpaks as part of my hoard is important to me. Why? Because I can.

The problems with flatpak archives

I want to get this out of the way first. Using flatpaks as an app archive requires some setup before you can install them and needs access to the internet.

Obviously, you need to install flatpak. In an offline scenario, I can do this with aptly as the flatpak package is in the Debian repos. But, to make this work, flatpak relies configuring the collection ID to allow for offline installs from archives. The collection ID is a property of the actual remote repository. Flathub has one, org.flathub.Stable.

While you have access to the internet, you have to do this:

flatpak remote-modify --collection-id=org.flathub.Stable flathub
flatpak update

If you don’t add the collection ID to your remote configuration, you will be greeted by an error saying “Remote ‘flathub’ does not have a collection ID set”. If you omit flatpak update, the error will say “No such branch (org.flathub.Stable, ostree-metadata) in repository”.

**This cannot be solved in an offline situation. **

Archiving flatpaks

I wish I could create an offline clone of flathub similar to aptly. Or, better yet, create a container registry like with Docker. It is not the end of the world, though. Since my offgrid PC has limited storage space, I am using my desktop to download a specific list of flatpaks, then using flatpak create-usb to make an archived version of the application I can install on any machine. This feature is built into flatpak.

Unfortunately, you have to install the flatpak, then create the archive. So, I’ve written an ugly script for downloading and installing a list of flatpaks. It is essentially just a list of flatpaks to install and then the path to where to save the archive. Here is my ugly script:

#! /bin/bash

# Set up logging
logfile="/path/to/your/flatpak_install.log"  
exec &> >(tee -a "$logfile")  

# Update flatpak
flatpak update 2>&1 | tee -a "$logfile"

# Install applications
flatpak install -y --noninteractive flathub \
    org.videolan.VLC \
    org.libreoffice.LibreOffice \
    io.mpv.Mpv \
    org.geany.Geany \
    com.github.tchx84.Flatseal \
    io.github.flattool.Warehouse \
    com.nextcloud.desktopclient.nextcloud \
    md.obsidian.Obsidian \
    org.kde.kdenlive \
    org.gimp.GIMP \
    org.audacityteam.Audacity \
    fr.handbrake.ghb \
    org.keepassxc.KeePassXC \
    com.github.paolostivanin.OTPClient \
    org.mozilla.Thunderbird \
    info.mumble.Mumble \
    net.minetest.Minetest \
    io.freetubeapp.FreeTube \
    org.gnome.baobab \
    org.mozilla.firefox \
    com.makemkv.MakeMKV \
    org.flameshot.Flameshot \
    org.openrgb.OpenRGB \
    com.transmissionbt.Transmission \
    com.github.zocker_160.SyncThingy \
    com.bitwarden.desktop \
    com.tomjwatson.Emote \
    org.gnome.SimpleScan \
    org.gnome.PowerStats \
    org.localsend.localsend_app \
    org.gnome.Calculator \
    dev.lizardbyte.app.Sunshine \
    xyz.armcord.ArmCord \
    org.kiwix.desktop \
    org.gnome.Boxes \
    org.onlyoffice.desktopeditors \
    com.moonlight_stream.Moonlight \
    org.cryptomator.Cryptomator \
    io.github.mpobaschnig.Vaults 2>&1 | tee -a "$logfile"  

# Create archive
for app in \
    org.videolan.VLC \
    org.libreoffice.LibreOffice \
    io.mpv.Mpv \
    org.geany.Geany \
    com.github.tchx84.Flatseal \
    io.github.flattool.Warehouse \
    com.nextcloud.desktopclient.nextcloud \
    md.obsidian.Obsidian \
    org.kde.kdenlive \
    org.gimp.GIMP \
    org.audacityteam.Audacity \
    fr.handbrake.ghb \
    org.keepassxc.KeePassXC \
    com.github.paolostivanin.OTPClient \
    org.mozilla.Thunderbird \
    info.mumble.Mumble \
    net.minetest.Minetest \
    io.freetubeapp.FreeTube \
    org.gnome.baobab \
    org.mozilla.firefox \
    com.makemkv.MakeMKV \
    org.flameshot.Flameshot \
    org.openrgb.OpenRGB \
    com.transmissionbt.Transmission \
    com.github.zocker_160.SyncThingy \
    com.bitwarden.desktop \
    com.tomjwatson.Emote \
    org.gnome.SimpleScan \
    org.gnome.PowerStats \
    org.localsend.localsend_app \
    org.gnome.Calculator \
    dev.lizardbyte.app.Sunshine \
    xyz.armcord.ArmCord \
    org.kiwix.desktop \
    org.gnome.Boxes \
    org.onlyoffice.desktopeditors \
    com.moonlight_stream.Moonlight
do
    flatpak --verbose create-usb "/path/to/your/backup/directory/flatpak_archive/" "$app" 2>&1 | tee -a "$logfile" 
done

# Compress for archive
tar -cvf "/path/to/your/backup/directory/flatpak_archive_$(date +%Y-%m-%d).tar.gz" "/path/to/your/backup/directory/flatpak_archive/.ostree" | pigz 2>&1 | tee -a "$logfile"

A couple of notes:

  • I added -y --noninteractive so it wouldn’t prompt me for every install.
  • I added logging since I normally run this in the background and want to review if there are any errors.
  • At the end I compress all of the archived flatpaks so I can send it to my NAS. It also makes it easier to move them to other machines, like this offgrid build.

Offline flatpak installs

Okay, so now you’ve setup flatpak, configured flathub, and updated the collection-id for the offgrid PC. In the scenario I want to install one of these flatpaks, you use the --sideload flag.

flatpak install --sideload-repo=/path/to/your/backup/directory/flatpak_archive/.ostree/repo flathub name.flatpak.Flatpak

This will install the flatpak from the archive, including any of the other bits needed. I’ve tested this several times and works like a charm.

Docker

I have already written about how to save and import containers in a previous post. The post also details how I run my own container registry. But, you can easily use the same post for downloading, archiving, and then installing a container from the archive using the load flag.

One important bit of info: Using the save flag does not save the container data. It only saves the image. If you need the volumes saved, along with the data, of existing running containers, you need to save the data in a different way. I am in the habit of always using bind mounts to data in custom folders so the data is more portable. Keep this in mind in case you are trying to archive a container plus all of its data. When you load it onto another system and are offline, you will be disappointed to see it is only the image.

Here is the short version, copy and pasted from my previous post:

How-to save a container

Once you’ve identified the container you want to save and make sharing via sneakernet, you will save it like this:

docker save -o output/path/filename.tar name:tag

Here is an example from my archive:

docker save -o /mnt/usb_storage/libreoffice_`date +%Y-%m-%d`.tar lscr.io/linuxserver/libreoffice:latest

In this command I am saving to /mnt/usb_storage with the file name and appending the date so I know when it was saved. Last is where to grab the image.

Here is an updated script for pulling the containers I use, saving them, compressing into a single archive, and then removing the containers from the system.

#!/bin/bash

# Define the list of containers to pull
containers=(
  "jasongdove/ersatztv:latest-vaapi"
  "searxng/searxng"
  "lscr.io/linuxserver/jellyfin:latest"
  "deluan/navidrome:latest"
  "lscr.io/linuxserver/nextcloud"
  "filebrowser/filebrowser"
  "photoprism/photoprism"
  "lscr.io/linuxserver/syncthing:latest"
  "lscr.io/linuxserver/calibre-web:latest"
  "ghcr.io/kiwix/kiwix-serve:3.7.0"
  "lscr.io/linuxserver/heimdall:latest"
  "archivebox/archivebox"
  "lscr.io/linuxserver/freshrss:latest"
  "frooodle/s-pdf:latest"
  "ghcr.io/advplyr/audiobookshelf"
  "akhilrex/podgrab"
  "lscr.io/linuxserver/emulatorjs:latest"
  "grimsi/gameyfin:latest"
  "neethumohan1212/kolibri"
  "lscr.io/linuxserver/minetest:latest"
  "ghcr.io/openzim/zimit"
  "ghcr.io/usememos/memos:latest"
  "lscr.io/linuxserver/libreoffice:latest"
  "sigoden/dufs"
  "fallenbagel/jellyseerr:latest"
  "danielszabo99/microbin"
  "ghcr.io/shaarli/shaarli:latest"
  "vaultwarden/server:latest"
  "lscr.io/linuxserver/vscodium:latest"
  "ghcr.io/nagimov/agendav-docker:latest"
  "zefhemel/silverbullet"
)

# Get today's date in YYYYMMDD format
today=$(date +%Y%m%d)

# Create a temporary directory
mkdir -p temp_docker_images

# Pull and export each container
for container in "${containers[@]}"; do
  echo "Pulling container: $container"
  docker pull "$container"
  if [ $? -eq 0 ]; then
    echo "Successfully pulled $container"

    # Export the image to a tar file in the temporary directory
    docker save "$container" > "temp_docker_images/${container##*/}-${today}.tar"
    echo "Exported $container to temp_docker_images/${container##*/}-${today}.tar"

    # Remove the original image
    docker rmi "$container" 
  else
    echo "Error pulling $container"
  fi
done

# Compress all tar files in the temporary directory into a single tar.gz using pigz
pigz -p 4 temp_docker_images/*.tar -f docker_images_${today}.tar.gz 

# Remove the temporary directory
rm -rf temp_docker_images

echo "All container pull, export, and compression operations completed."

How to load a container

Getting it onto the new system is simple. Obviously, you are gonna need Docker installed and configured on the new system. Then, you can load the image onto the system like this:

docker load -i /path/to/file/filename.tar

Once this completes, you can run docker images and see it on the system. Now you can use like you would normally, either with docker run or docker compose files.

Doing this can obviously be scripted and automated with cron.

Android apps

Although this isn’t for the offline PC I’m bringing to my made up scenario, I imagine I will be bringing my mobile device, too. That means I will want access to Android apps I may not have installed or if I need to reload my phone, reinstall critical apps.

For this I am using fdroidcl to download apps from F-Droid, IzzyOnDroid, and a couple other repos and then saving them to a directory on my data drive for this project. I am running fdroidcl on my desktop to “prep” for this offgrid situation and then transferring.

We all know I am also running it and saving the output to my NAS 😉.

Installing and configuring fdroidcl

Installing is simple as it is in the apt repos. below is a copy/paste from a previous post where I described how I sideload apps on my ChromeOS device.

For the uninitiated, fdroidcl is a command line client for downloading apk’s from F-Droid and other repos mapped to the config. I use fdroidcl for creating a local archive of about 30 apps in my data hoard. I wrote a simple script that grabs the apk’s from seven various repos and saves to my NAS.

This is what my fdroidcl config looks like:

{
        "repos": [
                {
                        "id": "f-droid",
                        "url": "https://f-droid.org/repo",
                        "enabled": true
                },
                {
                        "id": "f-droid-archive",
                        "url": "https://f-droid.org/archive",
                        "enabled": false
                },
                {
                        "id": "IzzyonDroid",
                        "url": "https://apt.izzysoft.de/fdroid/repo",
                        "enabled": true
                },
                {
                        "id": "NewPipe",
                        "url": "https://archive.newpipe.net/fdroid/repo",
                        "enabled": true
                },
                {
                        "id": "DivestOS",
                        "url": "https://divestos.org/fdroid/official",
                        "enabled": true
                },
                {
                        "id": "GuardianProject",
                        "url": "https://guardianproject.info/fdroid/",
                        "enabled": false
                },
                {
                        "id": "FediLab",
                       "url": "https://fdroid.fedilab.app/repo",
                        "enabled": true
                },
                {
                        "id": "Guardian_Project",
                        "url": "https://guardianproject.info/fdroid/repo",
                        "enabled": true
                },
                {
                        "id": "Bitwarden",
                        "url": "https://mobileapp.bitwarden.com/fdroid/repo/",
                        "enabled": true
                }
        ]
}

I then created a little ugly script for downloading the apks I want to archive and then compress them.

#! /bin/bash

# Download apps
fdroidcl download \
    net.minetest.minetest \
    it.niedermann.owncloud.notes \
    com.nextcloud.client \
    com.nextcloud.talk2 \
    org.videolan.vlc \
    de.danoeh.antennapod \
    org.tasks \
    com.fsck.k9 \
    org.mozilla.fennec_fdroid \
    com.beemdevelopment.aegis \
    com.kunzisoft.keepass.libre \
    com.termux \
    com.github.catfriend1.syncthingandroid \
    org.moire.ultrasonic \
    com.amaze.filemanager \
    org.jellyfin.mobile \
    org.xbmc.kodi \
    danielmeek32.compass \
    at.tomtasche.reader \
    io.anuke.mindustry \
    com.limelight \
    org.ppsspp.ppsspp \
    d.d.meshenger \
    com.retroarch \
    org.ligi.survivalmanual \
    com.cylonid.nativealpha \
    org.localsend.localsend_app \
    com.apk.editor \
    ws.xsoh.etar \
    at.bitfire.davdroid \
    com.nononsenseapps.feeder \
    com.artifex.mupdf.viewer.app \
    net.gsantner.markor \
    ua.com.radiokot.photoprism \
    com.bobek.compass \
    com.coboltforge.dontmind.multivnc \
    sushi.hardcore.droidfs \
    fr.gouv.etalab.mastodon \
    com.iiordanov.freeaRDP \
    org.mian.gitnex \
    com.x8bit.bitwarden \
    com.example.flutter_http_server \
    com.foobnix.pro.pdf.reader


# Download additional APKs
cd ~/.cache/fdroidcl/apks &&
wget https://download.kiwix.org/release/kiwix-android/kiwix.apk &&
wget https://briarproject.org/apk/briar.apk &&
# Update this link before running
wget https://github.com/termux/termux-app/releases/download/v0.118.1/termux-app_v0.118.1+github-debug_arm64-v8a.apk &&
wget https://github.com/termux/termux-app/releases/download/v0.118.1/termux-app_v0.118.1+github-debug_armeabi-v7a.apk &&
wget https://github.com/termux/termux-x11/releases/download/nightly/app-arm64-v8a-debug.apk &&
# Update this link before running
wget https://dist.torproject.org/torbrowser/13.5.7/tor-browser-android-aarch64-13.5.7.apk &&
wget https://dist.torproject.org/torbrowser/13.5.7/tor-browser-android-armv7-13.5.7.apk

# Compress
tar  --use-compress-program="pigz -k " -cvf ~/tmp/fdroid_apks_$(date +%Y-%m-%d).tar.gz ~/.cache/fdroidcl/apks

Installing on Android

Once I have an archive of the apks, I just need to transfer them to the mobile device and tap the package to install (after giving permission to the file explorer app, of course).

The other option is to install onto the device by connecting it with a USB cable and then using adb. Don’t have adb installed? No worries! It is in our aptly repo.

The script above is my main script for grabbing the apks I want to archive, which includes several apps that need the internet to work (like Tor). They take up such little room, I left them in the script for my offgrid build, too.

Up next

Now that I have my hardware, the distro, and all my applications ready to go, my next post will finally be about the data I have archived.

- - - - -

Did you like this post? Give it an upvote by clicking on the arrows below! Sending me an upvote is like you and I giving each other a high five.

🙏 😎

Thank you for reading! If you would like to comment on this post you can start a conversation on the Fediverse. Message me on Mastodon at @cinimodev@masto.ctms.me. Or, you may email me at blog.discourse904@8alias.com. This is an intentionally masked email address that will be forwarded to the correct inbox.

If you enjoy the random stuff I write here, post to Mastodon, or watch on YouTube, and are feeling generous, I am open to tips of Ko-fi.