(This is a repost of this reddit post https://www.reddit.com/r/selfhosted/comments/1fbv41n/what_are_the_things_that_makes_a_selfhostable/, I wanna ask this here just in case folks in this community also have some thoughts about it)

What are the things that makes a selfhostable app/project project good? Maybe another way to phrase this question is, what are the things that makes a project easier to self-host?

I have been developing an application that focuses on being easy to selfhost. I have been looking around for existing and already good project such as paperless-ngx, Immich, etc.

From what I gather the most important thing are:

  • Good docs, this is probably the most important. The developer must document how to self-host
  • Less runtime dependency–I’m not sure about this one, but the less it depends on other services the better
  • Optional OIDC–I’m even less sure about this one, and I’m also not sure about implementing this feature on my own app as it’s difficult to develop. It seems that after reading this subreddit/community, I concluded that lots of people here prefer to separate identity/user pool and app service. This means running a separate service for authentication and authorization.

What do you think? Another question is, are there any more good project that can be used as a good example of selfhostable app?

Thank you


Some redditors responded on the post:

  • easy to install, try, and configure with sane defaults
  • availabiity of image on dockerhub
  • screenshots
  • good GUI

I also came across this comment from Hacker News lately, and I think about it a lot

https://news.ycombinator.com/item?id=40523806

This is what self-hosted software should be. An app, self-contained, (essentially) a single file with minimal dependencies.

Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services.

I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.

Do you agree with this?

  • Shimitar
    link
    fedilink
    English
    34 months ago

    Has docker compose file for deployment

    Can be hosted on sub-path and not only subdomain

    Can easily be integrated into SSO lime authelia

  • mel
    link
    fedilink
    English
    34 months ago

    I totally disagree with the quote from hackernews. Having the option to use sqlite is nice to test it, but going with postgresql or mariadb allows you to have better performance if you use rdbms. Also, packaging with containers allows to have one standardized image for support if some third party packaging (from a distro repo) is bugging to test it further. To me, a good gui really depends on what service is provided. For kanidm (IAM), I don’t care this much of a web admin panel, the cli is really intuitive and if you need some graph views of your users, you can generate some diagram files. Considering OIDC/LDAP, I’d rather have OIDC implemented for two reasons : I can point my users to the (really minimalist) kanidm ui where they have a button for each app allowed. Also, the login informations are only stored in kanidm, no spreading of login password.

    I saw a comment about not needing to rely on many third services but I partly disagree with it. Using nextcloud as a mixed example, using elastic search for full text search is better than reimplementing it, but the notify_push should not be as separated as it is (it is here because I understood, apache-php and websockets does not mix well).

    All in all, the main criterias for me are :

    • SSO with OIDC, but ldap is good enough
    • Good documentation
    • easy deployment to test, prod deployment can be more advanced
    • Not reimplement the weel eg if you need full text search, meilisearch or elastic can do it better than you will, so don’t try to much (a simple grep for a test instance is enough)
    • If you need to store files, having remote stores is nice to have (webdav or s3)
  • @T156@lemmy.world
    link
    fedilink
    English
    84 months ago

    Ease of installation/use, I think, is the main big one, and one of the biggest obstacles.

    People who want to give self-hosting a try aren’t going to be particularly fond of having to jump through a whole bunch of different configs, and manually set everything up.

    They want something that they can just set up and go, without having to deal with server hosting, services, and all of that. Something you can just run on your computer, leave it be, and use it with relatively little fuss.


    Second to that, would definitely be a case of better documentation/screenshots. A lot of self-hosted things, like Lemmy, didn’t provide much documentation of what the actual user side of it does, only what you need to do to set it up, which isn’t going to make me want to use the software, if I have no idea what it’s supposed to do, and how it compares to other things that do the same.

  • 𝘋𝘪𝘳𝘬
    link
    fedilink
    English
    74 months ago

    To me the number one thing is, that it is easy to setup via Docker. One container, one network (ideally no network but just using the default one), one storage volume, no additional manual configuration when composing the container.

    No, I don’t want a second container for a database. No I don’t want to set up multiple networks. Yes, I already have a reverse proxy doing the routing and certificates. No, I don’t need 3 volumes for just one application.

    Please just don’t clutter my environment.

    • @hono4kami@slrpnk.netOP
      link
      fedilink
      English
      64 months ago

      No, I don’t want a second container for a database.

      Unless you’re talking about using SQLite:

      Isn’t the point of Docker container is to only have one software/process running? I’m sure you can use something like s6 or other lightweight supervisor, but I feel like that’s seems counterintuitive?

      • 𝘋𝘪𝘳𝘬
        link
        fedilink
        English
        24 months ago

        To me, the point of Docker is having one container for one specific application. And I see the database as part of the application. As well as all other things needed to run that application.

        Since we’re here, lets take Lemmy for example. It wants 6 different containers with a total of 7 different volumes (and I need to manually download and edit multiple files before even touching anything Docker-related).

        In the end I have lemmy, lemmy-ui, pictrs, postgres, postfix-relay, and an additional reverse proxy for one single application (Lemmy). I do not want or need or use any of the containers for anything else except Lemmy.

        There are a lot of other applications that want me to install a database container, a reverse proxy, and the actual application container, where I will never ever need, or want, or use any of the additional containers for anything else except this one application.

        So in the end I have a dozen of containers and the same amount of volumes just to run 2-3 applications, causing a metric shit-ton of maintenance effort and update time.

        • @traches@sh.itjust.works
          link
          fedilink
          English
          14 months ago

          I can see why editing config files is annoying, but why exactly are two services and volumes in a docker-compose file any more difficult to manage than one?

          • 𝘋𝘪𝘳𝘬
            link
            fedilink
            English
            14 months ago

            See it in a broader scope. If I’d only host Lemmy with is multiple mandatory things, I couldn’t care less, but I already have some other applications that I run via Docker. Fortunately I was able to keep the footprint small, no multiple containers or volumes for one application, but as said: those exist. And they would clutter the setup and make it harder to maintain an manage.

            I also stand by my point that it is counter-intuitive to have multiple containers and volumes for just one single application.

            • @traches@sh.itjust.works
              link
              fedilink
              English
              14 months ago

              Ok but is there room for the idea that your intuitions are incorrect? Plenty of things in the world are counter-intuitive. ‘docker-compose up -d’ works the same whether it’s one container or fifty.

              Computer resources are measured in bits and clock cycles, not the number of containers and volumes. It’s entirely possible (even likely) that an all-in-one container will be more resource-heavy than the same services split across multiple containers. Logging from an all-in-one will be a jumbled mess, troubleshooting issues or making changes will be annoying, it’s worse in every way except the length of output from ‘docker ps’

              • 𝘋𝘪𝘳𝘬
                link
                fedilink
                English
                14 months ago

                docker ps or Portainer as a nice web-UI wrapper around the Docker commands are the two main use cases with Docker I have have on a regular basis.

                No, thank you. I am not going to maintain fifty containers and fifty + X volumes for just a handful of applications and will alway prefer self-contained applications over applications that spread over multiple containers for no real reason.

        • @Zeoic@lemmy.world
          link
          fedilink
          English
          24 months ago

          I agree with this. If you are going to be using multiple containers for a single app anyways, what is the point of it being in multiple containers? Stick all of it in one container and save everyone the hassle.

          • @schizo@forum.uncomfortable.business
            link
            fedilink
            English
            14 months ago

            It’s because of updates and who owns the support.

            The postgres project makes the postgres container, the pict-rs project makes the pict-rs container, and so on.

            When you make a monolithic container you’re now responsible for keeping your shit and everyone else’s updated, patched, and secured.

            I don’t blame any dev for not wanting to own all that mess, and thus, you end up with seperate containers for each service.

    • ÚwÙ-Passwort
      link
      fedilink
      English
      24 months ago

      I prefer this, but if the options are available its shows me that soneone actually thought about it while creating the software/conatiner

    • @silmarine@discuss.tchncs.de
      link
      fedilink
      English
      14 months ago

      I came here to basically say this. It’s especially bad when you aren’t even sure if you want to keep the service and are just testing it out. If I already have to go through a huge setup/troubleshooting process just to test the app, then I’m not feeling very good about it.

    • @traches@sh.itjust.works
      link
      fedilink
      English
      74 months ago

      I disagree with pretty much all of this, you are trading maintainability and security for easy setup. Providing a docker-compose file accomplishes the same thing without the sacrifice

      • separate volumes for configuration, data, and cache because I might want to put them in different places and use different backup strategies. Config and db on SSD, large data on spinning rust, for example.
      • separate container for the database because the official database images are guaranteed to be better maintained than whatever every random project includes in their image
      • separate networks because putting your reverse proxy on a different network from your database is just prudent
  • @bandwidthcrisis@lemmy.world
    link
    fedilink
    English
    104 months ago

    Before even getting to documentation, I see so many projects that don’t have a short summary of what they do (and maybe what to not expect them to do).

    As an example, Home Assistant. I can tell that it involves home automation, so can I replace Google Home with it? It seems like it doesn’t do voice recognition without add-ons and it can work with Google Assistant. Do I still need accounts with the providers of smart appliances, or can it control my bulbs directly?

    None of that is very clear from the website.

    I’ve seen plenty of other projects where it’s assumed there’s no need to explain it’s overall purpose.

  • @tatterdemalion@programming.dev
    link
    fedilink
    English
    4
    edit-2
    4 months ago
    • Has a simple backup and migration workflow. I recently had to backup and migrate a MediaWiki database. It was pretty smooth but not as simple as it could be. If your data model is spread across RDBMS and file, you need to provide a CLI tool that does the export/import.

    • Easy to run as a systemd service. This is the main criteria for whether it will be easy to create a NixOS module.

    • Has health endpoints for monitoring.

    • Has an admin web UI that surfaces important configuration info.

    • If there are external service dependencies like postgres or redis, then there needs to be a wealth of documentation on how those integrations work. Provide infrastructure as code examples! IME systemd and NixOS modules are very capable of deploying these kinds of distributed systems.

  • @Prunebutt@slrpnk.net
    link
    fedilink
    English
    44 months ago

    Please be mindful of HDD spindown.

    If your app frequently looks up stuff in a database and also has a bunch of files that are accessed on-demand, then please have an option to separate the data-directory from the appdata-directory.

    A lot of stuff is self-hosted in homes and not everyone has the luxury of a dedicated server room.

    • @hono4kami@slrpnk.netOP
      link
      fedilink
      English
      14 months ago

      separate the data-directory from the appdata-directory

      Would you mind explaining more about this?

      • @Prunebutt@slrpnk.net
        link
        fedilink
        English
        34 months ago

        Take my setup for jellyfin as an example: There’s a database located on the SSD and there’s my media library located on an HDD array. The HDD is only spun up when jellyfin wants to access a media file.

        In my previous setup, the nextcloud database was located on a HDD, which resulted in the HDD never spinning down, even if the actual files are never really accessed.

        In immich, I wasn’t able to find out if they have this separation, which is very annoying.

        All this is moot, if you simply offer a tiny service which doesn’t access big files that aren’t stored on SSDs.

        • @sugar_in_your_tea@sh.itjust.works
          link
          fedilink
          English
          24 months ago

          Exactly. Separate configuration and metadata from data. If the metadata DB is relatively small, I’ll stick it on my SSD and backup to my HDD on a schedule.

  • @Passerby6497@lemmy.world
    link
    fedilink
    English
    34 months ago

    Configuration by config file is preferred but not mandatory for me, but a docker image is mandatory for me to even try the app anymore. And the ability to backup and restore state is key, preferably in such a way that I can write my backup to a mounted smb share rather than writing locally and copying to the network.

    I’m running everything on commodity or 2nd hand gear, so failures aren’t unheard of. I had one of my micro PCs cook itself this year, and the majority of my services on that box fit that mold (mostly), so I got them back up pretty quick. Though, I did run into issues with container backups not working (because they write the backup like a database, so it has to be a local write for a db lock) and had to start from scratch .

  • @thelittleblackbird@lemmy.world
    link
    fedilink
    English
    64 months ago

    My points are totally in the other direction:

    • stable, this is critic, if the app is not able to performs its duties with. 2 weeks uptime, then it is bad. This also applies to random failures. I don’t want to spend endless days to fix it
    • docker, with a all-in-image, and as a nice to have the possibility to connect external docker composes for vpn, or databases
    • a moderate use of resources, not super critic, but nobody likes to have ram problems

    And then as a second league that lean the balance:

    • integration with LDAP or any central user repo
    • relatively easy to backup and restore
    • relatively low level of break changes from version to version
    • the gui / ease of use (in like with the complexity of the problem I want to address)
    • sane use of defaults and logging capabilities

    That’s all from my side

  • @sugar_in_your_tea@sh.itjust.works
    link
    fedilink
    English
    20
    edit-2
    4 months ago

    Not something so complex that it requires docker.

    I disagree. Docker makes things a lot easier and I’m going to use it regardless.

    My rule is pretty simple: not PHP. PHP requires configuring a web server, so either that’s embedded in the docker image, (violates the “do one thing” rule of docker) or it’s pushed onto the user. This falls under the dependencies part, but I uniquely hate dealing with standalone web servers and I don’t mind configuring databases, so I called it out.

    I actually tried switching to OCIS from Nextcloud specifically to avoid PHP, but OCIS is even more complex so I bailed.

    Give me an example configuration that works out of the box and detailed documentation about options and I’ll be happy. Don’t make me configure a web server any particular way, and do let me handle TLS myself. If you do that, I’ll probably check it out.

  • drkt
    link
    fedilink
    English
    24 months ago

    I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.

    Do you agree with this?

    Yes. If I have to spend an hour reading your documentation just to find out how to run the damn thing, I’m not going to run it.

    I hate docker with a passion because it seems like the people who develop on it forego writing documentation because docker “just works” except when it doesn’t.

    I archived one of my github repos the other day because someone requested I add docker support. It’s a project I made specifically to not use docker…

  • Matt
    link
    fedilink
    English
    10
    edit-2
    4 months ago

    Not something so complex that it requires a docker

    Docker is the thing that sandboxes your services from the host OS. I’d rather use Podman because of the true non-root mode, but Docker is still based. Plus, you can use Docker Swarm if you don’t want to switch to Kubernetes (though you don’t have easy storage integration for persistence).

    • @daddy32@lemmy.world
      link
      fedilink
      English
      24 months ago

      Docker is also the thing that allows the distribution of the app as “single file with minimal dependencies”.

      • @ancoraunamoka@lemmy.dbzer0.com
        link
        fedilink
        English
        14 months ago

        Not a single app, not minimal dependencies. It’s a file that gets processed and creates many gigabites of leftovers, with an enormous runtime and piles of abstractions

    • @jabjoe@feddit.uk
      link
      fedilink
      English
      14 months ago

      The problem is when Docker is used to gift wrap a mess. Then there are rotting dependencies in the containers. The nice thing about Debian packaged things is the maintainer is forced to do things properly. Even more so if they get it into the repos.

      My preference is Debian Stable in LXC or even KVM for services. I only go for Docker if that is the recommended option. There is stuff out there where the recommend way is their VM image which is full of their soup of Dockers.

      Docker is in my pile of technologies I don’t really like or approve of, but don’t have the energy to really fight.

  • @vividspecter@lemm.ee
    link
    fedilink
    English
    54 months ago

    Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services.

    This comment is a bit silly. Databases just make sense for many services, although many could just use sqlite which would be fine (and many do). Redis etc is usually optional and might increase performance in some cases.

    I wouldn’t be a fan of something requiring docker, but it’s often just the easiest way to deploy these types of services (and the easiest way to install it as a user).

    Anyway, I’ll echo that clear, up-to-date documentation is nice. I shouldn’t have to search through actual code or the bug/issues section to find current information (but I get this is very challenging). And I’d rather projects didn’t make Discord a source of documentation (especially not the primary one).

    I’ll add that having a NixOS module is a big plus for me, but I don’t expect the developers themselves to maintain this.

    • @ancoraunamoka@lemmy.dbzer0.com
      link
      fedilink
      English
      14 months ago

      Unless you are a business with millions of users, I would be really skeptical of redis improving performance. At the end of the day, any simple KV store can replace it. It’s not like postgres where having sqlite in it’s place means not only a different performance profile, but also different semantics, sql dialect, rpc protocol, etc