thegeeklab/content/posts/2020/create-a-static-site-hosting-platform/index.md

---
title: "Create a static site hosting platform"
date: 2020-07-30T01:05:00+02:00
aliases:
  - /posts/create-a-static-site-hosting-platform/
authors:
  - robert-kaussow
tags:
  - Automation
  - Sysadmin
resources:
  - name: feature
    src: "images/feature.jpg"
    params:
      anchor: Center
      credits: >
        [AltumCode](https://unsplash.com/@altumcode) on
        [Unsplash](https://unsplash.com/s/photos/coding)
---

There are a lot of static site generators out there and users have a lot of possibilities to automate and continuously deploy static sites these days. Solutions like GitHub pages or Netlify are free to use and easy to set up, even a cheap webspace could work. If one of these services is sufficient for your use case you could stop reading at this point.

<!--more-->

As I wanted to have more control over such a setup and because it might be fun I decided to create my own service. Before looking into the setup details, lets talk about some requirements:

- deploy multiple project documentation
- use git repository name as subdomain
- easy CI workflow

The required software stack is quite simple:

- Nginx as web server to deliver static files
- Minio S3 as files backend

Of course, Minio could be removed from the stack but after a few tests, my personal impression was that Minio is a way better to handle file sync and uploads instead over SSH/SCP, at least in a CI driven workflow. The whole workflow is nearly the same as for GitHub pages. Right beside your projects source code in the Git repository you will have a `docs` folder which contains the required files to build the documentation site. It's up to you what static site generator to use. Instead of using e.g. the `gh-pages` branch to publish the site, a CI system will build the documentation form the docs folder and publish it to a prepared Minio S3 bucket. The Nginx server will basically use the defined bucket as source directory and convert every sub-directory into something like `https://<reponame>.mydocs.com`.

As a first step, install Minio and Nginx on a server, I will not cover the basic setup in this guide. To simplify the setup I will use a single server for Minio and Nginx but it's also possible to split this into a Two-Tier architecture.

After the basic setup it's required to create a Minio bucket e.g. `mydocs` using the Minio client command `mc mb local/mydocs`. To allow Nginx to access these bucket to deliver the pages without authentication a bucket policy `mc policy set download local/mydocs` need to be set. This policy will allow public read access. In theory, it should also be possible to add authentication headers to Nginx to server sites from private buckets but I have not tried that on my own.

Preparing the Minio bucket was the easy part, now Nginx need to know how to rewrite the subdomains to sub-directories and properly deliver the sites. Let's assume `mydocs` is still used as the base Minio bucket and `mydocs.com` as root domain. Here is how my current vHost configuration looks like:

<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<!-- spellchecker-disable -->
{{< highlight nginx "linenos=table" >}}
upstream backend_mydocs {
    server localhost:61000;
}

server {
    listen       80;
    server_name  ~^(www\.)?(?<name>(.+\.)?mydocs\.com)$;

    return 301 https://$name$request_uri;
}

server {
    listen       443 ssl;
    server_name  ~^((?<repo>.*)\.)mydocs\.com$;

    ssl_certificate [..];
    ssl_certificate_key [..];
    client_max_body_size 100M;

    recursive_error_pages on;

    location / {
        proxy_pass http://backend_mydocs/mydocs/${repo}${request_path};
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_connect_timeout 300;
        proxy_intercept_errors on;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        rewrite ^([^.]*[^/])$ $1/ permanent;
        error_page 404 = /404_backend.html;
    }

    location /404_backend.html {
        proxy_pass http://backend_mydocs/mydocs/${repo}/404.html;
        proxy_intercept_errors on;
    }
}
{{< /highlight >}}
<!-- spellchecker-enable -->
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

We will go through this configuration to understand how it works.

**_Lines 1-3_** defines a backend, in this case it's the Minio server running on `localhost:61000`.

**_Lines 5-10_** should also be straight forward, this block will redirect HTTP to HTTPS.

**_Line 14_** is where the magic starts. A named regular expression is used to capture the first part of the subdomain and translate it into the bucket sub-directory. For a given URL like `demoproject.mydocs.com` Nginx will try to serve `mydocs/demoproject` from the Minio server. That's what **_Line 23_** does. Some of you may notice that the used variable `${request_path}` is not defined in the vHost configuration.

Right, another configuration snippet needs to be added to the `nginx.conf`. But why is this variable required at all? For me, that was the hardest part to solve. As the setup is using `proxy_pass` Nginx will _not_ try to lookup `index.html` automatically. That's a problem because every folder will at least contain an `index.html`. In general, it's required to tell Nginx to rewrite the request URI to `/index.html` if the origin is a folder and ends with `/`. One way would be an `if` condition in the vHost configuration but such conditions are evil[^if-is-evil] in most cases and should be avoided if possible. Luckily there is a better option:

<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<!-- spellchecker-disable -->
{{< highlight nginx "linenos=table" >}}
map $request_uri $request_path {
    default $request_uri;
    ~/$ ${request_uri}index.html;
}
{{< /highlight >}}
<!-- spellchecker-enable -->
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

[Nginx maps](https://nginx.org/en/docs/http/ngx_http_map_module.html) are a solid way to create conditionals. In this example set `$request_uri` as input and `$request_path` as output. Each line between the braces is a condition. The first line will simply apply `$request_uri` to the output variable if no other condition match. The second condition applies `${request_uri}index.html` to the output variable if the input variable ends with a slash (and therefor is a directory).

**_Line 38-41_** of the vHost configuration tries to deliver the custom error page of your site and will fallback to the default Nginx error page.

We are done! Nginx should now be able to server your static sites from a sub-directory of the Minio source bucket. I'm using it since a few weeks and I'm really happy with the current setup.

[^if-is-evil]: [This article](https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/) from the Nginx team explains it very well.