thegeeklab/content/posts/create-a-static-site-hosting-platform/index.md

title	date	authors	tags	resources
Create a static site hosting platform	2020-07-30T01:05:00+02:00	robert-kaussow	Automation Sysadmin	name src params feature images/feature.jpg anchor credits Center [AltumCode](https://unsplash.com/@altumcode) on [Unsplash](https://unsplash.com/s/photos/coding)

There are a lot of static site generators out there and users have a lot of possibilities to automate and continuously deploy static sites these days. Solutions like GitHub pages or Netlify are free to use and easy to set up, even a cheap webspace could work. If one of these services is sufficient for your use case you could stop reading at this point.

As I wanted to have more control over such a setup and because it might be fun I decided to create my own service. Before we look into the setup details, lets talk about some requirements:

• deploy multiple project documentation
• use git repository name as subdomain
• easy CI workflow

The required software stack is quite simple:

• Nginx as web server to deliver static files
• Minio S3 as files backend

Of course, Minio could be removed from the stack but after a few tests, my personal impression was that Minio is a way better to handle file sync and uploads instead over SSH/SCP, at least in a CI driven workflow. The whole workflow is nearly the same as for GitHub pages. Right beside your projects source code in the Git repository you will have a docs folder which contains the required files to build the documentation site. It's up to you what static site generator to use. Instead of using e.g. the gh-pages branch to publish the site, a CI system will build the documentation form the docs folder and publish it to a prepared Minio S3 bucket. The Nginx server will basically use the defined bucket as source directory and convert every sub-directory into something like https://<reponame>.mydocs.com.

As a first step, install Minio and Nginx on a server, I will not cover the basic setup in this guide. To simplify the setup I will use a single server for Minio and Nginx but it's also possible to split this into a Two-Tier architecture.

After the basic setup we need to create a Minio bucket e.g. mydocs using the Minio client command mc mb local/mydocs. To allow Nginx to access these bucket to deliver the pages without authentication we need to set a bucket policy mc policy set download local/mydocs. This policy will allow public read access. In theory, it should also be possible to add authentication headers to Nginx to server sites from private buckets but I have not tried that on my own.

Preparing the Minio bucket was the easy part, now we need to teach Nginx to rewrite the subdomains to sub-directories and deliver the sites properly. Let us assume we are still using mydocs as the base Minio bucket and mydocs.com as root domain. Here is how my current vHost configuration looks like:

{{< highlight nginx "linenos=table" >}} upstream backend_mydocs { server localhost:61000; }

server { listen 80; server_name ~^(www.)?(?(.+.)?mydocs.com)$;

return 301 https://$name$request_uri;

}

server { listen 443 ssl; server_name ~^((?.*).)mydocs.com$;

ssl_certificate [..];
ssl_certificate_key [..];
client_max_body_size 100M;

recursive_error_pages on;

location / {
    proxy_pass http://backend_mydocs/mydocs/${repo}${request_path};
    proxy_http_version 1.1;
    proxy_buffering off;
    proxy_connect_timeout 300;
    proxy_intercept_errors on;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    rewrite ^([^.]*[^/])$ $1/ permanent;
    error_page 404 = /404_backend.html;
}

location /404_backend.html {
    proxy_pass http://backend_mydocs/mydocs/${repo}/404.html;
    proxy_intercept_errors on;
}

} {{< /highlight >}}

We will go through this configuration to understand how it works.

Lines 1-3 defines a backend, in this case it's the Minio server running on localhost:61000.

Lines 5-10 should also be straight forward, this block will redirect HTTP to HTTPS.

Line 14 is where the magic starts. We are using a named regular expression to capture the first part of the subdomain and translate it into the bucket sub-directory. For a given URL like demoproject.mydocs.com Nginx will try to serve mydocs/demoproject from the Minio server. That's what Line 23 does. Some of you may notice that the used variable ${request_path} is not defined in the vHost configuration.

Right, we need to add another configuration snippet to the nginx.conf. But why do we need this variable at all? For me, that was the hardest part to solve. As the setup is using proxy_pass Nginx will not try to lookup index.html automatically. That's a problem because every folder will at least contain an index.html. In general, it's required to tell Nginx to rewrite the request URI to /index.html if the origin is a folder and ends with /. One way would be an if condition in the vHost configuration but such conditions are evil¹ in most cases and should be avoided if possible. Luckily there is a better option:

{{< highlight nginx "linenos=table" >}} map $request_uri $request_path { default $request_uri; ~/$ ${request_uri}index.html; } {{< /highlight >}}

Nginx maps are a solid way to create conditionals. In this example set $request_uri as input and $request_path as output. Each line between the braces is a condition. The first line will simply apply $request_uri to the output variable if no other condition match. The second condition applies ${request_uri}index.html to the output variable if the input variable ends with a slash (and therefor is a directory).

Line 38-41 of the vHost configuration tries to deliver the custom error page of your site and will fallback to the default Nginx error page.

We are done! Nginx should now be able to server your static sites from a sub-directory of the Minio source bucket. I'm using it since a few weeks and I'm really happy with the current setup.

---

1. This article from the Nginx team explains it very well. ↩︎