Source code for my blog/gemlog. It used to be on gopher, hence the name
git clone
Log | Files | Refs

commit 58e28b99f2f8416cbd0c6dd8f5720e9cb4bba621
parent a86617089f5157f65dc2135fd5fdd89670c69bb2
Author: Charlie Stanton <>
Date:   Tue, 17 Nov 2020 18:23:02 +0000

Adds all_the_protocols

Mconfig.json | 10++++++++++
Aposts/ | 24++++++++++++++++++++++++
Aposts/ | 122+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Aunpublished_posts/ | 3+++
Aunpublished_posts/ | 251+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 410 insertions(+), 0 deletions(-)

diff --git a/config.json b/config.json @@ -1,6 +1,16 @@ { "posts": [ { + "title": "All the protocols", + "file": "all_the_protocols", + "date": "2020-11-17 18:20:28" + }, + { + "title": "How this site is built", + "file": "build_process", + "date": "2020-08-21 08:51:10" + }, + { "title": "Reviving ex in 2020", "file": "ex", "date": "2020-08-05 15:27:08" diff --git a/posts/ b/posts/ @@ -0,0 +1,24 @@ +There are nowhere near as many protocols as there should be. When I'm writing this there are 8944 RFCs +in the IETF RFC index, which is pitiful for how much the internet is doing. +There aren't enough RFCs because obviously no company producing a proprietary protocol is going to +make it available to the world with an RFC, but open source developers largely don't seem to be +bothering either. + +If you are making an open source project that communicates over the internet to do something, for example +a whiteboard for groups collaberating on something, then either use an existing RFC or make a new one. +Otherwise someone else will come along with their open source client that does the same thing, and the +two won't be able to communicate. Open source protocols are pointless if they aren't standardised. + +RFCs can seem scary, but I've read quite a few now and they're not that bad. Having an +RFC for your protocol also helps contributors to your project learn how it works, which is great since +documentation is also far too often lacking in open source projects. + +I once set myself a goal of submitting code to something I actually use and getting it accepted, and I +eventually got round to having a very small contribution to vim. I'm now going to work on contributing +to an RFC. Not necessarily one I use, just any of them. + +Hopefully this rant doesn't make me a hypocrite as someone who's never been to an IETF meeting, but +I'll fix that soon enough, and I'd encourage you to do the same. + +Feel free to email with any thoughts. Writing super long and well researched +posts is hard so I'm gonna try these short ones with the occasional longer one. diff --git a/posts/ b/posts/ @@ -0,0 +1,122 @@ +Obviously most sites in 2020 are enourmous beasts, running megabytes of +javascript on both the client and server. This one isn't, and I thought I'd take +some time to explain how it does work. I want to write an article critical of +the web as a whole at some point so I haven't put any reasoning for why it works +this way in this post. + +### This is no ordinary site + +This is both a blog and a phlog. All of the posts are available on both a +website and a gopher server, both at If you haven't heard of +gopher, go have a look! I hadn't until recently but it is a lot of fun. I got +started +[here]( + +### Server Side: HTTP + +There is basically nothing on the http server. It is a rust executable using +actix that serves files from a directory. All the pages on this site are just +html files. The only interesting thing it does is redirect / to /index.html. + +### Server Side: Gopher + +Even simpler, I didn't need a gopher server library since the protocol is +wonderfully simple. Again a rust executable is serving static files, and / +serves a file called index. + +### Server Hosting + +These are hosted on a debian buster server with vultr. I wrote a couple of +systemd unit files for my executables and they run 24/7 and autorestart should +they fail. + +### Client Side + +The phlog (gopher log) just serves the raw markdown files that I write, but the +blog has some additional styling. A tiny bit of css which uses css-grid to +layout the pages and make them responsive, as well as gruvbox colours. + +### Directory structure + +The project root has: + +- `config.json` - list of posts with metadata +- `gopher` - built gopher files +- `gopher_src` - scripts and templates for building gopher files +- `html` - built html files +- `html_src` - scripts and templates for building html files +- `posts` - post markdown files +- `gopher-server` - rust gopher server +- `web-server` - rust web server +- `` - publish everything +- `Tupfile` - tupfile (see later) + +### Build process + +I've hinted that I write markdown posts but obviously they are published as +html. To generate all the pages for the site I've used a bunch of python scripts +connected together with tup, which I hadn't used before but I quite like now. So +far there are 4 python scripts: + +- `gopher_src/` which generates the gopher index +- `html_src/` which generates index.html +- `html_src/` which generates the nav menu snippet of html (not a + whole page) from a json list of posts. +- `html_src/` which generates the post pages. + +These are all pretty simple but `html_src/` is the most complex and +interesting so I'll break that down briefly. + + #!/usr/bin/python3.8 + import argparse, json, sys, os, subprocess, shutil + + parser = argparse.ArgumentParser(description="Generate html for post from markdown") + parser.add_argument("post", help="Post markdown file name") + args = parser.parse_args() + + with open(os.path.join("posts",".md"), encoding="utf-8") as post_file, open("config.json", encoding="utf-8") as config_file, open("html_src/post_template.html", encoding="utf-8") as template_file, open("html_src/nav.html", encoding="utf-8") as nav_file: + config = json.load(config_file) + post_metadata = next((p for p in config["posts"] if p["file"] ==, None) + if not post_metadata: + print("Post not found") + sys.exit(1) + for line in template_file: + if line == "{content}\n": + sys.stdout.flush() +["markdown"], stdin=post_file, stdout=sys.stdout) + sys.stdout.flush() + elif line == "{header}\n": + sys.stdout.write(""" + <h2>{title}</h2> + <em>{date}</em> + """.format(title=post_metadata["title"], date=post_metadata["date"])) + elif line == "{nav}\n": + shutil.copyfileobj(nav_file, sys.stdout) + else: + sys.stdout.write(line) + +This isn't a python tutorial so I'll be quick. It takes an argument with the +name of the post (not including the extension). It finds the metadata for the +post in the json file, then reads the template line by line looking for +substitution points which is substitutes accordingly. + +The `Tupfile` that runs all of these scripts is: + + : foreach html_src/*.css |> cp %f %o |> html/%B.css + : |> ./html_src/ > %o |> html_src/nav.html + : foreach posts/*.md | html_src/nav.html |> ./html_src/ %B > %o |> html/%B.html + : foreach posts/*.md |> cp %f %o |> gopher/%b + : | html_src/nav.html |> ./html_src/ > %o |> html/index.html + : |> ./gopher_src/ > %o |> gopher/index + +Finally I have a `` that runs `tup` and then `rsync` to copy all the +updates to the server. + +The git repo with all this stuff in can be accessed using +`git clone git://`. + +### Conclusion + +I was pretty pleased with myself once I got this working so thought I'd write +this, but it'll probably improve further. Maybe I'll make this a series. As +always my email is diff --git a/unpublished_posts/ b/unpublished_posts/ @@ -0,0 +1,3 @@ +Operating systems are already compiled, but they could be more compiled. +It would be possible to have the whole source code for an OS in one place, managed by the user, +and installing software is actually diff --git a/unpublished_posts/ b/unpublished_posts/ @@ -0,0 +1,251 @@ +I'm not sure where the term "the small internet" came from, but I've decided I like it. +It describes an alternative approach to that of the web. More focused on usefulness to individuals +and less on profitability for corporations running it. + +### What exactly is the small internet? + +I don't know. I've tried a few terms on a few search engines but haven't had a whole +lot of success. So I'm gonna define what I think it should be: + +- Everything on the gopherspace +- Everything in the gemini project +- A few things on the web + +Ooh, that last one's probably controversial! But not all of the web protocols are that bad. + +This is inspired by +[this post]( +which also suggests that some of the web is in fact, small enough. + +### I'm no expert + +I've done a bit of web development and a lot of web using, but I'm certainly +not qualified to be writing a post like this. I will overlook things and get +things wrong, but hopefully the idea is still useful. + +### Which parts of the web are ok? + +I'm gonna split up a website into 4 parts: HTML, CSS, JS and media. +Each of these have different amounts which I consider acceptable. + +### JS + +This is the simplest one, there's no excuse for running a script on a webpage. +If for some reason you need to run code on the users computer, they should +download the code and run it. This gives them a lot more control over what is +run on their computer and native programs tend to be of much higher quality than +'web apps'. + +### Media + +Slightly more involved, I think a good approach here is to have all media be download links, +but allow a browser to substitute that link with an inline viewer if that's what the +user wants. Unless configured otherwise, all pages will load only text, but +media can be added should the user want to view it. This means users with fast +connections can use the web similarly to now, but if not you're still fine. + +Implementing this doesn't mean removing everything except text from your site. +It means being aware that a lot of people don't computers or internet access as +fast as yours, and sites should be designed with that in mind. + +### HTML + +This is where it gets tricky, according to +[this page]( +there are 141 different HTML elements. Obviously HTML is a lot more than just +the elements but I'll just focus on these for now. + +### Definitely necessary + +- `<html>` +- `<base>` +- `<head>` +- `<link>` +- `<meta>` +- `<style>` +- `<title>` +- `<body>` +- `<h1>`, `<h2>`, `<h3>`, `<h4>`, `<h5>`, `<h6>` +- `<div>` +- `<span>` +- `<li>` +- `<ol>` +- `<p>` +- `<pre>` +- `<ul>` +- `<a>` +- `<br>` +- `<code>` +- `<em>` +- `<s>` +- `<strong>` + +### Probably useful + +- `<address>` +- `<article>` +- `<aside>` +- `<footer>` +- `<header>` +- `<hgroup>` +- `<main>` +- `<nav>` +- `<section>` +- `<blockquote>` +- `<dd>` +- `<dl>` +- `<dt>` +- `<figcaption>` +- `<figure>` +- `<hr>` +- `<main>` +- `<bdi>` +- `<bdo>` +- `<cite>` +- `<q>` +- `<sub>` +- `<sup>` +- `<caption>` +- `<col>` +- `<colgroup>` +- `<table>` +- `<tbody>` +- `<td>` +- `<tfoot>` +- `<th>` +- `<thead>` +- `<tr>` +- `<button>` +- `<datalist>` +- `<fieldset>` +- `<form>` +- `<input>` +- `<optgroup>` +- `<option>` +- `<select>` +- `<textarea>` + +### Maybe worthwhile + +- `<abbr>` +- `<b>` +- `<data>` +- `<dfn>` +- `<i>` +- `<kbd>` +- `<mark>` +- `<rb>` +- `<rp>` +- `<rt>` +- `<rtc>` +- `<ruby>` +- `<samp>` +- `<small>` +- `<time>` +- `<u>` +- `<var>` +- `<wbr>` +- `<del>` +- `<ins>` +- `<meter>` +- `<legend>` +- `<label>` + +### Probably unneeded + +- `<area>` +- `<audio>` +- `<img>` +- `<map>` +- `<track>` +- `<video>` +- `<embed>` +- `<picture>` +- `<source>` +- `<output>` +- `<progress>` +- `<details>` +- `<summary>` + +### Definitely awful + +- `<iframe>` +- `<object>` +- `<param>` +- `<canvas>` +- `<noscript>` +- `<script>` +- `<dialog>` +- `<menu>` +- `<slot>` +- `<template>` + +### Deprecated + +- `<acronym>` +- `<applet>` +- `<basefont>` +- `<bgsound>` +- `<big>` +- `<blink>` +- `<center>` +- `<command>` +- `<content>` +- `<dir>` +- `<element>` +- `<font>` +- `<frame>` +- `<frameset>` +- `<image>` +- `<isindex>` +- `<keygen>` +- `<listing>` +- `<marquee>` +- `<menuitem>` +- `<multicol>` +- `<nextid>` +- `<nobr>` +- `<noembed>` +- `<noframes>` +- `<plaintext>` +- `<shadow>` +- `<spacer>` +- `<strike>` +- `<tt>` +- `<xmp>` + +### Why is finding a good subset of HTML so hard? + +I think a large part of the issue is that the roles of HTML and CSS overlap a +bit too much. Lots of these HTML elements are just aesthetic changes and could +be done with CSS instead. You could use HTML tables to layout your site, or you +could use flexbox or css-grid or something else. Because css-grid and HTML +tables do very similar jobs its hard to justify keeping both of them, so which +should go? + +I'm a bit of a fan of css-grid, so I'd be in favour of keeping that. I think +it's quite an elegant way of laying out a page, so I use it on this site. + +### + +### CSS + +Of the 4 parts I have the least experience with CSS, but I reckon we could get +rid of almost all of it. I won't do another exhaustive list because I wouldn't +know what most of the items were, but here are some thoughts: + +### Quite nice + +css-grid +colours +vw, vh, px +@media + +### Awful + +animations +transitions + +### Stuff used on this site + +