As what as you like

  • Home networking like a pro - Part 1 - Network Storage

    Introduction

    This is part one in a series of posts about some hardware I recommend (or otherwise!) for people who want to bring some semi-professional flair to their home network.

    The first topic is storage - specifically, Network Attached Storage.

    Background

    For the last few years, I was running a Mac Mini with two 3TB drives in a RAID1 array in a LaCie 2big Thunderbolt chassis (US UK), with the Mac running macOS Server to provide file sharing (AFP and SMB), and Time Machine backups for the rest of the network.

    This was a very nice solution, in that the Mac was a regular computer, so I could make it do whatever I wanted, but it did have the drawbacks that the Thunderbolt chassis only had two drive bays, and I had trouble getting the Mac to run reliably for months at a time (I ran into GPU related kernel panics, perhaps because it was attached to a TV rather than a monitor).

    Around the time I was selecting the Mac/LaCie, most NAS devices in a similar price range were very underpowered ARM devices, and could do little more than share files, but in 2017 almost all NAS devices are much more powerful x86 devices that often have extensive featuresets (e.g. running containers, VMs, hardware accelerated video transcoding, etc.) so I decided it was time to switch.

    Solution

    I ended up choosing a Synology DS916+ (US UK), popped one of the 3TB drives (Western Digital Red (US UK)) out of the LaCie and into the Synology, and set about migrating my data over. I then moved the other drive, and put in two more 3TB drives, all of which are running as a single Synology Hybrid RAID volume with a BTRFS filesystem (note that the Hybrid RAID really seems to just be RAID5).

    I configured the Synology to serve files over both AFP and SMB, and enabled its support for Time Machine via AFP. I was also able to connect both of its Ethernet ports to my switch (a ZyXEL GS1900 24 port switch, which I will cover in an upcoming post) and enabled LACP on each end to bond the two connections into a single 2Gb link.

    So, how did it work out?

    The AFP file sharing is great, and works flawlessly. SMB is a little more complex, because recent versions of macOS tend to enforce encryption on SMB connections, which makes them go much slower, but this can be disabled. I tested Time Machine over SMB, which is officially supported by Synology, but is a very recent addition, and it proved to be unreliable, so that is staying on AFP for now.

    Something I was particularly keen on, with the Synology, was that it has an “app store” and one of the available applications is Docker. I was running a few UNIX daemons on the Mac Mini which I wanted to keep, and Docker containers would be perfect for them, however, I discovered that the version of Docker provided by Synology is pretty old and I ran into a strange bug that would cause dockerd to consume all available CPU cycles.

    For now, the containers are running on an Intel NUC (which will also be covered in an upcoming post) and the Synology is focussed on file sharing.

    Open Source

    Synology’s NAS products are based on Linux, Samba, netatalk and a variety of other Open Source projects, with their custom management GUI on top. They do publish source, but it’s usually a little slow to arrive on the site, and it’s not particularly easy (or in some cases even possible) to rebuild in a way that lets you actively customise the device.

    Conclusion

    Overall, I like the Synology, but I think if I’d known about the Docker issue, I might have built my own machine and put something like FreeNAS on it. More work, less support, but more flexibility.

    The recent 5-8 drive Synologies now support running VMs, which seems like a very interesting prospect, since it ought to isolate you from Synology’s choices of software versions.

  • Hyper Key in macOS Sierra with Karabiner Elements

    Over the last few years, various people have used Karabiner to remap Caps Lock to cmd+shift+opt+ctrl, which is such an unusual combination of modifier keys, that it effectively makes Caps behave as a completely new modifier (which we have collectively called “Hyper”, in reference to old UNIX workstation keyboards).

    And for a time, it was good.

    Then came macOS Sierra, which changed enough of the input layers of its kernel, that Karabiner was unable to function. Thankfully, Karabiner’s author, Fumihiko Takayama, began work on a complete rewrite of Karabiner, which is currently called Karabiner Elements.

    Initially, Elements only supported very simple keyboard modifications - you could swap one key for another, but that was it. Various folk quickly got to work offering quick hacks to get a Hyper key to work, and others started to try to work around the missing support, with other tools.

    I’m very glad to say that it is now possible to do a proper Hyper remap with Karabiner Elements (and to be clear, none of this is my work, all credit goes to Fumihiko).

    Here’s how you can get it:

    • Download and install https://pqrs.org/latest/karabiner-elements-latest.dmg
    • Launch the Karabiner Elements app, go to the Misc tab and check which version you have, if it’s less than 0.91.1, click either Check for updates or Check for beta updates until you get offered 0.91.1 or higher, then install that update and re-launch the Karabiner Elements app.
    • You probably want to remove the example entry in the Simple Modifications tab.
    • Edit ~/.config/karabiner/karabiner.json
    • Find the simple_modifications section, and right after it, paste in:
    "complex_modifications": {
        "rules": [
            {
                "manipulators": [
                    {
                        "description": "Change caps_lock to command+control+option+shift.",
                        "from": {
                            "key_code": "caps_lock",
                            "modifiers": {
                                "optional": [
                                    "any"
                                ]
                            }
                        },
                        "to": [
                            {
                                "key_code": "left_shift",
                                "modifiers": [
                                    "left_command",
                                    "left_control",
                                    "left_option"
                                ]
                            }
                        ],
                        "type": "basic"
                    }
                ]
            }
        ]
    },
    
    • As soon as you save the file, Elements will notice it has changed, and reload its config. You should immediately have a working Hyper key 😁

    If you’re not confident at your ability to hand-merge JSON like this, and don’t need anything from Elements other than the basic defaults, plus Hyper, feel free to grab my config and drop it in ~/.config/karabiner/.

    Supplemental note for High Sierra

    I’ve only tested this very briefly on High Sierra, but I had to disable SIP to get the Elements .kext to load. I’m not quite sure what’s going on, but I reported it on GitHub. (Note that you can re-enable SIP after the kext has been loaded successfully once)

    Update

    Many people like to turn Caps into Hyper, but also have it behave as Escape if it is tapped on its own. As of Karabiner Elements 0.91.3 this appears to be possible by adding this to the manipulator:

    "to_if_alone": [
        {
            "key_code": "escape",
            "modifiers": {
                "optional": [
                    "any"
                ]
            }
        }
    ],
    

    (thanks to Brett Terpstra for the sample of this

  • New blog setup

    Bye bye Blogger, hello GitHub Pages. This means no more comments, and probably a bunch of posts have terrible formatting at the moment, but at least my blog is now just static Markdown files I can edit nicely 😁

    (Also the domain has changed, cmsj.net is the new tenshu.net :)

  • Changing GPG key

    It’s been 15 years and my GPG key is now looking hilariously out of date - 1024bit DSA key. Yuck. So, time to start over. Below is a statement about the change, with details of my new and old keys, signed by both keys. Since it will almost certainly not paste properly out of a browser, I have also uploaded it to GitHub: here.

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    - -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA256
    
    Hello world
    
    My name is Chris Jones and I am changing my GPG key.
    The original key fingerprint is: 6C99 9021 9B3A EC6D 4A28  7EE7 C574 7646 7313 2D75
    The new key fingerprint is: 79D2 2E89 6591 210E 45F3  75D3 BCA2 36E2 E19F 727D
    
    You should find this message signed by the new key, and that combined message signed by the old key, as proof that I am doing this. I have also updated my keybase.io account and secured a couple of signatures on the new key to start rebuilding my place in the web of trust.
    
    Thank you for your time,
    
    Chris
    - -----BEGIN PGP SIGNATURE-----
    Version: GnuPG/MacGPG2 v2.0
    
    iQIcBAEBCAAGBQJXxffoAAoJECwUb1eIgw1qUMEP+wfaB84vYC4tHdo9OS9szaQv
    cmNW0sIuDx/RQr4iYrcs+8QTQf2I3FPXUc6auhyF4J7Lntu67sTKI9zyfsDm0IW9
    NbIzysTP1Y35lJPA12VM9O9IRaf5G7J57BKAmAuUbpnY7icIzA0MoD4SwCgcRXtA
    QPZd8JVPDLaNpwb1O5rdQLBAdo+OUjF+bB8jZzzoORX0oVVdGhaGJFVhuq8aV1dY
    0YB/nZr71ZApUVnvSZBfj1FHsgXZ5Fai70iI+oAox/Gj6BJ0IJhBIk5hLAzAiXoR
    8EMmtqkWkKc8Jd3NonMzKRGF+qT+G3YuZIDmSptOZWjJ5volT25bYFknEBxPslwC
    TiBGSs6rKz9RhfxGxCmM350zBIVFFCn+RNCWrgn/z4OJp4xSvZJ0IwqB/CwkrmUb
    0ZhG44O2W5lpuSCDh1dhCsiryq4JeSiUy1GENyHl8eIkXjzDOjKTt6OT8wYFFPL7
    XheyIfvMbRNh86o79Skch6Qoyh7nvVALAwsLVHKSDtQRzHbVF6ED9h2ISxdABiZ2
    CkiJ95bf8JQeNVoqLJ78uwSYN96AyGPXfMQKG45SavgkzNLyqeoI1iMJE2yYuVIy
    Z9XUaKhoDI9ERLps+Fw6NY+v2BVSKTvl5MDEDHmfvjK0m3x6C4tF3/QdgwRsJmvE
    2XMWXLrBkcZx9tnYNFIG
    =9PwA
    - -----END PGP SIGNATURE-----
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG/MacGPG2 v2.0
    
    iQEcBAEBCAAGBQJXxff1AAoJEPH//8xn2jUBNOcIAL4fqY/DViwNj2/Va4ePEg1w
    9uUWhGnMUcG4CsQGkhtODU6Qg45inrWnG0VE8jwhGilP4w5tQoIe+m53cUp5m/Rv
    ueRBvfDxBw/SrF6eFZ1SGkXv6kcUkOYjueKsDtxObaX9dN7PrDUljtZWpGTzE77k
    5EWPGfUT89oXa2eGwYnr6t7t9f76cO9eKFck7rIWT+p1tzmF6amm7IjoS8gsjfSb
    lPk3PoC0G71wSseh7iesIgw+vZRZ7tYg59RdpwWLmZjQJVhMzW/QpX87CPAM2m0A
    OcyivXLlbaSZ58AsHZSIA4ZjeoDnlWNsFHemBUSAOMa03b4JtgnGbTaHTZhiLc8=
    =/lkZ
    -----END PGP SIGNATURE-----
    
  • Raspberry Pi project: PiBus

    The premise of this project is simple - my family lives in London, there’s a bus route that runs along our road, and we use it a lot to take the kids places. The reality of using the bus is a little more complex - getting three kids ready to go out, is a nightmare and it’s not made any easier by checking a travel app on your phone to see how close the bus is.

    I don’t think there is much I can do to make the preparation of the children any less manic, but I can certainly do something about the visibility of information about buses, mainly thanks to the excellent open APIs/data provided by Transport For London. So, armed with their API, a Python 3 interpreter and a Raspberry Pi, I set out to make a little box for the kitchen wall which will show when the next 3 buses are due to arrive outside our house. The code itself is easy enough to throw together because Python has libraries for everything (it also helps if you don’t bother to design a decent architecture!). Requests to fetch the bus data from TfL, json/iso8601 to parse the data, Pillow to render it as an image, and APScheduler to give it a simple run-loop. The question then becomes, how to display the data. The easiest answer would be a little LCD screen, but that brings with it the downside of having a backlight in the kitchen, which would be ugly and distracting, and it also raises the question of viewing angles. Another answer would be some kind of physical indicator, but that requires skills I don’t have time for. Instead, I decided to look for an E-Ink display (think Kindle) - it would let me display simple images without producing light. The first option I looked at was the PaPiRus, but it’s in the window between its crowdfunding drive having finished, and being available to buy. The only other option I could find was the E-Paper HAT, from Percheron Electronics, which also started life as a crowdfunding project, but is actually available to buy. Unfortunately, these displays are super fragile, which I discovered by destroying the first one, but Neil at Percheron was super helpful and I quickly had a new display and some tips about how to avoid cracking it. My visualisation of this data isn’t going to win any awards for beauty, but it serves its purpose by showing a big number to tell us how many minutes we have, and I managed to minimise the number of times you see the white-black-white refresh cycle of the eInk display with partial screen updates. Here are some photos of the project in various stages of construction: | | |————————————————————————————————————————————————————————————————————————————| | | | Freshly assembled out of the box |

     
    The smallest USB WiFi adapter I’ve ever seen
    Sadly I had to make some modifications to the
    PiBow case to fit this particular rPi
     
    Running one of the eInk display test programs

     Initially I was rather hoping I could use the famous font that TfL (and London Transport before it) use, which is known as Johnston, but sadly they will not licence the font outside their own use and use by contracted partners. There is a third party clone of the font, but it may have legal issues, presumably because TfL values their braaaaaand. Instead, I decided to just drop the idea of shipping a font with the code, and exported Courier.ttf from my laptop to the Pi directly.

     
    This would have been nice, but I cannot have nice font things.

    I did briefly try Ubuntu Mono, which is a lovely font, but the zeros look like eyes and it freaked me out.

     
    PIBUS IS WATCHING YOU

    The display needs to handle various different situations, most obviously, when no data can be fetched from the API. Rather than get too bogged down in the details of whether our Internet connection is down, TfL’s API servers are down, London is on fire, or it’s just night time and there are no buses, I went for a simple message with a timestamp. Once this has been displayed, the code skips any further screen updates until it has valid data again. This makes it easy to see when a problem occurred.

    Maybe aliens stole the Internet, maybe it's a bus strike.
    It doesn't matter.

    I also render a small timestamp on valid data screens too, showing when the last data fetch happened. This is mostly so I can be sure that the fetching code isn’t stuck somehow. Once I trust the system a bit more, this can probably come out.

     
    The final design, showing a fallback for when there is data for  0 < x < 3 buses
     
    Data for three buses, plenty of time to get ready for the second one!

    So there it is, project completed! Grab the code from https://github.com/cmsj/pibus, install the requirements on a Pi, give money to the awesome Percheron Electronics for the E-Paper HAT (and matching PiBow case), throw a font in the directory and edit the scripts for the bus stop and bus route that you care about!