• Custom LUA HAProxy HealthCheck

    I’ve recently been tasked with using HAProxy to front a LDAP server in AWS to allow SSL access to the LDAP backend. This was essentially following this guide from AWS. It is a great shame that AWS don’t offer SSL termination in the Directory Service itself, but the HAProxy/ELB solution isn’t too complex and does work well.

    While testing out some failure scenarios I did however spot an issue which I didn’t like.

    There exists a situation whereby if (for whatever unlikely reason) an HAProxy instance cannot reach any of its downstream LDAP backends the ELB health check of the HAProxy instance will still pass and thus the instance will still be considered healthy. The ELB will still send traffic to the instance, but then things go south because the instance cannot see any LDAP backends and the request fails.

    I spent a bit of time googling for a solution and didn’t find anything which solved this, what we really want is for HAProxy to stop listening on the TCP port of the front end when all the backends are unavailable, however that isn’t an option.

    The solution I came up with was to use a custom HTTP health check endpoint, which runs a simple LUA script inside HAProxy. This uses the new LUA support, added around to HAProxy around version 1.6.1. Not all the binaries I found had it compiled in as an option so we ended up using the official Docker container.

    The way this works is you define a custom HTTP front end in your haproxy.cfg like this:

    global
        # Lots removed for brevity!
        lua-load /usr/local/etc/haproxy/haproxy-smart-tcp-healthcheck.lua

    Custom HTTP Health check endpoint

    frontend status-lua bind *:8000 mode http http-request use-service lua.status_service

    LDAP frontend

    frontend ldap_front bind *:1389 description LDAP Service option socket-stats option tcpka timeout client 5s default_backend aws-ldap

    Downstream AWS LDAP backends

    backend aws-ldap balance roundrobin server directory1 10.0.0.1:8081 check port 8081 server directory2 10.0.0.2:8081 check port 8081 option tcp-check

    
    This now runs the LUA service `tcp_healthcheck` defined in the file `haproxy-smart-tcp-healthcheck.lua` whenever a request is made for `/` on port `8000`.
    
    The script itself turned out to be pretty simple, once I got my head around some LUA quirks!
    
    All it does, is query the HAProxy state for all the servers associated with a given backend (named `aws-ldap` on line 15). Then we loop that list of backends, and if _any_ of them are `UP` we return a 200 status with the message `Found some servers up`, if they are all down we return a 500 status with the message `Found NO servers up`. 
    
    <pre class="wp-block-code"><code>-- This service checks all the servers in the named backend (see the 
    -- backend_name var). If _any_ of them are up, it returns 200 OK. If 
    -- they are all down it returns a 500 FAILED.
    -- 
    -- This is intended to be used as a HTTP health check from an upstream
    -- load balancer, without this check the most intelligent health check 
    -- that could be performed is a simple TCP check on the HAProxy frontend.
    -- This would not fail in the event that HAProxy cannot see *any* of its
    -- downstream servers
    
    core.register_service("tcp_healthcheck", "http", function (applet)
    
        -- Harcoded backend here, if anybody knows how to pass vars into Lua
        -- from the haproxy.cfg please shout!
        local backend_name = "aws-ldap"
        local r = ""
    
        backend = core.proxies[backend_name]
        servers = backend["servers"]
    
        local any_up = false
    
        for k, v in pairs(servers) do
    
            status = v.get_stats(v)["status"]
    
            -- If _any_ of the servers are up, we will return OK
            if (status == "UP") then
                any_up = true
            end
    
        end
    
        if ( any_up ) then
            core.log(core.debug, "Found some servers up")
            applet:set_status(200)
            r = "OK"
        else
            core.log(core.debug, "Found NO servers up")
            applet:set_status(500)
            r = "FAILED"
        end
    
        applet:start_response()
        applet:send(r)
    end)</code>
    

    With this all installed in the HAProxy instance it was a simple matter of configuring the ELB in AWS to use a HTTP health check for the path / on port 8000.

    The AWS Console config for the ELB Health check looks like this:

    And you can see the Listeners configuration is still in TCP passthrough mode, with TLS termination performed at the ELB.

    The one optimisation I’d like to make to this but have not yet found a solution for, is to pass the name of the backend to query through to the script as an argument. Currently it is hardcoded into the script which would mean duplicating the script if you need more than one of these custom health checks, that would be a pain. However in our use-case the HAProxy is only fronting a single LDAP directory so we can live with this single hardcoded piece fo configuration!


  • Importing non GoPro videos into GoPro Quik

    I’ve been using our GoPro quite a bit this summer which the kids have really enjoyed. I also recently got myself a DJI Spark, which is also great fun.

    However I really wanted to be able to import the Spark videos into the new GoPro Quik desktop app because it makes short work of turning a load of clips into short snappy videos.  Sadly it only offers to import videos originating from the GoPro.

    I figured there are only so many ways the software can check to determine if a given file was made by a GoPro or not, so I got to poking around and after about half an hour I found a solution which works on all my current DJI video files.

    If you run this ffmpeg command on any a DJI video (although it probably works for iPhone or other videos), it adds the magic field that Quik looks for, and then it happily imports the video.

    ffmpeg -i dji-video.mp4 
    -map_metadata
    -1
    -codec copy
    -metadata:s handler=“GoPro AVC encoder”
    gopro-video.mp4
    ```

    I know almost nothing about the millions of options ffmpeg accepts, so this is mostly copy and paste from various google searches, but it works fine from what I can tell!


  • Still alive!

    I’m still here, and so is a very much neglected blog. I will try and make amends in the near future!


  • Hb King Desert Fox Dead

    Spoke too soon!

    After the stripped hex and the first (!) smoking electronics incident I had hoped we’d have a period of trouble free use. Sadly this was not to be. After replacing the hex I drove the car slowly across the living room and all seemed well.

    The following weekend the lad and I went out to the woods to have a bash, but as he drove away the car stuttered and then stopped, within seconds of stopping smoke was again pouring out of the bodywork ūüôĀ. I managed to rip open the top of the body and get the LiPo disconnected to prevent any more damage, but that was clearly the end of that.

    Now I’m normally a fan of Hobby King, I almost exclusively use their batteries and servos in my Aircraft however this was one failure too far. I can forgive the stripped hex (although I’ve never seen that before) but two sets of electronics going up in smoke (literally!) with the second being a much higher spec than the first is too much for me to want to pour more good money after bad.

    It was decided to chuck it up in the loft and move onto something else. But what next I wonder?

    For reference, both times the car was running on a 2s LiPo, we never ran it on anything else.


  • Hb King Desert Fox Stripped Hex

    While we were on holiday the lad took the ‘Fox out onto the nearby beach and had a good bash around, sadly after a little while it seemed to be lacking punch when you pull the throttle which stopped play.

    When I got it back to the cottage and had a look into what might have caused this, I noticed the hex in one of the rear wheels had completely rounded off! I’ve never seen this before, but I guess given the price of the thing it is to be expected that things will fail. Thankfully, the spare wheel which I had assumed was just for decoration on the rear of the body, was in fact a full spare. I ordered a new set of drive hex’s, aluminium this time and swapped the damaged wheel for the spare. We were back on the road again!


  • Tiling, gah

    IMG_2478I don’t know quite why, but I decided to tile the horrible downstairs bathroom. I’ve never done tiling before so it was a case of learning on the job. Things I wish I’d considered before I started:

    1. Choosing a ‘brick’ pattern means more cutting, and more fiddling around at the corners.
    2. Choosing a bevel edge tile makes grouting much more effort.
    3. Choosing a really small room for your first attempt isn’t wise.
    4. Choosing a room with a pitched ceiling means lots of cutting awkward angles. See 1!

    Oh well, its done now. Onward!


  • Right, thats it then.

    So 2 months to the day of the operation I’m back at work today. I imagine my inbox will be verging on hell and I also imagine I’ll be breaking out the ctrl-a, delete solution.

    I’ve managed to crack on with a few projects over the break:

    • The Sand Scorcher is all but done, some paint around the windows and decals to do.
    • Nuala’s boat now has lights
    • Edwards boat is now running brushless, is water tight again and is blue.
    • The Fournier is finished.
    • The Taylor Craft is finished.
    • The Desert Fox burst into flames.
    • A Dominus 10TR arrived to replace it.
    • The downstairs loo is finished - although I only had a small bit of grouting to do.
    • The BlackJack now has a motor and ESC - Just need to solder on the connectors and fit the motor into the boat.
    • I’m about 75% through Doom on the PS4.
    • I’m about 50% through Drive Club on the PS4.
    • I’m about 30% through uncharted 4 on the PS4.
    • My aquarium¬†now has a temperature sensor connected to the Raspberry Pi Zero.
    • The blog has had a massive tidy up of pages.
    • I’ve written a load of update blog posts.
    • I’ve started on a Nigel Hawes Can-Do.
    • We’ve had the kitchen design done, the fitters arrive w/c 19th September.
    • We’ve picked the new living room furniture.
    • I’ve bought a new tele for the living room.

    Phew. I thought I was taking it easy, but that seems like a lot!


  • Weekend in Wiltshire

    In the last few weeks before I was due to go back to work we had a little family break, first we popped down to devon for 4 days, then another few in Wiltshire on the way back. The weather throughout the holiday was perfect with lots of clear sunny days. Whilst in Wiltshire I couldn’t resist having a little fly of the Dynamic-S, which I don’t think I’ve flown since 2015! It flew, as ever, brilliantly. There wasn’t enough wind to venture onto the slope, but on the flat field at the top of the White Horse hill I had a lot of fun while my daughter looked on.

    IMG_2408After waiting to patiently while I put a few packs through the Dynamic, we couldn’t resist putting a pack through our new RC car on the way back home. ¬†The Dominus worked well on the gravel track, with the Daughter slowly getting to grips with the steering!


  • Sea Nymph refirb

    So this turned out to be a bit of a saga.

    Ever since I finished my daughters boat, my son was a little verbal about how her boat was quicker than his. So I promised at some point when I had some time I’d refit it to make it quicker.

    Initially I built it with a 380 brushed motor, the plan being to make it slow enough for him to get used to the steering. Making it quicker wouldn’t be hard then, bung in a brushless motor!

    This was until he dropped it off its bloody stand onto a concrete path leading to a nice leak. Sigh.

    After finding the leak, epoxying it back together so it was watertight the hull looked a little sorry for itself so I decided to paint it, he chose blue this time, few coats of Halfords Acrylic and its looking good.

    I took the lazy way out with the motor, figuring if I could find a 380 sized brushless in-runner I could mount it in the existing mount, the only one I could find which fitted was the mount was a little one from Overlander.

    The new ESC, motor and gear went back in quite quickly, and a test run in the bath proved the max current at WOT was well below what the motor and ESC are rated at. One problem did manifest itself however, I have overpowered the thing so much that it practically emptied the bath and tried to make a break for orbit. ¬†I’ve wound the throttle travel down on the TX, and will try to find a prop with less pitch to slow it down a bit!


  • Sand Scorcher Progress

    While having some time off work one of the unfinished, long term projects I wanted to crack on with was painting the Sand Scorcher. I had filled, sanded and primed it a while back so I gave it a few coats of white before leaving it for a week.

    I pondered over masking the blue for quite some time, the curves had me worried. Initially I thought I could cut out some masks, in-fact that was the plan until quite by chance I stumbled across some Tamiya flexible masking tape while browsing in my local HobbyCraft. It turns out this is wonderful stuff, making short work of all the curves on the ‘box art’ scheme.

    Now its sitting on a shelf waiting for me to polish it up with some finishing compound and then apply the decals.