Year of Hope
18th — Zombies. I enjoy zombie stories. Is that odd? I’ve watched every Walking Dead episode and really liked the World War Z movie starring Brad Pitt. I’ve listened to all the Mountain Man books on Audible and I’m currently reading the excellent Surviving The Evacuation 18 book series by Frank Tayell. So if you’re worried about Zombies, read the Zombie Survival Manual
10th — So I’ve hit a stumbling block on my web project. After doing some significant XCA testing today, I’ve identified that patients with extensive medical histories can push beyond the 1 MB limit on Lambda functions when they are used as ALB targets. It’s not too much of a problem for the history page as I’ve already come up with an optimisation strategy to keep the response body below 1 MB.
The main problem is that some PDF’s are in the order of 7 MB big. Too large to be used as a direct response from the Lambda function. To get around this, I’ll be learning tomorrow how to upload large documents to S3, generating a pre-signed URL with an expiration date/time to restrict access.
9th — As part of a migration project for HiPRES, I needed to start moving reference data to the new system/database. I’ve wrote up the process I used to export the PostgreSQL data to a CSV file.
3rd — Spent an hour drawing this picture titled “First Dance”.
2nd — Spent a nice couple of hours drawing this colourful owl. Drawn using Derwent coloursoft pencils; versatile, professional-quality coloring pencils in a range of vibrant colors. Someone from Twitter asked me to draw a version of this for their newly built bar/shed. My first commission, yeah 🎨.
The image above is displayed using a different method than the other images on this page. It’s using the picture element to present the most optimised image based on what your browser can support. It’s also using the native browser support for lazy image loading.
The image also shows a solid background colour as a fallback placeholder if the images fail to load. On devices with slow internet connectivity, this should present an image placeholder with a meaningful colour until the image loads. This is useful to prevent against cumulative layout shift which is considered bad for front-end performance.
I’m actually in the process of writing a command-line tool to make this easier for developers. Given the source input image as a JPG file, it will create .WEBP and .AVIF file formats, picking the five most prominent colours from the photograph to use as the background colour fill. It will then generate the necessary HTML code snippet to use within your web page. I’ll be writing a complete tutorial very soon.
24th — Someone I follow on twitter likes to share three positives from their day. I’ve found it useful to sit quietly and contemplate at the end of day, reflecting on what has been positive. Reframing the days negatives an to focus on those positives. Here are mine for today:
- Attacked the ironing mountain.
- Had a lovely BBQ with the family.
- Sat outside, soaking up the evening sun while reading a great novel and drinking a glass of Jam Shed red wine.
I use curl, if not daily, certainly weekly. This is an interesting interview with the maintainer of the project as curl turns 23 years old.
17th — I’ve just finished writing a short story : Agent 13. A story I started writing for my son’s 13th birthday. Following in the tradition of the story I wrote for his Twelfth birthday. Kira Blackwood is Agent 13. Hired to track down Thomas who has just completed the heist of the decade. A heist that he hoped would make him very rich.
16th — Wow; time is flying by, not posted in a while. Now that the Covid restrictions have been lifted a bit and up to six people can meet outdoors, I met up with some fellow brass band players and had a practice session. It was really nice being able to make music with other people.
I’ve been making good progress with the Go web application and I’ve been compiling a list of notes about using Lambda functions for web applications. I’ll publish these soon under a title ‘Leveraging Lambda’. In the meantime I’ve published a short guide to AWS tagging best practices and how to automate the tagging using the AWS CLI
Here are the latest SCC stats for the web project. I think it’s time to show these to the boss.
29th — I’ve been busy coding a new Go web application over the last three weeks. So far I’ve wrote almost five thousand lines of code; and according to scc the estimated effort to reproduce is 5.9 months at a cost of over £81,000.
19th — This week I had to interface with an STS (Security Token Service) server to obtain a SAML assertion for our regional Health Information Exchange. The SOAP message used to submit to the STS server needed to contain a wsse:Security header. For this I had to encode the password digest based on combining a nonce with some other key data items. I’ll be writing this up in a future post to show how I managed to do this using Golang.
12th — It’s been a couple of weeks since I last posted. I’ve been busy working on a new web application. The application is going to be made up entirely of Lambda functions triggered from an ALB (Application Load Balancer). So I’ve been learning quite a lot recently. I’m going to publish my learnings soon so I’ve got it fully documented for my future self.
I also had to quickly learn how to make my Go HTTP clients proxy aware. Turns out it wasn’t too difficult. Just a couple of extra lines of code.
21st — I’ve had a week off work and it’s been amazing. Today I learned a great tip for sending the output of a terminal command to the linux clipboard, so that the contents can be pasted into my code editor. First you need to install the xclip command then you can use the syntax go run main.go | xclip -selection c to capture the output.
13th — Found Litestream today. It’s a backup utility which streams SQLite changes to S3-compatible storage. This allows you to quickly recover to the point of failure if your server goes down. It makes a bold claim. Stop building slow, complex, fragile software systems. Safely run your application on a single server.
There are many benefits to running a start-up business on a single server. Beside the costs savings, simplifying your infrastructure to a single server reduces your time to recovery and makes it easier to move your application to a different provider. You can scale up the VM as your business grows, and many business won’t reach the scale that requires multiple servers.
But this SQLite, nobody writes production applications with SQLite, right?
Well it can be done. David Crawshaw has a conference talk (worth watching) and blog post on building single process applications on SQLite.
With Litestream continually backing up your SQLite database, it’s certainly worth investigating. I’d like to try it on a high traffic data API.
12th — I love stumbling across sites like Old Book Illustrations and reviewing old drawings from a time gone by. Some of the drawings under the technical section would make great retro covers for tech books.
11th — I’ve been taking it easy this last week. The internet blocking measures I put in place for my son have now been applied to myself. This is to stop me working outside of my contracted hours. It was hard for the first couple of days, but I’m now finding it liberating. I’m also practicing saying “no”. The extra time is allowing me to read for pleasure, which I’m enjoying again.
Today, I created a quick demo video of using Audio Recorder on Linux to capture the system audio on a Ubuntu system.
8th — (❄️) Over the weekend I came to the realisation that I’m struggling with my current situation. I’ve been neglecting my own well being. I’m sad, low, exhausted and demotivated all due to burn out. The last twelve months has been unrelenting pressure with high stress workloads. The last three months have been even worse, I’ve been consistently working fifty to sixty hours a week, sacrificing many weekends to deliver projects. Something has to change. I can’t continue like this. I’ve been suffering headaches and stomach pains, all stress related.
My imposter syndrome feelings have increased over the last several months and isn’t helping the situation. Doing some research I can certainly identify with this trait:
The expert: These individuals are always trying to learn more and are never satisfied with their level of understanding. Even though they are often highly skilled, they underrate their own expertise.
So this weekend I pressed the reset button.
I took afternoon naps to recover the body, and made time for some hobbies that I have neglected. I played the cornet, read a book for pleasure, and did some pencil sketching:
I doubt it will be a quick recovery, but I’ve made the start. I’ll be sticking to a strict thirty seven hour week and working hard to using my weekends as downtime to recover. I’ve also scheduled in some annual leave.
5th — It’s been a busy week at work. Started learning Terraform to provision cloud infrastructure as code. I’ll be revisiting some older manual build projects to redefine them as terraform scripts.
Worked with the RDS data API to allow supplier access to an Aurora serverless database using the AWS command line utility. This allowed me to keep the RDS server private and grant secure access to the supplier using their IAM access keys.
30st — (🚶♂️) Out for a late afternoon walk and managed to capture this amazing sunset.
28th — Published first draft of Writing a Windows Service in Go.
27th — Have been busy working with suppliers to design and build AWS infrastructure to support reporting from our regional HIE (Health Information Exchange). We’ll be using Logstash to centralize, transform & stash application logs and ATNA data into an AWS Elastic Search cluster. We used this tutorial on how to install and configure Logstash.
Found a good video which outlines best practices checklist for using Amazon Elasticsearch service.
21st — (🚶♂️🎺) Starting to see improvement in my cornet playing; the consistency in practice is starting to pay off. Need to keep it up now, ready for a time when we’re allowed to meet as a band again. The new season of The Infinite Monkey Cage, podcast has been released. Always funny and informative. Finished watching Bridgerton on NetFlix; a fun look at London high society in the 1800’s - Very enjoyable watch.
19th — Published Sky router configuration, how to automatically disable the internet on a schedule.
18th — (🚶♂️🎺) Started a two week break from work. Going to spend the time walking (weather permitting) and practicing the cornet. I’m also going to use the time to learn some new skills, in particular Terraform.
16th — Published barcode scanning, a look at how to create a SPA (Single Page Application) to barcode receipt / reject laboratory samples at scale.
14th — Published Amazon Rekognition. Using the Go Rekognition API to OCR text from an image and to identify subjects within a photograph. Looking at how to set this up so that I can index all my photos within my photo albums. If I use something like bleve modern text indexing in Go, I’ll be able to search for photos based on their content. Like search for all photos contain Castles.
13th — (🎺) Spent a couple of hours investigating the features of AWS Rekognition for image identification. Doing some preparation for an upcoming longer blog post. Using the image below the system identified the contents as: Bird, Animal, Jay, Finch, Blue Jay with various levels of confidence ranging from 90.49 - 99.89
11th — (🎺) Most of the day was spent trawling through CSV files to ascertain why the files hadn’t been processed by our results ingestion service. In every instance the files either contained invalid or corrupted data 🤷♂️. On the plus side I learned some new grep skills.
10th — (🚶♂️🎺) Spent some quality time practicing the cornet. Went through several of Jean-Baptiste’s Arban Cornet Method studies, paying particular attention to tone and quality of sound. I really need to focus on getting the most out of each practice session. I particularly like the Andante con spirito on page 106.
Updated the family computer to OSX Big Sur and now the machine is really slow. We’ve had the computer for a while now and it’s probably time to try doing a fresh install of the operating system. A project for next weekend.
9th — Cold, but sunny. Time for some self care:
7th — (❄️) Used Excelize to produce exception reports for a project I’m working on. It was quite easy to set-up with a very intuitive API. Jotted down some PostgreSQL Tips. Started listening to The Cipher podcast.
5th — (🎺) Spent the afternoon debugging an issue with a Windows NT service I’d written in Go. Evaluated possible Go packages for generating Excel spreadsheets and found Excelize. The country moves into a third national lock down due to Covid19.
4th — First official day back at work. Lethal black ice, worse I’ve ever seen. Started work building a new cloud micro-service. Configured two CentOS servers and installed Docker onto them. Provisioned a load balancer and SSL certificate. Learned how to monitor and grep the Docker logs for exceptions; so I can now set-up a cron job to monitor the logs every fifteen minutes and alert first line support if exceptions go over a threshold.
3rd — (🚶♂️🎺) Jotted down some notes on how to manipulate images from the Linux command line. Downloaded and installed Krita a professional free and open source painting program made by artists for artists. Started teaching my ten year old how to use it, as he has a flare for drawing.
2nd — Switched the favicon for this site to an embedded base64 encoded image. Removing the need to make an additional HTTP request. Added a preconnect header tag, to speed-up the loading of Google fonts. Added the contact form which used the static forms service.
1st — (🚶♂️🎺) Wrote some proof of concept code to submit an image to AWS’s Rekognition machine learning API to extract batch number & expiry dates from the packaging. The AI is very good id the image is the correct way up, less so if the image is rotated either 90 or 180 degrees. It’s promising. Need to test in different lighting conditions and if the packaging is not straight.
Spent an hour or so working on a new short story, Agent 13.
Started listening to The Black Tapes Podcast; a serialized docudrama about one journalist’s search for truth, her subject’s mysterious past, and the literal and figurative ghosts that haunt them both. Do you believe?