Besides being Queen’s drummer and writer of such hits as ‘Radio Ga Ga’ and ‘A Kind Of Magic’, Roger Taylor has released seven albums under his own name. While his solo songs might not be as well known to the average punter as those of bandmates Freddie Mercury (‘Living On My Own’, ‘Barcelona’) or Brian May (‘Driven By You’, ‘Too Much Love Will Kill You’), one could make the case that his oeuvre is the strongest of the three. He was the first in the band to release solo material, beginning with the single ‘I Wanna Testify’ in 1977 and then the album Fun In Space in 1981.

Taylor’s latest album, Violence Insane in a Beautiful World, is the impetus for a short tour of the UK, which made a stop at Edinburgh’s Usher Hall last night (22 September). The show opened with a banger in ‘Strange Frontier’, one of his classic 80s tracks, bleakly anthemic and imbued with Cold War angst (‘Some day soon they’ll drop the big one / No more Dad and no more Mum’). This was followed by ‘Nation of Haircuts’, a satire of the superficiality of late-90s Britain (‘We don’t make ships and we don’t make cars, but we look real good hanging out in bars’). A strong start.

Taylor has always been a great singer, and he remains in fine voice, most evident last night on quieter numbers such as his lovely cover of John Lennon’s song ‘Jealous Guy’ (accompanied on piano by multi-instrumentalist Tina Keys). Some of the higher notes on older tunes like ‘I’m In Love With My Car’ and ‘Man On Fire’ might not be quite so readily attainable these days, but it would be unfair to expect any singer to have the same range as they did 40 or 50 years ago.

The new songs really shine live. ‘Violence Insane’ is the darkest thing he’s released since ‘Nazis 1994’, and it hit hard live, with its ominous pulsing beat, swirling synths and darkly distorted vocals. Its opposite number is ‘A Beautiful World’, featuring the Ndlovu Youth Choir from South Africa, written from the point of view of an alien looking down on Earth. ‘Don’t fuck it up,’ Taylor pleads. He took to the drum kit during this song and peppered fills over the gently shuffling groove.

Taylor has never shied away from penning songs about subjects and figures that irk him. Two numbers in this category (about ‘a couple of c**ts’) were featured in the show: ‘Dear Mr Murdoch’, originally from 1994, and ‘Chump’, from Violence Insane…. (Can you guess who the latter might be about?) To lift the mood, he followed with ‘Under Pressure’, the 1981 collaboration between Queen and David Bowie.

The other Queen tracks in the set list were ‘These Are The Days Of Our Lives’, ‘Rock It (Prime Jive)’ (sung by drummer Tyler Warren), ‘A Kind Of Magic’, and, to close, ‘Radio Ga Ga’. And from his solo catalogue: ‘Surrender’ (featuring a lovely vocal duet with Tina Keys), ‘I See You Now’, ‘Absolutely Anything’, ‘Don’t Photograph Food’, and ‘Come On Summer (It’s Party Time)’ (also featuring the South African choir, who were not present at the gig, but who will appear on stage at the London show on the 29th). Besides ‘Jealous Guy’, the evening’s other cover was Led Zeppelin’s ‘Rock and Roll’.

In conclusion, it would be fair to call his first show in Edinburgh since 1999 a success. The sell-out audience was really into it, and Taylor and his band (which also includes Spike Edney on keyboards, Christian Mendoza on guitar and Neil Fairclough on bass) were on form and looked to be having fun.

It’s great to see that, after a career spanning 50+ years, Taylor is as creative and up for performing as ever. He doesn’t seem to have lost any of the drive and enthusiasm that sustained him during the early, lean years of Queen, when he scraped a living running a second-hand clothes stall with Freddie Mercury in London’s Kensington Market. He doesn’t need the money now, obviously, so it’s all about the music. And he might not be done yet: in an interview with Billboard he said he hasn’t ruled out another album at some point in the future. Nice! Keep on rockin’, Rog.

Full set list

1. Strange Frontier (Strange Frontier, 1984)
2. Nation of Haircuts (Electric Fire, 1998)
3. Gangsters Are Running This World [non-Purple version] (Outsider, 2021)
4. Surrender (Electric Fire )
5. I See You Now (Violence Insane in a Beautiful World, 2026)
6. I’m In Love With My Car (Queen, A Night At The Opera, 1975)
7. Absolutely Anything (Outsider)
8. Violence Insane (Violence Insane…)
9. Don’t Photograph Food (Violence Insane…)
10. These Are The Days Of Our Lives (Queen, Innuendo, 1991)
11. Rock It (Prime Jive) (Queen, The Game, 1980)
12. Jealous Guy (John Lennon, Imagine, 1971)
13. Dear Mr Murdoch (Happiness?, 1994; 2011)
14. Chump (Violence Insane…)
15. Under Pressure (Queen, Hot Space, 1982)
16. Man on Fire (Strange Frontier)
17. Rock and Roll (Led Zeppelin, Led Zeppelin IV, 1971)
18. A Kind Of Magic (Queen, A Kind Of Magic, 1986)
19. Encore: A Beautiful World (Violence Insane…)
20. Encore: Come On Summer (It’s Party Time) (Violence Insane…)
21. Encore: Radio Ga Ga (Queen, The Works, 1984)

Related posts

• Roger Taylor 2026 tour ticket tracker
• Hand-written lyrics for lost Freddie Mercury songs up for auction

* I continued counting until an hour or so before each gig started and the ticket map went offline. Resales tickets count as sold.
** All 514 seated tickets (Level 2) were sold for the London date. Standing area tickets are not countable on the Ticketmaster website. As of 22 September, you could only buy resale tickets, which indicated all floor tickets had been sold. However, on the 25th, standard (non-resale) tickets were available again. It seems venue capacity is in the region of 1,800. (Gigs with Ivan has a write-up of the venue in concert setup. He mentions gig capacity being unclear, but thinks it’s 1,800.).

Other note: all VIP tickets were sold in all venues. Some may be available via resale – it’s not something I tracked.

Filming

It looks like the London concert was filmed. Videographers CreoKinetics posted on Instagram on 30 September that they’d been ‘providing and operating 5x fixed rig cameras and 1x 4K PTZ onstage […], alongside a crew of front of house cameras.’ We got a live album (The Outsider Tour Live, 2022) out of the previous tour. Some kind of live audio-visual release for the Violence Insane… era would be just braw!

Related post: Roger Taylor rocks the Usher Hall.

Lyrics to unknown songs

There are plenty of lyrics that I can’t attribute to existing songs, including a few with a working title at the top of the page. Such named songs include ‘Tripping Love’, ‘I Can’t Sleep at Nights’, ‘Don’t Give Up’ and ‘The River of Lasting Love[?s]’. These could in theory be early versions of other songs. At first glance, it’s tempting to flag ‘Tripping Love’ as an early version of ‘Mr Bad Guy’, as it includes the line ‘I’m walking rainbows in the sky’, similar to ‘Let’s go chasing rainbows in the sky’ from the album’s title track; and the lyrics on the opposing page certainly are for an early or alternative version of ‘Mr Bad Guy’. However, ‘Tripping Love’, underlined at the top of the page, is clearly marked as the song’s title, and the lyrics don’t scan with ‘Mr Bad Guy’ if you sing the familiar melody in your head. Perhaps it’s a track being written in parallel with ‘Mr Bad Guy’ and the lines about rainbows were being tried out in both.

The Solo Collection (2000) was about as comprehensive as it was possible to be in terms of presenting available Freddie-related outtakes and alternative versions, so we have to assume no demos of these mystery tracks are in the official archive. Could some demos exist as home recordings? Mary Austin, Freddie’s longtime close personal friend to whom he left half of his estate, notes in her recent book Freddie Mercury: A Life in Lyrics (2026, HarperCollins) that a number of cassette tapes, ‘the kind Freddie had used to record himself when he was writing’, were found in his piano stool in 2023 while readying his possessions for auction¹. No information is given as to what, if anything, is on these cassettes. (There does exist a leaked home demo of unreleased Mr Bad Guy track ‘New York’).

When composing a song, Freddie tended to come up with the music first before focusing on the lyrics later², so I have to imagine melodies and chords would have existed for these song drafts, even if only in Freddie’s head. It goes without saying that it would be nice to hear any demo recordings he made of them.

Interesting track listings

Also in amongst the items in the collection are a couple of track listings. One (on the right in the image above) is shorter and divided in the middle with a row of ‘X’s, suggesting a two-sided album running order. Besides tracks which ended up on the album or as B-sides, there are demos which appeared in the Freddie Solo Collection box: ‘Money Can’t Buy Happiness’, ‘Gazelle’ and ‘God is Heavy’. ‘You Are The Only One’ is noted in Queen: Complete Works (Georg Purvis, updated edition, 2018, Titan Books) as being from the Kind of Magic sessions (1985–6)³. I guess we can now say the song started life a bit earlier. ‘Whipping Boy’ may be a Queen track started during the Hot Space sessions in 1981–2 (Purvis). ‘Piece of My Heart’ will be what became ‘Let Me Live’ on Queen’s Made In Heaven (released 1995). ‘Killing Time’ may be a Freddie version of Queen drummer Roger Taylor’s solo track of the same name from Strange Frontier (1984). I have nae idea about ‘Press On (Presto)’, ‘New Piano Song’, ‘One Night Stand’, ‘A Desperate Man’, ‘Make it On My Own’ and ‘Flesh is on the Bone’.

‘Killing Time’ is interesting: it appears in both lists, with a tick next to it in the first, and a question mark in the second⁴. Judging from the track names, the second listing is from a later date, suggesting the track was in the running for inclusion on the album for at least some period of time. Does this mean a complete Freddie solo version exists? I don’t know, but it would be amazing to hear. Freddie did provide backing vocals on Roger’s version (Purvis) – check out the bit from 3:11. If this song was also worked on by Queen during sessions for The Works, it could well turn up on any future Works box set. There was a lot of cross-pollination between Freddie, Roger and Queen at the time, and any interesting outtakes or alternative versions would likely be reserved for a Queen archival project⁵.

Who plays what on Mr Bad Guy

Finally, there is a listing of who plays what on the Mr Bad Guy tracks, giving us much more information than the album liner notes do. The three-page document, entitled ‘Musicians’ and dated 14 February 1985 (ten weeks before the album’s release on 29 April), describes who plays what instruments and on which tracks⁶. Roger Taylor is down as having played drums on ‘Man Made Paradise’, ‘Made in Heaven’ and ‘There Must Be More To Life Than This’, as well as drum fills on ‘Let’s Turn It On’. I don’t believe this has been confirmed before now (Roger mentions in the The Great Pretender documentary (2012) that he played on the album, without specifying in what capacity, only mentioning Freddie asking him to come to Munich to add some backing vocals). The drummer credited in the liner notes, Curt Cress, is listed under ‘Love Me Like There’s No Tomorrow’, ‘Mr Bad Guy’, ‘She Blows Hot and Cold’ and ‘My Love is Dangerous’, as well as for drum fills on ‘Your Kind of Lover’. It’s of course entirely possible that, like Brian May’s contribution to ‘She Blows Hot and Cold’, Roger’s parts were subsequently replaced by the musician who appears in the official credits⁷.

Random tidbits

• A random factoid known to the more dedicated of fans is that Freddie was approached to audition for a part in Puccini’s opera La bohème opposite Linda Ronstadt. The offer letter is in the auction and dated 13 July 1984. It notes, in a rather condescending manner: ‘The role of Rodolfo is very difficult vocally. I have enclosed a selection of the score that we will be hearing for the audition. If Mr. Mercury is interested we would like him to sing for the director in early August. Please let us know as soon as possible.’ There are a few notes hand-written on the score in all caps, the tone of which is quite arsey: ‘Sing only these lyrics, not others’; ‘1. Do not sing in another key’; ‘2. Sing high notes in full voice, not falsetto’ (with the ‘not’ double-underlined). Freddie was taken aback at being approached, and wasn’t interested, finding the prospect of singing opera for six or seven nights a week unappealing (‘I’d be bored to death’)⁸.
• A printed draft of album credits ends ‘This album is dedicated to all cat lovers[.] Fuck the rest of you cunts..Freddie Mercury’. The eventual dedication was the slightly tamer ‘This album is dedicated to my cat Jerry – also Tom, Oscar and Tiffany, and all the cat lovers across the universe – screw everybody else.’ Another draft has ‘fuck everybody else’ (and can be seen in the collage at the top of this post).

⁂

All said, for a Queen and Freddie geek like myself, this is an interesting cache of documents! Don’t think I can afford to bid on any of them, unfortunately.

Related posts

• Roger Taylor rocks the Usher Hall
• Roger Taylor 2026 tour ticket tracker

tl;dr: I found the AScore font in /Applications/Logic Pro.app/Contents/Frameworks/MACore.framework/Versions/A/Resources, installed it with Font Book, and can now open Logic’s PDFs in Affinity without rendering problems.

I use Apple’s Logic Pro to produce written musical scores. Logic is a bit of an eccentric choice for this purpose: its scoring functionality has its share of quirks, and probably doesn’t quite match a dedicated scoring application like Dorico in terms of power and flexibility. But it gets the job done and is actually quite capable.

Often I want to open a PDF of a score exported from Logic in an application like Affinity or Adobe Illustrator for light editing. Logic uses internal fonts called ’AScore‘ and ’AScoreParts’ for musical glyphs (note heads, time signatures etc.), which don’t render when a PDF produced by Logic is opened in a vector editing tool (you get a ‘missing font’ message).

The problem is most noticeable when using Internal as the symbol font (see screenshot below). However, even when using a third-party font like Jazz or Petaluma (my preferred hand-written font; see later screenshot), PDFs will fall back to using the Internal Font (AScore and AScoreParts) when a glyph/symbol is unavailable in the third-party font, or is perhaps mapped differently.

This problem bumbazed me for the longest time, but I finally figured out a fix.

AScore and AScoreParts aren’t installed along with the other system fonts, but I knew they had to be somewhere in the file system. Maybe I could find them and install them along with the others, which would allow them to be picked up by other applications. To my delight, this theory panned out. I located the fonts, installed them, and I can now open Logic PDFs in Affinity for editing (which is sometimes necessary when I can’t quite get Logic to format exactly the way I want it to – as I said, the scoring functionality can be quirky).

Installing the fonts

On my machine (running Logic Pro 12.3 on macOS Tahoe 26.6.1), AScore and AScoreParts are stored in /Applications/Logic Pro.app/Contents/Frameworks/MACore.framework/Versions/A/Resources. To install them, I opened Finder and selected ‘Go to Folder…’ from the ‘Go’ menu, pasted in the aforementioned path (adjust it if your Logic app lives elsewhere, such as in the Users folder), and hit Enter. I then saw AScore.ttf and AScoreParts.ttf listed. Woop. I double-clicked both to open and install them via Font Book. They’re then picked up when I open my PDF scores in Affinity. (I’ve only actually encountered AScore in the wild, but I installed AScoreParts too in case I use one of its symbols in the future.)

The Logic folder structure may well change with a subsequent version. If you’re reading this and the fonts are no longer at the path where I found them, try searching via Terminal:

1. cd '/Applications/Logic Pro.app'
2. find . -name "*AScore*"

(I was originally going to post all this as a reply to a thread on the Affinity forums, but I see they’re now read-only, replaced with a Discord community. That’s not a lot of use for people searching for answers. Apparently, moving communities to Discord is a trend.)

One study predicts that AI water consumption globally will be between 0.38 and 0.60 km³ in 2027. Animal agriculture, by comparison, is much, much worse, consuming 228 km³ annually. In other words, exploiting animals for their flesh, milk and eggs consumes roughly 380× to 600× as much water as AI does. To put the volumes in perspective: in 2027 AI is predicted to consume between a twentieth and a twelfth of the volume of water in Loch Ness (Scotland’s largest loch by volume, and the largest body of water by volume in Britain), whereas animal farming already consumes just over 30 entire Loch Nesses annually!

(‘Consumption’ here means water which is evaporated and lost, as opposed to ‘withdrawal’, which is largely returned to the system. Water withdrawal for AI in 2027 is predicted to be between 4.2 and 6.6 km³. Animal agriculture, in contrast, withdraws 566 km³ per year: 513 km³ for feed, 40 km³ for drinking water and 13 km³ for ‘service’ [non-potable] water.)

This is not to say there isn’t a problem with water usage by AI, especially in localities hosting data centres. And there are of course other environmental impacts of both AI and animal agriculture, not to mention the serious ethical violations inherent in the latter.

However, if water conservation and the environment in general concern you, you’re much better off adopting a plant-based diet (and advocating for it to others, including institutions and governments) than worrying too much about AI.

Featured post: Hand-written lyrics for lost Freddie Mercury songs up for auction.

This blog post write-up is both for my own future reference and for anyone else who might find it helpful.

Contents

• Introduction
• Adding functionality with custom modules
• The View
  • Outputting the HTML via the Fields section
  • Best practice
  • Filtering the View output
  • Accessing feed custom fields with Relationships
  • Add some AJAX
• Refreshing the feeds
• Conclusion

Introduction

The core functionality is based on a View and the Aggregator module. The site also uses the Rate and Voting API modules to enable users to ‘react’ to any news item by tapping on one of the emojis. The theme is a subtheme of Olivero, which ships with Drupal core. I quickly set up Drupal using SiteGround’s app installer. (Handily, the install comes with Composer, Drush and git out of the box.)

Adding functionality with custom modules

As well as contributed modules Aggregator and Rate, there are three custom modules:

‘Cookie Voter’

Cookie Voter alters the Rate module so that it uses cookies instead of IP addresses to track anonymous emoji reactions. Using IP addresses can cause users on shared networks to see others’ reactions appear as their own, which is confusing. BuzzFeed takes a similar approach, storing user reactions in the browser’s localStorage. On the other hand, the Bear blogging platform appears to use IP addresses to record upvotes. In Bear’s case that’s a sensible approach as votes affect a list of trending posts. At AnimalRights.fyi, emoji reactions simply offer users an informal way to engage with news stories.

‘Custom Headline Filter’

Custom Headline Filter provides a Views filter for removing duplicate (or overly similar) headlines. Such headlines can appear due to malformation of the incoming RSS feed, crossposting between websites, or when multiple news outlets report on the same story. There is a similarity threshold which can be adjusted to taste in the Views UI. The similarity is calculated by PHP’s similar_text function.

‘Custom Twig Extensions’

I needed to use PHP’s preg_replace function, and wanted to avoid installing the Twig Extensions module; so I created Custom Twig Extensions. (I try to install as few modules as possible to keep the complexity at a minimum and make site maintenance easier.)

Custom modules summary

All three modules were basically written by Claude (3.5 Sonnet), though there was a fair amount of back and forth between it and myself. In the case of Cookie Voter, for example, I asked it to refactor this Drupal 7 code for Drupal 8+, but it took much prompting to achieve the desired result. ‘We’ eventually landed on a solution after I fed the AI some relevant code from the Rate module codebase. This is a useful tip I will bear in mind when problem solving with a chatbot in future: don’t assume it already ‘knows’ a given codebase. (I had a conversation with ChatGPT about why supplying codebase excerpts for more obscure coding problems might be necessary.)

The View

Here’s a screenshot of the View, which is where the user-facing page is built:

Outputting the HTML via the Fields section

The Fields section forms the HTML output of the View, with Twig as the templating language. Here’s the Fields section aggregated into a single template representation:

<div class="news-item {{ title_1 }} fade-in-quick">
  <h3 class="headline{% if field_podcast == '1' %} icon-podcast{% endif %}" iid="{{ iid }}"><a href="{{ link }}" target="_blank">{{ title }}</a></h3>

  {# Get first paragraph of description and remove HTML tags #}
  {% set first_paragraph = description|split('</p>')[0]|trim %}
  {% set first_paragraph = first_paragraph starts with '<p>' ? first_paragraph[3:] : first_paragraph %}
  {% set first_paragraph = first_paragraph|striptags|trim %}

  {# Handle truncation #}
  {% set last_char = first_paragraph|last %}
  {% set last_three = first_paragraph|slice(-3) %}
  {% set last_nine = first_paragraph|slice(-9) %}

  {%
    if last_char not in ['.', '!', '?', '…', ':'] and
    last_three != '[…]' and
    first_paragraph|slice(-2) != '."' and
    first_paragraph|slice(-6) != '&nbsp;' and
    last_nine != ' ... more'
  %}
    {% set first_paragraph = first_paragraph ~ '.' %}
  {% elseif last_char == '…' %}
    {% set first_paragraph = first_paragraph|custom_replace('/(?<!\s)…$/u', ' […]') %}
  {% elseif last_nine == ' ... more' %}
    {% set first_paragraph = first_paragraph|custom_replace('/ \.\.\. more$/', ' […]') %}
  {% elseif last_three == '...' %}
    {% set first_paragraph = first_paragraph|custom_replace('/(?<!\s)\.{3}$/u', ' […]') %}
  {% elseif last_char == ':' %}
    {% set first_paragraph = first_paragraph|custom_replace('/:$/', '.') %}
  {% endif %}

  {# Hide descriptions that include these strings #}
  {% set excluded_patterns = [
    '©',
    'Image courtesy of',
    'Image supplied by',
    'Image credit',
    'The post',
    'Image:',
    'No abstract',
    'If you enjoyed this episode',
    'Published on'
  ] %}
  {% set is_excluded = excluded_patterns|filter(pattern => pattern in first_paragraph)|length > 0 %}

  {% if first_paragraph|length > 5 and not is_excluded %}
  <div class="views-field views-field-description">
    {{ first_paragraph|custom_replace('/&nbsp;/', '')|custom_replace('/&amp;/', '&') }}
    {% if field_include_feed_description == '1' %}
      {{ description_1 }}
    {% endif %}
  </div>
  {% endif %}

  <div class="meta">via {{ field_website }}
  {% if field_donate %}
    <span>
      <span class="time">{{ timestamp }}</span><span> {{ field_donate }}</span>
    </span>
  {% else %}
    <span class="time">{{ timestamp }}</span>
  {% endif %}
  </div>
</div>

<div class="comment fade-in-quick">{{ field_comment }}</div>

Most of the logic here is a bunch of heuristics which tidy up the HTML contained within the RSS feeds, eg standardising the ellipsis style for truncated descriptions, and removing descriptions that don’t provide any value to the reader.

You’ll see in the screenshot that a number of fields are set to ‘hidden’. Hiding a field makes its value available to output in the ‘Rewrite results’ section of subsequent fields. This allows you to combine or perform logic on two or more fields at once. The Field ‘Aggregator feed item: Title’, for instance, uses both the ‘Link’ and ‘Podcast?’ fields. The following markup is from the ‘Title’ field configuration under ‘Rewrite results’:

<div class="news-item {{ title_1 }} fade-in-quick">
  <h3 class="headline{% if field_podcast == '1' %} icon-podcast{% endif%}" iid="{{ iid }}"><a href="{{ link }}" target="_blank">{{ title }}</a></h3>

Here’s a screenshot of the UI for the ‘Title’ field:

Best practice

It’s probably better practice to create a Twig template file override instead of scattering the template across the View’s GUI as I’ve done here. Using the GUI is great for quickly prototyping Views, but you may later want to port it to a Twig template file so you can see the full template at a glance. (I may do this for my View as part of a project refinement exercise.)

Having all the code in a single template file also makes it easy to track changes. That said, it’s still possible to track changes with the GUI approach by exporting the configuration (drush cex) after making changes then committing the output to git.

Filtering the View output

The ‘Filter criteria’ section removes certain items entirely, such as sponsored posts and recipes. ‘Filter headlines (exposed)’ provides a text field by the which the user can search to filter news items by their headline.

Accessing feed custom fields with Relationships

The View is set up to list Aggregator feed items specifically; not the actual feeds. The ‘Aggregator feed’ relationship (under Advanced in the top-right of the Views screenshot) allows us to include custom fields from Aggregator feeds themselves. These include ‘Link’ fields for the feed’s website and a page where the user can donate to or otherwise support the website. Here’s a screenshot of /admin/config/services/aggregator/fields:

The ‘field_aggregator_item_rss_item_metadata’ Relationship links the View to a custom content type called ‘RSS Item Metadata’, which allows us to attach additional metadata to individual feed items. An ‘Entity reference’ field allows us to search for the news item we want. We can also add a comment beneath a news item or pin it to the right-hand column. Here’s the ‘Manage fields’ UI:

Add some AJAX

Make sure ‘Use AJAX’ (under ‘Other’) is set to ‘Yes’. This allows for pagination and filtering (‘Filter by headline’) without reloading the entire page.

Refreshing the feeds

In order to fetch new items as they’re published, I’ve set a cron job to run every 10 minutes. Drupal’s automated cron system only runs when Drupal processes a request, such as when someone visits the site, so I’ve disabled it by setting the ‘Run cron every’ dropdown at /admin/config/system/cron to ‘Never’, and added a cron directive via my hosting’s cron UI. It looks something like this (__string__ is available at /admin/config/system/cron):

*/10 * * * * curl -s -o /dev/null https://animalrights.fyi/cron/__string__

Using an external cron trigger in this way means the cron is guaranteed to run every 10 minutes.

There is an ‘Update interval’ field on each feed (/aggregator/sources/FEED_ID/configure), which I set to ‘15 mins’, ‘1 hour’, ‘1 day’ etc., depending on how frequently the given website tends to post new content. When the cron runs, it checks when each feed was last refreshed, and if the time elapsed since the last refresh exceeds the update interval, the feed will be fetched again, and any new items will be displayed by the View.

Additionally, news items older than a year are deleted, via the directive $config['aggregator.settings']['items']['expire'] = 31536000; in settings.php. (The ‘Discard items older than’ field at /admin/config/services/aggregator/settings is set to ‘Never’, and is overridden by settings.php.)

Conclusion

This post is an overview rather than a step-by-step tutorial. I may explore specific aspects of the website’s functionality in more depth in future posts.

Concerning Drupal as a platform, I wouldn’t necessarily recommend it for side projects like this. Drupal is complex, which is fine because it’s powerful; but managing the system (mainly running regular updates and managing configuration) can be a massive ball-ache. I only chose it for AnimalRights.fyi because I already build Drupal sites professionally. (And LLMs like Claude and ChatGPT make working with Drupal and similarly complex platforms much easier.)

Now that I have ZX Spectrum hardware for the first time in a couple of decades, I thought it would be a good time to see whether I could create a ‘Hello, world’ program in Z80 assembly language, convert it to machine code, and run it.

Reader caution: I don’t really know what I’m doing here. This blog post is more or less personal documentation. I’ve tried to avoid inaccuracies, but feel free to correct me in the comments.

I found the take on ‘Hello, world’ described on Benjamin Blundell’s website to be nice and succinct, so I used that. Here it is:

org $8000
  ld bc, MY_STRING

MY_LOOP:
  ld a, (bc)
  cp 0
  jr z, END_PROGRAM
  rst $10
  inc bc
  jr MY_LOOP

END_PROGRAM:
  ret

MY_STRING:
  defb "Hello, world!"
  defb 13, 0

I’ll use the Pasmo assembler to translate the instructions into binary machine code so that the Spectrum can understand it.

Some notes, based on Benjamin’s blog post, on what each line does:

org $8000

This is an assembler directive to set the starting (‘origin’) memory location (address) of the proceeding program ($8000 is 0x8000, or 8000 in hex, or decimal 32768). This location is safely within a 48K Spectrum’s non-system memory. (See Dean Belfield’s website for details on how memory is mapped in the both 48 and 128K models.)

ld bc, MY_STRING

Load register pair BC with the starting address of MY_STRING. Registers are small storage areas on the processor where data can be manipulated. We need to move a given value stored in memory into a register before anything can be done with it (eg print it on screen, do a calculation with it). We don’t need to think about this when working in a higher-level language like BASIC or JavaScript: memory management is handled for us. However when writing assembly language we need to move values explicitly between memory and the registers.

Regarding the address of our string, you may be wondering, as I did: ‘Hello, world!’ isn’t actually written to memory until later in the code, so how does the assembler know at this stage what the address is? The answer is that most assemblers, including Pasmo, run two passes, the first of which will read through the assembly code to determine the address of each label. Only on the second pass will the machine code be generated. So when the instruction ld bc, MY_STRING is encountered on pass two, the assembler knows the starting address of MY_STRING. This concept and process is referred to as ‘forward referencing’.

MY_LOOP:

Add label MY_LOOP to mark a loop that will cycle through each byte of whatever is stored starting at location MY_STRING.

ld a, (bc)

Load register A (the accumulator) with the value of the first byte at address MY_STRING (which will be the ‘H’ from ‘Hello, world!’ on the first go-around of this loop).

cp 0

Compare the contents of register A with 0. 0 is the last byte, after Hello, world! and a carriage return, set by the instruction defb 13, 0 later.

jr z, END_PROGRAM

Jump to END_PROGRAM if the above comparison is true (ie the value in A is 0).

rst $10

Call the ROM routine at address 0x10, which prints whatever is in register A to the screen. RST (or ‘restart’) is the same as CALL, except it uses only one byte, and is faster. However, you can only use it with eight specific ROM addresses, one of which is 0x10, which we call here. (Source: ‘Spectrum Machine Language for the Absolute Beginner’, page 128.) You could think of these routines as built-in helper functions which perform a specific task.

inc bc

Increment register pair BC so it moves to the next byte in memory (ie the next character in our string).

jr MY_LOOP

Jump back to the top of the loop.

END_PROGRAM:

Another label. Labels can have arbitrary values, and help us and the assembler navigate the code.

ret

Exit program.

defb "Hello, world!"

Define bytes with the string we want to print.

defb 13, 0

Define two more bytes — a carriage return (13) and string terminator (0) — so that cp 0 earlier in the code can check whether we’ve reached the end of the string.

I saved the file as helloworld.asm.

Converting our program into machine code

We’ll use the command line tool Pasmo (a Z80 cross-assembler) to assemble our machine code and create our .tap file, which we can then run on a Spectrum emulator or actual Speccy hardware. The TAP will comprise helloworld.asm in machine code form, and a BASIC loader program. The latter loads the machine code into memory and runs it. (Alternative assemblers include zasm, and Odin and Zeus if you’d like to develop on an actual Spectrum.)

Build and install Pasmo

I’m using macOS here, but the steps should be similar on other platforms.

1. First install CMake, a C++ build tool. I’ll use Homebrew: brew install cmake
2. Download Pasmo: git clone https://github.com/jounikor/pasmo.git
3. cd pasmo/pasmo (steps 3 to 7 are taken from the Pasma README.md)
4. mkdir build
5. cd build
6. cmake ../
7. make
8. Copy Pasmo to /usr/local/bin/ so that you can run it from anywhere: sudo cp pasmo /usr/local/bin/
9. Relaunch the shell: exec zsh -l
10. Verify that you can run Pasmo by viewing the manual page: pasmo man

Assemble our program and create the .tap file

1. Before we run the assembly process, add the Pasmo directive END $8000 at the end of your helloworld.asm file. This will prompt Pasmo to include a RANDOMIZE USR statement in the BASIC loader program. This will ensure our machine code runs when we launch the .tap file
2. Run pasmo --tapbas helloworld.asm helloworld.tap

Open helloworld.tap in Fuse or another emulator, or on an actual Spectrum Next, as I did. It should look like this:

The BASIC loader program

This is the BASIC loader program that Pasmo created:

10 CLEAR 32767
20 POKE 23610,255
30 LOAD "" CODE
40 RANDOMIZE USR 32768

Line by line:

• 10 CLEAR 32767 — ensure that the BASIC interpreter doesn’t write to memory above address 32767 (as well as clear any variables that are already stored there). This value should be a byte before the start of where our machine code will be stored.
• 20 POKE 23610,255 — ‘avoid a[n] error message when using +3 loader’. (Source: spectrum.cxx in the pasmo-0.5.5 codebase.)
• 30 LOAD "" CODE — load the next binary code file the Spectrum finds (which would typically have been on a cassette tape back in the day, located just after the BASIC loader program).
• 40 RANDOMIZE USR 32768: call (run) the machine code that’s stored starting at address 32768, which we specified in our helloworld.asm code on line 1 (org $8000), and at the end (END $8000) as Pasmo requires. $8000 is a shorthand for hexadecimal 8000 (0x8000), or decimal 32768.

What next?

I just received my copy of 40 Best Machine Code Routines for the ZX Spectrum by John Hardman and Andrew Hewson (with a new chapter on the Next by Jim Bagley), and it seems like an accessible guide to using machine code. I’ll try out some of the routines.

The tutorial ‘ZX Spectrum Machine Code Game in 30 Minutes!’ by Jon Kingsman looks intruiging, and promises the ability to program in machine code in the time it takes to consume a large cup of tea. I tend to ‘learn by doing’ so I think I may tackle this next.

Sources

• Assembly on the ZX Spectrum - Part 1, Benjamin Blundel
• Original source of this ‘Hello, world’ code may have been DIDIx13/hello-world
• Spectrum Machine Language for the Absolute Beginner, ed. William Tang. 1982 (2020 reprint)
• Jonathan Cauldwell, How to Write ZX Spectrum Games, v1.1 (?2020): ‘Chapter Eighteen - Making Games Load and Run Automatically’
• Spectrum memory map, Dean Belfield
• Pasmo docs
• ChatGPT (GPT-4)

Featured post: Hand-written lyrics for lost Freddie Mercury songs up for auction.

Requirements

If you have not already done so, install the following tools:

1. Docker Desktop
2. Homebrew: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
3. Composer: brew install composer
4. Node/npm: brew install node
5. DDEV: brew install ddev/ddev/ddev
6. mkcert: mkcert -install

Installation

I initially followed this Digital Ocean guide and distilled the key steps into the below sequence. (The Digital Ocean tutorial also covers Linux.)

1. mkdir __project_name__
2. cd __project_name__
3. ddev config --project-type=drupal11 --docroot=web --create-docroot
4. ddev start
5. ddev composer create-project "drupal/recommended-project"
   • This will install the latest stable version of Drupal. To choose an older version, say v10, run ddev composer create-project "drupal/recommended-project:^10".
6. ddev composer require "drush/drush"
   • If you get a PHP version compatibility error running this command:
     1. Update php_version in .ddev/config.yaml.
     2. ddev restart
     3. Run step 6 again.
7. ddev exec drush site:install --account-name=admin --account-pass=admin (or replace admin with a more secure username and password.)
8. ddev launch

Bish, bash, bosh: a new Drupal site, with the URL https://__project_name__.ddev.site/, should now open in your browser. Nae bother.

That’s it

You might like to install Bootstrap5 as a starter theme: composer require 'drupal/bootstrap5'. And these modules:

1. Simple XML sitemap: composer require drupal/simple_sitemap && ddev drush pm:enable simple_sitemap
2. Admin Toolbar: composer require drupal/admin_toolbar && ddev drush pm:enable admin_toolbar admin_toolbar_tools admin_toolbar_search
3. Pathauto: composer require drupal/pathauto && ddev drush pm:enable pathauto

At this point you might commit the code to a git repo, then get on with developing your site.

If you have any corrections or optimisations, please comment below.

For this blog post I’ve chosen a practical, real-life example from my archive of ChatGPT conversations. The problem in this case: I wanted to download a bunch of files (PDFs mostly) for which I had a list of URLs. My initial thought was to look into whether I could run something in the browser, for three reasons:

1. I’m a front-end dev whose natural habitat is the browser, so I should be able to whip up a solution quickly.
2. You don’t need to worry about authentication (if the files are only accessible when you’re logged in) as you can run the code from within an authenticated browser session.
3. The code can be run by others easily on their own machines.

On being presented with this problem, rather than start from a blank slate, I had a conversation with ChatGPT (GPT-4) about it. I’ve reproduced a lightly edited version of it below.

I’m aware that in blogging about a ChatGPT conversation I could be accused of being a bore (like somebody going on about the dream they had last night). However, large language model-based products, like ChatGPT and GitHub Copilot, look like they’re ushering in a major change in how sofware developers work, so I thought why not write about how I’m using them.

ChatGPT maintains context throughout the conversation so I can ask it questions about the information it’s given me (‘What does async do?’ and ‘Is that called “destructuring”?’), and ask it to refactor code to meet new requirements (‘Can you refactor downloadPDF() to handle multiple PDFs?’, ‘Can I determine the MIME type?’).

If you’re in a hurry, you can jump straight to the final iteration of the code.

JavaScript function to fetch and download PDF.

To fetch and download a PDF using JavaScript, you can use the fetch function to get the PDF data as a Blob and then create a download link using that Blob. Here’s an example function that takes a URL and a filename as parameters:

async function downloadPDF(url, filename) {
  try {
	// Fetch the PDF data as a Blob
	const response = await fetch(url);
	const blob = await response.blob();

	// Create a download link and set its attributes
	const link = document.createElement('a');
	link.href = URL.createObjectURL(blob);
	link.download = filename;

	// Append the link to the document, click it, and remove it
	document.body.appendChild(link);
	link.click();
	document.body.removeChild(link);
  } catch (error) {
	console.error('Error while downloading PDF:', error);
  }
}

// Usage example
const pdfURL = 'https://example.com/path/to/your/pdf-file.pdf';
const pdfFilename = 'downloaded-pdf-file.pdf';
downloadPDF(pdfURL, pdfFilename);

Make sure to replace the pdfURL variable with the URL of the PDF file you want to download. The pdfFilename variable is the name that will be given to the downloaded file.

[Tangential question removed.]

Re the downloadPDF() function you wrote, can you refactor to handle a list of PDFs to download, each with a specific, defined value[?]

Certainly! Here’s a modified version of the downloadPDF function that accepts an array of objects, each containing the URL and filename for a PDF. The function will download all PDFs one after another.

async function downloadPDFs(pdfs) {
  // [Deleted]

  for (const pdf of pdfs) {
	const { url, filename } = pdf;
	try {
	  // Fetch the PDF data as a Blob
	  const response = await fetch(url);
	  const blob = await response.blob();

	  // Create a download link and set its attributes
	  const link = document.createElement('a');
	  link.href = URL.createObjectURL(blob);
	  link.download = filename;

	  // Append the link to the document, click it, and remove it
	  document.body.appendChild(link);
	  link.click();
	  document.body.removeChild(link);

	  // Add a delay to avoid triggering browser's security mechanisms
	  await new Promise((resolve) => setTimeout(resolve, 1000));
	} catch (error) {
	  console.error(`Error while downloading ${filename}:`, error);
	}
  }
}

// Usage example
const pdfList = [
  {
	url: 'https://example.com/path/to/your/pdf-file1.pdf',
	filename: 'downloaded-pdf-file1.pdf',
  },
  {
	url: 'https://example.com/path/to/your/pdf-file2.pdf',
	filename: 'downloaded-pdf-file2.pdf',
  },
  // Add more PDFs as needed
];

downloadPDFs(pdfList);

Replace the pdfList array with the URLs and filenames for the PDF files you want to download. The function will iterate through the list and download each PDF file with the specified filename.

Please note that downloading multiple files in quick succession might be flagged by some browsers as suspicious behavior, potentially causing the downloads to be blocked. To avoid this, a delay is added between downloads using setTimeout. You can adjust the delay time as needed.

What does the async before the function definition mean?

The async keyword before a function definition indicates that the function is asynchronous. An asynchronous function is a function that can perform operations without blocking the execution of other code, allowing other tasks to run concurrently.

Asynchronous functions use the await keyword within their bodies to pause the execution of the function until a Promise is resolved or rejected. When a Promise is resolved, the function continues execution from the point where it was paused.

Here’s a simple example to illustrate how async functions work:

async function fetchData() {
  console.log('Fetching data...');

  // Simulate an asynchronous operation (e.g., fetching data from an API)
  const result = await new Promise((resolve) =>
	setTimeout(() => resolve('Data fetched!'), 1000)
  );

  console.log(result);
}

console.log('Starting...');
fetchData();
console.log('Waiting for data...');

The output will be:

Starting...
Fetching data...
Waiting for data...
Data fetched!

As you can see, the async function does not block the execution of the rest of the code. While it’s waiting for the Promise to resolve, other code (in this case, the console.log('Waiting for data...')) can run.

In the downloadPDFs function example, the async keyword allows the function to wait for the fetching and downloading of each PDF without blocking the execution of other code. This is useful when working with network requests or other asynchronous operations.

In the refactored downloadPDFs() function, what does const { url, filename } = pdf; do? Is that called ‘destructuring’ or something?

Yes, you’re correct! The line const { url, filename } = pdf; is an example of object destructuring in JavaScript. Object destructuring is a syntax feature that allows you to extract properties from an object and create new variables that hold the values of these properties.

In the downloadPDFs() function, the pdfs parameter is an array of objects, where each object has a url and filename property. The line const { url, filename } = pdf; is used to extract the url and filename properties from the current pdf object in the loop. This creates two new variables url and filename holding the respective values, which can then be used in the following lines of code.

Here’s a simple example to illustrate object destructuring:

const person = {
  name: 'John',
  age: 30,
};

const { name, age } = person;

console.log(name); // Output: 'John'
console.log(age);  // Output: 30

In this example, the name and age properties are extracted from the person object, and new variables name and age are created with the corresponding values.

Can you do this with arrays, for example if pdfList was an array of arrays, each PDF array being the format ['https://example.com/pdfs/pdf1.pdf', 'pdf1.pdf']?

Yes, you can use array destructuring to handle the pdfList if it’s an array of arrays. Array destructuring works similarly to object destructuring but is used to extract elements from arrays. Here’s the modified version of the downloadPDFs() function that works with an array of arrays:

async function downloadPDFs(pdfs) {
  for (const pdf of pdfs) {
	const [url, filename] = pdf; // Array destructuring
	try {
	  // Fetch the PDF data as a Blob
	  const response = await fetch(url);
	  const blob = await response.blob();

	  // Create a download link and set its attributes
	  const link = document.createElement('a');
	  link.href = URL.createObjectURL(blob);
	  link.download = filename;

	  // Append the link to the document, click it, and remove it
	  document.body.appendChild(link);
	  link.click();
	  document.body.removeChild(link);

	  // Add a delay to avoid triggering browser's security mechanisms
	  await new Promise((resolve) => setTimeout(resolve, 1000));
	} catch (error) {
	  console.error(`Error while downloading ${filename}:`, error);
	}
  }
}

// Usage example
const pdfList = [
  [
	'https://example.com/path/to/your/pdf-file1.pdf',
	'downloaded-pdf-file1.pdf',
  ],
  [
	'https://example.com/path/to/your/pdf-file2.pdf',
	'downloaded-pdf-file2.pdf',
  ],
  // Add more PDFs as needed
];

downloadPDFs(pdfList);

In this version, the line const [url, filename] = pdf; uses array destructuring to extract the URL and filename from each array element in pdfList. The rest of the function remains the same.

Say some of the files were Word docs (.docx); can I determine the MIME type from the response so I can give the file to be downloaded the correct file extension?

Yes, you can determine the MIME type of the fetched file from the Content-Type header in the response object. Based on the MIME type, you can set the appropriate file extension for the downloaded file. Here’s the modified version of the downloadPDFs() function that handles different file types and sets the correct file extension:

async function downloadFiles(files) {
  for (const file of files) {
	const [url, filename] = file;
	try {
	  // Fetch the file data as a Blob
	  const response = await fetch(url);
	  const blob = await response.blob();

	  // Get the MIME type from the response headers
	  const contentType = response.headers.get('content-type');
	  let extension;

	  // Determine the file extension based on the MIME type
	  switch (contentType) {
		case 'application/pdf':
		  extension = '.pdf';
		  break;
		case 'application/vnd.openxmlformats-officedocument.wordprocessingml.document':
		  extension = '.docx';
		  break;
		// Add more cases for other MIME types if needed
		default:
		  extension = ''; // Fallback to no extension if MIME type is unknown
	  }

	  // Create a download link and set its attributes
	  const link = document.createElement('a');
	  link.href = URL.createObjectURL(blob);
	  link.download = filename + extension;

	  // Append the link to the document, click it, and remove it
	  document.body.appendChild(link);
	  link.click();
	  document.body.removeChild(link);

	  // Add a delay to avoid triggering browser's security mechanisms
	  await new Promise((resolve) => setTimeout(resolve, 1000));
	} catch (error) {
	  console.error(`Error while downloading ${filename}:`, error);
	}
  }
}

// Usage example
const fileList = [
  [
	'https://example.com/path/to/your/pdf-file1.pdf',
	'downloaded-pdf-file1',
  ],
  [
	'https://example.com/path/to/your/word-file1.docx',
	'downloaded-word-file1',
  ],
  // Add more files as needed
];

downloadFiles(fileList);

In this version, the function is renamed to downloadFiles() to reflect that it can handle different file types. The MIME type is obtained from the response headers using response.headers.get('content-type'), and a switch statement is used to determine the appropriate file extension based on the MIME type. The link.download attribute is set to the filename concatenated with the determined file extension.

Please note that this approach relies on the server providing the correct Content-Type header for the files. If the server doesn’t provide the correct MIME type, you might need to use an alternative method, such as checking the file extension in the URL or using a library to determine the file type based on the content.

And that was me with the code I needed. I manually tidied it up a bit, and this is the format of the final version that I ran:

async function downloadFiles(files) {

  // MIME type: file extension
  const fileExtensions = {
	'application/pdf': 'pdf',
	'application/vnd.openxmlformats-officedocument.wordprocessingml.document': 'docx',
	'application/msword': 'doc',
	'application/zip': 'zip'
  }

  const mimeTypes = [];

  // Collate MIME types
  Object.keys(fileExtensions).forEach(key => {
	mimeTypes.push(key);
  });

  for (const file of files) {
	const [url, filename] = file;

	try { // Fetch the file data as a Blob

	  const response = await fetch(url);
	  const contentType = response.headers.get('Content-Type');

	  if (response.ok && mimeTypes.includes(contentType)) {
		const blob = await response.blob();

		// Create a download link and set its attributes
		const link = document.createElement('a');
		link.href = URL.createObjectURL(blob);
		link.download = `${filename}.${fileExtensions[contentType]}`;

		// Append the link to the document, click it, and remove it
		document.body.appendChild(link);
		link.click();
		document.body.removeChild(link);
	  } else {
		console.error(`Error while downloading ${filename}: Invalid content type (${response.headers.get('Content-Type')}) or response error`);
	  }

	  // Add a delay to avoid triggering browser's security mechanisms
	  await new Promise((resolve) => setTimeout(resolve, 300));
	} catch (error) {
	  console.error(`Error while downloading ${filename}:`, error);
	}
  }
}

// Usage example
var fileList = [ // `const` → `var` so we run in single browser console session more than once
  ['https://example.com/path/to/your/pdf-file1.pdf', 'pdf-file1'],
  ['https://example.com/path/to/your/word-file1.docx', 'word-file1'],
];

downloadFiles(fileList);

This was a pretty straightforward series of prompts. The problem was a relatively simple one, and ChatGPT was able to output what I wanted from start to finish. In more complex scenarios, it might only get you started, or you’ll have enough code you want to ask it about that you’ll run out of tokens. ChatGPT also didn’t have any hallucinations in this session. When that happens you can politely let it know it’s havering and should provide a new answer.

And as to the way I went about solving this problem (running a script from the browser console), some may think it a bit inelegant or hacky. You’re probably right! But it did the job and that’s what matters. And thanks to ChatGPT it took less time and effort than it would have otherwise.

Disclaimer: I’m not a Drupal expert and share this post only to help others who may have a similar use case. I make no guarantees of best practice! This should work in Drupal 8 onwards. Refinements welcome in the comments.

The scenario here is that you want to create a number of landing pages which, within the body of each landing page, list their immediate sub-pages. Along with a hyperlink, you also want to output a thumbnail image which has been assigned to each sub-page.

Say you were building a website for a professional wrestling organisation called the Scottish Wrestling Federation (SWF) and had a number of categories under a parent Roster page: Men, Women, Broadcast team and Referees. Each landing page is a gallery of photos which link to a relevant profile page.

First create a custom Image field called field_wrestler_photo (most of the roster are wrestlers so we’ll ignore the fact that a few are referees and announcers in terms of the naming). Then set up a Menu block called Sub-pages within the Main content region at /admin/structure/block. You should now see a theme hook suggestion at /admin/structure/block/manage/sub_pages#edit-settings-style (within the HTML and style options settings). Let’s name the block menu__sub_pages.

Next add a function called swf_preprocess_menu__sub_pages() to our swf.theme file. This function will process the URL and alt text data and assign it to properties of $variables. The data will then be available to us in menu--sub_pages.html.twig in the templates/navigation folder.

We’ll be using the Node interface within our preprocess function in swf.theme, so first include it towards the top of the file:

use Drupal\node\Entity\Node;

Begin our function by setting up a loop to get the data for each page:

function swf_preprocess_menu__sub_pages(&$variables) {
  foreach ($variables['items'] as $key => $item) { // Loop through each page
    // ...
  }
}

For each page, get its URL (eg /roster/women/jean-meikle) and node path (eg /node/23). From the node path use preg_match() to get the node ID (23 in this example):

function swf_preprocess_menu__sub_pages(&$variables) {
  foreach ($variables['items'] as $key => $item) {
    $page_url = $variables['items'][$key]['url']->toString(); // Get page’s URL
    $node_path = \Drupal::service('path_alias.manager')->getPathByAlias($page_url); // Get node path, eg /node/23
    if (preg_match('/node\/(\d+)/', $node_path, $matches)) {
      $node_id = intval($matches[1]); // Get the numeric node ID
      // ...
    }
  }
}

Next, check for the property field_wrestler_photo and whether it has a value. Then get the Node object for this node ID:

function swf_preprocess_menu__sub_pages(&$variables) {
  foreach ($variables['items'] as $key => $item) {
    $page_url = $variables['items'][$key]['url']->toString();
    $node_path = \Drupal::service('path_alias.manager')->getPathByAlias($page_url);
    if (preg_match('/node\/(\d+)/', $node_path, $matches)) {
      $node_id = intval($matches[1]);

      if (
          $node_id &&
          Node::load($node_id)->field_wrestler_photo && // Has wrestler photo field
          count(Node::load($node_id)->field_wrestler_photo) // Field has value
      ) {
        $node = Node::load($node_id); // Get node object
        // ...
      }
    }
  }
}

You can now access the alt text, and use getFileUri() and the file_url_generator service to get the image’s URL. Finally, assign them to $variable properties so you can reference them in the Twig templates:

function swf_preprocess_menu__sub_pages(&$variables) {
  foreach ($variables['items'] as $key => $item) {
    $page_url = $variables['items'][$key]['url']->toString();
    $node_path = \Drupal::service('path_alias.manager')->getPathByAlias($page_url);
    if (preg_match('/node\/(\d+)/', $node_path, $matches)) {
      $node_id = intval($matches[1]);

      if (
          $node_id &&
          Node::load($node_id)->field_wrestler_photo &&
          count(Node::load($node_id)->field_wrestler_photo)
      ) {
        $node = Node::load($node_id);
        $image_alt_text = $node->field_wrestler_photo->alt; // Get alt text
        $file_uri = $node->field_wrestler_photo->entity->getFileUri(); // Get file URI
        $image_url = \Drupal::service('file_url_generator')->generateString($file_uri); // Get image URL
        $variables['items'][$key]['image_alt_text'] = $image_alt_text; // Store alt text
        $variables['items'][$key]['image_url'] = $image_url; // Store image URL
      }
    }
  }
}

Create a Twig template for the roster lists in templates/navigation/menu--sub_pages.html.twig:

{% import _self as menus %}

{{ menus.build_menu(items, attributes, 0) }}

{% macro build_menu(items, attributes, menu_level) %}
  {% import _self as menus %}
  {% if items %}
  {% for item in items %}
    {{ menus.add_link(item, attributes, menu_level) }}
  {% endfor %}
  {% endif %}
{% endmacro %}

{% macro add_link(item, attributes, menu_level) %}
  <div>
    <a href="{{ item.url }}">
     <img src="{{ item.image_url }}" alt="{{ item.image_alt_text }}" width="240" height="360"> {# Wrestler image #}
    </a>
    <div>{{ item.title }}</div> {# Wrestler name #}
  </div>
{% endmacro %}

You can now create a page for each person in the roster and arrange them into categories by editing the menu struture at /admin/structure/menu/manage/main.

You probably want to restrict the sub-pages block to only appear on the Roster pages by adding relevant paths at /admin/structure/block/manage/sub_pages#edit-visibility-request-path: /roster, roster/men etc.

We’ve focused here on a custom image field, but you can access any default or custom field in a similar fashion. For example, you could add field_wrestler_height and field_wrestler_weight fields and list each wrestler’s height and weight under their photo.

When coming up with a domain name for your latest venture, the possibilties now seem endless. Even if a .com is still available,² there’s probably something more eye-catching and a better fit for your project out there.

gTLDs can have their own tone, ranging from the irreverent (.fail and .lol) to the functional (.plumbing and .florist) to the apparently inexplicable (.ooo and .xyz), and they seem to have become somewhat more visiable of late. For example, the advice columnist and podcast host Dan Savage’s domain is now savage.love,³ and I’ve just seen freetesting.hiv in an Instagram ad.⁴ The former GoCompare.com (quite well known in the UK owing to their annoying TV ads) has gone for a full rebrand as Go.Compare, with the idea being that ‘searching’ for it on any device will take the user directly to the website rather than an intermediary search engine.

The project I’ve been working on for the past couple of years is a website with a bunch of jazz piano tips and tricks. In the process of thinking about what I should call it, I jotted down some ideas. I thought I’d publish them here as a blog post, along with comments on their suitability. I’m not sure whether the list provides much in the way of insight into how to best go about choosing the ‘perfect’ domain name, but I present it here nonetheless for anyone looking for inspiration.

(These are the domains in the order I jotted them down in my notes file. You can mostly interchange ‘jazz’, ‘jazzkeys’ and ‘jazztoolkit’, as it took some time before I settled on which variant I liked best.)

• jazz.wtf
  I like this. It’s irreverent. Reflects the state of feeling overwhelmed when learning a complex subject like jazz. Maybe a bit sweary for some people, though.
• jazz.clinic
  Nice. It might suggest that the website is some kind of a workshop, which it’s not. Maybe a bit too grand also: I don’t feel in a position to put on a ‘clinic’ in jazz piano. And aren’t clinics more things that drummers do?
• jazz.tips
  Like it. Short. Doesn’t suggest something too all-encompassing.
• jazztoolkit.fyi
  I like the informality of the ‘.fyi’ domain. Reflects the approach I’m taking. ‘Toolkit’ feels like it reflects what the site will be: a bunch of techniques and approaches to playing jazz piano.
• jazzpiano.how
  Cool domain, but .how suggests too grand a scope. The site will be a collection of tips, examples etc.; not a soup-to-nuts course in how to play jazz piano. I’m not sure even the most accomplished jazz piano player would be so arrogant as to think they know the ‘how’: making music is an individual, and often mysterious, thing.
• jazz.fyi
  Nice. However, I’m leaning towards focusing on just jazz piano and not jazz in general (though a lot of stuff can also be applied to other instruments). Domain is already registered so I’d need to look into making an offer.
• jazz.fail
  Like jazz.wtf, this is irreverent, and I like it; but maybe a bit negative sounding.
• jazzpiano.tools
  Goes well with the ‘toolkit’ framing.
• jazzkeys.tools
  Fine. I prefer ‘keys’ to ’piano’. It’s more informal, and I’m recording a lot of the examples on an electric piano anyway.
• jazztoolkit.online
  OK, I guess. There are better TLDs than .online.
• jazztoolkit.help
  Cool TLD that reflects what I want to achieve with the site.
• jazztoolk.it
  Too clever. Difficult to verbally convey to others. Not a huge fan of using country-level TLDs for purposes other than those for which they’re intended.
• jazz.help
  Fine, but at ~£100 it’s a bit expensive.⁵
• jazztoolkit.app
  I like the .app TLD, and the site does have app-like functionality; but it might confuse non-tech people as it’s a web app, which I don’t think means much to non-tech punters.
• 🌟 jazzkeys.fyi
  OK, this is perfect. Suggests an informal, non-prescriptivist approach: “Here’s what works for me; if it’s of use to you, then great.” Also rolls off the tongue. JazzKeys.fyi it is!

JazzKeys.fyi will be launching soon. Sign up to be notified!

tl;dr: if you’d rather just dig into the code, and not bother with this blog post, there’s a repo on GitHub.

This post describes a demo version — available at the repo linked above — of the code I deployed on my home page.

Contents

• Testing the API
• Authenticating
• Proxy scripts
  • header.inc
  • fetch.php: fetch a listing of gists for a user
  • fetch-id.php: fetch content of a specific gist
  • Cross-Origin Resource Sharing (CORS)
  • Folder structure
• Building the front end
  • Installing a custom build of highlight.js as a npm dependency
  • index.html
  • index.js
• Summary

Testing the API

You can query the API to fetch a list of gists for a specific user without authenticating. For example, running this code in your browser console will return a listing of my 15 most recent gists in JSON format (optionally replace donbrae with your own user name):

fetch('https://api.github.com/users/donbrae/gists?per_page=15').then(response => {

  // Success
  if (response.ok)
    return response.json(); // Returns to then()

  // Error
  return Promise.reject(response);

}).then(data => {
  console.log(data);
}).catch(err => {
  console.error(err); // Error
});

You can also fetch the actual content of a gist by using the /gists/{gist_id} endpoint, eg https://api.github.com/gists/d78ee08d2ffdc2f7b8442155f9cf7fa1.

Authenticating

For unauthenticted users, the API is rate-limited to 60 requests per hour. Creating a personal access token¹ and using it to authenticate with the API gets you 5,000 an hour.

When authenticating, we don’t want to expose the access token by making the request directly on the client side, so we can set up proxy scripts on a server to do the authenticating and make the API calls.

I wrote the scripts in PHP because I have some familiarity with the language, and it comes installed with your typical web hosting plan, but you can translate it to your preferred language.

Proxy scripts

The two proxy scripts — one to fetch the list of gists, and one to fetch a particular gist’s content — are in the repo donbrae/get-gists.

header.inc

header.inc contains code we’ll use in both scripts. First we add headers to set the Content-Type to application/json (we’ll be serving JSON), and the Cache-Control header to no-cache (we’ll be implementing our own caching system).

<?php
header('Content-Type: application/json');
header('Cache-Control: no-cache');

Next, the variable $opts contains our HTTP context options, including the request header values we’ll pass to the GitHub API. Replace <token> with your personal access token.

$opts = [
  'http' => [
    'method' => 'GET',
    'header' => [
      'User-Agent: PHP',
      'Content-Type: application/json',
      'Accept: application/vnd.github+json',
      'Authorization: token <token>'
    ]
  ]
];

There is also a function that writes files in a way that avoids potential write conflicts:

function writeFile($filename, $content, $append = false) {
  // https://softwareengineering.stackexchange.com/a/332544
  $unique_tmp_filename = uniqid('', true) . '.tmp'; // Create unique filename
  if ($append) {
    file_put_contents($unique_tmp_filename, file_get_contents($filename) . $content); // Concatenate contents of existing file with new content
  } else {
    file_put_contents($unique_tmp_filename, $content);
  }

  rename($unique_tmp_filename, $filename);
}

And a function to parse the response headers:

function getResponseHeaders($http_response) {
  // https://beamtic.com/parsing-http-response-headers-php
  $response_headers = [];
  foreach ($http_response as $value) {
    if (false !== ($matches = explode(':', $value, 2))) {
      $response_headers["{$matches[0]}"] = trim($matches[1]);
    }
  }
  return $response_headers;
}

And, finally, a function to get the HTTP response status code as an integer:

function getStatus($http_response) {
  // https://stackoverflow.com/a/52662522/4667710
  $status_message = array_shift($http_response);
  preg_match('{HTTP\/\S*\s(\d{3})}', $status_message, $match);
  $status = intval($match[1]);
  return $status;
}
?>

(You probably won’t need the last two functions if you’re not using PHP. PHP is weird.)

fetch.php: fetch a listing of gists for a user

Now onto the script that will actually get the data, fetch.php.

We start by including the header.php file:

<?php
require_once 'header.inc';

Next, name a file that we’ll use as a cache:

$cache_file = 'cached.json';

Now we’ll call the API and store the returned data in variable $content:

$context = stream_context_create($opts);
$content = file_get_contents('https://api.github.com/users/donbrae/gists?per_page=15', false, $context);

We’ll then use a couple of the other functions we declared in header.php to parse out the response status code and headers. This data is available in the $http_response_header array:

$status = getStatus($http_response_header);
$response_headers = getResponseHeaders($http_response_header);

If the status is 200, we output the JSON returned from the API:

if ($status === 200) {
  echo $content;

Next we add a conditional that checks the X-RateLimit-Remaining response header to determine whether we should write a copy of the returned JSON to our cache file:

  if (
    intval($response_headers['X-RateLimit-Remaining']) < 250 && // We may hit the API rate limit soon
    file_exists($cache_file) && (time() - filemtime($cache_file)) / 60 / 60 > 6 || // Cached file hasn’t been updated in last 6 hours
    !file_exists($cache_file) // Or there is no cache file
  ) {
    writeFile($cache_file, $content); // Write a cached version of JSON in case API rate limit is reached
  }

I set the threshold to an arbitrary 250 API calls remaning. The conditional could instead in theory be X-RateLimit-Remaining === 0, but the script that fetches the actual gist content (fetch-id.php) also counts towards API usage, so we can’t guarantee that fetch.php will be called when the rate limit is exactly zero.²

There is also a condition which checks that the cache file hasn’t been updated in the last six hours, to avoid continually writing a new cache file every time this script is called and X-RateLimit-Remaining is under 250.

Alternatively, if no cache file exists — regardless of other conditions — write one.

If another status is returned, and if there is a cached version, serve the cached version; otherwise, return an error message in JSON format:

} else if (file_exists($cache_file)) { // Status ?304, 403 or 422
  echo file_get_contents("cached.json"); // Serve cached copy
} else {
  echo "{\"error\": \"Cannot fetch list of gists. Error code: $status\"}";
}
?>

CORS

Add fetch.php to your server in a folder called gists. If you’re using another domain to host the proxy scripts — as I’m doing because my blog is a static site hosted on Netlify, which doesn’t support PHP — we’ll need to make sure Cross-Origin Resource Sharing (CORS) is set up properly. We need to tell our server hosting fetch.php to accept Ajax requests from our originating domain. In my case, the proxy domain — donbrae.co.uk — needs to accept requests from browsers viewing jamieonkeys.dev.

To do this — and assuming your server runs Apache — add a .htaccess file in the gists folder on donbrae.co.uk:

<IfModule mod_headers.c>
  Header set Access-Control-Allow-Origin: https://www.jamieonkeys.dev
</IfModule>

The website at jamieonkeys.dev should now be able to call https://donbrae.co.uk/proxy-php/gists/fetch.php and get the data in return.

fetch-id.php: fetch content of a specific gist

There is also a proxy script, fetch-id.php, to get the content of specific gists.

Again, we start by including the same header.inc file we used previously:

<?php
require_once 'header.inc';

On the front end we’ll be passing a gist ID in the querystring parameter gist_id, so capture that:

$gist_id = $_GET['gist_id'];

The GitHub API returns a status code of 304 for gists which haven’t been updated since a specified date. Requests which result in a 304 also don’t count towards your rate limit. We’ll create a caching system to return static JSON files from our server so we don’t need to keep calling the API.

Add a subfolder called on your server under gists called gistscache.

fetch-id.php continues:

$cached_file_path = "./gistscache/$gist_id.json";
$cached_file_exists = file_exists($cached_file_path);

if ($cached_file_exists) {
  // Get last updated date of file in GMT
  $last_modified = filemtime($cached_file_path); // Get its last modified date
  $last_modified_gmt = gmdate('D, d M Y H:i:s', $last_modified) . ' GMT'; // Format it so GitHub API accepts it in header

  // Add date to header so that API will return 304 if no updates have been made, and we can serve the cached copy; otherwise we can expect a 200 with the latest data
  array_push($opts['http']['header'], "If-Modified-Since: $last_modified_gmt");
}

fetch-id.php always checks for a cached copy of the requested gist in the gistscache folder, and if one exists, gets the file’s last modified date and adds it to the headers we’ll pass to the GitHub API.

If the gist has been updated since that date, a status of 200 will be returned along with the latest data. If the gist hasn’t been updated, a 304 will be returned and we’ll serve the cached data.

Next we make the API call and store the status code:

$context = stream_context_create($opts);
$content = file_get_contents("https://api.github.com/gists/$gist_id", false, $context);

$status = getStatus($http_response_header);

Add then we handle the API response:

if ($status === 200) {
  echo $content; // Serve JSON returned from API
  writeFile($cached_file_path, $content); // Add/update cached copy
} else if (
  $status === 304 && $cached_file_exists || // Not modified and we have a cached version
  $status === 403 && $cached_file_exists // We've likely reached our API limit (403 == Forbidden Gist)
) {
  echo file_get_contents($cached_file_path); // Serve cached copy
} else {
  echo "{\"error\": \"Cannot fetch gist. Error code: $status\"}";
}

If we get a 200 — meaning the gist has been updated since the last updated date we sent in the request headers — serve the returned JSON and write a copy to the cache. If we get 304 — meaning the gist hasn’t been updated — serve a cached copy. If we get a 403, it means we’ve reached our API limit, so also serve the cached copy. In any other case, serve an error message in JSON format.

Folder structure

In summary, the gists folder on your proxy server should look something like this:

.
├── .htaccess
├── cached.json
├── fetch-id.php
├── fetch.php
├── gistscache
│   ├── 0276d43b831af40d1bbe529549a66f84.json
│   ├── 2277dd0362789957fd5ce9ed4894c93b.json
│   ├── 35841f68de35bce70ea1bb4cd71ac5d1.json
│   └── [etc.]
└── headers.inc

Building the front end

If you’ve not done so already, grab the code for the front end by cloning get-gists on your local machine:

git clone https://github.com/donbrae/get-gists.git

cd into the get-gists folder and run npm install.

Next, start a local server with npm run start. The page should open in your default browser.

Let’s dig into the code.

Installing a custom build of highlight.js as a npm dependency

We use highlight.js to add syntax highlighting to gists that contain code. By default highlight.js is over 1 MB in size, so to minimise the amount of JavaScript we serve to the user, I created a custom build with only the languages I need, namely JavaScript, HTML, CSS, PHP and Markdown.

You’d normally install highlight.js from the public npm registry via npm install highlight.js, but I instead followed these steps:

1. cd to your development folder and clone the highlight.js repo: git clone https://github.com/highlightjs/highlight.js.git
2. cd highlight.js
3. npm install
4. Run a build for just the languages we need: node tools/build.js javascript xml css php markdown (xml includes HTML)
5. The files we create are in ./build. Run ls -l ./build to list them
6. cd to the root of the project folder you want to use the custom build with (in my case, get-gists)
7. mkdir src to create a src folder
8. mkdir src/highlight.js to create a folder for the highlight.js custom build
9. cd src/highlight.js
10. Use cp to copy over the custom build we created in step 4, eg cp ../../../highlight.js/build/highlight.js ./
11. Run npm init to set the folder up as an npm dependency
12. Answer the prompts. Call it highlight.js and make sure entry point is set to highlight.js (it should be auto-selected)
13. cd ../.. to root of your project folder
14. Run npm install ./src/highlight.js
15. You should now see highlight.js referenced as a dependency in your project’s main package.json: grep --color=always -e "^" -e "highlight.js" package.json
16. highlight.js can now be imported from within a JavaScript file in your project: import hljs from "highlight.js";

index.html

You’ll see in the repo that index.html is a basic HTML page with a button — which, when clicked, will initiate a fetch() of the gist data — and an empty <div> where we’ll add the returned data. We also include the main script, index.js:

<!DOCTYPE html>
<html>
  <head>
    <title>Get gists</title>
    <meta charset="UTF-8" />
  </head>

  <body>
    <h1>Get gists demo</h1>
    <button id="get-gists">Get gists</button>
    <div id="gists" class="hide fade"></div>

    <script src="src/index.js" type="module"></script>
  </body>
</html>

index.js

Note that, here, the two fetch calls we make call the GitHub API directly, but in production these can be swapped for the proxy URLs.

First in index.js we import our CSS files and JavaScript modules:

import './styles.css';
import './xcode.css';
import hljs from 'highlight.js';
import markdownit from 'markdown-it';

• styles.css is some basic CSS for styling the page that lists the gists
• xcode.css is a set of CSS classes that style the code to look like Apple’s Xcode IDE
• highlight.js is our custom build of the code-highlightling module
• markdown-it transforms gists written in Markdown to HTML

Next we define a couple of functions. For security reasons, escapeHtml() sanitises any HTML that is returned by the API, replacing certain characters with their HTML entity equivalents:

// https://stackoverflow.com/a/6234804
function escapeHtml(unsafe) {
  return unsafe
    .replaceAll('&', '&')
    .replaceAll('<', '&lt;')
    .replaceAll('>', '&gt;')
    .replaceAll('"', '&quot;')
    .replaceAll("'", '&#039;');
}

show() handles element fade-ins:

function show(el) {
  el.classList.remove('hide');
  setTimeout(() => {
    el.classList.add('show');
  }, 30);
}

Next, the cfg object defines various properties, such as the GitHub user account we’re fetching gists for, and the IDs of any gists we wish to exclude from our listing:

const cfg = {
  githubUser: 'donbrae',
  hideIds: [
    // IDs of Gists to exclude from page
    '2369abb83a0f3d53fbc3aba963e80f7c', // PDF page numbers
    'bfbda44e3bb5c2883a25acc5a759c8fc', // Bootstrap 5 colour gradient
    'ab4e15be962602b1bf4975b912b14939' // Apple Music shortcuts
  ],
  perPage: 15, // Number of gists to fetch from API
  gistsLimit: 10 // Maximum number of gists to add to page
};

So that we can display the name of the month next to each gist, we define an array of month name abbreviations:

const months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];

Next, we have a regular expression to parse text for URLs so we can turn them into links:

const urlRegEx = /(\b(https):\/\/[-A-Z0-9+&@#%?=~_|!:,.;]*[-A-Z0-9+&@#%=~_|])/gi; // Only transform https URLs. Source: https://www.codespeedy.com/replace-url-with-clickable-link-javascript/

We also grab the HTML element into which we’ll place the returned gists:

const gists = document.getElementById('gists');

And then add an event listener to the ‘Get gists’ button:

document.getElementById('get-gists').addEventListener('click', getGists);

This calls our main function, getGists():

function getGists(e) {
  const btnGetGists = e.target;
  btnGetGists.classList.add('fade'); // Fade out button

  function error(err, container, e) {
    console.error(err);
    const escapedHTML = escapeHtml(`${err.status} ${err.statusText}: ${err.url}`);
    container.innerHTML = `<div>${err.status} ${err.statusText}: ${err.url}</div>`;
    show(container);

    const button = e.target;
    button.parentNode.removeChild(button);
  }

  fetch(
    `https://api.github.com/users/${cfg.githubUser}/gists?per_page=${cfg.perPage}`
  )
    .then(function (response) {
      if (response.ok) return response.json();

      return Promise.reject(response);
    })
    .then(function (data) {

      if (data.error) {
        console.error(data.error);
        return;
      }

      /**
       * Pseudocode block 1: show listing of gists returned by API
       */
      // Filter returned items to remove specific gists by ID
      // Loop through filtered gists
      //   Get created date
      //   Get description
      //     Convert any URLs within description to links
      //     Transform backticked text to <code> elements
      //   Create <div> for gist with date, description and 'Get gist' button
      //     Add to button the gist ID as a `data` attribute
      // End loop
      // Add gists to DOM
      // Fade in gists

      /**
       * Pseudocode block 2: show specific gist's content
       */
      // Loop through 'Get gist' buttons in DOM
      //   Add on-click function
      //     Fetch 'https://api.github.com/gists/<gist-id>'
      //       Get gist type
      //         If type is text/html
      //           Escape the HTML
      //         If type is text/markdown
      //           Run the content through markdownit
      //       Add to DOM
      //         If text/markdown
      //           Add gist content in a <div>
      //         Else
      //           Add to UI in a code block
      //           Run code block through highlight.js
      //   End on-click function
      // End loop
      // Fade in gist
    })
    .catch((err) => {
      error(err, gists, e);
    });
}

The fetch() function is the same in structure as the one we tested at the start of this post.

You’ll see a couple of blocks of pseudocode which run when the API call is successful. I find that writing such a high-level overview of the various steps in a process helps me think through a problem without getting bogged down in specifics. Now that we’ve outlined the main steps in pseudocode, we can convert them to real code:

// [...]
      /**
       * Show listing of gists returned by API
       */
      const items = [];

      // Filter returned items to remove specific gists by ID
      const dataFiltered = data.filter(
        (gist) => cfg.hideIds.indexOf(gist.id) === -1
      );

      for (let i = 0; i < cfg.gistsLimit; i++) { // Loop through filtered gists
        if (dataFiltered[i]) {

          // Get created date
          const date = new Date(dataFiltered[i].created_at);
          const dateFormatted = `${date.getDate()} ${
            months[date.getMonth()]
          }`;
          const verb = !i ? 'Created ' : '';
          const year = ` ’${date.getFullYear().toString().slice(-2)}`;

          // Get description
          let description = dataFiltered[i].description.trim().length
            ? `<div>${dataFiltered[i].description}</div>`
            : '';

          // Convert any URLs within description to links
          description = description.replace(urlRegEx, function (match) {
            return `<a href="${match}">${match}</a>`;
          });

          // Transform backticked text to <code> elements
          description = description.replaceAll(/`(.+?)`/gi, function (match) {
            return `<code>${match.slice(1).slice(0, -1)}</code>`;
          });

          // Create <div> for gist with date, description and 'Get gist' button
          items.push(
            `<div class="gist-container">
              <h2><a href="${
                dataFiltered[i].html_url
              }">${
              Object.keys(dataFiltered[i].files)[0]
            }</a></h2> <span class="dt-published">${verb}${dateFormatted} ${year}</span>${description}
              <button class="get-gist display-block button button-sm mt-1" data-gist-id="${
                dataFiltered[i].id // Add to button the gist ID as a `data` attribute
              }">Show</button>
              <div id="gist-${dataFiltered[i].id}" class="gist-content hide fade"></div>
            </div>`
          );
        } else break;
      }

      gists.innerHTML = items.join(''); // Add gists to DOM
      btnGetGists.parentNode.removeChild(btnGetGists); // Remove 'Get gists' button
      show(gists); // Fade in gists

      /**
       * Show specific gist's content
       */

      // Loop through 'Get gist' buttons in DOM
      const getGists = document.querySelectorAll('.get-gist');
      getGists.forEach((getGistButton) => {

        // Add on-click function
        getGistButton.addEventListener('click', (e) => {
          const gistId = e.target.dataset.gistId;
          const btn = e.target;
          btn.disabled = true; // Make sure user only triggers one API call
          btn.classList.add('fade');

          const el = document.getElementById(`gist-${gistId}`);

          // Fetch 'https://api.github.com/gists/<gist-id>'
          fetch(`https://api.github.com/gists/${gistId}`)
            .then(function (response) {
              if (response.ok) return response.json();

              return Promise.reject(response);
            })
            .then(function (data) {

              if (data.error) {
                console.error(data.error);
                el.innerHTML = `<div class="gist-content">${escapeHtml(data.error)}</div>`;
                btn.parentNode.removeChild(btn);
                show(el);
                return;
              }

              const gistName = Object.keys(data.files)[0];
              const gist = data.files[gistName];
              let gistContent;

              // Get gist type
              if (gist.type === 'text/html') { // If type is text/html
                gistContent = escapeHtml(gist.content); // Escape the HTML
              } else if (gist.type === 'text/markdown') { // If type is text/markdown
                const md = new markdownit('default', { html: true }); // Run the content through markdownit
                gistContent = md.render(gist.content);
              } else {
                gistContent = gist.content;
              }

              // Add to DOM
              if (gist.type === 'text/markdown') { // If text/markdown
                el.insertAdjacentHTML( // Add gist content in a <div>
                  'beforeend',
                  `<div>${gistContent}</div>`
                );
              } else {
                el.insertAdjacentHTML( // Add to UI in a code block
                  'beforeend',
                  `<pre class="code" role="code">${gistContent}</pre>`
                );
                hljs.highlightElement(el.querySelector('pre')); // Run code block through highlight.js
              }

              btn.parentNode.removeChild(btn);
              show(el); // Fade in gist
            })
            .catch((err) => {
              error(err, el, e);
            });
        });
      });
// [...]

Summary

And that’s it! To summarise, we:

• used the GitHub API to fetch a listing and contents of public gists for a specific user
• created two proxy server scripts to authenticate with the GitHub API and fetch the data
• added a CORS directive to the proxy server to allow our calling domain to request data
• implemented basic caching to avoid unnecessary API data fetches
• added a couple of npm modules to tranform gists written in Markdown to HTML, and highlight the syntax of gists comprising code. The syntax highlighting module was a custom-built npm dependency to avoid serving the user unnecessary JavaScript

It turns out, however, that the Web Audio API’s AudioContext interface has a new property that returns an estimate in seconds of the output latency. The property, outputLatency, is currently supported in newer versions of Firefox and Chrome¹, and allows us to delay starting the visuals and have them be in sync with the audio.

Testing it out

You can test it by running the code below in the browser console.

First check the latency while your audio output is set to the built-in speakers or a pair of wired headphones:

var audioCtx = new AudioContext()
audioCtx.outputLatency

I get 0 (zero) when I just checked in Firefox on macOS.²

Now set the audio output to a pair of Bluetooth headphones and check the latency again:

audioCtx.outputLatency

I get 0.17780041666666666 seconds.

If I then switch back to the built-in speaker it‘s 0.02485258333333333 seconds. I can’t explain that; feel free to leave a comment if you know the reason.

You can alternatively test it on an updated version of the animated keyboard (below). Output latency will be shown beneath the controls when you press Play.

Safari and older browsers

Because browser support is incomplete, when using outputLatency we should first check that the property exists:³

const hasOutputLatency = window.AudioContext && 'outputLatency' in window.AudioContext.prototype ? true : false;

if (hasOutputLatency) {
  console.log('AudioContext.outputLatency is available');
}

Regarding Safari, I see there is a fixme in the WebKit code, so hopefully the feature will be available soon on iPhones and iPads.

Further reading

There is some great technical background in this post by a Paul Adenot, an engineer at Mozilla who added outputLatency (along with baseLatency and getOutputTimestamp) to Firefox.

Related post: Fix for missing ‘AScore’ font when editing Logic Pro PDFs.

Calendar

The first is the cal program, which prints an ASCII calendar.

cal by itself returns the current month with today’s date highlighted.

cal 2022 returns a calendar for the whole of 2022.

cal -3 returns the current and surrounding months. This may be the most useful default view for me:

It also works for the past: cal 1981 returns a calendar for 1981. You can also capture just a month: cal may 1981.

The weather

Another user brought up wttr.in, a service which provides a weather report in ASCII format.

You can run a simple curl wttr.in to retrieve a forecast based on your location:

Assigning alias weather to curl wttr.in

To make the curl wttr.in command more memorable, I decided to assign the alias weather to it. Adding aliases can be useful in general, so I thought I’d document the process:¹

1. cd to your home directory: cd ~/
2. Open the file .bashrc in your preferred text editor (which is nano for me): sudo nano .bashrc
3. Add a new line with an alias: alias weather='curl wttr.in'
4. Save the change (Control-O in nano), then exit (Control+X)
5. Run source .bashrc to add the function to the current terminal session²
6. You can now run weather to get a weather forecast for your current location

Passing an argument for location

wttr.in also lets you query the weather in a specific location. I updated my weather command to allow for this:

1. In your home folder, run unalias weather to remove the alias we just set
2. Open .bashrc in your text editor
3. Delete the line with alias weather='curl wttr.in'
4. Add a weather() function (you don’t need the alias keyword):

weather() {
  if [ $# -eq 0 ] # If no argument has been passed to this function
    then
      curl wttr.in
    else
      curl wttr.in/"$1" # Append location
  fi
}

5. Save and exit
6. Run source .bashrc
7. Now, by adding a location argument to weather you can see what the weather is like in, say, Glasgow (weather glasgow+scotland) or New York (weather new+york).³ weather with no argument returns a forecast based on your location.

There didn’t seem to a solution that matched quite what I was looking for, so I decided to roll my own, a demo of which you interact with below.

Approach

I wanted the solution to be no more complex than it needed to be, so I started with a piano keyboard SVG I found on Wikimedia Commons. I extended the graphic to cover all 88 keys, rounded the key edges, and added some gradients.

My idea was to have the program play an audio file and update the fill property of whichever <rect>s in the SVG represented the keys that were being played.

MIDI to JSON

MIDI was the obvious way to get the music (which I played and recorded in Logic) in the form of note-on, note-off and velocity data. I used Tone.js (‘a Web Audio framework for creating interactive music in the browser’) and Tone.js Midi to covert MIDI file data into a JSON format that could be understood by Tone.js.

When the user clicks ‘Play’, the MIDI events are scheduled by Tone.js; then, when the audio file has loaded, the audio and MIDI files are played simultaneously.

There isn’t a synth hooked up to the Tone.js instance, so the MIDI file is, in effect, silent. Using Tone.js’s Draw class, you can schedule code to run on each MIDI event, and I use this to add and remove note-on class names to the relevant UI elements.

Audio

I use Howler.js, a library built on the Web Audio API, to handle the playing of audio. Here it plays the examples in either M4A or WebM formats, depending on the browser.

UX

The colour red is used to show what the right hand is playing, and blue the left. The brighter the colour, the higher the velocity (i.e. the speed with which the key is being depressed).¹

Additionally, a circle appears above each key as it’s played: a filled-in circle for the right hand, and a circle outline for the left. The circles have a ~1s fade-out to help orient the user. (By orient I mean that when a given note starts playing, there is an indication to the user as to which note or notes were played immediately before.)

The tutorials feature a musical chart (score), often with explanatory annotations, for each example. The site is aimed mostly at players who already have at least some experience of playing piano, and such folk can probably already read Western music notation. To these users the animated keyboards may have limited value.

Even if you can read music, however, seeing the velocities indicated visually might he helpful if you’re looking to improve your command of the jazz or funk ‘feel’. For this funk groove, for example, the colours could help users see which notes are being ‘ghosted’.

Technical details

Playing audio when an iOS device is muted

On iOS, the Web Audio API can’t produce sound if the device is muted. This is probably a feature (rather than a bug) to prevent websites from playing sound when the user doesn’t expect it.

In the case of JazzKeys.fyi, users are explicitly tapping a ‘Play’ button and so very likely want to hear something even if their mute switch is active.

Unlike the Web Audio API, the HTML5 <audio> element plays sound no matter a device’s mute state, and so we can work around the no-sound-on-mute issue by simultaneously playing a silent audio file via an <audio> element.

Code structure

In a typical JazzKeys.fyi tutorial, there will be several musical examples. I structured this in code by creating the class ItemAudioState, from which a new object per example is instantiated on page load:

class ItemAudioState {
  constructor(
    name,
    buttonPlay,
    loop,
    tempo,
    filePath,
    midi,
    numberOfKeys
  ) {
    this.name = name; // e.g. lick-blues-1. Taken from `data-name` on .button-play
    this.buttonPlay = buttonPlay;
    this.loop = loop; // true/false
    this.tempo = tempo; // integer/undefined
    this.filePath = filePath; // e.g. /audio/lick-blues-1/lick-blues-1-straight-120 (file extensions added later)
    this.midi = midi; // Path to MIDI file
    this.numberOfKeys = numberOfKeys; // Number of keys on keyboard
  }
}

State management

Each instantiated instance of ItemAudioState is stored in the state object, in the itemsState property:

const state = {
  init: false,
  itemsState: {}, // Instantiated `ItemAudioState` objects
  activeItemState: undefined, // Reference to active item's object in `itemsState`. Active item == currently playing item (set on Play button click)
  audio: undefined, // Main howler object
  playing: false,
  loading: false,
  playingItemObjectName: undefined, // Name of the item currently being played
  playCount: 0, // Incremented after end of each play of main audio file
  updateAudio: false, // Was a relevant parameter (e.g. tempo or rhythm) changed during playing that requires the main audio file playing to be updated?
  isiOS: undefined, // true/false
  htmlAudio: undefined, // For iOS
  htmlAudioSetup: false, // Flag. Background HTML5 element created
  htmlAudioPlaying: false, // Flag. Background HTML5 has started playing
  tonejs: {
    itemsMidi: {}, // MIDI files as JS objects
    visuals: []
  },
  timeouts: {
    // Store timeouts in state object so they can be cleared (and we can prevent multiple instances thereof being fired). All timeouts in this object are cleared by playStopHowl()
    loading: undefined // If audio taking more than a brief moment to load, show 'loading' indicator
  }
};

Listeners are added to various UI elements, such as the ‘Loop’ switch and ‘Tempo’ dropdown menu. When there is a change in value to one of those UI elements, a matching property in the relevant object in state.itemsState is updated.

A listener is also added to the body to capture any change that‘s made in the UI. This updates the audio file path based on changes to tempo and rhythm (whose values themselves are updated by the element-specific listeners described in the previous paragraph). It also captures changes to elements added dynamically, such as the ‘Bar dimming’ switch².

Querying the state

The state can be updated at any time, including while audio is being played and the keyboard is animating. The updated state will then be queried at the relevant point in the code, such as when the program decides whether the audio should loop and/or load new audio.

The state management also handles the scenario in which the user interrupts the playing of one example by clicking ‘Play’ on another example. Here, the object in state.itemsState which corresponds to the newly activated example will become the source of what audio is to play and what keyboard is to animate.

HTML data attributes

The id and various data attributes in the HTML associate the ‘Play’ button with its accompanying UI elements:

<div class="flex controls">
  <button class="button button-play" data-name="figure-bebop-6" data-tempo="#tempo6" data-chord_name="#chord6" data-loop="#loop6" data-annotations="#annotations6" data-midi="88|49">Play</button>
</div>
<div class="flex controls">
  <div>
    <label for="tempo6">Tempo (speed)</label>
    <select id="tempo6" class="select tempo" data-name="figure-bebop-6">
      <option value="140">140bpm (1x)</option>
      <option value="105">105bpm (0.75x)</option>
      <option value="70">70bpm (0.5x)</option>
    </select>
  </div>
  <div>
    <label for="chord6">Chords</label>
    <select id="chord6" class="select chord-name" data-name="figure-bebop-6">
      <option value="">None</option>
      <option value="C7|F-rh-140">C7 | Fmaj7</option>
      <option value="Gm7-C7|F-rh-140">Gm7 C7 | Fmaj7</option>
    </select>
  </div>
  <div>
    <label class="form-switch">
      Loop
      <input type="checkbox" class="loop" id="loop6" data-name="figure-bebop-6">
      <i></i>
    </label>
  </div>
  <div>
    <label class="form-switch">
      Annotations
      <input type="checkbox" class="annotations" id="annotations6" checked data-name="figure-bebop-6">
      <i></i>
    </label>
  </div>
</div>

That all elements for each example share the same name data attribute allows the JavaScript to target the relevant object in state.³

Get notified on launch

And there we have it: a simple animated piano keyboard built in JavaScript. You can view the demo and edit its code at this CodeSandbox.

If you’d like to be notified when JazzKeys.fyi launches, feel free to enter your email.

Related post: Fix for missing ‘AScore’ font when editing Logic Pro PDFs.

macOS native shortcuts and modifiers

Useful shortcuts already defined by the system.

Window management

• Command-Backtick: cycle current application’s windows
• Command-H: hide active application’s windows¹
• Command-Option-H: hide all windows expect those belonging to the active application’s^1:1
• Hold Option while resizing window: change window dimension(s)² around the window’s current centre point
• Hold Option-Shift while resizing window: change window dimensions around the window’s current centre point while maintaining aspect ratio

(See also Rectangle Pro.)

Special characters

• Hold $KEY in text fields to get pop-up menu of given character with diacritic markings
• Control-Command-Space bar: open Character Viewer (emoji and symbols picker)
• Option-Hyphen: insert en dash (–)
• Option-Shift-Hyphen: em dash (—)
• Option-Close square bracket: opening single curly quote (‘)
• Option-Shift-Close square bracket: closing single curly quote (’)
• Option-Open square bracket: opening double curly quote (“)
• Option-Shift-Open square bracket: closing single curly quote (”)

You can also search the Character Viewer (Control-Command-Space bar) by name, for ‘en dash’ etc.

‘Hidden’ functionality

Holding the Option key while clicking or hovering over a UI element can reveal additional functionality or information. Some examples: the Wi-Fi and Bluetooth menubar icons; the ‘Plus’ button when building a Finder search (allowing you to enhance the search logic); and application dropdown menus in the menu bar.

One I find useful is ‘Copy as Pathname’ in the Finder Edit menu:

Custom OS mappings

You can assign application keyboard shortcuts in System Preferences > Keyboard > Shortcuts > App Shortcuts:

• Control-Option-Command-M in Finder: run Window > Merge All Windows
• Command-Option-I in Preview: Adjust Size… from the Tools menu
• Shift-Command-Forward slash in all apps: ‘Show Help menu’ (to search menu items):

Rectangle Pro

Third-party window manager. Includes window snapping à la Microsoft Windows. The keyboard shortcuts I use:

• Shift-Command-Enter: current window to medium size, centred (custom shortcut)
• Command-Option-Enter: ‘Almost Maximize’
• Control-Option-Command-Enter: full screen, with some padding, centred (custom shortcut)
• Control-Option-Command-Left Arrow: ‘Left Half’
• Control-Option-Command-Right Arrow: ‘Right Half’
• Control-Option-Command-Down Arrow: ‘Center’
• Control-Option-Command-Escape: ‘Restore’

Karabiner-Elements

Third-party keyboard customiser that I use to open and switch between commonly used apps with a single keystroke. It’s generally more efficient than using a trackpad or mouse, Spotlight, LaunchBar or Command-Tabing.

I generally assign Control-Shift-$KEY to these.

• Control-Shift-U: launch/switch to Apple Music
• Control-Shift-C: Calendar
• Caps Lock³: Finder
• Control-Option-Shift-D: Downloads folder in Finder
• Control-Shift-F: Firefox
• Control-Shift-D: Firefox Developer Edition
• Control-Shift-G: Google Chrome
• Control-Shift-I: iTerm
• Shift-Caps Lock: Logic Pro
• Control-Shift-M: Microsoft Teams
• Control-Shift-N: Nova
• Control-Shift-S: Safari
• Control-Shift-P: Spotify
• Control-Shift-T: Terminal
• Control-Shift-R: Transmit
• Control-Option-Shift-V: Visual Studio
• Control-Shift-V: Visual Studio Code

Assign shortcuts by editing profiles[0].simple_modifications in ~/.config/karabiner/karabiner.json⁴. For example, to open Firefox Developer Edition:

{
	"from": {
		"key_code": "d",
		"modifiers": {
			"mandatory": [
				"left_shift",
				"left_control"
			]
		}
	},
	"to": [
		{
			"shell_command": "open /Applications/Firefox\\ Developer\\ Edition.app"
		}
	]
}

Or to open Finder (or whatever you have assigned as your default file browser):

{
	"from": {
		"key_code": "caps_lock"
	},
	"to": [
		{
			"consumer_key_code": "al_local_machine_browser"
		}
	]
}

Open several Finder windows at specific locations

I use Control-Option-Shift-J to open several Finder windows⁵ at locations relevant to my JazzKeys.fyi project:

{
	"from": {
		"key_code": "j",
		"modifiers": {
			"mandatory": [
				"left_control",
				"left_shift",
				"left_option"
			]
		}
	},
	"to": [
		{
			"shell_command": "open ~/Documents/dev/JazzKeys.fyi\\ process\\ files\\ \\(Hazel\\); open ~/Documents/dev/jazztoolkit/site/audio; open ~/Library/Mobile\\ Documents/com~apple~CloudDocs/Logic\\ projects/jazzkeys.fyi/Jamming; open ~/Library/Mobile\\ Documents/com~apple~CloudDocs/Logic\\ projects/jazzkeys.fyi/Licks\\ etc."
		}
	]
}

Third-party app-specific

Some third-party apps which I find useful enough to assign keyboard shortcuts to. The shortcuts are defined within the apps themselves.

I generally assign Control-Option(-Command)-$KEY to these.

• Control-Option-C: open Copy ’Em (clipboard manager)
• Control-Option-F: Dato⁶ (menubar calendar app)
• Control-Option-Command-Space: DevUtils.app (developer tools)
• Control-Option-Command-F: Dropzone (file management and general productivity booster)
• Control-Z: Menuwhere (get the Mac menu bar wherever your cursor is)
• Control-Option-Command-N: Quick Note > new note
• Shift-Command-2: Snappy (take screenshots which remain in front of other windows) > new snap
• Control-Option-Command-C: System Color Picker (OS-wide access to an enhanced system colour picker) > pick colour

(I also use aText for text automation/expansion, but that’s probably a separate blog post.)

I wrote the original prototype in 2013 using jQuery, but it’s now 2021 so the first task was to refactor the codebase to use vanilla JavaScript.² For the animations — previously handled by jQuery’s animate() function — I ended up using the Web Animations API. It’s still at Working Draft status, but widely supported. I actually began by using the top/left position properties to move elements, and specifying the animation transitions in the CSS, but the frame rate across devices was patchy. The switch to using the WAAPI and translate() CSS function — which leverages hardware acceleration — resulted in improved performance and more fine-grained control of animations.

Here’s a basic example of the WAAPI and translate() in action³:

See the Pen Web Animations API by donbrae (@donbrae) on CodePen.

The howler.js library is used for audio⁴, and canvas-confetti for the ‘win’ animation.

The game works with both pointer and touch input, and can more or less function as a Progressive Web App (PWA) on mobile devices by way of iOS-specific metatags, favicons, splash screen graphics (iOS only), and a manifest.json file. (Offline access still needs to be implemented.) When added to the home screen on my iPhone it pretty much behaves as if it was a native app.

I built it mainly using VS Code and Firefox Developer Edition, with GitHub for version control and Netlify for hosting and easy git-based deployment. (Oh, and I created the sounds in Logic.)

Overall, this was a challenging and enjoyable side project. I started my programming journey writing games in Sinclair BASIC, so the process has had nostalgia value alongside giving me a chance to hone my dev chops. For anyone who’s interested, the code is available to view on GitHub, and you can play the game here.