It took Bob Dylan six days to record Blood on the Tracks. He liked to move fast: it was meant to take four. Sessions started in New York on 16 September 1974 with a band of musicians that the producer, Phil Ramone, had hired earlier in the day. They ran through 30-odd takes of ten different songs, with Dylan sometimes showing them the chords to the next song while the playback of the last was still running. The next day Dylan fired all of the band bar the bass player.

By the 19th, he was done, and Dylan’s label, Columbia, started planning for a late December release. Covers were printed, records pressed, but over the next few months, Dylan grew less and less happy with the finished songs. His unhappiness crystallised over conversations with his brother David at Christmas in Minnesota. Dylan reportedly called Columbia to delay the album’s release the day before it was due to ship. Over two days in late December – the 27th and 30th – he went into a studio in Minneapolis and re-recorded five of the songs with a group of local musicians hurriedly corralled by his brother; one of them even had a day job on the railroad.

At the time, Dylan was thought to be in a creative rut – perhaps burned-out entirely. Critics had been baffled by the string of albums that followed Blonde on Blonde in 1966; Greil Marcus in Rolling Stone opened his review of 1970’s Self Portrait by asking ‘What is this shit?’. Dylan, a man who knows how to nurse a grudge, waited 43 years for his revenge: in 2013 he hired Marcus to write more appropriately rapturous liner notes to the expanded archival release Another Self Portrait.

Whatever kind of rut he was in, it wasn’t a commercial one, and perhaps the public were wiser than the critics. Almost all those albums hit the top five in the US, and the most recent, Planet Waves, released in January 1974, had hit number one. It coincided with a two-month tour of the States alongside longtime collaborators The Band, Dylan’s first for eight years – a long time now, but an eternity for a rock star so early in the genre’s lifespan. Over 12 million people reportedly applied for tickets. Dylan was back; but he still had something to prove.

The suite of songs that he wrote for what would become Blood on the Tracks came together over the summer of 1974 when Dylan was living on his 100-acre farm in Hennepin County, Minnesota. Sara Lowndes, his wife of nine years, wasn’t with him; Ellen Bernstein, his 24-year-old mistress of four or five months, was. The latter relationship drifted to a close early the following year and Lowndes would eventually file for divorce in 1977, so the record has usually been approached through the lens of a ‘break-up album’. There’s a lot of truth in that, but it’s only part of the picture. Really, Blood on the Tracks is a kind of gazetteer of heartbreak: emotions and emotional responses alike, packed together, jostle one another – regret and longing, flight and abandonment, rage and acceptance, loyalty and betrayal, memory and hope. At least some of the album’s extraordinary vitality comes from the complex and competing psychological forces it somehow holds in balance.

The songs were all initially written in the same key using an open-E tuning. That is, the unfretted strings sound the notes E-B-E-G#-B-E. It’s a tuning most commonly found in blues guitar. In Dylan’s hands, almost all the chords are heavily flavoured with the tones generated by the choice of tuning. The high E string, for example, is almost always open throughout every song, nagging away at the top of almost every chord, insistently the same, an aural hint of limitation and constraint.

It’s with this tuning that the songs were recorded in New York. This might sound a minor technical point, but Dylan’s use of the open-E tuning speaks to the songs’ emotional wellsprings and perhaps reinforce the sense of deep introspection, if not self-involvement, of those recordings. In New York, Dylan sounds like a man trapped in inescapable torment, not least because his musical choices limit his possibilities. Listen to songs like ‘Simple Twist of Fate’ and ‘Shelter From the Storm’, recorded in New York and issued on the original LP: Dylan’s guitar is cycling through chord changes, but the sense of motion that you usually get from chord progressions is weakened, compromised. For all the movement, the ever-present open high-E reinforces a sense of emotional stasis or suspended animation: everything feels unresolved, you might say. In the same way, while the songs’ various personae are always in motion, travelling – hunted and hunting, searching and hoping – they are confronted everywhere with borders and limits, forced to wait for something that implicitly will never arrive.

Dylan in the early 1970s was the most bootlegged artist in the world, and it wasn’t long after the release of Blood on the Tracks on 20 January 1975 that illicit copies of the original test pressing began circulating. Many listeners prefer this version: Alex Ross, music critic of the New Yorker, thinks it comparable in its beauty and bleakness to Schubert’s Winterreise song cycle. For such listeners, Dylan’s decision to re-record five of the songs was a creative mistake. Certainly, if you like your melancholy 100-proof, the New York sessions are the ones for you.

Who knows why he actually changed his mind: the received wisdom is that he got cold feet about the sparseness of the finished record, but it’s not a coincidence that when Dylan did follow his brother’s advice, he approached the songs in standard tuning. The album’s emotional palette is enormously expanded by that choice, reinforcing the narrative multiplicity of the lyrics and reflecting a sense of generosity and release that is absent from the earlier recordings. Blood on the Tracks becomes less of a study of personal disintegration – although it remains that, too – and more of a generational study of a cohort who were now no longer young and who found themselves heading out into the unknowns of mid-life haunted by regret and hounded by failure. It’s an effect accentuated by the restless, shifting pronouns of arguably the album’s greatest song, ‘Tangled Up in Blue’, one of the five Minneapolis recordings.

When Blood on the Tracks was finally released, the critical reception was uncertain. It ‘would only sound like a great album for a while’, Jon Landau wrote in Rolling Stone. The public, more right than the critics, as before, took it to number one. Fifty years on, for all that its acoustic instrumentation looks back to Dylan’s earlier work, it still stands apart in his catalogue: it doesn’t quite belong to any of the distinct periods which mark his output. In its conception and concerns, it is Dylan at his most self-consciously artful, but that artfulness is in the service of the rawest and most unprotected writing of his career. For all the dazzling shifts of perspective, it’s the sound of an artist bereft of any defences but his art. And if the album’s title seems to beg the questions, whose blood, whose tracks, the answers ultimately come in the compromises and defeats, the defiant woundedness and persistence of the songs: all of ours, in time.

‘All you can do is do what you must do, and do it well,’ Dylan sings in the last verse of the last song on the record. He’s never done it better than this.