Charlie Chaplin, Mahatma Gandhi and the Eiffel Tower are all recognisable by their silhouettes. Early on in the British Museum’s sumptuous new exhibition, we are reminded that this is true also for the samurai. The kabuto helmet with horns or crescent. The sword hanging from the waist or else held aloft, ready to strike. The armoured horse, cantering into battle. One whole wall of an early room in the exhibition is given over to just such a scene: silhouettes moving through a purple-red dusk or dawn.

There’s a timelessness to this scene that captures the enduring place of the samurai in our imaginations. From Disney’s Shōgun and the computer game Assassin’s Creed: Shadows to kimono-clad sword-fighting hobbyists, our fascination with Japan’s warrior caste shows no signs of abating. Why is that? Surely it’s the intoxicating signature mix of romance, self-discipline, vivid display and unbridled brutality. Far removed as we now are from the samurai age, and with modern Japanese militarism 80 years in the past, we can enjoy all the fighting and intrigue with an easy conscience.

The curators of the British Museum’s new Samurai exhibition want us to enjoy far more than just this. They argue for a vision of the samurai that extends beyond enforcers and bloodletters. The samurai dominated Japan for more than half a millennium, and across much of that period the country was at peace. Armour stashed away, or placed lovingly on display, the samurai thrived as poets, artists, calligraphers and much else besides. This is how they would want us to remember them. For every great warlord like Oda Nobunaga – one of the great ‘unifiers’ of Japan in the 16th century – there was a poet like Matsuo Bashō or a dispenser of wise words such as Miyamoto Musashi (‘Think lightly of yourself, and deeply of the world’).

Strange to think that the samurai emerged in a country not hitherto known for military prowess. A thousand years ago, the emperor and his courtiers controlled most of Japan from their capital at Heian-kyō – now Kyoto. Japan’s ‘eternal city’ was the focal point of a famously refined culture. Pastimes included competitions to test people’s knowledge of tea and incense, and the sending back and forth between lovers of exquisitely-penned poems on fragrant paper. Readers of The Diary of Lady Murasaki or Sei Shōnagon’s Pillow Book often marvel at the close attention paid to clothing, from colours and grades of silk to the way that designs and combinations reflected the changing seasons.

These courtiers were known as ‘cloud-dwellers’, such was their removal from life in Japan as most people lived it. Aristocratic men hunted for sport, but there was little hint of a martial dimension to their culture. The enforcement of law and order, and defence against ‘barbarians’ living in Japan’s far north, was taken care of by an imperial army made up of conscripts. Morale seems rarely to have been high: to have a family member called into service meant losing that person’s labour while at the same time having to find the money to buy them weapons, armour and provisions.

Performance on the battlefield suffered as a result. These lines, attributed to Emperor Kanmu in around 789, after a particularly ignominious episode, hardly suggest a country on the verge of producing some of the world’s most celebrated warriors:

To advance not at all, and then to dismiss the troops – what kind of reasoning is there in this plan of Our generals? We know that it is because Our generals fear the ferocious rebels that they remain in garrison. They cleverly employ words to avoid facing their crimes. Nothing can surpass this in disloyalty…

It was around this time that the word ‘shōgun’ first came into use. The full title, Sei-i-Taishōgun, meant ‘barbarian-crushing generalissimo’. Its holder was charged with doing the emperor’s bidding, seeing off enemies and riff-raff while the monarch enjoyed a rarefied existence in Kyoto.

The ‘barbarians’ whom the hapless imperial army was occasionally sent out to fight included a group called the Emishi. They fought from horseback and were adept – terrifyingly so, to those forced to face them – at using a bow and arrow while riding at a gallop. It is from these Emishi that certain rural Japanese families, who established themselves as private security contractors across the 900s and 1000s, are thought to have learned a thing or two. Kyoto courtiers anxious about their provincial properties or their personal security made increasing use of such contractors. It is from the Japanese verb for serving or attending – saburau – that we derive the word ‘samurai’.

By the 1180s, two warrior families in particular – the Taira and the Minamoto, both offshoots of the imperial family – had become powerful enough, and sufficiently entangled in court politics, to compete with one another for political influence and to bring bloodshed to the streets of the capital. The result was a civil war between 1180 and 1185 whose victors, the Minamoto, adopted the title of shōgun. A role which had originally come with little executive authority, and every chance of a scolding from the emperor, now began to be used as a fig-leaf by military men with political ambitions. It was a means of wielding power while retaining the genteel and convenient fiction that final authority rested with the Emperor.

The Samurai exhibition compares the civil war of the 1180s with the Battle of Hastings or Agincourt. It was a foundational moment in Japanese history, marking the dawn of the medieval era and a long, eventful epoch of samurai rule.

Fighting was a major feature of that rule. A number of colourful scrolls on display here feature chaotic scenes of combat, with mounted warriors charging at one another across ground littered with arrows and stained with blood. Scenes like these were often inspired by the standout piece of literature from Japan’s medieval period. The earlier, classical era has become synonymous with Murasaki Shikibu’s The Tale of Genji (c. 1000), starring a handsome prince – Genji – as he encounters life’s possibilities and vicissitudes. Little more than two centuries later, The Tale of the Heike offered a dramatised – and occasionally very gruesome – account of the civil war. In one passage, we encounter the semi-legendary (perhaps entirely legendary) female samurai Tomoe Gozen chiding a battlefield opponent called Uchida for his lack of martial ability:

Does a man good enough to close with a woman draw his dagger as they fight and let her see it? Uchida, you know nothing of the old ways.

Then she clenched her fist and struck him hard in the elbow of his dagger arm. The blow was so powerful that the dagger fell out of his hand.

‘See… ’ she cried, ‘I am from a mountain village in Kiso, acknowledged the best in the land. I am the combat instructor you need!’

And she stretched out her left hand, seized Uchida’s faceplate, and forced his head down on to the pommel of her saddle; then she slipped her hand under the faceplate, drew her own dagger, wrenched the head round and struck it off.

Here was a new set of values and aesthetics in Japanese life: skill and valour, alongside self-discipline and self-sacrifice. Buddhism looms large, balancing out a respect for strategic nous and martial prowess with a conviction that everything depends, in the end, on karma. Familial bonds mattered deeply from very early on, in the form both of blood relationships and a hierarchy of loyal retainers. But what Europeans might recognise as a chivalric ideal was not formalised until the 1600s and 1700s. The period most closely associated with samurai in action, that of the Warring States (c. 1460s to 1600), was characterised instead by a mixture of rule-bound behaviour and the violent seeking of advantage by any means. A man like Oda Nobunaga, driven by a sense of destiny in uniting Japan under his own personal control, placed victory above all other concerns – earning himself the grim nickname of ‘Demon King’ in the process. Even the rules could be brutal – not least the taking of enemy heads for presentation to a commander as a gory invoice. Depending on the status of the deceased, generous rewards could be expected.

Samurai does a fine job of reflecting these realities while avoiding the temptation to reduce Japan’s warrior class to mere butchers. They were also social climbers, aspiring to – and sometimes comfortably matching – the refinement of court nobility. We encounter this fusion of art and war most vividly in armour design, and in particular the crafting of ostentatious helmets that reference the arts, mythology and the natural world. At a basic functional level, the job of a helmet was to protect the head while conveying a degree of pride and menace. Some were created with moustaches attached – useful for giving young and nervous men, perhaps treading a battlefield for the first time, the appearance of seasoned fighters. Armourers did not stop there, however. One of the helmets on display at the exhibition bears the beaked face and spray of hair of a tengu: a goblin-like figure from Japanese folklore. Another is shaped like a shark-fin, complete with engraved waves washing around the lower rim.

The Warring States period gave way, in the early 17th century, to more than 200 years of peace. Nestled as it is between preceding chaos and the mixed blessing that followed, of Japan’s opening to the modern world, the Tokugawa era (c. 1600 to 1868) is often recalled by Japanese as a golden age. This is certainly true of the first half of the period, when merchants and samurai together built a flourishing urban culture in cities like Edo (now Tokyo), Kyoto and Osaka.

Samurai has much to teach us about this period, rich as it was in artefacts of all kinds. One of the exhibition highlights is a flame-red women’s firefighting jacket, embroidered in silk- and gold-thread. Fires – euphemistically called ‘flowers of Edo’ – were one of the greatest threats to life in cities where the architecture consisted mostly of wood and paper. A rapid response was essential, to prevent winds whipping down narrow streets and fanning a local outbreak into a citywide conflagration. Samurai were charged with organising that response, women and men alike. This jacket features a torrent of fire-quenching water along the bottom, and is accompanied by a matching conical hat. It makes for a striking combination, almost as though control over fire were a shamanic power, requiring an outfit to match.

Anyone who has gone shopping in Japan for a single item and then found themselves beguiled by beautiful products and displays into maxing out the credit card will find evidence aplenty here that Japanese shopkeepers have always known how to conjure this outcome. We have smart lacquered travelling boxes, an incense game adorned with motifs from The Tale of Genji, and porcelain dishes featuring birds and chrysanthemums. We find signs, too, of samurai men and women reaching significant heights as artists in various Chinese-influenced styles. One of the images on display here shows a white cat blissfully asleep under the shelter of peonies. Another, by the celebrated samurai painter Chōbunsai Eishi, shows an imperial consort from ancient China seated on a throne festooned with flowers, playing a flute.

When not shopping, creating poetry and art, or engaging in scholarly discussions of Japan’s past and future, the samurai could be found in the pursuit of pleasure. Leaving their swords at the gate, they would enter the world of flesh and fantasy that was Yoshiwara: home to teahouses, theatres and courtesans. A handscroll, ink and colour on silk, allows us to peek inside a theatre teahouse at various scenes of relaxation, easy conversation and incipient romance. Not that many samurai would have regarded these things as shameful, needing to be taken care of in the dark. Elite courtesans travelled to appointments not via backstreets or in disguise but instead in full procession and in footwear designed to lift them above attendants and onlookers.

By the dawn of the 19th century, this world was coming to an end. Wealthy samurai, living the high life, were vulnerable to the perception amongst ordinary Japanese, villagers and townspeople alike, that they were freeloading: receiving generous rice stipends in return for – what, exactly? Penning poems? Carousing with courtesans? Much though later generations around the world may be glad that such a culture once existed, hard-pressed Japanese at the time struggled to see the point of such people.

Lower down the samurai hierarchy, meanwhile, families were struggling. They attended with care to the front of their homes, ensuring an impression to passers-by of cleanliness and prosperity. Inside, however, the tatami matting might be old and worn. Meals might be meagre. While merchants, in theory near the bottom of the social order, strode around in fine grades of silk and lived in great mansions, lower-ranking samurai eked out a living doing part-time jobs or selling off their families’ armour – even, in some cases, their daughters.

Unwelcome visits from westerners in the 1850s, firstly and most famously from the American Commodore Matthew C. Perry and his gunboats, brought these problems to a head and resulted in a short civil war in 1868-9. The victors were a handful of radical reformers, who quickly decided that though they themselves were mostly samurai, their class was now surplus to requirements. It was clear from looking across to Europe that the wars of the future would be fought using conscript armies wielding high-tech weapons. Samurai privileges, not least their stipends, were meanwhile unaffordable for a country with a long and expensive path to modernisation ahead of it.

A civil war brought the samurai to power, way back in 1185. Another, almost 700 years later, ended their supremacy. This was not, however, the end of the hold of the samurai over people’s imaginations – far from it. The final part of the Samurai exhibition explores that afterlife, beginning with Japanese attempts to mythologise them for political and cultural gain. A major contribution here was made by the writer Nitobe Inazō. In 1899, he published Bushidō: The Soul of Japan: an attempt to convince an international audience of the samurai’s virtues by comparing them with the European chivalric tradition.

Some in Europe were all too convinced. A striking wall poster by the Italian artist Gino Boccasile, made during the Second World War and reflecting Japan’s alliance with Italy and Nazi Germany, shows a monstrously large samurai standing in a turbulent sea and preparing to deliver the coup de grâce against two wrecked Royal Navy vessels far beneath him. It was inspired by Japan’s sinking of the HMS Prince of Wales and HMS Repulse in 1941. Nearby we find a German book, Die Samurai, likewise celebrating the martial virtues of fascism’s far eastern ally. In Japan itself, the European sabres used by the modern conscript army were replaced by samurai-style swords.

By the time the Second World War ended, few westerners were in the mood for samurai and swords. The leaders of a newly pacifist Japan preferred to sell their country abroad via its food and fine art instead. But as the final few exhibition displays attest, that state of affairs did not last long. Period dramas featuring the samurai became a staple of Japanese cinema, linked to an earlier and less globally controversial epoch of the country’s past. Western pop culture picked up on the trend: looming over visitors here is a model of Darth Vader, whose helmet – rounded top and flared base – was modelled on samurai armour. Not everyone notices George Lucas’ bit of borrowing here, perhaps because a visor was added to explain how Darth Vader would be able to breathe in space. Even the structure and characterisation in Star Wars owed something to the samurai: Lucas was inspired by Akira Kurosawa’s Hidden Fortress (1958), set in the Warring States period.

Critics have argued that this final section of Samurai feels a little like an add-on. But for all that it may lack standout exhibits, when compared with what came before, it feels essential for rounding out this epic story. All the more so since plenty of westerners actually encounter the samurai backwards, stumbling across their culture and aesthetics in a Star Wars film or a computer game and only later meeting them in their own time and place. Thanks to this nuanced and intelligent exhibition, packed with treasures, people now have brand-new points of entry into the world of the samurai. It is a remarkable achievement to have made that world, always at risk of hackneyed treatment, feel fresh, vital and real.