Lady Williams of Crosby obituary

Shirley Williams, Lady Williams of Crosby, who has died aged 90, was one of the most influential figures in British social democracy in the latter half of the 20th century. The former Labour cabinet minister who defected to co-found the Social Democratic party, of which she then became president, later served as leader of the Liberal Democrats in the House of Lords after the merger between the SDP and the Liberal party.

A dedicated egalitarian, she was the minister responsible for the abolition of the socially divisive 11-plus examination in most education authorities. She was an internationalist with a distinguished academic record, and in her later years spent more than a decade as professor of elective politics at the John F Kennedy school of government, Harvard University.

Although one of the most highly achieving women in British politics and one of the most popular with the electorate, Williams was self-deprecating about her suitability for the top posts, something she ascribed to a lack of self-belief, common among women of her generation and earlier ones. While still a cabinet minister, she ran against Michael Foot for the deputy leadership of the Labour party in 1976 (after Harold Wilson stood down) but never stood as a candidate for the top job.

She was the first member of the SDP to be elected as an MP – at the Crosby byelection in November 1981, when she sensationally overturned a Conservative majority of 19,727 to win by 5,289 – but then in effect conceded the leadership of the new party to her co-founder, Roy Jenkins. Refusing to stand for the post of parliamentary leader herself, despite strong indications that she would have won against Jenkins, instead she nominated David Owen for the contest and accepted the role of president.

Williams’s biggest political mistake, by her own admission, came earlier. The SDP had been launched by the so-called Gang of Four, Williams, Jenkins, Owen and Bill Rodgers, in January 1981 in protest at the Labour party’s leftward drift. The new party attracted great public support, much of it from people who had not previously belonged to a political party, and the hope was that it might “break the mould” of British politics. But in July 1981 Williams refused an invitation to fight as the SDP candidate in a byelection at Warrington, Cheshire, a moderate Labour seat with a sizeable Catholic community reflecting both her own religious and political commitments.

Later she wrote: “I did not dither. I quailed.” At the time she cited family commitments and her fear of losing for a second time so soon after her high-profile defeat in her Hertford and Stevenage constituency in the 1979 general election, which swept Margaret Thatcher to power. She recognised her error swiftly and stood successfully in Crosby a few months later, but as she acknowledged: “My reputation for boldness, acquired in the long fight within the Labour party, never wholly recovered.”

Had she fought Warrington or stood against Jenkins as leader – and polls suggested she would almost certainly have won both – the popular momentum behind the SDP under her leadership might have been sustained and might have led to the sort of success for the nascent party in the 1983 and 1987 general elections that could have changed the course of British political history. Instead, Jenkins stood unsuccessfully at Warrington and then, having won in Glasgow Hillhead the following year, he secured the party leadership – only to prove hugely unsuited for the role.

The SDP failed to break through electorally and merged with the Liberal party to form the Liberal Democrats after the 1987 election. Williams lost her Crosby seat in 1983, and was president of the SDP from 1982 to 1988.

One reason why Williams alone held such authority over the fate of the new party was the extraordinary level of her personal popularity. “Shirl the Pearl”, as she was labelled, was adored, particularly by female voters and especially in Labour seats where the SDP hoped to prosper. There was tension within the new party between the progressive liberals, represented by Jenkins, and the social democrats, personified by Williams, but while she lacked the worldly eminence of Jenkins, her popular appeal and direct speaking style gave her a great advantage.

She was, however, partly destabilised by a whispering campaign that she was indecisive, incapable and disorganised. Throughout her life she had difficulty organising her personal timetable, a fault that partly arose from her generosity in making herself available to others but also reflected an inability to prioritise.

The somewhat puzzling belief that she was, in her own words, “not quite good enough” to lead the new party was despite the fact that she had already served for five years as a cabinet minister and had scarcely been off the Labour frontbench in her first 15 years in the Commons after being elected MP for Hitchin in 1964 (the constituency became known as Hertford and Stevenage after boundary changes in 1974).

It was also odd because she had been born into an intensely political and intellectual household, through which passed many leading politicians, authors and artists and a host of distinguished and highly assertive, self-confident female writers who were friends of her parents.

Shirley – who was named after Charlotte Brontë’s eponymous “gallant little cavalier”, a champion of social justice – was born in Chelsea, London, the second child of the political scientist George Catlin and the pacifist author Vera Brittain. The pattern of her life and many of its defining influences owed much to the legacy of their unusual and curious parenting.

When she was 15 months old she was pushed in her pram through the streets of Brentford and Chiswick, in the first of her father’s several unsuccessful attempts to become a Labour MP. He was an outstanding academic but gained no pleasure from his erudition or the recognition it afforded him, seeking only political power and access to those who could wield it. George was a conspicuously disappointed man, forever name-dropping and behaving with embarrassing sycophancy in the presence of the powerful.

When she was old enough to notice, Shirley would weep inwardly at his inability to distinguish between the great and famous and the great poseurs. Yet he bequeathed to his daughter the gift of intellectual curiosity and with it a certain courage, an audacity to dare – as a little girl she was encouraged by him to climb the bookshelves in his study right up to the ceiling – and, most importantly, an unquestioning belief in feminism.

This last was something her father had inherited from his own mother, whose commitment to radical politics had caused her to leave her husband, a Church of England vicar, and their son in order to work for an East End charity. His understanding of his mother’s driven beliefs was of huge importance to him, and was to prove a significant aspect of his attraction to Brittain.

She was to be the main breadwinner of the family, and her elegiac classic Testament of Youth, about her experience of the first world war, was a huge commercial and literary success when it was published in 1933, leading to considerable fame in Britain and in the US, where she undertook lecture tours. But she was a remote mother, a worrier who was always anxious and concerned, yet also somewhat distantly absorbed with other causes.

Her daughter understood from her earliest years that she came third in her mother’s list of priorities, after writing and Shirley’s brother, John, who was two years older and bore a striking resemblance as a child to Vera’s brother, Edward, who had been killed in the trenches. It was something that the little girl did not resent but, rather, developed into an understanding of the need for her to be independent.

She also rejected some of her mother’s other concerns. Brittain was always scrupulously tidy, organised and punctual. Her daughter gained a degree of notoriety in adult life for being none of those things – she had already rejected them as a child. While her mother loved shopping for clothes, Shirley learned to loathe the activity and was consciously, deliberately, careless about her wardrobe and her appearance. She was once shamed about her unkempt looks by being given a hairbrush by a teacher in front of a class of teenaged girls.

When her father introduced Shirley to Nancy Astor – the first woman to take her seat as an MP – and said that his daughter hoped to become an MP in time, the great woman took one look at her and pronounced: “Not with that hair!”

In the view of one of Shirley’s close friends, although she became devoted to her mother as an adult, she was nevertheless profoundly ambivalent about her and suppressed, perhaps even unknowingly, a deep resentment towards her. Her friends from her university years remarked that at Oxford she chose to be known as Shirley Catlin and did not take the hyphenated name of both parents, as her brother had done.

Yet this difficult childhood also produced a further defining characteristic that would in time prove an invaluable personal and political asset – a desire to please. She did not like to make enemies, as she would demonstrate by her reluctance to be as ruthless as necessity demanded in the later power struggles within the SDP. She could be a warrior, she could take up cudgels, as she did in some political battles over education policy, over Britain’s role in Europe and over the future of the Labour party, but she did not want to be unpopular.

It meant that she was sometimes less than straightforward with the entire truth, not because she was devious but because she sought not to hurt people’s feelings. It sometimes infuriated her friends, however, that she had a capacity for fantasy, often about trivial matters such as the reason for her being late.

Her pursuit of popularity also led to the evolution of a particular political style. She exuded reasonableness, both in public debate and in the course of an interview on radio or television. But there was a certain trick involved. She would appear to agree with what an opponent had asserted, by smiling and genially seeming to acknowledge the point, while she was in fact simultaneously putting an entirely contrary point of view, but doing it so pleasantly that she would already have won over the audience before she had finished speaking. It worked wonders, whether she was speaking to the Labour conference or on the BBC’s Question Time, where it won her a Most Frequent Panellist award.

There was a chameleon quality to Williams that perhaps came partly from a peripatetic education and partly from a lifetime’s travel, particularly in the US, which she had learned to love as a child. Not only did her mother visit the US, but her father taught at Cornell University, New York.

Her early childhood was spent in the cushioned comfort of Chelsea. She went to a smart private nursery school, Mrs Spencer’s Academy, followed by Christ Church elementary school. At the outbreak of the second world war, she was sent to boarding school in Swanage, but air raids over Dorset persuaded her parents to evacuate her and her brother to the US. “Send us your children,” said a telegram from a family in St Paul, Minnesota, whom Brittain had met on a lecture tour.

As a pacifist and a sponsor of the Peace Pledge Union, Brittain felt that she could not herself leave the country. In June 1940, Shirley and John were packed off, unaccompanied but duly labelled, through the dangers of the Battle of the Atlantic, on board the ocean liner Duchess of Atholl. It would be the overture for her to a lifetime’s love affair with the US, culminating in her second marriage.

Returning in 1943, Shirley was briefly a pupil at St Paul’s girls’ school in Hammersmith, west London, before it was bombed and she was sent to stay with acquaintances in Cambridge, where she was tutored privately. She then attended Talbot Heath girls’ school in Bournemouth, close to the family’s second home in the New Forest, which Shirley came to love.

By this time she had become active in the Labour party, joining the Labour League of Youth and, aged 17, visiting postwar Germany as a delegate to the youth wing of the German Social Democratic party, part of a programme to re-educate young Germans about democracy. Before going to Somerville College, Oxford, in 1948, where she had been offered an open history scholarship, she worked as a cowhand on a community farm in Essex – established by conscientious objectors during the war and supported by her mother – and as a hotel waiter in Whitley Bay, North Tyneside.

Her years at Oxford were dominated by politics and her second love, acting. Shirley had become interested in amateur dramatics in the US and was among those screen-tested for the lead part in National Velvet, for which the producers were seeking a 12-year-old horse rider with an English accent. Elizabeth Taylor was chosen.

Williams played Cordelia in a celebrated Oxford University Dramatic Society production of King Lear, directed by Tony Richardson, which toured the US in the summer of 1950 and was also staged briefly in the West End. The part of Lear was taken by Peter Parker (later the chairman of British Rail), with whom she was for a time in love. Among her other contemporaries was Roger Bannister, with whom she ran while he was training and partnered at a dance on the evening after he ran a mile in 4 minutes 7.8 seconds.

It was her political life that dominated, however. She had known famous politicians from childhood. As a teenager during the war, she sat next to Herbert Morrison, then home secretary, for more than two hours in an air raid shelter in Park Lane. He subsequently took her to lunch at the Commons and became her political mentor. At Oxford in the immediate postwar period, established politicians came to speak to the political clubs, many of whose members would themselves become political luminaries in their time.

In 1950 she became the first woman to chair the Oxford University Labour Club, which had 1,500 members and of which the treasurer was Rodgers. She took a second-class degree in philosophy, politics and economics and applied successfully for a Fulbright scholarship at Columbia University, New York, to study economics and trade unions.

After only eight months, however, she returned to the UK, having been selected for her first parliamentary seat, as the Labour candidate for the safe Conservative constituency of Harwich, which she contested unsuccessfully in a byelection in 1954. She lost the seat again in the general election in 1955 and fought Southampton Test unsuccessfully in 1959 before her selection for the winnable seat of Hitchin, in Hertfordshire.

In the meantime she worked as a reporter for the Daily Mirror (1952-53), followed by five years at the Financial Times. In 1955 she married Bernard Williams, a philosopher whom she had first met at Oxford. In 1958 she left the FT and taught briefly in Ghana, where her husband had been invited to teach for a term.

Back in Britain once again, she became deeply involved in Labour politics and in 1960 succeeded Rodgers as general secretary of the Fabian Society. It provided her with a superb vantage point for the battles between left and right within the Labour party, to study policy issues and to forge friendships that would prove vital throughout her political career.

These apprenticeships paid off. Two days after her election as an MP in 1964, she was appointed as a parliamentary private secretary to the health minister, Kenneth Robinson. In 1966 she became a junior minister at the ministry of labour, the “bed of nails” as it was then termed by the minister, Ray Gunter. During the National Union of Seamen’s strike that year, Gunter was taken ill and Williams found herself reporting to the cabinet.

The following year she was rewarded with promotion to education minister, which brought her face to face with one of the bigger controversies of her career. She oversaw the introduction of comprehensive schools, including a policy that linked future funding to a local education authority’s plans to cooperate. No new money was made available for selective grammar schools.

The policy was implemented successfully – although resisted by a handful of authorities, which still today retain pupil selection through the 11-plus examination – but it continued to be a source of vicious personal criticism made of Williams. She remained unrepentant and in later years was a vociferous opponent of the academies programme introduced under Tony Blair’s Labour administration.

In 1966 she became a Home Office minister with responsibility for prisons. Four years later, after Wilson’s defeat to Edward Heath, she secured election for the first time to the shadow cabinet and also to the Labour party’s national executive committee, thanks in part to the respect she commanded among the trade unions, the leaders of which liked her warmth and natural egalitarianism. She was opposition spokeswoman on social services from 1970 to 1971, when she became shadow home secretary.

In 1973 she was moved to prices and consumer protection, and, on Wilson’s re-election as prime minister in 1974, entered the cabinet with a daunting brief as the secretary of state for that department at a time of huge oil price increases and with annual inflation running at 13.5%. When James Callaghan succeeded Wilson in 1976, he moved her back to education as secretary of state, a role she combined with being paymaster general.

Williams was already unpopular with the Conservative press because of her championing of comprehensive schools, and she compounded her vulnerability to rightwing media hostility in 1977 by joining the picket line and briefly being arrested during the Grunwick trade union dispute. It was a typically supportive act by Williams to demonstrate her backing for her own trade union, Apex (the Association of Professional, Executive, Clerical and Computer Staff), and for poorly paid Asian female workers, but the subsequent picket line violence led to furious press criticism.

By this time the Labour party was already descending into internal warfare over its own way forward, its approach to Europe, the infiltration of Trotskyites and a series of demands from the left for constitutional reform within the party. Williams had been associated with the Campaign for Democratic Socialism in the early 1960s as a result of two speeches she had made at the Labour party conferences of 1960 and 1961, the first criticising the party’s single-minded pursuit of nationalisation and, the following year, endorsing the principle of the emerging European common market. In 1971, along with 68 other Labour MPs, she had voted against a three-line whip in the Commons in the vote on Britain’s entry into Europe.

By 1978, Williams was profoundly concerned by what she saw as the party’s relentless drift to the left and was the first cabinet member to speak out against the Militant tendency. She was under attack in the press, partly because of her undiminished support for comprehensive education, and was subjected to an aggressive personal attack by the Sun shortly before the 1979 election, which had been forced on Callaghan by the so-called winter of discontent and the loss of a vote of confidence over Scottish devolution. Williams sued the Sun, which settled out of court, two days after she had lost her Commons seat as Margaret Thatcher swept into Downing Street.

Until this time, Williams had resisted any suggestion that she could have a role in a new centre party. She had memorably declared that such a party would have “no roots, no principles, no philosophy and no values”. But after the 1980 Labour conference, at which she spoke vehemently against the “red fascism” that was displayed by the buoyant left in the conference hall, she agreed with her fellow co-founders of the SDP to establish a Council of Social Democracy.

The 1981 Limehouse declaration, made outside Owen’s home in the East End of London, led to the launch of the new party, and while it was a heady time for the Gang of Four, there was no doubting the difficulty Williams felt personally in leaving Labour. She had at the time already been reselected as the Labour candidate for her former seat in Hertford and Stevenage and was also chairing the Fabian Society. She would admit privately in later life that in her heart, she still continued to identify strongly with the Labour party, within which she had worked so hard for so long.

From 1988 to 2000 she was a politics professor at Harvard University, and was created a life peer in 1993. She served as the leader of the Liberal Democrats in the House of Lords from 2001 to 2004.

Williams’s daughter, Rebecca, now a lawyer, was born in 1961, and she later helped to raise the orphaned children of her brother and a stepdaughter of her second marriage. Her marriage to Bernard Williams was happy enough, but suffered from the inevitable difficulties of two ambitious people pursuing different careers. Life was never easygoing with her first husband, a restless, witty, attractive, sometimes cruel intellectual. Neither was he faithful, and in 1970 he left her for the wife of one of his academic colleagues. The marriage was dissolved in 1974.

She later fell in love with a friend, the journalist and political scientist Anthony King, himself a widower, and they would have married but for Williams’s profound Roman Catholic faith. She sought an annulment of her first marriage which was submitted to more than one ecclesiastical tribunal and was not granted until after she and King had parted in 1976. In 1987 she married Richard Neustadt, by then also a widower, whom she first met 15 years earlier when he was a professor of government at Harvard. Neustadt died in 2003, following a fall.

In the course of her long career, Williams amassed garlands of academic honours, fellowships and honorary degrees. She was an honorary fellow of Somerville College, Oxford, and of Newnham College, Cambridge. After the collapse of communism, Williams became involved in helping the emerging democracies of eastern Europe and was a UN representative to the former Yugoslavia. She helped draft constitutions for Russia, Ukraine and South Africa and continued to travel widely.

When Gordon Brown became prime minister in 2007, she became an adviser on nuclear proliferation. She wrote a number of books including Politics Is for People (1981); Unemployment and Growth in the Western Economies (1984); A Job to Live (1985); Snakes and Ladders, a Political Diary (1996); and an autobiography, Climbing the Bookshelves (2009).

She continued to play an active role in the House of Lords during the Lib Dem-Conservative coalition government, and retired from the chamber in 2016.

She is survived by Rebecca.

• Shirley Vivian Teresa Williams, Lady Williams of Crosby, politician and academic, born 27 July 1930; died 11 April 2021