The aerial view of Canary Wharf in the opening sequence of BBC1's The Apprentice is the iconic image of Docklands, now fixed in the public consciousness. It's impossible to imagine the docks that Henry Mayhew witnessed in the 19th century, where the scents of tobacco, rum, coffee, spice and "the stench of hides" greeted the passer-by within a "forest of masts". Even the desolate wasteland in Bermondsey and Rotherhithe in the 1970s, just as the southside docks had given up the ghost, seems remote. Where the Design Museum now stands, alongside a colony of Conran restaurants, the late artist Derek Jarman once occupied an old grain warehouse, and scouted for coins and pilgrim badges on the shore below. That moment, before the builders and the bulldozers arrived, is documented in his Super-8 shorts of the time about Bankside and Shad Thames. Even by the mid-1980s, when I lived there, there remained suitable remnants of the past, between the skeletal warehouses fortressed by corrugated iron stained with graffiti ("Local land for local people") to bring a last-of-England theme to Jarman's The Queen Is Dead film for the Smiths.
Hints of the Dickensian past and the docks of Mayhew's days remained, when on hot summer nights spicy scents emanated from the creaking timber beams of wharves that had morphed into shell flats. It was there weeks after the Marchioness disaster in 1989 when fragments of records were washed up on the shore. I spotted two men scooping up the broken vinyl while wading into the water looking for the dead from the disco boat. "You get two and six if you find a body in the Thames," they told me. "It's an old river law."
At this point, the dockland areas on both sides of the river witnessed the transition of certain wharves into impenetrable apartment blocks with underground parking, and rumours of in-house gyms and swimming pools. The cash and the careers of those who occupied these conversions put them at odds with locals living in the social housing of the old London County Council flats that remained.
The creation of Canary Wharf, on which building began in 1988, moved the City further along the Thames into the East End. But the idea that this led to the area becoming colonised by "yuppies" is a myth that suited both those who needed a successful contemporary symbol of capitalism, and those who needed a folk devil to represent its evils.
As late as 1987, 29% of the residents on the Isle of Dogs were native "islanders" and another 40% came from elsewhere in east London. As Janet Foster, a lecturer at the London School of Economics, points out, many of the newbuilds were occupied by those with modest incomes who came "from a diverse range of backgrounds".
Foster spent much of the 1990s conducting research on the Isle of Dogs in east London, for the book Docklands: Cultures in Conflict, Worlds in Collision, published in 1998. "As a sociologist, I am interested in the underdog," she says. "When I started my research on the regeneration of London Docklands, I saw it as a straightforward story about the plight of powerless local people. It wasn't. It was a story about different worlds. I found within those worlds, by digging deep, some developers and planners with a social conscience who were passionate about Docklands."
Foster has returned to the subject as the evolution of Docklands has become relevant to the London Olympics, just three years away, and the Thames Gateway project. The developments within the former docks of east and south-east London were the first stage of the regeneration of the riverside – one that continues today. The Thames Gateway project extends the riverside regeneration from Docklands to Southend-on-Sea in Essex and the Isle of Sheppey in Kent. Those involved are hoping to learn from past mistakes. A native of the borough of Southwark, I lived in Rotherhithe in the 1980s throughout the redevelopment of its docks. The charge levelled at planners at the time was that they failed "to take the people with them".
This was central to the conflict that emerged between developers and the established communities – but time appears to have brought about a rethink from all sides. At least this was the outcome of a recent witness seminar, introduced by Foster, at Queen Mary, University of London. The event was organised by Eastside Community Heritage as part of a three-year project, Working Lives. Funded by the Heritage Lottery Fund, it sets out to document the memories of those who lived and worked within the Thames Gateway region, as part of an oral history archive. At the seminar, figures central to the work of the London Docklands Development Corporation (LDDC), the agency created by the Conservative government to implement change, reflected on its history. The corporation came into being in 1981 and was disbanded in 1998, when its powers were handed to local authorities.
"With the LDDC, the government concentrated its effort through one vehicle, and to some extent it worked," says Conor McAuley, a local Labour councillor since 1982, and chair of the London Thames Gateway Development Corporation's planning committee. "Today, there are too many regional and central government players."
Whatever the mistakes made by the LDDC, it is generally believed that its successes outweighed its failures.
"Looking at the skyscrapers, you feel like you're at the very centre of globalisation, as the docks were before for two centuries," says Jon Davis, a lecturer in postwar history at Queen Mary, University of London, who was raised in the East End. His family worked within the docks as carpenters and stevedores, and he is researching a book about Canary Wharf. "The LDDC legacy is undoubtedly mixed, but ultimately a tribute to dogged determination to do something," he says. "It is undeniable that the docks were dying and that the change to the whole industrial landscape of the East End was inevitable. The Royal Docks, like the wider Thames Gateway, need a bit of magic, a bit of uninhibited, high-minded, burning ambition. Then all of us to the east of London can be a part of the buzzy atmosphere of Canary Wharf."
An alternative view comes from George Pye, a fifth-generation "islander" and third-generation stevedore, who has been prominent in the local residents' action group since the 1970s. He believes one major mistake on the part of the LDDC was not keeping the enclosed docks to continue dock work. "The river has been used for thousands of years, and not using it is the greatest disservice to the community I know." Now an employee at Canary Wharf, he is resigned to the fact that "the machines, the computers, the banks have won".
I remember in Rotherhithe how many a former student, artist and activist would roll up and con their way into the warehouses purchased by housing trusts. Their "homelessness" made them a priority over south London natives on the waiting list.
Ironically, it would be these interlopers who became the most vocal, taking on the LDDC with their marching cries of "Local homes for local people". In the process, they grafted every lost cause – from Militant to Wages for Housework – on to a campaign that was born of the genuine grievance of white working-class locals, represented by the likes of Pye. Something similar occurred within the Labour councils that refused to work with the LDDC.
"Somebody needed to articulate the needs of the local communities to the LDDC and help it understand how its work could regenerate these areas," says McAuley. "The field was left clear to groups on the left and further left, who basically disagreed with the very existence of the LDDC, or to local resident groups who did actually engage with the corporation."
But equally, as he is quick to point out, it took a long time for the LDDC to understand that it needed to engage in social issues such as housing and education. Planners with a social conscience were the exception rather than the rule.
Throughout the 1970s, the 5,500 acres of former dockland was a bone of contention. The Conservatives saw it as a national resource for private and public investment. Labour planned to build council housing on the derelict sites, but appeared to have no idea how to regenerate the Docklands economically. Ultimately, it was the former vision that won out, bringing in international interest via Canary Wharf and City airport, and introducing transport links via the Jubilee line and the Docklands Light Railway to make the territory more accessible to residents and visitors.
It fell short by placing too much emphasis on the private sector and not staying true to its original promise to the public sector. This is something the government needs to address with the regeneration of the Thames Gateway today, says McAuley. "It can create the conditions for the private sector to regenerate the economy of the area, but it must step up to the plate on issues such as education, housing and health."
The monumental Canary Wharf is central to the story of the Docklands and its success, but it is not the whole story. Beyond it there remain pockets that still await regeneration and renewal. The current interest in its evolution should be a reminder that 30 years on it is no less a work in progress than the Thames Gateway or the London Olympics.