The Computational Limits of Deep Learning Are Closer Than You Think - Latest Govt Exam
Connect with us

Environment

The Computational Limits of Deep Learning Are Closer Than You Think

Published

on

Deep in the bowels of the Smithsonian National Museum of American History in Washington, D.C., sits a large metal cabinet the size of a walk-in wardrobe. The cabinet houses a remarkable computer — the front is covered in dials, switches and gauges, and inside, it is filled with potentiometers controlled by small electric motors. Behind one of the cabinet doors is a 20 by 20 array of light sensitive cells, a kind of artificial eye.

This is the Perceptron Mark I, a simplified electronic version of a biological neuron. It was designed by the American psychologist Frank Rosenblatt at Cornell University in the late 1950s who taught it to recognize simple shapes such as triangles.

Rosenblatt’s work is now widely recognized as the foundation of modern artificial intelligence but, at the time, it was controversial. Despite the original success, researchers were unable to build on it, not least because more complex pattern recognition required vastly more computational power than was available at the time. This insatiable appetite prevented further study of artificial neurons and the networks they create.

Today’s deep learning machines also eat power, lots of it. And that raises an interesting question about how much they will need in future. Is this appetite sustainable as the goals of AI become more ambitious?

Today we get an answer thanks to the work of Neil Thompson at the Massachusetts Institute of Technology in Cambridge and several colleagues. This team has measured the improved performance of deep learning systems in recent years, and show that it depends on increases in computing power.

Environmentally Unsustainable

By extrapolating this trend, they say that future advances will soon become unfeasible. “Progress along current lines is rapidly becoming economically, technically and environmentally unsustainable,” say Thompson and colleagues, echoing the problems that emerged for Rosenblatt in the 1960s.

The team’s approach is relatively straightforward. They analyzed over 1000 papers on deep learning to understand how learning performance scales with computational power. The answer is that the correlation is clear and dramatic.

In 2009, for example, deep learning was too demanding for the computer processors of the time. “The turning point seems to have been when deep learning was ported to GPUs, initially yielding a 5-15x speed-up,” they say.

This provided the horsepower for a neural network called AlexNet, which famously triumphed in a 2012 image recognition challenge where it wiped out the opposition. The victory created huge and sustained interest in deep neural networks that continues to this day.

But while deep learning performance increased by 35x between 2012 and 2019, the computational power behind it increased by an order of magnitude each year. Indeed, Thompson and co say this and other evidence suggests the computational power for deep learning has increased 9 orders of magnitude faster than the performance.

So how much computational power will be required in future? Thompson and co say that error rate for image recognition is currently 11.5 percent using 10^14 gigaflops of computational power at a cost of millions of dollars (i.e. 10^6 dollars).

They say achieving an error rate of just 1 percent will require 10^28 gigaflops. And extrapolating at the current rate, this will cost 10^20 dollars. By comparison, the total amount of money in the world right now is measured in trillions ie 10^12 dollars.

What’s more, the environmental cost of such a calculation will be enormous, an increase in the amount of carbon produced of 14 orders of magnitude.

The future isn’t entirely bleak, however. Thompson and co’s extrapolations assume that future deep learning systems will use the same kinds of computers that are available today.

Neuromorphic Advances

But various new approaches offer much more efficient computation. For example, in some tasks the human brain can outperform the best supercomputers while running on little more than a bowl of porridge. Neuromorphic computing attempts to copy this. And quantum computing promises orders of magnitude more computing power with relatively little increase in power consumption.

Another option is to abandon deep learning entirely and concentrate on other forms of machine learning that are less power hungry.

Of course, there is no guarantee that these new techniques and technologies will work. But if they don’t, it’s hard to see how artificial intelligence will get much better than it is now.

Curiously, something like this happened after the Perceptron Mark I first appeared, a period that lasted for decades and is now known as the AI winter. The Smithsonian doesn’t currently have it on display, but it is surely marks a lesson worth remembering.


Ref: The Computational Limits of Deep Learning. arxiv.org/abs/2007.05558.

Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Environment

Women in the newsroom: Why South Africa leads the way

Published

on

Mahlatse Mahlase is group editor-in-chief of Eyewitness News, a major broadcaster in South Africa. It’s a role she would have been unlikely to fill 30 years ago, under the apartheid regime, whose hierarchy put Black women on a bottom tier.

But no matter where you look in the world, women’s leadership in the news media has lagged – and still does. South Africa, in fact, is a relative success story. Today, women lead nearly half of the country’s major publications and broadcasters, according to a recent analysis by Oxford University. That puts it ahead of the United States, the United Kingdom, and even countries held up as models of gender parity, like Finland.

Yet South African women’s gains have been especially hard-won, and incomplete. Women hold few of journalism’s purse strings, for instance. 

Ms. Mahlase says she has reflected on representation through a new prism since she gave birth seven months ago to her first child, a boy. 

“I want my son to grow up in a world where he has female bosses,” she says. “I want him to grow up listening to women experts on the radio, hearing them as contributors to changing the world and not just as victims of an unfair system.”

Johannesburg

Nwabisa Makunga can point to the exact moment she knew she would become a journalist.

It was April 1993 and she was 11 years old, watching the TV broadcast of the funeral of assassinated anti-apartheid leader Chris Hani with her family in their living room. The presenter was a Black reporter named Noxolo Grootboom, and her powerful tribute to Mr. Hani brought Ms. Makunga’s parents to tears.

“She was a woman who looked like me, who spoke like me, telling the story of a man who had been so important to people from my community,” she recalls. “I said to myself – this is what I’m going to do. I’m going to be like Noxolo, and I’m going to tell stories.”

Three decades later, Ms. Makunga is the editor-in-chief of the Sowetan, one of South Africa’s most-circulated daily newspapers. She is part of a generation of women who have risen through the ranks to take leadership of many of the country’s most important news outlets. Today, these women – most of them Black, Asian, and multiracial – lead nearly half the country’s major publications and broadcasters, according to a recent analysis by Oxford University. That makes the country a global leader for women’s leadership in the media, putting it ahead of the United States, the United Kingdom, and even countries held up as models of gender parity, like Finland.

And the achievement is doubly remarkable in South Africa, where only a generation ago the entire country was run largely by white men. Editors like Ms. Makunga grew up not only with few role models who looked like them, but also in a world where the media was used as a weapon of the apartheid regime, premised on white supremacy – enforcing a hierarchy in which they were the bottom tier.

Alexander Joe/AFP/Getty Images/File

A vendor sells a Daily Sun edition honoring former South African President Nelson Mandela’s birthday on July 18, 2013. The country’s transition to democracy opened up opportunities for many of today’s journalists.

Hard-won gains

Even after apartheid formally ended, these editors came up in newsrooms where its effects lingered. They were told their accents were “too African,” or their looks “too tempting.” Even as the newsrooms around them began to change from mostly white to mostly not, they still found themselves edged out of the pub trips and golf games where their male colleagues networked their way to the top.

“Make no mistake: Those statistics changing are the result of a fight,” says Mahlatse Mahlase, group editor-in-chief of Eyewitness News, a major breaking-news broadcaster. Women in South Africa’s newsrooms “have fought racism and they’ve also fought the patriarchy. The discrimination came from every side.”

Although they’ve now arrived, many of these same leaders note that South Africa’s media transformation remains incomplete. Women hold few of journalism’s purse strings, for instance. Just 19% of the board members at media companies here are female, according to a 2018 academic study of women in the media called “Glass Ceilings.” The vast majority of the country’s major media houses have a male CEO.

And while the proportion of Black, Asian, and multiracial women in the top echelons of South African media companies has grown from 6% in 2006 to 30% in 2018, that number still lags, in a country where they represent 46% of the population, according to the Glass Ceilings survey. Once they get there, meanwhile, they’re often placed under intense scrutiny and pressure.

“When men get to the top of their profession, they’re celebrated. When women get there, particularly Black women, there’s backlash, there’s trolling, there’s bullying,” says Glenda Daniels, an associate professor of media studies at the University of the Witwatersrand in Johannesburg and the co-author of the Glass Ceilings study. She uses the term “Black” as it is frequently applied in South Africa, to describe anyone who is not white. “Women at the top find they’re often exhausted just proving over and over that they deserve to be in the room.”

“Newsrooms figured out they needed to mimic what was happening in society so they wouldn’t end up on the wrong side of history.” – Mapula Nkosi, editor of the Daily Sun

Coming up the ladder

In 1994, the same year Nelson Mandela took the oath as South Africa’s first Black president, an ambitious young reporter named Mapula Nkosi walked into the Johannesburg newsroom of the Weekly Mail, which made its name opposing apartheid, to begin her career as a cadet reporter.

When she looked up the ranks, she says, she saw only men. But all around her, the country had a new buzzword: transformation. The process was often fuzzy, but the objective was clear: The structures that ruled the country should look more like the people who lived in it.

“Those newsrooms figured out they needed to mimic what was happening in society so they wouldn’t end up on the wrong side of history,” says Ms. Nkosi, now the editor of the Daily Sun, the country’s most-circulated daily newspaper. “The whole time I was coming up I had people on one side of me saying, ‘We don’t understand why you’re in this position,’ and people on the other side saying, ‘We are so excited to see where you go.’”

Those competing forces followed many of South Africa’s women journalists through their rise. Each time they pulled themselves up another rung on the ladder, they heard whispers – that they only got this job because leadership wanted a woman, that they’d never make it.

But when they did, the conversation changed.

“When you do well, you become almost genderless,” says Paula Fray, a media consultant and the first woman to edit the Saturday Star, a major South African newspaper. “But if you don’t do well, then your failings represent your whole gender.”

Meanwhile, women leaders in the newsroom “carry the added responsibility of making these environments more welcoming for women,” says Mary Papayya, a veteran journalist and editor who now sits on the board of the South African Broadcasting Corp. That, she notes, is a particularly difficult task at a time when many outlets have seen their budgets slashed and are now contending with a pandemic that has made news reporting both more dangerous and less lucrative.

And the fight for change goes beyond who sits in the corner office. In South Africa, like many countries, men are disproportionately both the experts and subjects of news stories. A recent analysis of the South African media’s coverage of the COVID-19 pandemic by the organization Media Monitoring Africa, for instance, found that 80% of those quoted in stories about the virus were men.

Ms. Mahlase, the radio news editor, says she sees these kinds of statistics through a new prism since she gave birth seven months ago to her first child, a boy. 

“I want my son to grow up in a world where he has female bosses,” she says. “I want him to grow up listening to women experts on the radio, hearing them as contributors to changing the world and not just as victims of an unfair system.”

Source link

Continue Reading

Environment

Women in the newsroom: Why South Africa leads the way

Published

on

Mahlatse Mahlase is group editor-in-chief of Eyewitness News, a major broadcaster in South Africa. It’s a role she would have been unlikely to fill 30 years ago, under the apartheid regime, whose hierarchy put Black women on a bottom tier.

But no matter where you look in the world, women’s leadership in the news media has lagged – and still does. South Africa, in fact, is a relative success story. Today, women lead nearly half of the country’s major publications and broadcasters, according to a recent analysis by Oxford University. That puts it ahead of the United States, the United Kingdom, and even countries held up as models of gender parity, like Finland.

Yet South African women’s gains have been especially hard-won, and incomplete. Women hold few of journalism’s purse strings, for instance. 

Ms. Mahlase says she has reflected on representation through a new prism since she gave birth seven months ago to her first child, a boy. 

“I want my son to grow up in a world where he has female bosses,” she says. “I want him to grow up listening to women experts on the radio, hearing them as contributors to changing the world and not just as victims of an unfair system.”

Johannesburg

Nwabisa Makunga can point to the exact moment she knew she would become a journalist.

It was April 1993 and she was 11 years old, watching the TV broadcast of the funeral of assassinated anti-apartheid leader Chris Hani with her family in their living room. The presenter was a Black reporter named Noxolo Grootboom, and her powerful tribute to Mr. Hani brought Ms. Makunga’s parents to tears.

“She was a woman who looked like me, who spoke like me, telling the story of a man who had been so important to people from my community,” she recalls. “I said to myself – this is what I’m going to do. I’m going to be like Noxolo, and I’m going to tell stories.”

Three decades later, Ms. Makunga is the editor-in-chief of the Sowetan, one of South Africa’s most-circulated daily newspapers. She is part of a generation of women who have risen through the ranks to take leadership of many of the country’s most important news outlets. Today, these women – most of them Black, Asian, and multiracial – lead nearly half the country’s major publications and broadcasters, according to a recent analysis by Oxford University. That makes the country a global leader for women’s leadership in the media, putting it ahead of the United States, the United Kingdom, and even countries held up as models of gender parity, like Finland.

And the achievement is doubly remarkable in South Africa, where only a generation ago the entire country was run largely by white men. Editors like Ms. Makunga grew up not only with few role models who looked like them, but also in a world where the media was used as a weapon of the apartheid regime, premised on white supremacy – enforcing a hierarchy in which they were the bottom tier.

Alexander Joe/AFP/Getty Images/File

A vendor sells a Daily Sun edition honoring former South African President Nelson Mandela’s birthday on July 18, 2013. The country’s transition to democracy opened up opportunities for many of today’s journalists.

Hard-won gains

Even after apartheid formally ended, these editors came up in newsrooms where its effects lingered. They were told their accents were “too African,” or their looks “too tempting.” Even as the newsrooms around them began to change from mostly white to mostly not, they still found themselves edged out of the pub trips and golf games where their male colleagues networked their way to the top.

“Make no mistake: Those statistics changing are the result of a fight,” says Mahlatse Mahlase, group editor-in-chief of Eyewitness News, a major breaking-news broadcaster. Women in South Africa’s newsrooms “have fought racism and they’ve also fought the patriarchy. The discrimination came from every side.”

Although they’ve now arrived, many of these same leaders note that South Africa’s media transformation remains incomplete. Women hold few of journalism’s purse strings, for instance. Just 19% of the board members at media companies here are female, according to a 2018 academic study of women in the media called “Glass Ceilings.” The vast majority of the country’s major media houses have a male CEO.

And while the proportion of Black, Asian, and multiracial women in the top echelons of South African media companies has grown from 6% in 2006 to 30% in 2018, that number still lags, in a country where they represent 46% of the population, according to the Glass Ceilings survey. Once they get there, meanwhile, they’re often placed under intense scrutiny and pressure.

“When men get to the top of their profession, they’re celebrated. When women get there, particularly Black women, there’s backlash, there’s trolling, there’s bullying,” says Glenda Daniels, an associate professor of media studies at the University of the Witwatersrand in Johannesburg and the co-author of the Glass Ceilings study. She uses the term “Black” as it is frequently applied in South Africa, to describe anyone who is not white. “Women at the top find they’re often exhausted just proving over and over that they deserve to be in the room.”

“Newsrooms figured out they needed to mimic what was happening in society so they wouldn’t end up on the wrong side of history.” – Mapula Nkosi, editor of the Daily Sun

Coming up the ladder

In 1994, the same year Nelson Mandela took the oath as South Africa’s first Black president, an ambitious young reporter named Mapula Nkosi walked into the Johannesburg newsroom of the Weekly Mail, which made its name opposing apartheid, to begin her career as a cadet reporter.

When she looked up the ranks, she says, she saw only men. But all around her, the country had a new buzzword: transformation. The process was often fuzzy, but the objective was clear: The structures that ruled the country should look more like the people who lived in it.

“Those newsrooms figured out they needed to mimic what was happening in society so they wouldn’t end up on the wrong side of history,” says Ms. Nkosi, now the editor of the Daily Sun, the country’s most-circulated daily newspaper. “The whole time I was coming up I had people on one side of me saying, ‘We don’t understand why you’re in this position,’ and people on the other side saying, ‘We are so excited to see where you go.’”

Those competing forces followed many of South Africa’s women journalists through their rise. Each time they pulled themselves up another rung on the ladder, they heard whispers – that they only got this job because leadership wanted a woman, that they’d never make it.

But when they did, the conversation changed.

“When you do well, you become almost genderless,” says Paula Fray, a media consultant and the first woman to edit the Saturday Star, a major South African newspaper. “But if you don’t do well, then your failings represent your whole gender.”

Meanwhile, women leaders in the newsroom “carry the added responsibility of making these environments more welcoming for women,” says Mary Papayya, a veteran journalist and editor who now sits on the board of the South African Broadcasting Corp. That, she notes, is a particularly difficult task at a time when many outlets have seen their budgets slashed and are now contending with a pandemic that has made news reporting both more dangerous and less lucrative.

And the fight for change goes beyond who sits in the corner office. In South Africa, like many countries, men are disproportionately both the experts and subjects of news stories. A recent analysis of the South African media’s coverage of the COVID-19 pandemic by the organization Media Monitoring Africa, for instance, found that 80% of those quoted in stories about the virus were men.

Ms. Mahlase, the radio news editor, says she sees these kinds of statistics through a new prism since she gave birth seven months ago to her first child, a boy. 

“I want my son to grow up in a world where he has female bosses,” she says. “I want him to grow up listening to women experts on the radio, hearing them as contributors to changing the world and not just as victims of an unfair system.”

Source link

Continue Reading

Environment

Trump Administration Penalizes Chinese Officials for Hong Kong Crackdown

Published

on

WASHINGTON — The Trump administration on Friday imposed sanctions on 11 senior Chinese Communist Party officials and their Hong Kong allies, including chief executive Carrie Lam, over their role in cracking down on political dissent in the southern Chinese territory.

These are the first sanctions against officials from China and Hong Kong over suppression of pro-democracy protests and dissent in the territory. They are being imposed as actions following an executive order President Trump signed last month seeking to punish China for its repression in Hong Kong.

“The United States stands with the people of Hong Kong and we will use our tools and authorities to target those undermining their autonomy,” Steven T. Mnuchin, the Treasury secretary, said in a statement.

The action is another in a series of measures the Trump administration has taken in recent months to ratchet up pressure on Beijing. Last month, the administration imposed sanctions on the Chinese government, including a senior member of the Communist Party, over human rights abuses against the largely Muslim Uighur minority.

Beijing announced in June that it was imposing a new national security law in Hong Kong to grant security agencies expansive powers to crack down on dissent.

Since then, American officials have debated how to get Beijing to roll back the law or how to penalize the action.

Last month, Mr. Trump signed an executive order ending the special status that the United States grants Hong Kong in diplomatic and trade relations, saying Hong Kong was no longer an autonomous entity, and officials are now beginning to treat the territory like mainland China.

Some administration officials had wanted Mr. Trump to announce during the signing that his government was imposing sanctions on Ms. Lam and other officials in China and Hong Kong. The president declined to do so then.

The current flurry of actions against China dovetails with a core part of Mr. Trump’s campaign strategy. His campaign aides are trying to show Mr. Trump is hitting China hard in order to shift the national conversation from his failures on managing the coronavirus pandemic and the economy.

The initial virus outbreak began in central China, and Mr. Trump has blamed Chinese officials for failing to contain it, though he also praised China’s leader, Xi Jinping, for efforts this winter.

Some of the administration’s China hawks are trying to set the two nations on a course for long-term confrontation and ensure that relations remain in a state of open rivalry even if Democratic candidate Joseph R. Biden Jr. wins the November election.

Source link

Continue Reading

Trending