Twitter and Facebook last month suspended hundreds of thousands of accounts and operations that they said were part of a Chinese state-linked disinformation campaign designed to discredit pro-democracy protests in Hong Kong.
A few days later, Google followed suit, banning 210 YouTube channels that it said it had identified as part of a similar disinformation effort.
Twitter published a list of the most active 936 accounts it banned and more than 3.6 million of their tweets, but it has not detailed how it ascertained that the accounts were connected to the Chinese state.
In its statement, Twitter highlighted behaviors that allowed it to trace many of these accounts to mainland China — a cause for suspicion given the ban on Twitter within the country. A Twitter spokesperson declined to elaborate on how these accounts were identified.
Twitter did say that the suspended accounts were all used at some point to promote China's official narrative on Hong Kong's protests, which casts the mass movement as a willfully destructive mob pushing for regime change in Hong Kong.
NPR's data team and Beijing correspondent took a deep dive into the more than 900 accounts Twitter identified as linked to a Chinese state effort to discredit Hong Kong protesters. Here's what we found.
Tweets about, well, all sorts of things
The suspended Twitter accounts did not share exclusively pro-Beijing posts or content against the Hong Kong protests.
In fact, the accounts with the highest number of retweets or likes were about pornography. Tweets about cute animals also performed well.
Many of the accounts were created years ago — one as early as 2007 — and tweeted in various languages, including Indonesian, Arabic and English.
Bots for hire?
Given the diversity of content the suspended accounts shared, experts say it is likely that at least a significant fraction of them were either bots for hire or compromised accounts hacked from their original owners. NPR's research surfaced several instances where, after years of tweeting in one language, accounts suddenly began tweeting in Chinese in the last year and a half.
This thread, posted by a Chinese data scientist who asked NPR not to use his name because of political sensitivities, delves into one suspended account and the disinformation campaigns for which it was apparently used.
I've been having a lot of fun coming up with life stories for accounts suspended by Twitter. Buckle in for a wild ride.— Air-Moving Device (@AirMovingDevice) August 22, 2019
Meet 披荆斩棘 (@saydullos1d), from Cottonwood Colorado, a Denver suburb. She joined Twitter in 2013 and slowly built a following of over 21,000.
Accounts used repeatedly for different disinformation or promotional campaigns over the years could have been purchased through what's known as the darknet — the portion of the global Internet that is uncatalogued and accessible only through specialized browsers.
"In the same places where you can buy software exploits from Russian hackers and ransomware, there are also actors that offer what they call disinformation as a service: either building up an entity positively on social media or negatively, by tearing them down or spreading compromising information," says Priscilla Moriuchi, a research head at Recorded Future, a digital risk and intelligence company in the Boston area.
Stealing and repurposing social media accounts is also common, says Moriuchi: "After selling credit card numbers, selling leaked credentials on a massive scale is a pretty large business enterprise, and the credentials are really cheap."
Documents show that Chinese government agencies pay to acquire more social media followers. A tender posted on Aug. 16 by Chinese state-run outlet China News offers 1.25 million yuan ($176,900) to acquire more Twitter followers. Another government tender posted a few days later offers 750,000 yuan ($106,120) to acquire more Facebook and Twitter followers to support an exposition of Asian countries being held in China's Guangxi province in September.
Last year, The New York Times reported that China's state news agency Xinhua was among dozens of clients that paid a prominent bot factory to acquire Twitter followers.
Demonstrations in Hong Kong began on March 31 against a bill that would have allowed the extradition of crime suspects to mainland China, but they expanded into a movement to improve democracy and lessen the central Chinese government's influence in the city. On Sept. 4, Hong Kong's leader killed the bill, but protests continue.
NPR found activity from the now-suspended Twitter accounts increased after the Hong Kong protests began. But the accounts were most active more than two years earlier: right after the 2016 U.S. presidential election.
These accounts did not begin widely tweeting in Chinese until late 2017 — and many of the initial tweets were about Guo Wengui, a Chinese entrepreneur and fugitive now living in New York City who claims to have compromising information on key Chinese leaders.
Seeding accounts to sow dissent
"Seeding," or creating new accounts and developing their network and audience, is key to mounting successful disinformation campaigns.
"You can't instantaneously do [disinformation campaigns]," says Clint Watts, senior fellow at the Foreign Policy Research Institute in Philadelphia. "Every day they're probably seeding more accounts," Watts says. "You don't come out with an account and start broadcasting right away your influence message, because you don't have an audience."
Watts adds that seeding and using accounts too quickly can raise red flags. Diving into the data, we see a sudden surge of Chinese-language accounts beginning in late 2017.
Between December 2017 and February 2018, the majority of these new accounts tweeted in Chinese. From then on, posts in Chinese climbed from effectively zero to between 15% and 35% of tweets.
Engage and retreat
After these new Chinese-language accounts emerged, we see a corresponding uptick in engagement with them. Simply put, they were retweeted and liked much more than previous Chinese-language tweets by the banned accounts.
The most dramatic rise in such engagement happened in early December 2018. A series of protests organized by retired military veterans had broken out across China, embarrassing the ruling Chinese Communist Party, which has long drawn political support from veterans. Tweets discrediting the protests, along with reshares and likes of those tweets, quickly spread.
The more recent second spike in engagement with the now-suspended accounts occurred around June of this year, just as the Hong Kong protests were escalating.
In both events, tweet engagement spiked and quickly dropped off within weeks.
Historically, many of China's state-backed disinformation campaigns on English-language social media have used its many accounts to retweet state propaganda messages to give the illusion that such views are mainstream and widely supported, according to a Recorded Future analysis from earlier this year.
"With social media, a lot of what becomes acceptable is about volume. If you just say something enough times, it becomes truth on social media," says Recorded Future's Moriuchi.
"Will not cease"
Twitter faces an uphill battle against disinformation. The company has drawn broad criticism for failing to prevent these campaigns from spreading.
"These shutdowns are probably a very small percentage of what these countries actually have operating," Watts says.
In its statement, Twitter acknowledged one reality of disinformation campaigns: "It is clear that information operations and coordinated inauthentic behavior will not cease."