US President Donald Trump is well-known for conducting policy via Twitter. But does that mean the medium favored him in the first presidential debate? On the face of it, it seems like it could: Trump has made Twitter his primary outlet for policy statements, where his 82.4m followers dwarf Biden’s 6.4m. And Trump is notorious for setting Twitter buzzing with incendiary statements.

But notoriety on Twitter isn’t the same thing as influencing opinion — and influencing opinion is what matters in presidential debates. Texture AI took up the challenge of evaluating which of the two candidates' debate performances had the biggest impact on Twitter. 

Our initial hypothesis was simple: if a candidate’s message lodged with the Twitter audience, then tweets by that audience would echo the policy positions of that candidate. The question is, how can you measure something as subtle as the ‘echo’ of a candidate’s message?

The traditional solution would be to measure how many of the words and phrases used by a candidate were repeated in tweets about the candidate. However, this method has poor sensitivity: there are many tweets that will echo a message but do so in language that was not used by the candidate — especially in a short-form medium like Twitter, which penalises long communications. 

We solved this problem by making use of document embedding algorithms. These shallow neural networks can represent text documents numerically. They do this by capturing the pattern of the words that make up a document — as opposed to the words themselves. They are based on the supposition that words relating to the same concepts will be used in similar patterns. (For instance, the word CAT will be used in similar ways to the word FELINE.) The great advantage of document embedding algorithms is that they can be used to score documents for semantic similarity — and thereby provide a measure of echo strength.

Immediately after the debate, we harvested 650k tweets that mentioned either Trump or Biden. At the same time, we parsed the debate transcript into each candidate’s talking points on the main topics in the debate. We focused on the six topics that featured as key debating points: the Supreme Court appointment, the Covid-19 pandemic, the economy, the integrity of the election, Trump’s tax returns, and race.

The tweets, combined with the topics, gave us a corpus of texts on which to train our document embedding algorithm. The result was an algorithm that had ‘learned’ the semantic patterns in the data and could make precise, meaningful comparisons between tweets and candidates’ talking points. The results were unequivocal: Donald Trump outperformed rival Joe Biden in shaping opinion on Twitter with respect to the COVID-19 pandemic, race, and the Supreme Court appointment, while tweeters reproduced more of Biden’s message on election integrity.

Furthermore, when people tweeted about Biden the preferred topics were race and the economy; their preferred topics when they tweeted about Trump were the Supreme Court and the COVID-19 pandemic.

However, that is only half of the story. In addition to message echo, we took scores on how trusting and positive tweeters were when tweeting about each candidate. These measures are important because they measure attitude — and attitude is a better predictor of voting patterns. On both measures, Biden stole the show: he outscored Trump on all topics, with the only real contest being on the economy, where both were close on positivity.

Finally, we investigated whether the candidates differed in terms of the divisiveness of the language they prompted. Strongly divisive language uses aggressive terms like ‘destroy’ and ‘annihilate’, as well as profanity and terms of abuse. On every debate topic, tweets about Biden were more trusting and tweets about Trump were more divisive. For example, tweets about Trump were more likely to use inflammatory words like ‘attack’, while tweets about Biden used more trusting language, emphasising competence, positivity, and integrity.

Given the chaos of the debate, what can our analyses tell us about how people received the candidate’s performances? Three themes emerged very clearly. 

First, Trump was relatively successful at getting the content and tone of his messages to proliferate on Twitter. His talking points were echoed relatively strongly, and his divisive tone showed a clear signal in our analyses. Trump has made Twitter his primary outlet for policy statements, and this may have allowed his messages to land better, compared to Biden’s denser, more policy-oriented strategy of communication.

Second, although Trump is largely dictating the tone and content of Twitter discussion, he also seems to be less well liked on the platform. Former Vice President Biden’s lead in opinion polls is being reflected on Twitter - when he is tweeted about, more positive, trusting language is being used.

Third, new methods in natural language processing allow us to extract insights from unstructured text data at a speed and scale that has never been available before. Before now, analysing the semantics of over a half a million tweets would have required a team of human analysts working for weeks. With document embedding algorithms, this can be achieved in an afternoon.

The relative speed and breadth of our analyses allow us to offer timely and surprising insights into public discourse. Through assessing the vast majority of Twitter’s reaction to the debate, we were able to sample public opinion in a way that is unbiased by pollsters leading questions. Apart from being relatively younger and urban, Twitters users are broadly representative of Americans across the political and demographic spectrum. We believe these factors make our analyses well-placed and relevant in an ever shifting and increasingly chaotic geopolitical landscape.

For queries on Texture Echo, the technology underwriting this analysis, please contact us via the email address below.

Contact: human@texture.ai