Walking Through Twitter: How to Mine a National Follow Network

Abstract

Twitter is used by individuals, grassroots movements, and political and social elites to directly communicate to the public and influence opinion. The platform appears relatively accessible to researchers because the majority of accounts post publicly and its API remains comparably open. However, after Twitter introduced restrictive rate limits in 2012 (Puschmann & Burgess, 2012), the global follow network stopped being accessible to the majority of researchers (for exceptions, see e.g., Myers, Sharma, Gupta, & Lin, 2014).
This development results in a lack of independent research on a key mechanism for information diffusion and a global infrastructure for influence. While it is widespread research practice to address this lack by using proxies for networks of attention on Twitter, such as mention, co-hashtag, or retweet networks (e.g., Himelboim, Smith, Rainie, Shneiderman, & Espina, 2017), all of these rely on active communication. Therefore, silent listening may be underrepresented.
This situation is aggravated by a change in how Twitter assigns account IDs. Until recently, its consecutive numbering scheme allowed a few independent, technically and monetarily costly projects such as the Australian Tracking Infrastructure for Social Media Analysis (TrISMA) (Bruns et al., 2016) to collect details of public accounts globally. Based on this, national follow networks could be captured and analysed (e.g., Bruns & Enli, 2018; Bruns, Moon, Münch, & Sadkowsky, 2017). However, Twitter closed this possibility by assigning account IDs at random, undermining further data mining efforts following this collection strategy.
Therefore, we present a large-scale test of a new sampling technique, building on the rank-degree method (Salamanos, Voudigari, & Yannakoudakis, 2017). As this walk-based technique only requires local information to sample a graph, we were able to adapt it as a data mining method for the follow networks of influential Twitter users, using the cost-free standard Twitter API. This approach has been tested on an ongoing collection of influential accounts in the German-speaking Twittersphere. So far, we have compiled a network of more than 200,000 accounts and 840,000 edges.
First results indicate that this network sample is suitable for investigating large-scale functional and topical communication structures in the Twittersphere. For example, we can detect topical clusters of prominent and influential accounts, or identify bridges between rather disconnected groups. Further, descriptive statistics show that the sample exhibits similar distribution patterns for indicators of influence and activity as a near-complete collection of German-speaking Twitter accounts from 2016. This is evidence that our sample represents an influential backbone of the German Twittersphere.
In light of the growing restrictions on the Twitter API, this shows that our new technique for gathering and analysing selected network samples is a valuable alternative to more brute-force approaches. Even though Twitter has made it almost impossible for independent researchers to capture comprehensive, large-scale follow networks, our method works around these restrictions to produce a robust perspective on the overall structure of these networks, for the German Twittersphere and other contexts. The code for this approach will be published under an open-source licence.