Enhancing the ‘Intelligence Studies Network’ website
The ‘Intelligence studies network’ website is now 1 year old (created in December 2022)! Initially, I created this site as a testing ground to improve my Python and data analysis capabilities. As soon as I learn new stuff, I tell myself ‘why shouldn’t I try to implement it on the website’. Over time, this has turned from a playing ground to a proper website for the sources of intelligence studies. So, we are where we are now! (Disclaimer: I don’t have a developer background. So, I am wholeheartedly open to your criticism).
In the last two months, the website has transformed with new sources and functionalities (apparently, the passive voice decided to be my instinct voice and the latent hero which doesn’t reveal the fact that this is not high on my priority list at all. You’ll understand what I mean if you know me!).
Sources
Initially, I was thinking of adding academic publications only. But when I read really good long reads on intelligence history and studies, I thought other types (blog posts, newspaper articles, podcasts etc.) should also be added.
In addition to new functionalities, I have also included more sources in the Zotero group library which is the main database of the website for publication sources, with more than 1000 sources being added since December (what a nice Xmas break!). The library now covers more sources on ‘global intelligence’ (a term used to identify publications looking at non-UK-USA intelligence topics). The sources are still manually curated by using semi-automation methods. Although every effort is being made, the library might be far from complete. Sources will continue to be added.
New home page
Search keyword(s)
New search functionalities are now available on the home page. Users are now able to search keywords, authors, collections, publication types, and publication years. For example, you can search for a keyword or a phrase in titles or abstracts of sources (quotation mark is recommended for phrases).
When you search for a keyword or phrase, the sources are listed after which users can filter by publication types or download the search result as CSV. Furthermore, there is an option that allows you to generate a dashboard based on the search results. The dashboard will display the number of publications by type (e.g. journal articles, books, or book chapters) and publication year. It also shows the author names and number of publications as a bar chart. One of the useful visuals is the Wordcloud generated from titles (I admit in advance that there are things that still need improving).
Search author
The home page also has a dedicated section for searching authors in the field of intelligence studies. To disambiguate different name variations of the same author, a code snippet is applied. Similarly, users can generate a dashboard for one author’s publications.
Search collections
The Zotero library has different collections each of which represents a research field in intelligence studies, such as intelligence history, covert action, counterintelligence, or global intelligence. One source may be associated with more than one collection. The collections are manually curated as the process is not straightforward enough to create a machine-learning algorithm (I’m very open to suggestions).
Publication types
You can also search for a specific publication type within the collection. This might be good if you would like to concentrate on peer-reviewed sources such as journal articles or to find theses. Similar to other search options, a dashboard can be generated.
Publication years
One of the interesting features is that you can filter down items in the library to a specific year or period. This can be quite useful to see what the trends were and which topics were discussed in a selected year. For example, blog posts were quite popular following journal articles in 2023 among intelligence scholars to engage with their audience. Also, according to the library data, 2023 was the most productive year so far in terms of publications on intelligence studies.
Surprise me
This section is designed to suggest 5 random sources in the library. If you are a real intelligence geek and follow every publication timely (or a follower of Dan Lomas who is doing a marvellous job in sharing publications on intelligence), this part may not surprise you.
Dashboard
There is nothing new in this section. You can explore the dashboard to see interactive visuals and find more information in my previous blog.
Global intelligence library
The global intelligence section of the library has also been updated. Countries in titles are automatically extracted and listed on the page to understand which countries’ intelligence histories or systems are being researched — except for the UK and the USA. The automatic extraction script is not yet smart enough though. For example, the following title can be categorised under both countries ‘X’ and ‘Y’: Intelligence activities of the country ‘Y’ in country ‘X’ which clearly should be categorised under ‘Y’ only (still working on this to resolve the issue).
Institutions
On the institutions page, you can find organisations that perform research or work on intelligence. For example, you can find and filter academic programs in different countries, look up research centres whose activities are largely on intelligence studies, and even check which museums are available that exhibit objects about intelligence.
Wrapping up and some thoughts
This website can be useful for researchers of intelligence studies or just intelligence enthusiasts. People can use it for various purposes, like doing literature reviews, checking trends in publications about intelligence over time, finding academic/research programs for their careers, or surfing on it just for fun.
What is more important is to have disciplined-specific, human-curated lists — like the Intelligence studies bibliography — in this era of Generative AIs. These relatively smaller lists or datasets can play an important role in finding more trustworthy responses from AI models if they are used for training a model. As David Myers from Data Licensing Alliance says ‘No matter the size of the data set, the relevance of the data is what really matters’.
This site is still in its baby steps. I am still not quite sure where it goes next. I’m eager to improve it as long as I have time, energy, and the will but I believe I’ve already pushed the Streamlit app to its limits. The one thing is for sure so long as Zotero, Streamlit, and GitHub are alive, this page and the library will be available for everyone.
I should also include a disclaimer here. This site and the library itself don’t claim to be the largest source of intelligence studies. You may notice that not all sources are yet listed. Also, the list is curated based on the contributors’ (so far only two apparently) subjective views and efforts.
If you have any ideas/suggestions or wish to collaborate, feel free to get in touch with me.
