Exploring whether NASA FIRMS can accurately be used for open-source intelligence activities
NASA Fire Information for Resource Management System (NASA FIRMS) satellites have been around for more than a decade and have been adopted for numerous objectives throughout their operational history. As a near real-time active fire / thermal anomaly location geographical information system (GIS), it provides key information for natural resource managers and also UN organisations for monitoring emergency projects. However, it is the Russian invasion of Ukraine that has additionally revealed how useful this system can be in the hands of the open-source intelligence (OSINT) community, especially in gathering data about the conflict, tracking changes on the frontlines or in verifying information on events on the ground, such as shelling or bombing.
Based on the first six months of the military conflict (from 24th February 2022 to 24th August 2022), this study intended to test whether data gathered from NASA FIRMS could accurately be used for the above-mentioned OSINT activities, and if so, to identify the system’s potentials and limitations. This testing also served the purpose of validating whether NASA FIRMS as an OSINT tool could be used as a primary source to provide first-hand evidence about events in a war where, due to the nature of military conflicts, opposing parties tend to resort to subterfuge, and oftentimes, secondary sources are unreliable.
Important disclaimer regarding this study: the thermal anomalies have no nationality; therefore, it cannot be determined which of the opposing parties has generated them. The study draws on news and reports from the field and does not wish to adopt a position in debates on the origins of the shellfire.
Methodology
This report begins with an introduction to NASA FIRMS and then moves from the bigger picture to the smaller, detailing specific events. As such, the analysis drills down from the macro level to the micro and investigates six (three bigger; three smaller) cases to identify the minimum scale of events that can be tracked using NASA FIRMS technology.
For the study, FIRMS data from all three instruments (MODIS, VIIRS S-NPP, VIIRS NOAA-20) in .csv format from 1st January 2021 to 1st September 2022 on Ukraine was requested and subsequently received from NASA. Overall, there were 218,477 entries (GPS coordinates and other data) from which two groups of data were compiled: one covering the period from 24th February 2021 to 24th August 2021, as a reference (the six months when there was no war in Ukraine — 68,412 GPS coordinates and other data), and one covering the period from 24th February 2022 to 24th August 2022, as test data (the same six months the following year, when the war in Ukraine was waging — 103,541 GPS coordinates and other data). The reference dataset was the standard against which comparisons were made with the test data.
For the big data analysis and visualisation, Microsoft Power BI and Datawrapper tools were used, in addition to images from the NASA FIRMS website.
The list of references and suggested readings can be found at the end of the post.
NASA FIRMS in a nutshell
Since some of the strengths and weaknesses are embedded in its technology, it was important to conduct a preliminary investigation of the capabilities of the NASA FIRMS instruments prior to an in-depth analysis of the gathered data.
NASA FIRMS was originally developed at the University of Maryland from NASA funds to provide wildfire data. The system uses two types of sensors: the Moderate Resolution Imaging Spectroradiometer (MODIS) on the Terra and Aqua satellites and the Visible Infrared Imaging Radiometer Suite (VIIRS) on the Suomi-NPP and NOAA 20 satellites. To detect wildfires, the sensors search for thermal anomalies and hot spots while passing overhead.¹
The detection itself is performed by a contextual algorithm that monitors strong emissions of mid-infrared radiation from fires. MODIS data is available from November 2000 (for Terra) and from July 2002 (for Aqua) to the present, where one pixel represents approximately 1 km, while VIIRS 375 m data is currently available from 20th January 2012 to the present (for Suomi NPP) and 1st January 2020 to the present (for NOAA-20), where one pixel represents approximately 0.375 km.²
Generally, under suitable conditions, the system can detect both flaming and smouldering fires of 1,000 m2 in size, but under optimal conditions as small as 50 m2 in size. However, there can be a multitude of reasons as to why a fire goes undetected, the most obvious being when it breaks out and dies down between satellite observations. A fire can also go under the radar when a cloud, heavy smoke or tree canopy obscures it; when the satellite instruments fail; or when the fire itself is too small or too cool in temperature.³
The bigger picture – the macro level
Starting with the bigger picture, it is important to note that agriculture and industry had significant shares in the Ukrainian economy in peacetime.⁴ Therefore, it is unsurprising that more than 100,000 thermal anomalies are detected by NASA FIRMS in the country every year, and most of these correspond nicely with the locations of agricultural or industrial centres.
Two heatmaps were created with Microsoft Power BI to see if the gathered data on Ukraine for 2021 (the six months of no war, from 24th February to 24th August 2021) differed from the data for 2022 (the same six months during the war, from 24th February to 24th August 2022) — in other words, to see if there was any change in the number of thermal anomalies or locations etc. that could be interpreted as resulting from the conflict.
The red areas show where the most thermal anomalies occurred, while the yellow and blue areas show where the least occurred.
From a cursory glance at the 2021 heatmap, it is conspicuous that the eastern and southern regions of the country contain more red areas than the western region. This phenomenon may be rooted in the fact that the former regions (especially the Donbas) are the most densely urbanised and industrialised.⁵
This pattern was also reflected in the data, especially in and around Mariupol, where the Azovstal Steel Works and other heavy industrial facilities are located,⁶ and also in and around Donetsk, which are the country’s scientific and industrial centres.⁷ As such, these areas generate an above-average number of thermal anomalies. Moreover, it worth noting that there has been an ongoing, low-intensity military conflict — the so-called ‘War in Donbas’ — in this region since 2014.⁸
Compared to the reference heatmap for 2021, the heatmap for 2022 is significantly different in many ways. First and foremost, there are many more red areas in the eastern and southern parts of the country; furthermore, there are new red areas north of Kyiv and around the cities of Chernihiv and Kharkiv.
These new thermal anomalies correspond with the main passage of the Russian invasion⁹ and can therefore be considered consequences of the military conflict. Meanwhile, most of the remaining hot spots are false positives and generated by either natural, agricultural or industrial activity.
We can conclude from the number of thermal anomalies detected in each year that something unusual began taking place on the ground in the second year, since there was a significant (more than 50%) increase in the number of incidents from the first six-month period in 2021 (68,412) to the second in 2022 (103,541).
In terms of the number of daily thermal anomaly detections, it is apparent that compared to the 2021 data, there are several unusual spikes in the 2022 data. The most striking is the enormous spike in March (peaking at 4,444 thermal anomalies on the 23rd) followed by several smaller ones in May, July and August during the six-month observation period.
If the number of daily cases in 2021 is deducted from the number of daily cases in 2022, then the picture is even clearer. In 2022 there were 102 days when the number of daily detections exceeded the daily detections in 2021, and only 80 days when it did not. This means that on more than half of the observed days during the six-month period of 2022, more thermal anomalies were detected than on the same days in 2021.
Specific events – the micro level
In total, data for six events was analysed: three large-scale battles spanning several days and three smaller conflicts. The bigger ones consisted of area attacks (shelling or bombing), while the smaller ones were precision strikes (guided missile attacks or sabotage). I combined the data and images and produced the following set of schematic maps of the territories: a so-called Standard map showing the territories without any thermal anomalies, a Reference Map showing the thermal anomalies for 2021, and a Test Map showing the thermal anomalies for 2022. Moreover, in the case of the longer battles, Animated Gifs have also been added to illustrate the day-to-day changes.
Chronologically, the Battle of Kyiv was one of the first major events in the Russian invasion of Ukraine, starting on 25th February and ending with the withdrawal of the Russian forces on 31st March. Nine shellings are indicated on the map (from the many that happened during the more than month-long battle): Russian shelling of the city of Kyiv (26th February–31st March 2022);¹⁰ Russian shelling of Bucha (February–March 2022);¹¹ Ukrainian shelling of Russian forces in Bucha (February 2022);¹² Russian shelling of Kurenivka, Rusanivka and Boyarka (1st March 2022);¹³ Russian shelling of Borshchahivka (4th March 2022)¹⁴ and Russian shelling of Irpin (6th March 2022).¹⁵
The Standard Map shows Kyiv and the surrounding neighbourhood with no thermal anomalies, while the Reference Map for 2021 depicts how Kyiv normally looks during peacetime, with several thermal anomalies but no large clusters of these hot spots. However, on the Test Map for 2022, it is evident that there is a dramatic change. The northern, eastern and western neighbourhoods of Kyiv, especially, are saturated with detections, but thermal anomalies are widespread in the southern neighbourhood as well.
Some of the above-mentioned shelling events can be observed in the animated gif. Several thermal anomalies were detected in the city of Kyiv from 11th March and in and around Bucha from 25th February, but none were detected in Kurenivka, Rusanivka, Boyarka, Borshchahivka or Irpin on any of the given dates. Therefore, it can be concluded that NASA FIRMS instruments picked up an unprecedented number of thermal anomalies, with some of these events taking place in and around Kyiv.
In addition, the siege of Mariupol was one of the key events in the Russian invasion of Ukraine and included the attack on the city’s massive, highly defensible, Soviet-era industrial complex that is Azovstal Iron and Steel Works and is where the Ukrainian forces had retreated to by 22nd April.¹⁶ The bunkers under the facilities were built in the early 30s with war in mind and intended to shelter thousands of workers.¹⁷ The siege of the steel works took several weeks, during which the Russians subjected it to both artillery and air strikes.¹⁸
The Standard Map shows Mariupol and the steel works facilities when there are no thermal anomalies present. On the Reference Map for 2021, two principal and several smaller hot spots are visible. The two principal ones are in the north, over the Azovmash Factory and the Illyich Iron and Steel Works, and in the south, over the Azovstal Iron and Steel Works. These plants generate a vast amount of heat when they operate. However, when they do not — as in wartime — the case is different. As Test Map, 2022 shows, due to the war, the factories’ operation had been suspended in the north (no thermal anomalies detected), and both the number and distribution of the detected thermal anomalies in the south are different, meaning that these hot spots around the Azovstal Iron and Steel Works were generated by intense events.
As the gif shows, the first thermal anomalies appear on 27th April, which is in line with the statements made by Petro Andryushchenko, an aide to the city mayor, who claimed there had been no break in air strikes on the Azovstal plant, even when Russian President, Vladimir Putin, stated that there was no need to bomb it since he had declared victory in Mariupol.¹⁹ The bombing nevertheless continued over the following days, as both the NASA FIRMS data and reports from the ground confirm.²⁰ In fact, thanks to Maxar Technologies satellite imagery, visual confirmation of the events is also available.²¹ This indicates that NASA FIRMS worked well and recorded the bombings extremely precisely in this instance.
Zaporizhzhia Nuclear Power Plant (ZNPP) — which, in spite of its name, is located approximately 100 km from the city of Zaporizhzhia, near to Enerhodar — also features regularly in the news. The facility’s importance comes from the fact that it is the largest nuclear plant in Europe, with six VVER-1000 reactors, and generates 5,700 MW of energy (one-fifth of Ukraine’s electricity), and therefore has a critical role in Ukraine’s energy infrastructure.²² Since the Chernobyl incident, any potentially harmful event in or around a nuclear facility that threatens its stability has been given the highest priority; thus, the violent actions inflicted on the ZNPP and the surrounding area are relevant and newsworthy.
It is important to note that, in the case of the ZNPP, the sources of the shelling attacks are not known, since no one has accepted responsibility for them.
The Standard Map depicts the nuclear power plant and its surroundings when there are no thermal anomalies detected. On the Reference Map, 2021 there are only a handful of hot spots, since there was no active military conflict in the region at this time. However, Test Map, 2022 shows the location in August, where areas to the north and south of the ZNPP are flooded with thermal anomalies, which is an indication of active fires, in line with the reports on shelling around the nuclear facilities.
Taking a look at the day-to-day progressions, it is apparent that the reports were accurate. Although the attacks on the 330kv transmission on 5th August²³ and the dry fuel storage facility on 6th August are not visible on the NASA FIRMS maps,²⁴ ²⁵ the attacks at the end of the month are.
When it comes to precision strikes, it is important to remember that the NASA FIRMS instruments can detect both flaming and smouldering fires in an area covering 1,000 m2, and under ideal conditions, an area covering just 50 m2.
As part of the battle of Kyiv, the strike on the 385–m high Kyiv TV Tower was among the most documented events. Pictures and video footage recorded the Russian missile attack on the tower²⁶ and the surrounding neighbourhood following the hit.²⁷
What is intriguing, however, is the fact that the NASA FIRMS instruments were unable to detect any thermal anomalies around the TV tower on 1st March, which confirms the information about the sensors’ limitations when it comes to the sizes of hot spots.
The same applies to the next precision strike example: the HIMARS attack on the Antonovsky Road Bridge in Kherson Oblast, near to Kherson city; this bridge crosses the Dnieper River and was critical for Russian logistics.²⁸ The US–made HIMARS light multiple rocket launchers had been operating in Ukraine since the end of June²⁹ and are famous for their high-precision attacks and capability for engaging point and area targets.³⁰
Since this attack, other than some likely false positives (anomalies that are nowhere near the bridge), no thermal anomalies can be observed on the Test Map for 2022; thus, it can be said that NASA FIRMS instruments were unable to pick up the small hot spots generated by the HIMARS hits on the bridge itself, which is consistent with the satellite systems’ known limitations.
Similar conclusions can be drawn from the attack against the Saky Airbase in Crimea. The Russian airbase was bombarded by Ukrainian forces on 9th August, and at least nine aircraft were damaged.³¹ The blasts and accompanying smoke were visible even from the nearby beach,³² indicating widespread destruction.
However, despite the high level of damage and smoke visible from kilometres away, it is notable that the NASA FIRMS instruments did not detect any thermal anomalies on the ground on the given day. Since heavy smoke or even tree canopy can obscure a fire, it is likely that something of this nature prevented the systems from being able to detect the hot spots.
And the cherry on top
Thanks to the excellent work of Def Mon (@DefMon3) / Twitter and Ragnar Gudmundsson / Twitter, there is lots of data available on the locations of the Russian shelling in Ukraine — and much more on Ragnar’s data visualisation project: Tracking Russia’s losses in Ukraine (google.com).)
The data has been tracking the shelling since 9th June 2022, but it is noteworthy from the above map that the locations of Russian shelling correspond with the locations of the previously analysed macro level NASA FIRMS thermal anomaly detections. This further proves that NASA FIRMS data is reasonably accurate in detecting area attacks (it is worth remembering that this type of deep and forceful artillery attacks was one of the main tactics of the Soviet army³³ and it seems not much has changed since then) and confirms that the daily reports on the shelling were also precise.
It is also notable that the number of Russian shelling attacks per day shows an almost steady growth during the almost three-months observation period.
Potential and limitations
As evidenced in the abovementioned examples, with respect to the Russian invasion of Ukraine, NASA FIRMS as an OSINT tool is not the holy grail of military conflict tracking. Nevertheless, it is also true that it possesses a certain degree of usefulness. By comparing datasets from different years, atypical spikes in the daily data can reveal if something ‘unusual’ is happening on the ground. The tool can also help verify major events if they are large enough to be detected by satellites passing overhead.
Simply put, the smaller the size of the target (observed event), the bigger the uncertainty factor of the detection.
This characteristic is not the fault of NASA FIRMS but rather it is embedded in its technology, since the system was designed and calibrated to detect fires of 1,000 m2 in magnitude (or as small as 50m2 under optimal conditions) — not to locate, track or monitor military conflicts.
Conclusions
Based on the findings outlined in this paper, in using NASA FIRMS to track or monitor the Russian invasion of Ukraine, it can be said with a certain degree of certainty that ‘something is going on below’. Area attacks (where larger territories exhibit thermal anomalies) are visible on the maps, but single-point hits (where smaller regions exhibit such anomalies) are not. As with the case when wearing the wrong glasses creates blurry vision, NASA FIRMS instruments also generate extremely blurry images of events on the surface. This is unsurprising since the system was not intended to be a high-precision, open-source military intelligence tool. Therefore, it can be concluded that the satellite system fulfils the purpose for which it was designed: namely to detect flaming and smouldering fires.
For the system’s main purpose, this degree of accuracy is more than sufficient. However, as a primary OSINT tool — especially in cases of precision attacks — NASA FIRMS is not precise enough in isolation; as such, using it without the support of secondary evidence — such as photos, videos or reports from the ground — can be a bit ‘hit-and-miss’…although better than nothing.
If you would like to explore this dataset further, feel free to download the raw data from my Kaggle page, play around with it yourself and supplement the analysis with your own. And don’t forget to follow the work of Def Mon (@DefMon3) / Twitter and Ragnar Gudmundsson / Twitter.
Suggested readings
Agira Technologies. 2020. 30 Best Satellite Maps To See Earth in New Ways. February 6.
https://www.agiratech.com/satellite-maps-earth
Bellingcat. 2022. Scorched Earth: Using NASA Fire Data to Monitor War Zones. October 4.
https://www.bellingcat.com/resources/2022/10/04/scorched-earth-using-nasa-fire-data-to-monitor-war-zones/
Earth Observation Group, Payne Institute for Public Policy. 2022. Russia-Ukraine War as Seen by DNB.
https://eogdata.mines.edu/products/special_topics/russia_ukraine_war.html
The Swedish Institute of International Affairs. 2020. The Long War in Donbas: Causes and Consequences.
https://www.ui.se/globalassets/ui.se-eng/publications/ui-publications/2020/ui-report-no.-1-2020.pdf
References
1. ArcGIS StoryMaps. 2022. FIRMS: Fire Information for Resource Management System. January 25.
https://storymaps.arcgis.com/stories/9c011e1e569845c092eba5c377259202
2. Earth Data / NASA. FIRMS Frequently Asked Questions.
https://www.earthdata.nasa.gov/faq/firms-faq
3. Earth Data / NASA. FIRMS Frequently Asked Questions.
https://www.earthdata.nasa.gov/faq/firms-faq
4. Ukraine Invest. Key facts and figures about Ukraine.
https://ukraineinvest.gov.ua/invest-in-ukraine-now-2/key-facts-figures-about-ukraine/
5. Ukraine Invest. Overview of Regions.
https://ukraineinvest.gov.ua/analytics-research-2/overview-of-the-regions/
6. METINVEST. AZOVSTAL IRON & STEEL WORKS.
https://azovstal.metinvestholding.com/en
7. Ukraine Invest. Overview of Regions.
https://ukraineinvest.gov.ua/analytics-research-2/overview-of-the-regions/
8. Ukraine.ua. 2022. The invisible war: 8 years of battles in Donbas. July 14.
https://war.ukraine.ua/articles/8-years-of-war-in-donbas/
9. CNN. 2022. Russia launches military attack on Ukraine with reports of explosions and troops crossing border. February 24.
https://edition.cnn.com/2022/02/23/europe/russia-ukraine-putin-military-operation-donbas-intl-hnk/index.html
10. Washington Post. 2022. Battle for Kyiv: Ukrainian valor, Russian blunders combined to save the capital. August 24.
https://www.washingtonpost.com/national-security/interactive/2022/kyiv-battle-ukraine-survival/
11. OSCE. 2022. REPORT ON VIOLATIONS OF INTERNATIONAL HUMANITARIAN AND HUMAN RIGHTS LAW, WAR CRIMES AND CRIMES AGAINST HUMANITY. July 11.
https://www.osce.org/files/f/documents/3/e/522616.pdf
12. Reuters. 2022. Love letter, ID card point to Russian units that terrorised Bucha. May 5.
https://www.reuters.com/investigates/special-report/ukraine-crisis-bucha-killings-soldiers/
13. Держспецзв’язку — Telegram channel. 2022.
https://t.me/dsszzi_official/1464
14. Reuters. 2022. More explosions heard in Kyiv in sign assault is intensifying. March 4.
https://www.reuters.com/world/more-explosions-heard-kyiv-sign-assault-is-intensifying-2022-03-04/
15. NPR. 2022. Russia-Ukraine war: What happened today. March 4.
https://www.npr.org/2022/03/04/1084484564/russia-ukraine-war-what-happened-today-march-4
16. Yahoo News. 2022. Ukraine war: How the Azovstal steelworks turned into the final outpost in the brutal battle for Mariupol. April 22.
https://uk.news.yahoo.com/ukraine-war-azovstal-steelworks-turned-150700314.html
17. France24. 2022. ‘A chance to survive’: Ukraine’s fortress steel mills. May 11.
https://www.france24.com/en/live-news/20220510-a-chance-to-survive-ukraine-s-fortress-steel-mills
18. New York Times. 2022. Last Stand at Azovstal: Inside the Siege That Shaped the Ukraine War. July 24.
https://www.nytimes.com/2022/07/24/world/europe/ukraine-war-mariupol-azovstal.html
19. Reuters. 2022. Russian forces pummel Ukrainian fighters holed up in Mariupol steel plant — mayoral aide. April 27.
https://www.reuters.com/world/europe/russian-forces-pummel-ukrainian-fighters-holed-up-mariupol-steel-plant-mayoral-2022-04-27/
20. The Guardian. 2022. ‘Massive bombing strike’ on Azovstal plant, says Mariupol official. April 28.
https://www.theguardian.com/world/live/2022/apr/28/russia-ukraine-war-putin-warns-of-lightning-fast-retaliation-if-west-intervenes-war-has-cost-ukraine-600bn-zelenskiy-says-live?page=with:block-626ac2de8f08fff86b738258#block-626ac2de8f08fff86b738258
21. Maxar Technologies — Twitter channel. 2022.
https://twitter.com/Maxar/status/1520481307175829507
22. The Guardian. 2022. Zaporizhzhia nuclear power plant: everything you need to know. March 4.
https://www.theguardian.com/world/2022/mar/04/zaporizhzhia-nuclear-power-plant-everything-you-need-to-know
23. BBC. 2022. Zaporizhzhia: Russian rockets damaged part of nuclear plant, Ukraine says. August 5.
https://www.bbc.com/news/world-europe-62432873
24. The Guardian. 2022. Shelling temporarily disconnects Zaporizhzhia nuclear plant from Ukraine grid. August 25.
https://www.theguardian.com/world/2022/aug/25/zaporizhzhia-nuclear-plant-disconnected-ukraine-grid-russia
25. CNN. 2022. Russian-held Zaporizhzhia nuclear plant disconnected from power grid for second day after nearby fires. August 26.
https://edition.cnn.com/2022/08/25/europe/ukraine-zaporizhzhia-nuclear-plant-disconnected-intl/index.html
26. BBC. 2022. Ukraine conflict: Kyiv TV tower struck by Russian missile. March 1.
https://www.bbc.com/news/av/world-europe-60579847
27. ABC News. 2022. Russian missile hits Kyiv TV tower killing at least 5. March 1.
https://abcnews.go.com/Politics/live-updates/russia-ukraine/?id=82467772#83180798
28. France24. 2022. Ukraine targets supply routes to weaken Russian troops in Kherson. July 29.
https://www.france24.com/en/europe/20220729-ukraine-targets-supply-routes-to-weaken-russian-troops-in-kherson
29. CNN. 2022. All pledged rocket systems will be in Ukraine by mid-July, US defense official says. June 24.
https://edition.cnn.com/europe/live-news/russia-ukraine-war-news-06-24-22/h_5f3b4771de02e2e97a63e294d895bad9
30. Lockheed-Martin. 2022. HIMARS: A Critical Long-Range Precision Fires Capability for our key Indo-Pacific Allies.
https://www.lockheedmartin.com/en-us/news/features/2022/himars-long-range-precision-fires-capability-indo-pacific-allies.html
31. The Guardian. 2022. Russian warplanes destroyed in Crimea airbase attack, satellite images show. August 11.
https://www.theguardian.com/world/2022/aug/11/russian-warplanes-destroyed-in-crimea-saky-airbase-attack-satellite-images-show
32. The Washington Post. 2022. At least six explosions hit Crimea air base in nearly an hour. August 14.
https://www.washingtonpost.com/investigations/2022/08/14/crimea-airbase-attack/
33. Headquarters Department of the Army. 1984. The Soviet Army — Operations and Tactics, July 16.
https://irp.fas.org/doddir/army/fm100-2-1.pdf
