Big Data: Providing Fire Service and Emergency Response Communities with Tools to Predict and Respond
By
Desmond O’Neill
Viern Pierson
Joe Russo
Ari Wolfe
Chris Demaise
Trevor Wilson
The following document is the product of a CHDS project that asked homeland security experts to evaluate how a better use of data could improve homeland security. Because of the nature of the projects, we are making an exception to the editorial guidelines of the collection in order to leave the citations in the way they were offered, instead of using hyperlinks as we normally request.
Introduction — O’Neill
Imagine you are an officer within a large metropolitan fire department. Having arrived at the firehouse a few minutes early before beginning your 15-hour tour, you head to your office, fire up your MacBook Pro, and begin scanning over your battalion’s emails for the day. Seeing the new Fire Forecast post just hit your inbox, you click on it and wait. Within seconds your computer screen lights up with what could be considered every fire official’s dream: a cache of data predicting the fire station’s calls-for-service during the next 24 hours. Over the next several minutes, you take note of its predictions, including six medical emergencies, two gas emergencies, one structural fire, and a motor vehicle accident with injuries. The analytics recommend ten in-service personnel to handle the projected call volume, three less than the current number already scheduled to work. As you click open the embedded map icon you evaluate the road conditions within your station’s area of responsibility — in real time. With heavy snow anticipated within the next several hours, the estimated response times throughout your district are constantly updating. Flashes of blue, red, and yellow light up your screen pulling your attention towards the geo-locating markers indicating where the predicted types of emergencies are likely to occur. Although the projections are never failsafe, they provide you with enough information to ready your crew. During your nightly hour-long skills session you select the most appropriate training drills to ensure each team member is proficient at handling these predicted types of emergencies.
B. Integrating Big Data into the Fire Service
Public service agencies, such as fire departments, are capitalizing on the synthesis of big data in an effort to both enhance its capabilities as well as protect its personnel. Through the trawling of information captured by the vast array of private companies, social media platforms, and the ever increasing Internet of Things, big data seems ripe for the picking.[1] The difficulty lies, however, in gaining legitimate access to the various data sources, evaluating its accuracy, and then making sense of how it all fits together into a predictable model that is both user-friendly and up-to-date.
Within this white paper, we delve into the analytical value of using big data to increase the efficacy within the Fire Department City of New York (FDNY). Currently, the FDNY utilizes an algorithm called FireCast 2.0 “that helps the department manage inspected buildings and prioritize them based on fire risk.”2[2] Its utility, however, stops there. In an effort to add to the wealth of literature advocating the development of the next-generation FireCast, a system that mirrors the opening vignette, this paper incorporates an array of source material that underscores the necessary steps forward.
C. Chapter Overview
Chapter I begins with case studies that evaluate the big data analytical tools used by two prominent U.S. fire protection agencies to assess risk: the Atlanta Fire and Rescue Department’s Firebird and the California Department of Forestry and Fire Protection’s Scout. Within this chapter are the fundamental challenges that came with integrating these systems, the measures taken to overcome obstacles, and their successes thereof. An interview with a subject matter expert (SME) specific to this space adds further validity to this chapter. During the course of the conversation, the SME discussed the issues associated with implementing risk assessment tools into the fire protection ethos, a topic that extends beyond the scope of the available literature.
Chapter II focuses on the history of the FDNY’s current predictive risk algorithm, FireCast 2.0, including what metrics it measures and how those figures are ultimately tabulated into a risk score. An exploratory assessment is also discussed regarding the next generation FireCast. This enhanced version provides a concept of what a future model may look like using an array of sources outside the sphere of the current system. This chapter also sheds light on the FDNY’s institutional dogma that plagues its ability to source in-house data, information deemed vital for forecasting fires.
Chapter III expands on the concepts detailed in Chapter II by exploring other avenues of data sources that may show relevance in fire prediction, such as the New York Police Department, the New York Department of Finance, as well as sectors within the insurance and real estate industries. Further reviewed within this chapter is the importance of establishing formidable relationships with other data sharers in order to ensure the continuous influx of information. Chapter IV explores the technical, policy, and cultural issues that plague upgrading the current FireCast system. Challenges that include merging copious amounts of data from incompatible platforms, privacy violations resulting from the unintentional disclosure of personal data, and policy shortcomings that fail to acclimate to the increasing collection, analysis, and storage of information. The final chapter, Chapter V, is a synthesis of the work contained in the previous chapters. In addition, it focuses on the limitations of this research and offers recommendations towards the successful adoption of the FireCast 3.0 model.
CHAPTER I: CASE STUDY — Pierson
A. Introduction
The fire service has attempted to employ big data for fire prevention, detection and suppression. A prominent example of this effort is Firebird in Atlanta. Firebird is a fire risk and prevention tool conceived and developed by Atlanta Fire and Rescue in partnership with the Georgia Institute of Technology. This forward thinking tool consists of data mining from a wide variety of sources combined with algorithms to produce usable management data.
B. Firebird: Using Strategic Data to Predict Fire Risk and Prioritize Fire Inspections
Firebird was conceived as a big data tool used to identify buildings that are likely to have a fire hazard in the immediate future. Since implementation, it has been used to accurately predict fires 71% of the time in Atlanta and has a False Positive Rate of 20%. Employed correctly, Firebird permits city building and fire inspection personnel to prioritize inspections based on algorithmic probabilities, derived from risk assessments based data. This tool combines 8 separate data sets (fire incidents, fire inspections, crime incidents, liquor and business licenses, socioeconomics and demographics, commercial property information, parcel conditions, and Google places web application) to identify commercial properties with high fire risk. Firebird then produces an interactive map to prioritize fire inspections. In a study done by Atlanta Fire and Rescue Department utilizing Firebird they accurately predicted 8,223 fires, 5,022 of which were on the list provided by Firebird for inspection.
One of the primary impetuses of Firebird was the appreciation by the Atlanta Fire and Rescue Department (AFRD) that a significant number of buildings within Atlanta were unregistered and therefore uninspected. Inspectors would drive around Atlanta searching for buildings that appeared risky and in need of inspection. The AFRD estimated that only 10% of Atlanta’s commercial structures were receiving regular inspections. According to Matt Hinds-Aldrich, formerly a member of the Assessment and Planning section of the AFRD, ’“Intelligent door knocking is the crux of this thing — firefighters knocking on doors talking to residents about fire safety has been proven to be way more effective than sitting at grocery store talking to random people who pass by, intuitively I know where the challenging neighborhoods are and where we should focus, but I don’t know which street I should be on, what door to knock on, or what to expect. Do they speak Spanish? Are they likely to have children? Are there adults with limited mobility there? How many people live there?”’[3] Firebird was developed in conjunction with Georgia Tech’s Data Science for Social Good program, which used graduate students to use analytics to sort out buildings within the city and assign risk profiles.[4]
C. Development and Implementation Challenges
One of the major challenges with Firebird was merging data from different departments. In fact, some inspection data was stored on paper and various city entities used different naming conventions for the same data. All data needed to be digitized and converted into the same file format in order to be downloaded and processed by Firebird. Although this was a big challenge, shortcomings and lack of standardization in fire inspections and record keeping was a primary reason for Firebirds initiation. Additionally, as AFRD worked to compile existing data, the project team identified the need to incorporate and process the 19,397 new commercial property inspections.
Ultimately, the team combined information from commercial properties, cross-referenced with a predictive model of fire risk using machine-learning algorithms. The team employed both Vector Machine, supervised learning models with associated learning algorithms, and Random Forest, ensemble-learning method for classification and regression.[5][6] These programs provided a 71.36 percent success rate in predicting fires with only a 20 percent false positive rate. These changes were able to make a significant impact on mitigating fires in Atlanta. Even false positives provided useful information by identifying buildings that may catch fire in the future and should be considered high risk and in need of inspection.
Firebird is successful because it can use data to accurately and efficiently task inspection personnel in a way that mitigates fire risk. Prioritization of inspections allows fire inspectors to inspect buildings with the highest probability of risk. This ends the routine inspection of buildings that individual agencies prioritize and instead uses a complex algorithm to determine which buildings have the highest risk. Ultimately, available data is consolidated and evaluated in a mechanism that serves to efficiently mitigate risk to the public.
[1] Fiona Graham, “Can big data help fight fires and save lives,” BBC News, March, 26, 2013, http://www.bbc.com/news/business-21902070
[2] Spotfire Blogging Team, “Saving Lives with Big Data Analytics… Putting out Fires in NYC,” TIBCO blog, February 18, 2015, http://www.tibco.com/blog/2015/02/18/saving-lives-with-big-data-analytics-putting-out-fires-in-nyc/
[3] Roman, Jesse, “Online only,” NFPA Journal, June 2016, http://www.nfpa.org/news-and-research/publications/nfpa-journal/2016/may-june-2016/features/embracing-analytics/experian
[4] Roman, Jesse, “Embracing Analytics,” NFPA Journal, June 2016, http://www.nfpa.org/news-and-research/publications/nfpa-journal/2016/may-june-2016/features/embracing-analytics
[5] Cortes, C.; Vapnik, V. (1995). “Support-vector networks”. Machine Learning. 20 (3): 273–297. doi:10.1007/BF00994018
[6] Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome (2008). The Elements of Statistical Learning (2nd ed.). Springer. ISBN 0–387–95284–5.
Raven-SCOUT: The Incorporation of Enabling Technologies for Fire Services
Raven is an open source, mobile, web based command and control program developed by Jack Thorpe, formerly of the Defense Advanced Research Projects Agency (DARPA,) to assist first responders. Raven’s initial intent was to establish an intelligence process in which information flowed from the lowest level first responders to key decision makers as well as front line first responders. In 2016, development of the project with the financial support of DHS led to SCOUT, which is essentially Raven 6.0 and managed by the California Office of Emergency Services. Raven-SCOUT was conceived and designed with the fatigued emergency responder in mind. Designed so that any operator can easily use its advanced processing and computing, the system allows for any number of constituents to view and input information simultaneously. There is a Chat room associated with it that can be used to describe issues and help navigate units. Global Positioning Systems help track units on the ground on a map that resembles the military’s BFT (Blue Force Tracker). Users are able to create visualizations that can be shown to higher echelons of command or incoming units to help them understand what is happening on the ground. As long as an operator has Internet access they have the ability to utilize this highly advanced tool.
Raven-SCOUT’s uses the Internet to transfer data in real time to multiple sources and ensure they are receiving the same, concise material. Prior to Raven-SCOUT and NICS, systems operators used paper, land mobile radio, and face-to-face meetings in an attempt to convey any semblance of an operational order (OPORD). Historically, Incident commanders received and processed information as time elapsed, then commanders would begin the process of conveying that information and command and control decisions at either a briefing or via radio. Operational orders written in pen and paper took time to distribute. Orders conveyed via radio were subject to information being misheard or too many seeking to speak at the same time. Circumstances continue to evolve as directives are filtering through the chain of command. Raven-SCOUT bypasses these issues, by using the Internet and allowing all users to communicate simultaneously and cohesively. Raven-SCOUT data is administered instantaneously to all units that are a part of the operation and is available on any operating system and browser.
Much like Firebird, one of the primary development challenges with Raven-Scout was source data naming conventions. A multitude of agencies operated using multiple information management systems. This is of particular concern in California due to the multiple emergency service entities at the state, county and municipal level, each with their own command and control structures. Additionally, the various entities employ communication tools including paper, radios, mobile data terminals, (MDT) and cell phones. Finally, one of the key challenges has been funding. While the original funding for the development of Raven-SCOUT was DARPA, the subsequent evolution of this open source project has included Australia, the State of California and DHS.
CHAPTER II: PREDICTIVE FIREFIGHTING WITHIN THE FIRE DEPARTMENT OF THE CITY OF NEW YORK (FDNY) — Russo
A. Introduction
The adoption of predictive firefighting will increase the safety of New York City’s Firefighters and citizens. Firebird in Atlanta demonstrate the value of big data to structural fire and rescue efforts. While the FDNY has taken early steps toward the direction of a more comprehensive, fire predictive program more data is needed. The more data and variety the program receives will lead to more accurate prediction. With more data, perhaps it is possible the program will not only predict that a structural fire will occur, it will also provide the type of occupancy. Having this kind of knowledge is unprecedented and would provide responders with the ultimate preparedness tool.
B. FireCast
The FDNY has begun to use big data with the development of a “fire risk” algorithm called FireCast. FireCast is currently integrated into the new Risk-Based Inspection System (RBIS). When the FDNY is tasked with having to inspect over 330,000 buildings per year, the department is simply overwhelmed. The annual goal is to inspect just ten percent of that number.[1] Due to time constraints, fire companies can only conduct building inspection three days a week. FireCast 2.0 is the algorithm in RBIS that aids in assigning a risk score to all buildings entered into the system.[2]
FireCast 2.0 used various data associated with a structure to project a risk score. This score was recalculated daily, in order to capture the most current information. tatic building data such as size, age, and construction type, historical “performance” data (violations issued, past fire incidents, etc.), and a wide range of information related to any kind of public service requests (complaints, non-fire emergencies, etc) fed the algorithm The risk score was based on correlation between the variables and the incidence of fire. The result was a score that ranks the building on its relative likelihood of having a fire incident. This is how the department defined risk.
After building inspection, an FDNY officer returns to quarters and enters the updated inspection information. FireCast 2.0 processed this new data so RBIS can update the risk score daily. According to Government Technology magazine, FireCast is an algorithm that organizes data from five city agencies into approximately 60 risk factors, which are then used to create lists of buildings that are most vulnerable to fire.[3] According to the FDNY, “Since FireCast 2.0 was deployed in 2013, more than 16 percent of the city’s fires were in buildings that had been inspected in the past 90 days, suggesting that not only had the right structures been moved to the top of the list, but also when the firemen returned to fight the fires, they had up-to-date information on the layout of the buildings.”[4] FireCast 3.0 is the next iteration of the algorithm and is far more sophisticated. It plans to use 7500 metrics across 17 city agency data streams.[5]
C. Metrics for Predictive Firefighting
For this project, a similar methodology could easily be applied to predict different kinds of emergency responses (gas leaks, building collapses, etc) instead of strictly fires. While the methodology is somewhat straightforward, the practicality of obtaining useful data and manipulating it into a singular format is extremely challenging. Good data is hard to find, even harder to get, and wrangling it into the format needed is challenging. This chapter outlines a theoretical process to accomplish this goal.
Predictive firefighting is only possible with vast amounts of data from a multitude of sources. This project includes data from the following sources: FDNY responses, fire code violations, 311 service requests, the Department of Buildings, private sector agencies, and weather. In theory, by plugging this data into a predictive algorithm it would forecast what a work shift would look like for each individual company of the FDNY. This model will only improve its predictions as time passes and more data is available. The most important data that needs collection comes from within the FDNY. The department’s response data provides metrics for how fire units are mitigating various emergencies around the city.
The following FDNY response data will be used in this predictive firefighting algorithm:
a.) Date, day of the week, time — marks exactly when the incident occurred.
b.) Type of response — initial call to emergency dispatchers that caused a response by emergency units. This includes gas emergencies, structural fires, carbon monoxide incidents, motor vehicle accidents and fires, structural collapse, medical emergencies, elevator emergencies, hazardous material and decontamination incidents, electrical emergencies, water and steam incidents, and brush fires among many others.
c.) Duration of response — measures the time from dispatcher notification to response unit arrival at the scene. NYC’s 911 system continues to suffer from changes made back in 2009.[6] An improved 911 system could greatly reduce response times.
d.) Outcome of response — Due to the Unified Call Taker (UCT) system many runs turn out to be erroneous or for a completely different type of emergency.[7] This metric ensures the appropriate response data is collected.
e.) Scale of response — The amount of units used at the scene of an emergency to mitigate an incident.
f.) Location of response — This includes identifying information for the location of the incident. Not only would the street address require collection but also other building descriptors such as BIN and Box numbers. Each building in New York City has its own Building Identification Number (BIN). The BIN is the unique identifier used by city agencies to identify buildings.[8] Each address in the city is assigned an alarm box number. The term “box” refers to the Fire Alarm Boxes, which at one time lined street corners and in front of certain buildings. Each Fire Alarm Box was given a specific number by the FDNY’s Bureau of Communications. Even if the physical fire alarm box is no longer at a specific address or street corner, the address or street corner is still assigned that fire alarm box’s number.
g.) Identities and response order of units — Provides the algorithm with the individual units that responded and the order in which they arrived. Each alarm response creates a ticket that lists the order in which companies should arrive based on their firehouse location to the incident. Several factors can influence this order of arrival. Traffic conditions play the biggest role in the order of response. Other factors include inspection activities that pull fire companies to the outer edge of their response district and relocations that call particular fire companies to other areas of the city.
h.) Identities of regularly assigned units to that box — Collects the unit identities if regularly assigned companies did not respond.
i.) Elevation above sea level to response location — New York City is one of the most vertical cities in the world. When a unit arrives on the scene, it can still take several minutes before responders reach the emergency. Collecting this metric provides a more accurate time of arrival.
The aforementioned metrics are agency centric and should be relatively easy to collect. However, their acquisition is restricted by organizational culture and the main reason FireCast 3.0 never launched. The hoarding of this information severely limits the effectiveness of such an algorithm. In order for the theory of predictive firefighting to work, the free flow of information is essential. This is especially true regarding information from within the FDNY. Trying to understand the bureaucratic landscape and the organizational change barriers that inhibit the free flow of data requires future research.
Additional metrics that are essential to creating a predictive firefighting algorithm include but are not limited to:
a.) Fire code violations — Notice of Violation and Violation orders given by the FDNY and assigning a risk score based on the type of violation.
b.) 311 service requests — day and time of request, along with the nature of the request and the identity of the responding agency.
c.) Department of Building data — Certificate of occupancy, presence and nature of violations, and any open work permits assigned to the particular occupancy.
d.) Weather data from the National Oceanic and Atmospheric Administration (NOAA) — One of the major parts of responder size-up at an emergency is the weather. The weather can greatly impact response, can dictate procedure and is at times the cause of the emergency.
Each of these agencies collects data that is very relevant to the FDNY predictive programs. The third generation of FireCast is intended to gather additional data to enhance the building fire risk score however it neglects other emergencies that require the department’s response. Our model takes it a step further. Collecting this data for a predictive firefighting model will not only predict potential fires, but other types of emergencies as well.
D. Applying Big Data to Structural Fire Operations
When an incident commander arrives at the scene of a structural fire he or she is trained to follow certain procedures based on fire behavior and unit operations. Like most professions, the more you do something at work, the more confident you feel. The same holds true for chief officers managing a fire scene. While responses to emergencies are at an all-time high, the number of fires in New York City continues to decline.[9] This lack of fire duty does not provide new chief officers the experience that former FDNY chiefs had. The tacit knowledge senior experienced chief officers developed going to fires will take significantly longer for new chiefs to develop; if the development can happen at all. With the right metrics and the right algorithm, perhaps newly promoted chief officers can use big data to make up for a lack of tacit knowledge.
Collecting the following data can provide another tool for incident commanders managing fire operations:
a.) Type of response — What kind of fire is burning, i.e., contents only, structural, electrical.
b.) Duration of response — Time metrics collected from different points in the operation from the moment the call was received, to the moment the fire was put under control. A critical time metric is exactly when water is placed on the fire. This provides information on how long it took from the receipt of the call to extinguishment operations. It also allow provides how long it takes from the moment water hits the fire to full extinguishment.
c.) Weather — NOAA data for weather conditions when fire occurred.
d.) Type of building — Whether the fire is in a private dwelling, old law tenement, new law tenement, a taxpayer or a high-rise office building, determines the procedures taken to extinguish the fire.
d.) Building Construction — The footprint and makeup of the building is one of the most critical metrics. This includes the height, width and length of the building and the building materials used that make the building either fireproof, fire-resistive, non-fireproof, wood-frame, metal or timber.
e.) Fire protection — The fire protective systems inside the building play a role in operations and extinguishment. These include any sprinkler systems, standpipes, or fire escapes.
f.) Type of hose line used — The attack hose line used to extinguish the fire can vary in size. Smaller, more mobile lines are 1 3/4” and are used most often. This size hose line fights fires in private dwellings and tenements. A larger 2 1/2” hose line is used for commercial occupancies and fireproof high rise buildings. The larger hose line provides additional gallons per minute and is necessary to safely attack the fire from the interior of the building. The larger hose line takes longer to put into operation and impacts how quickly water gets placed on the fire.
g.) Fire location — How high above sea level the fire is burning impacts the amount of time it takes to get water on the fire for extinguishment.
An incident commander can use big data and real time analysis as a tool to assess on-going operations at a fire scene. While staffing the command post, there are several factors that might cause the chief officer to give a second alarm, which lets the dispatcher know more units are needed on scene. Some of the determining factors are the type of building and the location of the fire. Fire spreading into specific areas of a building would also require a chief to call for additional units. If enough data was collected using the metrics discussed an algorithm could begin to predict the amount of time it should take to extinguish a particular fire.If a structural collapse were to to take place within the city information related to the age and inspection history of adjacent building may also be needed to aid the safety of first responders. An incident commander with less experience would now have a guide to help shape his or her decisions while managing a fire or disaster scene.
Chapter III: LAW ENFORCEMENT AND OTHER DATA SOURCES — Wolfe
A. Introduction
As has been suggested in previous chapters, in order for predictive firefighting applications such as the proposed Firecast 3.0 to reach their full potential, they will need to include data from sources outside of the firefighting and building inspection community. Chapter 2 recommended the inclusion of weather data from the National Oceanic and Atmospheric Administration (NOAA) because of the influential or even causal connection between weather and the risk of structural fires. The following chapter will attempt to identify other external data sources that may impact risk ratings of particular structures in particular communities. These data will include: open source and limited internal law enforcement records (in this case, those of the New York Police Department (NYPD)), as well as other public records from the New York Department of Finance (NYDF). This chapter will also explore the possible utility of non-governmental sources, including the insurance and real estate industry. The chapter will conclude with recommendations for further research.
B. Data Source Transparency — A Lesson from 21st Century Policing
It might be argued that fire departments have not faced the same level of public scrutiny as police agencies, but that does not mean that public perception is not of concern to the fire services’ leadership.[10] Police professionals, from beat officers to chiefs, make decisions based on the law, and specifically civil rights and privacy issues on a daily basis. Whether it is a patrol officer developing probable cause to search a car,[11] or a police commissioner considering a policy on how to store and retrieve body camera footage,[12] an understanding of the law and public perception of how that law is enforced is integral to police work.
This may be less true of the fire professional. While there are certainly laws that govern all professions, most firefighters need not have a sophisticated understanding of the law every time they enter a blazing building. This paper explores the collection and use of geographic, demographic, and socio-economic data to more safely and efficiently deploy fire resources, in a similar way that predictive policing data collection is meant to serve law enforcement agencies. Given the controversy surrounding the much older practice of predictive policing,[13] the fire services should adopt sound ethical checks and balances for predictive firefighting in order to avoid negative public perception.
A 2015 report by the President’s Task Force on 21st Century Policing lists “Technology and Social Media” as one of the six pillars of policing. But it also challenges law enforcement leaders to “be able to identify, assess, and evaluate new technology for adoption and do so in ways that improve their effectiveness, efficiency, and evolution without infringing on individual rights.”[14]
As the use of technology for predictive firefighting is relatively new, it would be advisable to follow the latest “best practices” of ethical conduct by progressive police agencies. Ensuring that data collected are secure, anonymous in terms of personal identification information (PII), and that the purpose and usage of collection is transparent and reproducible help to ensure that fire services maintain public trust while optimizing public service.
C. Law Enforcement Data Resources
As this whitepaper has been using the FDNY as a test case for the implementation of a national predictive firefighting program, this chapter’s examination of relevant law enforcement data will be limited to the NYPD. The NYPD and many other New York City agencies publish data on “NYC Open Data,” an open source, web-based application that is available to the public.[15] These include limited crime statistics by precinct. The data are broken down by agency and category, and the metadata can be accessed using multiple digital formats.[16] Such compatibility makes NYC Open Data ideal for predictive firefighting purposes, as its sources can be easily identified then categorized and assigned risk values without a high degree of sophisticated programing.
NYC Open Data has also ties into NYC 311, which provides a user-friendly platform for the public to make nuisance complaints from rodents to garbage on the sidewalk.[17] While these data are not available for public review in the same way as location-specific crime statistics, such information, if shared, might have significant relevance to the assignment of fire risk ratings. But, in order to assign a rating to a specific building, data would need to be extracted directly from NYPD and NYC 311 databases, as the public platforms do not provide address information.
Here again, it would be important to consider the ethical and legal implications of such data sharing. While it might be acceptable to share that a particular address has multiple complaints for noise, or that officers responded there for a domestic disturbance call, it would be quite another matter to say that a particular person was associated to the complaint or the call. Furthermore, such information would be of little use in assigning a risk rating to the building itself. To share more specific information directly out of police and other City of New York databases, interagency protocols would have to be established to ensure the data was consistent, accurate and shared in a way that was legal, transparent and acceptable to the public.
Clearly such data integration would require software compatibility across multiple databases, and the framework for such infrastructure has already been built in New York City. In 2013, the Mayor’s Office of Data Analytics (MODA) created DataBridge, an analytics platform that integrated databases for 40 agencies.[18] The MODA claims that DataBridge has enabled analytics that have identified stores selling bootlegged cigarettes, restaurants that were illegally disposing of cooking waste, and structures that posed fire risk because of illegal subdivision.[19]
Such outcomes are good examples of different agencies with distinct missions using each other’s data in new ways, which might diverge significantly from their original purpose when collected. While illegal subdivisions seem to directly correlate to a building’s susceptibility to fire — and such cooperation between the building department and FDNY have already been established as part of FireCast 2.0 (See Chapter 2) — the cooking violations involved a more novel approach. In this instance data from the Business Integrity Commission, which certifies that restaurants have a permit to cart offending grease, was compared to geospatial data from the city’s sewer system. The result was the identification of offending restaurants that were dumping their grease in the sewer with 95 percent certainty.[20] Success stories like these suggest that there are many possible sources for data that might affect fire risk ratings.
D. Private Sector Resources
Currently, New York City does not use privately-owned networks such as Everbridge’s Nixle.com in order disseminate public information and press releases, but such sources should not be discounted in jurisdictions that use them.[21] One of the challenges of using data from Nixle, is that the data are generally not quantitative but qualitative in nature. This makes it difficult to identify, aggregate and assign a risk value to their data without sophisticated programming. For example, if a Nixle message warns the public about a rash of burglaries in a given neighborhood, this may be deemed a risk factor. The fact that neighborhood houses are often unattended, or the buildings have not been updated with modern security measure might, over time, be deemed relevant to fire prediction. However, agencies might also post yearly reminders to Nixle, reminding residents that burglaries occur around the holidays, even if they are not, in fact, occurring in the neighborhood. Without qualitative analysis such data may not be of much value as a way to accurately assign a fire risk rating.
Social media sites such as Twitter and Facebook also employ qualitative data, but the quantity of data is so massive that analytics can be used to aggregate and compare the data in ways that are useful to public safety agencies. Law Enforcement agencies, including the NYPD, already use social media data for predictive policing, but the practice is not without controversy.[22][23] Particular patterns among Tweets or posts might be analyzed in ways that are pertinent to predictive firefighting as well. However, unless the analytics prove to be extremely accurate and clearly in the public interest, the controversy of monitoring social media posts, might not be worth the risk to public trust in the firefighting services.
A public-private partnership between municipalities and industries that collect data relevant to building condition, maintenance, value and historical context might prove even more valuable than monitoring social media, and possibly less controversial.
Two examples of open-source data platforms that might be used to fire risk rating analytics, are real estate websites and insurance databases.
One of the most recognized real estate websites, Zillow.com, provides data on building sale dates, last sale prices, current price estimates, and foreclosure history, all of which might be relevant to fire risk ratings.[24] While some of this information may already be accessible through interagency cooperation in cities like New York that have a data sharing platform in place, it might not be available in others, thus Zillow is another viable option. Zillow also partners with the credit agency, TransUnion, in order to measure negative equity in particular buildings, a factor which might prove to be an indicator of arson risk.[25] Because Zillow is an open source and publicly accessible website, use of its data might be viewed more favorably than the use of government agencies’ data, even if it is of a similar nature.
Insurance databases, such as Location, Inc. and Sanborn, are another source of privately collected information, which are specifically designed to assess fire risk ratings. Both companies use a combination of historical data and geospatial mapping to help assess risk.[26] In their public relations material, Location, Inc. claims to use data based on “millions of policy years of fire loss data and over 20 proprietary geospatial models” to predict “behavior-related” fire risk incidents stemming from negligence and arson to create its risk index.
Given that industries such as real estate marketing and insurance underwriting have a profit motivation for increasing their data analytics capabilities, it is possible that they may provide better indices of fire risk than many municipalities have internal resources to analyze. Cost factors and incompatibility between data systems might be present barriers to incorporating such data into public fire prediction analytics, but further research is recommended to evaluate a partnership with these industries.
E. Conclusion
It is important to mention that the relevance of particular types of data to a structure’s fire risk may not be immediately evident. But, over time, specific data sets could be correlated and prioritized to determine their relationship to risk. For example, a high incidence of vandalism complaints might suggest a building is neglected, in poor condition or in need of maintenance, which might affect its risk rating. But if the frequency of vandalism complaints were matched with real estate records that indicate high property values, the complaints could have very little impact on risk. They might simply indicate that the property owners are reporting vandalism more often because they are sensitive to its impact on those values. As data collection grows in volume and diversity, the risk calculations can be continually improved. In order for this to happen, it is critical that the FDNY field personnel report outcome data as discussed in Chapter II. Thus, the relationship between useful predictive data and useful outcome data becomes cyclical and symbiotic. As will be suggested in the conclusion of this white paper, such symbiosis can only happen if cooperation between agencies becomes ingrained in the culture of municipal government. This may require incentives by federal agencies that could provide financial support, training and education, and access to standardized data platforms to enhance analytical capabilities.
Chapter IV: POLICY, LEGAL, AND IMPLEMENTATION CONCERNS — DeMaise
A. Introduction
Building inspections are one of the few proactive measures that firefighters can perform to reduce the property destruction and casualties that result from fires, building collapses, and other natural and manmade disasters. It was recently revealed that there was a lack of fire inspections at a warehouse in Oaklyn, CA where a fire took the lives of 36 people and injured scores.[27] In order to move away from the city’s previous attempts to locally track inspections on handwritten cards maintained at the Fire Company level, the FDNY engaged in a program called FireCast 2.0 in March of 2013.[28] The program was designed to examine characteristics of buildings that had fires and compare them to similar ones. Since then the program has been subject to several upgrades, however our research indicates that it is falling short of meeting the predictive rescue needs of our nation’s premier fire department. Many of the challenges are related to technical deficiencies, however policy and cultural issues are also inhibiting the growth of this program.
B. Policy and Technical Details
The Firecast 3.0’s cycle of data acquisition, processing, and response has been proven effective and will serve the immediate needs of our program. However, our goal will be to add data from disparate sources that will enhance the current risk assessment program. Merging data from various structured and unstructured platforms will prove very challenging in the near term. These challenges will be derived from data housed in legacy systems as well as emerging proprietary technologies that do not want to work with other platforms. To mitigate these problems, the National Fire Protection Agency (NFPA) has issued new codes and standards for fire service data analysis. NFPA 950, Data Development and Exchange for the Fire Service and NFPA 951, Guide to Building and Utilizing Information were written in anticipation of increased data inquiry by fire departments.[29] These standards will require any data that may be developed in the future to be memorialized in a uniform format. Standardization includes the uniformity of date, incident typing in National Fire Incident Reporting System (NFIRS) coding and text in American Standard Code for Information Interchange (ASCII) characters.[30]
NFPA 950 provides that:
● 5.3.3:The authority having jurisdiction (AHJ) shall adopt extensible markup language (XML) as defined in ISO 8879 and Geographic Markup Language (GML) as defined in ISO 19136 as the data coordination and sharing standard.
The International Standards Organization (ISO) requirements in NFPA provide credibility to the program’s efforts to unify the format of data. Extensible Markup Language (XML) is a simple, very flexible text format derived from SGML (Standard Generalized Markup Language.) SGML is a standard for defining generalized markup languages for documents. Most computer literate people are familiar with HTML or Hyper Text Markup Language. HTML is also considered to be an application of SGML and is used for creating web pages and web applications. The World Wide Web is composed primarily of HTML documents transmitted over browsers and servers using Hyper Text Transfer Protocol (HTTP). XML however is playing an increasingly important role in the exchange of a wide variety of data on the Web and elsewhere.[31] By standardizing fire service data now, future data acquired from sensors on firefighters equipment, vehicles, and other elements of the emerging Internet of Things will prove valuable to the predictive algorithm. However, how can data from outside sources be integrated into our program?
Data processing systems need to employ programming languages that are optimized for data analysis and computing efficiency. Systems such as Hadoop use Structured Query Language (SQL), a standardized database language that can process disparate data sources, may meet the mission requirements of public safety. Industry analysis indicates that some of these programs are not as efficient due to the lag associated with translating disparate data into a single format.[32] Unfortunately these lags can detract from the services performed by first responders in critical incidents and ultimately inhibit cultural acceptance of technology by service members.[33] In the future, increasing processor capabilities will improve and these enormous amounts of data will be analyzed and distributed to the field with positive results.
C. Program Roadmap and Legal Considerations
The U.S. Department of Commerce has published a Public Safety Analytics Roadmap that suggests agencies consider the following data concerns before launching into an analytics project.
a.) Privacy: If accessing citizen data, public safety organizations should effectively articulate what the data is to be used for and why it access should be authorized. Privacy violations can result from intentional or unintentional acts, therefore effective policy and oversight are essential.
b.) Regulatory: Federal, state, and local considerations may present challenges for public safety agencies to access data needed to aid fire prevention. Agencies using this approach need to also understand the impact that analysis of this data may before implementing the program.
c.) Staffing: As data analysis programs move forward, personnel will be needed to design programs, support its daily operation, as well as train first responders in its use.
d.) Policy: Standard operating procedures will be needed before analytics can be properly used. If procedures are not in place the data may be misused or not used as effectively as it should.
e.) Training: Initial training prior to deployment of the technology as well as refresher training will ensure that users are maximizing the potential of the data and its subsequent analysis.
f.) Funding: These programs require a strategic funding approach for data storage, system maintenance, and program updates as new and emerging services become available. Hardware, training, and staffing needs will also require fiscal support to maintain an effective program.
g.) Clear Benefit: In some cases analytics programs provide an unforeseen burden upon busy public safety agencies. The benefit of these programs must clearly outweigh the burden upon the agency, both fiscal and operational.
h.) Legal Authority: Resources that are being shared between public and private sectors require significant levels of approval. Legal precedence may be required before engaging in collaborative research and development projects.[34]
Data integration is the combination of technical and business processes used to combine data from disparate sources into a meaningful and valuable way.[35] The ability to merge disparate forms of data is very challenging because the numerous data sources and the lack of common data standards.[36] NIST has identified that even data sets that are properly structured have varying levels of quality and formatting leading to challenges related to reconciling the information for processing.[37]
NIST recommends establishing clear data governance programs for the “overall management of the availability, usability, integrity, and security of the data employed in an enterprise.” The first step to implementing one of these programs is to define the owners of the data or its custodians. Policy must be developed that specifies who is accountable for the data including its accuracy, accessibility, consistency, completeness, and updating. Processes must be defined on how data will be stored, archived, backed up, and protected. A set of standards and procedures must be developed that defines how the data is to be used by authorized personnel. Finally, a set of control and audit procedures must be in place to ensure compliance with government regulations.[38]
Gathering, analyzing and distributing ever-evolving streams of data will be no easy task. Sensors, weather, and dynamic databases that are continuously changing every hour may be inhibited by Internet connection disruptions. Agencies that use data such as this will need to design an architecture that can connect to external data sources and incorporate relevant information into application workflows and anticipate disruptions.[39] Algorithms and statistical models may enable analytics to be conducted in spite of disruptions in connectivity. This would be more efficient than processes that store data before analysis.
However, some of the data received by public safety agencies contains personal information. When receiving data that contains Personally Identifiable Information (PII) many legal mandates need to be considered. The Privacy Act of 1974, the Health Insurance Portability and Accountability Act of 1996 (HIPPAA) and the Confidential Information Protection and Statistical Efficiency Act (CIPSEA) all impact data gathered by government agencies.[40] These mandates recognize the threat that the IT environment poses to this data and imposes restrictions on its handling, retention and destruction. Agencies that access PII are required to establish administrative, technical, and physical safeguards. One way to navigate around these requirements is to anonymize the PII data prior to collection and analysis. Once that data has been de-identified it may be retained and analyzed for its value in fire prevention. This would be particularly important if the fire department is to make use of any data from police departments, health industries, or private sector that may contain PII.
D. Implementation
Section 164.502 of the privacy rule permits two methods for the de-identification of data. The expert identification method allows for a properly trained and experienced individual to determine if the methods to identify the data are effective enough to limit the risk of identification of an individual to “very small”[41] The second method, which we will use for this research program is referred to as the safe harbor method. The safe harbor method is a prescriptive de-identification method that specifies identifiers that will be removed prior to the agency taking possession of the data.[42] This approach will require the use of an algorithm that we will develop to eliminate the possibility of privacy act violations for any of the data processed through this program.
Public safety is beginning to appreciate advances in technologies that can distribute data real time. This can enhance the ability of first responders to arrive at critical incident locations faster than their predecessors and better prepared for the event. Data can be distributed in a variety of formats depending upon user needs. Geospatially represented data can be very helpful in defining the event as it is unfolding and the impact it may have on the region. Emerging geospatial awareness platforms can map fixed infrastructure and operational status. Dynamic forms of data such a first responder location and computer aided dispatch information all lend to a more complete picture of situational awareness in real time. Analysis from NIST indicate that public safety will be able to realize significant improvements in live streaming data processing in the next five to ten years.[43] This emerging capability will aid fire prevention programs by delivering data to the firemen in the field for more timely proactive measures.
E. Conclusion
NIST Analytics Working Group suggests the following workflow be incorporated for agencies that wish to engage in active data analytics programs:
This workflow engages the user in an organized process of developing an effective data analysis program.[1] It also encourages the user to conduct an iterative process that returns to the design phase to review the status of the program for improvements as new technologies and relevant data sets emerge. Programs such as FireCast will continue to evolve. It is our hope that additional data sources, program objectives, and an understanding of data related legal concerns will lead to a more comprehensive, predictive rescue program for fire departments of the United States.
CHAPTER V. CONCLUSION: LOOKING TO THE FUTURE — Wilson
A. Introduction
The incorporation and use of Big Data is an integral component of our collective economic fortunes, health and welfare, and security as a nation. As this report makes clear, it is also an increasingly important component of the discipline of fire prevention and response. In today’s environment, Fire agencies are awash in data; increasingly, the information generated in the course of fire operations is disseminated through enabling tools, like Raven-SCOUT described in Chapter I, that enhance tactical situational awareness and command functions. Translating the real-time capture and generation of tactical data — and combining it with additional publicly available and governmental datasets — for predictive modeling of risk is an evolutionary next step for fire services. The potential applications are transformational and holistic and include: providing accurate predictions on specific buildings in fire-prone urban (and rural) areas most at risk; helping inform targeted public education and outreach regarding fire dangers in identified high-risk areas; building better training curriculum tailored to a given firefighter’s needs based on the historical and environmental details of his/her firehouse; preventing long term health risks (e.g.,) based on analysis of a given firefighter’s career history of exposure to known cancer-causing substances; or influencing budgeting decisions.[1]
A. Existing Challenges and Constraints
Not surprisingly, the embrace and transition of fire and emergency response agencies toward Big Data has also surfaced acute challenges of operating — managing, integrating, and disseminating — large unstructured datasets that serve mission-enhancing purposes.[2] For example, as noted in Chapter II, FDNY’s greatest challenge may well be organizational capacity: including organizational reticence to share data; also, data governance challenges for a department not traditionally designed and resourced with the human capital required for analytical data mining, evaluation, and sense-making.
As described in chapter III, because the use of technology and large unstructured data is a relatively new venture for fire services, leveraging the “best practices” of sister disciplines — such as law enforcement — that are already implementing predictive analytic tools makes sense. Part of this is that the purpose and usage of collection is transparent and reproducible help to ensure that fire services maintain public trust while optimizing public service. For example, ensuring that any data collected are secure and anonymous — in terms of personal identification information (PII) — while less of an issue than it is for law enforcement, is crucial in assuring that the relative high level of public trust of fire services is maintained.
A cornerstone of any successful predictive model is the incorporation of an evaluative process. Advances in computing power and data analytics incentivize the collection of ever larger volumes of unstructured data. This is partially a necessary function of machine learning: as data collection grows in volume and diversity, the risk calculations can be continually improved; yet it may also lead to bureaucratic pathologies of goal displacement, where the inputs related to a specific goal begin to override the desired outcomes the inputs are meant to bring about.[3] As it pertains to Big Data, goal displacement occurs when the practice of data aggregation becomes a priority in and of itself. Additionally, the feasibility of processing and analyzing increasingly large and disparate patterns via machine learning algorithms becomes an end rather than a means to predict and more efficaciously prevent harm. Therefore, ensuring that forecasts made via Big Data analysis are compared against actual results is an indispensable component of the process. As it pertains to fire services, this makes the collection of incident data imperative.
C. Recommendations
This paper identifies the following policy recommendations that deserve prompt Administration attention. These are:
· Empower the Organizational Capacities of Fire Service Departments to Implement Big Data. The Administration should empower state and local departments by building governmental expertise that can be shared with the FDNY and other fire services investing in computational tools for predictive firefighting. Such assistance should help facilitate the implementation of systems by providing templates and best practices that anticipate and account for legal, resource, and technological challenges associated with Big Data frameworks. Success will require symbiosis that can only happen if cooperation between agencies becomes ingrained in the culture of municipal government. This may also require incentives by federal agencies that could provide financial support, training and education, and access to standardized data platforms to enhance analytical capabilities.
Provide Resource Assistance. The Administration should press Congress to pass legislation that enhances national data standards that mirror and enhance the efforts currently underway by the NFPA and NIST. Symbiosis regarding data collection and uniformity standards can only happen if cooperation between agencies becomes ingrained in the culture of municipal government. This may require incentives by federal agencies — akin to National Institute of Justice (NIJ) and FEMA grant and research calls related to prevention issues like counter-terrorism.
[1] Jesse Roman, “Training the Brain,” NFPA Journal, November 1, 2016, available at: http://www.nfpa.org/news-and-research/publications/nfpa-journal/2016/november-december-2016/features/big-data
[2] Kevin C. Desouza, Realizing the Promise of Big Data: Implementing Big Data Projects, IBM Center for The Business of Government, Using Technology Series (2014), available at: https://www.hsdl.org/?view&did=752145
[3] In his work on bureaucracy the sociologist Robert Merton detailed the structural inefficiencies inherent to bureaucracies, describing the tendency to supplant the mission of a bureaucratic entity with the rules used to attain that mission. Merton termed this perversion of means over ends “goal displacement” and concluded the assumed efficacy of divvying up bureaucratic roles — if structured poorly — could just as likely lead to greater inefficiency as to a more effective division of labor.
[1] Randy Rieland, “How Data and a Good Algorithm Can Help Predict Where Fires Will Start,” Smithsonian Magazine, March 2, 2015, http://www.smithsonianmag.com/innovation/how-data-and-good-algorithm-can-help-predict-where-fires-will-start-180954436/
[2] Brian Heaton, “New York City Fights Fire with Data,” Government Technology Magazine, May 15, 2015, http://www.govtech.com/public-safety/New-York-City-Fights-Fire-with-Data.html
[3] Ibid.
[4] Randy Rieland, “How Data and a Good Algorithm Can Help Predict Where Fires Will Start,” Smithsonian Magazine, March 2, 2015, http://www.smithsonianmag.com/innovation/how-data-and-good-algorithm-can-help-predict-where-fires-will-start-180954436/
[5] Brian Heaton, “New York City Fights Fire with Data,” Government Technology Magazine, May 15, 2015, http://www.govtech.com/public-safety/New-York-City-Fights-Fire-with-Data.html
[6] Reuven Blau and Jonathan Lemire, “New York City Confirms new 911 system is Mess,” New York Daily News, May 4, 2012, http://www.nydailynews.com/new-york/new-york-city-confirms-new-911-system-mess-article-1.1072941
[7] Ibid.
[8] David Elner, “NYC Data: A primer on property data,” December 4, 2015, http://blog.davidelner.com/nyc-property-data-primer/
[9] Rich Calder, “Just 5 percent of FDNY are actually for fires,” New York Post, December 10, 2015, http://nypost.com/2015/12/10/just-5-percent-of-fdny-calls-are-actually-for-fires/; Jennifer Fermino, “FDNY responded to record number of fire calls in 2015, but deaths were down,” New York Daily News, January 8, 2016, http://www.nydailynews.com/new-york/fdny-responded-fires-2015-deaths-article-1.2490523
[10] Fire Service Image Task Force, Taking Responsibility for a Positive Public Perception, (Fairfax, VA: International Association of Fire Chiefs, 2013)
[11] “Vehicular Searchs,” Justia: US Law, 2016, accessed December 11, 2016, http://law.justia.com/constitution/us/amendment-04/15-vehicular-searches.html
[12] “Body Worn Camera Toolkit: Policy,” Bureau of Justice Assistance, 2016, accessed December 11, 2016, https://www.bja.gov/bwc/Topics-Policy.html
[13] American Civil Liberties Union (ACLU), “Statement Of Concern About Predictive Policing By Aclu And 16 Civil Rights Privacy, Racial Justice, And Technology Organizations,” ACLU, August 31, 2016, https://www.aclu.org/other/statement-concern-about-predictive-policing-aclu-and-16-civil-rights-privacy-racial-justice
[14] President’s Task Force on 21st Century Policing, Final Report of the President’s Task Force on 21st Century Policing, (Washington, DC: Office of Community Oriented Policing Services, 2015), 2
[15] NYC Open Data, “About NYC Open Data,” The City of New York, 2016, Accessed December 10, 2016, https://data.cityofnewyork.us/stories/s/About-This-Site/9v8v-e23e
[16] Ibid.
[17] “NYC 311: The Official Website of the City of New York,” The City of New York, 2016, http://www1.nyc.gov/311/index.page
[18] Rutrell Yasin, “How Analytics is Making NYC’s and Buildings Safer,” GCN, October 4, 2013, https://gcn.com/articles/2013/10/04/gcn-award-nyc-databridge.aspx
[19] Ibid.
[20] Alan Feuer, “The Mayor’s Geek Squad,” The New York Times, March 23, 2013, http://www.nytimes.com/2013/03/24/nyregion/mayor-bloombergs-geek-squad.html?pagewanted=all&_r=0
[21] “Everbridge Nixle,” Nixle.com, accessed December 10, 2016, https://www.nixle.com
[22] Richard Winton, “L.A. Will Roll Out Predictive Policing by Monitoring Tweets in Real-Time,” Los Angeles Times, September 23, 2016, http://www.latimes.com/local/lanow/la-me-ln-predicting-hate-crimes-20160922-snap-story.html
[23] Josmar Trujillo, “Op-Ed: Why NYPD’s ‘Predictive Policing’ Should Scare You,” CityLimits.org, January 29, 2015, http://citylimits.org/2015/01/29/why-nypds-predictive-policing-should-scare-you/
[24] Zillow, “Data,” Zillow, accessed December 15, 2016, http://www.zillow.com/research/data/
[25] Ibid.
[26] Sanborn, “Sanborn Fire Insurance Maps,” accessed December 12, 2016, http://www.sanborn.com/sanborn-fire-insurance-maps/
Location, Inc., “Location, Inc. Introduces New Index to Help Insurers Assess Structure Fire Risk Based on Human Negligence,” January 27, 2016, http://www.locationinc.com/media/fire-risk-assessment-pr/
[27] Jusha Elinson, “No Fire Inspections of Warehouse where 36 People Died, Oaklyn Fire Chief Says,” Wall Street Journal, December 13, 2016, http://www.wsj.com/articles/no-fire-inspections-of-warehouse-where-36-died-oakland-fire-chief-says-1481667873?mod=whatnext&cx_navSource=cx_picks&cx_tag=poptarget&cx_artPos=4#cxrecs_s.
[28] NFPA, “How RBIS Works.”
[29] Jesse Roman, “In Pursuit of Smart,” NFPA Journal, November 2014, http://www.nfpa.org/news-and-research/publications/nfpa-journal/2014/november-december-2014/features/in-pursuit-of-smart.
[30] NFPA, “NFPA Standards Development Site, 950,” accessed on November 30, 2016, http://submittals.nfpa.org/TerraViewWeb/ViewerPage.jsp?id=950-2015.ditamap&toc=false&draft=true.
[31] W3C, “Extensible Markup Language (XML) , accessed on December 7, 2016, https://www.w3.org/XML/
[32] Ye Zhou, Amir, Esmailpour, “Improvements in Big Data Hadoop,” ASEE 2014 Zone I Conference, April 3–5, 2014, University of Bridgeport, http://www.asee.org/documents/zones/zone1/2014/Student/PDFs/215.pdf
[33] Charles Werner, “Technology, Change And The Cheese,” February 28, 2001, http://www.firehouse.com/article/10544520/technology-change-and-the-cheese
[34] Ryan Felts, Marc Leh, Tracy McElvaney, “Public Safety Analytics R & D Roadmap,” accessed on November 13, 2016, 26. http://nvlpubs.nist.gov/nistpubs/TechnicalNotes/NIST.TN.1917.pdf.
[35] IBM, “What is Data Integration?” accessed on November 30, 2016, http://www.ibm.com/analytics/us/en/technology/data-integration/.
[36] Ryan Felts, Marc Leh, Tracy McElvaney, “Public Safety Analytics R & D Roadmap.”
[37] Ibid., 26.
[38] Margaret Rouse, “Data Governance,” accessed on November 30, 2016, http://searchdatamanagement.techtarget.com/definition/data-governance.
[39] Ibid.
[40] Simon Garfinkle, “Deidentifying Government Data Sets,” NIST, December 2016, http://csrc.nist.gov/publications/drafts/800-188/sp800_188_draft2.pdf.
[41] Jacqueline Klosak and Anna HSIA, “HHS Issues New Guidance on De-identifying Heath Protected Information,” January 2013, https://docs.google.com/document/d/1k1dDoVZchYau7lst0EZ1w4BYJpO3fexiSBVQ6Gt5PVQ/edit#.
[42] Simon Garfinkle, “Deidentifying Government Data Sets,” 32.
[43] Ryan Felts, Marc Leh, and Tracy McElvaney, “Public Safety Analytics R & D Roadmap,” 38.
