Reshaping healthcare through big data: an Australian perspective

Lachlan McGowan
Sep 30, 2019 · 10 min read

In recent years, companies have extracted an unprecedented amount of information from ‘big data’. Big data has the potential to revolutionise various industries, influencing better decision-making in order to improve business outcomes (Gandomi & Haider, 2015). Organisations must continually update their capacity to store and interpret data using the latest data science tools in order to gain a greater understanding of the patterns and trends that they wish to exploit. Perhaps the most notable industry where big data can be used to great effect is healthcare, as the benefits of analysing big data from eHealth such as electronic health records (EHRs) can lead to higher quality health outcomes for patients (Raghupathi & Raghupathi, 2014). However, big data in healthcare is subject to intense scrutiny due to the personal nature of health data (Bhatt, Dey & Ashour, 2017). For example, researchers collecting statistical data on the likelihood of HIV/AIDs in certain demographics will require data on a patient’s age, gender, sexuality, and sexual experiences in order to improve health outcomes for sufferers of the disease. As a result of the personal nature of such data, many countries including Australia lack the necessary foundations to properly analyse it as lawmakers struggle to develop policy that balances the privacy of individuals with the exchange of health data for analytics.

Significant problems that require innovative solutions

Australia is considered a healthy country, with a high average life expectancy. Males are expected to live for 79.5 years, and females for 84 years (Australian Institute of Health and Welfare, 2018). However, healthcare in Australia is currently facing an ongoing crisis due to an aging population (Figure 1), and the growing demand for higher quality tests, treatments, and technologies is placing a strain on health services (Australian Government, 2016). As a result, the cost of healthcare has risen and the quality has slowly declined (Australian Institute of Health and Welfare, 2018). Innovation in the form of big data may alleviate these issues by measuring the value of healthcare, improving medical research, and paving the way for more personalised and targeted healthcare. This will help to reduce the strain on health services posed by the aging population and demand for higher quality health services.

Figure 1: The expected proportion of people over the age of 65, 1970–2050

(Garcia, 2016)

Big data can measure the value of healthcare

Of all the potential opportunities for big data in healthcare, the ability to measure the effectiveness of healthcare and identify the most cost-effective patient diagnosis and treatment strategies is the most beneficial. In Australia there are many instances where medication, tests, and treatments administered to patients are of little use to the patient’s overall health, and in some cases, had the potential to cause great harm (Garcia, 2016). In 2013, the Australian Commission on Safety and Quality found that 24.4% of antibiotic prescriptions were deemed inappropriate, and in addition, the Medicare Benefits Schedule report of 2016 found that the number of pathology tests increased from 46% in 2003–04 to 54% in 2013–14 (Australian Commission on Safety and Quality in Healthcare 2013; Australian Government 2016). These statistics are very concerning as the number of inappropriate prescriptions and tests could indicate an over-diagnosis and over-treatment of symptoms for some patients (James et al. 2015). Big data can remedy this issue as it can highlight the effectiveness of medicines, tests, and treatments that contribute to higher quality healthcare, thus reducing the number of inappropriate prescriptions made and allowing for greater efficiency in healthcare delivery.

Big data can improve medical research

Big data in healthcare may also improve the quality of medical research. In Australia health data is accessible through EHRs which provide real time data on a patient’s medical history. EHRs are beneficial as they can be used to coordinate care between healthcare providers and create better clinical trials, instrumental in developing better medication for treatment (Raghupathi & Raghupathi 2014). However, according to the Australian Government in 2013, the access of EHRs by users and practitioners has diminished (Figure 2). While the merging of EHR datasets allows researchers to examine and predict the early onset of diseases, many practitioners and the general public are fearful of the risks of sharing health data (Andrews, Gajanayake & Sahama, 2014). In addition, the Australian Government alongside policy makers have restricted the linkage of health data for medical research (Canaway et al. 2019). Thus, if big data is to improve the quality of medical research, the Australian Government will need to promote the benefits of exchanging health data to the public.

Figure 2: Weekly accesses for EHR, July 2012-December 2013

(Garcia, 2016)

Personalised and targeted healthcare

Big data in healthcare also has the potential to transition patient care from a broad ‘one size fits all’ approach to a more personalised and targeted approach, which is beneficial as not every patient will be experiencing the same symptoms or need the same treatment. Personalised healthcare is a relatively new concept and is defined as any healthcare that is specifically tailored towards an individual’s disease or predisposition to disease (Mathur and Sutton, 2017). For example, WebMD uses a form of personalised healthcare where patients can input their symptoms as keywords to search medical literature covering thousands of medical conditions and find the condition that best fits their symptoms. Another example of personalised healthcare is patient monitoring apps on smart devices such as health and fitness apps, which allows for greater communication and information sharing between patients and their practitioners (Panahiazar et al. 2014). This is especially useful as it provides practitioners with real-time information on their patients’ progress, which enables specific care to be provided to a patient’s individual needs, instead of care that broadly covers symptoms experienced by the patient.

The impact of big data on Australian healthcare

The implementation and analysis of big data in healthcare will have a major impact in Australia, with the potential to revolutionise healthcare with a transition from a paper-based system of care to an EHR paperless system. This will likely reduce medication errors and wait times, providing greater efficiency overall. The application of big data in healthcare will also provide practitioners and nurses with the most credible base of evidence in undertaking evidence-based clinical practice (EBP).

Greater efficiency

The transition to an EHR paperless system will greatly improve the efficiency and quality of healthcare delivery as medication errors and wait times will be significantly reduced. Medication errors are a significant problem around the world and are usually caused by human error and when patients move between healthcare providers (Wheeler et al. 2018). In a recent study investigating the rate of medication errors in acute care hospitals, the results found that up to 9% of all medication administered to patients were errors (Roughead et al. 2016). Luckily, EHRs can reduce the likelihood of medication errors as they are digital in nature and are updateable. In Australia, waiting time has increased steadily over the years, causing a burden on patients in need of desperate care. According to the Australian Institute of Health and Welfare, the median waiting time for emergency departments and elective surgery has risen from 36 days in 2013–14 to 40 days in 2017–18 (Australian Institute of Health and Welfare, 2018). This shows that the application of big data in healthcare is likely to greatly improve the efficiency of healthcare if implemented correctly.

Greater evidence

EBP is a process that encourages clinicians to seek the best available evidence and critically apply it to an individual patient’s needs (Parrish, 2018). While EBP is imperative in health as the continual study of scientific evidence for an injury or illness can lead to higher quality health outcomes, many practitioners and nurses fail to apply EBP as they lack time to study new evidence and the evidence available is either outdated or inaccurate (Weiss, 2017). By implementing big data in healthcare, practitioners and nurses will be able to easily locate the best available evidence as improvements in medical research will reveal which evidence is more accurate.

Barriers to the implementation of big data in Australian healthcare

While the benefits of analysing big data in healthcare are numerous, the barriers preventing big data are just as challenging. In order to gain the best knowledge from big data, organisations must continually update their capacity to store and interpret data. Unfortunately, current policy settings in Australia are too restrictive and there are privacy concerns surrounding the security of exchanging personal health data (Andrews, Gajanayake & Sahama, 2014). In order to address these barriers and allow companies to analyse health data effectively, policy settings must address significant legislative and social challenges.

Privacy concerns

The greatest barrier hindering the innovation of healthcare in Australia is the prevalence of individual and public concerns regarding the security of health data. In July 2012, the Australian Government attempted to utilise EHRs by establishing Australia’s first national EHR system: My Health Record. The aim was to improve the exchange of health data between healthcare providers and promote higher quality health outcomes, however public opinions regarding the government’s ability to safeguard data was mixed, with many criticising the initial three-month deadline to ‘opt out’, believing that the government was accessing their health data without consent (El Emam, Moreau & Jonker, 2011). When introducing the My Health Record system to the Australian public, the government failed to explain the health benefits of such a system, or the securities that would be put in place to protect the private information of users. In a national survey conducted by the Internal Medicine Journal, 82% of respondents preferred control over their health record and only 66% agreed to have their records shared between all their healthcare providers (Lehnbom, Brien & McLachlan, 2014). As a result of government communicative incompetence, engagement with the system plateaued. Moving forward, it is imperative that the government provide greater transparency over the benefits of sharing health data so the necessary social license can be generated to support such endeavours (Canaway et al. 2019).

Policy developments

As a result of My Health Record, lawmakers were quick to respond to the public’s privacy concerns and enacted strong legislation which restricted the access of big data, which made it difficult for researchers to analyse data to support better healthcare outcomes (Garcia, 2016). In Australia, access and linkage of health data is restricted by The National Health Act 1953, which restricts the linkage of data between MBS and PBS items, The My Health Records Act 2012, which restricts an individual’s health data from being collected, used or disclosed without consent, and proposed amendments to the Privacy Act of 1988, which makes it a criminal offence to re-identify or encourage the re-identification of de-identified datasets. To remedy this issue, the government will need to create less restrictive policy settings that balances the privacy of individuals with the exchange of health data for analytics, which will allow researchers to identify new medication, tests, and treatments that correlate with greater quality health outcomes.


To summarise, the benefits of implementing big data in Australian healthcare are numerous, as big data can be used to evaluate the effectiveness of medicine, tests, and treatments administered by healthcare providers, which in turn could lead to greater quality healthcare and allow for more personalised healthcare. Big data can also reduce medication errors, provide greater efficiency in healthcare delivery, and support evidence-based practice. However, for big data to become instrumental in improving healthcare in Australia, significant privacy concerns must be addressed by promoting the benefits of exchanging of health data and creating a system where health data is easily accessible and secure. Additionally, policy settings must balance the privacy of individuals with the free exchange of health data for analytics, as the current policy settings are too restrictive and prevent researchers from accessing vital data to improve healthcare outcomes.

Reference List

Australian Government. (2016). Medicare Benefits Schedule Review Taskforce, Interim Report to the Minister for Health 2016, September 2016.

Australian Government. (2016). Reform of the Federation White Paper; Issues Paper 3, Increased Demand (leading to rising health expenditure).

Australian Institute of Health and Welfare. (2018). Australia’s health 2018. Australia’s health series no. 16. AUS 221. Canberra: AIHW.

Andrews, L., Gajanayake, R. and Sahama, T. (2014). The Australian general public’s perceptions of having a personally controlled electronic health record (PCEHR). International Journal of Medical Informatics, 83(12), pp.889–900.

Bhatt, C., Dey, N., Ashour, A. (2017). Internet of Things and Big Data Technologies for Next Generation Healthcare, Springer International Publishing, Cham, Switzerland

Canaway, R., Boyle, D., Manski‐Nankervis, J., Bell, J., Hocking, J., Clarke, K., Clark, M., Gunn, J. and Emery, J. (2019). Gathering data for decisions: best practice use of primary care electronic records for research. Medical Journal of Australia, 210(S6), pp.S12-S16.

El Emam, K., Moreau, K. and Jonker, E. (2011). How Strong are Passwords Used to Protect Personal Health Information in Clinical Trials?. Journal of Medical Internet Research, 13(1), p.e18.

Gandomi, A. and Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), pp.137–144.

Garcia, R. (2016). The Mckell Institute Big Data, Big Possibilities How Australia Can Use Big Data for Better Healthcare, December 2016

James, R., Upjohn, L., Cotta, M., Luu, S., Marshall, C., Buising, K. and Thursky, K. (2015). Measuring antimicrobial prescribing quality in Australian hospitals: development and evaluation of a national antimicrobial prescribing survey tool. Journal of Antimicrobial Chemotherapy.

Lehnbom, E., Brien, J. and McLachlan, A. (2014). Knowledge and attitudes regarding the personally controlled electronic health record: an Australian national survey. Internal Medicine Journal, 44(4), pp.406–409.

Mathur, S. and Sutton, J. 2017, Personalized medicine could transform healthcare, Biomedical Reports, vol 7, no 1, pp.3–5,.

Panahiazar, M., Taslimitehrani, V., Jadhav, A. and Pathak, J. (2014). Empowering personalized medicine with big data and semantic web technology: Promises, challenges, and use cases. 2014 IEEE International Conference on Big Data (Big Data), pp.790–795.

Parrish, D. (2018). Evidence-Based Practice: A Common Definition Matters. Journal of Social Work Education, 54(3), pp.407–411.

Raghupathi, W. and Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health Information Science and Systems, 2(1).

Roughead, E., Semple, S. and Rosenfeld, E. (2016). The extent of medication errors and adverse drug reactions throughout the patient journey in acute care in Australia. International Journal of Evidence-Based Healthcare, 14, pp.113–122.

Runciman, W., Hunt, T., Hannaford, N., Hibbert, P., Westbrook, J., Coiera, E., Day, R., Hindmarsh, D., McGlynn, E. and Braithwaite, J. (2012). CareTrack: assessing the appropriateness of health care delivery in Australia. The Medical Journal of Australia, 197(2), pp.100–105.

Sullivan, C., Staib, A., Ayre, S., Daly, M., Collins, R., Draheim, M. and Ashby, R. 2016, Pioneering digital disruption: Australia’s first integrated digital tertiary hospital, The Medical Journal of Australia, vol 205, no 9, pp.386–389,.

Wheeler, A., Scahill, S., Hopcroft, D. and Stapleton, H. (2018). Reducing medication errors at transitions of care is everyone’s business. Australian Prescriber, 41(3), pp.73–77.

Trends in Data Science

Trends in Data Science publishes evidence backed discussion…

Lachlan McGowan

Written by

Trends in Data Science

Trends in Data Science publishes evidence backed discussion papers, outlining trends in data science and the potential for data science to drive innovation across sectors. TDS is written by students on the University of Technology Sydney, Masters in Data Science and Innovation.

Lachlan McGowan

Written by

Trends in Data Science

Trends in Data Science publishes evidence backed discussion papers, outlining trends in data science and the potential for data science to drive innovation across sectors. TDS is written by students on the University of Technology Sydney, Masters in Data Science and Innovation.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store