How to prevent Insufficient Logging and Monitoring

5 min readFeb 14, 2019

Security in web applications is one of the top most important topics in the security environment. After all, the web application at the forefront is the interface to the Internet. The OWASP Top-10, the 10 most dangerous vulnerabilities in web applications, contain a vulnerability that is actually not a real one. Due to the “insufficient logging and monitoring”, compromises are sometimes not detected at all or detected much too late. On average, it takes up to seven months for a hacker attack to be detected. Sensors built into the application can provide a remedy, identify attackers on the first attempt and initiate protective measures themselves. Why insufficient logging and monitoring in the top 10 nevertheless has its right to exist and how the OWASP AppSensor project provides more security is the subject of this article.

OWASP A10: Insufficient Logging and Monitoring

In professional environments, logging for traceability of events is important. The risk is that failed logins, high-value transactions or errors are insufficiently logged. The OWASP community has therefore included “A10: Insufficient Logging & Monitoring” in the OWASP Top 10 — even before risks such as cross-site request forgeries (CSRF) or open redirects. The latter has been omitted from the latest ranking of 2017. Many criticized this decision because “logging and monitoring” is not a clear vulnerability like the other risks. Rather, it is a best practice guide to protect the application. OWASP justifies this by stating that logging and monitoring are the cornerstones of a modern and secure system. For others, the real surprise — considering the long time it takes for an attack to be detected — is that logging and monitoring has not yet been on the list.

With regard to this risk, OWASP classifies the chance for attacks based on this vulnerability into “medium”, prevalence “high” and detectability “low”. The effects are listed as somewhat difficult to define, mainly because of the way attacks are initiated.

Why was logging and monitoring added to the OWASP Top 10 list?

Statistics show that identifying attacks is often difficult. The delay before Advanced Persistent Threats (APT) are identified is 98 days in the financial sector and 197 days in online trading. That’s almost seven months. 58 percent of the financial service providers surveyed and 71 percent of online merchants estimate the probability of being able to contribute to a significant improvement in this situation in the coming year to be rather low.
In view of more than 50 network attacks per month at 83 percent of financial service providers and 44 percent of online merchants, this situation is alarming. Dr Larry Ponemon, founder of the Ponemon Institute, explains:

“The most important conclusion of our study is that companies must increase their investments in security, both on the personnel side and in tools and technologies, in order to be able to identify security incidents efficiently and precisely and to react adequately. […] It takes far too long to detect complex threats. Once an attacker has gained access to a network, he has plenty of time to often cause irreparable damage.”

The cost of an attack can be reduced by quickly identifying the threats. A recent Kaspersky study concludes that prevention of cyber attacks is the best way for organizations to prevent costly security incidents. If an attack should occur despite precautions, a quick reaction is required. An immediate identification of the incident costs large enterprises 456,000 US dollars. If the incident remains undetected for one week, the costs even double to 1.2 million US dollars.

Monitoring is required so that the logs do not simply overflow unnoticed on the log server. Logging alone is therefore not enough, much more logs have to be processed and checked regularly. This enables unusual events and anomalies to be detected in good time. Recent experiments show how even attacks can be predicted with the help of machine learning.

When is an application vulnerable?

According to OWASP policy, insufficient logging, detection, and monitoring occur in the following cases:

Verifiable events such as logins, failed logins and high-value transactions are not logged.
Warnings and errors result in no, insufficient or unclear log messages. This includes obscure error logging without sufficient detail for forensics to understand.
Application and API logs are not monitored for suspicious activity.
Logs are only stored locally. Logs that are not backed up run the risk of being deleted by intruders accessing a system. In this way, the intruders conceal their traces, so that the source of the intrusion is not traceable.
Adequate alarm thresholds and reaction escalation processes are absent or ineffective.
Penetration tests and scans by DAST tools (e.g. OWASP ZAP) do not trigger any warnings.
The application cannot detect, escalate or warn against active attacks in real time.

In addition to these errors, there are also organizational reasons why logging and monitoring fail:

Lack of a formal escalation plan after a violation.
Missing automated auditing and monitoring of security frameworks and/or lack of qualified security personnel to analyze log data.
Poor authentication management.
Insufficient training for logging and monitoring.

How can insufficient logging and monitoring be prevented?

An attacker spends a lot of time examining an application or system to find the vulnerabilities. If a system does not have sufficient logging and monitoring, the attacker can calmly search for errors and vulnerabilities. This increases the chance of successfully finding and exploiting a vulnerability. Ideally, the application has monitoring software that alerts the user to this threatening investigation. If not, at least one mechanism is needed to inform about the intrusion of an attacker.

OWASP recommends the following measures according to the risk of the data stored or processed by the application:

All login, access control, and server-side input validation errors should be logged with sufficient user context to identify suspicious or malicious accounts. Logs should be retained for a period of time that allows delayed forensic analysis.
Ensure that logs are created in a format that can be easily used by central log management tools.
High-value transactions should have an audit trail with integrity controls to prevent manipulation or deletion.
Effective monitoring and alerting should be established so that suspicious activities can be detected and responded to in a timely manner.
A response and recovery plan shall be established for incidents, such as NIST 800–61 rev. 2.

There are both commercial and open source frameworks. OWASP names the solutions “OWASP AppSensor”, “Web Application Firewalls” like “ModSecurity” with the “OWASP ModSecurity Core Rule Set” and log correlation software with user-defined dashboards and notifications.

In addition to the measures defined by OWASP, Dave Whitelegg of IBM suggests additional security measures:

A separate and dedicated, security hardened server platform to capture and store events in the audit log.
The use of network time synchronization technology to synchronize system clocks. This also enables automated monitoring tools to analyze event patterns that occur in real time.
Strong access control to logs.
The creation of a formal incident response plan.
Ensuring 24/7 monitoring by implementing a warning system for monitoring personnel.

Chris Bihary of Tap Into Technology believes that human insight is an appropriate tool to protect a system. You should consider the following:

Know your base traffic to determine what is not normal.
Identify the presence of unknown/unauthorized IP addresses in wireless networks.
Be careful with multiple failed login attempts for system authentication and event logs.
Track suspicious network activity after hours.
Investigate inexplicable system reboots or shutdowns.
Keep an eye on services and applications that are configured to start automatically without permission.