Introducing Dow Jones Hammer
Today, Dow Jones Tech is pleased to open source Dow Jones Hammer, a DevSecOps tool that lets you identify and proactively fix misconfigurations in cloud workloads.
Built by the Dow Jones Product Security team, Dow Jones Hammer is designed to scale security across your multiple cloud environments by identifying insecure configurations, automatically remediating problems and enabling security guardrails for your products.
In today’s Agile development world, developers can provision and deploy an entire suite of products with the click of a button, meaning security is often overlooked in development and deployment. To keep up with quick deployments, security teams need tools that work at the speed of business.
Dow Jones Hammer was developed to secure our AWS environment, with the help of native AWS resources like CloudFormation, Lambda functions, Dynamodb, SNS, CloudWatch and EC2 compute instances. Technologies the tool uses include Python 3.6, AWS, Atlassian Jira, Slack, and Serverless.
The Secret Sauce: How It Works
Dow Jones Hammer is broken down into four components — identification, correlation, reporting and auto-remediation.
In the identification stage, Dow Jones Hammer runs from the central account and scans for policy violations across all AWS accounts in all regions using Lambda functions, storing detected issues in a DynamoDB vulnerability database. Those issues are then analyzed in the correlation stage, identifying the issue’s ownership, criticality, and remediation steps needed to fix the issue. If the issues are easily exploitable, it marks them critical and stores metadata information back to the vulnerability database.
During reporting, Dow Jones Hammer provides near real-time reporting on Slack, as well as logging bug tickets into Jira. If an issue is critical enough, Dow Jones Hammer will set a counter for developers to fix the issue on time.
Some business-critical functions will go through auto-remediation if the engineering team doesn’t fix the issues in the given time-frame. The auto-fixes are designed so that ideally they do not impact any production systems. All actions taken against the resources are logged into the bug ticket, as well as the previous state of the resources for developers to review.
Security Services
The current Dow Jones Hammer toolkit comes with detection and protection capabilities with the following suite of services which deviates from established policies and best practices.
- IAM Inactive Keys: Detects and deactivates IAM access keys unused for “N” number of days.
- IAM Keys Rotation: Detects the access keys which have not been rotated for N days, and helps in deactivating them. Note: Deactivating in-use IAM keys should be done in accordance with product teams, as they may be used within source code.
- S3 Buckets Public Access — ACL: Detects S3 buckets which are publicly accessible and have read/write ACL(Access Control List) permissions.
- S3 Buckets Public Access — Bucket Policy: Detects S3 buckets which are publicly accessible and have overly-permissive read/write Bucket policies (e.g., *:* ). Locks down the bucket to be accessible over RFC 1918 private networks. Note: Remediating S3 bucket policies should be enabled in accordance with product teams.
- Insecure Services: Detects security groups open from public internet (e.g., 0/0, 134.x.x.x,…). Locks down to RFC 1918 private networks for non-web ports.
- CloudTrail Logging Issues: Detects CloudTrail logging status and permission issue.
- EBS Un-encrypted Volumes: Detects not encrypted at rest EBS volumes.
- EBS Public Snapshots: Detects publicly accessible EBS snapshots.
- RDS Public Snapshots: Detects publicly accessible RDS snapshots.
We are constantly working on enhancing the way Dow Jones Hammer works by adding more protection capabilities and incorporating more integration with other security tools for issue identification. Our goal is to make Dow Jones Hammer’s identification of issues as close to real-time as possible. We are also looking into avenues for providing run-time scanning and protection capabilities with the tool.
Statistics
The following graph in Jira shows the impact of Dow Jones Hammer in hundreds of our Dow Jones AWS accounts.
Each upward spike depicts a feature release, followed by auto-remediation of that feature. The auto-remediation step results in a downwards spike, illustrating the immediate reduction of the risk.
Impact at Dow Jones
Providing fast feedback to our developers makes them accountable for their defects, thus nudging the developers to resolve issues quickly. This instills the confidence that our development community will prioritize and fix issues in a timely manner. Most of the issues reported by Dow Jones Hammer are fixed by developers within a few days of reporting them.
Based on our approach at Dow Jones, we recommend running Dow Jones Hammer initially in “detection mode,” then slowly turn on “auto-remediation mode” in Development Accounts (for each feature), followed by Production Accounts with appropriate Change Control practices.
At Dow Jones, our product security team is committed to continually enhancing the developer experience with scalable and developer-friendly security tools. We believe in moving swiftly by lighting gentle fires to help developers solve issues and are excited to see how Dow Jones Hammer helps other organizations reduce exposure risks.
You can check out Dow Jones Hammer here. We look forward to your feedback and collaborating on further development.
If you have any questions, you can reach us at hammer@dowjones.com or on GitHub.
