Security is a key consideration for many organizations when evaluating adoption of the public cloud. Regulatory requirements, data security, navigation of the shared responsibility model for security between the cloud provider and the organization–the list of considerations is significant.
Addressing all these security concerns in the context of the public cloud offers significant opportunities, however. Most importantly, doing so creates an opportunity to rebuild security processes in an automated, agile manner. Unlike on-premises data centers, which are typically built by hand, use a wide variety of solutions, and lack a definitive source of truth for auditing purposes, the complete configuration of one’s cloud infrastructure is just a few API calls away.
Moreover, in Two Sigma’s case, our governance processes encompass the entire company’s public cloud usage, representing many different engineering teams, with an array of responsibilities. The potential for sprawl is clear in such a scenario. Automating cloud security is therefore not just an option, but a necessity.
Since cloud resources can be fully configured and provisioned via API, security controls and standards can be built into the deployment process, ensuring that the cloud security model adapts to a constantly changing cloud environment. This paradigm is incredibly empowering for security teams, which are traditionally stuck with manual security workflows built on top of a hand-provisioned environment.
This post details an example of how Two Sigma’s cloud security team took advantage of the opportunity presented by moving to the public cloud to build a more agile, more scalable, and ultimately more secure cloud environment.
Goals
We approached the project in a disciplined, systematic way, identifying the following key objectives at the outset:
- To be a business enabler, not a blocker
- To automate security checks and processes
- To minimize the operational burden of security services
Keeping these overarching goals in mind, let’s dive into a specific case of a workflow that previously had been manual and work-intensive, and then into the approach we took to solving this in a more automated, cloud-native manner.
The drawbacks of manual workflows
One of the responsibilities of our team is to develop and maintain a set of baseline security policies and standards for using public cloud environments.
When a Two Sigma business is embarking on a new initiative in the public cloud, part of the process requires an audit of the environment by the cloud security team, to validate that the design and configuration meet the core policies and specific risk considerations for that business and project.
In addition to this initial review, a periodic audit and review to detect configuration drift over time is a prudent security practice, and the team would schedule additional periodic audits based on the sensitivity of the environment.
When one steps back and considers the confluence of multiple cloud providers, each with a myriad of service offerings with varied security characteristics, extensive audit guidelines, and increasing scale, it becomes apparent that keeping up with this level of manual effort would require an ever-expanding workforce.
Such an expansion would, however, add significant friction and increase the risk of human error and oversight. This type of work also isn’t particularly exciting or engaging; the team could add greater value in other initiatives.
For all practical purposes, approaching cloud security this way would run counter to our first goal, above–being a business enabler rather than a blocker. We were thus highly motivated to find a more effective solution.
Automating security audits
The team had previously built its own serverless auditing system in Amazon Web Services (AWS). We felt, however, after analyzing several options, that Forseti (an open-source security tooling project) was the optimal fit for Google Cloud Platform (GCP).
Forseti brought nearly all of the necessary features to successfully monitor and audit our environment from a security perspective. The fact that it is open-source meant that Two Sigma’s team could actively contribute to it to address any gaps we found, or to develop features we desired, while also benefiting from the rapid innovation Google and other power users of the system were adding to it.
We hypothesized that each these characteristics would support our three core objectives. To test this belief, the team initially deployed Forseti into a development environment and began to experiment with it.
At a high level, Forseti can take an inventory of many of your cloud resources, and then build a model from this inventory. Custom rules that represent the desired security posture are then developed and the Forseti Scanner module will take those rules, compare them to the resources in your inventory model, and determine if there are any violations. For example, it’s possible to craft a rule that determines whether any of one’s BigQuery datasets are accessible by all authenticated users in GCP (not just those in one’s specific organization or project)–this would be a significant security gap when storing sensitive information in these tables.
When Two Sigma first started using Forseti, all violations were written into Google Cloud Storage (GCS) in Comma-Separated Value (CSV) format. It is of the utmost importance to securely store the results of an audit in an immutable fashion.To this end, we decided to modify the Forseti code base to support writing violations into an immutable system. The team chose to send these violations from Forseti into Google’s Cloud Security Command Center (Cloud SCC). This service has a resource type called Findings, which allow you to record information about a security event or violation in a secure fashion.
Getting connected with the team working on Forseti at Google was a straightforward process, and pretty soon we were up and running with a development branch. Previously, Forseti had added the ability to move violations into Cloud SCC via writing to GCS. However, to make things immutable, and to give ourselves more control over the Finding structure, we decided to write code that would directly integrate Forseti and Cloud SCC, via the Cloud SCC API. Iterating on the code and getting support from the team at Google was a smooth process, and pretty soon, we had some working code merged into the development branch.
A couple of small pull requests followed, to get the code working in conjunction with automated testing performed on the Forseti project, after which we felt increasingly confident that this solution would indeed help us meet each of our main objectives.
The next important piece of the architecture was a notification mechanism that would alert internal teams to potential security gaps in our environments, and suggest steps on how to remediate them. To that end, the team wrote some very straightforward code using the Cloud SCC API mentioned above, which reads the Findings we generate from Forseti, pulls them into our internal notification and alerting system, and then updates the Findings in Cloud SCC to reflect that they have been ingested.
An enhanced security posture, with greater agility
Approaching cloud security in a scientific, systematic way–starting with clearly defined objectives, then rigorously tailoring and testing potential solutions–helped deliver the results we sought. The finished system provides a solid foundation for continuously auditing cloud resources, along with a unified alerting system to enable a quick response to any security events.
Moreover, the Forseti integration is just one of the many workflows that Two Sigma has automated in an effort to increase the overall security posture of our cloud presence, while also enabling agility and consistency for the organization. Other examples include a secure deployment pipeline for Infrastructure-as-Code, as well as a series of serverless functions that abstract away resource creation from a user, to control security and consistency.
Additional resources
Two Sigma collaborated with partners at Google at their Cloud Next ‘18 conference, and gave a talk called Security and Compliance Monitoring with Forseti. Check it out for more context on the project itself, its potential application, and our specific experience with Forseti (as well as some live demos).
Additionally, the team gave a talk at Next ‘18 on our full usage of Cloud SCC, called Cloud Security Command Center: Control of Your Vulnerabilities on GCP, which may also prove interesting for those looking to improve their cloud security capabilities.
