Writing multi-cloud compliance tests in Cucumber using Golang’s godog framework
Using BDD to test cloud guardrail controls on AWS, Azure and GCP
Recently we have been building out examples of Compliance as Code against the three main public cloud providers — AWS, Azure and GCP. We decided to adopt a Behaviour Driven Development (BDD) approach using Cucumber. In this approach we define our control requirements in Gherkin and implement tests of the controls in our programming language of choice — in this case Golang — to support the deployment of guardrails around the cloud environments. To do this in Golang, we use Godog which is now the official Cucumber project for Golang.
One of the interesting problems I came across is how to have a single Gherkin feature file work with different Cucumber test implementations for each of the different cloud providers, without having to duplicate identical feature files for each provider implementation.
Let me go through an example in which I implement the same test for both AWS and Azure. Here we have part of a scenario from the feature file (features/whitelisting.feature):
@non_intrusive_test
@service.object_storage
@whitelisting
@CCO:CHC2-SVD030
@csp.aws
@csp.azureFeature: Object Storage Has Network Whitelisting Measures Enforced As a Cloud Security Architect
I want to ensure that suitable security controls are applied to Object Storage
So that my organisation's data can only be accessed from whitelisted IP addresses Rule: CHC2-SVD030 - protect cloud service network access by limiting access from the appropriate source network only@detective
Scenario: Check Object Storage is Configured With Network Source Address Whitelisting Given the CSP provides a whitelisting capability for Object Storage containers
When we examine the Object Storage container in environment variable "TARGET_STORAGE_CONTAINER"
Then whitelisting is configured with the given IP address range or an endpoint
Note how we tag the features and scenarios with @. This will help us to run selective tests in Godog. From this feature file we run the Godog command line tool godog to generate test snippets for us.
You can implement step definitions for undefined steps with these snippets. Here’s the output for the above scenario:
func theCSPProvidesAWhitelistingCapabilityForObjectStorageContainers() error {
return godog.ErrPending
}func weExamineTheObjectStorageContainerInEnvironmentVariable(arg1 string) error {
return godog.ErrPending
}func whitelistingIsConfiguredWithTheGivenIPAddressRangeOrAnEndpoint() error {
return godog.ErrPending
}func FeatureContext(s *godog.Suite) {
s.Step(`^the CSP provides a whitelisting capability for Object Storage containers$`, theCSPProvidesAWhitelistingCapabilityForObjectStorageContainers) s.Step(`^we examine the Object Storage container in environment variable “([^”]*)”$`, weExamineTheObjectStorageContainerInEnvironmentVariable) s.Step(`^whitelisting is configured with the given IP address range or an endpoint$`, whitelistingIsConfiguredWithTheGivenIPAddressRangeOrAnEndpoint)
}
Normally, I would just replace return godog.ErrPending in each function with test implementations. However, I wanted to have the test run as part of the CI pipeline and have the test for AWS and Azure run in parallel by switching environment variables.
To achieve this, first I have declared an interface to the main test (whitelisting_test.go):
package mainimport (
…
)const csp = “CSP”// EncryptionInFlight is an interface. For each CSP specific implementation
type AccessWhitelisting interface {
setup()
isCspCapable() error
examineStorageContainer(containerName string) error
whitelistingIsConfigured() error
teardown()
}func FeatureContext(s *godog.Suite) {
…
s.BeforeSuite(state.setup)
s.Step(`^the CSP provides a whitelisting capability for Object Storage containers$`, state.isCspCapable)
s.Step(`^we examine the Object Storage container in environment variable “([^”]*)”$`, state.examineStorageContainer)
s.Step(`^whitelisting is configured with the given IP address range or an endpoint$`, state.whitelistingIsConfigured)
s.AfterSuite(state.teardown)
}
Note there are couple things I have done here.
- I have renamed all the functions to shorter form to be easier to work with. Compare it to the original one generated from
godog. - I have defined an interface containing all of the prescribed functions for the test, plus
setupandteardownfunctions so that I can perform some pre-test setup and post-test clean up. - Have the
FeatureContextcall the interface’s method mapping them to the scenario steps in the feature file
This is not enough though. I have to instantiate an instance of the interface in the FeatureContext.
func FeatureContext(s *godog.Suite) { var state AccessWhitelisting cspEnv := strings.ToLower(os.Getenv(csp))
switch cspEnv {
case "azure":
state = &AccessWhitelistingAzure{}
case "aws":
state = &AccessWhitelistingAWS{}
default:
log.Panicf(“Environment variable CSP is defined as \”%s\””, cspEnv)
} s.BeforeSuite(state.setup)
s.Step(`^the CSP provides a whitelisting capability for Object Storage containers$`, state.isCspCapable)
s.Step(`^we examine the Object Storage container in environment variable “([^”]*)”$`, state.examineStorageContainer)
s.Step(`^whitelisting is configured with the given IP address range or an endpoint$`, state.whitelistingIsConfigured)
s.AfterSuite(state.teardown)
}
We have a state (I call it state because it holds state information for passing between different scenario steps) that is of the type AccessWhitelisting interface. By reading from an environment variable called ${CSP} it will switch between creating a concrete instance of this interface — AccessWhitelistingAzure or AccessWhitelistingAWS.
Now we can implement concrete instances of this interface with our Cloud Provider specific test code. In Golang, interfaces are implemented implicitly. Below is the Azure example:
package main…// AccessWhitelistingAzure azure implementation of access whitelisting for Object Storage featuretype AccessWhitelistingAzure struct {
ctx context.Context
policyAssignmentMgmtGroup string
tags map[string]*string
bucketName string
storageAccount azureStorage.Account
runningErr error
}func (state *AccessWhitelistingAzure) setup() {…
_, err := resource.CreateGroupWithTags(state.ctx, azureutil.GetAzureResourceGP(), state.tags) if err != nil {
log.Fatalf(“failed to create group: %v\n”, err.Error())
} log.Printf(“Created Resource Group: %v”, azureutil.GetAzureResourceGP())
}func (state *AccessWhitelistingAzure) teardown() {
err := resource.Cleanup(state.ctx)
…
}func (state *AccessWhitelistingAzure) isCspCapable() error {
return nil
}func (state *AccessWhitelistingAzure) examineStorageContainer(containerNameEnvVar string) error {
… state.storageAccount, state.runningErr = storage.GetStorageAccountProperties(state.ctx, resourceGroup, accountName) if state.runningErr != nil {
return state.runningErr
} networkRuleSet := state.storageAccount.AccountProperties.NetworkRuleSet result := false // Default action is deny
if networkRuleSet.DefaultAction == azureStorage.DefaultActionAllow {
return fmt.Errorf(“%s has not configured with firewall network rule default action is not deny”, accountName)
} // Check if it has IP white listing
for _, ipRule := range *networkRuleSet.IPRules {
result = true
log.Printf(“IP WhiteListing: %v, %v”, *ipRule.IPAddressOrRange, ipRule.Action)
} // Check if it has private Endpoint white listing
for _, vnetRule := range *networkRuleSet.VirtualNetworkRules {
result = true
log.Printf(“VNet whitelisting: %v, %v”, *vnetRule.VirtualNetworkResourceID, vnetRule.Action)
}… if result {
log.Printf(“Whitelisting rule exist. [Step PASSED]”)
return nil
}
return fmt.Errorf(“no whitelisting has been defined for %v”, accountName)
}func (state *AccessWhitelistingAzure) whitelistingIsConfigured() error {
// Checked in previous step
return nil
}
Conclusion
With a cloud provider agnostic Gherkin feature document, we can support different implementations of the same Cucumber test in Go by defining an interface and implementing concrete instances for each target CSP.
