The business needs could be any — the merging of 2 firms and so their software systems, the need to comply with regulation, to save costs, to offer flexibility, to have a better customer experience, to improve productivity and to allow seamless data flow.

What are the different ways of integrating 2 or more software systems?

What are some of the ways of testing the integration of 2 or more software systems?

What are the some of the challenges in integration testing?

Lack of information of the system being integrated to as well as no access to the code are impediments, but a well developed test methodology based on the specifications of the system can lead to effective testing in an application context which was completely unknown when the system was developed.

#components #integrationtesting #qualityengineering

Despite numerous efforts to improve software quality over the years, the software industry still struggles to produce flawless software. It’s clear that achieving a higher level of product quality requires a greater level of testing maturity. With the growing awareness of the cost of poor testing, firms are investing more resources in software testing and finding ways to conduct testing more effectively and efficiently. One way they’re doing this is by making improvements to their testing processes.

TMMi has become a popular means of achieving this. Firms choose TMMi primarily for improving the quality of their product, reducing the risk associated with the product, making the testing process more efficient, comparing their testing practices to an international standard, and raising the profile of their testing team.

Firms that use TMMi have reported various benefits, including improved product quality (test effectiveness), increased test efficiency, shorter test execution times, and a higher Defect Detection Percentage.

Security testing can be either performed statically or dynamically. The dynamic way is mimicking how an actual bad actor may get into the system. Dynamic testing can be categorized into the below 3

Web Application Security Testing involves exploring the whole application in order to determine all URLs/resources available, then performing malicious requests against every identified resource and evaluating every response of the application in order to determine possible security issues on the targeted URL.

API Scanning Security Testing involves testing of every endpoint with a focus on authentication, input validation, or error handling. The testing includes a wide range of input data, from authentication credentials to potentially harmful payloads like SQL injection.

Behavior Driven Security Testing helps in identifying security vulnerabilities that could be targeted through multiple entry points in the system. This approach combines the above web application and api scanning security tests, to simulate attack scenarios that a hacker might use.

Integration into CI/CD does not differ much from how other tests are orchestrated to build/ run through CI/ CD. There is decision making involved in which security tests needs to run where and when in the integration, delivery and deployment phase of a pipeline.

There are for sure some challenges involved viz.

• Keeping the run-time of a pipeline to a minimum
•Dealing with increasing complexity of security tests

But overall, with the continued cyber threats, DevSecOps is a good first line of defense.

#devsecops #securitytesting #qualityengineering

Culture is a tricky thing to define — it’s like a mix of attitudes, beliefs, and behaviors that set one group of people apart from another. But if we’re trying to compare two different groups, we can talk about something called cultural distance. Basically, it’s how different their values, beliefs, and customs are from each other. And this can show up in all sorts of ways — what’s important, what’s not, what’s considered polite or rude, what’s seen as right or wrong. Culture is so complex that we often have to break it down into smaller pieces, like looking at the national culture, the culture of a particular organization, or even the culture of a single team.

The behavioral difference which is the result of the above cultural distance can be accentuated by more complex outsourced service and also by the nature of the leadership and management techniques.

Though there is no one size fits all solution to this, as long as one is aware of the distance and consciously tries to bridge it, or uses it as a factor in deciding a partner, all should be good.

The US Food and Drug Administration (FDA) defines

Verification as confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.

Validation as confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled.
Process validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications.
Design validation means establishing by objective evidence that device specifications conform with user needs and intended use(s).

In other words, validation means ensuring that we elicit from the users the right requirements which when fulfilled will meet the needs of the users. This will be ‘validated’ by actually using the product from a user perspective.

And verification means, once we set out to build the product, at each step, say design, building, testing, we ensure that the criteria/ requirements at each step are met as defined.

Failures lead to recalls and an analysis of the most frequent medical device recalls leads to one of the reasons being software design issues due to inadequate testing, software bugs, or inadequate validation of software changes. Examples include wearable bluetooth connected wrist watch de-activating monitoring, network cards failing resulting in inability to receive or transfer analytics data.

IEC 62304 is the relevant standard specifying the lifecycle requirements for medical devices and medical device software. Though much of the software testing involved in medical devices overlaps with testing of software products in nature, due to compliance requirements, more attention has to be paid to requirement traceability, managing IP. And due to hardware devices being involved, more attention has to be paid to static code analysis, unit testing, run time analysis and code coverage.

#medtech #qualityengineering

It is a way of bucketing test cases into various buckets based on the effort expended in creating and maintaining the test cases. It is mainly divided into 3 parts

• Unit Tests (~70%): mainly written by developers, are shorter and quicker to run
• Service Tests (~20%): mainly consists of API contracts tests and integration tests
• User Interface Tests (~10%): mainly consists of e2e (end-to-end) UI tests

The above follows from the shift left philosophy — meaning the earlier the defects are found, the cheaper it is to fix it and the faster we can deploy features. Unit testing is closer to the development phase and hence we do the maximum testing using unit tests. Service testing is a little further away and UI test a little more than that.

As with any framework, this is a starting point and is open to adaptation per the firm’s needs. It could be very likely that websites built using angular or react could have UI unit test cases and not e2e test cases. A team can come up with its own name for each of the layers, as long as it is consistent within the firm. Maybe name the service tests layer to be integration tests. Maybe it need not be a pyramid but more a diamond, since a firm has more integration tests (say #stripe?).

#softwareengineering #testpyramid

Transformation is inevitable mostly, so how to go about transforming a legacy quality management system. Though every enterprise is its own being, the below transformation characterizes what needs to exist at a minimum for an effective quality management system.

The framework for this transformation is defined in terms of six distinct areas: metrics standardization, process standardization, measurement, reporting, quality analytics, and culture & leadership.

Culture & Leadership: It has been proven time and again that the most important ingredient in the success of any endeavor is the people(/employees) component. And it would be impossible to keep the people motivated and driven without the active participation of the executive leadership team who can help by aligning the quality objectives to the strategic goals and following through on that.

Quality Analytics: mainly depends on 2 parameters viz. defect classification and defect prediction. Defect classification involves bucketing defects into various levels of severity and priority so that they can be fixed accordingly. Defect prediction involves using historical data (with the aid of even machine learning) to predict the most defect prone component of an application. This helps in resource planning as well as measuring test effectiveness.

Reporting: mainly serves 2 purposes, providing a singular source of quality metrics and displaying trending so as to aid decision making. The granularity of the reporting can be modified based on the audience, whether it be engineering team, leadership or external customer.

Measurement: In addition to the primary barometer of software quality, viz. defects which need to be measured, it is also of value to measure escalations from customers, time taken to complete testing activities, number of team members involved in testing, churn of issues while still in the software development cycle.

Process Standardization: This is applicable to relatively larger teams and alludes to processes like using either waterfall or agile for all program teams, using the same automation framework for similar technologies, using the same coding practices, using the same code review process, using the same defect classification process. This helps in an appreciable improvement in velocity.

Metrics Standardization: This follows naturally from the above two — viz. Process standardization and measurement. Meaning, follow standard metrics across the entire enterprise so that when they are used in measurement, they help in deriving a like for like comparison picture.

It goes without saying that in order to validate the effectiveness of the transformation in terms of the incoming defect statistics, periodic failure rates, and release cadence timeline, we need to capture the before and after profile of the team.

#transformation #enterprise #qualityengineering

A monolith is one whose modules cannot be executed independently while a microservices architecture’ modules can.

From a quality engineering perspective, both monolithic and microservice software architectures can be evaluated against certain software quality attributes:

• Availability — is defined as the degree to which an application or system is operational and accessible when required for use. Microservices are created based on the service they provide, and it is less likely that the entire application can go down since that presumes that all microservices go down together which is unlikely. Microservices based applications are thus more available than monolith ones.
• Complexity — is defined as the degree to which an application or system’s design or implementation is difficult to decipher and validate. Since monolithic applications incorporate all features, its complexity is higher as compared to a microservice that incorporates only one feature.
• Testability — is defined as a degree of effectiveness and efficiency with which test criteria can be established for an application or system and tests can be performed to determine whether those criteria have been met. There are different kinds of tests and here there is a mixed result in the sense that integration tests for microservices are more involved than for monoliths.
• Coupling — is defined as a measure of interdependence among components or modules in a software system. Since microservices mostly only encapsulate a singular feature independently within itself, they are less likely to depend on other microservices. Hence microservices are more loosely coupled in the context of an entire application or system.
• Security — is defined as the degree to which an application or system protects information and data. Here the results are more or less similar with both sharing the same vulnerabilities in terms of user privileges, cross-site scripting attacks, and command line privileges with microservices having the additional job of ensuring messages between services are appropriately secured.
• Deployability — is a measure of the time needed for the production of deployable artefacts and includes validation, code compilation, test execution, static code analysis and packaging of the application into a deployable format. Mostly, due to the fanning out of functionalities or features to various microservices, and due to the fact that at any time only a handful of features are modified, from a logistics perspective there are less moving parts in a microservices to productionize the features. Hence microservices are more deployable.

#monolith #microservices #qualityengineering