How Twitter does Automated Testing

It was 8:00 AM and I entered the Zoom call. Kalisha was at home, greeting the morning with a coffee. Smiling like it wasn’t a Tuesday morning.

I recently embarked on a journey to find out how the top players from the Valley are delivering you a bug-free experience.

The fact was, Kalisha isn’t just your average Software Engineer. Over the past few years she had been handling testing for the massive online enterprise which we know as Twitter.

She isn’t the only one with that burden, since Twitter has an enormous number of teams.

I only had 15 minutes to pick her brain.

Here is the transcript.

1. What does your role involve on a daily basis?

My main responsibility is to make sure the changes do not generate any cross-browser issues for our components.

Each person that goes to Twitter should have a smooth experience, regardless of their browser choice.

2. Doesn’t that make you a Software Tester?

Not sure. Maybe. I honestly can’t put a label on any techie working in the Valley nowadays, all of us seem to be doing everything.

Writing code. Testing the code. DevOps. Product Meetings. You name it.

3. How do you know when the testing is done?

It depends. There are 2 types of situations.

One where you need Exhaustive Testing, and one where you don’t.

Exhaustive Testing should only be applied to mission-critical software, where you risk losing lives or other types of serious damage.

In other situations, it doesn’t make sense, because the cost would be too high.

4. What’s something that most of us seem to get wrong?

I see some devs out there focusing on Accessibility, but they don’t seem to grasp the concept correctly.

The idea of Accessibility is to make your website accessible to as many people as possible.

But I’m seeing a narrow focus on user disabilities.

Don’t talk about Accessibility if you‘re not testing on Firefox and Safari.

You might have 0.01% of users that require screen readers, but you probably have 24% of users who aren’t using Chrome.

FYI, Safari is the 2nd most used desktop browser around the world.

You better be testing on Safari.

5. What tools are you using? Would they work for us as well?

We are using a long list of tools and services for testing.

A tool is useful only if it helps you save time and resources.

Here are two of my favorites:

Postman

Endtest

I love the fact that neither of them require any coding skills.

6. Why is UI testing moving to JavaScript?

It’s not. Who told you that?

There are mainly two ways of doing functional UI testing.

Through the JavaScript layer or through the WebDriver layer.

You should always aim to do the functional testing through the WebDriver, because that mimics a real user.

Testing through JavaScript has the following disadvantages:

1. It can’t handle multiple browser tabs.

How are you going to test that email validation from the Sign Up flow?

You actually need to click on the link from the email to see that it works.

2. It can barely handle iframes.

How are you going to test that Stripe integration?

3. And worst of all, it doesn’t mimic a real user.

Ever tried to do a Click with JavaScript?

That Click will work even if the element is hidden or covered by another element.

Your tests will be green, but your users will be stuck.

Use JavaScript for unit tests, but not for functional UI tests.

Remember, a browser is more than just a JavaScript interpreter.

7. Testing tools, should we build or should we buy?

Again. It depends.

We have an enormous R&D staff and even we usually avoid building our own testing tools.

One way or another, you’ll still end up paying for it.

If you buy a tool, it has a price tag.

If you build your own, the resources and time invested by you and your colleagues will translate into significant spendings for your company.

Unless you’re working for free.

9. What’s the process behind that? How do I make that decision?

You start by making a list of requirements.

Is there at least one tool on the market that meets those requirements?

Good. Sign up for a free trial. Do a POC. See if it works for you.

If your use case is so unique, you might have to build it.

In both cases, you should calculate the Return On Investment.

How much time and resources you pour into that initiative and what’s the return you’re getting.

If I need a tool, I would like to start using it today. I don’t want to wait 7 months to try a tool that is in a constant state of “almost ready”.

Waiting for 7 months means that I’m not getting any returns in that time frame.

10. But why am I seeing people building their own testing tools?

I don’t think you’ll find anyone in Silicon Valley doing that.

That practice is a bit outdated.

Maybe testers are hobbyists, they like building things.

Maybe they have unique situations that involve IoT or smart devices.

But building that classic internal mammoth testing framework by stitching together different Selenium libraries and unoptimized code from StackOverflow isn’t going to get you far.

11. How do I know if my automated tests are good enough?

The idea of automation is that it helps you save time and resources.

Your tests should be running in the background, on the cloud.

Multiple test executions, in parallel. A full regression in less than 15 minutes.

They should be tightly integrated with your CI/CD system, but not dependent on it.

If you’re still triggering them manually or reviewing the results manually, you’re not there yet.

12. What’s one advice that I should follow?

Start testing your website on Safari.

It surpassed Firefox and it’s not going to stop there.

And it’s not going to use Chromium anytime soon.