Introducing Appdiff, Mobile App Testing Reimagined

By jason arbon, co-founder & CEO of Appdiff

Today we are proud to introduce Appdiff, the fastest and smartest way to test your mobile apps. What do we mean by fastest and smartest?

• Fastest—Appdiff returns results in about 15 minutes, and without any code, configuration, or work from your team to get started.
• Smartest—Appdiff provides performance and UI automation analysis that only the top app teams could afford to build, and are making that power available to every app team on the planet. By smartest, we also mean that app teams should stop doing all the busy work that consumes their days.

Appdiff is a reimagining of software testing and automation.

Appdiff has built intelligent bots that can test your mobile apps. These bots are now smart enough to have booked Uber cars without my permission and deleted my Starbucks gift cards. App teams get their time back because the bots can do their busy work. The bots have been quietly trained on over 10,000 top mobile apps, and they are now ready for yours.

What’s the “Diff”?

Simply put, Appdiff puts your app through its paces. Hundreds of little bot brains run in parallel to tap, type, and swipe their way through an app, and at each step, Appdiff records the position of every element in your app and measures the timing of action and transitions. A typical bot run generates more than 30,000 individual test points.

This alone is great data, but the real magic happens when Appdiff sees the next version of your app. The bots tirelessly go through the same motions, but this time they also compare your app to the previous version. If anything in your app got slower, or a button or image was added or removed, the bots notice and tell you about it — before you ship your app so you can still roll back the change.

What does the Diff mean for you?

• Manual testers won’t miss new features that slip into the build.
• Most app functionality will be automatically tested for regression coverage. If the app still works the same as last time, you don’t need to know.
• New features are automatically discovered and explored, so you don’t need to add new regression tests manually.
• You can catch performance regressions before they slip into production.
• Your hand-crafted tests never break when the app changes, because there are no hand-crafted tests.

The combination of intelligent bots and the Diff means every app team gets a completely automated regression test suite, with no code and no setup required, just actionable test results.

How does a bot know what is good?

Every smart developer and tester asks this question: How does a bot know what is the correct functionality in my app when I barely know?

The key is in the Diff. Thanks to Agile, Lean, and Continuous engineering practices, today’s app teams are 10x faster than those of a decade ago. When you have new builds every week, perhaps every day or every hour, the most important thing to measure is change. A human cannot possibly test and measure every aspect of a new build, add new test cases for changes, or write test scripts for every feature. Knowing what changed then lets you know where to focus your time.

This focus on incremental change and quality is really just borrowed from my time working on the Chrome browser. At Google, we built continuously, practiced an organic form of agile, and operated lean in a way to let us quickly add and remove features based on data. It was a chaotic environment that looks similar to what many app teams are practicing today, so how did the team ship the world’s highest quality browser? We gathered as much data as possible, manual and automated. Forget the illusion that there is ever a perfect build, just focus on making sure new code never goes untested and no regressions go unnoticed. At Appdiff, we have automated much of that data gathering and differencing for mobile apps.

Fast moving companies don’t have the time to wait for full test suites to be manually crafted and executed. What if you could get broad test coverage in minutes, before moving on to the next build? Appdiff is the first answer to delivering automation at scale for modern app teams.

What can Appdiff tell me about my app?

Performance: Appdiff automatically analyzes transitions between pages in your app. Would you like to know the slowest part of your app’s user experience, the part that needs attention? Appdiff simply points you to the slowest part of your app from your user’s perspective — no clicking, no data tables, no charts and graphs.

Then you go further. Appdiff will show the fraction of your app that is slow or fast, and even automatically compare your app’s performance against the average of other apps in your category. If your app is twice as slow as the typical app, it’s time to work on performance.

Going back to the Diff, Appdiff also automatically compares performance measurements between the current and previous build. If anything has sped up or slowed down, you’ll know within minutes of sending your app to us.

All this data has been lovingly integrated into a single performance analysis screen. You could even show it off at your next engineering standup meeting.

UX Diffs: If your user experience changed in the new app version, Appdiff lets you know within minutes too. Imagine buttons were added, checkboxes moved, images removed, or tapping a button takes you to somewhere new. Appdiff catches these changes automatically. More importantly, if nothing changed, you can sit back, relax, and deploy to production.

Debugging with Inspector: The Inspector view gives developers and testers context to what the bot was doing when the issue was found, along with data about memory, network, and CPU utilization at every step. This way you can visually inspect an issue and know how to reproduce it.

Navigating your app: Manual testers and product managers love to explore new versions of their app, reviewing for usability, functionality, and localization. They also like to look at things from the perspective of a single screen. The Navigator allows exploration of the app for quick testing and validation. From the Navigator, you will see every action taken on a screen, the average performance of each action, and the steps leading up to the page. If an issue is found, testers can quickly add a comment that is shared with everyone on the team.

What we haven’t shown you

What we haven’t shown you is a new record-and-playback tool that breaks with each new version of your app. There is no new SDK that you need to add and instrument to your code base. There is no new test framework which requires a whole new code base and developers to leverage and maintain all that code. There isn’t a new simulator or device lab. These aren’t the results of humans manually regression-testing your app.

This is a glimpse into the future of software testing, using smart bots and lots of data analytics, so you can focus on the hard stuff.

Get started today at https://www.appdiff.com.