Test Acceleration Interview with Yves Junqueira
The Digital Anarchist Network recently interviewed YourBase co-founder Yves Junqueira to discuss software test acceleration. Host Alan Shimel asked some thought-provoking questions about test acceleration: how it works, whether AI/ML could help, and what types of companies may benefit from test acceleration.
This article contains a summary of Yves’ responses edited for readability.
What is software test acceleration?
Software test acceleration is a process of determining which tests actually need to execute based on changes in the codebase. Small changes to code should not require hours of testing. Unfortunately, most software testing is applied without consideration of the changes to the code.
Using test acceleration, product development teams can be sure that only the tests relevant to code changes are executed. Test selection is just one method of accelerating software testing. We also look at how tests are executed and we optimize that process. For example, many testing suites are single-threaded and can be architected for concurrent, multi-threaded execution.
YourBase’s software traces every piece of code to every test that’s executed. We generate a dependency graph based on that trace and we determine which tests need to run for which pieces of code. There is very little overhead cost to this process after we have completely built the dependency graph.
Why does software test acceleration exist?
When building software, it is important to know that the software is working as expected. Tests are the primary vehicle for determining whether code is meeting or exceeding functional and non-functional specifications. As codebases grow, complexity and dependencies increase, requiring more sophisticated and thorough testing.
Large codebases also require larger teams and increased coordination. It’s very important that software developers don’t step on each other’s toes. Developers can easily get tripped up by code branch merges, changes to core functions, and deployment timelines. Furthermore, to prove that code changes are working as expected, more tests get written.
Over time, the number of tasks in a codebase can grow quickly and become a very big problem for developer productivity. The test suite that takes an hour today might have only taken minutes to complete just a year ago. Developers are hired to produce and deploy functional code. When they’re waiting for tests to complete, they have to switch contexts and do something else.
The more work developers do, the slower their codebase gets. It takes time to understand code before changing it and code changes can impact testing results for a variety of reasons. This work can be quite difficult and lead to a decrease in developer happiness. If developers have to spend more avoiding problems than actually building, they lose motivation and everyone suffers.
What do developers need to accelerate their tests?
YourBase can either run locally or integrate with popular continuous integration tools. This allows multiple developers to work at the same time in the same code base. They can be making different code changes, test those changes, and not have to worry about conflicting with the work of their teammates.
Even better, YourBase keeps track of what tests are executed. Developers don’t have to repeat the same tests their colleagues have already run on the same codebase. The only tests that will execute during development are the ones relevant to the code changes being made.
Does YourBase use AI/ML for test acceleration?
YourBase does not use artificial intelligence or machine learning for test acceleration. We use simple data structures and algorithms based on the dependency graph our software generates. This is a very important point for us.
While we certainly appreciate the capabilities of AI, ML, and other statistical approaches, we would only have limited confidence that we are testing the right things. Imagine you are a financial company, like a hedge fund, where these types of tests are extremely important. If you don’t catch a certain bug, it can cause millions of dollars in losses.
We guarantee to our users that the correct tests run against any detected code changes. If we were using an AI approach, we might only have a 90% confidence that we are running the right tests in our clients’ code. These statistical approaches need algorithms that have to make decisions about the right tests for particular changes. It’s very difficult to get right.
Our customers need assurance that test acceleration is not leading to any regressions. They need full confidence in our approach. Our approach is a little intrusive at first because we watch every single function call during testing. It takes a while for the first executions, but it allows us to provide full confidence to our customers.
Who can benefit from software test acceleration?
Fortunately, YourBase can accelerate testing for small and very large codebases. We have customers that are startups just getting started and want to have the fastest possible development process. And we also have Fortune 500 customers with very large codebases that want to keep moving fast.
Customers are demanding that software companies evolve their products quickly because their needs are changing quickly. By reducing software test execution times by up to 90%, we are able to help companies get to market much faster, saving them significant time and money.
We are able to help product teams change their software at high velocity with the most productivity possible. Even though the codebases are large, they can move as quickly as if the codebase was much smaller.
We are very proud of our ability to help large companies move as quickly as small companies. I worked at Google for 10 years and my co-founder John Ewart was a principal engineer at Amazon. We know what it’s like to work in large codebases and we fully understand the impact of test acceleration on developer productivity in competitive markets.
How can I get started with test acceleration?
If you’d like to try test acceleration on your codebase, you can download our Python product at YourBase.io. We are working on our Ruby product and hope to launch that soon. Our technology has to be implemented on a per-language basis. Feel free to ask us about your codebase’s language.
We think all test suites should take 10 minutes or less to execute. Please reach out to us if your test suite takes hours or days to complete. We would love to help you lower that number and increase your developer productivity.