A reduction of 1-2 working days per month. Accelerate development cycle while maintaining quality with test automation.

With the evolution of technologies such as electric vehicles, self-driving cars, and the rise of sharing economy, the demand for vehicles is diversifying.

Meanwhile, Carview Inc. is developing a platform that connects people and services related to cars. It operates Japan’s largest car information site, carview!, which provides the latest information on new and used cars, and the social networking website, Minna no Car Life (Minkara), which attracts diehard car fans.

In 2015, they collaborated with Yahoo! JAPAN to further expand the number of users. Carview also develops new services centered around cars. We interviewed Mr. Michisuke Kitamura, who oversees the development department, and Mr. Tomokazu Manabe, a QA engineer, about quality assurance and operations after introducing Autify.

(Mr. Michisuke Kitamura, General Manager, Development Division, Carview Inc. (left), Mr. Tomokazu Manabe, engineer (right))

– What are your primary business operations at Carview?

Michisuke:We provide online services related to cars. Our main offering is the social networking website Minna no Car Life (Minkara), which diehard car fans use to connect. We also offer Carview!, a comprehensive car information website, which presents information on brand-new cars, motor shows, used cars, and more.

We also provide Kaitori Carview, where users can get a quote from multiple car buyers; Carview Clear Shaken, an automobile inspection brokerage service; pitte, a marketplace for car accessories and skill sharing. We are a company that develops a wide range of media and commerce systems centered around cars.

We’ve also established a subsidiary, KENKEY Inc., and started providing KENKEY, a service that deals with construction machinery. The QA (Quality Assurance) team manages that too.

– It sounds like there is a wide range of responsibilities. What challenges did the QA team face before implementing Autify?

Tomokazu: The QA team consists of three people, including the manager, and there are only two members who directly deal with tests. Given how few of us there are, efficiency was crucial. Before Autify, we used Selenium to write programs and run tests.

Development and maintenance were very time-consuming. Maintenance would take ages when a program or screen changed because the automated test needed to be edited as well. We only launched KENKEY last year, but we had to create about 20 Test Scenarios, from extensive scale tests to minor ones. We had to construct them to verify the entire operation and the whole process took about a year.

– You did it all by yourself?

Tomokazu: Yes. I’m the only QA member who has programming knowledge. I prioritized so that I could implement automated tests in between manual testing.

Even within the QA testing domain, people seem to think that things that must be done manually should be done manually. It seems never-ending if we spent a whole year trying to automate tests for one service. It needs to be quicker.

– Does your company have a fast release cycle?

Tomokazu: One of our services releases every Wednesday. I’ve been running automated tests every week, even when I was writing code manually.

Michisuke: KENKEY is once a week, other services may have a slightly shorter cycle, or it may be irregular. From a business point of view, we want to release frequently. At the same time, we have to ensure quality. We respond to customer requests and inquiries while simultaneously modifying UI and adding new features. We are part of the Yahoo group, so we try to cooperate with Yahoo! JAPAN to have a short PDCA cycle, such as running A/B tests.

– Can you tell us about any challenges you encountered when implementing Autify?

Tomokazu: I worked on replacing the automated test code written in Selenium with Autify. Test Scenarios were already completed, so I thought it would be a good way to get acquainted with Autify.

At the time, Autify didn’t have as many functions as it does now, and I implemented it with the support team’s help. Autify has released a lot of new features since then, so most of the issues I encountered back then wouldn’t happen now. I’m very impressed and grateful for Autify listening to feedback and improving their services quickly.

– That means a lot to us. Thank you for taking a chance on our service, even when Autify was still in its infancy.

Tomokazu: Regarding challenges that we faced; it wasn’t easy to implement tests because our services have a step of uploading a CSV file. We’ve had to write code with Javascript just for that part, but I think it’s something we programmers love! I like that Autify has the potential to automate testing in any system.

– We’ve recently added a function that expands the functionality by using JavaScript steps. You can add arguments in JavaScript steps or pass values between JavaScript steps. We’ve managed to implement a two-step verification using that feature! I understand what you’re saying completely as a former engineer.

Tomokazu: An engineer’s work isn’t just about working and recording. It really gets me excited when there’s a problem to resolve. I requested a function where a single button allows you to use an open URL. I think that would be useful for many of us.

– Autify’s policy is to provide functions that are frequently used in JavaScript so that anyone can use them easily. Thank you for submitting your request.

What kind of creative ideas did you have when you implemented and operate Autify?

Tomokazu: I try not to overcomplicate steps. If the Test Scenario is too long, it’s difficult to identify what the error is. I try to keep it as short as possible. I focus on checking if things that worked before the release are still working. I check any additional items manually.

We verify assertion steps and error items at the time of input, so we usually don’t have recurring issues unless we create a new screen. We try to make Test Scenarios as slim as possible by removing those unnecessary steps. Testing everything would take forever, so I look at the system as a whole and pinpoint what really needs testing.

– You don’t blindly test everything just because it’s automatic. Instead, you focus on the essentials and create a simple design.

Tomokazu: Exactly. Even though it’s faster with Chrome, it can take over 10 minutes to run and complete a single Test Scenario. Checking inputs takes even longer. If you want a rapid release cycle, I think you would have to slim down Test Scenarios to some extent.

– Were there any other ideas?

Tomokazu: Since Autify published API, I began starting up Autify from Jenkins using that. I use the API to get the result and programmed it to handle errors and successes on Jenkins.

We implemented the function to upload CSV files before the JavaScript function was released, so that’s the only part we have Java code. I’ve created a pipeline where after operation A, I run Java code on Jenkins to handle aspects that Autify can’t. When that’s finished, I go back to Autify, and so on. I was very proud of myself when we solved that issue!

– What kind of effect did you see after implementing Autify?

Tomokazu: Around April 2020, which was a while after introducing Autify, Kitamura announced to the entire Development Headquarters that the QA team is using Autify to check before release. Now, the whole company knows about Autify.

Until then, we were only using Autify to test_ KENKEY and _Minkara, but we have expanded automated testing to other services. Now, any developer and planner can run Autify before release.

Currently, the QA team creates the Test Scenario, but we are incorporating Autify jobs within the CI/CD cycle of the release to operate Autify before releasing.

– You can verify with a press of a button and run a test plan. Once that’s done, it’s ready for release. You wouldn’t even have to press a button in the future, and it will be fully automated.

Michisuke: We occasionally perform nighttime maintenance on infrastructure platforms. Manabe has made Scenarios for those too. We would like for the platform teams to test with Autify soon.

– How many work hours has Autify reduced?

Tomokazu: It’s been one year since implementing Autify, and overall, we’ve been able to reduce a total of 1-2 working days per month. We used to spend a whole week writing test code, but now it takes one day if I put my head down, so it’s been a significant reduction in working hours. Had we continued our slow pace, I don’t think all systems within the company would have been fully automated.

– I often hear that horizontal expansion to other departments isn’t easy. How do you communicate within your company?

Michisuke: There are occasions when we talk about the Development Headquarter’s policies at company-wide meetings, so we communicate then. We introduced Autify and explained the benefits of how automated testing saves time, and that UI changes won’t require additional maintenance. The development team was busy writing programs, so they weren’t too keen at first, but after they actually used it, they understood how beneficial it would be.

Autify checks basic parts important to our business. This alone has been useful for preventing critical failures.

Tomokazu: For the type of service that we provide, I think automatically checking the communication every morning is just right. We can see whether essential functions such as “apply for a used car” are working correctly.

Michisuke: The QA team used to call it “morning check.” We used to check manually, but now we use Autify and automate it via Jenkins. Autify makes it so easy to creating Test Scenario. Change in UI used to be a problem, but Autify can handle that too, which reduces maintenance costs too.

It’s such a useful tool. In fact, the screen had an overhaul last week when we launched a new project. I thought I would have to re-record, but I tried running a test, and it completed with zero issues. It made me appreciate the effectiveness of Autify’s AI.

– That’s great! It’s something that we can’t experience on our end, so I’m glad you feel that way.

What are your plans for quality assurance?

Michisuke: CI/CD will continue to be the basis, and we will increase frequency. Another thing is that I’d like to be aware of fundamental issues as quickly as possible. I’m sure there are other ways to utilize Autify, so I’d like to discover them as well.

We haven’t made use of the feature to take and download screenshots. I think it would be handy for creating manuals, for example, so I’d like to utilize that.

– Do you have any advice for those planning on working with test automation?

Tomokazu: As I mentioned earlier, I recommend designing Test Scenarios focusing on checking whether the basics work instead of trying to run complicated tests. It’s easier to identify issues when something goes wrong, and you can always add essential steps later. Autify is a fun tool for people like me, who are engineers at heart. At the same time, non-engineers who usually don’t write code can use it for testing. I recommend that you give it a try.

Michisuke: The world is changing rapidly, and there’s a demand for faster releases. Maintaining quality and more rapid release seems contradictory. I think test automation is an essential part of the solution for this. Autify is so effective because creating Test Scenarios is straightforward and maintenance is equally effortless.

It may be a challenge for the whole company to implement it, but it’s easier if you start small and gradually get other departments to try it out.

– Thank you very much. Do you have anything you’d like to add?

Michisuke: If you love cars and are interested in development, please take a look at our careers page. Thank you.

(Interviewer: Autify, Inc. Ryo Chikazawa)