Now, first-tier Internet giants including Google, Facebook, and eBay are gradually implementing a new model of "no full-time testing, testing is done by developers". The original full-time business function testing team is gradually shrinking, and some are even gone. , And the original test and development team is gradually transforming into an engineering productivity (Engineering Productivity) team. The reason why these Internet giants are able to implement this new model well is because they have solved the two biggest problems of this model:
How can developers be competent in testing?
How does the engineering effectiveness team empower developers to help them efficiently complete high-quality tests?
This article will discuss these two issues. First, let's take a look at the problems and obstacles that developers will encounter when doing their own tests.
1What problems will developers encounter when doing their own tests? Problems caused by human nature
First of all, from a human point of view, developers usually belong to "creative thinking", and the code they develop is like a son, and I think the implementation is great no matter how you look at it; while testers belong to "destructive thinking", testers Its responsibility is to find as many potential defects as possible, and full-time testers usually have accumulated a large number of typical error-prone patterns in previous test practices, so testers are often more objective and comprehensive than developers Do adequate testing.
The problem of thinking inertia
From a human point of view just now, if from a technical point of view, developers will test themselves, there will be serious "thinking inertia", usually developers did not consider the branching and processing logic in the design and development process, in The same will not be considered when doing the test by yourself. For example, for a function, which has an input parameter of type String, if the developer does not consider the possibility of Null value in String when implementing the function, then the code implementation will not deal with the Null value, and the result will be Even when testing, it will not design Null worthy test data. Such a "one-stop" lack will leave a hidden danger to the quality of the code. To make matters worse, for this situation, even if the code coverage index is enabled to measure the completeness of the test, it cannot effectively expose such problems, because the code to deal with Null is not written at all, so how can code coverage be talked about?
The complexity of the tested environment and the test execution environment
When there is a full-time test, the test is done by a full-time tester, who is usually responsible for setting up the tested environment and managing the test execution environment. The tested environment is easy to understand, that is, System Under Test (SUT). The test execution environment refers to the machine used to execute test cases. For example, for GUI tests on the Web, the simplest test execution environment is the browser on your local machine. But for large Internet companies, the test execution environment is far more complicated than you think. Usually some large test execution clusters, or even internal test execution private clouds, such as the GUI test execution environment built with Selenium Grid, often such clusters will have hundreds or thousands of machines, for example, Appium+Selenium Grid There are often thousands of devices in a mobile device test cluster built. Now that there are no full-time testers, it is necessary for developers to manage, maintain and build these test infrastructures by themselves. In fact, this is not worth the gain. The workload itself has not been reduced, but a group of people have been changed to do the same thing. , And the development effort should often be spent on building new business functions, rather than on maintaining the test infrastructure.
Problems with test data preparation
Test data preparation is an indispensable key step in the testing process. When there is a full-time test, testers prepare test data. On the one hand, testers often understand the system under test better than developers at a global level. The design and generation of test data will also be more efficient. On the other hand, testers have accumulated many methods and gadgets for test data generation in the past testing process. Now these all need to be done by the developers themselves, which undoubtedly further increases the workload of the developers, and the developers often lack a systematic understanding of cross-module and cross-system test data preparation, often in order to generate a non-self business field Data and spend a lot of learning costs. For example, suppose that the developer of the "buyer module" needs to test the "commodity purchase" operation, then "goods that can be sold" need to be prepared in advance, which means the developer of the "buyer module" You need to know the details of the "seller module" and "commodity module" to generate "goods that can be sold". This type of problem will be more serious in the current mainstream microservice architecture, because in order to generate a piece of test data, it may be necessary to call many services in turn.
Test execution and CI/CD integration issues
For different business development teams, the automated testing frameworks used in each stage may be different. For example, some will use Java-based Selenium, and some will use JavaScript-based Nightwatch, etc. When there are full-time testing, a variety of different testing frameworks and CI/CD integration is done by testers from various business teams and CI/CD personnel. Now there is no full-time testing. This part of the work needs to be completed by developers themselves and CI/CD personnel. This requires Developers not only need to be very familiar with the details of the automated testing framework (in order to better integrate with CI/CD, in many cases, open source testing frameworks or self-developed testing frameworks will be used for secondary development, such as improving the retry mechanism, increasing coverage statistics, etc. Etc.), you must also understand the CI/CD pipeline design and script design, and then you need to support the running command line of the automated test framework and the parameters that need to be exposed (test case Git path, test execution environment, test report path, etc.) Script written into CI/CD. These tasks have greatly dispersed the energy of development, which is very unfavorable for improving the efficiency of development.
Attribution of failed test cases
When there is full-time testing, developers often only pay attention to modifying part of the related test cases. If there are failed test cases in the full regression test of a module or service, the testers usually follow up to analyze the specific reasons, and coordinate solutions before they can Release online. But now that the developer is responsible for all tests, he must focus on the overall test. To give a practical example, for example, the development engineer of the "user login" service fixes a defect, and then submits the code after passing the local self-test. Then, unfortunately, the CI/CD pipeline full regression test found some The use cases failed. Although these failed use cases have nothing to do with this code modification, in order to ensure that their modifications can go online smoothly (the CI/CD pipeline requires that only 100% of the full regression tests can be passed before they can go online), he must be one by one It is obviously very unreasonable and not agile to analyze the failed test cases and then find the corresponding people to coordinate and solve them.
In the final analysis, the nature of these problems will directly affect the progress and efficiency of the essential work of developers, so how should we solve or alleviate these problems to a certain extent? This is the question to be discussed next, how does the engineering effectiveness team empower developers to help developers complete high-quality tests efficiently.
2The engineering effectiveness team empowers developers to carry out high-efficiency and high-quality tests
The basic idea of ​​empowerment is to allow developers to focus more on the test itself, and liberate them from the work of auxiliary testing (such as setting up a test execution environment, CI/CD integration, etc.). These auxiliary testing work is determined by "engineering effectiveness". Service or related support tool chain to solve uniformly. This idea coincides with the design idea of ​​the very popular Service Mesh. Service Mesh also allows service developers to focus all their energy on the realization of business functions without the need to care about the basis of communication between services. Facilities, such as service registration and discovery, fuse mechanism, etc. will be implemented by Service Mesh in a way that is transparent to business applications. These "engineering effectiveness" services or related support tool chains are usually designed and developed by the engineering effectiveness team that originally transformed from testing and development. Then let's take a look at what kind of "engineering efficiency" services or related support tool chains can be provided, and in what ways can we solve or alleviate the problems caused by the development of our own tests mentioned above.
Test Execution Service
All test execution initiation at each stage of CI/CD is through Test Execution Service (TES, Test Execution Service), and TES is integrated with CI/CD in a decoupled manner through a unified Web Service interface. Whether it’s the CI/CD pipeline or the developers performing the tests, they are all initiated through TES. The only difference is that developers generally use TES’s UI interface to initiate tests, while CI/CD is initiated by calling TES’s Restful API directly in the pipeline script. test. The input parameters of the test execution service are also very simple and intuitive, usually including only the test framework name, test case set version number, test case path, test report acquisition method, synchronous/asynchronous execution switch, etc. Once TES is called to initiate a test, how to call Jenkins job, how to package and download the test jar, how to find a suitable test execution environment, how to initiate a test, and how to collect test reports are completely transparent to users. It is conceivable that now, when developers are integrating with CI/CD and executing tests, they no longer need to care about the command line to execute the test, the Jenkins job that initiates the test, the specific execution environment of the configuration, the test, the acquisition of test reports, etc. information. This will greatly improve the efficiency and convenience for developers to perform their own tests.
Test Data Service
As mentioned earlier, cross-module and cross-system test data preparation is a challenge for the development of self-testing, especially now that a large number of microservices architecture is adopted, this problem will become more prominent. Test Data Service (TDS, Test Data Service) will provide a consistent test data preparation entry for all types of tests in the form of a Web Service interface. Whether the development is to do API testing, GUI testing, or performance testing, you can prepare test data of various combinations and magnitudes by calling the TDS Web Service or UI. TDS itself is also a development platform, any developer can contribute new data type support through scaffolding code, and the TDS platform itself has metadata management capabilities with its own Core Service and built-in database, which can provide information such as the quantity and quality of test data Management. The following figure shows a schematic diagram of a typical TDS architecture design for reference.
Test execution environment service (Test Bed Service)
As mentioned earlier, the test execution environment is very large and complex for large enterprises. In order to facilitate developers to use the test execution environment, or in order to make the test execution environment transparent to developers, it is necessary to introduce the test execution environment service (TBS , Test Bed Service). The main responsibility of TBS is to manage, create, expand/shrink test execution clusters. A common test execution environment architecture is shown in the figure below. TBS will dynamically decide the number and type of nodes in the test execution cluster based on the queue of test cases waiting to be executed. Docker and Kubernetes are usually used to implement TBS Gird management.
Build Engineering Productivity Tools Store (Engineering Productivity Tools Store)
Similar to the App Store concept, various test gadgets and efficiency-improving tool sets can be unified in the Engineering Productivity Tools Store for centralized version management. For example, at the beginning of the article, we mentioned that there is a blind zone in thinking when developing our own tests. For parameters like String, Null worthy use cases may be omitted. We can develop a small tool to automatically generate boundaries based on the input parameter types of the tested function. Test cases, such as String type parameters will definitely generate Null, SQL injection attack strings, non-English characters, long strings, etc., so that blind spots in development can be systematically avoided. There are many other tools like this, and I will share them with you in the future.
3 Test as a Service (TaaS, Test as a Service) global architecture
In addition to the above, there are actually other things such as Test Report Service (TRS, Test Report Service), Global Test Configuration Service (GRS, Global Registry Service), and Mock Service (Unified Mock Service) for decoupling API testing. Expand one by one. It needs to be emphasized that many of the services mentioned here have already been implemented in some enterprises and achieved good results. Finally, end this article with the overall architecture diagram of Test as a Service.
Closed Pod Electric Vape 2.6ml Pod
Closed Pod Electric Vape 2.6Ml Pod,Electric Vape 2.6Ml Pod,Max Puffs E-Cigarette Box,Cigarette Bar
KENNEDE ELECTRONICS MFG CO.,LTD. , https://www.axavape.com