A/B testing and gated release, and B testing grayscale

A/B testing and gated release, and B testing grayscale
1. The theoretical products of A/B testing and phased release are multidimensional. The purpose of the testing is to achieve the final delivery of the system, including design experience, interactive experience, system quality, and operation support, A set of systems that are adequate in all aspects, rather than those defined in the document, must constantly evolve. The question of testing goes through the process from product design to code to the final operation, and finally promotes Product improvement and repeats. It is in line with the essence of agile Internet thinking. 2. A/B Testing some terms related to phased release: Bucket Testing: there is no clear definition of this, however, it is usually a comparative test outside China to test whether the swimming pool has water leakage. Place a bucket of water in the swimming pool to indicate the internal and external water levels. After a period of time, if the external water level drops significantly (more than XXX inches), it indicates that the pool is leaking. This is not directly related to software testing, but it is a comparative test between two solutions to identify defects. 1.2 Multivariate Testing: This is a term used in marketing. It is usually used to compare the effectiveness of marketing programs in a complex environment with multiple variables. 1.3 A/B Testing (A/B Testing): defined by Wikipedia, "Web design (usually user experience) it is an experimental method used to distinguish between the two types of web design to maximize the benefits of the target (such as click rate ". It is mainly used to compare the advantages and disadvantages of the two designs. Bucket Test and Multivariate Testing are variants of A/B Testing, because they may involve comparisons in multiple scenarios. A/B testing is also used for comparison of marketing channels, which is consistent with the definition, because Web pages are A marketing channel. 1.4 gray-scale delivery: "gray-scale release refers to a release method that can smoothly transition between black and white. AB test is A gray release method that allows A user to continue using A, and some users start using B. If the user has no objection to B, the scope is gradually expanded, migrate all users to B. Gray release ensures the stability of the entire system. problems can be found and adjusted during initial gray release to ensure the impact ."
3. Differences between A/B testing and phased release and Traditional testing: 2.1 There can be multiple ready-made products to actually test (bucket testing) 2.2 A/B testing is A method that supports multi-variable testing. 2.3 A/B testing is A set of systems. It is an implementation method of phased release, testing and O & M have been integrated into a process.
4. Comparison of different algorithms between A/B testing and phased release using 3.1 Recommendation Systems, comparison of different variables 3.2 comparison of different solutions in the design scheme 3.3 design adjustment, solution adjustment 3.4 fault control

If your system needs to optimize some areas that you cannot predict and control. Test A/B to control future risks.

Data is an important basis for optimizing the system. If you want to optimize the system, you need to accumulate data.