Game to do A/b test error

Summary: Some of the more recent interesting insights about how Zynga operates are on the social news site Reddit, one of the 520 employees who were sacked last week. I would like to say a few two posts in particular. Post 1: In terms of gaming, I think they're off

Some interesting insights about how Zynga operates recently appeared on the social news site Reddit, one of the 520 employees sacked last week. I would like to say a few two posts in particular.

Post 1: On the game side, I think their overall concept of pulling all the players ' behavioral data is great, but their reliance on it is not that amazing. It makes development very analytical and intuitive, and it's easy to know where a game is interesting. Although the difficulty is how to draw data about their behavior

Post 2: At some point, the company seems to be switching from really trying to innovate to trying "the way it worked before". I'm not sure how many games have really done a A/b test in the past year to discover new fun and popular points. The company's expertise in data collection, data analysis, and game psychology can help us improve our gaming experience.

Zygna seems to suffer from the hidden costs of a/b test.

The dominant cost of A/b test is obvious: A/b test takes time to design, implement, and evaluate, which represents an opportunity cost, indicating the benefits of other work during this time period (note: Opportunity costs are also referred to as optional costs, alternative costs.) Opportunity cost is the opportunity cost for a commercial company to produce a commodity with a certain amount of time or resources, and the chance to lose the opportunity to produce other best alternatives. Source, Baidu Encyclopedia). The dominant cost of A/b test is often cited as a reason not to conduct a full-scale test, but I believe these costs are generally overstated.

If it takes about 30 minutes to design and implement A/b test, the organization does not have sufficient analytical infrastructure resources to perform A/B testing through some internal tools. It is not a problem in itself: a product like Dataeye is designed to ease the burden on developers by providing "analytics services."

A real data-driven organization is in the state of a permanent A/b test and optimization. The opportunity cost of a A/b test in such an organization is negligible.

However, the hidden costs of A/b test have a greater impact. A/b test stealth cost arises because a product team is overly dependent on iterative, incremental improvements. This over-reliance binds the product to the infrastructure and the existing flaws in the product.

Incremental improvements are not entirely unimportant: usually in a process or percentage growth, for example, the game's login page conversion, total revenue, session length will get some growth. This chart is a process that goes through 10% of the cyclical growth rate:

But this continuous testing optimization has made it impossible for a team to focus on the real problem of their products. This is not an opportunity cost (i.e. "should we choose a further A/b test or fix the product's structural problems?"), but rather than identify or consider the weaknesses of the core product, because the improvement is considered a panacea. This chart is a process that goes through 10% cycle growth: 20%.

The hidden costs of A/b test are not reflected in the misuse of resources, they reflect the idea that scientific data can replace product development. A/b test failed product is like carrying a gunshot to the gym: it's not done well, and in the end it's futile.

The manufacturers of data problems are actually abandoning the real macro-level issues, giving way to the improvement of the micro process, in fact, did not solve the fundamental problems of products, but to some extent, accelerate the spread of the problem speed.

A/b test is a tool that rather than a product development strategy, it should be used to bring greater advantages to products that have already been successful, rather than letting them think their intuition is wrong with the product (GRG Game Research Group Note: He is only suitable for some micro-process improvements, Rather than strategy-level development strategies cannot solve the product's essential problems.