Notes on the startup of the oldest programmers: full-text search, data mining, and recommendation engine applications 42

Customer-oriented
Originally, Wu Yan and Zhang shaozhi discussed how to improve system concurrency in a fully informal meeting. It is estimated that Zhang shaozhi has not made any special preparations, but as Wang Wenbin, he presided over the first technical meeting, this has special significance. Wu Yan hoped to express to Wang Wenbin and Zhang shaozhi at this meeting that although he was not the technical director, he still had an influence that could not be ignored in terms of technology. Therefore, Wu Yan needed to do anything at this meeting.
First of all, this meeting will certainly discuss database bottlenecks. Previously, all operations were performed based on databases due to human resources. This will not cause any problems when the number of users is small, however, when the user concurrency is large, the database performance problems will become increasingly prominent. To solve this problem, we can obviously get some program relief through the MySql cluster, but it is not a fundamental solution. Therefore, we need to consider using a system architecture that supports large-capacity concurrency.
According to Wu Yan's speculation, Zhang shaozhi is likely to propose a NoSql solution, which can not only solve the database bottleneck problem, but also conduct large-scale restructuring of the system. All developers, wu Yan himself and others both want to build a new system of their own, instead of maintaining the old system of others. system maintenance usually means an endless nightmare. At this point, zhang shaozhi may be no exception. At this point, Wang Wenbin may also want to reduce his dependence on the old system, which is more conducive to his future work. Therefore, from this point alone, Wang Wenbin and Zhang shaozhi share the same interests. But this means that a large part of the code needs to be rewritten, and it also means a large amount of capital occupation. From the company's perspective, at least at this stage, it is not necessary.
Wu Yan compared NoSql with NewSql in the previous stage and thought that NewSql may be a better solution for the current project. Through the configuration of the background database software, the implementation meets the high-capacity concurrency requirements without having to make any modifications to the existing Code. This solution seems better than NoSql. It's just that programmers usually like to solve problems through software programming, rather than through system software and hardware configuration. This is why Wu Yan hesitated before NoSql and NewSql in the previous section, now, after jumping out of the programmer's view, it is logical to make the choice of NewSql.
After making the decision, Wu Yan stayed up late to familiarize himself with the concepts of NewSql and NoSql again, and made sufficient preparations for tomorrow's technical seminar.
After reading all the materials, it was more than a minute in the morning, and there was no one in the company. Just now, Wu had packed up his items and moved them to a vacant space in the hall, freeing up the office for Wang Wenbin.
Just now, some independent offices have not been hot yet, and they have no more. It seems that they may not have the life of sitting in an independent office! Wu Yan couldn't help but feel deeply in his heart.
After preparing all this, Wu Yan dragged his tired body back to rest.
Wu Yan arrived at the company around eight o'clock and there was no one in the company. Wu Yan sat down in his new position in the hall, turned on his computer, and began to read the ebook "Product Manager Bible". As a product manager, he had to do a good job.
Wu Yan is reading a book. At this time, the company's front-end arrived first. The front-end was a girl younger than 20 years old, called Zhang xinyan. She just graduated from college this year. Seeing Wu Yan sitting in the new seat in the hall, he said with amazement: "Mr. Wu, how have you sat down here ?"
"Oh, this is my new position. Our company has a new CEO who will sit in my original office. By the way, today our new CEO is here, Mr. Wang, who came with Mr. Zhao two days ago. You still have an impression !" Said Wu.
Zhang xinyan nodded thoughtfully and said, "I'm a little impressed ."
"By the way, when he comes over today, you will take him directly to his office !" Wu Yan said.
"Okay !" Zhang xinyan replied that the expression on his face was somewhat complicated. He could see that he was surprised by the sudden personnel changes in the company, and he felt some sympathy for Wu Yan's situation or other feelings, anyway, Zhang xinyan's expression also gave Wu Yan a feeling of being in the back. Fortunately, Zhang xinyan turned around in the hall and went to the front desk.
Other employees also arrived one after another. They saw Wu Yan sitting in the hall and had some strange feelings in their eyes. Wu Yan had to explain it to them over and over again. This was his new position, today, the company wants a new CEO to sit in their original office. It can be seen that each of them has a strange expression. In their eyes, Wu Yan feels uncomfortable, it was a torment, but at last all the employees arrived, and this embarrassing experience could finally come to an end.
Wang Wenbin arrived at the company in less than, and Zhang xinyan at the front-end took him directly to Wu Yan's original office. Upon passing through the hall, Wang Wenbin saw Wu Yan at a glance and quickly walked over to shake hands with Wu Yan and said, "Hello, Mr. Wu! Sorry, it takes up your office !"
Wu Yan also responded: "It doesn't matter. You are welcome! You should tidy up in the office first. We will have a meeting later !"
"Okay, okay !" Wang Wenbin agreed while walking into his office with Zhang xinyan.
At around ten o'clock, the meeting on improving the high-capacity concurrency of the system started on schedule. Only Wu Yan, Wang Wenbin, and Zhang shaozhi attended the meeting. Wu Yan presided over the meeting. Therefore, Wu Yan first made an opening speech: "Let me introduce you first. This is Mr. Wang, the new CEO of our company. In the future, Mr. Wang will be mainly responsible for R & D. Of course, to achieve a smooth transition, emails from all technical departments can be copied to me, and I will participate in meetings of all technical departments ."
Wu Yan's remarks surprised Zhang shaozhi. He suddenly changed his boss and didn't have any preparation at all. It's no wonder that Zhang shaozhi will have a sudden feeling.
Wu Yan then introduced Zhang shaozhi to Wang Wenbin: "This is Zhang shaozhi, department manager of the R & D department. He used to be responsible for system development in shangcong network. He is a master's degree in computer science from qingbei University and is very competent."
Wang Wenbin said hello to Zhang shaozhi: "Hello! Please support more in the future !"
Zhang shaozhi also said, "Yes, no !"
Then Wu Yan entered the topic: "The main topic of our meeting today is to discuss the high-capacity concurrency of the system. This problem is no problem when the number of users is small, however, once the number of users comes up, the database bottleneck will become obvious in our current architecture. I have talked with Shao Zhi about this problem before. I would like to introduce Shao Zhi's solution to this problem ."
Zhang shaozhi cleared the noise, and then Wu Yan said: "I think the main problem in our system is that all operations, including logon operations, are directly performed on the MySql database, due to database performance problems, our current system performance is not good in the case of high-capacity concurrency!"
Wang Wenbin nodded his head, wrote something in the book, and asked, "How do you think this problem should be solved ?"
Zhang shaozhi continued: "There are two solutions to this problem: short-term solutions and long-term solutions. In the long term, we should migrate to the Hadoop architecture. In the short term, we need to convert MySql to NoSql. Because NoSql is developed specifically for MySql performance issues and solves the storage and management of unstructured or semi-structured data, many SNS websites currently use this technology in large quantities, we are faced with similar situations here, so I think we should adopt the NoSql solution."
After listening to Zhang shaozhi, Wang Wenbin said with some doubts: "I also have some knowledge about NoSql. However, it seems that NoSql is very different from common databases. I don't know if it is like this ."
"Yes !" Zhang shaozhi affirmed: "a common database is built on a relational model and is suitable for processing structured data, but inefficient at processing unstructured data. At the same time, their programming models are different ."
"Is it difficult for us to use NoSql to switch from common databases to NoSql ?" Wang Wenbin asked.
"It should not be difficult. Another original design of NoSql is to change the complexity of relational databases. It is much easier for programmers to learn NoSql than to learn relational databases !" Zhang shaozhi spoke in a certain tone.
"Well !" Wang Wenbin nodded and said, "If the conversion is not difficult, it can also improve the system's high-capacity concurrency capability. I think it can be converted to NoSql !"
After listening to Wang Wenbin's words, Zhang shaozhi's face showed a proud look.
At this time, Wu Yanyan said at the right time: "Shao Zhigang said that it makes sense. Indeed, many companies are currently using NoSql technology, which can indeed improve the high-capacity concurrency performance of the system. However, based on the current status of our project, we have already adopted MySql in large quantities. If all of them are switched to NoSql, although the technology is not difficult, the workload will be very huge, in the current condition of limited human resources, this may not be worth the candle!"
After listening to Wu Yan, Wang Wenbin asked Zhang shaozhi: "Do you think the workload problem Mr. Wu mentioned just now is a big problem ?"
"The workload will indeed be relatively large," Zhang shaozhi said with some hesitation: "But this can solve the high-capacity concurrency problem. This is a solution with a long pain rather than a short pain ."
Wang Wenbin turned his head to Wu Yan. Wu Yan continued: "In fact, NoSql is not the only solution to the problem of insufficient performance of relational databases, the newly emerging NewSql is also a good solution."
Wang Wenbin said at this moment: "NewSql? There are too many new terms in your technical field, and SQL has come out of NoSql before you understand it. Now there is NewSql. What are these things? Is there a big difference ?"
Wu Yan went on to say: "Shao Zhi just mentioned NoSql. In fact, NoSql is a general term for a type of products. NoSql includes three different system architectures, including documents such as Apache CouchDb and MongoDb, there are also Graph-based storage methods such as OrientDb and Pregel, and the largest number of Key-Value structures such as Memcached and Redis, among which MongoDb is the most popular, because it is similar to a relational database, Redis and Memcached are very common."
Wu Yan combed NoSql products last night. Now he is selling NoSql products. Since these technical problems can be explained in a very systematic manner, it can obviously increase the persuasiveness of his views.
Wu paused. Apparently, Wang Wenbin and Zhang shaozhi had no objection to the technical analysis he had just analyzed, so he began to talk about it: "NewSql is a technology that has only recently emerged. They believe that the performance problem of relational databases is not caused by relational databases themselves, but is caused by the implementation of various relational databases using technologies that have been around for more than two decades, therefore, we will focus on using the latest technologies to recreate relational databases. There are also three representative technologies. The first type is to develop a new storage engine for relational data such as MySql, so that MySql can support high concurrency, distributed storage, and performance issues, such as Xeround and Akiban. The second type is MySql Sharding technology, that is, adding another entity before MySql to achieve concurrency scaling, for example, ScaleBase, the third type is the brand new database design, which is divided into two types, one is MySql compatible, and the other is based on the brand new protocol, such as VoltDb and NimbusDb."
What Wu Yan said clearly has already put Wang Wenbin in the information overload state. Even Zhang shaozhi explained Wu's topic to him, which is somewhat unfeasible.
Wu Yan thought that the results had been achieved and he could come up with his own conclusions. Then he said, "according to the features of our project, I suggest using this architecture. You can check it out, is it feasible."
After listening to Wu Yan's long speech, Wang Wenbin and Zhang shaozhi are full of expectations for Wu's conclusions.
Wu Yan said: "I suggest that you use Key-Value-based NoSql Technology for frequently accessing databases, such as user name and password verification during logon. Memcached can be used for specific products, instead of Redis, Memcached is selected because Memcached service will be integrated in Versions later than MySql 5.6. We will definitely upgrade MySql to Versions later than MySql 5.6 in the future, using Memcached can reduce the nodes required by the system and reduce the difficulty of system management. I don't think there are a lot of frequently accessed content, so the workload is not heavy ."
Wu Yan analyzed: "For MySql databases, we can consider using extended storage engine products to automatically solve concurrency performance problems, at the same time, our application can not be modified. The only thing we need to do is data migration."
After talking about all the solutions, Wu Yan concluded: "This is my basic idea. Do you have any comments ?"
Wang Wenbin took the pen and seemed to want to remember something, but it seemed that there was too much content and there was no way to write it. He didn't say anything at the moment.
After hearing this, Zhang shaozhi said, "I think Wu's solution is quite good. This can indeed improve performance and reduce the workload. I originally wanted to use MongoDb, but it does have a lot of work ."
Wang Wenbin nodded and did not say anything.
Wu said: "Shao Zhi still has some work on improving the system availability. At present, it has not been completed. In the next stage, after Zhao Qigang's product design plan is released, new feature development is also required. In addition, the new merchants and coupon features introduced by Mr. Wang have a huge workload in these aspects. Currently, they only have three people, and the workload is not small, therefore, it is best to use the most economical method to solve the large-capacity concurrency problem, because we have not reached the stage of the large-capacity concurrency bottleneck."
Wu Yan's words seem to have inspired Wang Wenbin. Wang Wenbin said: "Yes, I agree with Mr. Wu's opinion just now. We should focus on new function development. As for performance improvement, it is best to adopt an economical approach."
Wu Yan's goal was initially achieved, and he achieved an influence on technical issues in an initial position. Wu Yan's mood was relaxed.
Wang Wenbin went on to say: "Shao Zhi, you will write a meeting minutes later, send them to me and Wu Zong, and then you will submit the implementation plan. Thank you !"
Wang Wenbin's first meeting ended like this. Wu Yan was very satisfied with the meeting results, but he was somewhat depressed.
On the way back to the office, Wang Wenbin said to Zhang shaozhi, "Are you going to have something to do later? If you are not busy, come to my office !"
"I'm not busy !" Zhang shaozhi replied, and Wang Wenbin walked into his office.
The two men closed the door and talked about the lunch time. Wu Yan does not know what the two talk about. However, Wang Wenbin hopes that Zhang shaozhi will be able to work independently from Wu Yan, while Zhang shaozhi may still wonder whether he should listen to Wang Wenbin or Wu Yan.
At noon, the company went to lunch and held a welcome banquet for Wang Wenbin.
After everyone was seated, Liang xiujuan first introduced: "Dear colleagues, this is next to me, Mr. Wang, the new CEO of our company. He will be responsible for the overall work of the company, we welcome Mr. Wang to give you a speech."
Wang Wenbin owes a debt and says, "Hello, everyone! I am very glad to have the opportunity to work with you. I have already been in touch with some colleagues this morning. I am very fond of our team and feel that our team is really talented, we can certainly make a big event. I have just arrived at the company. I am not very familiar with the situation and hope to get your support. I want to say a few words next. It will take an hour or two if the time is not long !"
"Ah ?" The employees made an exclamation.
"But let everyone look at such a big table of delicious food. When the yellow flowers are getting cooler, It's too disappointing. So everyone will have dinner first and we will talk about it later !" Wang Wenbin is also humorous.
The employees began to eat.
Liang xiujuan then asked Wang Wenbin: "Is it necessary for Mr. Wang to come to the company ?"
"Adaptive, adaptive! It feels like you have a sense of belonging at home !" Wang Wenbin answered without thinking.
"That's good! We do not have any business experience at present. Mr. Wang has a wealth of experience in this field. We can lead everyone to succeed quickly !" Liang xiujuan said with a slight compliment.
"Where, where! Rich experience, dare not do it! Everyone works together !" Wang Wenbin said with ease.
Everyone enjoyed this meal. Although Wang Wenbin was at home with everyone for the first time, he was not at all restrained and could control the atmosphere at the dinner table. Wu Yan was very impressed.
On the way back, Wang Wenbin was originally with Wu Yan and Liang xiujuan, but soon Zhao Qigang took advantage of this opportunity and chatted with Wang Wenbin separately, now, Wu Yan and Liang xiujuan are on the same stage.
Wu Yan said to Liang xiujuan: "It seems that you are quite satisfied with this king !"
Liang xiujuan said thoughtfully: "Okay! I have inquired about it. Wang Wenbin was originally named No. 3 in yimei.com. yimei.com fell down this year. Their first and second leaders started their business again, that is, Wang Wenbin is still keeping his original mess, so that the original resources of yibu.com will belong to him. It is said that his capabilities are average."
"Oh, that's it! Then why did Zhao arrange him ?" Wu Yan always thought that Wang Wenbin, as Zhao Haibo said, is an extremely rare management talent. At this point, we will certainly be able to do a good job for the company! According to Liang xiujuan, Wang Wenbin seems to be not very competent, so Wu Yan asked in an uncertain manner.
"I am also thinking about this, because, as a matter of course, Wang Wenbin should be a member of the core management team and should have real shares of the company, however, Zhao Haibo only gave him a virtual stock, so Zhao Haibo is not very optimistic about him. I feel that Zhao Haibo means to take over the position of CEO first and find a suitable candidate for another job in the future, he should be a transitional character." Liang xiujuan analyzed.
Wu Yan listened to Liang xiujuan's analysis and said, "I think your analysis is justified. I think there may be another one, that is, Wang Wenbin can check and balance with us, in this way, Zhao Haibo does not have to worry about his voice."
Wu Yan wants Liang xiujuan to stand on his own side, so he always regards both of them as a whole when talking.
"Well, you can even see this. It seems that you have been a little bit of a person in the past !" Liang xiujuan said.
Wu Yan was a little embarrassed about this and said, "Where is it? Just think about it !"
Wu Yan felt that there was A bottom in his mind. He felt that Liang xiujuan was still on his own side. He never forgot what Xiao A said. Liang xiujuan was A talented person, as long as you work together with Liang xiujuan, You can do big tasks.
The next step is the product design meeting to be held the day after tomorrow. Wu Yan has carefully read Zhao Qigang's product design documents. In the past, Wu Yan had to read these things without any technical skills. However, Wu Yan read the suggestions carefully several times. He was surprised that he not only understood all the suggestions of Zhao Qigang, but also had his own opinions on many of them, therefore, Wu Yan is working hard to sort out his own ideas according to Zhao Qigang's ideas.
However, Zhao Qigang also seems to have a problem. From his active talk with Wang Wenbin today, he seems to want to use the company's personnel changes to seek greater benefits by standing again. Wu Yan never thought about these things before. He thought these people were boring all day, but now they are sensitive to these things. At the same time, he thought about the relationship between people, it is also very fun.
Zhao Qigang is very enterprising, but he seems a little anxious. He always has the urge to take shortcuts, probably because he is young, because he just graduated from his undergraduate course for two years. Therefore, we should cut down his authority and give him some more specific things to do. We should first put away his opportunistic acute temper. However, these need to be discussed with Liang xiujuan before they can be decided.
As soon as he got back to the company, Wu Yan began to study books on the design of Internet products. Now he is reading a book about the design of the SNS interface downloaded from the Internet, in the past, these non-technical books are also fascinating, and there are also many seemingly simple but profound things. For example, if you use the first or second person to communicate with users on an SNS website, there are many examples of these two terms, such as MySpace and MyYahoo, youTube and others all adopt the second person, but it is interesting that the first person, such as MySpace, is in a dilemma. This is not an accidental phenomenon, but a second person, it is more conducive to the creation of a communication atmosphere between the system and the user, more in line with the concept of SNS, so SNS is more likely to succeed in the second person.
In the past, Wu Yan certainly dismissed the above things, so he would not have the opportunity to apply the knowledge to the project. Although the technical content is not high, however, it can often play a greater role than high-depth technology. Therefore, Wu Yan is glad that he has the opportunity to jump out of pure technology fields and consider more important issues.
However, Wu Yan also found that his technical background also had advantages, and he quickly understood some concepts. Moreover, he knew the implementation details, so he certainly had a deeper understanding of some problems.
After several days of preparation, Wu Yan has fully organized his ideas on product design, and is waiting for this product design seminar to be held.
Coming soon: Product Design Seminar