Ruby on Rails Case Study: changingthepresent.org

Five months ago, arvato systems found me saying:

• I need to lead an important project.
• You can use my favorite technologies.
• The team members (at least part) can be selected by me.

By the way, they also told me that the results of this project can change the whole world! I was cheated by such a thing before, but this time, I believe it is true. In this article, I will tell you the technical details behind the extraordinary creativity of this project. But first, I need to let you know its name --changingthepresent.org. This site is targeted at the $250 billion gift market each year. In the past, we were nothing more than sending a fruit cake or a pair of hairy slippers. With changingthepresent, you can give more meaningful and unforgettable gifts, for example, one hour for cancer researchers and one acre of rainforest ownership. It makes gifts more convenient and has its own profit model. But what surprised me most is that this idea has changed people's way of thinking.
Background if you have some knowledge about nonprofit organizations, You can roughly guess the changingthepresent operating mode. The technology used is similar to other industries that need to gather users through the Internet. However, in changingthepresent, people do not have a relationship with each other, but instead donate and have no choice. At the Lance Armstrong Foundation, the Children's Rescue Association, the first book (American poverty children's book donor), and The sitleclub) with the support of nearly 200 advisory institutions and 380 non-profit organizations, changingthepresent's market operation is on the right track. But here I want to tell you the technical story of this project.
In this article, I will show you the entire process of developing this site using Ruby on Rails. You will understand some of the key features and common plug-ins we use. The technology we use is not shocking, but it may help you to understand what we do every day. My goal is to give you a general understanding of trustworthy technologies in a team collaboration, production environment, tools we use, and rails frameworks that we think are very important. I will also provide some link resources, but will not discuss specific issues in detail. If you want to learn more, leave a message. If you are interested, I will be happy to write a series of articles and discuss your questions in depth.
At the earliest stage, I joined the team as a master programmer. Because of its excellent scalability and stability, the Team Leader preferred Java, but after evaluating the entire solution, it was found that the investment was huge and the system was complicated. We later focused on Ruby on Rails, which takes into account efficient productivity, programming model definition, and running performance. Of course, it is not foolproof. After carefully analyzing the existing sites and our business plans, we feel that we have the ability to design a system that processes millions of requests each day. Although there are no such success stories, we believe that such a level of system can be developed through the aggregation cache and shared-nothing cluster technologies.
I went back in March. Our work is very productive. We can launch demos every week, and the progress is fast. The market is paying attention to this project. By November, I joined wellgood LLC and served as a CTO. In December, we released the first beta version of the site. Rails plays a vital role in improving productivity and final success.
We are still far from the completion of the entire site-so far, only about 5% of the original plan has been completed-but we have accumulated enough, I believe that the subsequent work will become smoother and smoother. Although we have initially conducted benchmark and stress tests, we still cannot ensure that the current version can cope with the growth of access traffic for a long time, because almost no rails site provides a precedent. However, we are confident in the following aspects:

• We can achieve rapid development, testing, and release.
• With the support of a small amount of hardware, we can extend the system to a higher traffic level.
• We can add hardware without modifying the software system as needed.
• Our five-person development team can complete development and management during the Early Growth of the site.
• We can solve possible scalability and performance problems in the future.

For the above aspects, productivity is the most important thing for us. My team must be ahead of competitors to meet the needs of users and investors as quickly as possible. The rest of this article mainly introduces some of our practices from a technical point of view. I will provide information links for problems that are hard to clarify in this article.
First, let's have a general understanding of the overall system. Most Rails Applications use the lamp (Linux + Apache + MySQL + PHP) architecture. For this site, we use sun hardware and F5 bigip for load balancing. Like many other emerging rails sites, our application server is mongrel (mainly Processing dynamic content. And Lighttpd), the database is MySQL. Our databases are deployed in master/Master/Slave Mode to meet fail-over, performance, and scalability requirements.
We chose text drive, which has rich experience in operating large-traffic rails websites, to host our content. Like other Internet sites, we primarily focus on read-only traffic, write traffic mainly concentrated on several personalized social network templates, and user shopping history. We also found that the activerecord model needs to be optimized through analysis. The key is to reduce the number of database requests. In addition, we also use the memcache plug-in to implement the cache policy. Apart from the cache layer, we all adopt the traditional rails Configuration Policy.
Each donor donated with enthusiasm. They often prepare many pictures for gifts. Organizations and organizations that receive donations will also select images that best present gifts to stimulate the interest of visitors. There is no doubt about the value of images. Through the promotion of images, gifts can be given to those who need them as soon as possible, enhancing donor confidence and pride. However, a large number of image downloads undoubtedly increase the burden on servers. Mongrel features not static content, so we use URL rewriting technology to manage images. At present, we are also preparing to use panther express for image caching to speed up access.
Some Java and. NET developers and related vendors have warned us that Ruby on Rails may not meet our scalability requirements, but we are not very worried about this. I believe that software technology has been developed for so many years and has accumulated a lot of experience and lessons that are equally applicable to rails. The cornerstone of rails, lamp, is also the mainstream architecture guideline for various large web sites. For me, the more important question is how to achieve the needs of investors, users and bosses as quickly as possible. Now, I spend more time on teams, tools, and implementation processes.
Process and team rails declared: "We are agile, we are happy ." I fully agree with this. I love agile development. We warmly embrace the following agile principles:

• We focus on shortening the development cycle and sticking to the weekly iteration. We should release at least one new version every week.
• We rely on developers to develop and write test cases, rather than testing in a missed phase to ensure quality.
• We are not afraid to restructure the system to optimize the business model. Through refactoring, we have improved code quality, requirement understanding accuracy, and system flexibility.
• For sub-requirements, we implement weekly management; for important requirements, we implement monthly management.
• We use the weekly release demo to maintain close contact with users.

We use TRAC/SVN to execute source code principles, requirement tracking, and release plan management. The high integration of TRAC and SVN brings great convenience to our work. After heavyweight automated testing and lightweight manual testing, we incrementally modify our products based on the testing results and implement branch management for important revisions. We usually do not need to make major revisions when using the rails platform.
Our team members are distributed in three time zones and may soon be more dispersed. Members can communicate with each other at any time to solve the problem as soon as possible. Although it depends on long-distance communication, I still hope that my team will have more excellent rails developers, rather than working close, but there are too many new users. So far, the benefits of this decision are obvious. We also try to increase the number of meetings and shorten the meeting time. We hold regular meetings every day via Skype and hold weekly technical meetings attended by market personnel. We also insist on releasing the periodic plan every week.
Some people say that rails does not support expansion of complex solutions. I do not agree with this statement. Compared with similar technologies, rails has the highest productivity. Rails has a dramatic impact on our development process, team building, and management. The lower the productivity of the technology used, the longer the iteration cycle; the larger the team, the more time it takes to check and coordinate-after both Ruby and Java project management, this point of view is more supported. Using other technologies requires building a larger team (for example, 10 to 15 people), while we can minimize the number of teams. At present, we have five Ruby developers. at most two of them are responsible for HTML design. It may be more appropriate to add seven people in the future. At that time, we will have a second development team dedicated to solving complex and key problems.
The tool we use is super simple. For example, code coverage testing uses rcov, integration testing uses selenium, and all testing and coverage tools are driven by rake. The coverage rate of our test cases is generally around 85%, especially after strict tracking of the coverage rate, which is basically within the range of 79% to 90%. Because we often modify user interfaces, selenium usage is not ideal. However, I think the success rate will gradually increase after the user interface is gradually fixed. At present, we mainly Conduct Unit and function tests, but have made great progress in exploring other testing methods.
Our developers are free to choose encoding tools. So far, no one has ever used the integrated development environment from the beginning to the end. The main source code editing tool we use is textmate. Rails benchmarks and debugging tools with logs and test functions are sufficient to help us complete system debugging. Refactoring is a bit of a headache, especially when you have to modify the database model. Therefore, once you find that there is enough good ide, we will still consider using it. However, in my experience, I think our current maintenance speed can be three times faster than that of Java applications.
Although the core architecture seems a bit radical in the selection of the project solution, the most important thing is to continue to adhere to it as always. Our core architecture will still be centered on rails. On the Database End, we use DHH (Note:Loyal to rails author David Heinemeier Hansson, who does not forcibly require database constraints such as integrity or use database views. In terms of testing, we mainly rely on the rails framework. Of course, as the size of test cases increases, we are also actively trying other tools. Many aspects are still under exploration and change when they are not final. But in general, we use the classic rails MVC model supported by rest WebService. This is also detailed below.
Like many online community sites, the model first optimizes read operations because most of our content is static and cannot be modified once a day on average. By analyzing competitor statistics, we found that to increase the system capacity, caching stable content is very important, such as charitable organizations (such as UNICEF.Note:United Nations Children's Fund, UNICEF), gifts (such as one hour for cancer researchers) and institutional missions (such as world peace and medical research ). These contents do not change frequently and can be cached stably. We use the rails plug-in actsascacheable to implement static content caching. Its backend is deployed with a distributed second-level cache service (memcache). However, in special cases, activerecord APIs are usually used directly.
This site also includes the workflow mechanism. For example, if a charity submits a content, the changingthepresent administrator reviews and approves the content. We must provide the ability to activate and freeze any gifts at any time, and we prefer to mark records as deleted rather than directly deleting them from the database. To support this custom workflow, we introduced a plug-in named actsasstatemachine. This plug-in supports using domain specific language to express the state machine. For details about DSL, see cross-border: rails plug-in.
I realized the power of DSL only when I knew it was used to communicate with our business users. Ruth Ann hacking is responsible for site content collection and management. Although she has no programming experience, she is a fencing player who has participated in the World Cup. Because she can use a military knife (or even a ball pen) to kill me at any time, I tried to give her fun all day. I gave her the code containing the state machine at one time. Unexpectedly, she finished reading the code from start to end and found several bugs! Ruth Ann is so happy that I live now. This is the life with rails. There are always some small things hidden in its framework. It makes me happy to jump out from time to time.
User Interfaces our user interfaces are built on the basis of the classic rails controller-View-layout mode. New restful controllers are not yet supported, but will be added later. We only use Ajax in a few places-such as the greeting card generation wizard and on-site editing. All in all, our goal is to make user interfaces as concise and clear as possible. To improve the scalability of the site, we have taken the following measures:

• Use as few images as possible. For example, the site has many rounded corners, and we use JavaScript libraries for rendering. Of course, this policy will change shortly after pantherexpress is supported.
• Reduce page cache. In our site, there are many user-defined things, so there are few static pages and the page cache is of little value. Currently, the cache is mainly implemented at the activerecord level, and pantherexperss is used for image caching.
• At first, we showed the gifts to users in the order of the first letter of their names. But soon, it was found that all users could only start browsing experience from the gifts named after "a", so they changed to a random sorting method and cached results every day. Instead of implementing this function through the view, we use the MySQL random function to design a user-defined searcher with the current time as the Random Seed.