"Reprint" How to write a frame: steps (bottom)

Description: When writing this article, the author completely writes out the things in his mind, and does not refer to any material, so it may be incomplete for each content, and cannot be used as a complete reference. There are some ways to learn things that everyone has their own preferences, not feel right and wrong.

Unit Test

The framework we wrote before is only a framework that can be used in the most basic cases, and as a framework we cannot predict how developers will use it in the future, so we need to do a lot of work to ensure that the framework is not only functional but also robust. Writing application code, most projects do not write unit tests for many reasons:

1. Project rush time, even do some input verification there is no time to do, where there is time to write test code.
2. The quality requirement of the project is not high, as long as the function can be available under the standard operation flow.
3. The project is basically not going to change or the temporary project, once the test pass is always like this, there is no iteration.
4. ......

For the framework, on the contrary, there is no framework for matching unit tests (that is, it is very scary to use a manual method of testing, such as calling some methods in main to observe the log or output, or to run the sample project to see if the functionality is normal, which is very frightening) for the following reasons:

1. The high degree of automation, the short time required for regression, and even integration into the build process are not achievable by manual testing.
2. The framework must have a lot of iterations and refactoring, every modification although only changed a function, but may affect the B and C functions, manual test you may only verify that a is normal, easy to ignore B and C, using unit testing as long as all functions have coverage, It is almost impossible to omit the potential problems caused by the modifications and also to feed back the compatibility issues caused by the modifications.
3. As previously said, once the framework is open, the user of the framework may use your framework in a variety of ways, with different environments causing a lot of weird boundary input or illegal input, and requiring rigorous boundary testing of the code using unit tests to ensure that the framework can survive in harsh environments.
4. Unit testing can also help us improve the design, when writing unit tests when it is difficult to find the target code to simulate difficult to build effective unit tests, then the target code may be strong dependency or responsibility is too complex, a unit test is highly covered by the framework is often well-designed, in line with the cohesion-poly low-coupling framework.

If the time requirements of the framework are not particularly tight, the introduction of unit testing can be introduced in the process of the main line, the sooner the maturity of the introduction of the framework may be higher, the less chance of refactoring rework later, the framework reliability will certainly be greatly improved. I had written a class library project before, did not write unit tests, used the class library in the project for some time and did not have any problems, and then took a little time to write unit tests for the class library, to my surprise, More than half of all the APIs provided by my class library are not unit tested (originally thought to be a mature class library that actually contains dozens of bugs), and even one of the APIs is used in my project. You might ask, why is there a problem when using this API without a problem and when the unit tests are in place? As mentioned before, I am the architect of the framework and I know the best practices when I use the API provided by the class library, so I have a special setting for the class library when I use it, if it is not exposed through unit tests, Then other people will encounter a potential bug when they use this class library.

Demonstration projects

Writing a sample project is not just for reference, but also to help you improve the framework, for example projects, it is best to take into account the following points:

1. A site or system that is meaningful, rather than simply demonstrating the features. This is because, most of the time only those real business logic will expose the problem, when demonstrating the characteristics of the time we always have some set of thinking will avoid many problems. Or you can provide two items, a purely demo feature, and a sample project.
2. Overwrite as many features or use difficulties as possible, provide some comments in the code of the project, many developers do not like to read the document, but prefer to look at the sample project to get started directly (imitate the sample project, or directly with the code in the sample project to modify).
3. The code in the project, especially the code that involves the use of the framework, must be standardized, and the reason for that is that, as the architect of the framework, you don't want everyone copying code that's stuck in a messy code.
4. If your project is targeting more than just Web projects, the sample project is best served with two versions of the web and desktop, one that makes it easy for you to discover the differences in usage due to different environments, and to give different best practices for different types of projects.

Perfecting logs and exceptions

A good framework not only needs to be well designed, and the processing of logs and exceptions is also a very important criterion, here are some counter examples:

1. There is no uniform standard for the use of various levels of logging, and even a level of logging is always used.
2. Almost without any log, the framework runs completely in a black box.
3. The log logs are many and have no actual meaning, but are used to observe the contents of variables when debugging.
4. The exception type uses only exception, does not use a more materialized type, and does not have a custom type.
5. The exception message text only writes "error" words, does not write clearly the specific problem lies.
6. Always just throw an exception, let the exception rise to the outermost layer, and give it to the user of the framework to handle.
7. Use exceptions to control the code flow, or you should use the return value when the method does not achieve the desired effect.

In fact, personally, a framework of the main logic code is not necessarily the most difficult, the most difficult is the processing of some details, so that the framework to maintain a set of standardized unified log and exception of the use of the framework is a difficult for developers, the following are some suggestions for logging:

1. The first thing to do is to have a specification for the log level used by the framework, such as definition:
   1. Debug: Used to observe the running flow of the program, only open during debugging
   2. INFO: Used to inform the program of the State or stage of the change, can be opened in the test environment
   3. WARNING: An error or an exception that informs a program that it can recover itself, or that does not affect the execution of the mainline process, can be opened in a formal environment
   4. Error: Used to inform the program is not recoverable, the main line process is interrupted, need to develop or operations personnel aware of the intervention errors or anomalies, need to open in the formal environment
2. According to the above level specification, in the need to record the log where the log, in addition to the debug level of log other logs can not be too much, if the framework is always running when the output of dozens of warnning is also easy for users to ignore the real problem.
3. The logging message needs to be clear, preferably with some contextual information, such as "unable to find the profile xxx.config under XXX, the framework will use the default configuration" instead of "Load configuration failed!" "

Here are some suggestions for using exceptions:

1. The framework cannot complete the functionality represented by the API name due to configuration errors or use errors or run errors, consider throwing the transformed exception, letting the caller know what happened, and the framework can establish its own error handling mechanism
2. For predictable errors, and the type of error can be enumerated, consider informing the caller in the form of a return value that the subsequent logic can be handled according to different results
3. A warning or error log can be logged if an error can be retried or does not affect the return that is encountered by callers who are not able to resolve the internal functionality implementation of the framework
4. You can accompany each module with a custom exception type that contains contextual information (such as viewexception can contain viewcontext), so that an exception can be handy to know which module is having problems and can get environmental information when an exception occurs
5. If the exception crosses the implementation level (for example, from the framework to the application), then it is better to do a wrapper conversion (such as the hint that the file read failed to load the configuration file failed), otherwise the upper layer does not know how to deal with these internal problems, internal problems need to be handled by the framework itself
6. Exception Log can be recorded and the current operation is closely related to the parameter information, such as the search path, view name and so on, the information about the method does not have too many records, the exception is generally with the call stack information
7. If possible, when an exception occurs, you can analyze why this problem occurs, in the exception information to some problem-solving suggestions or help links to facilitate users to troubleshoot problems
8. Exception handling from bad to good levels is when a serious problem arises:
   1. The user knows nothing, the integrity and logic of the program is compromised.
   2. The user doesn't know what's wrong or how to solve it.
   3. Users can clearly know what's wrong, but can't solve
   4. The user not only knows what happened, but also can quickly solve the problem through the guidance of the exception message.

Perfect configuration

The part of the configuration can be left to the framework to write almost again to write, because this time can already think clearly which configuration is:

1. Need to be publicly available for user configuration, and configuration will vary depending on the environment
2. Need to be publicly available to users to configure, configuration and deployment environment Independent
3. Only needed within the framework for the framework developers to configure
4. No need to be a configuration, as long as you centrally store this setting in your code

There are generally several ways to configure:

1. Configure through a configuration file, such as an XML file, a JSON file, or a property file
2. Configure with annotations or attributes (Annotation/attribute) (classes, methods, parameters)
3. Configuration by code (such as a separate configuration class, or a configuration API that implements a configuration class or calls a framework)

Many frameworks provide a variety of configurations, such as spring MVC support the configuration of the above three ways, the individual feel that the configuration, we should be treated differently, rather than without the brain to all the configuration items are provided in the above three ways to provide configuration, we have to consider the high cohesion and low coupling principle, for the web framework, Cohesion need to consider more than low coupling, my recommendation is to provide different configuration options for different configurations:

1. If the configuration item needs to be configured by the user, especially if it is related to the environment, it is best to configure it using configuration methods, such as open ports, memory, and number of threads, but be aware that:
   1. All configuration items need to have a default value, if the configuration is not found using the default value, if the configuration is not reasonable to use the default value (you do not want to use the framework of the people of the thread pool inside the min set to 999999, or timer interval set to 0 milliseconds it?) ）
   2. When the framework starts to detect all the configuration, if not reasonable to give hints, most people will only look at the start of the log, when the use of no matter
   3. I wonder if you would prefer XML or JSON or key-value pairs for the configuration file format?
2. For all configurations that are only made at development time, try not to use the configuration file and make the configuration as close as possible to the objects it configures:
   1. If the configuration of the set extension type of the framework integrity, it can provide code configuration, such as the MVC framework we want to implement the various iroute, iviewengine, etc., it is best to provide iconfig interface to enable developers to implement the interface, So they can know what can be configured, and the code is the document.
   2. If the configuration of the model, action, such as the model validation rules, filter, etc. are all annotated way to configure
3. Some people say that it is very flexible to use configuration files for configuration, and the use of code and annotations to configure is not flexible and may be intrusive. I think it's still a trade-off, and my advice is not to put too much of the framework inside the configuration file, to make it harder for users (and most of the time, most people just copy the configuration in order to complete the configuration, not to use configuration files to configure your framework for real flexibility, Look at the Internet so SSH configuration file copy to copy to know.
4. Finally, I suggest a lot of things too internal for the lightweight application framework can not provide any configuration options, only need to be defined in a constant file, so that a real need to do two development developers to modify, for a framework if a sudden exposure to hundreds of "advanced" configuration items to users, they will be dizzy.

Providing status services

The so-called state service is to reflect the framework of the internal operation of the service, many open Source services or systems (Nginx, MONGODB, etc.) have provided similar modules and functions, as a framework I think it is necessary to provide some internal information (mainly configuration, data statistics and internal resource status) out, This allows people who use your framework to understand the workings of the framework at the time of development or on the line, and we give two examples of this information for a Web MVC framework we mentioned earlier:

1. Routing configuration
2. View engine configuration
3. Filter configuration

For a socket frame, there are some differences, the socket framework is stateful, and its state service provides information in addition to the configuration information currently in effect, more than the current framework to reflect the status of some resources and statistical data:

1. Various configurations (pool configuration, queue configuration, cluster configuration)
2. Socket-related statistics (total Open, total shutdown, send and receive data per second, total data received, current open, etc.)
3. The current state of the various pools
4. The current status of various queues

Status services can be provided in the following ways:

1. Code, for example, if the developer implements the Ixxxstateaware interface, you can push some information for its implementation class, or you can set up a statecenter directly in the framework to expose all the state information of the framework.
2. Automatic log mode, for example, if the statelogginginterval=60s option is turned on in the configuration, our frame will automatically output the log once a minute, showing the state inside the frame
3. interface, such as opening a restful interface or listening to an additional port to provide status services, allowing users to integrate raw data and other monitoring platforms
4. internal and external tool mode
   1. For example, we can provide the framework with a dedicated page (/_route) to render the configuration of the route (even if we can let the developer directly enter the address on this page to test the matching of the route, the state service is not necessarily only seen), so that the development and testing can be more convenient debugging
   2. We can also provide a proprietary tool for the framework to view the state information of the framework (which, of course, might be a network service that connects the framework to get the data), so that even if the framework is used in multiple machines, we may have only one monitoring tool

If there is no State service, then the framework is a black box at run time, and if the state service is detailed enough, we can easily troubleshoot some functional or performance issues. It is important to note, however, that the body service may degrade the performance of the framework, and we may need to perform a one-time test of the state service to eliminate the loss performance in the State Service (some data collection can be unexpectedly lossy).

Check thread safety

The framework's support for multithreaded environments is an important evaluation criterion for frame quality, and it is often seen that even some mature frameworks can have multi-threading problems. Here are a few things to cover:

1, you can't predict how the user of the framework will instantiate and save your API's entry class, and if your entry class is used as a singleton, will there be a single-threaded problem in the case of concurrent calls?

This is an old topic, have said many times before, you in the design framework when the heart if you put a class into a singleton class but do not provide a singleton mode, you can not ask users to help you achieve a singleton. This involves more than just multithreading, and there may be performance issues. For example, the cacheclient of a client who has seen a distributed cache requires the user to keep a single cacheclient for a cache cluster (because there is a connection pool), but each time a cacheclient is instantiated, A few hours later, tens of thousands of half-dead sockets cause the network to collapse. I've seen the code comment for the entrance factory of a class of libraries that says that the person using the xxxfactory is using it as a singleton (because it caches a lot of data), but the person who uses it doesn't notice the comment, and each time it instantiates a xxxfactory, causing the GC to crash. So I think as the framework of the designer developers, it is best to put the framework of best practices directly into the API, making it impossible for users to make mistakes (say a word before, again, a good framework will not let the user wrong). You might say that for cacheclient example, it is impossible to make a singleton, because my program may need to use more than one cache of clusters, another idea, we can completely in the packaging layer, through a cacheclientcreator and the like The Cacheclient class to manage multiple singleton cases. Even in some extreme cases, you can't just provide a way for the user to go, also need to do some testing mechanism within the framework, to remind users in a timely manner, "We found that you use the framework, which may create problems, do you intend to do that?" "

2, if your entry class is a singleton, do you have a shared resource in your class, and will your API be called in parallel to ensure thread safety for those resources? There are several difficulties in solving multithreading problems:

1. It's hard to think that this code is going to be called concurrently. For example, an init () method, a config () method, you always assume that the consumer will call and call only once, but the fact is not necessarily the case, sometimes the caller himself is not sure how many times my container calls me this code.
2. OK, to solve the multi-threading problem of all kinds of irritability, then all the methods involved in sharing resources are locked. A rough (granular) lock on a method can lead to a dramatic drop in performance or even a deadlock problem.
3. You think you're using an elegant lock-free code or a concurrent container but you don't get the idea. We tend to use a large number of concurrent collections in the heart of the problem to solve the multi-threaded problems and achieve excellent performance, but you think this is to solve the problem of thread safety but in fact there is no, we can not assume that both A and B are thread-safe, but the entire code snippet for A and B method calls is thread-safe.

I don't have a good solution for multithreading, but here are a few things I think you can try:

1. You need to go through the code very carefully, place the shared resources, and the related methods and classes, and don't assume anything, as long as the API is exposed, assuming it might be called concurrently. Shared resources are not necessarily static resources, and even if the resources are non-static, it can be problematic to operate on the same object's resources in a concurrent environment.
2. Generally for the public API, as the architect of the framework we need to make sure that all static methods (or instance methods of the Singleton class) are thread-safe, we can not do this for instance methods (because of performance reasons), but we need to explicitly prompt the user method's non-thread safety in the comments, If concurrent calls are required, handle thread-safety issues on your own.
3. You can see if it's possible to make these resources (fields) into local variables within the method, sometimes we don't really need classes to hold a field, just because multiple methods have to use the same things and write them around.
4. For some of the methods associated with low frequency use of some resources is not necessary to use concurrent containers, the direct use of rough method of resource locking or even methods level lock, first to ensure that there is no thread safety, if you do a pressure test after the performance problem to solve.
5. Some of the resources associated with high-frequency methods can use concurrent containers, but you need to think carefully about whether the code will have thread-safety issues and, if necessary, design some multithreaded environment unit tests for the code to validate.

Performance Testing and optimization

As mentioned before, you don't anticipate how your project will be used in terms of traffic, and we don't want the framework to have significant performance gaps compared to similar frameworks (if you're doing an ORM framework or RPC framework, this is essential), So after the framework is basically complete we need to do benchmark:

1. Identify several test cases to cover the main process and some important extensions as much as possible
2. Find a few mainstream similar frameworks, implement the same test cases, achieve the time to be simple, try not to rely on other external frameworks
3. For these frameworks and their own frameworks, use stress test tools to run these test cases in the same environment and platform, using charts to draw the execution time under different pressures (as well as the consumption of major resources such as memory and CPU)
4. If there are significant gaps, use the Profiling Tools to troubleshoot and optimize, such as:
   1. Optimizing the implementation of thread safety within a framework
   2. Do some caching for the code within the framework (caching of reflected metadata, etc.)
   3. Reduce call levels
   4. These adjustments may break the original mainline process or make the code difficult to understand, leaving the relevant comments
5. Constant stress testing and optimization process, each attempt to optimize the performance of 5%~20%, although the more it may be more difficult to later, if the discovery is not optimized (the performance analysis tool shows that the distribution of performance is very uniform), you can look at other frameworks for this part of the work implementation of the Code logic

Encapsulation and expansion

Personally think a framework if only can use that is the first level, can be easily extended or two development that is another level, if our keel stage work is good enough, the framework is a three-dimensional full frame, then this part of the workload will be much smaller, Otherwise we need to refactor the framework a lot so that we can reach this level.

1. We need to look at all the types of frameworks and see what types we are planning to provide for developers to enhance, extend, or replace, to respond to these types of structural adjustments.
   1. For example, if you want to be augmented, you need to consider it from an inheritance perspective.
   2. For example, if you want to be expanded, you need to consider it from a provider perspective.
   3. For example, if you want to be replaced, you need to provide a replacement for the component in the configuration
2. We need to fine-tune these types again:
   1. Check whether the closed closed up, the open open up the
   2. Whether enhanced extensions or replacements can cause side effects
   3. For new incoming types, do enough checking when receiving and using
   4. The perfection of related logs

Refactoring or refactoring

Just refactoring this thing can actually say a book, in fact, I have a bit of code cleanliness, here are some of my own code when the focus on the place:

1. Format: Use the IDE to format your code and references every time you commit your code (of course, implementations may need to configure the IDE for the code style you like)
2. Name: Keep the whole class and interface naming unified, a variety of er,provider, Creator, Initializer, Invoker, selector represents a thing, do not use Hanyu Pinyin name, if the English is not good enough to check the dictionary, Sometimes I even read some source code for a name to see how the foreigner named the object or the method
3. Access Control modifiers: This is a very difficult detail to do because there are too many places to have access control modifiers, and what level modifiers are given often depends on the framework's extension. Can at the beginning of the time to give the minimum permissions, when necessary to slowly improve, for example, in addition to the method must give public places (such as the common API or implementation interface), as far as possible to the private, in the hierarchy of inheritance to give the protected, for the class can be given the default package /assembly permissions, and then go to public when a compilation error occurs
4. Attribute/getter, setter: For non-Pojo class field of the public also want to think carefully, whether it is necessary to have setter, because once the outside can be set an internal field of the class, then not only can change the internal state of the class, you have to consider how to deal with this change, is not a thread Security issues and so on, even to consider whether it is necessary to open getter, whether the information inside the class should be exposed to external
5. Method: Consider whether the existence of each method in the current class is reasonable, whether this belongs to the current class should do things, whether the method does too many things too few things
6. Parameters: Need to think, for each method to invoke the parameters, should be passed to the method, or let the method to get it, should pass multiple parameters, or encapsulate a context to the method
7. Constants: Try to use enumerations or static strings instead of some of the constants or magic numbers used by the framework, and you need to classify the constants in a single category not a single brain heap in a constant class consts

In addition to some of the problems mentioned above, I think the most important sentence for refactoring is: do not let the same piece of code appear two times, mainly around the principle of refactoring will often solve many design problems, to achieve this goal may require:

1. Do almost live classes use inheritance to avoid code duplication (refining the superclass) and use the template method to leave the variance to the subclass implementation
2. The constructor method can be called hierarchically, and the main construction method can be as long as one, do not implement too much logic in the construction method
3. If the code of the method can be duplicated, consider extracting a smaller public method to invoke the method (refining method), or consider using a lambda expression to extract the more granular repetitive code (refining logic)
4. You can use the IDE or some code analysis tools to analyze duplicate code, and if you can do anything to avoid these repetitions, the code quality can be improved by one level

In fact, it is not necessarily in the reconstruction of the time to deal with all of the above problems, if you write code when all with these consciousness to write, then the burden of refactoring will be smaller (but the burden of writing code is relatively large, need to consider packaging problems, elegant problems, log anomalies, multithreading problems, etc., So writing a code that can be used and writing a good set of code is really not the same thing.

Project documentation

If you want someone else to use your framework, it is necessary to provide and maintain a project document in addition to the sample project, and I recommend that the document be divided into these sections:

1. Feature Features:
   1. Equivalent to a project brochure that allows others to be attracted by the highlights of your project
   2. Each feature can be a word to introduce
2. Beginner Getting Started Get started:
   1. Introduce the basic position and function of frame
   2. Start with the download and let the user know how to use the framework in a step-by-step way
   3. Reading time of the entire document is less than 10 minutes
3. Beginner Tutorial Tutorials:
   1. Provide 5~10 articles on the user's point of view to introduce the main functional points of the project
   2. Or a step-by-step way to teach you to use the framework to complete a small project (such as CRUD)
   3. Describe best practices for using the framework
   4. Reading time of the entire document within 8 hours
4. Manual Manual:

   "Reprint" How to write a frame: steps (bottom)