RESTful API Design and engineering practices

This is a creation in Article, where the information may have evolved or changed.

Statement

This article is just a summary of my personal reading materials and engineering practices, which may not be the best practice. But hopefully, some help will be given to readers who have doubts about restful API design and engineering practice.

Objective

The RESTful principle was put forward by Roy Fielding in the fifth chapter of his thesis.

The RESTful API is to the backend developer, just as the UI is to the UI designer. RESTful APIs As with all UIs, a standard, friendly, and consistent user experience is of paramount importance. To achieve the above goal, the API needs to meet the following requirements:

• Comply with the WEB specifications and common conventions as much as possible
• The invocation interface is straightforward, readable and unambiguous.
• API style consistency, call rules, incoming parameters and return data have a uniform standard
• Provides a simple and flexible way for clients to access data
• Efficient, secure and easy to scale

Safety

Secure connection with HTTPS

The use of HTTPS to ensure the security of the entire API call process, which can effectively prevent eavesdropping and tampering.

In addition, because of the security of communications, a more convenient Token mechanism can be used to simplify the validation process without having to sign each request again.

For non-HTTPS API calls, do not redirect it to HTTPS, and return a call error directly to prohibit unsafe calls.

Using JWT to implement the Authorization mechanism

The authentication mechanism based on JWT is a better scheme in the condition-free authentication mechanism . For specific implementations and how to use them, please read the JWT.

When using JWT, be sure to use HTTPS.

Design of the API

Version control

The iteration of the API is inevitable. Versioning of APIs is necessary to ensure that the developer's interests are not compromised by frequent iterations of the API during the iteration.

There are two viewpoints about where the version information is saved:

• Url
• HTTP Header

Theoretically, it should be placed in the HTTP header. But individuals feel that better practices are placed in URLs, which makes it more intuitive to see the version of the API that is currently in use.

However, there are tradeoffs between how to use the--strip API versioning:

• The main version number is placed in the URL to indicate the overall structure of the API
• The time-based minor version number is placed in the HTTP header to indicate minor changes in the API, such as the abandonment of the parameter field, the change of Endpoint, etc.

123	GET https://api.stripe.com/v1/charges http/1.1stripe-version:2017-01-27

Implementation method:

• API callers add version information to the request Headers to indicate the API version they have requested
• The API provider adds the version information corresponding to the Request Headers in the Response Headers to indicate the API version being responded to

Routing

The key to RESTful API design is to "decompose the API into logical resources and modify the resources through HTTP methods that have specific meanings (GET, POST, PUT, PATCH, DELETE)."

Like what:

• GET /posts-Get Post List
• GET /posts/8-Gets the specified post
• POST /posts-Create a new post
• PUT /posts/8-Update post with ID 8
• PATCH /posts/8-Partial update of post with ID 8
• DELETE /posts/8-Delete post with ID 8

Among them, there are a few points to note.

Use nouns to name resources, not verbs

Read more.

Endpoint use plural instead of singular

Even though it is wrong to use a plural form to represent a single resource, always use the plural form in order to ensure consistent URL formatting. Also, do not deal with the complex transformations of special words in English, such as Goose/geese.

The association between resources

If there is an association between the two resources, such as the association between posts and comments, you can map the comment to the route of the post in the form of:

• GET /posts/8/comments-Get a list of comment #8 posts
• GET /posts/8/comments/5-Get the contents of the comment #5 of the post #8
• POST /posts/8/comments-Create a new comment for the post #8
• PUT /posts/8/comments/5-Update the contents of the comment #5 of the post #8
• PATCH /posts/8/comments/5-Partially update the contents of the comment #5 of the post #8
• DELETE /posts/8/comments/5-delete the comment of post #8 #5

In addition, if the comment has its own corresponding route, for example, it is /comments best not to use the above /posts/:post_id/comments/:comment_id path, but instead use its own corresponding route. This is done for a two-point reason:

1. Avoid duplicate API designs
2. Use shorter, more convenient routes

Conditional request

Please read the HTTP conditional requests.

Query parameters

Query parameters are implemented via the URL's query Parameters.

Resource sequencing

Query Parameters in 排序关键字=[-]字段1,...,[-]字段N the form of stitching:

• Sort keywords: You can choose your own, do not conflict with other fields, such as sort .
• Field: The , list of fields to delimit if the field prefixes are in - descending order.

1	/entrypoint?sort=-age,sex

Resource filtering

Query Parameters in 字段=值 the form of stitching.

1	/entrypoint?age=35&sex=male

Resource Field filtering

API consumers do not always need all the content of the resource. The ability to provide on-demand return fields is a great benefit for reducing traffic consumption and accelerating API calls.

Querystring in 字段过滤关键字=字段1,...,字段N the form of stitching:

• Field Filter Keywords: You can choose, do not conflict with other fields, such as fields .
• Field: , the list of fields to separate.

1	/entrypoint?since=1499410815441&count=10&age=35&fields=age,sex

Full-Text Search

Sometimes, the basic resource filtering function is not enough. At this point, you may be using ElasticSearch or other Lucene-based full-text search tools.

When full-text search is used, the parameters of the full-text search should be provided through the query parameters of the resource API. The data returned after the query is complete should be consistent with the normal list query.

Like what:

1	Get/messages?q=return&state=read&sort=-priority,created_at

Set up aliases for common queries

Some queries are often used to give developers more comfort when they are set up with individual names. For example, query for recently read messages:

1	Get/messages/recently_read

Request Content

If you do not need to be compatible with older systems, use JSON first.

Ensure that the requested header is received Content-Type application/json , otherwise it will be returned 415 Unsupported Media Type .

Response Status Code

HTTP defines a number of meaningful status codes, but not all of them can be used. Some of the commonly used status codes are listed below:

• OK-Responds to a successful GET, PUT, PATCH, or DELETE operation. can also be used on POST operations that do not create new resources
• 201 Created-Responds to POST operations that create new resources. Should take a location header pointing to the new resource address
• 204 No Content-responds to successful requests that do not return a response body (such as a DELETE request)
• 304 Not Modified-http when the cache header is in effect
• The bad request-requests exception, such as the body of the request cannot be resolved
• 401 Unauthorized-No certification or authentication is illegal. When the API is accessed through the browser, it can be used to pop up an authentication dialog box
• 403 Forbidden-When authentication is successful, but the authenticated user does not have access to the resource
• 404 Not Found-Request a resource that does not exist
• 405 Method Not allowed-The requested HTTP methods do not allow the current authenticated user to access
• 410 Gone-Indicates that the currently requested resource is no longer available. Useful when calling an older version of the API
• 415 Unsupported Media Type-if the content type in the request is wrong
• 422 unprocessable Entity-used to indicate a checksum error
• 429 Too Many requests-denied access due to request frequency reach limit

Response Content

• If you do not need to be compatible with older systems, use JSON first.
• After you create and modify the operation, return all the information for the resource.

Error handling

Errors generally fall into two categories:

• Client Request Error
• Service Side Response Error

About the use of status codes

Client request error using 400 series status code, service side response error using 500 series status code.

About the response body

When an error occurs, the response body returned by the API should provide some useful information to the developer:

• The only error code
• Useful error Messages
• If possible, provide a description of the error details

In JSON format, it looks like this:

12345

{ "code": 1234, "message": "Something bad happened:(", "description": "More details on the error here"}

For PUT, PATCH, POST requests, when a checksum error occurs, additional fields are used errors to provide the error details, such as:

12345678910111213141516

{ "code": 1024x768, "message": "Validation Failed", "errors": [ { "code": 5432, "field": "First_ Name ", " message ":" First name cannot has fancy characters " }, { " code ": 5622, " field ":" Passwo Rd ", " message ":" Password cannot be blank " } ]}

More meta-information

If more meta information is needed, it can be placed in an HTTP header. The more common meta-data are:

• Paging information
• Request frequency limit information (already mentioned)
• Certification information (already mentioned)
• Version information (already mentioned)
• ......

To avoid naming conflicts, when you set up an HTTP header, you should prefix the custom fields in the HTTP header, such as OpenStack:

123	Openstack-identity-account-idopenstack-networking-host-nameopenstack-object-storage-policy

Previously, the designer and the implementation of the protocol used X- prefixes to differentiate between custom and non-custom HTTP headers, but the practice proved that the problem was solved by introducing many problems. So RFC 6848 has begun to scrap this practice.

In addition, although the HTTP specification does not specify the size of the HTTP header, in some platforms it is limited in size. For example, the header size of node. JS cannot be higher HTTP_MAX_HEADER_SIZE (the default 80KB), so it is designed to protect against DDOS attacks based on HTTP headers.

Turn on gzip compression

Remember to turn on gzip compression.

Call Frequency limit

Server resources are limited, and in order to prevent intentional or unintentional high frequency requests, it is necessary to limit the frequency of requests.

RFC 6585 introduces an HTTP status code 429 Too Many requests to solve this problem.

Of course, it would be better to notify the API consumer before reaching the call cap. There are some standards missing in this area, but it is a popular practice to use HTTP response headers.

Implementation: The API provider maintains the frequency limit information of the API caller and notifies the API caller of the call to the API through multiple Response Headers:

• X-Rate-Limit-Limit: Maximum number of requests allowed in the current time period
• X-Rate-Limit-Remaining: Number of requests remaining in the current time period
• X-Rate-Limit-Reset: How many seconds are there, and the limit on the number of requests will be reset

Why is x-rate-limit-reset using the remaining seconds instead of the timestamp (timestamp)?

Timestamps contain a lot of useful but not necessary information, such as dates and time zones. What API consumers really want to know is when they can continue to initiate requests, using seconds for consumers to minimize the cost of processing. And the use of seconds also avoids the clock offset problem.

Cache

HTTP has built-in caching policies. You just need to add a few headers to the API response and check some of the request headers when processing the request.

There are two scenarios: ETag and last-modified.

ETag

When processing a request, the response contains an HTTP header called the ETag, whose value can be a hash or checksum of the contents of the resource. The value of the ETag should change as the content of the resource changes. If the HTTP request header is included If-None-Match , and its value is the same as the ETAG value of the requested resource, then the API returns the 304 Not Modified status code instead of outputting the resource content.

Last-modified

And the ETag works about the same, the difference is that the head uses a timestamp. The response header last-modified contains a timestamp in RFC 1123 format, which is used to verify the value of If-modified-since (HTTP protocol accepts three different date formats, so it should be handled for all three format servers).

API Documentation

• Keep documents synchronized with API updates
• Make documents publicly accessible and easy to find
• The documentation should provide a complete example of the invocation (GitHub and Stripe are familiar with this path)

Testing of the API

Black box test

Black-Box testing is a test method that does not care about the application's internal results and how it works, but only about whether the results are correct.

When testing black boxes, you should not Mock any data.

Also, when writing tests, make assumptions about the system state as much as possible, but in some scenarios, you need to know exactly where the system is currently in order to add more assertions to provide test coverage. If you have this requirement, you can pre-populate the database with the following two methods:

• Select a subset of production environment data to run the black box test
• Populate the database with manually constructed data before running the black box test

Unit Test

In addition to the black box test, the unit test must also be written honestly.

The derivation of the API

The derivation of the API is unavoidable. Document changes through documentation and inform developers, such as CHANGELOG, blogs, deprecation schedules, mailing lists, and so on, to gradually discard old APIs.

Design an excellent API

Standing on the shoulders of giants, a lot of understanding and learn from the API design method, is a very good learning and practical methods.

• GitHub API
• Twilio API
• Stripe API
• Digitalocean API

Problems with RESTFul APIs

1. Lack of extensibility. A simple user interface may return only a few pieces of information, such as user name, Avatar, etc. As the API continues to evolve, you may need to return more information, such as age, nickname, Signature, and so on. Most of the time the client just needs some of the information, but the interface still transmits all the information, this situation increases the network transmission, especially for mobile applications are particularly unfriendly, but also require the client to extract the required data. Instead of building two APIs that are roughly the same as the return field, it increases the complexity of the backend implementation, either by adding business logic or by increasing the difficulty of maintenance. Of course, this problem can be solved by using query parameters, but this undoubtedly increases the amount of code redundancy.
2. Complex data requirements require multiple API calls. For example, the client to display the content of the article, you may want to invoke the article interface, comment interface, user information interface. To make a complete view of a resource, there are several separate calls that need to be made, so the data acquisition method is very inflexible.

The future of the API

Maybe it's graphql and Falcor.

Reference

• Best practices for designing a pragmatic RESTful API
• Practices for Writing node. js REST APIs
• Best Practices for API versioning?
• RESTful Service API Design Best Engineering practices and solutions for common problems