Kinto Introduction to open source lightweight JSON storage

This article is from my translation of the Infoq Chinese station, the original address is: Http://www.infoq.com/cn/news/2015/12/introducing-kinto

Kinto is a lightweight JSON storage service that provides synchronization and sharding capabilities, and is easy to use.

It has been widely used within Mozilla, based on the Apache V2 license.

Basically, Kinto is a service that the client application can use to store and retrieve JSON data. To provide synchronization and sharing of these data, Kinto introduces some basic concepts.

• Bucket: The bucket can be considered a namespace: it is assumed to be stored in a different bucket. There is no conflict between the collection names.
• Collection: The collection of records.
• Record: The data that is actually stored.
• Group: A named principal list that defines permissions.

A record is the smallest unit of data. By default, it is modeless. JSON can include no matter what the data. Collection is a set of record. We manipulate the record in the form of a list. Ability to filter and sort them. Buckets are an abstract concept for organizing collections and their permissions.

Kinto provides a bucket named default, whose collection and record can only be visited by the current user.

Each object manipulated by Kinto shares some common properties:

• A unique identifier
• A version that will increment when changed
• A set of permissions

These concepts are very similar to disks, where buckets are partitions, collection are folders, and the record is a file.

Kinto has the concept of a user group. A group will have a list of members, and they belong to a bucket.

Permissions refer to groups rather than individual users, which allows us to easily define "roles", especially when applying the same set of permissions to several objects.

To control the viewing, creation, alteration, and deletion of records. Kinto can define permissions on buckets, Group, collection, and a single record. Since there is a concept of succession between buckets, collection and record. Therefore, Kinto takes into account the issue of inheriting permissions from the parent object. Say. Suppose a bucket defines a permission to allow anonymous users to read. Then each record of the collection in the bucket can be read. This is the result of inheritance. The permission definition for creating a new object is in the parent object. For example, the permission to create a record is defined in collection. Create collection or group permissions defined in buckets.

Only the permissions to create a new bucket are controlled by the server configuration.

The main application scenarios for Kinto are as follows:

• Provides a common web database for front-end applications
• Build collaborative applications with granular permissions control
• can store encrypted data
• Synchronize application data between devices

It is worth mentioning that Kinto now has been widely used in Firefox and Firefox OS. It is primarily used for global synchronization of settings with various information, and it is also the preferred solution for browser extensions and personal data storage in Web applications.

The main features of the Kinto are as follows:

• Data synchronization
• Fine-grained permission control
• JSON Schema validation
• Universal and multi-client support
• Open source, self-hosted
• A good design

Highlights of Kinto are:

• Take full advantage of HTTP best practices
• Pluggable authentication mechanism
• Pluggable storage, caching, and permissions support
• Configurable via INI file or environment variable
• Built-in monitoring support
• Cache control

Two-way record synchronization is a very difficult thing to do.

Kinto took some short cuts, which provided some basic safeguards for concurrency control and change polling only. And will not take the initiative to solve the conflict problem. Basically, each object has a version that is added after each change. Kinto does not save the old version of the object.

The client is able to obtain a list of changes that have occurred since a particular version of the record collection, and Kinto can also prevent accidental updates of objects by it.

The Kinto Sync feature was designed and developed by the Mozilla Firefox Sync team.

Before you start building your own data storage service. Let's take a look at the products that already exist on the market.

The Kinto team started with the idea of using and expanding existing community projects. Did not intend to invent the wheel again. Just, after a few attempts. The team found that both of these solutions did not solve the problems that the team faced very well. Let's say fine-grained permission control, etc.

As a result, the team started building their own data storage services based on the experience gained from developing Firefox sync.

The table below shows the comparative features of Kinto with similar products on the market, which helps readers understand the advantages of Kinto compared to other competing items.

[1] At the moment, hoodie's user plugin will proactively let users authenticate through, but they are solving the problem.

[2] Couchdb will map/reduce as a query mechanism, which is not very good for the newcomers to understand.

[3] Remote storage is able to use "LS" for a folder, except that the content items are not well sequenced. There are no pagination.

[4] for conflict handling, Kinto uses the same mechanism as the remote Storage.

[5] Remote storage supports OAuth2.0 implicit Grant Flow.

[6] The discovery of the go-to-center was supported in the plan, just not yet realized.

[7] The Remote storage does not define any default implementation (as it is a protocol), but it can be easily started with JavaScript and node. js.

Interested readers can also learn about the choices and motivations behind creating kinto.

Here are some FAQs on Kinto that will help readers better understand the design and usage scenarios of this lightweight JSON storage.

Can the Kinto encrypt the data?

Kintoserver will store whatever data is passed over, regardless of whether the data is encrypted or not. We think encryption should be a client-done thing. You can easily encrypt your data with the kinto.jsclient we provide.

Does Kinto provide a package download tool for the operating system I'm using?

No, just this idea is very good. Maintaining packages for several platforms is a very time-consuming task. Besides, we're still a small team now. For now, you can easily execute the server through the makefile or dockerfile we provide. Ability to easily install Kinto with PIP. There is also a mirror in the Docker hub. Just, let's say you're willing to help us maintain the Kinto package on your favorite operating system. Then we will be extremely grateful!

Why would I choose to use Python instead of other languages?

We like Python. Because it is a simple and expressive language, it provides powerful data structures at the same time and is easy to learn. As a result, the development team chose to use Python.

In addition Mozilla's OPS team is also familiar with deploying and managing Python applications in a production environment.

Just, Python itself is just an implementation detail. The kinto is defined by the HTTP protocol. Can be implemented in whatever language you are using.

Can I store files in Kinto?

Not now. But we have designed a file storage feature that is now being implemented and is expected to be announced next year. At the same time, suppose you want to try the latest version. We are looking forward to your feedback.

What is Cliquet? What are the differences between Cliquet and Kinto?

Cliquet is a toolset for designing microservices. Kinto is a server built from this toolset.

I encountered a exception error. Where did it go wrong?

The ability to learn specific information about errors through troubleshooting.

Interested readers can try Kinto, here is Kinto's documentation.

Kinto Introduction to open source lightweight JSON storage