Like planting potatoes on Mars? See how others use Docker in Big data analytics

30 years later, when Matt Damon recalled his magnificent life, the most memorable, perhaps not to participate in the Normandy beachhead of the lost battle "save Private Ryan", not in the panic of amnesia in exile "Spy Shadow Heavy", is not frozen in more than 30 years of spring and Autumn Dream "Star Crossing", But the day of the potato on the Red Planet, Mars rescue.

Space station astronauts are said to have eaten the lettuce grown in space. A NASA botanist also said that humans can grow potatoes on Mars, and that more than potatoes can be planted. In the 2014, the Netherlands had a group of curious scientists who had planted 14 crops, including tomatoes, carrots and wheat, on simulated lunar soil and simulated Martian soil.

So, the 15-year-old was hired as a software engineer in the U.S. National Laboratory "space Otaku" Andy Will, even if writing novels, is also very reliable!

After watching "Mars Rescue", this year's expected sci-fi blockbuster left "Star Wars 7".

It is said that "Star Wars 7" in North America's pre-sale of more than $50 million, 70% are 18-49-year-old male fans, the average age of 34 years, 67% people will be in the release of a week to see the fast ...

The above data are issued by a company called Movio, what is Movio?

Movio mainly has two products, Movio cinema and Movio Media,movio Cinema co-operate with major cinemas (already covering 52% of North America's screens, global 24.5%), providing personalized service to cinema customers through big data analytics; Movio Media mainly serves film manufacturers and publishers, using data collected on Movio cinema to provide effective advice to producers and publishers. in short, is a determined to use big data analysis and the Internet to change the film industry company.

And Movio is the heavy user of Docker. Movio to enable its code to grow in complexity while remaining resilient, and to start using microservices architecture and Docker. The following is a technical blog from Movio, which focuses on how it uses Docker in a development environment.

Docker encapsulates the application in a "full" OS environment and runs in a container. Containers are similar to virtual machines, but more like physical machines: for example, memory is not allocated to containers in advance and requires containers to be dynamically applied to the host.

The use of containers means that library dependencies (libraries dependencies) can be installed independently of the host system. Other dependencies, such as configuration files, can be configured with default values in advance and placed in the appropriate location. With Docker, it's the equivalent of every application, with a standard set of environments running in the same way, regardless of the operating system. Libraries and packages installed in the host operating system do not affect the application because they are independent of each other.

The main use of Docker in the development of Movio is:

Run Anywhere

Program apes are each their own for multiple distributions of OS X and Linux, resulting in a non-uniform development environment that is very painful. While the application runs in the container, it means that everyone can rely on the same configuration. And when you need some personalized configuration, you can also set parameters when you start the container.

For example, you can load a volume in a container and store the data generated during the container. For example, you have a database container where you want the data to be saved after the container is stopped or deleted.

To facilitate the implementation of relevant practices, can be tested on some container platforms. For example, www.alauda.cn also provides a persistent storage service that abstracts cloud storage into the most intuitive storage volumes and can be hung directly on containers just as easily as local operations on Docker's storage volumes. On the basis of the storage volume, to further enhance the durability of the data, the cloud also provides "storage volume backup", which can be used to capture and save the storage volume Flash.

Facilitates integration between individual components

If you have a microservices that needs to be integrated with another small group of MicroServices, running Docker pull can download that micro service to your environment without worrying about environmental dependencies.

This process completely replicates the interactions of the various services in the production environment, which also makes us believe that the results seen in the test environment are fully reflected in the production environment.

The cloud supports multi-container release management, compatible with Docker-compose, which describes the application's container components through the Yaml file, as well as the dependencies between containers, and deploys a complete multi-container application with one click.

composable Docker Images

Based on this, we can simply reuse the work done before, we have a series of core base images, but also means that our images are based on the same basis. Also, mirrors can be reused across groups.

Docker images are managed hierarchically, each part is implemented as one layer, and the base layer can be shared, with 2 benefits: reducing the Docker library and shortening the time it takes to download new images.

The cloud provides high-performance local registry services for creating private, public mirrored warehouses and providing a full range of mirroring services for uploading, downloading, building, and hosting

Enhance collaboration in development, testing, and operations

With Docker, developers have a more holistic mindset, and they consciously think about the need to run the application outside of the development environment.

The test environment will be replicated in the production environment and standardized, because Docker mirrors are standardized, and pull down can be run immediately, without the need to configure the document step-by-step.

The challenges that Docker brings

The implementation mechanism of Docker also means that there will be a lot of base mirroring dependencies. For example, we have a clojure-written micro-service that is built on a clojure image, but the clojure image is built on a Java image, and the Java image is based on a common base image. Once the underlying image has changed, all of the intermediate mirrors have to be re-built, which undoubtedly adds additional overhead.

Because Docker is based on the Linux kernel, it is necessary to run a Linux operating system on the OS X system in the virtual machine. The official tool is based on Visualbox, files and folders to mount to the VM first, then mount to the Docker container, although this tool can do this automatically, but the Chown and chmod operations do not work.

File names in OS x are case-insensitive, but most Linux systems are differentiated. If an incorrect name occurs exactly in the Mount path, the path cannot be found.

These are the challenges of using Docker in a local host environment, and if you use the CaaS service, such as a psionic cloud, you can avoid the above problems, focus your energy on the core business, and enjoy the benefits of Docker!

Summarize

There will certainly be a learning curve in the early days of using Docker, but in general, Docker does improve our productivity. In the past, it takes several days for new hires to be able to configure their local environment and to be plagued by environmental problems. With Docker, the production environment's reliance on the local environment becomes small. As we develop, this increase in efficiency becomes more and more evident.

Like planting potatoes on Mars? See how others use Docker in Big data analytics