Use of environment variables when DEA starts an application instance in cloud Foundry

In cloud foundry V2, when an application user needs to start an application instance, the user sends a request to the Cloud Controller through cf cli, And the Cloud Controller forwards the request to the DEA through NATs. The main task of running the startup is DEA. DEA is to start a warden container, copy the content such as droplet into the container, and finally configure the specified environment variable, start the application startup script under these environment variables.

This article will explain how DEA configures environment variables for application instance startup in cloud foundry.

DEA receives application startup requests and Its Execution Process

In this section, the execution process of the DEA application startup request is described in the form of code.

1. First, the DEA subscribe to the message of the corresponding topic. The topic is "Dea. # {Bootstrap. UUID}. Start", meaning "application startup message of its own DEA ":

         subscribe("dea.#{bootstrap.uuid}.start") do |message|        bootstrap.handle_dea_directed_start(message)      end

2. After receiving the subscription topic, run Bootstrap. handle_dea_directed_start (Message), which means "processing application startup requests through Bootstrap class instances ":

       def handle_dea_directed_start(message)      start_app(message.data)    end

3. The entry for processing can be considered as the start_app method in the above Code:

       def start_app(data)      instance = instance_manager.create_instance(data)      return unless instance      instance.start    end

4. In the start_app method, first create an instance object through the instance_manager class instance. by executing the instance class method start, you can see that, the original source of the passed parameter is the message transmitted through the NATs message, that is, the message in 1:

       def start(&callback)      p = Promise.new do        ……        [          promise_droplet,          promise_container        ].each(&:run).each(&:resolve)        [          promise_extract_droplet,          promise_exec_hook_script('before_start'),          promise_start        ].each(&:resolve)        ……        p.deliver      end

5. The actual execution of application startup is implemented in the promise_start method:

       def promise_start      Promise.new do |p|        env = Env.new(StartMessage.new(@raw_attributes), self)        if staged_info          command = start_command || staged_info['start_command']          unless command            p.fail(MissingStartCommand.new)            next          end          start_script =            Dea::StartupScriptGenerator.new(              command,              env.exported_user_environment_variables,              env.exported_system_environment_variables            ).generate        else          start_script = env.exported_environment_variables + "./startup;\nexit"        end        response = container.spawn(start_script,                                   container.resource_limits(self.file_descriptor_limit, NPROC_LIMIT))        attributes['warden_job_id'] = response.job_id        container.update_path_and_ip        bootstrap.snapshot.save        p.deliver      end    end

In step 1, we can see that DEA involves the ENV Environment Variables and other information of the application. Finally, the container. Spawn method is used to start the application.

Configure the DEA environment variable

In step 1 of the preceding steps, the environment variable Env = env. New (startmessage. New (@ raw_attributes), Self) is created. The env class initialization method is as follows:

    def initialize(message, instance_or_staging_task=nil)      @strategy_env = if message.is_a? StagingMessage        StagingEnv.new(message, instance_or_staging_task)      else        RunningEnv.new(message, instance_or_staging_task)      end    end

It can be seen that an instance of the runningenv class is created, and the parameter is mainly sent by the Cloud Controller.

In the promise_start method, after the env variable is created, the start_script variable construction is selected by judging staged_info.

Analyze the code implementation of staged_info:

    def staged_info      @staged_info ||= begin        Dir.mktmpdir do |destination_dir|          staging_file_name = 'staging_info.yml'          copied_file_name = "#{destination_dir}/#{staging_file_name}"          copy_out_request("/home/vcap/#{staging_file_name}", destination_dir)          YAML.load_file(copied_file_name) if File.exists?(copied_file_name)        end      end    end

It is mainly to specify the path and then extract the variable from the path. The following uses a ruby application as an example. The content of the staging_info.yml file is:

---detected_buildpack: Ruby/Rackstart_command: bundle exec rackup config.ru -p $PORT

Therefore, the final content of @ staged_info is as above. In the instance. Start method, the command is bundle exec rackup config.ru-p $ port. With the command variable, build the start_script variable:

          start_script =            Dea::StartupScriptGenerator.new(              command,              env.exported_user_environment_variables,              env.exported_system_environment_variables            ).generate

As you can see, the DEA creates start_script through the startupscriptgenerator class. The parameters are three, the first is the involved command, and the last two are generated using the env variable.

Now let's look at the implementation of the exported_user_environment_variables method:

    def exported_user_environment_variables      to_export(translate_env(message.env))    end

This method extracts the information whose attribute is env from the message as the user's environmental variable.

Enter the Env. exported_system_environment_variables method implementation:

    def exported_system_environment_variables      env = [        ["VCAP_APPLICATION",  Yajl::Encoder.encode(vcap_application)],        ["VCAP_SERVICES",     Yajl::Encoder.encode(vcap_services)],        ["MEMORY_LIMIT", "#{message.mem_limit}m"]      ]      env << ["DATABASE_URL", DatabaseUriGenerator.new(message.services).database_uri] if message.services.any?      to_export(env + strategy_env.exported_system_environment_variables)    end

It can be seen that when the system environment variable is generated, an env array variable is first created, including vcap_application, vcap_services, and memory_limit. The vcap_application information is as follows:

    def vcap_application      @vcap_application ||=        begin          hash = strategy_env.vcap_application          hash["limits"] = message.limits          hash["application_version"] = message.version          hash["application_name"] = message.name          hash["application_uris"] = message.uris          # Translate keys for backwards compatibility          hash["version"] = hash["application_version"]          hash["name"] = hash["application_name"]          hash["uris"] = hash["application_uris"]          hash["users"] = hash["application_users"]          hash        end    end

 
This part of information contains the application name, Uris, users, version, and a series of other content. Note that the Code hash = strategy_env.vcap_application is used to call the vcap_application method in the runningenv class, as follows:

   def vcap_application      hash = {}      hash["instance_id"] = instance.attributes["instance_id"]      hash["instance_index"] = message.index      hash["host"] = HOSTStartupScriptGenerator      hash["port"] = instance.instance_container_port      started_at = Time.at(instance.state_starting_timestamp)      hash["started_at"] = started_at      hash["started_at_timestamp"] = started_at.to_i      hash["start"] = hash["started_at"]      hash["state_timestamp"] = hash["started_at_timestamp"]      hash    end

The above code records a lot of information about the application instance, including instance_id, instance_index, host, port, started_at, started_at_timestamp, start, state_timestamp, and so on.

The vcap_services information is as follows:

     WHITELIST_SERVICE_KEYS = %W[name label tags plan plan_option credentials syslog_drain_url].freeze    def vcap_services      @vcap_services ||=        begin          services_hash = Hash.new { |h, k| h[k] = [] }          message.services.each do |service|            service_hash = {}            WHITELIST_SERVICE_KEYS.each do |key|              service_hash[key] = service[key] if service[key]            end            services_hash[service["label"]] << service_hash          end          services_hash        end    end

This part mainly looks for the existence of the message and the value of whitelist_services_keys as the key value. If so, add the services_hash variable.

Then, DEA runs the code env <["database_url", databaseurigenerator. New (message. Services). database_uri] If message. Services. Any ?, The role of this part of code is mainly to understand the meaning of databaseurigenerator, which is not very clear to the author.

Later, the DEA executes the code to_export (ENV + strategy_env.exported_system_environment_variables). This part of content is very important, mainly to go to the class where the strategy_env object is located to view the exported_system_envi:

    def exported_system_environment_variables      env = [        ["HOME", "$PWD/app"],        ["TMPDIR", "$PWD/tmp"],        ["VCAP_APP_HOST", HOST],        ["VCAP_APP_PORT", instance.instance_container_port],      ]      env << ["PORT", "$VCAP_APP_PORT"]      env    end

As you can see, this mainly contains the environment variables for running information, such as the home directory, TMP temporary directory, host address for running the application instance, and port number for running the application instance. The most important part is the port number of the application instance. In my other blog post cloud foundry, the DEA and warden communication completes the application port listening, it involves how to open a port through the warden server, and finally use the DEA, and pass the startup script to the application instance in the form of environment variables. Both vcap_app_port and port are the port number enabled by Warden container.

After analyzing the three parameters of startupscriptgenerator, you must enter the generate method of the startupscriptgenerator class:

    def generate      script = []      script << "umask 077"      script << @system_envs      script << EXPORT_BUILDPACK_ENV_VARIABLES_SCRIPT      script << @user_envs      script << "env > logs/env.log"      script << START_SCRIPT % @start_command      script.join("\n")    end

The above code is the process of creating the startup script. Here, @ system_envs is the previously analyzed env. exported_system_environment_variables, and @ user_envs is env. exported_user_environment_variables. There are two other scripts: export_buildpack_env_variables_script and start_script. The code for the export_buildpack_env_variables_script script is as follows, which means to execute all sh scripts in a certain path:

    EXPORT_BUILDPACK_ENV_VARIABLES_SCRIPT = strip_heredoc(<<-BASH).freeze      unset GEM_PATH      if [ -d app/.profile.d ]; then        for i in app/.profile.d/*.sh; do          if [ -r $i ]; then            . $i          fi        done        unset i      fi    BASH

The start_script code is as follows:

    START_SCRIPT = strip_heredoc(<<-BASH).freeze      DROPLET_BASE_DIR=$PWD      cd app      (%s) > >(tee $DROPLET_BASE_DIR/logs/stdout.log) 2> >(tee $DROPLET_BASE_DIR/logs/stderr.log >&2) &      STARTED=$!      echo "$STARTED" >> $DROPLET_BASE_DIR/run.pid      wait $STARTED    BASH

After the start_script is created, return to the promise_start method in instance. Rb and execute

response = container.spawn(start_script,                                   container.resource_limits(self.file_descriptor_limit, NPROC_LIMIT))

That is, the application instance is started. For details, go to the spawn method implementation in the iner class.

Indicate the source for reprinting.

This document is more out of my understanding and is certainly flawed and wrong in some places. I hope this article will be helpful to anyone who has access to dea_ng in cloud foundry V2 to start an application instance. If you are interested in this and have better ideas and suggestions, please contact me.

My mailbox: [email protected]
Sina Weibo: @ lianzifu ruqing

Use of environment variables when DEA starts an application instance in cloud Foundry