Greenplum installation Experiment

Document directory

• 1.3.1 modify the/etc/sysctl. conf file
• 1.3.2 modify the/etc/security/limits. conf file
• 1.3.4 modify Io algorithm/boot/GRUB/menu. lst
• 1.3.5 configure the NTP service
• 1.3.6 install the XFS File System

Chapter 2 Configuration Requirements and Environment Settings

 

1.1 Linux operating system parameters

Modify the hosts files of each host separately (the three hosts are identical)

VI/etc/hosts

127.0.0.1 localhost

 

192.168.10.177 MDW-ext

192.7.1.small mdw-1 MDW

192.7.2.20.mdw-2

192.7.3.small mdw-3

192.7.4.small mdw-4

 

192.7.1.1 sdw1-1 sdw1

192.7.2.1 sdw1-2

192.7.3.1 sdw1-3

192.7.4.1 sdw1-4

 

192.7.1.2 sdw2-1 sdw2

192.7.2.2 sdw2-2

192.7.3.2 sdw2-3

192.7.4.2 sdw2-4

1.2 Network Configuration

Make sure that the NIC of each host is correctly configured.

/Etc/sysconfig/network SuSE

/Etc/sysconfig/network-scripts RedHat

Each network adapter corresponds to the file ifcfg_eth [,]

	Master		Segment1		Segment2
Eth0	192.7.1.20.	Mdw-1 (MDW)	192.7.1.1	Sdw1-1 (SDW)	192.7.1.2	Sdw2-1 sdw2.
Eth1	192.7.2.6.2	Mdw-2	192.7.2.1	Sdw1-2	192.7.2.2	Sdw2-2
Eth2	192.7.3.small	Mdw-3	192.7.3.1	Sdw1-3	192.7.3.2	Sdw2-3
Eth3	192.7.4.small	Mdw-4	192.7.4.1	Sdw1-4	192.7.4.2	Sdw2-4
EXT	192.168.10.177

 

 

1.3 Linux operating system-level parameter 1.3.1 modify the/etc/sysctl. conf file

The following parameters in the greenplum master and segment host/etc/sysctl. conf:

Kernel. SEM = 250 64000100 512

Kernel. shmmax = 500000000

Kernel. shmmni = 4096

Kernel. Shmall = 4000000000

Kernel. SEM = 250 64000100 512

Kernel. sysrq = 1

Kernel. core_uses_pid = 1

Kernel. msgmnb = 65536

Kernel. msgmax = 65536

Net. ipv4.tcp _ syncookies = 1

Net. ipv4.ip _ forward = 0

Net. ipv4.conf. Default. accept_source_route = 0

Net. ipv4.tcp _ tw_recycle = 1

Net. ipv4.tcp _ max_syn_backlog = 4096

Net. ipv4.conf. All. arp_filter = 1

Net. ipv4.conf. Default. arp_filter = 1

Net. Core. netdev_max_backlog = 10000

VM. overcommit_memory = 2

1.3.2 modify the/etc/security/limits. conf file

The following parameters in greenplum master and segment host/etc/security/limits. conf:

* Soft nofile 65536

* Hard nofile 65536

* Soft nproc 131072

* Hard nproc 131072

 

1.3.3 set disk pre-Read/etc/init. d/after. Local (RedHat:/etc/rc. Local)

Dell

Place blockdev -- setra 16384/dev/SD * in the configuration file *

Restart the server and check blockdev -- getra/dev/SD *

HP

In the configuration file, place blockdev -- setra 16384/dev/CCISS/C? D? *

Restart the server and check blockdev -- getra/dev/CCISS/C? D? *

1.3.4 modify Io algorithm/boot/GRUB/menu. lst

Add elevator = deadline at the end

After the restart, run CAT/sys/block/*/queue/schedue. The returned content contains

[Deadline]

1.3.5 configure the NTP service

 

1.3.6 install the XFS File System

On the master node host, format the/dev/SDB disk into an XFS file and mount it to/data.

On the segment node host, format the/dev/SDB disk into an XFS file and mount it to/data1.

On the segment node host, format the/dev/SDC disk into an XFS file and mount it to/data2.

 

View the disk file system and usage DF-th

Master Node

Format the disk as an XFS File System

Mkfs-t xfs/dev/SDB

Mkdir-P/data/Master

Add the mount option to the/etc/fstab file.

/Dev/sdb/Data XFS logbufs = 8, noatime, nodiratime 1 1

Segment Node

Mkfs-t xfs/dev/SDC

Mkdir-P/data1/primary

Mkdir-P/data2/Mirror

Add the mount option to the/etc/fstab file.

/Dev/sdb/data1 XFS logbufs = 8, noatime, nodiratime 1 1

/Dev/SDC/data2 XFS logbufs = 8, noatime, nodiratime 1 1

1.4 users and their home directories

	User	Home Directory	Installation Directory	Data Directory
Master	Gpadmin	/Home/gpadmin	/Usr/local/greenplum-DB	/Data/Master
Segment1	Gpadmin	/Home/gpadmin	/Usr/local/greenplum-DB	/Data1/primary
Segment2	Gpadmin	/Home/gpadmin	/Usr/local/greenplum-DB	/Data1/primary

Create a gpadmin user and data directory on the master node

Groupadd gpadmin

Useradd-G gpadmin-D/home/gpadmin

Mkdir/data/Master

Chown-r gpadmin: gpadmin/data/Master

Modify the/home/gpadmin/. bash_profile File

. Bash_profile Linux System

. Bashrc SuSE System

Add the following content:

Source/usr/local/greenplum-DB/greenplum_path.sh

Export master_data_directory =/data/master/gpseg-1

Create a gpadmin user and data directory on all segment nodes

Method 1: Create gpadmin users and data directories on each segment node.

SSH sdw1

Groupadd gpadmin

Useradd-G gpadmin-D/home/gpadmin

Passwd gpadmin

Mkdir/data1/primary

Chown-r gpadmin: gpadmin/data1/primary

 

SSH sdw2

Groupadd gpadmin

Useradd-G gpadmin-D/home/gpadmin

Passwd gpadmin

Mkdir/data1/primary

Chown-r gpadmin: gpadmin/data1/primary

 

Method 2: run the GP command to install the SDK.

1. Install the GP database software on the master before using the GP command.

2. Create root user equivalence

The file all_hosts contains information about the NIC of all hosts.

VI all_hosts

Mdw-1

Mdw-2

Mdw-3

Mdw-4

Sdw1-1

Sdw1-2

Sdw1-3

Sdw1-4

Sdw2-1

Sdw2-2

Sdw2-3

Sdw2-4

Execution Environment Variable

Source/usr/local/greenplum-DB/greenplum_path.sh

Exchange key

Gpssh-exkeys-F all_hosts

Equivalence Verification

Gpssh-F all_hosts

3. Use gpssh for unified execution on all nodes

Create the file all_segs, which only contains the host names of each node.

VI all_segs

Sdw1

Sdw2

Gpssh-F all_segs

Groupadd gpadmin

Useradd-G gpadmin-D/home/gpadmin

Passwd gpadmin

Mkdir/data1/primary

Chown-r gpadmin: gpadmin/data1/primary

4. Create a gpadmin user equivalent.

Run the following command on the master node:

Su-gpadmin

Exchange key

Gpssh-exkeys-F all_hosts

Equivalence Verification

Gpssh-F all_hosts

Chapter 2 system verification 2nd operating system verification

Create a file VI hostfile_gpcheck

MDW

Sdw1

Sdw2

Root User execution

Source/usr/local/greenplum-DB/greenplum_path.sh

Gpcheck-f/home/gpadmin/gpconfigs/hostfile_gpcheck-m MDW

 

2.2 verify system performance

 

• Network Performance (gpnetmask *)

Test each network segment by Subnet

VI hostfile_gpchecknet

Mdw-1

Sdw1-1

Sdw2-1

Gpcheckperf-F hostfile_gpchecknet-R n-d/tmp

 

• Disk I/O performance (DD test)

• Memory bandwidth (stream test)

Create a file VI hostfile_gpcheckperf

Sdw1

Sdw2

$ Gpcheckperf-FHostfile_gpcheckperf-RDS-d-D/data1/primary-D/data2/primary \

-D/data1/mirror-D/data2/Mirror

 

Chapter 2 Linux environment Installation Guide 3rd install database software on the master node

Root User decompress the installation package and execute Installation

Unzip greenplum-db-4.1.1.3-build-4-RHEL5-x86_64

/Greenplum-db-4.1.1.3-build-4-RHEL5-x86_64.bin

 

The default installation path is/usr/local/greenplum-DB.

3.2 install database software on all segment nodes

# Gpseginstall-F all_segs

3.3 initialize the GP Database

Find the Initialization Configuration template (gpinitsystem_config) in the installation directory (/usr/local/greenplum-DB/docs/cli_help/gpconfigs). Modify the parameters in the template as follows:

Array_name = "EMC greenplum DW"

Seg_prefix = gpseg

Port_base = 40000

Declare-A data_directory = (/data1/primary/data1/primary/data1/primary/data1/primary)

Master_hostname = MDW

Master_directory =/data/Master

Master_port = 5432

Trusted shell = SSH

Check_point_segment = 8

Encoding = Unicode

# Pai_port_base = 50000 # optional

# Replication_port_base = 41000 # optional

# Pai_replication_port_base = 51000 # optional

# Declare-A pai_data_directory = (/data2/mirror/data2/mirror/data2/mirror) # optional

Machine_list_file =/home/gpadmin/gpconfigs/all_segs

The gpadmin user executes the following command to initialize the database:

Gpinitsystem-C gpinitsystem_config

3.4 modify user authentication and concurrency restrictions

VI/data/master/gpseg-1/pg_mirror.conf

Add the following content to allow any network segment to connect to any database through password verification by any user (or change it to another authentication method based on security requirements)

Host All all 0.0.0.0/0 MD5

 

VI/data/master/gpseg-1/PostgreSQL. conf

Master Node

Max_connections = 100

Max_prepared_transactions = 100

Segment Node

Max_connections = 500

Max_prepared_transactions = 100

 

Restart Database

Chapter 2 Performance Monitor installation 4th install performancemonitor database and activate performancemonitor agents

Run the following command under the gpadmin user and restart the database:

$ Su-gpadmin

$ Gpperfmon_install -- enable -- password P @ $ word -- Port 5432

$ Gpstop-R

Check whether the gpperfmon database is created normally

$ PS-Ef | grep gpmmon

$ Psql gpperfmon-C 'select * fromsystem_now ;'

4.2 install performancemonitor Console

Unzip greenplum-perfmon-web-4.1.1.3-build-4-RHEL5-x86_64.zip

/Greenplum-perfmon-web-4.1.1.3-build-4-RHEL5-x86_64.bin

The default installation path is/usr/local/greenplum-perfmon-web-4.1.1.4-build-4

# Chown-r gpadmin: gpadmin/usr/local/greenplum-perfmon-web-4.1.1.4-build-4

Add environment variables to/home/gpadmin/. bashrc

Source/usr/local/greenplum-perfmon-web-4.1.1.4-build-4/gpperfmon_path.sh

Run the following command to configure the console as prompted:

$ Gpperfmon-Setup

$ Gpperfmon -- start ["Instancename "]

$ Gpperfmon -- stop ["Instancename "]

$ Gpperfmon -- restart ["Instancename "]

$ Gpperfmon -- status ["Instancename "]

Access website

Https: // localhost: 28080/