linux commands for hadoop administration

=============== installation and Landing commands =============================================================Mount: Mount command. Designate storage media as a directory in the system (access resources in other file systems under running Linux), such as Mount Optical drives Mount/dev/cdromMount the CDROM to view content under directory/mnt/cdrom.Umount: Uninstall command, such as Uomunt/dev/cdrom.Exit: Ex

The Linux command line attracts most Linux enthusiasts. A normal Linux user typically has about 50-60 commands to handle daily tasks. Linux commands and their transformations are the most valuable treasures for

Linux's IP commands are similar to Ifconfig, but the former is more powerful and intended to replace the latter. With the IP command, you can easily perform some network administration tasks with just one command. Ifconfig is a command that has been deprecated in net-tools and has not been maintained for many years. Many of the enhanced commands are available in

hdpdata92m hivedata66m journaldata344m logs303m Spark-1.6.2-bin-hadoop2.627M sqoop260m zkdata[root@hadoop01 ~]# Du-sh *|grep [G] [filter out directories and files of GB-size]3. 2G Apps2. 2G Soft[root@hadoop01 ~]# Du-sh *|grep [MG] [filter out directories and files of size MB and GB]3. 2G apps82m flume150m hdpdata92m hivedata66m journaldata344m logs2. 2G soft303m Spark-1.6.2-bin-hadoop2.627M sqoop260m zkdata[root@hadoop01 ~]# du-sh *|grep [MG]|SORT-NR [filter out directories and files of size MB

directory]# find/media-name ' *java* '[Decompression tar.gz]# tar ZXPF vmwaretools-9.9.0-2304977.tar.gz[Copy directory 1 to directory 2]# cp–r {Directory 1} {directory 2}[Create a directory]# Mkdir/usr/java[Copy file]# Cp/tmp/jdk-6u45-linux-x64.bin/usr/java[Move file]# mv xxx xxx[Rename folder]# MV hadoop-1.2.1 Hadoop[Toggle Current Working Directory]# cd XXX[Ch

to the ASCIl code.Conv = ebcdic converts an ASCIl code to an EBCDIC code.Conv = IBM converts an ASCIl code to an alternate EBCDIC code.Conv = block converts a variable bit to a fixed character.Conv = ublock converts a fixed bit to a variable bit.Conv = ucase converts lowercase letters to uppercase letters.Conv = lcase converts uppercase letters to lowercase letters.Conv = notrunc: the output file is not truncated.Conv = swab exchange each pair of input bytes.Conv = do not stop processing when n

1. Sudo-sEnter root user rights mode2. VimModify file I into edit mode ESC +: wq! Exit Edit, Save exit3 tailView Files4. sudo passwd rootSet the password for the root account5. Reboot-h NowRebooting the system6. TAR-XVFExtract7. Sudo gedit/etc/profileNotepad-like editing8. HostnameCurrent machine Name9. Ifconfig-aInformation such as current machine IPTen. Df-laView disk usageDu-sh * View the memory size of each folder under the current path11.rz-ary--o-sync12.

successfully.Add a user to the user group and use "-G" to indicate which user group the user belongs toSuch as:#useradd-G Hadoop HaduserThen cat/etc/passwd to view the new user.3. Login ssh, upload files in ssh, from one host to another, these hosts are on the same LAN.For example: The current host IP is 10.0.1.7, another host IP with this host on the same LAN is 10.0.1.8After landing ssh, to transfer a file from the 7 host to another 8 host, with t

for a password,2. Enable the Sudoers profile using alias modeVim/etc/sudoersUser_alias useradmin = hadoop,%hadoop,%useradmin #表示hadoop用户, Hadoop Group, useradmin GroupCmnd_alias useradmincmnd =/usr/sbin/useradd,/usr/sbin/usermod,/user/bin/passwd [A-Za-z]*,! /USR/BIN/PASSWD root #命令别名, which means that useradd usermod/

Linux common commands (man Histroy)1.linux Man Manual:Usage:# Man command query command HelpEg:man ls650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/5A/22/wKioL1T3yGuy-Q5IAAGgYSihLvY181.jpg "title=" man Ls.jpg "alt=" Wkiol1t3yguy-q5iaaggysihlvy181.jpg "/>Man Manual Chapter1 User Commands (Users command)2 sys

using administrator20. View Port Usagenetstat -tln | grep 8080Viewing usage of port 808021. See which program the port belongs tolsof -i :808022. View the processps aux|grep javaView Java processesps auxView All Processes23. List the contents of the directory in a tree viewtree aPs:mac using the tree command24. File downloadwget http://file.tgzInstall wget command under Maccurl http://file.tgz25. Network detectionping www.taobao.com26. Telnetssh [emailprotected]27. Printing informationecho $JAV

remove the parameter Rum-parts, you can write a script name to run, not the folder name.* * * * * Root run-parts/etc/cron.hourly4 * * * Root run-parts/etc/cron.daily4 * * 0 root run-parts/etc/cron.weekly4 1 * * Root run-parts/etc/cron.monthly[[email protected] etc]# ls cron*Cron.deny crontabCRON.D:0hourly SysstatCron.daily:Logrotate Makewhatis.cron Mlocate.cron prelink Readahead.cron tmpwatchCron.hourly:0anacron Mcelog.cronCron.monthly:Readahead-monthly.cronCron.weekly:99-raid-check[[email prot

.lessPaging through files, using when files are too long/keyword search keywordsQ exitV gedit = GNOME EDIT650) this.width=650; "src=" Https://s3.51cto.com/wyfs02/M02/9C/5F/wKiom1lvaH6zMvzzAAFa6kKXAFo535.png "title=" Screenshot from 2017-07-19 22-10-03.png "alt=" Wkiom1lvah6zmvzzaafa6kkxafo535.png "/> 13.gedit = text editor for GNOME edit graphics NBSP gedit file Open File and edit ctrl+s Save ctrl+n Open a new editing interface ctrl+o Opening a new file 14.rm = remove Delete

In practice, it is sometimes necessary to remotely start programs on other Linux hosts from the Web management interface, which can be easily met by SSH protocol. In fact, in the Hadoop architecture, this principle is used when the DN is started on the NN. GANYMED-SSH2 is an open source project that implements the SSH protocol, the project address is: http://ganymed-ssh-2.googlecode.com/(download source to

Linux commands involved in setting up the Spark Environment Linux commands involved in setting up the Spark environment (simple and easy to use) From one server responsible for content to another server: scp jdk-6u37-linux-x64.bin spark@10.126.45.56:/home/spark/opt Tar compr

, -pcpu | head 20’View system Open File LsofLsof (list open files) is a tool that lists open files for the current system.In a Linux environment, everything is in the form of files, with files that not only access regular data, but also access to network connectivity and hardware. So, such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) sockets, the system assigns a file descriptor to the application in the background, regardle

: The actual time, from the command line start execution to the elapsed time of the end of the run; note: The sum of CPU time and system CPU time is CPU time, that is, the total amount of time the command consumes CPU execution. The actual time is greater than the CPU time, because Linux is a multitasking operating system, often when executing a command, the system also handles other tasks.Another problem to be aware of is that even though the same co

protected] ~]#Role: To switch other users to root, enter a password, it is recommended that the "-" band, you can bring the root environment variables and working directory at the same time, otherwise, due to the problem of environment variables error.2. System Management Commands (PS and kill)PS: Displays a list of processes that the user is running on the current system.Kill: outputs a specific signal to the specified PID (process number) and behav

Linux Common commands1, view current yum source Yum list2, view information about your computer and the operating system uname-a3, view current version information Cat/proc/version4, view release information Cat/etc/Issue5, and view the currently running processes. PS [options] See documentation6, hostname temporary entry into force hostname Hadoop-dn-017, Bash command now the Shell interface, reload the sy