Crontab problem summary

If you don't pay attention to crontab, there will be problems sooner or later. Once this problem occurs, you will always remember it, because it is a tough problem.
-- A predecessor

A crontab is set.
30 0 * CD/home/work/user/huangbx/research/getfeature/data/current; SH resample. Sh &>/dev/null
$ Sh resample. Sh can be run.
$ Head-5 resample. Sh
# Sampling fact data
Set-x
G_date = 'date-d "3 days ago" + % Y % m % d'
However, you cannot run it in crontab.
I learned from the Internet that crontab cannot run normally because environment variables are not necessarily identifiable in crontab. However, the resample. Sh does not involve the use of environment variables.
After several attempts, I finally found that Code In the first line of the Chinese comments caused by the problem, add #! /Bin/sh and then you can run it.
Summary:
In crontab, you must pay attention to the use of environment variables.
#! /Bin/sh is not required, but when there is no sha-bang, do not add a Chinese comment after "#" in the first line !!
It is best to add sha-bang #! /Bin/sh
Supplement:
I didn't pay special attention to the crontab failure caused by environment variables. Today I met it.
Problem description: cron executes multiple operations in a certain sh file, which calls both external shell scripts and external Python scripts. From the running log, some scripts are called, while some Python scripts are not called. Python scripts are not called, and mysqldb module (third-party module) is used ).
This script can be executed normally by directly using the sh command.
Error message:
Traceback (most recent call last ):
File "areafile. py", line 2, in <module>
Import mysqldb
File "build/bdist. linux-x86_64/egg/mysqldb/_ init _. py", line 19, in <module>
File "build/bdist. linux-x86_64/egg/_ mysql. py", line 7, in <module>
File "build/bdist. linux-x86_64/egg/_ mysql. py", line 6, in _ Bootstrap __
Importerror: libmysqlclient. so.15: cannot open shared object file: no such file or directory
Mysqldb needs to call the MySQL database, but the system does not know where your MySQL is installed :(
Solution:
Add a sentence to the shell script of the general control.
Export LD_LIBRARY_PATH =/home/work/local/mysql5/lib/MySQL
(That is, from ~ /. After the LD_LIBRARY_PATH field in bash_profile) Program Finally, it can be started normally in crontab.
Explanation:
1 )~ /. Bash_profile &&~ /. Bashrc
"/Etc/profile ","~ /. Bash_profile "and other configuration files will be automatically executed. The execution process is as follows: when logging on to the Linux system, first start "/etc/profile", and then start "~ /. Bash_profile ", if "~ /. Bash_login "and "~ /. Profile "the file will also be executed when it exists "~ /. Bash_profile "is called in sequence.
Next let's take a look "~ /. What is in the bash_profile file?
$ Cat ~ /. Bash_profile
#. Bash_profile
# Get the aliases and functions
If [-f ~ /. Bashrc]; then
.~ /. Bashrc
Fi
# User specific environment and startup programs
Path = $ path: $ home/bin:/home/work/local/Python/lib/python2.5/Site-packages/Django/bin/: $ home/bin: /home/work/local/mysql5/bin /;
LD_LIBRARY_PATH =/home/work/local/mysql5/lib/MySQL
Alias py = '/home/work/local/Python/bin/python'
Export path LD_LIBRARY_PATH
Unset Username
You can see ~ /. Bash_profile file is called first ~ /. Bashrc, and then load path and LD_LIBRARY_PATH.
. Bash_profile and. bashrc
/Etc/profile: This file sets the environment information for each user in the system. When the user logs on for the first time, this file is executed.
And collect shell settings from the setting file in the/etc/profile. d directory.
/Etc/bashrc: execute this file for every user running bash shell. When bash shell is opened, the file is read.
~ /. Bash_profile: each user can use this file to enter the shell information dedicated to his/her use. When a user logs on,
The file is executed only once! By default, it sets some environment variables to execute the user's. bashrc file.
~ /. Bashrc: This file contains bash information dedicated to your bash shell. When you log on and every time you open a new shell,
The file is read.
~ /. Bash_logout: execute this file every time you exit the system (exit bash shell.
/Etc/profile is a global function, in which the set variables act on all users ,~ The variables set in/. bash_profile can inherit the variables in/etc/profile and act on users.
~ /. Bash_profile is interactive and login to run in bash.
~ /. Bashrc runs in bash in interactive non-login mode.
Generally, the two settings are roughly the same, so the former usually calls the latter. Http://blog.chinaunix.net/u2/63775/showart_527708.html)
However, when running crontab, the program is called in the non_login method ~ /. Bash_profile is not called in advance. Therefore, the running environment of crontab is much smaller than the running environment of bash in the login mode. If the program involves ~ /. Bash_profile: The environment variables used, so some programs that can run normally in the login mode cannot run under crontab.
In my program, the system cannot identify mysqldb, so the solution is to add the following sentence to the shell script of the general control:
Export LD_LIBRARY_PATH =/home/work/local/mysql5/lib/MySQL
More recommended solutions:
Add in cron
LD_LIBRARY_PATH =/home/work/local/mysql5/lib/MySQL
In this way, everything using MySQL in cron can run smoothly:) and this makes the operation clearer.
Ultimate recommended solution:
30 12 **** source ~ /. Bashrc & CD/home/work/mydir &./myproj
2) LD_LIBRARY_PATH
There is a set of shared libraries (*. So) in Linux ). The shared library is searched and loaded through/lib/lD. So (runtime shared library loader. LD. So searches for shared libraries in the standard path (/lib,/usr/lib. However, if the third-party library is not installed in the standard path, the library cannot be found when the program runs, similar to the following error:
LD. so.1: curl: Fatal: libgcc_s.so.1: Open failed: no such file or directory
You can set the environment variable LD_LIBRARY_PATH to allow lD. So to find a shared library with a non-standard path. Multiple paths can be set in LD_LIBRARY_PATH, separated by a colon. The path in LD_LIBRARY_PATH is located prior to the standard path.
In ~ /. Add the following code to bash_profile (for example, add the so file of MySQL to LD_LIBRARY_PATH)
LD_LIBRARY_PATH = $ LD_LIBRARY_PATH:/home/work/local/mysql5/lib/MySQL
Export LD_LIBRARY_PATH
Because ~ /. Bash_profile will be loaded (and loaded only) Once upon user login, and LD. So will automatically find and load the shared library in the standard path and LD_LIBRARY_PATH.
Disadvantages of LD_LIBRARY_PATH: (refer to http://xahlee.org/UnixResource_dir/_/ldpath.html)
"For security reasons, LD_LIBRARY_PATH is ignored at runtime for executables that have their setuid or setgid bit set. this severely limits the usefulness of LD_LIBRARY_PATH. "............ "LD_LIBRARY_PATH is one of those insidious things that once it gets set globally for a user, things tend to happen which cause people to rely on it being set. eventually when LD_LIBRARY_PATH needs to be changed Or removed, mass breakage will occur! "............ "Nowadays you specify the run-time path for an executable at link stage with the-R (or sometimes-rpath) flag to lD. there's also ld_run_path which is an environment variable which acts to LD just like specifying-R. before all this you had only-L, which applied not only during compile-time, but during run time as well. there was no way to say "use this directory during compile time" but "use this other directory at run time ". there were some rather spectacular failure modes that one cocould get in to because of this."
This article also shows how to properly use LD_LIBRARY_PATH: (although I don't fully understand it, I 'd like to post it and look forward to understanding it in the near future)
1) Never ever set LD_LIBRARY_PATH globally.
If you must ship binaries that use shared libraries and want to allow your clients to install the program outside a 'standard' location, do one of the following:
Ship your binaries as. O files, and as part of the install process relink them with the correct installation library path.
Ship executables with a very long "Dummy" Run-Time library path, and as part of the install process use a binary editor to substitute the correct install library path in the executable.
2) If you are forced to set LD_LIBRARY_PATH, do so only as part of a wrapper.
3 ). remove the link-time aspect of LD_LIBRARY_PATH ..... it wocould be much cleaner if LD_LIBRARY_PATH only had influence at run-time. if necessary, invent some other environment variable for the job (ld_link_path ).
3) LD. So. conf
In addition to LD_LIBRARY_PATH, you can also set/etc/lD. So. conf. Run ldconfig to generate lD. So. cache. LD. So will also be searched from LD. So. cache when searching for public libraries.
However.
"Some OS's (e.g. linux) have a retriable loader. you can configure what run-time paths to look in by modifying/etc/lD. so. conf. this is almost as bad a LD_LIBRARY_PATH! Install scripts shoshould never modify this file! This file shoshould contain only the standard library locations as shipped with the OS ."
The runtime linker detailed behavior of LD_LIBRARY_PATH can refer to http://docs.sun.com/app/docs/doc/819-0690/chapter6-63352? A = View

From: http://hi.baidu.com/huangboxiang/blog/item/f798a7dc3eb096e877c63833.html

 

As we all know, crontab is a good job. It can regularly execute some tasks to help you monitor system conditions and help you repeat mechanical tasks every day. However, crontab has a bad problem, that is, it does not read environment variable parameters from the user profile file by default, which often leads to a successful execution of a script by hand, however, an error occurs when the crontab attempts to periodically execute the program.
In the past, I used a silly method to directly specify all environment variable parameters in the script. Each time I write a script, I have to write a lot of paths, such as environment variable parameters such as LD_LIBRARY_PATH.
Later I found that I could execute the user's profile file directly in the script, and it would be OK.
If it is a script in Linux, use the default # in the script header #! /Bin/sh. If the script is in the Solaris environment, use # in the script header #! /Bin/KSh
Then write the following in the first part:
###################
./Etc/profile
.~ /. Bash_profile
##################
In this way, crontab can read the user's environment variable parameters when executing the script... A little trick. ^_^
Appendix:
If you submit in cron, note the following:
Do not assume that c r o n knows the required special environment, but it does not. Therefore, make sure that all necessary paths and environment variables are provided in the s h e l script, except for some automatically set global variables.
If c r o n cannot run the corresponding script, the user will receive an email explaining the cause.