When we recently compiled spark, we encountered a problem:
After spark_hadoop_version = 0.20.2-cdh3u5 spark_hive = true SBT/SBT assembly is executed, an error is returned:
Error: Protocol HTTPS not supported or disabled in libcurl while accessing https://github.com/apache/spark.git/info/refs
Fatal: http request failed
A problem occurred during git clone. It is suspected that curl does not support HTTPS. After checking, I found that curl supports HTTPS.
The problem is located. Git and git estimate that libcurl does not support HTTPS during compilation.
The cost of manual repair is too high and it has been fixed for a long time, which is found in an article about stackoverflow about this issues: http://stackoverflow.com/questions/19015282/how-do-i-enable-https-support-in-libcurl
Some people mentioned the use of brew doctor, which is only available on Mac OS. However, Linux has also been ported to linuxbrew.
Brew is an installation toolkit similar to Apt-Get and yum. The following features are supported:
- Can install software to a home directory and so does not require sudo
- Install software not packaged by the native distribution
- Install up-to-date versions of software when the native distribution is old
- Use the same package manager to manage both your Mac and Linux machines
- Linuxbrew allows the package to be installed in the user's home directory, so that sudo is no longer required;
- You can use linuxbrew to handle the missing packages in the Linux Package Manager;
- If the version of the package included in the Linux Package Manager is too old, you can use linuxbrew to install the latest version of the package.
1. Install linuxbrew: Clone the installation from git and configure the environment variables as follows:
git clone https://github.com/Homebrew/linuxbrew.git ~/.linuxbrewexport PATH="$HOME/.linuxbrew/bin:$PATH"export LD_LIBRARY_PATH="$HOME/.linuxbrew/lib:$LD_LIBRARY_PATH"
2. reinstall git: delete git and reinstall linuxbrew to install git:
[[email protected] shengli]# brew install git --with-brewed-curl ==> Installing dependencies for git: pkg-config, makedepend, openssl, curl==> Installing git dependency: pkg-config==> Downloading http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz......
Solve the problem. Reference: http://www.linuxeden.com/html/news/20140312/149399.html https://github.com/Homebrew/linuxbrew
Original article: Reprint please indicate from: http://blog.csdn.net/oopsoom/article/details/39231091
Brew software package management tool