Python Learning (i)
One: development tool installation
1,pycharm Download: Linux version
2, crack: Under Help-register-service input:
http://elporfirio.com:1017/or http://idea.imsxm.com/
Second: Common plug-in download and installation:
1, download and install Pip
wget https://bootstrap.pypa.io/get-pip.py --no-check-certificatesudo python get-pip.py
2, download and install Setuptools
wget Https://bootstrap.pypa.io/ez_setup.py-O-| sudo pythonsudo apt-get install Python-setuptools
3, install Networkx and NumPy and Matplotlib
sudo pip install networkx sudo apt-get install python-numpy python-scipy python-matplotlib Ipython ipython-noteb Ook Python-pandas python-sympy Python-nose
4, install the py4j, because the rear to use spark
sudo pip install py4j
5,pycharm Configuring Spark
Open Pycharm and create a project. Then select "Run", "Edit Configurations", "Environment variables" to add Spark_home directory and Pythonpath directory. -Spark_home:spark installation directory-Python directory under the Pythonpath:spark installation directory
6, two examples
from Import = Sparkcontext ("local""test"= Sc.parallelize ([1, 2, 3, 4])print"rdd.collect ():%s", str ( Rdd.collect ())
ImportNetworkx as NXImportMatplotlib.pyplot as Pltdefdraw_graph (graph):#extract nodes from graphnodes = Set ([N1 forN1, N2inchGraph] + [n2 forN1, N2inchGraph]) #Create Networkx Graphg=NX. Graph ()#Add Nodes forNodeinchnodes:g.add_node (node)#Add Edges forEdgeinchGraph:g.add_edge (edge[0], edge[1]) #Draw Graphpos =Nx.shell_layout (g) Nx.draw (g, POS)#Show Graphplt.show ()#Draw ExampleGraph = [(20, 21), (21, 22), (22, 23), (23, 24), (24, 25), (25, 20))]draw_graph (graph)
Third, reference website:
Using Pycharm to configure the Python development environment for Spark (base)
Pycharm Latest 2017 Activation code
installation and use of Networkx,matplotlib under Ubuntu
Python Learning (i)