Pandas Merge,join and concatenate

Concatenating objects

In [3]: pieces = [Df[:3], Df[3:7], df[7:] "in [4]: concatenated = concat (Pieces) in [5]: concatenated out[5]: 0 1 2 3 0 0.469112-0.282863-1.509059-1.135632 1 1.212112-0.173215 0.119209-1.044236 2-0.861849-2.10456 9-0.494929 1.071804 3 0.721555-0.706771-1.039575 0.271860 4-0.424972 0.567020 0.276232-1.087401 5-0.673690 0.113648 -1.478427 0.524988 6 0.404705 0.577046-1.715002-1.039268 7-0.370647-1.157892-1.344312 0.844885 8 1.075770-0.109050 1 .643563-1.469388 9 0.357021-0.674600-1.776904-0.968914 in [6]: concatenated = concat (pieces, keys=[' A ', ' second ', ' Third '] in [7]: concatenated out[7]: 0 1 2 3 a 0 0.469112-0.282863-1.509059-1.1356 1 1.212112-0.173215 0.119209-1.044236 2-0.861849-2.104569-0.494929 1.071804 Second 3 0.721555-0.70677
       1-1.039575 0.271860 4-0.424972 0.567020 0.276232-1.087401 5-0.673690 0.113648-1.478427 0.524988 6 0.404705 0.577046-1.715002-1.039268 Third 7-0.370647-1.157892-1.344312 0.844885 8 1.075770-0.109050 1.643563-1.469388 9 0.357 021-0.674600-1.776904-0.968914 in [8]: concatenated.ix[' second '] out[8]: 0 1 2 3 3 0.721 555-0.706771-1.039575 0.271860 4-0.424972 0.567020 0.276232-1.087401 5-0.673690 0.113648-1.478427 0.524988 6 0.40470 5 0.577046-1.715002-1.039268

frames = [Process_your_file (f) for f with files] result = Pd.concat (frames) 
 

In [one]: DF out[11]: a b c d mPXqv-1.294524 0.413738 0.276662-0.472035 ah4pw-0.013960-0.36 2543-0.006154-0.923061 C30FM 0.895717 0.805244-1.206412 2.565646 3EWtQ 1.431256 1.340309-1.170299-0.226169 1gQh9 0.41 0835 0.813850 0.132003-0.827317 kqwv8-0.076467-1.187678 1.130127-1.436737 8udgh-1.413681 1.607920 1.024180 0.569605 K  A8VN 0.875906-2.211372 0.974466-2.006747 kddli-0.410001-0.078638 0.545952-1.219217 yZsRv-1.226825
      -0.727707 in [[]: Concat ([Df.ix[:7, [' A ', ' B ']], df.ix[2:-2, [' C ']]], ...: df.ix[-7:, [' d ']]], Axis=1 ...: out[12]: A b c d 1gqh9 0.410835 0.813850 0.132003-0.827317 3EWtQ 1.431256 1.340309-1.170299-0.226169 8U DGh-1.413681 1.607920 1.024180 0.569605 ah4pw-0.013960-0.362543 nan nan ka8vn nan nan 0.974466-2.006747 kddl
I nan nan NaN-1.219217 kqwv8-0.076467-1.187678 1.130127-1.436737 c30fm 0.895717 0.805244-1.206412 nan mPXqv-1.294524 0.413738 Nan nan yzsrv nan nan NaN-0.727707 in [[]: Concat [Df.ix[:7, [' A ', ' B ']], df.ix[2:-2, [' C ']], .... : df.ix[-7:, [' d ']]], Axis=1, join= ' inner ') ...: out[13]: a b c D 3EWtQ 1.431256 1.340309-1. 170299-0.226169 1gqh9 0.410835 0.813850 0.132003-0.827317 kqwv8-0.076467-1.187678 1.130127-1.436737 8udgh-1.413681 1 .607920 1.024180 0.569605 in []: Concat ([Df.ix[:7, [' A ', ' B ']], df.ix[2:-2, [' C ']], ...: df.ix[-7:, [' d ']]], Axis=1, Jo In_axes=[df.index]) ...: out[14]: a b c d mPXqv-1.294524 0.413738 nan nan ah4pw-0.013960-0.3625 0.895717 nan nan c30fm 0.805244-1.206412 nan 3EWtQ 1.431256 1.340309-1.170299-0.226169 1gqh9 0.410835 0.813850 0.132 kqwv8-0.076467-1.187678 1.130127-1.436737 8udgh-1.413681 1.607920 1.024180 0.569605 KA8Vn nan nan 0.9744 003-0.827317 66-2.006747 Kddli nan nan NaN-1.219217 yzsrv nan nan NaN-0.727707

concatenating using Append
A useful shortcut to Concat are the append instance on methods and Series. These methods actually predated concat. They concatenate along axis=0, namely the index

in [[]: Df1
out[22]:
           A        B        C         D
2000-01-01 0.176444 0.403310-0.154951 0.301624
2000-01-02 -2.179861-1.369849-0.954208 1.462696
2000-01-03-1.743161-0.826591-0.345352 1.314232 in
[to]: DF2
Out[ :
           A        B        C
2000-01-04 0.690579 0.995761 2.396780
2000-01-05 3.357427-0.317441-1.236269
2000-01-06-0.487602-0.082240-2.182937
in [[]: Df1.append (DF2)
out[24]:
           A        B        C         D
2000-01-01 0.176444 0.403310-0.154951 0.301624
2000-01-02-2.179861-1.369849-0.954208 1.462696
2000-01-03-1.743161-0.826591-0.345352 1.314232
2000-01-04 0.690579 0.995761 2.396780 nan
2000-01-05 3.357427-0.317441-1.236269 nan
2000-01-06- 0.487602-0.082240-2.182937 NaN

Ignoring indexes on the concatenation axis

In [£]: concat ([Df1, DF2], ignore_index=true) in
[%]: Df1.append (DF2, Ignore_index=true)

More concatenating with group keys

In [$]: pieces = [df.ix[:, [0, 1]], df.ix[:, [2]], df.ix[:, [3]], in
[%]: result = concat (pieces, Axis=1, keys=[' one ', ' two ', ' three '] in
[/]: Result
out[45]: one                 two      three
  0         1         2        3
0- 0.014805-0.284319 0.650776-1.461665
1-1.137707-0.891060-0.693921 1.613616
2 0.464000 0.227371-0.496922 0. 306389
3-2.290613-1.134623-1.561819-0.260838
4 0.281957 1.523962-0.902937 0.068159
5-0.057873- 0.368204-1.144073 0.861209
6 0.800193 0.782098-1.069094-1.099248
7 0.255269 0.009750 0.661084 0.379319
8-0.008434 1.952541-1.056652 0.533946
9-1.226970 0.040403-0.507516-0.230096

Database-style Dataframe joining/merging

Left-use keys ' from-left frame-only
right-use keys ' from ' right ' frame only
outer use Union ' keys from both FRA Mes
Inner use intersection the keys from both frame