Pandas read large file read_table c-engine cparsererror:error tokenizing data

Workaround:

Pd_data = pd.read_table (comment_file,header=none,encoding='utf-8', engine=' python ')

Official website Analysis:

engine : {' C ', ' Python '}, optional

Parser engine to use. The C engine was faster while the Python engine was currently more feature-complete.

 

1,

iterator  : boolean, default False

Return Textfilereader object for iteration or getting chunks With get_chunk () .

or get

from Chunk

 pd_data = pd.read_table (comment_file,header=none,encoding= ' utf-8 ', iterator=true) 
 # Print (pd_data) 
 # Pd_ data_t = pd.read_table (comment_file,header=none,encoding= ' utf-8 ', engine= ' python ') 
 # return; 
 loop = True 
 Chunk_data = [] 
 Chunk_size = 1024x768 
 While loop: 
 Try: 
 pd_data_tmp = Pd_data.get_chunk (chu nk_size) 
 Chunk_data.append (pd_data_tmp) 
 except stopiteration: 
 loop = False 
 df = pd.concat (chunk _data,ignore_index=true) 

Pandas read large file read_table c-engine cparsererror:error tokenizing data