Sesame HTTP:, sesame http

Sesame HTTP:, sesame http

As long as your Scrapy Field name is the same as the database Field name. Congratulations, you can copy this SQL concatenation script. To store MySQL Data.

The specific Splicing code is as follows:

Def process_item (self, item, spider): if isinstance (item, WhoscoredNewItem): table_name = item. pop ('table _ name') col_str = ''row_str = ''for key in item. keys (): col_str = col_str + "" + key + "," row_str = "{}'{}',". format (row_str, item [key] if "'" not in item [key] else item [key]. replace ("'", "\'") SQL = "insert INTO {} ({}) VALUES ({}) ON DUPLICATE KEY UPDATE ". format (table_name, col_str [1:-1], row_str [:-1]) for (key, value) in six. iteritems (item): SQL + = "{}= '{}',". format (key, value if "'" not in value else value. replace ("'", "\'") SQL = SQL [:-2] self.cursor.exe cute (SQL) # Execute SQL self. cnx. commit () # write operation

This SQL concatenation is implemented. If the database has the same data, it will be updated. If the database does not exist, it will insert the SQL statement.

The specific implementation is the first for loop, where the key is obtained as the MySQL field name and VALUES is used as the SQL values (spliced into an inserted SQL statement)

The second for loop implements the concatenation of field name = VALUES.

And the SQL statement in the first for loop forms the insert into XXXXX on duplicate key update. If yes, update the SQL statement inserted if no data exists.