Python Django batch import without repeated data, pythondjango

Python Django batch import without repeated data, pythondjango

This article shares the python Django implementation code for batch import of non-duplicate data for your reference. The specific content is as follows:

The procedure is as follows:

# Coding: UTF-8 import OS. environ. setdefault ("DJANGO_SETTINGS_MODULE", "www. settings ") ''' when the Django version is greater than or equal to 1.7, add the following two statements: import djangodjango. setup () otherwise, the error django will be thrown. core. exceptions. appRegistryNotReady: Models aren't loaded yet. '''import djangoimport datetimeif django. VERSION> = (1, 7): # automatically determines the VERSION of django. setup () from keywork. models import DevData f = open('cs.csv ') WorkList = [] next (f) # Move the file tag to the next line x = y = 0for line in f: parts = line. replace ('"','') # replace "in the dictionary with null parts = parts. split (',') # Press; to segment the string if DevData. objects. filter (serv_id = parts [0], day_id = parts [24]). exists (): x = x + 1 else: y = y + 1 WorkList. append (DevData (serv_id = parts [0], serv_state_name = parts [1], acc_nbr = parts [2], user_name = parts [3], acct_code = parts [4], product_id = parts [5], mkt_chnl_name = parts [6], mkt_chnl_id = parts [7], mkt_region_name = parts [8], mkt_region_id = parts [9], mkt_grid_name = parts [10], sale_man = parts [11], sale_outlets_cd1_name = parts [12], completed_time = parts [13], remove_data = parts [14], user_flag = parts [15], pro_flag = parts [16], service_offer_id = parts [17], service_offer_name = parts [18], finish_time = parts [19], staff_name = parts [20], staff_code = parts [21], org_name = parts [22], prod_offer_name = parts [23], day_id = parts [24]) f. close () DevData. objects. bulk_create (WorkList) print "duplicate data" + str (x) + "entries, imported data" + str (y) + "entries"

This article mainly uses the exists () in the model to determine whether the imported data and database are repeated. If there are no duplicates, use bulk_create to import the database in batches!

For more information, see my other article:Batch import data

The above is all the content of this article, hoping to help you learn.

Articles you may be interested in:

• Read and Write python files and use mysql to insert examples in batches (use python to operate mysql)
• Python implements batch file name replacement and content replacement
• Batch cropping of Images Using python
• Python uses loops to create folders in batches.
• Python tips: Batch capturing beautiful pictures
• Python converts the SVG format to png and pdf format for code sharing in batches.
• Python batch file name modification implementation code
• How to obtain the file Suffix in python and update the file Suffix in the directory in batches
• Python Django batch import data
• Django framework uses ajax to import data in batches