Nhib1_vs Entity Framework: a performance test (zhuan)

As part of my thesis I measured the performance of some. net orm frameworks including NHibernate and Entity Framework. measuring was done by implementing two simple applications using the same table structure and doing the same operations on the same data.

Note: before reading this article please see my views on comparing ORM tools. Also see the follow up post to this comparison for revised results.

I measured the time it took for each framework to complete these operations:

• Store
• Read over relations
• Read by ID
• Update
• Delete

The results were somewhat interesting. Here is a short summary of what I 've found out.

Operation \ Number of operations	NHiberante-4 K	Entity Framework-4 K	NHiberante-40 K	Entity Framework-40 K	Winner
Store	37,37	9,19	1500	98	Entity Framework
Read over relations	1, 01	0, 54	10, 13	4,18	Entity Framework
Read by ID	3, 06	25, 22	246	230	Nhibunt with smaller amount of objects
Update	6, 61	7,34	77	72	Both
Delete	3, 35	16,76	58	1824	Nhib.pdf

Read on if you're interested in the details of the performance measurements.

Update: The source code of the program used to measure is nowavailable for download.

How performance was measured

The data model I 've set up to measure with was pretty simple. I used three entites:CompanyEntity which had multipleEmployeeS all of which had multipleReportS all with a 1-n relation.

The detailed results for each operation are as follow (note that the graphs use logarithmic scale ):

 

Storage

 

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	4,95	20, 64	112	280	575	1107	15000
Entity Framework	2, 11	6, 18	18, 4	31,3	51,7	77	98

Update(2009. 08. 24 .):
After nhib1_improvements by Tuna Toksoz (download source) this metric changed the following way:

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	1, 22	3, 25	8, 98	13,11	28	38	46
Entity Framework	2, 11	6, 18	18, 4	31,3	51,7	77	98

Read over relations

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	0, 21	0, 66	1, 81	3, 31	5, 0	8, 24	10, 13
Entity Framework	0, 17	0, 4	0, 96	1, 76	2, 47	3,91	4,18

Read by ID

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	0, 28	1, 31	13, 0	37,6	80	175, 5	246
Entity Framework	5, 56	17,03	48	83	132	195	230

Update

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	1, 74	4,54	17,1	33,2	43, 47	68	77
Entity Framework	1, 99	5, 29	14,2	24, 2	41,9	59	72

Delete

Framework/No. of operations	1 K	3 K	8 K	14 K	23 K	33 K	40 K
Nhib.pdf	0, 78	2, 31	8, 83	20, 6	28	46	58
Entity Framework	1, 78	8, 19	52,2	133	320	1014	1895

Conclusion

The performance measurements only provided significant differences in two cases.

• When it comesStoring data, Entity Framework proved to be significantly faster than nhib.pdf
• WhenDeleting data, Nhib1_was much faster than Entity Framework

My conclusion however after the test was that in those scenarios when the number of the operations is not too high performance does not differentiate that much. I wocould choose between these two tools based on which ones provides faster, more efficient and more straightforward development with the current project.

Give me the code

As defined of the commenters pointed out a comparison is not really valid without publishing the code enabling validation of this measurement. the program I 've written to do the test can therefore be downloaded: NHibernate vs Entity Framework source code.

To set up and run the project after downloading an unzipping the package follow these steps:

• Create two MSSQL Databases (e.g. EntityFrameworkTest and NHibernateTest)
• Run CreateTables. SQL on both databases to create the database structure
• In the App. config section of the MeasurementController project set the connection strings for nhib.pdf and Entity Framework (replace !!! Marks)
• Build the solution
• Run the MeasurementController project
• Run times are written to the output and also saved in files in the bin/Data folder

Some additional comments for understanding the source:

• Upon every run test data is created in an XML file. From this XML file Company, Emlpoyee and Report objects are created in-memory as the test structure
• This test structure is passed to the NHibernate and Entity Framework implementations of the Store, Read, Read by ID, Update and Delete tests (in the source additionally search is also measured) on each test run.