Performance Test: Linq to SQL vs Data Adapters/DataSet

LINQ to SQL is nifty. Because it is two-tiered, it's usefulness is limited. However, it's neat. The real question is: how does it perform?

A Runtime Query passes SQL to SQL Server. A Procedure Query accesses a Stored Procedure.

Basically, I accessed a simple SQL table (ID int identity primary key, Name varchar(50)) with 47k records, retrieving the one and only record with a specified name.

Results

Here's how it worked out.

image

Conclusion

The winner is? It's hard to say. Though runtime Linq is clearly not.

LINQ to SQL is pretty impressive. Considering the power of the runtime query function, the cost (which must be the construction of the query?) is close to other typical data access methods.

I ran it 40 times to make sure any SQL execution plan cache would benefit all techniques.

Note the Sleep(10) prevented 100% processor utilization. It made everything 1000 milliseconds longer - which is interesting, too.

Source Code

To execute this code, it will take some work. You will need to create the Context for LINQ to SQL and the typed DataSet for the data adapters. You will also need the database and the data.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using EntityContext;
using ConsoleApplication2.DataAccessDataSetTableAdapters;
using System.Data.SqlClient;
using System.Data.Common;
using System.Data;

namespace ConsoleApplication2
{
class DataAccess
{
static string m_ConnStr = "";
static public void Test()
{
string _Name = "Jason A. Clark";
System.Diagnostics.Stopwatch _Stopwatch =
new System.Diagnostics.Stopwatch();
int _Repeat = 100;

LinqProc(_Name, _Stopwatch, _Repeat);
//LinqRuntime(_Name, _Stopwatch, _Repeat);
//DataSetProc(_Name, _Stopwatch, _Repeat);
//DataSetRuntime(_Name, _Stopwatch, _Repeat);
//DataReaderX(_Name, _Stopwatch, _Repeat);
}

private static void DataSetRuntime(
string _Name,
System.Diagnostics.Stopwatch _Stopwatch,
int _Repeat)
{
// dataset runtime query
_Stopwatch.Reset();
_Stopwatch.Start();
for (int i = 0; i < _Repeat; i++)
{
TestTableTableAdapter _DataAdapter;
_DataAdapter = new TestTableTableAdapter();
DataAccessDataSet.TestTableDataTable _DataTable;
_DataTable = _DataAdapter.GetDataByName(_Name);
System.Diagnostics.Debug.Write(
_DataTable.Rows[0]["Name"]);
System.Threading.Thread.Sleep(10);
}
_Stopwatch.Stop();
Console.Write(
_Stopwatch.Elapsed.TotalMilliseconds.ToString() + ",");
}

private static void DataSetProc(
string _Name,
System.Diagnostics.Stopwatch _Stopwatch,
int _Repeat)
{
// dataset proc query
_Stopwatch.Reset();
_Stopwatch.Start();
for (int i = 0; i < _Repeat; i++)
{
TestTableTableAdapter _DataAdapter;
_DataAdapter = new TestTableTableAdapter();
DataAccessDataSet.TestTableDataTable _DataTable;
_DataTable = _DataAdapter.GetDataByProc(_Name);
System.Diagnostics.Debug.Write(
_DataTable.Rows[0]["Name"]);
System.Threading.Thread.Sleep(10);
}
_Stopwatch.Stop();
Console.Write(
_Stopwatch.Elapsed.TotalMilliseconds.ToString() + ",");
}

private static void LinqRuntime(
string _Name,
System.Diagnostics.Stopwatch _Stopwatch,
int _Repeat)
{
// linq runtime query
_Stopwatch.Reset();
_Stopwatch.Start();
for (int i = 0; i < _Repeat; i++)
{
DataAccessLinqDataContext _db;
_db = new DataAccessLinqDataContext();
var _Results = from item in _db.TestEntities
where item.Name == _Name
select item;
System.Diagnostics.Debug.Write(
_Results.First().Name);
System.Threading.Thread.Sleep(10);
}
_Stopwatch.Stop();
Console.Write(
_Stopwatch.Elapsed.TotalMilliseconds.ToString() + ",");
}

private static void LinqProc(
string _Name,
System.Diagnostics.Stopwatch _Stopwatch,
int _Repeat)
{
// linq procedure query
_Stopwatch.Reset();
_Stopwatch.Start();
for (int i = 0; i < _Repeat; i++)
{
DataAccessLinqDataContext _db;
_db = new DataAccessLinqDataContext();
var _Results = _db.TestProcedure(_Name);
System.Diagnostics.Debug.Write(
_Results.First().Name);
System.Threading.Thread.Sleep(10);
}
_Stopwatch.Stop();
Console.Write(
_Stopwatch.Elapsed.TotalMilliseconds.ToString() + ",");
}

private static void DataReaderX(
string _Name,
System.Diagnostics.Stopwatch _Stopwatch,
int _Repeat)
{
// datareader runtime query
_Stopwatch.Reset();
_Stopwatch.Start();
for (int i = 0; i < _Repeat; i++)
{
string _Sql;
_Sql = string.Format(
"select * from tb_Test where Name = '{0}'", _Name);

using (SqlConnection _SqlConnection =
new SqlConnection(m_ConnStr))
{
SqlCommand _SqlCommand;
_SqlCommand = new SqlCommand(_Sql,
_SqlConnection);
_SqlConnection.Open();

SqlDataReader _SqlDataReader;
_SqlDataReader = _SqlCommand.ExecuteReader();

_SqlDataReader.Read();
System.Diagnostics.Debug.Write(
_SqlDataReader["Name"]);
_SqlDataReader.Close();
_SqlConnection.Close();
}
System.Threading.Thread.Sleep(10);
}
_Stopwatch.Stop();
Console.Write(
_Stopwatch.Elapsed.TotalMilliseconds.ToString());
}
}
}

.Net/C# Loop Performance Test (FOR, FOREACH, LINQ, & Lambda)

There's a lot of new syntax. I started to wonder: do these cuties perform? I tried it out and these are my results.

FOR loop

The traditional FOR loop has been the performance champion. Its cryptic syntax has foiled developers resorting to the FOR snippet or turning to FOREACH. But does FOR still compete?

FOREACH loop

Visual Basic invented the FOREACH flow operator. If you have ever forgotten RecordSet.MoveNext() you know how lovely FOREACH is. FOREACH has never outperformed FOR, but it's simplified syntax has made it a favorite among developers. How does it perform in the new syntax landscape?

LIST.FOREACH loop

In the .Net Framework v2.0 the FOREACH delegate was delivered with the new LIST object. This allowed developers to iterate a list without the clutter of a FOREACH loop. Isn't that handy? I understand using delegates with SORT, but I am skeptical of the FOREACH. How does it perform?

LINQ "loop"

LINQ is the Language Integrated Query. It allows developers to put "backwards" SQL in their code. It can iterate, filter, and join enumerable lists. It's powerful. I like it. It has promise. But I was anxious to see how this new developer candy stacked up against the classic loop operators.

LAMBDA => "loop"

Lambda's cryptic syntax has many developers scratching their head. Its steep learning curve is no joke. Most developers (like me) can't separate its purpose from LINQ. However, language complexity is a matter of familiarity. Does this complexity pay off when it comes to performance? Let's take a closer look.

And here are my results:

Note: this is Framework 3.5 SP0 - to be fair.

image

Those are the raw numbers; here are the pretty pictures.

image

Conclusion:

The new syntax stacks up nicely. Clearly LINQ and Lambda can accomplish more complex solutions than a simple loop. However, in this simple scenario they are "just as" good. That's nice.

NOTE: this is an update from my previous post where the numbers were flawed because of a bug in the code. Sorry if you noticed that.

Here's the code I used to test:

This is a simple little console project.

Run it yourself if you doubt the numbers.

static void Test()
{
System.Diagnostics.Stopwatch _Stopwatch =
new System.Diagnostics.Stopwatch();
int _Repeat = 100;

// setup the source list we will iterate

List<int> _List = new List<int>();
for (int i = 0; i < 1000000; i++)
_List.Add(i);

// for

_Stopwatch.Reset();
_Stopwatch.Start();
List<int> _HoldFor;
for (int i1 = 0; i1 < _Repeat; i1++)
{
_HoldFor = new List<int>();
for (int i2 = 0; i2 < _List.Count - 1; i2++)
if (_List[i2] % 2 == 0)
_HoldFor.Add(_List[i2]);
}
_Stopwatch.Stop();
Console.WriteLine("for:\r\n "
+ _Stopwatch.Elapsed.TotalMilliseconds.ToString());

// foreach

_Stopwatch.Reset();
_Stopwatch.Start();
List<int> _HoldForEach;
for (int i1 = 0; i1 < _Repeat; i1++)
{
_HoldForEach = new List<int>();
foreach (int i2 in _List)
if (i2 % 2 == 0)
_HoldForEach.Add(i2);
}
_Stopwatch.Stop();
Console.WriteLine("foreach:\r\n "
+ _Stopwatch.Elapsed.TotalMilliseconds.ToString());

// delegate foreach

_Stopwatch.Reset();
_Stopwatch.Start();
List<int> _HoldForEachDelegate;
for (int i1 = 0; i1 < _Repeat; i1++)
{
_HoldForEachDelegate = new List<int>();
_List.ForEach(delegate(int i)
{
if (i % 2 == 0)
_HoldForEachDelegate.Add(i);
});
}
_Stopwatch.Stop();
Console.WriteLine("list.foreach:\r\n "
+ _Stopwatch.Elapsed.TotalMilliseconds.ToString());

// linq

_Stopwatch.Reset();
_Stopwatch.Start();
for (int i1 = 0; i1 < _Repeat; i1++)
{
var _x = from integer in _List
where integer % 2 == 0
select integer;
}
_Stopwatch.Stop();
Console.WriteLine("linq:\r\n "
+ _Stopwatch.Elapsed.TotalMilliseconds.ToString());

// lambda

_Stopwatch.Reset();
_Stopwatch.Start();
for (int i1 = 0; i1 < _Repeat; i1++)
{
var _x = _List.Where(i => i % 2 == 0);
}
_Stopwatch.Stop();
Console.WriteLine("lambda:\r\n "
+ _Stopwatch.Elapsed.TotalMilliseconds.ToString());

Console.WriteLine(string.Empty);
}

An Architect’s Balance with Developer Independence

Software projects of any significant size have architects. Architects scope the technical plan and make strategic technical decisions. A hands-on architect might build prototypes, interfaces, and mock ups.

A lead developer is sometimes wrongly considered an architect. Lead developers lead developers and make tactical technical decisions. They see a project through to its delivery.

Sometimes one person gets both roles.

Recently, I have appreciated the contracting forces of Nazi-like software rigor and the individual creative freedom – like micro-management versus macro-management.

I created a series of interfaces to build a wizard-like UI. The implementation was up to the developers. Dozens of wizard steps later, implementations varied greatly. It "got the job done" but was a cluster.

For the sake of simplicity or future maintainability by the customer, it needed to be refactored with a consistent implementation across each step.

I refactored each wizard step. The resulting code was simpler (50% less code) and consistent (since I knew the implementation pattern in advance).

But here's where my trouble started.

The customer wondered why my solution architecture was changing or was insufficient to address this 6 months prior to it occurring. What was all that "prep" work for?

Non-technical project members misunderstood refactoring as rewriting. They wondered why I didn't leave "good enough alone" as they encountered the inevitable I introduced.

My developers felt embarrassed when their implementation was not the selected pattern. They were frustrated. Their code was changed and my new bugs were appearing in their task list.

And here's where I went wrong.

As both the architect and lead developer, I didn't have the correct relationship with the developers or time on the project (a very tight timeline) to sufficiently code review.

Resting on too many assumptions with the customer, I never bothered to educate them in the subtle differences between architecture and design.

Misreading the non-technical project team's willingness to "do it right" I didn't bother educating them in the necessity and the value of refactoring code.

In the end, the code was simpler, consistent and maintainable. But the resulting team friction disenfranchised my later opportunities to refactor other ailing areas.

It's a lesson I have learned.

At the heart of the issue is the desire (as the lead) to ensure the overall excellence of the solution versus my desire to respect the creative desires of developers and maintain a cohesive team.

Which is more important?

From the bottom up, a cohesive, effective team is vital to project success. Regular turn over dilutes the vision, atrophies morale, and undermines consistency. Frustrated teams lose velocity, decrease quality, and are miserable. So, is the team the most important?

From the top down, a quality solution is vital to project success. Decreased quality increases maintenance costs, reduces shelf life, and threatens timelines and budgets. Mediocre software is more expensive, less stable, and slow. So, is software structure most important?

The answer is easy; doing it is difficult.

The customer writes the checks, but developers can leave – screwing up a project. A developer writes cool code, but customers can reject it – foiling success. Each must compromise for the other.

Instill in the customer and project team a common vision – each is the other's key to success. Good might just be "good enough" but it should never be the goal. A shared drive to excellence, simplicity, and mutual benefit – that's the real key.

What about developer independence?

There's no quick answer. Sometimes a developer is plain wrong. Sometimes a developer's idea is better than yours. You can't stifle creativity, but you have to ensure you have the time and discipline to adequately code review work. And communicate to your developers that code review and code refactoring is going to happen. Don't marry your code. But, be sure customers and non-technical team people understand this is vital and as the architect or lead developer, you must be allotted sufficient time.

Software is a Mess

Everywhere you look, software is over budget, behind schedule, insecure, unreliable, and hard to use. Anytime an organization attempts to introduce a new system, or upgrade an old one, it takes a colossal risk; today, large information-technology projects are technological tar pits that immobilize institutions. Studies regularly report that two-thirds of such projects encounter major delays, significant cost overruns, or both. The U.S. government has found it nearly impossible to introduce or upgrade large-scale software systems: decade-long efforts at the Federal Aviation Administration and the FBI have collapsed in chaos.

Businesses have fared no better. To give a single example, McDonald's executives dreamed of a Web-based management system they called Innovate that would track the real-time flow of burgers, fries, and chicken nuggets in every one of their restaurants around the world. By the time they gave up and canceled the project, they had to write off $170 million of its estimated $1 billion total cost.

Such failures add up. Every year, according to a 2002 study by the National Institute of Standards and Technology, software failures cost $59.5 billion. But the price of bad software can also be measured in human misery--and even in lives lost. During the 1991 Gulf War, a Patriot missile battery didn't fire at an incoming Scud because of faulty software; the direct hit on a barracks killed 28 U.S. soldiers.

The past half-century of computing has seen wonderful progress. Programmers have abandoned punch cards and teletypes. They have given us a computer on every desktop, tools for work, toys for play, and a network that links homes and businesses to form a teeming global pool of information and entertainment. This progress has been fueled by the exponential curve of Moore's Law, Intel founder ­Gordon Moore's prediction that microchips' power would double (or their cost would halve) every one to two years.

But even as Moore's Law has made each year's new computers faster and cheaper, the flexibility and utility of our computer systems have been limited by the slower, uneven evolution of software. One formulation of this problem is known as Wirth's Law, after programming expert Niklaus Wirth: "Software gets slower faster than hardware gets faster."

Read more