Testing Query Response Times

  • Thread starter Thread starter Dale Fye
  • Start date Start date
D

Dale Fye

Have never really spent much time trying to optimize queries, but have some
time on my hands, and am trying to speed up a query that returns about
10,000 records.

Would like to know how anyone here goes about setting up a test, and
evaluating the results.

My first thought was to create and save the query
Then create a procedure to create a recordset from the query, move to the
end of the recordset, then capture the elapsed time between starting the
process and completing it. But would really appreciate
 
Hi Dale,

I have done something similar:

' store starting date/time

DoCmd.OpenQuery "qryXXX"
DoCmd.GoToRecord acDataQuery, "qryXXX", acLast
DoCmd.Close acQuery, "qryXXX"

' store ending date/time then use DateDiff()

Clifford Bass
 
Clifford,

The query I was looking at creates a ranking based on the value of a numeric
field. I've seen the technique below used on numerous occassions, but it
returns duplicate rankings if there is no secondary field to generate a sort
order.

SELECT tbl_RankTest.SomeNumber,
(Select Count(*) FROM tbl_RankTest as T
WHERE T.[SomeNumber] <= tbl_RankTest.[SomeNumber]) AS Rank
FROM tbl_RankTest
ORDER BY tbl_RankTest.SomeNumber;

I've also used a function (fnRank) I created a while back (shown below) in
the 2nd query (also shown below). Since this function uses a static
variable, you have to call it once to reset that variable with reset set to
true, but then when you use it in a query, and pass it any field value from
the recordset, it returns sequential numbers.

SELECT tbl_RankTest.SomeNumber,
fnRank([ID]) AS Rank
FROM tbl_RankTest
ORDER BY tbl_RankTest.SomeNumber;

Public Function fnRank(SomeValue As Variant, Optional Reset As Boolean =
False) as long

Static myRank As Long

If Reset Then
myRank = 0
Else
myRank = myRank + 1
End If
fnRank = myRank

End Function

So tonight I ran a simple test,
1. created a table with 10000 records with random numbers between 0 and
5000
2. Ran the 1st query shown above through my "evaluation subroutine" (see
discussion below)
3. Ran the 2nd query shown above through my "evaluation subroutine"
4. Indexed the field, and reran query 1
5. Reran query 2

Results :
Step2 Seconds
2 94
3 0
4 16
5 0

Obviously there is a huge reduction (83%) in the time it took to generate
the of the first, but I was amazed in the performance of the function. Did
not think it would be nearly that fast.

My only concern with my function, is that when I run the query from the
query grid, and scroll from one "page" of results to the next, it
recalculates the value in the Rank column (but it only appears to do it for
the visible rows). This behavior also shows up when I used the query
results in a form and when I used them in a report. I also noted that when
I used it in a report, if I sorted by that column, it actually runs the
query twice, so the first number in the rank column ends up being the number
of records in the result set + 1. Strange behavior.

Given the speed advantage of the query using my function over the other one,
I quess I could use a make-table or an append query to get the results into
a temp table for use in my report, then delete the records later.

Dale
 
Hi Dale,

I can easily see why the first query would be inefficient. When you
think about it, it is doing 10,000 summary queries in addition to the main
query.

It is strange the convolutions one has to go through to get a
sequential numbering of the results of a query. All, I realize, because
sequential numbering is not part of set theory.

An alternative to a temporary table would be a permanent ranking column
in the table. Then create a query that sorts the data as needed and includes
the ranking field. Then do an update query based off of the previous query
that updates the ranking using function. It would need to be done only
whenever the data changes. That in essence is something that I do in at
several cases. Although I usually have to do it in code due to the
complicatedness (is that a word) of how the rankings are determined. This
works when the rankings only need calculating on an occasional basis. If
they needed calcuation say every few minutes, then something else would have
to be done.

The recalculation of the function value is actually more of a problem
than just with reports. If you are displaying the query and click on one of
the rows it recalculates. And it redoes all of them if you Alt-Tab to
another application and back. This is something that I noticed with the
Rnd() function in queries. The constant recalcuation whenever something is
touched or sorted.

My thoughts, for what worth you may find in them,

Clifford Bass

Dale Fye said:
Clifford,

The query I was looking at creates a ranking based on the value of a numeric
field. I've seen the technique below used on numerous occassions, but it
returns duplicate rankings if there is no secondary field to generate a sort
order.

SELECT tbl_RankTest.SomeNumber,
(Select Count(*) FROM tbl_RankTest as T
WHERE T.[SomeNumber] <= tbl_RankTest.[SomeNumber]) AS Rank
FROM tbl_RankTest
ORDER BY tbl_RankTest.SomeNumber;

I've also used a function (fnRank) I created a while back (shown below) in
the 2nd query (also shown below). Since this function uses a static
variable, you have to call it once to reset that variable with reset set to
true, but then when you use it in a query, and pass it any field value from
the recordset, it returns sequential numbers.

SELECT tbl_RankTest.SomeNumber,
fnRank([ID]) AS Rank
FROM tbl_RankTest
ORDER BY tbl_RankTest.SomeNumber;

Public Function fnRank(SomeValue As Variant, Optional Reset As Boolean =
False) as long

Static myRank As Long

If Reset Then
myRank = 0
Else
myRank = myRank + 1
End If
fnRank = myRank

End Function

So tonight I ran a simple test,
1. created a table with 10000 records with random numbers between 0 and
5000
2. Ran the 1st query shown above through my "evaluation subroutine" (see
discussion below)
3. Ran the 2nd query shown above through my "evaluation subroutine"
4. Indexed the field, and reran query 1
5. Reran query 2

Results :
Step2 Seconds
2 94
3 0
4 16
5 0

Obviously there is a huge reduction (83%) in the time it took to generate
the of the first, but I was amazed in the performance of the function. Did
not think it would be nearly that fast.

My only concern with my function, is that when I run the query from the
query grid, and scroll from one "page" of results to the next, it
recalculates the value in the Rank column (but it only appears to do it for
the visible rows). This behavior also shows up when I used the query
results in a form and when I used them in a report. I also noted that when
I used it in a report, if I sorted by that column, it actually runs the
query twice, so the first number in the rank column ends up being the number
of records in the result set + 1. Strange behavior.

Given the speed advantage of the query using my function over the other one,
I quess I could use a make-table or an append query to get the results into
a temp table for use in my report, then delete the records later.

Dale
 
I wonder if this query would be faster, slower, or end up in the middle

SELECT A.SomeNumber, Count(B.SomeNumber) as Rank
FROM tbl_RankTest as A INNER JOIN tbl_RankTest as B
ON A.SomeNumber <= B.SomeNumber
GROUP BY A.SomeNumber
ORDER BY Count(B.SomeNumber);

'====================================================
John Spencer
Access MVP 2002-2005, 2007-2008
The Hilltop Institute
University of Maryland Baltimore County
'====================================================
 
John,

At first I thought that might be quicker, but the results were actually
significantly worse.

un indexed: 286
indexed: 345

Then I thought about it, and realized that what we were doing was basically
creating a almost 50 million record 10^4 x (10^4-1)/2 recordset , so it
probably isn't surprising that these took significantly longer. What was
surprising was that the query on the indexed field actually took longer than
the one on the un-indexed field.

Any ideas?

Dale
 
Hi John and Dale,

At first I thought this would have the same problem as the initial
query you used Dale. But it produces even weirder results.

The small sample data (note there there are some duplicates according to
your information):

SomeNumber
234
272
321
345
345
543
543
654
707
717
727
727
747
747
757
789
987
987

The results of the initial query, which skips some rankings and
produces some duplicates:

SomeNumber Rank
234 1
272 2
321 3
345 5
345 5
543 7
543 7
654 8
707 9
717 10
727 12
727 12
747 14
747 14
757 15
789 16
987 18
987 18

The results of John's query:

SomeNumber Rank
789 3
987 4
757 4
717 9
707 10
654 11
747 12
727 16
321 16
272 17
234 18
543 26
345 30

Now, is that odd or what?!

If I change the <= to a >= it looks a bit better:

SomeNumber Rank
234 1
272 2
321 3
654 8
707 9
717 10
345 10
543 14
757 15
789 16
727 24
747 28
987 36

So, that works except for the duplicate values. It also only displays
only one instance of those that are duplicated. If there were no duplicates
I expect it would work.

Clifford Bass
 
Back
Top