cross-vendor LIMIT/OFFSET in SQL

Questions and answers regarding general SQL and backend databases

cross-vendor LIMIT/OFFSET in SQL

Postby ROCLASI » Tue Jan 28, 2014 8:45 pm

At ServoyCamp3 I did a session about some advanced SQL where I showed CTE's, Window Functions and JDBC Scalar Functions.
Someone asked me if there was a cross-vendor way of doing LIMIT/TOP. I said that there was none.
Thinking about this question a little more I have to conclude I was probably wrong with my answer (to a degree).
The SQL standard has 'FETCH FIRST n ROWS' for this but that is (still) not supported by all database vendors.
But you could do it using a Window Function and as far as I know only MySQL doesn't support these (yet?).

But let me explain the situation first.
SQL Server and Sybase both use the syntax 'SELECT TOP 5 myColumn, otherColumn FROM myTable' to get the first 5 rows to the result set.
PostgreSQL, MySQL use 'SELECT myColumn, otherColumn FROM myTable LIMIT 5'. Oracle however has a special-internal column named rownum that you can use in a WHERE clause.
Now most current database versions do support the row_number() window function so we could use this.

Code: Select all
SELECT  myColumn,
        otherColumn,
        row_number() OVER (ORDER BY myPK) AS rownr
FROM myTable

This will return 3 columns where rownr is an alias for the window function.
However to filter on it we would have to use that full function in the WHERE clause which is not allowed (you get an error that window functions can not be used in the WHERE clause).
So we have to wrap this query into another to be able to filter on the rownr like so:
Code: Select all
SELECT myColumn, otherColumn FROM (
    SELECT  myColumn,
            otherColumn,
            row_number() OVER (ORDER BY myPK) AS rownr
    FROM myTable
) t
WHERE rownr <= 5

This will give us the first 5 rows of the result set.
Now if we want to get rows 6 to 10 (i.e. using an OFFSET) to do some paging through the result set we can use the same approach like so:
Code: Select all
SELECT myColumn, otherColumn FROM (
    SELECT  myColumn,
            otherColumn,
            row_number() OVER (ORDER BY myPK) AS rownr
    FROM myTable
) t
WHERE rownr >= 6 and rownr <= 10

Or you can use the BETWEEN syntax as well:
Code: Select all
SELECT myColumn, otherColumn FROM (
    SELECT  myColumn,
            otherColumn,
            row_number() OVER (ORDER BY myPK) AS rownr
    FROM myTable
) t
WHERE rownr BETWEEN 6 AND 10


There is however one important thing to keep in mind.
ALWAYS use the same ORDER BY in the row_number() Window Function as you use on the main query or else you can get unexpected results.

Hope this helps.
Robert Ivens
SAN Developer / Servoy Valued Professional / Servoy Certified Developer

ROCLASI Software Solutions / JBS Group, Partner
Mastodon: @roclasi
--
ServoyForge - Building Open Source Software.
PostgreSQL - The world's most advanced open source database.
User avatar
ROCLASI
Servoy Expert
 
Posts: 5438
Joined: Thu Oct 02, 2003 9:49 am
Location: Netherlands/Belgium

Re: cross-vendor LIMIT/OFFSET in SQL

Postby lwjwillemsen » Tue Jan 28, 2014 11:52 pm

Hi Robert,

First thanks for your very nice SQL advanced session at ServoyCamp 2014.
Secondly thanks for your after input since I raised my finger on this one.

Currently we use in our solutions some global general SQL wrapper functions to deal with some database vendor differences.
As everybody probably will know you can ask Servoy for the database vendor of a database.
Example : databaseManager.getDatabaseProductName('user_data')

But it is good to know you can achieve the same (and more) with the standard window function row_number().

Thanks again,

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 680
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: cross-vendor LIMIT/OFFSET in SQL

Postby ROCLASI » Wed Jan 29, 2014 7:48 pm

You're absolutely welcome.
By the way, if you want to know more about what database (what version and more) your connection is using you can use the JDBC-Metadata plugin I wrote over a year ago.
It's of course Open Source. ;)
Robert Ivens
SAN Developer / Servoy Valued Professional / Servoy Certified Developer

ROCLASI Software Solutions / JBS Group, Partner
Mastodon: @roclasi
--
ServoyForge - Building Open Source Software.
PostgreSQL - The world's most advanced open source database.
User avatar
ROCLASI
Servoy Expert
 
Posts: 5438
Joined: Thu Oct 02, 2003 9:49 am
Location: Netherlands/Belgium

Re: cross-vendor LIMIT/OFFSET in SQL

Postby lwjwillemsen » Wed Jan 29, 2014 10:13 pm

Robert, thanks for pointing to your plugin.
I did not know of this plugin or I forgot...

I'll tell my colleagues about this JDBC-plugin.

Ps. maybe an idea for a future (PostgreSQL) session :
the query plan analyzer from pgAdmin with those great graphs...
The analyzer gave me a lot of insight into database query performance and
when indexes were used or not.
So maybe the theme should be : "indexes and database (query) performance"
BTW : I have a nice Servoy use case...

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 680
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: cross-vendor LIMIT/OFFSET in SQL

Postby ROCLASI » Fri Feb 07, 2014 7:40 am

lwjwillemsen wrote:Ps. maybe an idea for a future (PostgreSQL) session :
the query plan analyzer from pgAdmin with those great graphs...
The analyzer gave me a lot of insight into database query performance and
when indexes were used or not.
So maybe the theme should be : "indexes and database (query) performance"
BTW : I have a nice Servoy use case...,


I was actually already working on a presentation on indexing in general but couldn't finish it in time for ServoyCamp. So I'll probably show it at the next conference.
But I am always interested in use cases. So if you want you can send it to me via a forum DM if you want.
Robert Ivens
SAN Developer / Servoy Valued Professional / Servoy Certified Developer

ROCLASI Software Solutions / JBS Group, Partner
Mastodon: @roclasi
--
ServoyForge - Building Open Source Software.
PostgreSQL - The world's most advanced open source database.
User avatar
ROCLASI
Servoy Expert
 
Posts: 5438
Joined: Thu Oct 02, 2003 9:49 am
Location: Netherlands/Belgium

Re: cross-vendor LIMIT/OFFSET in SQL

Postby lwjwillemsen » Fri Feb 07, 2014 8:46 am

Hello Robert,

Ok, I will. I'll have to make some time for it though...

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 680
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands


Return to SQL Databases

Who is online

Users browsing this forum: No registered users and 15 guests