An MS Access SQL Transformation Odyssey

Recently, we’ve added support for the MS Access database in jOOQ 3.3. This may well have been our hardest integration so far. The MS Access database has its ways, and many of them, too. But luckily, jOOQ’s internal SQL transformation abilities are already very advanced, as we’ve shown previously in a blog post about the emulation of row value expression IN predicates.

In this post here, we want to show you how we can emulate a multi-value INSERT statement transparently through various steps until it works also for MS Access.

Consider the following standard SQL-92 INSERT statement:

INSERT INTO books (author, title)
VALUES ('George Orwell', '1984'),
       ('Leo Tolstoy', 'War and Peace');

With jOOQ, we can trivially write it like this:

DSL.using(configuration)
   .insertInto(BOOKS, AUTHOR, TITLE)
   .values("George Orwell", "1984")
   .values("Leo Tolstoy", "War and Peace")
   .execute();

See the manual for more details. The above multi-record INSERT syntax is supported by a variety of databases, but not by these ones:

• Firebird
• Ingres
• MS Access
• Oracle
• SQLite
• Sybase Adaptive Server Enterprise

But luckily, the above syntax can be emulated using INSERT .. SELECT

INSERT INTO books (author, title)
SELECT 'George Orwell', '1984'
UNION ALL
SELECT 'Leo Tolstoy', 'War and Peace';

Note that some databases require a FROM clause in most SQL statements. So does MS Access. A simple way to emulate what Oracle calls DUAL with MS Access is this:

INSERT INTO books (author, title)
SELECT 'George Orwell', '1984'
FROM (SELECT count(*) FROM MSysResources) AS DUAL
UNION ALL
SELECT 'Leo Tolstoy', 'War and Peace';
FROM (SELECT count(*) FROM MSysResources) AS DUAL

For simplicity, let’s just assume that DUAL actually exists:

INSERT INTO books (author, title)
SELECT 'George Orwell', '1984'
FROM DUAL
UNION ALL
SELECT 'Leo Tolstoy', 'War and Peace';
FROM DUAL

But this syntax is not supported, again, by MS Access as you can see in the manual.

It reads:

Syntax
Multiple-record append query:

INSERT INTO target [(field1[, field2[, …]])] 
[IN externaldatabase]
SELECT field1[, field2[, …]
FROM tableexpression

tableexpression: The name of the table or tables from which records are inserted. This argument can be a single table name or a compound resulting from an INNER JOIN, LEFT JOIN, or RIGHT JOIN operation or a saved query.

There’s clearly no room for UNION ALL clauses, but we can use a “saved query” in the FROM clause. Given our original intent, this would roughly translate to:

INSERT INTO books (author, title)
SELECT *
FROM (
  SELECT 'George Orwell', '1984'
  FROM DUAL
  UNION ALL
  SELECT 'Leo Tolstoy', 'War and Peace';
  FROM DUAL
)

… or so you might think. Unfortunately, the above attempt results into this error message:

SELECT * cannot be used in an INSERT INTO query when the source or destination table contains a multi-valued field

So we need to explicitly select each column from our newly created derived table. But those columns don’t have names (yet). The standard way to assign names to a derived table’s columns is by using derived column lists, which rename both the table and all of its columns in one go. We’ve previously blogged about this here.

For our SQL statement, this means:

INSERT INTO books (author, title)
SELECT a, b
FROM (
  SELECT 'George Orwell', '1984'
  FROM DUAL
  UNION ALL
  SELECT 'Leo Tolstoy', 'War and Peace';
  FROM DUAL
) t(a, b)

By now, you might have guessed that our SQL transformation odyssey is not over yet. MS Access does not support derived column lists, and it’s in good company. These databases don’t support them either:

• H2
• Ingres
• MariaDB
• MS Access
• MySQL
• Oracle
• SQLite

This means that we need to emulate yet another SQL feature, which results in the following query:

INSERT INTO books (author, title)
SELECT a, b
FROM (
  -- This subselect defines column names
  SELECT '' AS a, '' AS b
  FROM DUAL
  WHERE 1 = 0
  UNION ALL
  -- This subselect provides our data
  SELECT *
  FROM (
    SELECT 'George Orwell', '1984'
    FROM DUAL
    UNION ALL
    SELECT 'Leo Tolstoy', 'War and Peace';
    FROM DUAL
  ) t
) t

And we’re set. Almost. Let’s substitute the actual DUAL expression back into the query

INSERT INTO books (author, title)
SELECT a, b
FROM (
  -- This subselect defines column names
  SELECT '' AS a, '' AS b
  FROM (SELECT count(*) FROM MSysResources) AS DUAL
  WHERE 1 = 0
  UNION ALL
  -- This subselect provides our data
  SELECT *
  FROM (
    SELECT 'George Orwell', '1984'
    FROM (SELECT count(*) FROM MSysResources) AS DUAL
    UNION ALL
    SELECT 'Leo Tolstoy', 'War and Peace';
    FROM (SELECT count(*) FROM MSysResources) AS DUAL
  ) t
) t

Now ain’t that a beauty!?

Real life scenario

In real life, you would probably work around this limitation in one way or another because no one manually writes (or reads and maintains) such SQL. You would probably resort to multiple executions of single-record INSERT statements, to batch processing, or whatever. But in real life, you would also support SQL Server or Oracle or some other database in addition to MS Access, and you’d constantly run into these kinds of issues which you’d have to patch manually. This can get very frustrating!

Unless, of course, you use jOOQ and forget about the above details in order to simply write the following, intuitive, standard SQL statement:

DSL.using(configuration)
   .insertInto(BOOKS, AUTHOR, TITLE)
   .values("George Orwell", "1984")
   .values("Leo Tolstoy", "War and Peace")
   .execute();

… which works precisely like this in all of jOOQ’s 16 supported RDBMS. What are you waiting for?