Skip to content

Conversation

@fdcastel
Copy link
Member

@fdcastel fdcastel commented Feb 10, 2026

v4 - Firebird ODBC Driver Modernization

Builds available at https://github.com/fdcastel/firebird-odbc-driver/releases/tag/v4.0.1-preview.

Summary

This PR is a comprehensive modernization of the Firebird ODBC driver, addressing crash bugs, ODBC specification violations, Unicode data corruption on Linux/macOS, and significant performance bottlenecks. It also replaces the build system with CMake, adds a 401-test Google Test suite, and reorganizes the source tree.

The changes are organized into 4 commits for easier review:

  1. build: remove legacy build files, vendored headers, and dead code -- 383 files removed. Safe to skim; this is just deletions.
  2. feat: modernize driver -- CMake build, source reorg, fixes and improvements -- The core changes. Source moved to src/, all driver improvements applied. Git detects 137 renames so git diff -M shows only the actual code changes (~5,300 lines of real modifications).
  3. test: add comprehensive Google Test suite (401 tests, 34 suites) -- New tests only. Can be reviewed independently.
  4. docs: add project documentation and developer guidelines -- Documentation only.

Reviewing tip: For commit 2, use git diff -M or GitHub's "detect renames" option to see only the meaningful code changes rather than full file deletions + additions.


Motivation

I started this project to address real bugs I was hitting while using the Firebird ODBC driver in production. What began as a few targeted fixes grew into a larger effort as I kept finding interconnected issues -- the crash bugs led me to the GUARD macro pattern, which led me to entry-point hardening, which revealed the missing SQLSTATE mappings, and so on.

The driver has a solid foundation -- it correctly uses the Firebird OO API for connections, transactions, statements, and result sets. But it has accumulated technical debt over its 20+ year history, and some areas (Unicode handling, error mapping, safety at the API boundary) needed significant work.

I understand this is a large PR and may take time to review. I'm happy to discuss any changes, split things differently, or revert specific decisions if they don't align with the project's direction.

This work also led to a secondary project: ODBC Crusher. What began as a tool to investigate and learn more about ODBC drivers has since grown into a fully usable tool (still under active development).


What Changed (and Why)

🔴 Crash Fixes

Issue Fix
GUARD_HDESC/GUARD_HSTMT/GUARD_HDBC macros dereference handles before checking for null Added NULL_CHECK macro -- check first, then dereference
SQLCopyDesc crashes when source descriptor has no records Null guard for sour.records + early return when headCount == 0
OdbcObject::postError uses sprintf into 256-byte stack buffer Changed to snprintf with 512-byte buffer
64 unsafe C-style exception downcasts: (SQLException&)ex Replaced with direct catch (SQLException& exception) clauses
No null checks at ODBC entry points -- invalid handles cause immediate access violations Added explicit null checks; return SQL_INVALID_HANDLE

🟡 ODBC Specification Compliance

  • SQLSTATE mapping: Previously only 3 SQL error codes and ~19 ISC codes had explicit SQLSTATE mappings. Now: 121 SQLSTATE entries, 100+ ISC mappings, 130+ SQL code mappings, with dual ODBC 2.x/3.x strings.
  • SQLGetDiagRec: returned SQL_NO_DATA_FOUND (ODBC 2.x constant) instead of SQL_NO_DATA.
  • SQLGetDiagField: dereferenced StringLengthPtr without null check.
  • SQLGetInfo: wrong storage size for numeric InfoTypes; crashed on NULL InfoValuePtr.
  • SQLCloseCursor: returned SQL_SUCCESS when no cursor was open (should be SQLSTATE 24000).
  • SQLDescribeColW: returned SQL_CHAR/SQL_VARCHAR instead of SQL_WCHAR/SQL_WVARCHAR.
  • SQLSetConnectAttr: silently accepted unsupported attributes (no default error path).
  • SQLSetStmtAttr: missing cursor-state validations (24000/HY011).
  • SQL_ATTR_ODBC_VERSION: SQLGetEnvAttr always returned SQL_OV_ODBC3 regardless of what was set.
  • SQL_SCHEMA_USAGE: checked supportsCatalogsInIndexDefinitions() instead of the schema equivalent.
  • ODBC 3.8: SQL_OV_ODBC3_80 now accepted; driver reports "03.80"; SQL_ATTR_RESET_CONNECTION, SQL_GD_OUTPUT_PARAMS, SQL_ASYNC_DBC_FUNCTIONS implemented.
  • SQL_ASYNC_MODE: was incorrectly reporting SQL_AM_STATEMENT while rejecting async enable with HYC00. Now correctly reports SQL_AM_NONE.

🟢 New Features

  • Savepoint isolation: Per-statement SAVEPOINT/RELEASE/ROLLBACK TO SAVEPOINT when autoCommit=OFF. Failed statements don't corrupt the transaction.
  • IBatch (Firebird 4+): Array parameter execution now uses IBatch for a single server roundtrip instead of N individual roundtrips. Falls back to row-by-row for Firebird 3.x. Inline BLOB support via BLOB_ID_ENGINE policy.
  • SQL_GUID: CHAR(16) CHARACTER SET OCTETS (FB3) and BINARY(16) (FB4+) are mapped to SQL_GUID. All conversion methods implemented.
  • Query timeout: SQL_ATTR_QUERY_TIMEOUT implemented using IAttachment::cancelOperation(fb_cancel_raise) with a timer thread. Returns SQLSTATE HYT00 on expiry.
  • SQLCancel: Now functional -- calls cancelOperation() to interrupt in-flight operations.
  • ConnSettings: New connection string parameter to execute SQL on connect.
  • Server version detection: getServerMajorVersion()/getServerMinorVersion() for version-gated behavior (e.g., FB4+ types like INT128, DECFLOAT, BINARY/VARBINARY).
  • Connection pool reset: SQL_ATTR_RESET_CONNECTION rolls back pending transactions, closes open cursors, and resets statement attributes.

⚡ Performance

  • SRWLOCK: Replaced Win32 kernel Mutex (CreateMutex/WaitForSingleObject) with user-mode SRWLOCK on Windows. ~20ns vs ~1-2μs per lock/unlock.
  • 64-row prefetch buffer: IscResultSet fetches 64 rows at a time from Firebird, serving subsequent SQLFetch calls from the buffer. Measured: 10.88 ns/row for 10K×10 INT columns (embedded).
  • ConvertingString stack buffer: 512-byte stack buffer eliminates virtually all W-API heap allocations.
  • Native UTF-16 metadata caching: OdbcString w-cache in DescRecord -- SQLDescribeColW, SQLColAttributeW, SQLGetDescFieldW, SQLGetDescRecW, SQLGetDiagRecW, SQLGetDiagFieldW all read from cached UTF-16 strings via memcpy, bypassing ConvertingString entirely.
  • LTO: Link-Time Optimization enabled for Release builds.
  • Miscellaneous: BLOB object pooling, Value::getString buffer reuse, clearErrors fast path, std::to_chars for float→string, ODBC_FORCEINLINE on hot helpers, alignas(64) on Sqlda::buffer.

🔤 Unicode

The original Unicode implementation assumed SQLWCHAR == wchar_t, which is only true on Windows. On Linux (wchar_t = 4 bytes), this caused complete data corruption for all W-API functions.

  • Introduced ODBC_SQLWCHAR typedef (always 16-bit on all platforms).
  • All codec signatures and *ToStringW conversion functions now use SQLWCHAR* directly.
  • Unified to a single UTF-8↔UTF-16 codec (Utf16Convert.h).
  • CHARSET defaults to UTF8 when not specified in the connection string (was NONE, which used locale-dependent mbstowcs on Linux).
  • FSS codec and convVarStringSystemToStringW fixed to use SQLWCHAR* instead of wchar_t*.

🏗️ Build System & Infrastructure

  • CMake 3.20+: Replaces 14 platform-specific build configurations.
  • FetchContent: Firebird public headers fetched from GitHub (pinned to v5.0.2). Google Test and Google Benchmark fetched for testing.
  • Git-based versioning: Version extracted from tags via cmake/GetVersionFromGit.cmake.
  • WiX v5 MSI installer: installer/Product.wxs for Windows deployment.
  • CI: GitHub Actions for Windows x64 and Linux x64/ARM64, with Firebird 5.0 database for integration tests.
  • Source reorg: All source moved to src/ and src/IscDbc/.

🧹 Code Quality

  • OdbcError chain: std::vector<std::unique_ptr<OdbcError>> replaces raw linked list.
  • IscConnection::statements, IscStatement::resultSets, IscBlob::blobs: std::vector replaces LinkedList.
  • OdbcObject, OdbcError, OdbcEnv: diagnostic fields made private/protected.
  • EnvShare: Meyer's Singleton (construct-on-first-use) fixes static initialization order.
  • Concrete IscDbc classes marked final for devirtualization.
  • ~3,750 lines of dead code removed (Phase 13).
  • ATL/DTC distributed transaction support removed entirely.
  • ODBC escape sequence processing removed (SQL sent as-is to Firebird).

🗑️ What Was Removed

Component Reason
Builds/ (14 configs) Replaced by CMake
FBClient.Headers/ (60 files) Fetched via CMake FetchContent
Headers/ (SQL.H, SQLEXT.H) System SDK headers used
OdbcJdbcSetup/ (30 files) GUI not maintained; use odbcconf/odbcinst
Install/ (HTML help, InnoSetup) Replaced by WiX MSI
Res/ (5 locale files) i18n not supported
Test/, Tests/, JdbcTest/, TestInstall/ Replaced by Google Test
ChangeLog files Git history replaces these
ATL/DTC support Not maintained; Firebird doesn't support XA/DTC
ODBC escape sequences Legacy feature; SQL passthrough is standard practice

Testing

  • 401 tests across 34 Google Test suites.
  • Tests run against a real Firebird 5.0 database (both locally and in CI).
  • Test categories include: null handle safety, connection options, catalog functions, data type conversions (results and parameters), array binding, descriptors, cursors (forward-only and scrollable), error handling, ODBC 3.8 compliance, SQL_GUID, BLOBs, savepoints, multi-statement handles, query timeout, connection pool reset, Unicode (WCHAR), and more.
  • Tests gracefully skip (GTEST_SKIP()) when no database connection is available.
  • CI runs on Windows x64 and Linux x64/ARM64.

Breaking Changes

  • Build system: CMake is now required. Legacy build configurations (MSVC .sln, makefiles, MinGW, Borland) are removed.
  • Vendored headers: Firebird headers are no longer in the repository. CMake fetches them at build time.
  • Source layout: Source files moved from root to src/ and src/IscDbc/.
  • CHARSET default: When not specified in the connection string, CHARSET now defaults to UTF8 (was NONE). This is a correctness improvement -- CHARSET=NONE with locale-dependent mbstowcs was producing incorrect results on non-UTF-8 locales.
  • ODBC escape sequences: {fn}, {d}, {ts}, {oj} escape processing has been removed. SQL is sent as-is to Firebird. Applications using ODBC escapes should switch to native Firebird SQL syntax.
  • ATL/DTC: Distributed transaction support (ATL COM) has been removed.
  • OdbcJdbcSetup GUI: The setup dialog has been removed. Use odbcconf or odbcinst for driver registration.

Transparency Note

This work was substantially assisted by AI (LLMs). I believe in being upfront about this: the AI was used for code generation, analysis, testing, and documentation throughout this project. Every change was reviewed, tested against a real Firebird database, and verified on CI. The detailed issue tracking in Docs/FIREBIRD_ODBC_MASTER_PLAN.md documents the reasoning behind each change.

I believe the results speak for themselves: 401 passing tests, crash bugs eliminated, correct SQLSTATE mapping, functional Unicode on Linux, and measurable performance improvements. But I understand if the use of AI tooling is a concern, and I'm happy to discuss any specific changes in detail.

Remove obsolete infrastructure accumulated over 20+ years:

Build systems (Builds/):
- 14 platform-specific configurations: MSVC 6.0-2022, MinGW, Borland C++,
  GCC for Linux/macOS/FreeBSD/Solaris/AIX/HP-UX
- Replaced by CMake (added in following commit)

Vendored headers:
- FBClient.Headers/: 60 Firebird header files (~25K lines total, including
  IdlFbInterfaces.h). Now fetched via CMake FetchContent at build time
- Headers/: Microsoft ODBC SDK (SQL.H, SQLEXT.H). System SDK headers used

Removed components:
- OdbcJdbcSetup/: GUI setup dialog (30 files) — use odbcconf/odbcinst
- Install/: HTML help (5 languages), InnoSetup scripts — replaced by WiX
- Test/ + Tests/ + JdbcTest/ + TestInstall/: old test infra — replaced by
  Google Test suite
- Res/: locale resource files (5 languages) — i18n not supported

Dead code from driver source:
- LinkedList.cpp/.h: replaced by std::vector (Phase 5)
- Lock.cpp/.h: unused RAII wrapper
- ServiceManager.cpp/.h: backup/restore GUI support (880 lines, no callers)
- SupportFunctions.cpp/.h: ODBC escape sequences (removed by design)
- Engine.h: obsolete macros conflicting with <algorithm>
- IscOdbcStatement.cpp: merged into IscStatement
- LoadFbClientDll.h: replaced by new version in src/IscDbc/
- OdbcDateTime.cpp/.h: replaced by FbDateConvert.h
- TransactionResourceAsync.cpp/.h: ATL/DTC support (removed)
- WriteBuildNo.h: replaced by git-based versioning
- Build artifacts: .exp, .manifest, MinGW .def

Old CI: msbuild.yml, linux.yml, arm64 variants
ChangeLog files: ~4,600 lines (git history replaces these)
…ments

This commit contains all driver source code changes, the new CMake build
system, and CI/CD infrastructure. Source files are moved from the root
into src/ and src/IscDbc/ subdirectories.

BUILD SYSTEM:
- CMake 3.20+ replaces all legacy build systems
- Firebird headers fetched via FetchContent (pinned to v5.0.2)
- Google Test/Benchmark fetched via FetchContent for testing
- LTO enabled for Release builds; SRWLOCK on Windows
- WiX v5 MSI installer (installer/Product.wxs)
- Git-based versioning (cmake/GetVersionFromGit.cmake)
- Build script: firebird-odbc-driver.build.ps1 (Invoke-Build)
- CI: GitHub Actions for Windows x64 + Linux x64/ARM64

CRASH FIXES (Phase 0):
- GUARD_HDESC/HSTMT/HDBC macros: null check before dereference
- All ODBC entry points: null handle checks (SQL_INVALID_HANDLE)
- postError: snprintf with 512-byte buffer (was sprintf into 256)
- 64 unsafe C-style exception casts replaced with catch clauses

ODBC SPEC COMPLIANCE (Phase 1):
- ISC→SQLSTATE mapping: 121 states, 100+ ISC codes, 130+ SQL codes
- Dual ODBC 2.x/3.x SQLSTATE support (getVersionedSqlState)
- SQLGetDiagRec: returns SQL_NO_DATA (not SQL_NO_DATA_FOUND)
- SQLGetDiagField: null StringLengthPtr check
- SQLGetInfo: fixed numeric storage and NULL InfoValuePtr handling
- SQLCloseCursor: returns 24000 when no cursor open
- SQLDescribeColW: returns SQL_WCHAR/SQL_WVARCHAR (not SQL_CHAR)
- SQLSetConnectAttr: default error path for unknown attributes
- SQLSetStmtAttr: cursor-state validations (24000/HY011)
- BufferLength validation for W APIs (HY090 for odd values)

ENTRY POINT HARDENING (Phase 2):
- try/catch on 9 methods missing exception handling
- clearErrors() at all entry points
- Thread safety always enabled (removed DRIVER_LOCKED_LEVEL_NONE)

FEATURES (Phases 4, 8, 9, 11):
- Savepoint isolation: per-statement savepoint/rollback in autoCommit=OFF
- Server version detection: getServerMajorVersion/Minor for FB3/4/5
- ConnSettings: SQL executed on connect via connection string param
- Array parameter binding: column-wise and row-wise, operation ptr
- IBatch: single server roundtrip for batch params on FB4+ (with
  inline BLOB support via BLOB_ID_ENGINE policy)
- SQL_GUID: mapped from CHAR(16) OCTETS / BINARY(16), with all
  conversion methods (GUID↔string, GUID↔binary)
- ODBC 3.8: SQL_OV_ODBC3_80, SQL_ATTR_RESET_CONNECTION, driver
  version "03.80", SQL_GD_OUTPUT_PARAMS, SQL_ASYNC_DBC_FUNCTIONS
- SQLGetTypeInfo: thread-safe (per-instance copy), sorted by
  DATA_TYPE, multi-row returns, FB4+ type dedup
- Query timeout: timer-based cancelOperation(fb_cancel_raise),
  SQLSTATE HYT00 on expiry
- SQLCancel: functional via cancelOperation
- Connection pool reset: rollback pending TX, close cursors, reset
  statement attributes

OO API MODERNIZATION (Phase 9):
- ~35 dead ISC function pointers removed from CFbDll
- IscUserEvents migrated from isc_que_events to IAttachment::queEvents
- TPB construction via IXpbBuilder (replaces manual byte-stuffing)
- Julian-day date math replaced by shared FbDateConvert.h helpers
- Error handling unified: both OO and legacy use IUtil::formatStatus
- isc_vax_integer replaced by inline helper
- Concrete IscDbc classes marked final (devirtualization)

PERFORMANCE (Phase 10):
- SRWLOCK replaces Win32 kernel Mutex (~20ns vs ~1-2μs)
- 64-row prefetch buffer in IscResultSet (amortizes fetch overhead)
- ConvertingString: 512-byte stack buffer (eliminates heap allocs)
- BLOB object pooling per result set
- Value::getString buffer reuse
- clearErrors fast path with [[likely]]
- std::to_chars for float→string conversion
- Sqlda metadata rebuild skip on re-execute
- ODBC_FORCEINLINE on hot conversion helpers
- alignas(64) on Sqlda::buffer

UNICODE (Phase 12):
- ODBC_SQLWCHAR typedef (always 16-bit on all platforms)
- All codec signatures use ODBC_SQLWCHAR* (not wchar_t*)
- *ToStringW functions use SQLWCHAR* directly (fixes Linux)
- Single UTF-8↔UTF-16 codec (Utf16Convert.h)
- OdbcString class: UTF-16-native metadata caching
- Direct W-API output: SQLDescribeColW, SQLColAttributeW,
  SQLGetDescFieldW, SQLGetDescRecW, SQLGetDiagRecW, SQLGetDiagFieldW
  bypass ConvertingString entirely (zero-copy from cached OdbcString)
- CHARSET defaults to UTF8 when not specified
- Linux CHARSET=NONE uses UTF-8 codec (not locale-dependent)

CODE QUALITY (Phases 5, 13):
- OdbcError: std::vector<unique_ptr> replaces raw linked list
- IscConnection/Statement/ResultSet: std::vector replaces LinkedList
- OdbcObject/OdbcError/OdbcEnv: private/protected visibility
- EnvShare: Meyer's Singleton (construct-on-first-use)
- returnStringInfo: SQLINTEGER* delegates to SQLSMALLINT* overload
- snprintf/swprintf macro guards for MSVC >= 1900
- .clang-format configuration
- i18n code removed from ConnectDialog.cpp
New test suite using Google Test, replacing the old MSBuild/VSTest-based
tests. Tests cover the full ODBC API surface against a real Firebird 5.0
database, with graceful GTEST_SKIP() when no connection is available.

Test organization (by topic):
- test_null_handles.cpp: 65 tests via direct DLL loading (no Driver
  Manager) — every ODBC function with NULL/invalid handles
- test_connect_options.cpp: connection attributes, timeout, async mode,
  query timeout, connection reset (36 tests)
- test_catalogfunctions.cpp: all 12 catalog functions + type info
  ordering and version-gated types (29 tests)
- test_result_conversions.cpp: data type conversions in results — INT,
  BIGINT, FLOAT, DOUBLE, NUMERIC, VARCHAR, DATE, TIME, TIMESTAMP,
  truncation, NULL, cross-type (35 tests)
- test_param_conversions.cpp: parameter type conversions (18 tests)
- test_data_types.cpp: SMALLINT through TIMESTAMP, precision, scale,
  parameterized insert/select (18 tests)
- test_array_binding.cpp: ODBC Array of Parameter Values — column-wise,
  row-wise, NULL handling, operation ptr, 1000-row, multi-type (17 tests)
- test_errors.cpp: error handling, SQLSTATE mapping, diagnostic row
  count, truncation indicators (18 tests)
- test_descriptor.cpp: IRD/ARD/IPD, SQLGetDescRec, SQLCopyDesc, desc
  field count, copy-desc crash fix (13 tests)
- test_odbc38_compliance.cpp: ODBC 3.8 features — env version, driver
  version, getdata extensions, async DBC, reset connection (12 tests)
- test_guid_and_binary.cpp: SQL_GUID mapping, UUID roundtrip, type
  coverage, FB4+ types (14 tests)
- test_prepare.cpp: SQLPrepare/Execute lifecycle (10 tests)
- test_cursors.cpp: cursor behavior, commit/rollback, close/re-exec (7)
- test_cursor_commit.cpp: cursor behavior across transactions (6 tests)
- test_cursor_name.cpp: set/get cursor name, truncation, duplicates (9)
- test_scrollable_cursor.cpp: all fetch orientations (9 tests)
- test_stmthandles.cpp: 100+ simultaneous handles, interleaving (4)
- test_multi_statement.cpp: multi-handle on one connection (4 tests)
- test_blob.cpp: small/large/null BLOB read/write (3 tests)
- test_savepoint.cpp: savepoint isolation across errors (4 tests)
- test_escape_sequences.cpp: passthrough verification (6 tests)
- test_data_at_execution.cpp: SQL_DATA_AT_EXEC / SQLPutData (6 tests)
- test_bindcol.cpp: dynamic unbind/rebind mid-fetch (5 tests)
- test_wchar.cpp: SQL_C_WCHAR bind/fetch, truncation, empty (8 tests)
- test_descrec.cpp: SQLGetDescRec for all column types (10 tests)
- test_conn_settings.cpp: ConnSettings connection param (3 tests)
- test_server_version.cpp: server version detection (4 tests)
- test_odbc_string.cpp: OdbcString UTF-16 class unit tests (26 tests)
- bench_fetch.cpp: Google Benchmark for fetch throughput

Infrastructure:
- test_main.cpp: shared GTest main with connection string from env
- test_helpers.h: FirebirdODBCTest fixture, SQLWCHAR comparison helpers
- tests/CMakeLists.txt: GTest + GBenchmark via FetchContent
- Docs/FIREBIRD_ODBC_MASTER_PLAN.md: comprehensive registry of all
  known issues, architectural analysis, improvement roadmap, and
  implementation status. Serves as the single source of truth.
- Docs/firebird-api.MD: Firebird OO API reference for driver authors
- Docs/firebird-driver-feature-map.md: feature comparison matrix
- Docs/CONVERSION_MATRIX.md: OdbcConvert dispatch table documentation
- Docs/PERFORMANCE_RESULTS.md: benchmark methodology and results
- AGENTS.md: guidelines for AI-assisted development on this project
@irodushka
Copy link
Contributor

Oh my Lord...
@fdcastel you did a great job, didn't you?..
Do you really need any additional test cases assuming you aleady have >400 tests?))))

@mrotteveel
Copy link
Member

ODBC escape sequences: {fn}, {d}, {ts}, {oj} escape processing has been removed. SQL is sent as-is to Firebird. Applications using ODBC escapes should switch to native Firebird SQL syntax.

Isn't support of those escapes required for ODBC compliance? Or did ODBC do away with them?

@irodushka
Copy link
Contributor

ODBC escape sequences: {fn}, {d}, {ts}, {oj} escape processing has been removed. SQL is sent as-is to Firebird. Applications using ODBC escapes should switch to native Firebird SQL syntax.

Isn't support of those escapes required for ODBC compliance? Or did ODBC do away with them?

Agree with @mrotteveel

What's the reason to drop this logic? I remember fixing an issue with these escapes a while back - #207. It was 2.5 years ago only, so we can suggest this function is on demand and is being used.

@mrotteveel
Copy link
Member

Savepoint isolation: Per-statement SAVEPOINT/RELEASE/ROLLBACK TO SAVEPOINT when autoCommit=OFF. Failed statements don't corrupt the transaction.

This shouldn't be necessary. As far as I know, the engine already uses server-side savepoints to undo work on statement failure. A statement failure does not "corrupt" a transaction (and if it does, that would be a bug that should be fixed in the engine, not in a client).

If execution of an individual statement fails, all its effects are already undone, so trying to do the same client-side is just unnecessary overhead. Using savepoints client-side only makes sense when covering multiple executions.

@mrotteveel
Copy link
Member

ATL/DTC distributed transaction support removed entirely.

I'm not so familiar with the ODBC landscape, but is there a specific reason for this (e.g. is this deprecated technology?). Firebird supports real two-phase commits, and thus support for distributed transaction should be available, I think.

@mrotteveel
Copy link
Member

OdbcJdbcSetup GUI: The setup dialog has been removed. Use odbcconf or odbcinst for driver registration.

Isn't this the thing that is shown in the ODBC Data Source Administrator in Windows? If so, it should remain in place.

@irodushka
Copy link
Contributor

OdbcJdbcSetup GUI: The setup dialog has been removed. Use odbcconf or odbcinst for driver registration.

Isn't this the thing that is shown in the ODBC Data Source Administrator in Windows? If so, it should remain in place.

+1! Naturally it is! Use odbcconf or odbcinst for driver registration. - that's a great recommendation for MSOffice users)))
Well, maybe it's a good idea to replace by-pixel control drawing (CDsnDialog::DoModal()) with smth more advanced, like QT or smth, but remove it?...

@fdcastel
Copy link
Member Author

Thanks for the feedback, guys! I will address them soon. 👍🏻

Do you really need any additional test cases assuming you aleady have >400 tests?))))

Well... There's always room for improvement 😄

@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

Isn't support of those escapes required for ODBC compliance? Or did ODBC do away with them?

There's actually a choice, one that Microsoft calls "DBMS-specific grammar":

https://learn.microsoft.com/en-us/sql/odbc/reference/develop-app/choosing-an-sql-grammar

Therefore, there are really two choices of grammar to use: the SQL-92 grammar (and the ODBC escape sequences) and a DBMS-specific grammar. Of the two, only the SQL-92 grammar is interoperable, so all interoperable applications should use it. Applications that are not interoperable can use the SQL-92 grammar or a DBMS-specific grammar. DBMS-specific grammars have two advantages: They can exploit any features not covered by SQL-92, and they are marginally faster because the driver does not have to modify them.

I'm using the v4-preview driver alongside a few other tools, and so far nothing has raised any issues.

I removed this primarily for performance reasons -- and to make follow-on work (of reducing reallocations and string transliterations) simpler. If there’s a strong case for keeping it, we can add it back.

My two cents: this is a legacy feature from the early days of ODBC, when the goal was to make applications fully database-agnostic. I took the introduction of “DBMS-specific grammar” above as a signal from Microsoft that this level of abstraction isn’t really necessary anymore.

@fdcastel
Copy link
Member Author

Savepoint isolation: Per-statement SAVEPOINT/RELEASE/ROLLBACK TO SAVEPOINT when autoCommit=OFF. Failed statements don't corrupt the transaction.

This shouldn't be necessary. As far as I know, the engine already uses server-side savepoints to undo work on statement failure.

Mark, you’re right. I’m also finding this statement very strange. I’ll investigate it. It seems like it may be a misinterpretation of what was done when savepoint support was added.

@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

ATL/DTC distributed transaction support removed entirely.

I'm not so familiar with the ODBC landscape, but is there a specific reason for this (e.g. is this deprecated technology?). Firebird supports real two-phase commits, and thus support for distributed transaction should be available, I think.

This was another of those "this is making things complicated so let's cut it for now" moments during the CMake migration.

ATL is a very old Windows technology, and it was making it harder to keep the codebase truly multiplatform.

  • Yes, the code WAS wrapped in #ifdefs. But it still was giving me headaches. And I was aiming for the simplest path -- at least until everything was working again.

Another reason is that I didn't see any ATL usage in the PostgreSQL ODBC Driver (which I used as a reference). On a quick skim, it looks like they DO support distributed transactions -- so if we decide to bring them back, we shouldn’t need ATL to do it.

My two cents: in modern architectures, distributed transactions are typically handled at the application level (e.g., saga patterns, event sourcing) rather than via DTC. In 25+ years of using Firebird, I particularly never once needed them.

So, if it's genuinely needed, we can bring it back -- but this would increase the code complexity for the sake of benefiting only a tiny fraction of Windows users.

@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

OdbcJdbcSetup GUI: The setup dialog has been removed. Use odbcconf or odbcinst for driver registration.

Isn't this the thing that is shown in the ODBC Data Source Administrator in Windows? If so, it should remain in place.

You're both right, folks!

  1. I mistakenly assumed the driver was fine because I'd only tested the main window of ODBC Administrator:

    image
  2. But after your comments, I tried adding a User DSN and... 💣

    image

I'll dig into how we can bring that support back. Ideally in a modern way, if possible.
but no, @irodushka... definitely NOT using Qt! 😄

@fdcastel
Copy link
Member Author

Well, maybe it's a good idea to replace by-pixel control drawing (CDsnDialog::DoModal()) with smth more advanced, like QT or smth, but remove it?...

Sorry guys. At first I genuinely thought @irodushka was joking.

But then I realized I know basically NOTHING about ODBC drivers outside of Windows.

Is there an equivalent to the ODBC Administrator on Linux or macOS?

  • Are those DSN setup forms really Windows-only (as I assumed)?
  • or do we actually need something similar on other platforms too?

@fdcastel fdcastel marked this pull request as draft February 12, 2026 01:31
@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

I'm converting this PR to a draft for now. Thank you VERY much for all the precious insights, guys!

I'm going back to the drawing board. For two (great) reasons:

Adopting fb-cpp could potentially shrink this codebase even further -- possibly eliminating the entire IscDbc folder (unless I’m overlooking something).

Also: C++20 and vcpkg for dependencies! ❤️

@mrotteveel
Copy link
Member

mrotteveel commented Feb 12, 2026

Isn't support of those escapes required for ODBC compliance? Or did ODBC do away with them?

There's actually a choice, one that Microsoft calls "DBMS-specific grammar":

https://learn.microsoft.com/en-us/sql/odbc/reference/develop-app/choosing-an-sql-grammar

That is a choice for an application developer, not an ODBC driver developer.

And that section explicitly says (emphasis mine):

Drivers must modify SQL-92 SQL and the ODBC-defined escape sequences to DBMS-specific SQL.

And

Because most SQL grammars are based on one or more of the various standards, most drivers do little or no work to meet this requirement. It often consists only of searching for the escape sequences defined by ODBC and replacing them with DBMS-specific grammar. When a driver encounters grammar it does not recognize, it assumes the grammar is DBMS-specific and passes the SQL statement without modification to the data source for execution.

In other words, support for escape sequences is required.

I'm using the v4-preview driver alongside a few other tools, and so far nothing has raised any issues.

I removed this primarily for performance reasons -- and to make follow-on work (of reducing reallocations and string transliterations) simpler. If there’s a strong case for keeping it, we can add it back.

My two cents: this is a legacy feature from the early days of ODBC, when the goal was to make applications fully database-agnostic. I took the introduction of “DBMS-specific grammar” above as a signal from Microsoft that this level of abstraction isn’t really necessary anymore.

They are a part of the ODBC specification, and as far as I can tell, drivers are required to support them. See above, and ODBC and the Standard CLI, so removing them is not an option.

For example, SQLPrepare Function says (emphasis mine):

The driver can modify the statement to use the form of SQL used by the data source and then submit it to the data source for preparation. In particular, the driver modifies the escape sequences used to define SQL syntax for certain features.

That is phrased unconditionally. Similar phrasing is used in SQLExecDirect Function.

Maybe you haven't seen applications that use them, that doesn't mean that they're not out there. And especially given Firebird is an underdog, removing features that can bridge the gap in syntactic difference sounds like a big no to me.

@mrotteveel
Copy link
Member

mrotteveel commented Feb 12, 2026

Is there an equivalent to the ODBC Administrator on Linux or macOS?

I believe unixODBC also has a configuration GUI (ODBCConfig), but I'm not familiar with it.

  • Are those DSN setup forms really Windows-only (as I assumed)?

Looking at https://www.unixodbc.org/doc/UserManual/ it has a form, but I'm not sure how it's populated (and that page looks like it hasn't been touched since the turn of the century, with references to Star Office 5 and the GUI screenshots).

  • or do we actually need something similar on other platforms too?

I guess (but again, I'm not that familiar with ODBC!)

@mrotteveel
Copy link
Member

Thanks for the feedback, guys! I will address them soon. 👍🏻

Do you really need any additional test cases assuming you aleady have >400 tests?))))

Well... There's always room for improvement 😄

For comparison, Jaybird has over 10,000 tests.

@fdcastel
Copy link
Member Author

That is a choice for an application developer, not an ODBC driver developer.

You are correct, Mark.

It seems that in my eagerness to bring simplicity and finalize the tasks, I misinterpreted the Microsoft documentation.

I'll bring them back. Ideally, with a more modern parser. 👍🏻

@irodushka
Copy link
Contributor

irodushka commented Feb 12, 2026 via email

@fdcastel
Copy link
Member Author

the code that works and was fine tested about 2 years ago - see the link to the ticket above.

Sorry, what link / ticket?

Also, I’ll be in dire need of those tests, now. Reintegrating the old code at this point would probably be a no-go.

@mrotteveel
Copy link
Member

the code that works and was fine tested about 2 years ago - see the link to the ticket above.

Sorry, what link / ticket?

#207 as mentioned in this comment

@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

#207 as mentioned in this comment

Thanks, Mark! My oversight.

P.S. This tool would be enormously useful!

UPDATE: Damn! It's a GUI! 😅

@irodushka
Copy link
Contributor

irodushka commented Feb 12, 2026 via email

@mrotteveel
Copy link
Member

I think that this might be a little too much change for a single pull request (but ultimately, that is for @irodushka to judge)

@fdcastel
Copy link
Member Author

Please clarify this for me. Do you mean that removing this code (processing of escape characters) cannot be undone in your PR?

Since this change was made early in the project -- before we removed dead code and performed other optimizations -- unfortunately, that’s correct.

If you strongly prefer keeping the original implementation, we can certainly revisit that approach. It would require additional time, but it’s doable.

That said, I believe the most effective path forward is to build a new parser and, with the author's permission, leverage the test suite from ODBCQueryTool/SQLComponents.

@fdcastel
Copy link
Member Author

fdcastel commented Feb 12, 2026

I think that this might be a little too much change for a single pull request (but ultimately, that is for @irodushka to judge)

It SURELY is.

Honestly, I’m not entirely happy with how things turned out either.

As I mentioned in the original post, I initially set out to fix just a few small pieces. However, it quickly evolved into something much larger, to the point where splitting it into smaller PRs was no longer feasible.

My two cents: I’m not attached to the specific implementation. What matters to me is passing tests and maintaining solid code coverage. As long as those are in place, the code itself can be refactored or replaced as needed.

@irodushka
Copy link
Contributor

irodushka commented Feb 12, 2026 via email

@fdcastel
Copy link
Member Author

I FULLY understand you, @irodushka -- I’d probably do the same if I were in your shoes.

Enjoy the festivities! I’ll brainstorm something on my end during Carnival, here! 😄

@fdcastel
Copy link
Member Author

I've been thinking about how we should move forward with this PR, and I'd like to propose the following approach.

As always, opinions, criticism, and suggestions are very welcome.

General Strategy

  • Do not merge this PR.
    Instead, we keep it open as a reference for the intended end state -- a kind of “target vision” for where we want to go.

  • I will continue maintaining this branch (let's call it “unofficial-v4-preview”) for internal use and possibly for selected production scenarios.

    • This way, I'll be acting as the primary beta tester for it.
    • Of course, I'll provide unofficial builds for anyone interested in testing it.
  • In parallel, and with @irodushka's guidance, I will start submitting separate, well-scoped PRs, each focused on a clearly defined feature or improvement (to be agreed upon together).

    • The goal is to incrementally achieve the same end result as this large PR -- excluding the problematic decisions(*) -- in a way that is easier to review, validate, and merge.

(*) Some decisions in the current PR were mistakes (e.g., removing ODBC escape sequences and removing the ODBC Administrator UI forms). These should not be carried forward.

Initial Breakdown

The following outlines an initial plan for how we should approach this problem.

Each item will be submitted as a separate PR, potentially structured as a sequence of dependent commits (building on one another).

1. Build System

  • Port the build system to CMake.
  • Remove legacy build configurations (MSVC .sln, makefiles, MinGW, Borland).

2. Test Suite

  • Port the test suite only. Many tests will initially fail.
    • That's acceptable. It will at least give us a clear target to work toward.
  • Add any additional tests that @irodushka may have.
  • Add ODBCQueryTool tests for escape sequences (and any others considered useful).

3. GitHub Actions automation

  • Adopt GitHub Actions for build and tests
    • Ideally, also for releases -- though that decision belongs to @irodushka. It's not required for now.

The changes above are the most necessary. Without them, the remaining tasks would require significant manual testing and verification effort, making progress inefficient and resource-intensive.

4. Next Steps: Following the Original Plan -- Without the Bad Parts (*)

From this point on, we largely follow the original roadmap, excluding the problematic decisions:

5. Final Cleanup (Very Last Step)

  • Reorganize source layout: Move source files from the repository root into src/.
  • Remove dead code.

@irodushka
Copy link
Contributor

Hi

@fdcastel - agree with you plan. This is exactly what I meant yesterday between the lines of my message%)
Except for one point - I think it's better to implement a test suite first. This way we get the basic point of testing coverage and success/failure stats AS IS, before any futher changes/fixes/enhancements are made. And, very important for me - we get a tool to check regress in the next steps.

What do you think about the following plan? -

DevOps phase

So, in the first step we should get a test suite (with some test cases that are expected to fail) and it should be run manually.

The 2nd step - a new build system. Well, I dont think removing the last actual sln (MsVc2022.win folder) is reasonable. All other *.win solutions - ok, they are outdated, but MsVc2022.win should be kept for the customers who prefer to build from the native Visual Studio sln.
+++ Remove Firebird headers from the repository task - seems reasonable to include this in this step.
Testing: no regress and no enhancements are expected at this stage.

And the 3rd step - a new CI (GitHib actions). At this stage, we move from manual launch of test suite to automated.
Testing: still, no regress and no enhancements are expected at this stage.

Bugfix phase

Here we should fix the known errors and misbehavior cases. Every case must be issued as a separate ticket, branch and PR.
Testing: No regression, but improvements are expected - some of the previously failed tests should pass. If there's no test coverage for a given case, additional tests should be created.

Improvement phase (new features, refactoring & optimizations)

[to do closer to the point]

@irodushka
Copy link
Contributor

Well, maybe it's a good idea to replace by-pixel control drawing (CDsnDialog::DoModal()) with smth more advanced, like QT or smth, but remove it?...

Sorry guys. At first I genuinely thought @irodushka was joking.

But then I realized I know basically NOTHING about ODBC drivers outside of Windows.

Is there an equivalent to the ODBC Administrator on Linux or macOS?

  • Are those DSN setup forms really Windows-only (as I assumed)?
  • or do we actually need something similar on other platforms too?

OdbcJdbcSetup stuff is for Windows only. Without that dialog you have a chance to setup ODBC by RegEdit, but I never give you such an advice))

On Linux there is ODBCConfig tool (Qt-based, as you like, @fdcastel %) but I never use it, preferring to configure the ODBC drivers/DSNs manually in /etc/

Frankly speaking, have no idea how it works on Mac. I'd guess the unixODBC approach would work, given macOS's Unix base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants