Skip to content

Conversation

@jeroenptrs
Copy link

@jeroenptrs jeroenptrs commented Feb 23, 2025

Purpose

Sparked by the conversation on #50, I wanted to provide a way to easily provide statements through SQL files.

Tasks

  • configure wrangler, vite and ts to play nice with importing .sql files as text assets (I initially just copied everything to separate .ts files, but then I remembered wrangler allows you to bundle additional files as text using rules, ref here)
  • also had to include sql files in the vitest config

Verify

  • import a .sql file containing statements in code and use it as if it were a text file
  • start a starbase instance and make sure the imported sql code is ran somewhere
  • see the changes reflected in /studio

Before

No sql file support

After

Screenshot 2025-02-23 at 16 36 38

Anything I write in this file gets reflected when the DO initializes 🙂

Comment on lines 26 to 65
// Install default necessary `tmp_` tables for various features here.
const cacheStatement = `
CREATE TABLE IF NOT EXISTS tmp_cache (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"timestamp" REAL NOT NULL,
"ttl" INTEGER NOT NULL,
"query" TEXT UNIQUE NOT NULL,
"results" TEXT
);`

const allowlistStatement = `
CREATE TABLE IF NOT EXISTS tmp_allowlist_queries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sql_statement TEXT NOT NULL,
source TEXT DEFAULT 'external'
)`
const allowlistRejectedStatement = `
CREATE TABLE IF NOT EXISTS tmp_allowlist_rejections (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sql_statement TEXT NOT NULL,
source TEXT DEFAULT 'external',
created_at TEXT DEFAULT (datetime('now'))
)`

const rlsStatement = `
CREATE TABLE IF NOT EXISTS tmp_rls_policies (
"id" INTEGER PRIMARY KEY AUTOINCREMENT,
"actions" TEXT NOT NULL CHECK(actions IN ('SELECT', 'UPDATE', 'INSERT', 'DELETE')),
"schema" TEXT,
"table" TEXT NOT NULL,
"column" TEXT NOT NULL,
"value" TEXT NOT NULL,
"value_type" TEXT NOT NULL DEFAULT 'string',
"operator" TEXT DEFAULT '='
)`

this.executeQuery({ sql: cacheStatement })
this.executeQuery({ sql: allowlistStatement })
this.executeQuery({ sql: allowlistRejectedStatement })
this.executeQuery({ sql: rlsStatement })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For what it's worth, all of these will soon be moving away from being directly built in (when possible) to being their own plugins as well as to not potentially pollute new instances with tables or features that don't make sense for a users use case.

Copy link
Member

@Brayden Brayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do very much like the idea of enabling plugins themselves to include/import .sql files into their project, that part is great.

My fear would be including two default files (builtin.sql and user.sql) like this instead of adopting a more standardized migrations implementation – ref: https://github.com/lambrospetrou/durable-utils?tab=readme-ov-file#sqlite-schema-migrations

Current suggestion would be to include the work in this PR that allows .sql files to be included but at the moment not commit to adopting a two file default for SQL statements to instantiate a DO until we take a deeper dive into what a proper migrations setup could do for us.

Are you okay with that @jeroenptrs?

@jeroenptrs
Copy link
Author

@Brayden sounds good! this stemmed from not seeing a good encapsulated way to bootstrap a database, but moving to migrations (potentially as a plugin) makes sense. Will adapt the PR in a bit!

@jeroenptrs jeroenptrs changed the title feat: allow users to bring their own SQL file feat: allow SQL files to be imported Feb 24, 2025
@jeroenptrs
Copy link
Author

@Brayden since the scope of the PR changed I also renamed it, let me know if these changes make sense!

@jeroenptrs jeroenptrs requested a review from Brayden February 24, 2025 17:35
Copy link
Member

@Brayden Brayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work 🚀

@Brayden Brayden merged commit 931269e into outerbase:main Feb 25, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants