Skip to content

Database configuration and migration #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: develop
Choose a base branch
from
Draft

Conversation

JanJakes
Copy link
Contributor

@JanJakes JanJakes commented Apr 15, 2025

This PR extracts existing SQLite database configuration to a WP_SQLite_Configurator and implements WP_SQLite_Information_Schema_Reconstructor with the ability to backfill missing table data in information schema.

How it works:

  1. The configurator checks the SQLite driver version against a version stored in the database, and if the driver is ahead, it will execute the database (re)configuration (see WP_SQLite_Configurator::ensure_database_configured()), while acquiring an EXCLUSIVE lock to prevent race conditions (multiple requests triggering the same migration).
  2. When the configurator runs, it ensures that any internal tables are created (global variables, information schema), and then it proceeds to the WP_SQLite_Information_Schema_Reconstructor.
  3. The reconstructor then compares existing SQLite tables against tables recorded in the information schema, backfills those that are missing, and removes those that no longer exist.
  4. To backfill a table when in WordPress, and it is a WordPress table, the definition is taken from wp_get_db_schema().
  5. To backfill a non-WordPress table, the information schema data is derived from _mysql_data_types_cache, if it exists, otherwise directly from the SQLite schema using SQLite column affinity rules.

@JanJakes JanJakes force-pushed the driver-migration branch 2 times, most recently from 6148a0b to 2250ed3 Compare April 15, 2025 15:13
@JanJakes JanJakes force-pushed the driver-migration branch 4 times, most recently from 7d2c0c1 to 6e94ae8 Compare April 17, 2025 13:42
@JanJakes JanJakes requested a review from adamziel April 17, 2025 14:51

jobs:
verify-version:
name: Verify plugin version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's that workflow? Let's document its purpose inline or give it a name that explains the intent

@@ -12,6 +12,12 @@
* @package wp-sqlite-integration
*/

/**
* Load the "SQLITE_DRIVER_VERSION" constant.
* This constant needs to be updated whenever the plugin version changes!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So on releases?

private $sqlite;

public static function setUpBeforeClass(): void {
// if ( ! defined( 'PDO_DEBUG' )) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: commented code

$GLOBALS['wpdb']->show_errors = true;
}

// Mock symols that are used for WordPress table reconstruction.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Mock symols that are used for WordPress table reconstruction.
// Mock symbols that are used for WordPress table reconstruction.

// In WordPress, use "wp_get_db_schema()" to reconstruct WordPress tables.
$wp_tables = array();
if ( defined( 'ABSPATH' ) ) {
if ( wp_installing() ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add some context on why would this run during wp installation

// Avoid interfering with WordPress installation.
return;
}
if ( file_exists( ABSPATH . 'wp-admin/includes/schema.php' ) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're in WordPress context and that file is missing, should that be a fatal error?

|| str_ends_with( $data_type, 'blob' )
) {
$cols[] = sprintf(
'%s(%d)',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least decimal, timestamp, and datetime fields can also have parentheses with additional parameters. Maybe others too. Should we consider them here?

}
if (
"''" === $default_value
&& in_array( strtolower( $mysql_type ), array( 'datetime', 'date', 'time', 'timestamp', 'year' ), true )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale here? Let's document it

}
throw $e;
}
if ( str_ends_with( $mysql_type, ' KEY' ) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are some values that would trigger this condition? Let's document that inline.

if ( str_contains( $type, 'INT' ) ) {
return 'int';
}
if ( str_contains( $type, 'TEXT' ) || str_contains( $type, 'CHAR' ) || str_contains( $type, 'CLOB' ) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This came up for me as I read it: CLOB? Or BLOB? Also, in another line of this file we do strtolower and test for str_ends_with, let's be consistent here if that makes sense.

Then I've remembered sqlite uses substring to match a data type. And clob is actually a char type. Still, let's document that inline. These are all non-obvious behaviors

} else {
$unquoted = $quoted_identifier;
}
return str_replace( $first_byte . $first_byte, $first_byte, $unquoted );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's only run that replace in the if branch where that first byte is a quote, otherwise we risk corrupting byte sequences such as strassberg

@adamziel
Copy link
Contributor

adamziel commented Apr 17, 2025

Great start! And also a good idea to use a cascade of data sources for backfilling. There's some more way to go, though — I've commented on a bunch of potential issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants