Skip to content

Optional url for HEAD method #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ The full code of this example is in [example/](./example/).

## Compiling

To compile this project (only needed if you want to modify the library itself), make sure you have emscripten, then first compile sql.js, then sql.js-httpvfs:
To compile this project (only needed if you want to modify the library itself), make sure you have emscripten (min. 3.1.0), then first compile sql.js, then sql.js-httpvfs:

```
cd sql.js
Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
"license": "Apache-2.0",
"scripts": {
"dev": "webpack serve --mode=development",
"build": "webpack build --mode=production"
"build": "webpack build --mode=production",
"build-sql": "cd sql.js && yarn build && cd .. && webpack build --mode=production"
},
"dependencies": {
"comlink": "^4.3.0"
Expand All @@ -30,4 +31,4 @@
"files": [
"dist/*"
]
}
}
3 changes: 1 addition & 2 deletions sql.js/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,7 @@ EMFLAGS = \
-s EXTRA_EXPORTED_RUNTIME_METHODS=@src/exported_runtime_methods.json \
-s SINGLE_FILE=0 \
-s NODEJS_CATCH_EXIT=0 \
-s NODEJS_CATCH_REJECTION=0 \
-s LEGACY_RUNTIME=1
-s NODEJS_CATCH_REJECTION=0

# -s ASYNCIFY=1 \
# -s ASYNCIFY_IMPORTS='["sqlite3VdbeExec"]'
Expand Down
4 changes: 2 additions & 2 deletions src/lazyFile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
export type RangeMapper = (
fromByte: number,
toByte: number
) => { url: string; fromByte: number; toByte: number };
) => { url: string; headUrl: string; fromByte: number; toByte: number };

export type RequestLimiter = (bytes: number) => void;

Expand Down Expand Up @@ -169,7 +169,7 @@ export class LazyUint8Array {
/** verify the server supports range requests and find out file length */
private checkServer() {
var xhr = new XMLHttpRequest();
const url = this.rangeMapper(0, 0).url;
const url = this.rangeMapper(0, 0).headUrl;
// can't set Accept-Encoding header :( https://stackoverflow.com/questions/41701849/cannot-modify-accept-encoding-with-fetch
xhr.open("HEAD", url, false);
// // maybe this will help it not use compression?
Expand Down
8 changes: 6 additions & 2 deletions src/sqlite.worker.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
/// <reference path="./types.d.ts" />

import * as Comlink from "comlink";
import { Database, QueryExecResult } from "sql.js";
import initSqlJs from "../sql.js/dist/sql-wasm.js";
import wasmUrl from "../sql.js/dist/sql-wasm.wasm";
import { createLazyFile, LazyUint8Array, PageReadLog, RangeMapper } from "./lazyFile";
import { Database, QueryExecResult } from "sql.js";
import { SeriesVtab, sqlite3_module, SqljsEmscriptenModuleType } from "./vtab";

wasmUrl;
Expand Down Expand Up @@ -71,6 +71,7 @@ export type SplitFileConfigInner = {
| {
serverMode: "full";
url: string;
headUrl?: string;
}
);
export interface LazyHttpDatabase extends Database {
Expand Down Expand Up @@ -115,6 +116,7 @@ async function fetchConfigs(
: {
...configOut,
url: new URL(configOut.url, configUrl).toString(),
headUrl: configOut.headUrl ? new URL(configOut.headUrl, configUrl).toString() : new URL(configOut.url, configUrl).toString(),
},
virtualFilename: config.virtualFilename,
} as SplitFileConfigPure;
Expand Down Expand Up @@ -171,13 +173,15 @@ const mod = {
const serverTo = serverFrom + (to - from);
return {
url: config.urlPrefix + String(serverChunkId).padStart(config.suffixLength, "0") + suffix,
headUrl: config.headUrl ?? config.urlPrefix + String(serverChunkId).padStart(config.suffixLength, "0") + suffix,
fromByte: serverFrom,
toByte: serverTo,
};
};
} else {
rangeMapper = (fromByte, toByte) => ({
url: config.url + suffix,
headUrl: config.headUrl ? config.headUrl + suffix : config.url + suffix,
fromByte,
toByte,
});
Expand All @@ -189,7 +193,7 @@ const mod = {
mainVirtualFilename = filename;
mainFileConfig = config
}
console.log("filename", filename);

console.log("constructing url database", id, "filename", filename);
const lazyFile = createLazyFile(sql.FS, "/", filename, true, true, {
rangeMapper,
Expand Down