Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mops watch #254

Merged
merged 19 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions backend/main/registry/getDefaultPackages.mo
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ module {
case ("0.20.2") [("base", "0.11.1")];
case ("0.21.0") [("base", "0.11.1")];
case ("0.22.0") [("base", "0.11.2")];
case ("0.23.0") [("base", "0.11.2")];
case (_) {
switch (registry.getHighestVersion("base")) {
case (?ver) [("base", ver)];
Expand Down
2 changes: 1 addition & 1 deletion cli/check-requirements.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export async function checkRequirements({verbose = false} = {}) {
let config = readConfig();
let mocVersion = config.toolchain?.moc;
if (!mocVersion) {
mocVersion = getMocVersion();
mocVersion = getMocVersion(false);
}
if (!mocVersion) {
return;
Expand Down
18 changes: 18 additions & 0 deletions cli/cli.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import process from 'node:process';
import fs from 'node:fs';
import events from 'node:events';
import {Command, Argument, Option} from 'commander';

import {init} from './commands/init.js';
Expand All @@ -25,6 +26,7 @@ import {toolchain} from './commands/toolchain/index.js';
import {Tool} from './types.js';
import * as self from './commands/self.js';
import {resolvePackages} from './resolve-packages.js';
import {watch} from './commands/watch/watch.js';

declare global {
// eslint-disable-next-line no-var
Expand All @@ -33,6 +35,8 @@ declare global {
var mopsReplicaTestRunning : boolean;
}

events.setMaxListeners(20);

let networkFile = getNetworkFile();
if (fs.existsSync(networkFile)) {
globalThis.MOPS_NETWORK = fs.readFileSync(networkFile).toString() || 'ic';
Expand Down Expand Up @@ -392,4 +396,18 @@ selfCommand

program.addCommand(selfCommand);

// watch
program
.command('watch')
.description('Watch *.mo files and check for syntax errors, warnings, run tests, generate declarations and deploy canisters')
.option('-e, --error', 'Check Motoko canisters or *.mo files for syntax errors')
.option('-w, --warning', 'Check Motoko canisters or *.mo files for warnings')
.option('-t, --test', 'Run tests')
.option('-g, --generate', 'Generate declarations for Motoko canisters')
.option('-d, --deploy', 'Deploy Motoko canisters')
.action(async (options) => {
checkConfigFile(true);
await watch(options);
});

program.parse();
2 changes: 1 addition & 1 deletion cli/commands/bench.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ export async function bench(filter = '', optionsArg : Partial<BenchOptions> = {}
replica: config.toolchain?.['pocket-ic'] ? 'pocket-ic' : 'dfx',
replicaVersion: '',
compiler: 'moc',
compilerVersion: getMocVersion(),
compilerVersion: getMocVersion(true),
gc: 'copying',
forceGc: true,
save: false,
Expand Down
36 changes: 33 additions & 3 deletions cli/commands/replica.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {ChildProcessWithoutNullStreams, execSync, spawn} from 'node:child_proces
import path from 'node:path';
import fs from 'node:fs';
import {PassThrough} from 'node:stream';
import {promisify} from 'node:util';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Incorrect usage of promisify with spawn function

The child_process.spawn function does not follow the standard Node.js callback style and returns a ChildProcess instance, not a function that accepts a callback. Therefore, using promisify on spawn will not work as expected. This issue affects the import at line 6 and the usages at lines 148-154 and 159-165. Instead, consider creating a custom Promise-based wrapper around spawn to handle asynchronous execution and error handling properly.

To fix this, remove the import of promisify and implement a custom helper function:

-import {promisify} from 'node:util';

...

+import {SpawnOptions} from 'node:child_process';

+function spawnAsync(command: string, args: string[], options: SpawnOptions & {signal?: AbortSignal}): Promise<{ stdout: string; stderr: string; }> {
+  return new Promise((resolve, reject) => {
+    const child = spawn(command, args, options);
+    let stdout = '';
+    let stderr = '';
+    if (child.stdout) {
+      child.stdout.on('data', (data) => {
+        stdout += data.toString();
+      });
+    }
+    if (child.stderr) {
+      child.stderr.on('data', (data) => {
+        stderr += data.toString();
+      });
+    }
+    child.on('close', (code) => {
+      if (code === 0) {
+        resolve({ stdout, stderr });
+      } else {
+        const error = new Error(`Command failed with exit code ${code}: ${stderr}`);
+        (error as any).code = code;
+        reject(error);
+      }
+    });
+    child.on('error', (error) => {
+      reject(error);
+    });
+    if (options.signal) {
+      options.signal.addEventListener('abort', () => {
+        child.kill();
+      });
+    }
+  });
+}

...

- await promisify(spawn)('dfx', ['deploy', name, '--mode', 'reinstall', '--yes', '--identity', 'anonymous'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
+ await spawnAsync('dfx', ['deploy', name, '--mode', 'reinstall', '--yes', '--identity', 'anonymous'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
    if (error.code === 'ABORT_ERR') {
      return {stderr: ''};
    }
    throw error;
  });

...

- await promisify(spawn)('dfx', ['ledger', 'fabricate-cycles', '--canister', name, '--t', '100'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
+ await spawnAsync('dfx', ['ledger', 'fabricate-cycles', '--canister', name, '--t', '100'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
    if (error.code === 'ABORT_ERR') {
      return {stderr: ''};
    }
    throw error;
  });

Also applies to: 148-154, 159-165


import {IDL} from '@dfinity/candid';
import {Actor, HttpAgent} from '@dfinity/agent';
Expand Down Expand Up @@ -126,7 +127,7 @@ export class Replica {
}
}

async deploy(name : string, wasm : string, idlFactory : IDL.InterfaceFactory, cwd : string = process.cwd()) {
async deploy(name : string, wasm : string, idlFactory : IDL.InterfaceFactory, cwd : string = process.cwd(), signal ?: AbortSignal) {
if (this.type === 'dfx') {
// prepare dfx.json for current canister
let dfxJson = path.join(this.dir, 'dfx.json');
Expand All @@ -144,8 +145,27 @@ export class Replica {
fs.mkdirSync(this.dir, {recursive: true});
fs.writeFileSync(dfxJson, JSON.stringify(newDfxJsonData, null, 2));

execSync(`dfx deploy ${name} --mode reinstall --yes --identity anonymous`, {cwd: this.dir, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']});
execSync(`dfx ledger fabricate-cycles --canister ${name} --t 100`, {cwd: this.dir, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']});
await promisify(spawn)('dfx', ['deploy', name, '--mode', 'reinstall', '--yes', '--identity', 'anonymous'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
if (error.code === 'ABORT_ERR') {
return {stderr: ''};
}
throw error;
});

if (signal?.aborted) {
return;
}

await promisify(spawn)('dfx', ['ledger', 'fabricate-cycles', '--canister', name, '--t', '100'], {cwd: this.dir, signal, stdio: this.verbose ? 'pipe' : ['pipe', 'ignore', 'pipe']}).catch((error) => {
if (error.code === 'ABORT_ERR') {
return {stderr: ''};
}
throw error;
});

if (signal?.aborted) {
return;
}

let canisterId = execSync(`dfx canister id ${name}`, {cwd: this.dir}).toString().trim();

Expand All @@ -170,7 +190,17 @@ export class Replica {
idlFactory,
wasm,
});

if (signal?.aborted) {
return;
}

await this.pocketIc.addCycles(canisterId, 1_000_000_000_000);

if (signal?.aborted) {
return;
}

this.canisters[name] = {
cwd,
canisterId: canisterId.toText(),
Expand Down
2 changes: 1 addition & 1 deletion cli/commands/sources.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,5 @@ export async function sources({conflicts = 'ignore' as 'warning' | 'error' | 'ig
}

return `--package ${name} ${pkgBaseDir}`;
});
}).filter(x => x != null);
}
23 changes: 4 additions & 19 deletions cli/commands/sync.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import process from 'node:process';
import path from 'node:path';
import {execSync} from 'node:child_process';
import {globSync} from 'glob';
Expand All @@ -7,6 +6,7 @@ import {checkConfigFile, getRootDir, readConfig} from '../mops.js';
import {add} from './add.js';
import {remove} from './remove.js';
import {checkIntegrity} from '../integrity.js';
import {getMocPath} from '../helpers/get-moc-path.js';

type SyncOptions = {
lock ?: 'update' | 'ignore';
Expand Down Expand Up @@ -48,25 +48,10 @@ let ignore = [
'**/.mops/**',
];

let mocPath = '';
function getMocPath() : string {
if (!mocPath) {
mocPath = process.env.DFX_MOC_PATH || '';
}
if (!mocPath) {
try {
mocPath = execSync('dfx cache show').toString().trim() + '/moc';
}
catch {}
}
if (!mocPath) {
mocPath = 'moc';
}
return mocPath;
}

async function getUsedPackages() : Promise<string[]> {
let rootDir = getRootDir();
let mocPath = getMocPath();

let files = globSync('**/*.mo', {
cwd: rootDir,
nocase: true,
Expand All @@ -76,7 +61,7 @@ async function getUsedPackages() : Promise<string[]> {
let packages : Set<string> = new Set;

for (let file of files) {
let deps : string[] = execSync(`${getMocPath()} --print-deps ${path.join(rootDir, file)}`).toString().trim().split('\n');
let deps : string[] = execSync(`${mocPath} --print-deps ${path.join(rootDir, file)}`).toString().trim().split('\n');

for (let dep of deps) {
if (dep.startsWith('mo:') && !dep.startsWith('mo:prim') && !dep.startsWith('mo:⛔')) {
Expand Down
4 changes: 4 additions & 0 deletions cli/commands/test/mmf1.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ export class MMF1 {
this.output = [];
}

getErrorMessages() {
return this.output.filter(out => out.type === 'fail').map(out => out.message);
}

parseLine(line : string) {
if (line.startsWith('mops:1:start ')) {
this._testStart(line.split('mops:1:start ')[1] || '');
Expand Down
26 changes: 22 additions & 4 deletions cli/commands/test/reporters/silent-reporter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,25 @@ import {Reporter} from './reporter.js';
import {TestMode} from '../../../types.js';

export class SilentReporter implements Reporter {
total = 0;
passed = 0;
failed = 0;
skipped = 0;
passedFiles = 0;
failedFiles = 0;
passedNamesFlat : string[] = [];
flushOnError = true;
errorOutput = '';
onProgress = () => {};
Comment on lines +15 to +17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor to Avoid Redundant Default Values

The properties flushOnError and onProgress are assigned default values both in their declarations and in the constructor parameters. This redundancy can be eliminated to improve code clarity.

Suggested Refactor:

Option 1—Remove default values from property declarations:

- flushOnError = true;
- errorOutput = '';
- onProgress = () => {};

constructor(flushOnError = true, onProgress = () => {}) {
	this.flushOnError = flushOnError;
	this.onProgress = onProgress;
+	this.errorOutput = '';
}

Option 2—Keep defaults in property declarations and adjust constructor parameters:

constructor(flushOnError?: boolean, onProgress?: () => void) {
	if (flushOnError !== undefined) {
		this.flushOnError = flushOnError;
	}
	if (onProgress !== undefined) {
		this.onProgress = onProgress;
	}
}

Also applies to: 19-22


addFiles(_files : string[]) {}
constructor(flushOnError = true, onProgress = () => {}) {
this.flushOnError = flushOnError;
this.onProgress = onProgress;
}

addFiles(files : string[]) {
this.total = files.length;
}

addRun(file : string, mmf : MMF1, state : Promise<void>, _mode : TestMode) {
state.then(() => {
Expand All @@ -30,10 +41,17 @@ export class SilentReporter implements Reporter {
this.failedFiles += Number(mmf.failed !== 0);

if (mmf.failed) {
console.log(chalk.red('✖'), absToRel(file));
mmf.flush('fail');
console.log('-'.repeat(50));
let output = `${chalk.red('✖')} ${absToRel(file)}\n${mmf.getErrorMessages().join('\n')}\n${'-'.repeat(50)}`;

if (this.flushOnError) {
console.log(output);
}
else {
this.errorOutput = `${this.errorOutput}\n${output}`.trim();
}
Comment on lines +46 to +51
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor Error Output Accumulation for Efficiency

Accumulating error messages using string concatenation may become inefficient with a large number of errors. Consider using an array to collect error messages and join them when needed.

Apply this refactor to improve performance and maintainability:

-        else {
-          this.errorOutput = `${this.errorOutput}\n${output}`.trim();
-        }
+        else {
+          if (!this.errorOutputArray) {
+            this.errorOutputArray = [];
+          }
+          this.errorOutputArray.push(output);
+        }

Then, when you need to access the accumulated error output:

getErrorOutput() {
  return this.errorOutputArray.join('\n');
}

}

this.onProgress();
});
}

Expand Down
Loading