Skip to content

Commit 8505922

Browse files
andrasbacsaiandrasbacsai
authored
v4.0.0-beta.450 (#7351)
2 parents da17494 + 1047110 commit 8505922

File tree

18 files changed

+753
-128
lines changed

18 files changed

+753
-128
lines changed

‎.ai/core/deployment-architecture.md‎

Lines changed: 80 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,84 @@ Routes: [routes/api.php](mdc:routes/api.php)
270270
- **Build artifact** reuse
271271
- **Parallel build** processing
272272

273+
### Docker Build Cache Preservation
274+
275+
Coolify provides settings to preserve Docker build cache across deployments, addressing cache invalidation issues.
276+
277+
#### The Problem
278+
279+
By default, Coolify injects `ARG` statements into user Dockerfiles for build-time variables. This breaks Docker's cache mechanism because:
280+
1. **ARG declarations invalidate cache** - Any change in ARG values after the `ARG` instruction invalidates all subsequent layers
281+
2. **SOURCE_COMMIT changes every commit** - Causes full rebuilds even when code changes are minimal
282+
283+
#### Application Settings
284+
285+
Two toggles in **Advanced Settings** control this behavior:
286+
287+
| Setting | Default | Description |
288+
|---------|---------|-------------|
289+
| `inject_build_args_to_dockerfile` | `true` | Controls whether Coolify adds `ARG` statements to Dockerfile |
290+
| `include_source_commit_in_build` | `false` | Controls whether `SOURCE_COMMIT` is included in build context |
291+
292+
**Database columns:** `application_settings.inject_build_args_to_dockerfile`, `application_settings.include_source_commit_in_build`
293+
294+
#### Buildpack Coverage
295+
296+
| Build Pack | ARG Injection | Method |
297+
|------------|---------------|--------|
298+
| **Dockerfile** | ✅ Yes | `add_build_env_variables_to_dockerfile()` |
299+
| **Docker Compose** (with `build:`) | ✅ Yes | `modify_dockerfiles_for_compose()` |
300+
| **PR Deployments** (Dockerfile only) | ✅ Yes | `add_build_env_variables_to_dockerfile()` |
301+
| **Nixpacks** | ❌ No | Generates its own Dockerfile internally |
302+
| **Static** | ❌ No | Uses internal Dockerfile |
303+
| **Docker Image** | ❌ No | No build phase |
304+
305+
#### How It Works
306+
307+
**When `inject_build_args_to_dockerfile` is enabled (default):**
308+
```dockerfile
309+
# Coolify modifies your Dockerfile to add:
310+
FROM node:20
311+
ARG MY_VAR=value
312+
ARG COOLIFY_URL=...
313+
ARG SOURCE_COMMIT=abc123 # (if include_source_commit_in_build is true)
314+
# ... rest of your Dockerfile
315+
```
316+
317+
**When `inject_build_args_to_dockerfile` is disabled:**
318+
- Coolify does NOT modify the Dockerfile
319+
- `--build-arg` flags are still passed (harmless without matching `ARG` in Dockerfile)
320+
- User must manually add `ARG` statements for any build-time variables they need
321+
322+
**When `include_source_commit_in_build` is disabled (default):**
323+
- `SOURCE_COMMIT` is NOT included in build-time variables
324+
- `SOURCE_COMMIT` is still available at **runtime** (in container environment)
325+
- Docker cache preserved across different commits
326+
327+
#### Recommended Configuration
328+
329+
| Use Case | inject_build_args | include_source_commit | Cache Behavior |
330+
|----------|-------------------|----------------------|----------------|
331+
| Maximum cache preservation | `false` | `false` | Best cache retention |
332+
| Need build-time vars, no commit | `true` | `false` | Cache breaks on var changes |
333+
| Need commit at build-time | `true` | `true` | Cache breaks every commit |
334+
| Manual ARG management | `false` | `true` | Cache preserved (no ARG in Dockerfile) |
335+
336+
#### Implementation Details
337+
338+
**Files:**
339+
- `app/Jobs/ApplicationDeploymentJob.php`:
340+
- `set_coolify_variables()` - Conditionally adds SOURCE_COMMIT to Docker build context based on `include_source_commit_in_build` setting
341+
- `generate_coolify_env_variables(bool $forBuildTime)` - Distinguishes build-time vs. runtime variables; excludes cache-busting variables like SOURCE_COMMIT from build context unless explicitly enabled
342+
- `generate_env_variables()` - Populates `$this->env_args` with build-time ARG values, respecting `include_source_commit_in_build` toggle
343+
- `add_build_env_variables_to_dockerfile()` - Injects ARG statements into Dockerfiles after FROM instructions; skips injection if `inject_build_args_to_dockerfile` is disabled
344+
- `modify_dockerfiles_for_compose()` - Applies ARG injection to Docker Compose service Dockerfiles; respects `inject_build_args_to_dockerfile` toggle
345+
- `app/Models/ApplicationSetting.php` - Defines `inject_build_args_to_dockerfile` and `include_source_commit_in_build` boolean properties
346+
- `app/Livewire/Project/Application/Advanced.php` - Livewire component providing UI bindings for cache preservation toggles
347+
- `resources/views/livewire/project/application/advanced.blade.php` - Checkbox UI elements for user-facing toggles
348+
349+
**Note:** Docker Compose services without a `build:` section (image-only) are automatically skipped.
350+
273351
### Runtime Optimization
274352
- **Container resource** limits
275353
- **Auto-scaling** based on metrics
@@ -428,7 +506,7 @@ services:
428506
- `templates/compose/chaskiq.yaml` - Entrypoint script
429507

430508
**Implementation:**
431-
- Parsed: `bootstrap/helpers/parsers.php` (line 717)
509+
- Parsed: `bootstrap/helpers/parsers.php` in `parseCompose()` function (handles `content` field extraction)
432510
- Storage: `app/Models/LocalFileVolume.php`
433511
- Validation: `tests/Unit/StripCoolifyCustomFieldsTest.php`
434512

@@ -481,7 +559,7 @@ services:
481559
- Pre-creating mount points before container starts
482560

483561
**Implementation:**
484-
- Parsed: `bootstrap/helpers/parsers.php` (line 718)
562+
- Parsed: `bootstrap/helpers/parsers.php` in `parseCompose()` function (handles `is_directory`/`isDirectory` field extraction)
485563
- Storage: `app/Models/LocalFileVolume.php` (`is_directory` column)
486564
- Validation: `tests/Unit/StripCoolifyCustomFieldsTest.php`
487565

‎app/Console/Commands/SyncBunny.php‎

Lines changed: 188 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ class SyncBunny extends Command
1616
*
1717
* @var string
1818
*/
19-
protected $signature = 'sync:bunny {--templates} {--release} {--github-releases} {--nightly}';
19+
protected $signature = 'sync:bunny {--templates} {--release} {--github-releases} {--github-versions} {--nightly}';
2020

2121
/**
2222
* The console command description.
@@ -70,12 +70,25 @@ private function syncReleasesToGitHubRepo(): bool
7070
// Write releases.json
7171
$this->info('Writing releases.json...');
7272
$releasesPath = "$tmpDir/json/releases.json";
73+
$releasesDir = dirname($releasesPath);
74+
75+
// Ensure directory exists
76+
if (! is_dir($releasesDir)) {
77+
$this->info("Creating directory: $releasesDir");
78+
if (! mkdir($releasesDir, 0755, true)) {
79+
$this->error("Failed to create directory: $releasesDir");
80+
exec('rm -rf '.escapeshellarg($tmpDir));
81+
82+
return false;
83+
}
84+
}
85+
7386
$jsonContent = json_encode($releases, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES);
7487
$bytesWritten = file_put_contents($releasesPath, $jsonContent);
7588

7689
if ($bytesWritten === false) {
7790
$this->error("Failed to write releases.json to: $releasesPath");
78-
$this->error('Possible reasons: directory does not exist, permission denied, or disk full.');
91+
$this->error('Possible reasons: permission denied or disk full.');
7992
exec('rm -rf '.escapeshellarg($tmpDir));
8093

8194
return false;
@@ -158,6 +171,156 @@ private function syncReleasesToGitHubRepo(): bool
158171
}
159172
}
160173

174+
/**
175+
* Sync versions.json to GitHub repository via PR
176+
*/
177+
private function syncVersionsToGitHubRepo(string $versionsLocation, bool $nightly = false): bool
178+
{
179+
$this->info('Syncing versions.json to GitHub repository...');
180+
try {
181+
if (! file_exists($versionsLocation)) {
182+
$this->error("versions.json not found at: $versionsLocation");
183+
184+
return false;
185+
}
186+
187+
$file = file_get_contents($versionsLocation);
188+
$json = json_decode($file, true);
189+
$actualVersion = data_get($json, 'coolify.v4.version');
190+
191+
$timestamp = time();
192+
$tmpDir = sys_get_temp_dir().'/coolify-cdn-versions-'.$timestamp;
193+
$branchName = 'update-versions-'.$timestamp;
194+
$targetPath = $nightly ? 'json/versions-nightly.json' : 'json/versions.json';
195+
196+
// Clone the repository
197+
$this->info('Cloning coolify-cdn repository...');
198+
exec('gh repo clone coollabsio/coolify-cdn '.escapeshellarg($tmpDir).' 2>&1', $output, $returnCode);
199+
if ($returnCode !== 0) {
200+
$this->error('Failed to clone repository: '.implode("\n", $output));
201+
202+
return false;
203+
}
204+
205+
// Create feature branch
206+
$this->info('Creating feature branch...');
207+
$output = [];
208+
exec('cd '.escapeshellarg($tmpDir).' && git checkout -b '.escapeshellarg($branchName).' 2>&1', $output, $returnCode);
209+
if ($returnCode !== 0) {
210+
$this->error('Failed to create branch: '.implode("\n", $output));
211+
exec('rm -rf '.escapeshellarg($tmpDir));
212+
213+
return false;
214+
}
215+
216+
// Write versions.json
217+
$this->info('Writing versions.json...');
218+
$versionsPath = "$tmpDir/$targetPath";
219+
$versionsDir = dirname($versionsPath);
220+
221+
// Ensure directory exists
222+
if (! is_dir($versionsDir)) {
223+
$this->info("Creating directory: $versionsDir");
224+
if (! mkdir($versionsDir, 0755, true)) {
225+
$this->error("Failed to create directory: $versionsDir");
226+
exec('rm -rf '.escapeshellarg($tmpDir));
227+
228+
return false;
229+
}
230+
}
231+
232+
$jsonContent = json_encode($json, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES);
233+
$bytesWritten = file_put_contents($versionsPath, $jsonContent);
234+
235+
if ($bytesWritten === false) {
236+
$this->error("Failed to write versions.json to: $versionsPath");
237+
$this->error('Possible reasons: permission denied or disk full.');
238+
exec('rm -rf '.escapeshellarg($tmpDir));
239+
240+
return false;
241+
}
242+
243+
// Stage and commit
244+
$this->info('Committing changes...');
245+
$output = [];
246+
exec('cd '.escapeshellarg($tmpDir).' && git add '.escapeshellarg($targetPath).' 2>&1', $output, $returnCode);
247+
if ($returnCode !== 0) {
248+
$this->error('Failed to stage changes: '.implode("\n", $output));
249+
exec('rm -rf '.escapeshellarg($tmpDir));
250+
251+
return false;
252+
}
253+
254+
$this->info('Checking for changes...');
255+
$statusOutput = [];
256+
exec('cd '.escapeshellarg($tmpDir).' && git status --porcelain '.escapeshellarg($targetPath).' 2>&1', $statusOutput, $returnCode);
257+
if ($returnCode !== 0) {
258+
$this->error('Failed to check repository status: '.implode("\n", $statusOutput));
259+
exec('rm -rf '.escapeshellarg($tmpDir));
260+
261+
return false;
262+
}
263+
264+
if (empty(array_filter($statusOutput))) {
265+
$this->info('versions.json is already up to date. No changes to commit.');
266+
exec('rm -rf '.escapeshellarg($tmpDir));
267+
268+
return true;
269+
}
270+
271+
$envLabel = $nightly ? 'NIGHTLY' : 'PRODUCTION';
272+
$commitMessage = "Update $envLabel versions.json to $actualVersion - ".date('Y-m-d H:i:s');
273+
$output = [];
274+
exec('cd '.escapeshellarg($tmpDir).' && git commit -m '.escapeshellarg($commitMessage).' 2>&1', $output, $returnCode);
275+
if ($returnCode !== 0) {
276+
$this->error('Failed to commit changes: '.implode("\n", $output));
277+
exec('rm -rf '.escapeshellarg($tmpDir));
278+
279+
return false;
280+
}
281+
282+
// Push to remote
283+
$this->info('Pushing branch to remote...');
284+
$output = [];
285+
exec('cd '.escapeshellarg($tmpDir).' && git push origin '.escapeshellarg($branchName).' 2>&1', $output, $returnCode);
286+
if ($returnCode !== 0) {
287+
$this->error('Failed to push branch: '.implode("\n", $output));
288+
exec('rm -rf '.escapeshellarg($tmpDir));
289+
290+
return false;
291+
}
292+
293+
// Create pull request
294+
$this->info('Creating pull request...');
295+
$prTitle = "Update $envLabel versions.json to $actualVersion - ".date('Y-m-d H:i:s');
296+
$prBody = "Automated update of $envLabel versions.json to version $actualVersion";
297+
$output = [];
298+
$prCommand = 'gh pr create --repo coollabsio/coolify-cdn --title '.escapeshellarg($prTitle).' --body '.escapeshellarg($prBody).' --base main --head '.escapeshellarg($branchName).' 2>&1';
299+
exec($prCommand, $output, $returnCode);
300+
301+
// Clean up
302+
exec('rm -rf '.escapeshellarg($tmpDir));
303+
304+
if ($returnCode !== 0) {
305+
$this->error('Failed to create PR: '.implode("\n", $output));
306+
307+
return false;
308+
}
309+
310+
$this->info('Pull request created successfully!');
311+
if (! empty($output)) {
312+
$this->info('PR URL: '.implode("\n", $output));
313+
}
314+
$this->info("Version synced: $actualVersion");
315+
316+
return true;
317+
} catch (\Throwable $e) {
318+
$this->error('Error syncing versions.json: '.$e->getMessage());
319+
320+
return false;
321+
}
322+
}
323+
161324
/**
162325
* Execute the console command.
163326
*/
@@ -167,6 +330,7 @@ public function handle()
167330
$only_template = $this->option('templates');
168331
$only_version = $this->option('release');
169332
$only_github_releases = $this->option('github-releases');
333+
$only_github_versions = $this->option('github-versions');
170334
$nightly = $this->option('nightly');
171335
$bunny_cdn = 'https://cdn.coollabs.io';
172336
$bunny_cdn_path = 'coolify';
@@ -224,7 +388,7 @@ public function handle()
224388
$install_script_location = "$parent_dir/other/nightly/$install_script";
225389
$versions_location = "$parent_dir/other/nightly/$versions";
226390
}
227-
if (! $only_template && ! $only_version && ! $only_github_releases) {
391+
if (! $only_template && ! $only_version && ! $only_github_releases && ! $only_github_versions) {
228392
if ($nightly) {
229393
$this->info('About to sync files NIGHTLY (docker-compose.prod.yaml, upgrade.sh, install.sh, etc) to BunnyCDN.');
230394
} else {
@@ -249,6 +413,11 @@ public function handle()
249413

250414
return;
251415
} elseif ($only_version) {
416+
$this->warn('⚠️ DEPRECATION WARNING: The --release option is deprecated.');
417+
$this->warn(' Please use --github-versions instead to create a PR to the coolify-cdn repository.');
418+
$this->warn(' This option will continue to work but may be removed in a future version.');
419+
$this->newLine();
420+
252421
if ($nightly) {
253422
$this->info('About to sync NIGHLTY versions.json to BunnyCDN.');
254423
} else {
@@ -281,6 +450,22 @@ public function handle()
281450
// Sync releases to GitHub repository
282451
$this->syncReleasesToGitHubRepo();
283452

453+
return;
454+
} elseif ($only_github_versions) {
455+
$envLabel = $nightly ? 'NIGHTLY' : 'PRODUCTION';
456+
$file = file_get_contents($versions_location);
457+
$json = json_decode($file, true);
458+
$actual_version = data_get($json, 'coolify.v4.version');
459+
460+
$this->info("About to sync $envLabel versions.json ($actual_version) to GitHub repository.");
461+
$confirmed = confirm('Are you sure you want to sync versions.json via GitHub PR?');
462+
if (! $confirmed) {
463+
return;
464+
}
465+
466+
// Sync versions.json to GitHub repository
467+
$this->syncVersionsToGitHubRepo($versions_location, $nightly);
468+
284469
return;
285470
}
286471

0 commit comments

Comments
 (0)