Fix Gaussian splat Vulkan render target setup

[jeevan6996]

Type

• Bug fix (non-breaking change which fixes an issue): Fixes Gaussian Splatting crashes in Filament due to MSAA and sampleable depth RenderTarget assertion #7495
• New feature (non-breaking change which adds functionality). Resolves #
• Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

The Gaussian Splat Vulkan backend creates a render target with sampleable depth. Filament validates render-target compatibility when setRenderTarget is called and rejects that target while view MSAA is enabled. The backend was disabling MSAA only after binding the target, so the validation failed first.

Checklist:

• I have run python util/check_style.py --apply to apply Open3D code style to my code.
• This PR changes Open3D behavior or adds new functionality.
  • Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is updated accordingly. (Not applicable: this changes internal Vulkan setup ordering without changing an API or user-facing behavior.)
  • I have added or updated C++ and / or Python unit tests OR included test results (e.g. screenshots or numbers) here.
• I will follow up and update the code if CI fails.
• For fork PRs, I have selected Allow edits from maintainers.

Description

Move the existing MSAA disable operation before SetRenderTarget in the Gaussian Splat Vulkan output setup. This satisfies Filament's sampleable-depth/MSAA invariant at the point where it is synchronously checked. Render-target creation, shared depth usage, and post-processing behavior are otherwise unchanged.

Plan summary

• Goal: prevent the Vulkan Gaussian Splat render-target assertion reported in Gaussian Splatting crashes in Filament due to MSAA and sampleable depth RenderTarget assertion #7495.
• Requirements: keep the fix internal and Vulkan-specific; preserve existing resources and public APIs; avoid unrelated renderer refactoring.
• Implementation: configure the native Filament view's existing MSAA options before binding the render target.
• Test method: verify the ordering against the pinned Filament invariant, run Open3D's full style checker, configure the GUI/example build, attempt the focused GaussianSplat build, and rely on Linux/Windows CI for the platform-specific translation unit.
• Risk: the affected Vulkan backend is excluded on macOS. The change only reorders two existing operations, and CI provides the supported-platform compile check.

Validation

• PATH="$PWD/.venv/bin:$PATH" .venv/bin/python util/check_style.py --apply --no_parallel - passed for C++/CUDA, Python, and Jupyter files with the repository-pinned formatter versions.
• git diff --check - passed.
• CMake configuration with BUILD_GUI=ON, BUILD_EXAMPLES=ON, and BUILD_UNIT_TESTS=ON - passed using Clang 22.1.8 and system OpenBLAS.
• cmake --build build-llvm --target GaussianSplat --parallel 8 - compiled host shader tools and dependencies, then stopped during Metal AIR generation because the standalone Apple Command Line Tools installation does not provide xcrun metal; full Xcode is required for this macOS target.

No mock-only unit test is added. The ordering is inside the concrete GPU backend, existing visualization tests do not expose a hardware-independent seam, and introducing a production abstraction solely for this two-call ordering regression would expand the change beyond the reported fix. The changed Vulkan source is compiled by Linux/Windows CI.

[update-docs]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[ssheorey]

Hi @jeevan6996 thanks for taking a look at this! I cannot reproduce the bug from the original report. How did you reproduce it?

MSAA is supposed to be disabled.

[jeevan6996]

Thanks for checking. I should clarify one validation gap: I did not personally reproduce the Windows/Vulkan crash locally; my host is macOS, where this backend path is not available.

The reason I opened the PR is that #7495 includes a concrete Filament assertion, and the current code path appears consistent with it: FilamentView initializes MSAA as enabled, while GaussianSplatVulkanBackend::PrepareOutputTextures disables MSAA only after binding the render target. Since Filament validates the sampleable-depth/MSAA invariant during setRenderTarget, moving the existing MSAA disable before that call looked like the narrowest proposed fix.

If there is another setup path that should already disable MSAA before this point, then I may be missing that context. I’m happy to close or rework this based on your guidance. We could also ask the original reporter to confirm whether this patch resolves their Windows/Vulkan reproduction.

[jeevan6996]

I have rebased this PR onto the current main and added the requested CHANGELOG.md entry.

I cannot reproduce the Windows/Vulkan crash locally because my host is macOS and does not expose this backend. The proposed change is based on the assertion and call-order evidence in #7495: FilamentView starts with MSAA enabled, while PrepareOutputTextures previously called SetRenderTarget before disabling MSAA. Filament validates the sampleable-depth/MSAA compatibility during setRenderTarget, so the existing disable operation now runs before that validation.

I also checked the latest CI failures: the Linux/Windows build jobs completed compilation and tests, then failed in the artifact-attestation step due to the workflow missing id-token: write; the macOS failures are in Metal/OpenBLAS setup. None reported a compile or test failure in the changed source.

If the original reporter can confirm the Windows/Vulkan reproduction, that would provide the platform-specific validation this host cannot provide. If another initialization path already disables MSAA before this function, I am happy to adjust or close the PR.