Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDXL app: move await device to fix empty results #975

Merged
merged 2 commits into from
Feb 17, 2025
Merged

SDXL app: move await device to fix empty results #975

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
28ff441
move await device to fix empty results
daveliddell Feb 17, 2025
7fc9fb1
changed comment to explain why we need to wait there
daveliddell Feb 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions shortfin/python/shortfin_apps/sd/components/service.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,9 +386,11 @@ async def run(self):
await self._denoise(device=device)
if phases[InferencePhase.DECODE]["required"]:
await self._decode(device=device)
# Postprocessing needs the output data to be on the host. Even
# without postprocessing, we're done with the GPU, so we wait for
# it to finish here.
else:
# Decode and postprocess both need the output data to be on the host.
# With decode enabled, decode itself will wait for the data.
# With decode disabled, whether or not we're postprocessing,
# we're done with the GPU, so we wait for it to finish here.
await device
if phases[InferencePhase.POSTPROCESS]["required"]:
await self._postprocess(device=device)
Expand Down Expand Up @@ -546,6 +548,11 @@ async def _decode(self, device):
)
(cb.images,) = await fn(cb.latents, fiber=self.fiber)
cb.images_host.copy_from(cb.images)

# Wait for the device-to-host transfer, so that we can read the
# data with .items.
await device

image_array = cb.images_host.items
dtype = image_array.typecode
if cb.images_host.dtype == sfnp.float16:
Expand Down
Loading