Does show-o2 support image editing tasks?

[lky-violet]

Hello, thank you for your excellent work!

I would like to ask whether show-o2 supports image editing tasks.
If I want to fine-tune the model on an image editing dataset, should I refer to the training process in Stage 2?

Looking forward to your guidance.

[Sierkinhane]

Hi, sorry for the late reply. You can modify the configs for stage 1 by changing some hyper-params like the frozen params and modify the train_stage_one.py if you're fine-tuning for visual generation.

[lky-violet]

Thank you very much for your kind reply. After reviewing the code, I have a small question regarding the input to the Flow Head. Specifically, in modeling_showo2_qwen2_5.py at line 390, it appears that last_hidden_states is being used as the input noise to the diffusion model.

logits, last_hidden_states = outputs['logits'], outputs['hidden_states'][-1]

# diffusion head to predict vector fields
if hasattr(self, 'diff_proj'):
    last_hidden_states = self.diff_proj(last_hidden_states)
position_ids = torch.arange(last_hidden_states.shape[1], device=last_hidden_states.device).unsqueeze(0)
for layer in self.diffusion_head_a:
    last_hidden_states = layer(hidden_states=last_hidden_states,
                                           adaln_input=time_embeds,
                                           attention_mask=attention_mask,
                                           position_ids=position_ids,
                                           modality_positions=modality_positions,
                                           )[0]
v_pred = self.diffusion_head_b(last_hidden_states, time_embeds, modality_positions)

From my understanding, last_hidden_states is generated by the MLLM and might be more appropriate as a conditioning input rather than being used as the noise input to the diffusion process. I may have misunderstood the design here, so I would greatly appreciate it if you could kindly clarify this part.

Thank you again for your time and support!

[Sierkinhane]

We refer to this as a native unified multimodal model, as we directly model the diffusion modeling through the base LLM. There are a lot of native unified multimodal models with AR+Diffusion, such as show-o, transfusion, janusflow, and bagel.

[lky-violet]

Thank you very much for your prompt reply. I’d like to clarify that I’m not questioning the unified model architecture — I truly appreciate and admire your work.

Specifically, I’m just curious about whether the last_hidden_states passed to each layer in self.diffusion_head_a are used as the initialization of the noise input, or as conditioning information.

for layer in self.diffusion_head_a:
    last_hidden_states = layer(hidden_states=last_hidden_states,
        adaln_input=time_embeds,
        attention_mask=attention_mask,
        position_ids=position_ids,
        modality_positions=modality_positions,
        )[0]

If they are used purely as conditioning, it seems that image editing would become significantly simpler.

Thank you again for your excellent work and support!

[Sierkinhane]

Hi, for visual generation, we directly add noise to the unified visual representations to serve as the input for the unified model, and vision tokens in last_hidden_states serve as the direct input for the diffusion head, not as the conditional input.