Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VPlan] Run recipe removal and simplification after optimizeForVFAndUF. #125926

Merged
merged 2 commits into from
Feb 8, 2025
Merged

[VPlan] Run recipe removal and simplification after optimizeForVFAndUF. #125926

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d704a98
[VPlan] Run recipe removal and simplification after optimizeForVFAndUF.
fhahn Feb 5, 2025
43f5a9b
Merge branch 'main' into vplan-simpe-cleanup-after-optimize
fhahn Feb 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7685,6 +7685,8 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
VPlanTransforms::runPass(VPlanTransforms::unrollByUF, BestVPlan, BestUF,
OrigLoop->getHeader()->getContext());
VPlanTransforms::optimizeForVFAndUF(BestVPlan, BestVF, BestUF, PSE);
VPlanTransforms::simplifyRecipes(BestVPlan, *Legal->getWidestInductionType());
VPlanTransforms::removeDeadRecipes(BestVPlan);
Comment on lines +7688 to +7689
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which passes can/should be wrapped with runPass()?
Would be better to have each pass take care of printing and/or verification itself, somehow, rather than extracting this aspect to an explicit external wrapper.

VPlanTransforms::convertToConcreteRecipes(BestVPlan);

// Perform the actual loop transformation.
Expand Down
8 changes: 8 additions & 0 deletions llvm/lib/Transforms/Vectorize/VPlanPatternMatch.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,14 @@ struct Recipe_match {
if ((!matchRecipeAndOpcode<RecipeTys>(R) && ...))
return false;

if (!(std::is_same_v<VPWidenEVLRecipe, RecipeTys> || ...) &&
isa<VPWidenEVLRecipe>(R)) {
// Don't match VPWidenEVLRecipe if it is not explicitly part of RecipeTys.
// Otherwise we might match it unexpectedly when trying to match
// VPWidenRecipe, of which VPWidenEVLRecipe is a subclass of.
Comment on lines +173 to +177
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this relevant in general for any recipe that is a subclass of another?

return false;
}

assert(R->getNumOperands() == std::tuple_size<Ops_t>::value &&
"recipe with matched opcode the expected number of operands");

Expand Down
5 changes: 1 addition & 4 deletions llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -964,9 +964,7 @@ static void simplifyRecipe(VPRecipeBase &R, VPTypeAnalysis &TypeInfo) {
return R.getVPSingleValue()->replaceAllUsesWith(R.getOperand(1));
}

/// Try to simplify the recipes in \p Plan. Use \p CanonicalIVTy as type for all
/// un-typed live-ins in VPTypeAnalysis.
static void simplifyRecipes(VPlan &Plan, Type &CanonicalIVTy) {
void VPlanTransforms::simplifyRecipes(VPlan &Plan, Type &CanonicalIVTy) {
ReversePostOrderTraversal<VPBlockDeepTraversalWrapper<VPBlockBase *>> RPOT(
Plan.getEntry());
VPTypeAnalysis TypeInfo(&CanonicalIVTy);
Expand Down Expand Up @@ -1043,7 +1041,6 @@ void VPlanTransforms::optimizeForVFAndUF(VPlan &Plan, ElementCount BestVF,
}

Term->eraseFromParent();
VPlanTransforms::removeDeadRecipes(Plan);

Plan.setVF(BestVF);
Plan.setUF(BestUF);
Expand Down
4 changes: 4 additions & 0 deletions llvm/lib/Transforms/Vectorize/VPlanTransforms.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,10 @@ struct VPlanTransforms {
/// Lower abstract recipes to concrete ones, that can be codegen'd.
static void convertToConcreteRecipes(VPlan &Plan);

/// Perform instcombine-like simplifications on recipes in \p Plan. Use \p
/// CanonicalIVTy as type for all un-typed live-ins in VPTypeAnalysis.
static void simplifyRecipes(VPlan &Plan, Type &CanonicalIVTy);
Comment on lines +166 to +168
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be better for all live-ins to carry their type, e.g., by holding an underlying Undef to hold the Type for live-ins whose Value is not (yet) available, rather than supplying this type externally to VPTypeAnalysis, simplifyRecipes(), etc. In general, better to encode all IR information inside VPlan than to pass it as additional operands to VPlanTransforms.


/// If there's a single exit block, optimize its phi recipes that use exiting
/// IV values by feeding them precomputed end values instead, possibly taken
/// one step backwards.
Expand Down