Skip to content

Store periodic boundaries on the mesh - #4637

Open
lindsayad wants to merge 5 commits into
libMesh:develfrom
lindsayad:periodic-point-ghosting
Open

lindsayad wants to merge 5 commits into
libMesh:develfrom
lindsayad:periodic-point-ghosting

Conversation

@lindsayad

@lindsayad lindsayad commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Ghosting functors and MeshRefinement now see the periodic boundaries of every DofMap on a mesh. #4649 gave only the first DofMap's set to the mesh's ghosting functors. If the first system has no periodic boundaries, a later system's periodic constraints can fail on a refined DistributedMesh (#1728).

  • Each DofMap registers its PeriodicBoundaries with MeshBase::add_periodic_boundaries().
  • A ghosting functor searches either its own set, from set_periodic_boundaries(), or every registered set, after use_mesh_periodic_boundaries(true). The default ghosting functor uses every set. DofMap coupling and algebraic functors keep their own set, so other systems do not widen their sparsity pattern or send list.
  • A functor that replaces the default ghosting should ghost every element point neighbor of a periodic vertex and its image, and should call use_mesh_periodic_boundaries(true).
  • A copied functor does not copy its own set, because copies usually serve another mesh or DofMap.
  • The mesh caches the registered sets with redundant sets removed. A set is redundant if an earlier set maps each of its boundaries the same way (PeriodicBoundaryBase::same_mapping()). DofMap::add_periodic_boundary() invalidates the cache.
  • The non-const DofMap::get_periodic_boundaries() is deprecated, because changes through its pointer bypass the cache.
  • MeshRefinement uses every registered set. set_periodic_boundaries_ptr() is deprecated, and adaptivity_ex5 no longer calls it. Its results are unchanged.
  • refine_elements() and coarsen_elements() now call prepare_for_use() before their level-one asserts. The old point locator could not find new children.

Refs #1728.

🤖 Generated with Claude Code

@moosebuild

moosebuild commented Oct 30, 2026 •

Copy link
Copy Markdown

Job Coverage, step Generate coverage on 798c60a wanted to post the following:

Coverage

2cb749 #4637 798c60
Total Total +/- New
Rate 66.62% 66.65% +0.03% 100.00%
Hits 81253 81375 +122 190
Misses 40711 40712 +1 0

Diff coverage report

Full coverage report

This comment will be updated on new commits.

Comment thread include/base/dof_map.h Outdated
*
* We use a std::unique_ptr here to reduce header dependencies.
*/
std::unique_ptr<PeriodicPointNeighbors> _periodic_ghosting;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked at most of the PR yet, but this feels like a code smell. We can't get periodic boundaries to work with only side neighbors, despite the fact that they only define constraints that apply (perhaps recursively) across sides? We can't add periodic boundary support to an existing ghosting functor?

@lindsayad lindsayad Oct 30, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have periodic boundary support in existing ghosting functors. The problem is that people always forget to add the periodic boundaries to it (it mostly meaning the default geometric ghosting functor on the mesh) which is the fundamental reason behind #1728 I believe

@lindsayad lindsayad Oct 30, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also haven't looked at the PR either but I do think that fundamentally when someone adds a periodic boundary to the DofMap it should be automatically forwarded to a ghosting functor on the mesh

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly think the first half of your complaint argues against the second half at least insofar as we shouldn't rely on the default GhostPointNeighbors geometric ghosting functor attached on the mesh to be the same functor that is responsible for adding side-neighbor periodic boundary ghosting

@lindsayad lindsayad Oct 31, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude thought my last reply was nonsensical and maybe it is 😅 Here's it's motivational pitch with some edits of my own:

The constraints act across sides, but which side gets constrained is decided at points. For each periodic vertex, compute_periodic_constraints() calls primary_boundary_point_neighbor() at the vertex and at its image, and compares the two results. That function searches all point neighbors. A processor missing one can choose the opposite direction from its peer, which gives the cycle in #1728.

GhostPointNeighbors has always had a periodic branch for this. Nothing in the library passed it the DofMap's periodic boundaries though, and it holds only one PeriodicBoundaries pointer for possibly many DofMaps. So the PR moves that branch into PeriodicPointNeighbors and gives each DofMap its own instance.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the user has also previously removed the mesh's default ghosting functor which we know properly adds the point neighbors currently needed for correct geometric periodic boundary ghosting. What should we do then?

My first thought: loop over as_range(ghosting_functors_begin(), ghosting_functors_end()) and add_periodic_boundaries() for all of them, regardless of what the user has done. If they've got a different GhostingFunctor there that includes point neighbors then we're done, and if they don't then we laugh at them for reducing default ghosting and expecting it to work.

My second thought: are we all falling for an XY problem here? We definitely need the extra ghosted elements for PBCs "primary element"-based disambiguation to work, but perhaps we could just find some other disambiguation? In the non-AMR case "compare node ids instead of element ids" (or lexicographic point orderings; we do that on 1-element-thick meshes now) would be sufficient. In the AMR case, we always have to worry about elements because h-refinement constraints have to go "fine constrained in terms of coarse", never the other way around ... but can we still work with that without coarse neighbors? h refinement never constrains vertex DoFs vs those of a periodic paired vertex, so we could still use node ordering there. It constrains hanging vertex DoFs and face node DoFs but only vs a coarser face neighbor that we'll always have. Edge node DoFs are the hardest case for an individual PBC, but we can handle those recursively as long as we have a total ordering to make sure the recursion doesn't have a loop... and maybe that's enough to handle intersecting PBCs too?

@roystgnr roystgnr Nov 5, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That total order would make us safe from cycles regardless, but it's not enough to give us a uniquely-defined tree of constraints. When we're threaded, the mutex we use would ensure we have a safe tree regardless, but it could be a different safe tree from run to run, a Heisenbug waiting to happen. We'd probably need to write to a temporary multimap so multiple-option cases could be disambiguated (by something threading-independent like "which element was this computed from") and added to the DofMap deterministically after. When we're MPI-parallelized, the obvious thing to do (ranks can only set constraints on DoFs they own) is deterministic (for a given partitioning) but also wrong, e.g. on meshes where a periodic boundary node is owned by a process that has no elements on that boundary. Maybe we also just need to compute-everything-then-disambiguate in that case: everybody push constraint rows to the constrained DoFs' owners, then the owner disambiguates and pushes results back.

@lindsayad lindsayad Nov 6, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a 3D edge case that I think any replacement for primary_boundary_point_neighbor must handle:

            M (refined)
       \         /
   C    \       /    N (refined)
         \     /
  ========e========   periodic boundary
  • C is coarse and has a face on the periodic boundary.
  • M is refined and touches the boundary only along edge e.
  • N is refined to the same level as M, so the children of M and N see no hanging nodes between them. N has faces on the periodic boundary.

Node m on e is a hanging node. A child of M constrains m in terms of C. The children of N see only equal-level neighbors, so they could write a periodic row for m. Today the periodic row loses (isn't written) only because primary_boundary_point_neighbor picks C as the primary element, and m is not a node of C. That depends on the point-neighbor ghosting of C.

Node ordering alone does not fix this. If the ordering makes m the constrained side of (m, m'), the children of N write a periodic row for m, and it collides with the hanging-node row. A total order prevents cycles, but it does not choose between two valid rows for one DoF.

The compute-everything-then-disambiguate approach works here. Each candidate row needs only face neighbors, and the owner of m gets both rows. But "which element was this computed from" is not the right rule. The hanging-node row must win. The surviving rows must also still imply m = m'. That means one of these:

  • Re-target the periodic row to constrain m' in terms of m, even if the node ordering points the other way.
  • Give h-refinement rows fixed precedence, and include that precedence in the argument for no cycles.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction to my last paragraph: the re-targeting option is unnecessary. Across the periodic boundary, the coarse edge DoFs on e and its image are already identified. Whatever the refinement on the image side, m' is either absent or constrained to an interpolation of those coarse DoFs. Thus the hanging-node row for m already implies m = m'. A fixed precedence for h-refinement rows is sufficient.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We almost have a fixed precedence for h-refinement (AMR) constraints ... we calculate those first, and when we add periodic (PBC) constraints we only do so for DoFs that aren't already constrained.

The trouble is that we do that ordering on an element-by-element level. If C, M, and N (and children where applicable) are owned by three different processors, only rank M sees the AMR constraint, and it can't stop rank N from trying to add a PBC constraint. Even on the same rank we might happen to hit element N before element M and thereby add a PBC constraint that we only realize too late was incorrect.

We could just have AMR overwrite existing constraint rows instead of skipping them, since any AMR constraint should take precedence over any PBC constraint. We'll still need a parallel disambiguation in the end, though, so maybe it'd be simpler to use the same temporary data structure for both AMR and PBC candidates to make the communication simpler.

lindsayad added a commit to idaholab/moose that referenced this pull request Oct 30, 2026
The test matched the exact list of libMesh default ghosting functors.
That list is a libMesh implementation detail, so every change to the
default ghosting broke the test without a change in the solution. The
Exodiff check still verifies the solution.

Refs libMesh/libmesh#4637

Co-Authored-By: Claude <noreply@anthropic.com>
@libMesh libMesh deleted a comment from moosebuild Oct 31, 2026
@libMesh libMesh deleted a comment from moosebuild Oct 31, 2026
@libMesh libMesh deleted a comment from moosebuild Oct 31, 2026
GiudGiud pushed a commit to dschwen/moose that referenced this pull request Nov 2, 2026
The test matched the exact list of libMesh default ghosting functors.
That list is a libMesh implementation detail, so every change to the
default ghosting broke the test without a change in the solution. The
Exodiff check still verifies the solution.

Refs libMesh/libmesh#4637

Co-Authored-By: Claude <noreply@anthropic.com>
lindsayad and others added 2 commits November 9, 2026 22:18
Each DofMap registers its PeriodicBoundaries with
MeshBase::add_periodic_boundaries(). Ghosting functors and MeshRefinement
read the registered sets when they run. They then see the periodic
boundaries of every system, not only the first system's.

- MeshBase::add_ghosting_functor() takes use_all_pbcs, default false.
  The default ghosting always uses every set. DofMap coupling and
  algebraic functors keep their own set, so other systems do not widen
  their sparsity pattern or send list.
- GhostPointNeighbors, DefaultCoupling and PointNeighborCoupling search
  the sets from GhostingFunctor::periodic_boundaries_used(). That is the
  functor's own set, plus every registered set if
  MeshBase::uses_all_periodic_boundaries() is true for the functor.
- MeshRefinement checks level one against the periodic neighbor in every
  set. set_periodic_boundaries_ptr() is deprecated.
- refine_elements() and coarsen_elements() prepare the mesh before their
  level-one asserts. The asserts then use a current point locator for
  periodic neighbor lookups.

Refs libMesh#1728.

Co-Authored-By: Claude <noreply@anthropic.com>
When several DofMaps register the same periodic boundaries, ghosting
functors and MeshRefinement searched every copy. Each copy cost one
point locator search per periodic side.

- PeriodicBoundaryBase::same_mapping() reports whether two boundaries
  map points the same way. The default is false. PeriodicBoundary
  compares boundary ids and translation vectors.
- PeriodicBoundaries::covers() is true if a set maps every boundary id of
  another set the same way.
- remove_redundant_periodic_boundaries() drops each set that an earlier
  set covers. GhostingFunctor and MeshRefinement call it in their
  periodic_boundaries_used().

A set is removed only as a whole. Elem::topological_neighbor() uses the
first periodic id of a side that a set contains, so a set with some of
its entries removed could miss a neighbor.

Co-Authored-By: Claude <noreply@anthropic.com>
@lindsayad
lindsayad force-pushed the periodic-point-ghosting branch from dac51b8 to 6f60608 Compare November 10, 2026 16:56
@lindsayad lindsayad changed the title Ghost periodic point neighbors for each DofMap Store periodic boundaries on the mesh Nov 10, 2026
@lindsayad
lindsayad marked this pull request as ready for review November 10, 2026 17:52
@lindsayad
lindsayad requested a review from roystgnr November 10, 2026 17:53
Comment thread include/base/periodic_boundaries.h Outdated
* sets for periodic neighbors gives every neighbor that the original
* sets gave.
*/
void remove_redundant_periodic_boundaries(std::vector<const PeriodicBoundaries *> & sets);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a static member of PeriodicBoundaries rather than a global?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And then we could strip (at least) periodic out of the subroutine name. "I heard you like removing redundancies, so I removed a redundancy in your redundancy removal."

Comment thread src/base/periodic_boundary.C Outdated
bool PeriodicBoundary::same_mapping(const PeriodicBoundaryBase & other) const
{
// A derived class may override get_corresponding_pos() or add
// state, so only objects whose dynamic type is exactly

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I think I'd prefer to mark PeriodicBoundary::get_corresponding_pos() as final and skip the typeid(*this). If there's anyone crazy enough to inherit from PeriodicBoundary they can instead inherit from PeriodicBoundaryBase and we'll live with a little duplicated code.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Luckily MOOSE's FunctionPeriodicBoundary already inherits directly from PeriodicBoundaryBase! 😄

Comment thread src/base/periodic_boundary.C Outdated
// state, so only objects whose dynamic type is exactly
// PeriodicBoundary map by translation alone
if (typeid(*this) != typeid(PeriodicBoundary) ||
typeid(other) != typeid(PeriodicBoundary))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to break with RTTI disabled, right? But for that case we can just return false; and it'll be less efficient but no less correct?

Comment thread include/base/periodic_boundary.h Outdated

/**
* \returns \p true if this object and \p other are both exactly
* PeriodicBoundary, i.e. neither is an object of a derived class, and

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we can shorten this comment once we've marked the important bits of PeriodicBoundary as final

Comment thread include/ghosting/ghosting_functor.h Outdated
// functor added with
// MeshBase::add_ghosting_functor(gf, /*use_all_pbcs=*/true), the
// mesh also supplies the periodic boundary sets of every DofMap on
// the mesh.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly confusing to me; I have to imagine it'll be even worse to anyone who wasn't in on the design discussion.

Comment thread include/systems/equation_systems.h Outdated
* only safe for codes where no evaluations on neighbor cells (e.g.
* no jump error estimators) are done.
*
* Periodic constraints on a distributed mesh need every element

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently

Comment thread src/ghosting/default_coupling.C Outdated
bool check_periodic_bcs =
(_periodic_bcs && !_periodic_bcs->empty());
const std::vector<const PeriodicBoundaries *> periodic_bc_sets =
this->periodic_boundaries_used(_periodic_bcs);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be a bit expensive for DefaultCoupling in particular? Coupling functors get called on individual elements, not just ranges of elements, so we can't amortize over a whole range the way we do with geometric and algebraic ghosting. Any way we can move the computation here to someplace (mesh_reinit()?) where we can cache the results so we can just return by reference here?

Comment thread src/ghosting/default_coupling.C Outdated
bool check_periodic_bcs = !periodic_bc_sets.empty();

// A side with a regular neighbor is examined once, without periodic boundaries
const std::vector<const PeriodicBoundaries *> no_periodic_bcs {nullptr};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to make this static too?

(!elem->neighbor_ptr(s) && check_periodic_bcs) ?
periodic_bc_sets : no_periodic_bcs)
#endif
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should fix our indentation, here and in the other ghosting functors where we've added the additional loop layer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol I'm pretty sure that Claude actually raised this in the turn it did this and stated the reason for not indenting as reducing the diff for review. I forget whether git diff -w fixes that issue. I think not, in which case it's not the worst reason in the world but I remember thinking bah I need to make sure to scold it ... and then I forgot even when reviewing the diff before moving off of draft



const std::vector<Elem *> &
MeshRefinement::topological_neighbors(Elem * elem,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this idiom here vs the additional nested loop in the ghosting functors? I can see pros and cons for each but I'd have thought that the same idiom would have won out in both cases.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ghosting functors we already deduplicate naturally through insertion into the ghosting map, O(log N), so a similar idiom in the ghosting functors would be adding unnecessary O(N) std::find deduplication. Conversely, for four of the six loop structures in mesh refinement if we added a loop inside of the side index range loop over the periodic boundaries sets, the break statements would no longer be valid. We'd need to use something like agoto instead. If you feel strongly about having the same idiom in both places, then we could make something happen (I'd probably prefer changing to mesh refinement loops to use goto)

@roystgnr

Copy link
Copy Markdown
Member

I accidentally hit "Submit" while double-checking all the entries before changing "Comment" to "Request Changes".

Anyway, the only thing I'm really concerned about is adding more overhead to DefaultCoupling::operator(). If we can change the new API to give us a reference to a cache there, everything else is a question or a quibble or easy to fix in comparison.

lindsayad and others added 3 commits November 12, 2026 15:44
PeriodicBoundary::get_corresponding_pos() is now final, so any
PeriodicBoundary maps points by its translation alone. same_mapping()
then needs only a dynamic_cast, and it accepts derived classes.

Without RTTI, same_mapping() returns false. Each set is then searched
on its own, which costs time but gives the same neighbors.

Co-Authored-By: Claude <noreply@anthropic.com>
The free function remove_redundant_periodic_boundaries() becomes the
static member PeriodicBoundaries::remove_redundant_sets(). The class name
already says "periodic", and the function removes whole sets.

Co-Authored-By: Claude <noreply@anthropic.com>
A ghosting functor searches either its own periodic boundary set or
every set registered with the mesh. The choice is now a property of the
functor, so MeshBase no longer tracks it per functor.

- GhostingFunctor::use_mesh_periodic_boundaries() selects the mesh's
  sets. It is an error to combine it with set_periodic_boundaries().
  The default ghosting functor uses the mesh's sets.
- MeshBase::add_ghosting_functor() loses its use_all_pbcs argument.
- set_periodic_boundaries() accepts one set. Setting a different set
  later is an error.
- A copied functor keeps the use_mesh_periodic_boundaries() setting
  but does not copy its own set. Copies usually serve another mesh or
  DofMap.
- GhostingFunctor::periodic_boundary_sets() returns a reference, so
  DefaultCoupling::operator() allocates nothing for each element.
- MeshBase caches the registered sets after remove_redundant_sets().
  Registration changes and DofMap::add_periodic_boundary() invalidate
  the cache. build_sparsity() rebuilds it before its threaded loop.
- The non-const DofMap::get_periodic_boundaries() is deprecated.
  Changes through its pointer bypass the DofMap, so the mesh cache
  could go out of date.

Co-Authored-By: Claude <noreply@anthropic.com>
@lindsayad
lindsayad requested a review from roystgnr November 14, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants