Repository navigation
Conversation
| * | ||
| * We use a std::unique_ptr here to reduce header dependencies. | ||
| */ | ||
| std::unique_ptr<PeriodicPointNeighbors> _periodic_ghosting; |
There was a problem hiding this comment.
I haven't looked at most of the PR yet, but this feels like a code smell. We can't get periodic boundaries to work with only side neighbors, despite the fact that they only define constraints that apply (perhaps recursively) across sides? We can't add periodic boundary support to an existing ghosting functor?
There was a problem hiding this comment.
We already have periodic boundary support in existing ghosting functors. The problem is that people always forget to add the periodic boundaries to it (it mostly meaning the default geometric ghosting functor on the mesh) which is the fundamental reason behind #1728 I believe
There was a problem hiding this comment.
I also haven't looked at the PR either but I do think that fundamentally when someone adds a periodic boundary to the DofMap it should be automatically forwarded to a ghosting functor on the mesh
There was a problem hiding this comment.
I honestly think the first half of your complaint argues against the second half at least insofar as we shouldn't rely on the default GhostPointNeighbors geometric ghosting functor attached on the mesh to be the same functor that is responsible for adding side-neighbor periodic boundary ghosting
There was a problem hiding this comment.
Claude thought my last reply was nonsensical and maybe it is 😅 Here's it's motivational pitch with some edits of my own:
The constraints act across sides, but which side gets constrained is decided at points. For each periodic vertex, compute_periodic_constraints() calls primary_boundary_point_neighbor() at the vertex and at its image, and compares the two results. That function searches all point neighbors. A processor missing one can choose the opposite direction from its peer, which gives the cycle in #1728.
GhostPointNeighbors has always had a periodic branch for this. Nothing in the library passed it the DofMap's periodic boundaries though, and it holds only one PeriodicBoundaries pointer for possibly many DofMaps. So the PR moves that branch into PeriodicPointNeighbors and gives each DofMap its own instance.
🤖 Generated with Claude Code
There was a problem hiding this comment.
the user has also previously removed the mesh's default ghosting functor which we know properly adds the point neighbors currently needed for correct geometric periodic boundary ghosting. What should we do then?
My first thought: loop over as_range(ghosting_functors_begin(), ghosting_functors_end()) and add_periodic_boundaries() for all of them, regardless of what the user has done. If they've got a different GhostingFunctor there that includes point neighbors then we're done, and if they don't then we laugh at them for reducing default ghosting and expecting it to work.
My second thought: are we all falling for an XY problem here? We definitely need the extra ghosted elements for PBCs "primary element"-based disambiguation to work, but perhaps we could just find some other disambiguation? In the non-AMR case "compare node ids instead of element ids" (or lexicographic point orderings; we do that on 1-element-thick meshes now) would be sufficient. In the AMR case, we always have to worry about elements because h-refinement constraints have to go "fine constrained in terms of coarse", never the other way around ... but can we still work with that without coarse neighbors? h refinement never constrains vertex DoFs vs those of a periodic paired vertex, so we could still use node ordering there. It constrains hanging vertex DoFs and face node DoFs but only vs a coarser face neighbor that we'll always have. Edge node DoFs are the hardest case for an individual PBC, but we can handle those recursively as long as we have a total ordering to make sure the recursion doesn't have a loop... and maybe that's enough to handle intersecting PBCs too?
There was a problem hiding this comment.
That total order would make us safe from cycles regardless, but it's not enough to give us a uniquely-defined tree of constraints. When we're threaded, the mutex we use would ensure we have a safe tree regardless, but it could be a different safe tree from run to run, a Heisenbug waiting to happen. We'd probably need to write to a temporary multimap so multiple-option cases could be disambiguated (by something threading-independent like "which element was this computed from") and added to the DofMap deterministically after. When we're MPI-parallelized, the obvious thing to do (ranks can only set constraints on DoFs they own) is deterministic (for a given partitioning) but also wrong, e.g. on meshes where a periodic boundary node is owned by a process that has no elements on that boundary. Maybe we also just need to compute-everything-then-disambiguate in that case: everybody push constraint rows to the constrained DoFs' owners, then the owner disambiguates and pushes results back.
There was a problem hiding this comment.
Here is a 3D edge case that I think any replacement for primary_boundary_point_neighbor must handle:
M (refined)
\ /
C \ / N (refined)
\ /
========e======== periodic boundary
- C is coarse and has a face on the periodic boundary.
- M is refined and touches the boundary only along edge
e. - N is refined to the same level as M, so the children of M and N see no hanging nodes between them. N has faces on the periodic boundary.
Node m on e is a hanging node. A child of M constrains m in terms of C. The children of N see only equal-level neighbors, so they could write a periodic row for m. Today the periodic row loses (isn't written) only because primary_boundary_point_neighbor picks C as the primary element, and m is not a node of C. That depends on the point-neighbor ghosting of C.
Node ordering alone does not fix this. If the ordering makes m the constrained side of (m, m'), the children of N write a periodic row for m, and it collides with the hanging-node row. A total order prevents cycles, but it does not choose between two valid rows for one DoF.
The compute-everything-then-disambiguate approach works here. Each candidate row needs only face neighbors, and the owner of m gets both rows. But "which element was this computed from" is not the right rule. The hanging-node row must win. The surviving rows must also still imply m = m'. That means one of these:
- Re-target the periodic row to constrain
m'in terms ofm, even if the node ordering points the other way. - Give h-refinement rows fixed precedence, and include that precedence in the argument for no cycles.
🤖 Generated with Claude Code
There was a problem hiding this comment.
Correction to my last paragraph: the re-targeting option is unnecessary. Across the periodic boundary, the coarse edge DoFs on e and its image are already identified. Whatever the refinement on the image side, m' is either absent or constrained to an interpolation of those coarse DoFs. Thus the hanging-node row for m already implies m = m'. A fixed precedence for h-refinement rows is sufficient.
🤖 Generated with Claude Code
There was a problem hiding this comment.
We almost have a fixed precedence for h-refinement (AMR) constraints ... we calculate those first, and when we add periodic (PBC) constraints we only do so for DoFs that aren't already constrained.
The trouble is that we do that ordering on an element-by-element level. If C, M, and N (and children where applicable) are owned by three different processors, only rank M sees the AMR constraint, and it can't stop rank N from trying to add a PBC constraint. Even on the same rank we might happen to hit element N before element M and thereby add a PBC constraint that we only realize too late was incorrect.
We could just have AMR overwrite existing constraint rows instead of skipping them, since any AMR constraint should take precedence over any PBC constraint. We'll still need a parallel disambiguation in the end, though, so maybe it'd be simpler to use the same temporary data structure for both AMR and PBC candidates to make the communication simpler.
The test matched the exact list of libMesh default ghosting functors. That list is a libMesh implementation detail, so every change to the default ghosting broke the test without a change in the solution. The Exodiff check still verifies the solution. Refs libMesh/libmesh#4637 Co-Authored-By: Claude <noreply@anthropic.com>
The test matched the exact list of libMesh default ghosting functors. That list is a libMesh implementation detail, so every change to the default ghosting broke the test without a change in the solution. The Exodiff check still verifies the solution. Refs libMesh/libmesh#4637 Co-Authored-By: Claude <noreply@anthropic.com>
Each DofMap registers its PeriodicBoundaries with MeshBase::add_periodic_boundaries(). Ghosting functors and MeshRefinement read the registered sets when they run. They then see the periodic boundaries of every system, not only the first system's. - MeshBase::add_ghosting_functor() takes use_all_pbcs, default false. The default ghosting always uses every set. DofMap coupling and algebraic functors keep their own set, so other systems do not widen their sparsity pattern or send list. - GhostPointNeighbors, DefaultCoupling and PointNeighborCoupling search the sets from GhostingFunctor::periodic_boundaries_used(). That is the functor's own set, plus every registered set if MeshBase::uses_all_periodic_boundaries() is true for the functor. - MeshRefinement checks level one against the periodic neighbor in every set. set_periodic_boundaries_ptr() is deprecated. - refine_elements() and coarsen_elements() prepare the mesh before their level-one asserts. The asserts then use a current point locator for periodic neighbor lookups. Refs libMesh#1728. Co-Authored-By: Claude <noreply@anthropic.com>
When several DofMaps register the same periodic boundaries, ghosting functors and MeshRefinement searched every copy. Each copy cost one point locator search per periodic side. - PeriodicBoundaryBase::same_mapping() reports whether two boundaries map points the same way. The default is false. PeriodicBoundary compares boundary ids and translation vectors. - PeriodicBoundaries::covers() is true if a set maps every boundary id of another set the same way. - remove_redundant_periodic_boundaries() drops each set that an earlier set covers. GhostingFunctor and MeshRefinement call it in their periodic_boundaries_used(). A set is removed only as a whole. Elem::topological_neighbor() uses the first periodic id of a side that a set contains, so a set with some of its entries removed could miss a neighbor. Co-Authored-By: Claude <noreply@anthropic.com>
dac51b8 to
6f60608
Compare
| * sets for periodic neighbors gives every neighbor that the original | ||
| * sets gave. | ||
| */ | ||
| void remove_redundant_periodic_boundaries(std::vector<const PeriodicBoundaries *> & sets); |
There was a problem hiding this comment.
Should this be a static member of PeriodicBoundaries rather than a global?
There was a problem hiding this comment.
And then we could strip (at least) periodic out of the subroutine name. "I heard you like removing redundancies, so I removed a redundancy in your redundancy removal."
| bool PeriodicBoundary::same_mapping(const PeriodicBoundaryBase & other) const | ||
| { | ||
| // A derived class may override get_corresponding_pos() or add | ||
| // state, so only objects whose dynamic type is exactly |
There was a problem hiding this comment.
Honestly, I think I'd prefer to mark PeriodicBoundary::get_corresponding_pos() as final and skip the typeid(*this). If there's anyone crazy enough to inherit from PeriodicBoundary they can instead inherit from PeriodicBoundaryBase and we'll live with a little duplicated code.
There was a problem hiding this comment.
Luckily MOOSE's FunctionPeriodicBoundary already inherits directly from PeriodicBoundaryBase! 😄
| // state, so only objects whose dynamic type is exactly | ||
| // PeriodicBoundary map by translation alone | ||
| if (typeid(*this) != typeid(PeriodicBoundary) || | ||
| typeid(other) != typeid(PeriodicBoundary)) |
There was a problem hiding this comment.
This is going to break with RTTI disabled, right? But for that case we can just return false; and it'll be less efficient but no less correct?
|
|
||
| /** | ||
| * \returns \p true if this object and \p other are both exactly | ||
| * PeriodicBoundary, i.e. neither is an object of a derived class, and |
There was a problem hiding this comment.
And we can shorten this comment once we've marked the important bits of PeriodicBoundary as final
| // functor added with | ||
| // MeshBase::add_ghosting_functor(gf, /*use_all_pbcs=*/true), the | ||
| // mesh also supplies the periodic boundary sets of every DofMap on | ||
| // the mesh. |
There was a problem hiding this comment.
This is slightly confusing to me; I have to imagine it'll be even worse to anyone who wasn't in on the design discussion.
| * only safe for codes where no evaluations on neighbor cells (e.g. | ||
| * no jump error estimators) are done. | ||
| * | ||
| * Periodic constraints on a distributed mesh need every element |
| bool check_periodic_bcs = | ||
| (_periodic_bcs && !_periodic_bcs->empty()); | ||
| const std::vector<const PeriodicBoundaries *> periodic_bc_sets = | ||
| this->periodic_boundaries_used(_periodic_bcs); |
There was a problem hiding this comment.
This may be a bit expensive for DefaultCoupling in particular? Coupling functors get called on individual elements, not just ranges of elements, so we can't amortize over a whole range the way we do with geometric and algebraic ghosting. Any way we can move the computation here to someplace (mesh_reinit()?) where we can cache the results so we can just return by reference here?
| bool check_periodic_bcs = !periodic_bc_sets.empty(); | ||
|
|
||
| // A side with a regular neighbor is examined once, without periodic boundaries | ||
| const std::vector<const PeriodicBoundaries *> no_periodic_bcs {nullptr}; |
There was a problem hiding this comment.
Any reason not to make this static too?
| (!elem->neighbor_ptr(s) && check_periodic_bcs) ? | ||
| periodic_bc_sets : no_periodic_bcs) | ||
| #endif | ||
| { |
There was a problem hiding this comment.
We should fix our indentation, here and in the other ghosting functors where we've added the additional loop layer.
There was a problem hiding this comment.
lol I'm pretty sure that Claude actually raised this in the turn it did this and stated the reason for not indenting as reducing the diff for review. I forget whether git diff -w fixes that issue. I think not, in which case it's not the worst reason in the world but I remember thinking bah I need to make sure to scold it ... and then I forgot even when reviewing the diff before moving off of draft
|
|
||
|
|
||
| const std::vector<Elem *> & | ||
| MeshRefinement::topological_neighbors(Elem * elem, |
There was a problem hiding this comment.
Why this idiom here vs the additional nested loop in the ghosting functors? I can see pros and cons for each but I'd have thought that the same idiom would have won out in both cases.
There was a problem hiding this comment.
In the ghosting functors we already deduplicate naturally through insertion into the ghosting map, O(log N), so a similar idiom in the ghosting functors would be adding unnecessary O(N) std::find deduplication. Conversely, for four of the six loop structures in mesh refinement if we added a loop inside of the side index range loop over the periodic boundaries sets, the break statements would no longer be valid. We'd need to use something like agoto instead. If you feel strongly about having the same idiom in both places, then we could make something happen (I'd probably prefer changing to mesh refinement loops to use goto)
|
I accidentally hit "Submit" while double-checking all the entries before changing "Comment" to "Request Changes". Anyway, the only thing I'm really concerned about is adding more overhead to |
PeriodicBoundary::get_corresponding_pos() is now final, so any PeriodicBoundary maps points by its translation alone. same_mapping() then needs only a dynamic_cast, and it accepts derived classes. Without RTTI, same_mapping() returns false. Each set is then searched on its own, which costs time but gives the same neighbors. Co-Authored-By: Claude <noreply@anthropic.com>
The free function remove_redundant_periodic_boundaries() becomes the static member PeriodicBoundaries::remove_redundant_sets(). The class name already says "periodic", and the function removes whole sets. Co-Authored-By: Claude <noreply@anthropic.com>
A ghosting functor searches either its own periodic boundary set or every set registered with the mesh. The choice is now a property of the functor, so MeshBase no longer tracks it per functor. - GhostingFunctor::use_mesh_periodic_boundaries() selects the mesh's sets. It is an error to combine it with set_periodic_boundaries(). The default ghosting functor uses the mesh's sets. - MeshBase::add_ghosting_functor() loses its use_all_pbcs argument. - set_periodic_boundaries() accepts one set. Setting a different set later is an error. - A copied functor keeps the use_mesh_periodic_boundaries() setting but does not copy its own set. Copies usually serve another mesh or DofMap. - GhostingFunctor::periodic_boundary_sets() returns a reference, so DefaultCoupling::operator() allocates nothing for each element. - MeshBase caches the registered sets after remove_redundant_sets(). Registration changes and DofMap::add_periodic_boundary() invalidate the cache. build_sparsity() rebuilds it before its threaded loop. - The non-const DofMap::get_periodic_boundaries() is deprecated. Changes through its pointer bypass the DofMap, so the mesh cache could go out of date. Co-Authored-By: Claude <noreply@anthropic.com>
Ghosting functors and
MeshRefinementnow see the periodic boundaries of everyDofMapon a mesh. #4649 gave only the firstDofMap's set to the mesh's ghosting functors. If the first system has no periodic boundaries, a later system's periodic constraints can fail on a refinedDistributedMesh(#1728).DofMapregisters itsPeriodicBoundarieswithMeshBase::add_periodic_boundaries().set_periodic_boundaries(), or every registered set, afteruse_mesh_periodic_boundaries(true). The default ghosting functor uses every set.DofMapcoupling and algebraic functors keep their own set, so other systems do not widen their sparsity pattern or send list.use_mesh_periodic_boundaries(true).DofMap.PeriodicBoundaryBase::same_mapping()).DofMap::add_periodic_boundary()invalidates the cache.DofMap::get_periodic_boundaries()is deprecated, because changes through its pointer bypass the cache.MeshRefinementuses every registered set.set_periodic_boundaries_ptr()is deprecated, andadaptivity_ex5no longer calls it. Its results are unchanged.refine_elements()andcoarsen_elements()now callprepare_for_use()before their level-one asserts. The old point locator could not find new children.Refs #1728.
🤖 Generated with Claude Code