feature/dualtree #2

Open

christoph wants to merge 0 commits from feature/dualtree into main

pull from: feature/dualtree

merge into: christoph:main

christoph:main

christoph:ssevec

christoph:experiments

christoph:debug

christoph:master

christoph:structs

Conversation 0 Commits - Files changed -

christoph commented 2025-01-15 10:19:21 +01:00

Owner

Copy link

No description provided.

christoph added 40 commits 2025-01-15 10:19:21 +01:00

Call interface & unit tests of DualTreeTopology ... 6a55d02779

This new algorithm will use two binary trees, one  that defines the communication order, while the other defines the reduction order.

Helper functions for binary trees +tests 81306899df

Merge stash again 9ac4566477

Compute only outgoing intermediary results ... 5a8c160587

This excludes values that are just passed along the communication tree.

Fix unit test and boundary condition concerning subtree-locality b990625c29

Add Fuzzer 11544d30de

Add getter for comm tree children 73dba21b35

Functional dual tree implementation 90486fb279

Use full dual tree summation procedure in fuzzer d4a93fa203

Full Dual Tree Summation implementation ... 097a66a8f7

Support distributions with 0-length regions & arbitrary rank ordering
Fix various bugs related to 0-length regions
Clarify differences between rank & array order

Use iterative base-case in DualTreeSummation 681814f0ed

Pass -mavx2 to compiler 80b4b61ec5

Manually vectorize DualTreeSummation a11155b387

Remove dependency on std::format in release mode ... 0f89700996

Remove ubsan library flag

replace with in error case d69c01845a

Remove ubsan 052c1d4df1

Use MPI_STATUS_IGNORE instead of nullptr c5175160b1

Adapt message tag to fit into range guaranteed by MPI standard (max value is 32767) 07a2bc7ad2

Use unaligned AVX-2 loads ... 9a533f12a3

We can not fullfil the alignment criteria.

Fix passthrough criterion ... 745492ef8c

Previously, the algorithm would not recognize an element as passthrough if its subtree was not comm-local.

Add API calls for getting the reduciton mode 153bfd9266

Remove newlines from release output a2d29673d4

Re-enable random seed in fuzzer, use C-style distribution printing 9dc31796cd

Refine difficult distributions test bf4e9f4980

Restructure topology to support reduction of to received elements 3233f4e417

Fix unit tests, rename locally computed to outgoing 18d07f74af

Expand error message of fuzzer 681ffb23ea

Remove std::format in favor of printf for backwards compatibility 16776eced5

Remove std::format in favor of printf for backwards compatibility c5704eb401

Fix logging messages, print local array 47ec59894b

Replace C++ closure with block 70aa77804a

Remove with_comm_size_n statement b7ee516504

Replace scatterv in favor of std::copy_n ab5f333283

Replace std::map with std::unordered_map 0dc58880c4

Add profiling regions for dual tree summation ... 0d4f11d325

Move ScoreP compiler declaration to top of CMake file to avoid issues with build cache invalidation

Add missing region surrounding DTS benchmark ... 8801f20520

This tidies up the profiling output

Use pre-computed iterations instead of recursive accumulation procedure 12662f331f

Move to implicit coordinate indexing ... 140182f7b1

Previously, we stored incoming values in a map with the tree coordinate as key.
We now rely on the fact that the algorithm receives and accesses these values in-order.

Try to fix maximum stack size calculation, add documentation 0d69051d12

Fix stack size computation ... 5d27c546e0

While the reduce operation takes two operands from the stack, it also pushes it result resulting in a net stack size change of -1.

christoph added 1 commit 2025-01-15 11:24:13 +01:00

Include multiple allreduce types ... 5e31e318c6

Reduce + Bcast
Allreduce
Allreduce with `par_unseq`

christoph added 1 commit 2025-01-15 11:30:52 +01:00

Prevent compiler optimizations on result a05e924d08

christoph added 1 commit 2025-01-16 11:03:27 +01:00

remove vectorization from default allreduce variant 3ebeb833d1

christoph added 3 commits 2025-01-17 13:32:01 +01:00

Add DUAL_TREE mode to C library e9ba95b276

Rearrange code, clang-tidy 563e6ffd6a

Refactor DualTreeSummation constructor 5beb74e7b2

christoph added 2 commits 2025-01-19 16:40:01 +01:00

Add topology class for m-ary tree c91092dfe5

Dual tree summation with m-ary comm tree 02505085a0

christoph added 1 commit 2025-01-19 22:25:01 +01:00

Refactor operations into topology e61a35b759

christoph added 5 commits 2025-01-20 14:34:35 +01:00

Delete unneeded subtree helper cf2bd836ce

Add calculator for Dual Tree message counts 42cd540316

Output number of outgoing messages and communication parent 975d5d3ece

Fix assertion, m must be greater than 2 53a50c74f9

Add m to benchmark parameters 0e5d0d6533

christoph added 1 commit 2025-01-20 14:36:30 +01:00

ASSERT rather than EXPECT in fuzzer test 49aa9fa29c

christoph added 1 commit 2025-01-20 14:55:52 +01:00

Remove tbb, use stdbool 1b792764d9

christoph added 1 commit 2025-01-20 15:28:32 +01:00

Conditionally link against TBB ... 3f91fd2168

Required because of `std::execution::par_unseq` if TBB is installed.

This pull request has changes conflicting with the target branch.

• src/allreduce_summation.cpp
• src/c_wrapper.cpp

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feature/dualtree:feature/dualtree

git switch feature/dualtree

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff feature/dualtree

git switch feature/dualtree

git rebase main

git switch main

git merge --ff-only feature/dualtree

git switch feature/dualtree

git rebase main

git switch main

git merge --no-ff feature/dualtree

git switch main

git merge --squash feature/dualtree

git switch main

git merge --ff-only feature/dualtree

git switch main

git merge feature/dualtree

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

christoph/reprored!2

No description provided.