Commit bc7d89a7 authored by Alexis SALZMAN's avatar Alexis SALZMAN
Browse files

[xMeshTool] add a new Parmetis partition function with weight

This commit is adding template function setParmetisPartitionWithWeight that
use a set of weights per element to optimize ParMetis partitioning based on
multi criteria (the weights).  The template parameter (an int) is fixing the
number of weights per element. The interface is kept simple. Only an extra
xAttachedDataManagerAOMD is used to passe weights set per element.

No test case for now but it have been heavily tested in some apps.

This function is using almost the same algo as the one used for
setParmetisPartition in the cc file. It is rather dirty (code
duplication) but efficient.

It add also a new dependency in terms of cmake. xInterfaceParMetis must
now be added in cmake to use xLoadBalanceTools. But anyway from my point
of view it should already be the case. Normally xInterfaceParMetis should
embedded ParMetis and in cmake finding metis/parmetis lib should not be
done.... Modern cmake would do even more by setting all this dependency
setting for you .....
parent 3af9ff86
/*
/*
This file is a part of eXlibris C++ Library
under the GNU Lesser General Public License.
See the NOTICE.md & LICENSE.md files for terms
See the NOTICE.md & LICENSE.md files for terms
and conditions.
*/
#ifndef XLOADBALANCETOOLS
#define XLOADBALANCETOOLS
#include <vector>
#include <algorithm>
#include <iostream>
#include <vector>
#include "xAttachedDataManagerAOMD.h"
#include "xPartitionManager.h"
#include "xDataExchanger.h"
#include "xPartitionManager.h"
// graph_tools
#include "ParMetisInterface.h"
namespace xmeshtool
{
class moreDataEntity;
typedef xtool::xPartitionManager<xinterface::aomd::xAttachedDataManagerAOMD> partitionManager;
class moreDataEntity;
typedef xtool::xPartitionManager< xinterface::aomd::xAttachedDataManagerAOMD > partitionManager;
/// \brief given a mesh, a partition manager and targetproc migrate mesh entity of the mesh accross processor. The call is collective among all the processors.
///
/// The mesh and the partition manager must describe a "valid" partition : \n
/// - A top level entity (an entity that has no upward adjacencies) must be present only on one proc. \n
/// - The partition manager on entry must be correct. \n
/// Note that if the mesh is empty on all procs but one, the partition manager is correct if empty \n
/// ... It means that you can call this function on a mesh not yet partitionned ...\n
/// The input mesh must : \n
/// - be a mesh of simplices (only TET, TRI, EDGE and VERTEX entities) \n
/// - A complet mesh, meaning it contains all entities: \n
/// for each tet, all bounding tri must be present, for each tri all the bounding edges must be present, \n
/// for each edge all vertex must be present \n
/// Note that it is typically the case in xFem library (see xMesh construction : -> ModifyAllState) \n
/// - Only the one level adjacencies are needed : TET<->TRI<->EDGE<->VERTEX \n
/// If more are present, they will be ignored.\n
/// The \param targetproc drive the entities migration. All top level entity of the input mesh must have a value associated to it in this datamanager \n
/// this value tell on which proc one want to have the entity after the call to this function.
/// On output, If successful, on each proc :
/// - the mesh contain only the toplevelentities as defined by target, is complete and has all the one level adjacencies. \n
/// Note that other adjacencies are NOT maintained. \m
/// - The part_man is updated accordingly and correct on each proc. \n
/// pmore_data_entity is an optional argument to let the user communicate and clean up additional data associated to elements data
void migrateEntities(AOMD::mMesh &m, partitionManager & part_man, xinterface::aomd::xAttachedDataManagerAOMD< int > &targetprocfortoplevelentities, moreDataEntity * pmore_data_entity = nullptr, bool fixed_vertex_id=true);
/// \brief moreDataEntity is a base class to define call back used when an entity migrate entities.
///
/// Typical usage is to ensure some data associated to an entity can be retrieved after the migration, or to add some debugging messaga
class moreDataEntity{
/// \brief given a mesh, a partition manager and targetproc migrate mesh entity of the mesh accross processor. The call is
/// collective among all the processors.
///
/// The mesh and the partition manager must describe a "valid" partition : \n
/// - A top level entity (an entity that has no upward adjacencies) must be present only on one proc. \n
/// - The partition manager on entry must be correct. \n
/// Note that if the mesh is empty on all procs but one, the partition manager is correct if empty \n
/// ... It means that you can call this function on a mesh not yet partitionned ...\n
/// The input mesh must : \n
/// - be a mesh of simplices (only TET, TRI, EDGE and VERTEX entities) \n
/// - A complet mesh, meaning it contains all entities: \n
/// for each tet, all bounding tri must be present, for each tri all the bounding edges must be present, \n
/// for each edge all vertex must be present \n
/// Note that it is typically the case in xFem library (see xMesh construction : -> ModifyAllState) \n
/// - Only the one level adjacencies are needed : TET<->TRI<->EDGE<->VERTEX \n
/// If more are present, they will be ignored.\n
/// The \param targetproc drive the entities migration. All top level entity of the input mesh must have a value associated to it
/// in this datamanager \n
/// this value tell on which proc one want to have the entity after the call to this function.
/// On output, If successful, on each proc :
/// - the mesh contain only the toplevelentities as defined by target, is complete and has all the one level adjacencies. \n
/// Note that other adjacencies are NOT maintained. \m
/// - The part_man is updated accordingly and correct on each proc. \n
/// pmore_data_entity is an optional argument to let the user communicate and clean up additional data associated to elements
/// data
void migrateEntities(AOMD::mMesh &m, partitionManager &part_man,
xinterface::aomd::xAttachedDataManagerAOMD<int> &targetprocfortoplevelentities,
moreDataEntity *pmore_data_entity = nullptr, bool fixed_vertex_id = true);
/// \brief moreDataEntity is a base class to define call back used when an entity migrate entities.
///
/// Typical usage is to ensure some data associated to an entity can be retrieved after the migration, or to add some debugging
/// messaga
class moreDataEntity
{
public:
/// getInfo is called each time an entity will be send to proc sendto. the user as access to the buffer that will be send and can pack data into it.
virtual void getInfo( const AOMD::mEntity &entity, xtool::xMpiInputBuffer & buff , int sendto) const = 0;
/// setInfo is called just after an entiy has beencreated on a proc. all downward adjancencies to entity are allready present in the mesh.
/// Data set by the corresponding getInfo in proc with id receivedfrom
virtual void setInfo( AOMD::mEntity &entity, const xtool::xMpiOutputBuffer & buff , int receivedfrom ) = 0;
/// delEntity is called each time an entity will be delete. typical usage is to clean up some data associated to the entity, before it disappear from the mesh.
virtual void delEntity(AOMD::mEntity &entity) = 0;
};
/// \brief setRandomPartition set up random target values between 0 and nb_part for each top level entity in m.
xinterface::aomd::xAttachedDataManagerAOMD< int > setRandomPartition(AOMD::mMesh &m, int nb_part );
/// getInfo is called each time an entity will be send to proc sendto. the user as access to the buffer that will be send and
/// can pack data into it.
virtual void getInfo(const AOMD::mEntity &entity, xtool::xMpiInputBuffer &buff, int sendto) const = 0;
/// setInfo is called just after an entiy has beencreated on a proc. all downward adjancencies to entity are allready present
/// in the mesh. Data set by the corresponding getInfo in proc with id receivedfrom
virtual void setInfo(AOMD::mEntity &entity, const xtool::xMpiOutputBuffer &buff, int receivedfrom) = 0;
/// delEntity is called each time an entity will be delete. typical usage is to clean up some data associated to the entity,
/// before it disappear from the mesh.
virtual void delEntity(AOMD::mEntity &entity) = 0;
};
/// \brief setRandomPartition set up random target values between 0 and nb_part for each top level entity in m.
xinterface::aomd::xAttachedDataManagerAOMD<int> setRandomPartition(AOMD::mMesh &m, int nb_part);
/// \brief setParmetisPartition set up target proc values between 0 and nb_part for each top level entity in m, using parmetis. a
/// global numebring is needed for this version.
xinterface::aomd::xAttachedDataManagerAOMD<int> setParmetisPartition(
AOMD::mMesh &m, const partitionManager &part_man, const xinterface::aomd::xAttachedDataManagerAOMD<int> &global_vertex_id,
int nb_part);
/// \brief setParmetisPartition set up target proc values between 0 and nb_part for each top level entity in m, using parmetis.
/// Same as above, but the global numbering is created first internally and then destroyed.
xinterface::aomd::xAttachedDataManagerAOMD<int> setParmetisPartition(AOMD::mMesh &m, const partitionManager &part_man,
int nb_part);
/// \brief setParmetisPartitionWithWeight set up target proc values between 0 and nb_part for each top level entity in m, using
/// parmetis and given weight.
template <int NBW = 1>
xinterface::aomd::xAttachedDataManagerAOMD<int> setParmetisPartitionWithWeight(
AOMD::mMesh &m, const partitionManager &part_man, int nb_part,
xinterface::aomd::xAttachedDataManagerAOMD<std::array<xinterface::parmetis::ParMetisInterface::parmetis_indx_t, NBW>>
&weights);
/// \brief setParmetisPartition set up target proc values between 0 and nb_part for each top level entity in m, using parmetis. a global numebring is needed for this version.
xinterface::aomd::xAttachedDataManagerAOMD< int > setParmetisPartition(AOMD::mMesh &m, const partitionManager &part_man, const xinterface::aomd::xAttachedDataManagerAOMD< int > &global_vertex_id, int nb_part );
/// \brief number all vertices of the mesh so that vertices present in the partitionManager have the same number accros all
/// process. Collective among all the process of the MPI_Comm part_man was build with.
inline xinterface::aomd::xAttachedDataManagerAOMD<int> globalVerticesNumbering(AOMD::mMesh &m, const partitionManager &part_man);
/// \brief setParmetisPartition set up target proc values between 0 and nb_part for each top level entity in m, using parmetis. Same as above, but the global numbering is created first internally and then destroyed.
xinterface::aomd::xAttachedDataManagerAOMD< int > setParmetisPartition(AOMD::mMesh &m, const partitionManager &part_man, int nb_part );
/// \brief number all vertices of the mesh so that vertices present in the partitionManager have the same number accros all process. Collective among all the process of the MPI_Comm part_man was build with.
inline xinterface::aomd::xAttachedDataManagerAOMD< int > globalVerticesNumbering(AOMD::mMesh &m, const partitionManager &part_man );
/// \brief number all entities between b and e so that entities present in the partitionManager have the same number accros all
/// process. Collective among all the process of the MPI_Comm part_man was build with.
template <class ITER>
inline xinterface::aomd::xAttachedDataManagerAOMD<int> globalEntitiesNumbering(const ITER &b, const ITER &e,
const partitionManager &partman);
/// \brief number all entities between b and e so that entities present in the partitionManager have the same number accros all process. Collective among all the process of the MPI_Comm part_man was build with.
template<class ITER>
inline xinterface::aomd::xAttachedDataManagerAOMD< int > globalEntitiesNumbering(const ITER &b, const ITER &e, const partitionManager & partman);
/// \brief export partition to a stream
void exportPartition(AOMD::mMesh &m, std::ofstream &f,
xinterface::aomd::xAttachedDataManagerAOMD<int> &targetprocfortoplevelentities,
xinterface::aomd::xAttachedDataManagerAOMD<int> &entities_id);
/// \brief export partition to a stream
void exportPartition(AOMD::mMesh &m, std::ofstream &f, xinterface::aomd::xAttachedDataManagerAOMD< int >&targetprocfortoplevelentities,xinterface::aomd::xAttachedDataManagerAOMD< int >&entities_id);
void exportGMSH_V2_Dist(const AOMD::mMesh &m, const partitionManager &part_man, const std::string &filename_base);
void exportGMSH_V2_Dist(const AOMD::mMesh &m, const partitionManager &part_man, const std::string &filename_base);
} // end namespace xfem
} // namespace xmeshtool
#include "xLoadBalanceTools_imp.h"
#endif
/*
/*
This file is a part of eXlibris C++ Library
under the GNU Lesser General Public License.
See the NOTICE.md & LICENSE.md files for terms
See the NOTICE.md & LICENSE.md files for terms
and conditions.
*/
// standard includes
#include <vector>
// standard includes
#include <algorithm>
#include <fstream>
#include <functional>
#include <vector>
// Trellis/AOMD
#include "mMesh.h"
#include "mVertex.h"
#include "mEdge.h"
#include "mFace.h"
#include "mMesh.h"
#include "mRegion.h"
#include "mTet.h"
#include "mVertex.h"
#include "modeler.h"
// Trellis/model
#include "GEntity.h"
//Accumulate
// Accumulate
#include <numeric>
//Macro
// Macro
#include "xMacro.h"
namespace xmeshtool
{
inline auto range( const AOMD::mMesh & m, int dim) -> xtool::xRange<decltype(m.begin(dim))> {
assert (dim >= 0 && dim <=3);
return xtool::xRange< decltype(m.begin(dim)) > (m.begin(dim), m.end(dim) );
}
inline auto range(const AOMD::mMesh &m, int dim) -> xtool::xRange<decltype(m.begin(dim))>
{
assert(dim >= 0 && dim <= 3);
return xtool::xRange<decltype(m.begin(dim))>(m.begin(dim), m.end(dim));
}
class exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD{
class exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD
{
public:
exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD( const partitionManager & _partman ):partman(_partman){}
typedef const AOMD::mEntity * information_key_t;
information_key_t localObjectKey( const AOMD::mEntity & o) const {
return &o;
}
information_key_t remoteObjectKey(const xtool::xRemoteObject< AOMD::mEntity > & ro, const AOMD::mEntity &lo ) const{
exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD(const partitionManager &_partman) : partman(_partman) {}
typedef const AOMD::mEntity *information_key_t;
information_key_t localObjectKey(const AOMD::mEntity &o) const { return &o; }
information_key_t remoteObjectKey(const xtool::xRemoteObject<AOMD::mEntity> &ro, const AOMD::mEntity &lo) const
{
return ro.getObjectAddress();
}
xtool::xConstPartitionObject<AOMD::mEntity > getConstPartitionObject(const AOMD::mEntity &e) const{
}
xtool::xConstPartitionObject<AOMD::mEntity> getConstPartitionObject(const AOMD::mEntity &e) const
{
return partman.getConstPartitionObject(e);
}
protected :
const partitionManager &partman;
};
// strategy : all proc first count the vertices they own, then exchanges this numbers with an MPI_Scan. Now all the process know where to start the numbering of there owned vertices.
// numbering is done. then not owned vertices retrieve there number from owner proc during exchange.
//
template<class ITER>
inline xinterface::aomd::xAttachedDataManagerAOMD< int > globalEntitiesNumbering(const ITER &b, const ITER &e, const partitionManager & part_man){
xtool::xRange< ITER > entities_range(b,e);
const int nbowned = countOwned( b, e , part_man );
const MPI_Comm comm = part_man.getComm();
int iend;
MPI_Scan( const_cast<int *>(&nbowned), &iend, 1, MPI_INT, MPI_SUM, comm );
const int istart = iend - nbowned;
xinterface::aomd::xAttachedDataManagerAOMD< int > entities_numbering;
int i =istart;
for(auto p_entity : entities_range ){
if (part_man.getConstPartitionObject(*p_entity).isOwner()) entities_numbering.setData(*p_entity) = i++;
}
class exchangerInfoManagerNumberEntities {
public:
exchangerInfoManagerNumberEntities( xinterface::aomd::xAttachedDataManagerAOMD< int >& _entities_numbering ): entities_numbering(_entities_numbering){}
typedef xtool::homogeneous_data_style_trait data_style_trait EXLIBRIS_MACRO_WARNUNUSEDTYPE;
typedef xtool::send_and_recv_keys_communication_trait communication_trait EXLIBRIS_MACRO_WARNUNUSEDTYPE;
typedef const AOMD::mEntity * information_key_t;
typedef int information_t;
information_t getInfo( information_key_t key, int sendto){
const AOMD::mEntity & e = *key;
const int * i = entities_numbering.getData(e);
assert(i);
return *i;
}
protected:
const partitionManager &partman;
};
// strategy : all proc first count the vertices they own, then exchanges this numbers with an MPI_Scan. Now all the process know
// where to start the numbering of there owned vertices. numbering is done. then not owned vertices retrieve there number from
// owner proc during exchange.
//
template <class ITER>
inline xinterface::aomd::xAttachedDataManagerAOMD<int> globalEntitiesNumbering(const ITER &b, const ITER &e,
const partitionManager &part_man)
{
xtool::xRange<ITER> entities_range(b, e);
const int nbowned = countOwned(b, e, part_man);
const MPI_Comm comm = part_man.getComm();
int iend;
MPI_Scan(const_cast<int *>(&nbowned), &iend, 1, MPI_INT, MPI_SUM, comm);
const int istart = iend - nbowned;
xinterface::aomd::xAttachedDataManagerAOMD<int> entities_numbering;
int i = istart;
for (auto p_entity : entities_range)
{
if (part_man.getConstPartitionObject(*p_entity).isOwner()) entities_numbering.setData(*p_entity) = i++;
}
class exchangerInfoManagerNumberEntities
{
public:
exchangerInfoManagerNumberEntities(xinterface::aomd::xAttachedDataManagerAOMD<int> &_entities_numbering)
: entities_numbering(_entities_numbering)
{
}
typedef xtool::homogeneous_data_style_trait data_style_trait EXLIBRIS_MACRO_WARNUNUSEDTYPE;
typedef xtool::send_and_recv_keys_communication_trait communication_trait EXLIBRIS_MACRO_WARNUNUSEDTYPE;
typedef const AOMD::mEntity *information_key_t;
typedef int information_t;
information_t getInfo(information_key_t key, int sendto)
{
const AOMD::mEntity &e = *key;
const int *i = entities_numbering.getData(e);
assert(i);
return *i;
};
void setInfo(information_key_t key, const information_t &i, int receivedfrom){
AOMD::mEntity &e = const_cast<AOMD::mEntity &>(*key);
assert(!entities_numbering.getData(e)); // entity e should not have yet a number at this point.
entities_numbering.setData(e) = i;
void setInfo(information_key_t key, const information_t &i, int receivedfrom)
{
AOMD::mEntity &e = const_cast<AOMD::mEntity &>(*key);
assert(!entities_numbering.getData(e)); // entity e should not have yet a number at this point.
entities_numbering.setData(e) = i;
};
private:
xinterface::aomd::xAttachedDataManagerAOMD< int >& entities_numbering;
};
typedef const AOMD::mEntity * key_t;
typedef exchangerInfoManagerNumberEntities exchangerInfoManager;
typedef exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD exchangerKeyManager;
typedef xtool::xKeyContainerSendAndRecv< key_t > exchangerKeyContainer;
exchangerInfoManager exchanger_info_manager( entities_numbering);
exchangerKeyManager exchanger_key_manager( part_man);
exchangerKeyContainer exchanger_key_container( part_man.getComm());
exchanger_key_container.accumulateKeysOwnerScatter( b, e, exchanger_key_manager );
exchangeInformation(exchanger_key_container, exchanger_info_manager );
return entities_numbering;
}
inline xinterface::aomd::xAttachedDataManagerAOMD< int > globalVerticesNumbering(AOMD::mMesh &m, const partitionManager &part_man ){
auto vrange = range( m, 0);
return globalEntitiesNumbering(vrange.begin(), vrange.end(), part_man);
}
} // end namespace
private:
xinterface::aomd::xAttachedDataManagerAOMD<int> &entities_numbering;
};
typedef const AOMD::mEntity *key_t;
typedef exchangerInfoManagerNumberEntities exchangerInfoManager;
typedef exchangerKeyManagerSimpleSendAndReceiveKeysFromPartitionManagerAOMD exchangerKeyManager;
typedef xtool::xKeyContainerSendAndRecv<key_t> exchangerKeyContainer;
exchangerInfoManager exchanger_info_manager(entities_numbering);
exchangerKeyManager exchanger_key_manager(part_man);
exchangerKeyContainer exchanger_key_container(part_man.getComm());
exchanger_key_container.accumulateKeysOwnerScatter(b, e, exchanger_key_manager);
exchangeInformation(exchanger_key_container, exchanger_info_manager);
return entities_numbering;
}
inline xinterface::aomd::xAttachedDataManagerAOMD<int> globalVerticesNumbering(AOMD::mMesh &m, const partitionManager &part_man)
{
auto vrange = range(m, 0);
return globalEntitiesNumbering(vrange.begin(), vrange.end(), part_man);
}
inline bool isTopLevel(const AOMD::mEntity &e)
{
const int level = e.getLevel();
if (level == 3)
return true;
else
return !(e.size(level + 1));
}
template <int NBW>
xinterface::aomd::xAttachedDataManagerAOMD<int> setParmetisPartitionWithWeight(
AOMD::mMesh &m, const partitionManager &part_man, int nb_part,
xinterface::aomd::xAttachedDataManagerAOMD<std::array<xinterface::parmetis::ParMetisInterface::parmetis_indx_t, NBW>>
&weights)
{
xinterface::aomd::xAttachedDataManagerAOMD<int> global_vertex_id = globalVerticesNumbering(m, part_man);
std::array<int, 4> nb_top_dim = {{0, 0, 0, 0}};
for (int dim = 0; dim < 3; ++dim)
{
for (auto pentity : range(m, dim))
{
if (isTopLevel(*pentity)) ++nb_top_dim[dim];
}
}
nb_top_dim[3] = m.size(3);
const int nb_top = std::accumulate(nb_top_dim.begin(), nb_top_dim.end(), 0);
MPI_Comm comm = part_man.getComm();
int rank;
MPI_Comm_rank(comm, &rank);
int color = (nb_top) ? 1 : 0;
MPI_Comm newcomm;
MPI_Comm_split(comm, color, rank, &newcomm);
xinterface::aomd::xAttachedDataManagerAOMD<int> futurelem_rank;
if (color)
{
xinterface::parmetis::ParMetisInterface::parmetis_indx_t ncon = NBW;
int new_mpi_size;
MPI_Comm_size(newcomm, &new_mpi_size);
int iend;
MPI_Scan(const_cast<int *>(&nb_top), &iend, 1, MPI_INT, MPI_SUM, newcomm);
int istart = iend;
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_indx_t> elmdist(new_mpi_size + 1, 0);
MPI_Allgather(&istart, 1, MPI_INT, elmdist.data() + 1, 1, MPI_INT, newcomm);
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_indx_t> eptr(nb_top + 1);
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_indx_t> eind(nb_top_dim[0] + 2 * nb_top_dim[1] +
3 * nb_top_dim[2] + 4 * nb_top_dim[3]);
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_indx_t> elmwgt(nb_top * ncon);
eptr[0] = 0;
int pos = 1;
int weight_idx = 0;
xinterface::parmetis::ParMetisInterface::parmetis_indx_t index = 0;
for (int dim = 0; dim <= 3; ++dim)
{
for (auto pentity : range(m, dim))
{
if (isTopLevel(*pentity))
{
if (dim == 0)
{
const int *pnodeid = global_vertex_id.getData(*pentity);
assert(pnodeid);
eind[index++] = *pnodeid;
}
else
{
for (int in = 0; in < dim + 1; ++in)
{
const int *pnodeid = global_vertex_id.getData(*(pentity->get(0, in)));
assert(pnodeid);
eind[index++] = *pnodeid;
}
}
eptr[pos++] = index;
auto *pw = weights.getData(*pentity);
xinterface::parmetis::ParMetisInterface::parmetis_indx_t *pew = &elmwgt[weight_idx];
if (pw)
{
auto &rw = *pw;
for (int i = 0; i < NBW; ++i) pew[i] = rw[i];
}
else
{
cout << "Entity ...TODO... got no weight => null by default" << endl;
for (int i = 0; i < NBW; ++i) pew[i] = 0;
}
weight_idx += NBW;
}
}
}
xinterface::parmetis::ParMetisInterface::parmetis_indx_t wgtflag = 2;
xinterface::parmetis::ParMetisInterface::parmetis_indx_t numflag = 0;
xinterface::parmetis::ParMetisInterface::parmetis_indx_t ncommonnodes = 3;
if (nb_top_dim[2]) ncommonnodes = 2;
if (nb_top_dim[1]) ncommonnodes = 1;
xinterface::parmetis::ParMetisInterface::parmetis_indx_t nparts = nb_part;
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_real_t> tpwgts(ncon * nparts, 1.f / nparts);
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_real_t> ubvec(ncon, 1.05);
xinterface::parmetis::ParMetisInterface::parmetis_indx_t edgecut;
std::vector<xinterface::parmetis::ParMetisInterface::parmetis_indx_t> part(nb_top, -1);
std::array<xinterface::parmetis::ParMetisInterface::parmetis_indx_t, 3> options = {{1, 0, 0}};
// std::cout<< "start Parmetis " << std::endl;
xinterface::parmetis::ParMetisInterface::PartMeshKway(elmdist.data(), eptr.data(), eind.data(), elmwgt.data(), wgtflag,
numflag, ncon, ncommonnodes, nparts, tpwgts.data(), ubvec.data(),
options.data(), newcomm, edgecut, part.data());
int i = 0;
for (int dim = 0; dim <= 3; ++dim)
{
for (auto pentity : range(m, dim))
{
if (isTopLevel(*pentity))
{
futurelem_rank.setData(*pentity) = part[i++];
}
}
}
}
// std::cout<< "PARMETIS Done "<< "Edge cut "<< edgecut << std::endl;
return futurelem_rank;
}
} // namespace xmeshtool
......@@ -32,7 +32,7 @@ include_directories( ${PARMETIS_INCLUDE_DIR} ${METIS_INCLUDE_DIR} )
list(APPEND EXTERNAL_LIBRARIES ${PARMETIS_LIBRARIES} ${METIS_LIBRARIES})
list(APPEND xDistMesh_EXTERNAL_INCLUDES ${PARMETIS_INCLUDE_DIR} ${METIS_INCLUDE_DIR})
find_and_set(xInterfaceParMetis xInterfaceParMetis EXTERNAL_INCLUDES EXTERNAL_LIBRARIES REQUIRED )
set(LISTAOMDPARMETIS ${CMAKE_CURRENT_SOURCE_DIR}/testLoadBalanceToolsAOMD )
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment