I’ve written a python script that calls a package that interacts with cubit via the python API, and running into the following error:
ImportError: /opt/Coreform-Cubit-2021.11/bin/./libcubiti19.so: undefined symbol: H5Pset_dxpl_mpio
Any idea what might be going wrong here?
What is strange is that if I take the function call that uses cubit and put it into its own script, it works fine, but in this script, where the function call is within another function, I am getting this error.
Most likely your other function is loading a different HDF5 library from somewhere other than the Coreform-Cubit-2021.11/bin folder. That other HDF5 library doesn’t have the function H5Pset_dxpl_mpio that we require, thus the error. Do you have another installation of HDF5 somewhere on your system?
likely – as I’m currently running in a conda environment due to cubit’s incompatibility with the latest python version.
I think currently I have h5py installed via pip in both the environment and on my system. Any ideas on how to address/debug the issue?
We also use conda to build Cubit. Try adding hdf5=1.12.*=mpi_openmpi* to your conda environment, that should match the libhdf5 we use to build Cubit.
Didn’t resolve the issue. To clarify, you meant to export that as an environment variable? What I did was
export hdf5=1.12.*=mpi_openmpi*
and then ran the script again
No, you need to add that specification to your conda environment file.
Ok, still running into the same issue. I tried:
conda env config vars set hdf5=1.12.*=mpi_openmpi*
then reactivated the environment. Did not resolve.
conda env config vars sets an environment variable within your conda environment. The hdf5=1.12.*=mpi_openmpi* is not an environment variable, it’s a version specification string to tell conda what version of hdf5 to install in your environment. The string tells conda to install the openmpi variant of HDF5 version 1.12. Use it like this:
conda install hdf5=1.12.*=mpi_openmpi* <environment name>
If that fails to solve, that likely means that your environment has conflicting library dependencies. You can try to loosen that by changing the version string to something like this: hdf5=*=mpi, which won’t pin the version of HDF5 to be the same as Cubit’s, but at least it’ll get a parallel version built against MPI.
When I run that I get the following error
File "/usr/lib/python3.10/site-packages/conda/models/match_spec.py", line 760, in merge
raise ValueError("Incompatible component merge:\n - %r\n - %r"
ValueError: Incompatible component merge:
- 'mpi_mpich_*'
- 'mpi_mpich_tempest*'
Looks like this is an unresolved conda install issue, see conda install creates `ValueError: Incompatible component merge:` · Issue #11442 · conda/conda · GitHub. In the meantime, can you think of another way around the error I’m seeing?
That’s not really a problem with conda itself, it has to do with conflicting mpi versions. You have libraries in your conda environment that use the MPICH implementation of MPI, and Cubit uses the OpenMPI implementation of MPI. The two are not compatible. This is an unfortunate feature of the scientific computing landscape, almost everything needs to be built against the same implementation of MPI. The solution here is to go through your conda environment and make sure all your dependencies use the OpenMPI variant instead of the the MPICH variant, specifying the *mpi_openmpi section of the version string like above. It’s probably best if you create an environment.yml file and use that to fully specify your conda environment, see Managing environments — conda 4.13.0.post29+660f9d90 documentation.
This is a huge pain, unfortunately, but it’s the only way of making sure that all your dependencies are compatible. In computer science we call this dependency hell, and it’s something you’re sure to run into once your C/C++ program does much of anything non-trivial. Python tries to handle this for you, but Python is really just C under the hood and there’s only so much it can do.
Thanks for the answer, will look into this.
When will cubit’s python compatibility be updated past 3.8?
In the near future. We don’t have an exact date on that, but I’d say a reasonable expectation would be within a few months.