How to use conda-pack to relocate your conda environment?
Conda has become a widely adopted environment management tool for python developers, and developers that need some extra packages outside from python. However, relocating conda environment can be a pain in the stomach.
Conda provided two obvious ways to help us reproduce an environment. One only copy within the same conda installation (Method 3). The second is exporting environment but it contains hashes that will not exist soon in the future, some fixes will make it work (Method 2).
Conda-pack appears to be the best solution if you want to relocate your conda environment to a new conda installation (in cases that you messed up the first one), or a different computer. Note: conda-pack only works when the os system types are the same between origin and target. (This means that environments built on windows can’t be relocated to linux.)
Here is a step-by-step tutorial on relocating a conda environment.
Is an independent package for solving this problem. So far it worked the best for me. (link: https://conda.github.io/conda-pack/)
Here’s how you should do it (I found the instruction on official documentation unclear. Therefore this post.)
- Install conda-pack in the environment you want to copy
conda activate my_envconda install conda-pack
2. Pack the environment as a tar.gz
# This stores the output my_env.tar.gz at current location
conda pack -n my_env# Alternatively, provide output location
conda pack -n my_env --output /dir/to/my_env.tar.gz
3. Do you have conda in the target machine? (where you’re copying the environment to?) If you don’t, you need to first install conda in the target machine. Please refer to the official guide for installing conda or miniconda.
4. *this step is the key* On the target machine, make a directory within the envs/ folder within conda directory.
mkdir -p /dir/to/miniconda3/envs/my_env
tar -xzf my_env.tar.gz -C /dir/to/miniconda3/envs/my_env
5. Source the copied environment, and unpack, then deactivate.
6. Validate your newly installed conda environment
conda env list
conda activate my_env
7. Worked? Enjoy!
Didn’t work :( ? Leave a comment below and I’ll try it out with you later! Or try method 2 for now.
- Conda pack official website: https://conda.github.io/conda-pack/
is a command line tool for creating relocatable conda environments. This is useful for deploying code in a consistent…
Method 2: Sharing the .yml file with the specific version of packages
conda env export > environment.yml
conda env create -f environment.yml
However, with this method, you get a long and nasty UUID behind each package. (The UUID indicating the specific version you were using) However, most often than not, these versions already does not exist on conda-forge repository therefore it cannot be found if you simply create the environment with the yml file you just got.
There are some workarounds, supposed your code should be compatible with similar versions or newer versions (which is your responsibility to maintain your code), you could trim the suffix of all versions until the second “.” to make it work. (This may also work when you are trying to work with different operating system, but it is not garanteed.)
conda env export | cut -f -2 -d “=” | grep -v “prefix” > environment.yml
Method 3: clone an environment — doesn’t really work
conda create --name myclone --clone myenv
You’ll soon realize that cloning only works within the same conda installation so this does not work for our purpose of relocating conda environment into another location. So… use method 1 or 2 instead.