I am able to get a conda environment setup and running from within Google Colab. However, this environment seems to be transient in nature. It does not show up in any new notebook I create, and I am not able to run the notebook normally. I have to run all my python code in one cell... But at least I know using conda to create my own environment in Google Colab is possible.
Check to see if conda is already installed
!which conda
Nope, not installed. Go ahead and install it
I found a package cartopy that is not installed in my normal Google Colab environment. I want to use it so I will include it in my conda env create command below.
import cartopy
# try to run the bare minimum to get a new conda env working
conda_path = ''
try:
conda_path = !which conda
finally:
print('')
if (len(conda_path) == 0):
print('installing miniconda')
!wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh && bash Miniconda3-4.5.4-Linux-x86_64.sh -bfp /usr/local
!conda update conda -y -q
!source /usr/local/etc/profile.d/conda.sh
!conda init
!conda install -n root _license -y -q
else:
print('found miniconda')
conda_envs = !conda env list
res = [i for i in conda_envs if 'test36' in i]
if (len(res) == 0):
print('not found test36 env', len(res))
!conda create -y -q --name test36 python=3.6 libarchive cartopy
else:
print('found test36 env', len(res))
You should see a lot of install activity above, similar to this:installing miniconda --2020-06-27 22:45:59-- https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh Resolving repo.continuum.io (repo.continuum.io)... 104.18.201.79, 104.18.200.79, 2606:4700::6812:c94f, ... Connecting to repo.continuum.io (repo.continuum.io)|104.18.201.79|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh [following] --2020-06-27 22:45:59-- https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.131.3, 104.16.130.3, 2606:4700::6810:8203, ... Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.131.3|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 58468498 (56M) [application/x-sh] Saving to: ‘Miniconda3-4.5.4-Linux-x86_64.sh.1’
Miniconda3-4.5.4-Li 100%[===================>] 55.76M 180MB/s in 0.3s
2020-06-27 22:45:59 (180 MB/s) - ‘Miniconda3-4.5.4-Linux-x86_64.sh.1’ saved [58468498/58468498]
PREFIX=/usr/local installing: python-3.6.5-hc3d631a_2 ... Python 3.6.5 :: Anaconda, Inc. installing: ca-certificates-2018.03.07-0 ... ... unlinking: zlib-1.2.11-h7b6447c_3 installation finished. WARNING: You currently have a PYTHONPATH environment variable set. This may cause unexpected behavior when running the Python interpreter in Miniconda3. For best results, please verify that your PYTHONPATH only points to directories of packages that are compatible with the Python interpreter in Miniconda3: /usr/local Solving environment: ...working... done
Package Plan¶
environment location: /usr/local
added / updated specs:
- condaThe following packages will be downloaded:
package | build ---------------------------|----------------- openssl-1.1.1g | h7b6447c_0 3.8 MB...
conda-package-handling-1.6.1| py36h7b6447c_0 886 KB ------------------------------------------------------------ Total: 48.8 MBThe following NEW packages will be INSTALLED:
conda-package-handling: 1.6.1-py36h7b6447c_0 tqdm: 4.46.1-py_0The following packages will be UPDATED:
ca-certificates: 2018.03.07-0 --> 2020.1.1-0... tk: 8.6.7-hc745277_3 --> 8.6.10-hbc83047_0
Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done no change /usr/local/condabin/conda ... no change /usr/local/etc/profile.d/conda.csh modified /root/.bashrc
==> For changes to take effect, close and re-open your current shell. <==
Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
Current channels:
To search for alternate channels that may provide the conda package you're looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
not found test36 env 0 Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... done
environment location: /usr/local/envs/test36
added / updated specs:
- python=3.6
The following packages will be downloaded:
package | build
---------------------------|-----------------
_libgcc_mutex-0.1 | main 3 KB
...
zlib-1.2.11 | h7b6447c_3 103 KB
------------------------------------------------------------
Total: 49.4 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main ... zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done
check if conda exists again, now you should see something like "/usr/loca/bin/conda"
!which conda
Check on your environment
!env
If everything looks like I am showing, with no errors, you can run python code in your new conda environment. However, this will only persist for one cell. Then next cell will revert back to your base Google Colab environment. This is because you are using %%bash, this command will open a shell and run the commands within the same cell, but will close down when you exit the cell. You can test if the correct environment is active using conda env list and conda list. I can see my new environment is active (*) and the new package cartopy is available now.
%%bash
source activate test36
conda env list
conda list
python
import sys
import cartopy
#import libarchive
# maybe only need this the first time we run this notebook
sys.path.append('/usr/local/lib/python3.6/site-packages')
print("Python version")
print(sys.version)
The next cell will revert back to the default shell that was opened when you started Google Colab
import sys
print(sys.version)
Not sure how to get around this, and run multiple cells from within your own environment only invoking python once...
Also note that conda activate does not work. Only source activate works like this so far. Still looking at this option, but the hints in this error do not help.
%%bash
conda activate test36
python
import sys
# maybe only need this the first time we run this notebook
sys.path.append('/usr/local/lib/python3.6/site-packages')
print("Python version")
print(sys.version)
Additional info about the conda you just installed
!conda info
Location of your environment inside this container
!ls -rlt /usr/local/envs