Auto install all Python packages in src/edxapp-pip

README.rst

@@ -11,7 +11,7 @@ Requirements
 - This project requires **Docker 17.06+ CE**.
 - You are required to have the needed GCloud permissions. To authenticate your requests, follow the steps in: `Authentication methods <https://cloud.google.com/container-registry/docs/advanced-authentication>`_
 
-Desclaimer
+Disclaimer
 ----------
 - Apparently Docker behavior is inconsistent between Mac and Linux due to some core differences between both operating systems.
 - This README contains some known bugs and issues, make sure to search your issues here as they might be an issue we ran into before.
@@ -89,8 +89,13 @@ Currently the commands looks like this:
 
 If something goes wrong, check out the rest of this README for additional details.
 
-Environment Files
------------------
+Tahoe Specific Devstack Features
+--------------------------------
+The features below are specific to this Tahoe devstack repository and not found in
+the upstream devstck:
+
+Persistent Environment Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The environment files are now stored in the ``src/edxapp-envs`` directory near the ``edx-platform``
 so it can be edited using layman editors such as PyCharm and VSCode.
 
@@ -100,6 +105,28 @@ devstack.
 
 ``$ dev.up`` command brings those files outside the container.
 
+Persistent Custom Python Packages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Because of the statelessness of Docker containers, the devstack will forget any
+``pip install`` you'd do after restart.
+
+Knowing that most of the packages like Figures and Course Access Groups
+need to survive such restarts, this feature only works on LMS/Studio.
+
+The ``$ make tahoe.provision`` command will install all packages located inside
+the ``src/edxapp-pip`` directory. There's no need to run this command manually as it will
+run on every ``$ make dev.up``.
+
+For example to make Course Access Groups installed on every devstack startup:
+
+.. code:: bash
+
+    $ cd devstack/../src/edxapp-pip/
+    $ git clone git@github.com:appsembler/course-access-groups.git
+    $ cd ../../devstack
+    $ make dev.up  # Now the repository will always be installed on every startup
+
 Theme
 -----
 Tahoe themes are copied and available in ``edx-codebase-theme`` directory near the ``edx-platform``. Refer to `edx-theme-codebase#How to use on Devstack <https://github.com/appsembler/edx-theme-codebase#how-to-use-on-devstack>`_ for more info on setting up your custom theme.

provision-tahoe.py

@@ -7,6 +7,7 @@
 from shutil import move
 from path import Path
 from os import symlink
+from subprocess import call
 
 
 ENVIRONMENT_FILES = [
@@ -18,13 +19,17 @@
 
 SRC_DIR = Path('/edx/src/')
 ENVS_DIR = SRC_DIR / 'edxapp-envs'
+PIP_DIR = SRC_DIR / 'edxapp-pip'
 EDXAPP_DIR = Path('/edx/app/edxapp')
 
 
 def move_environment_files_to_host():
     """
     Move the json environment files to the host so they're editable.
     """
+    if not ENVS_DIR.exists():
+        makedirs(ENVS_DIR)
+
     for filename in ENVIRONMENT_FILES:
         container_path = EDXAPP_DIR / filename
         src_path = ENVS_DIR / filename  # The mounted directory in
@@ -45,11 +50,24 @@ def move_environment_files_to_host():
             symlink(src_path, container_path)
 
 
-def main():
-    if not ENVS_DIR.exists():
-        makedirs(ENVS_DIR)
+def install_auto_pip_requirements():
+    """
+    Install source pip packages (git repositories) that are checked out at `src/edxapp-pip`.
+
+    This useful to avoid the need to re-install pip requirements every time a `$ make dev.up` is done.
+    """
+    if not PIP_DIR.exists():
+        return
 
+    for package_dir in PIP_DIR.dirs():
+        setup_file = package_dir / 'setup.py'
+        if setup_file.exists():  # Ensure it's a proper Python package.
+            call(['pip', 'install', '-e', package_dir])
+
+
+def main():
     move_environment_files_to_host()
+    install_auto_pip_requirements()
 
 
 main()