Development Setup
Currently Backend.AI is developed and tested under only *NIX-compatible platforms (Linux or macOS).
The development setup uses a mono-repository for the backend stack and a side-by-side repository checkout of the frontend stack. In contrast, the production setup uses per-service independent virtual environments and relies on a separately provisioned app proxy pool.
There are three ways to run both the backend and frontend stacks for development, as demonstrated in
Fig. 4, Fig. 5, and Fig. 6.
The installation guide in this page using scripts/install-dev.sh
covers all three cases because the only difference
is that how you launch the Web UI from the mono-repo.
Fig. 4 A standard development setup of Backend.AI open source components
Fig. 5 A development setup of Backend.AI open source components for Electron-based desktop app
Fig. 6 A development setup of Backend.AI open source components with pre-built web UI from the backend.ai-app
repository
Installation from Source
For the ease of on-boarding developer experience, we provide an automated script that installs all server-side components in editable states with just one command.
Prerequisites
Install the followings accordingly to your host operating system.
-
Ensure that you have all of the Python versions specified in
pants.toml
withpyenv
. (both Python 3.9.x and Python 3.10.8 at the time of writing, but please consult your copy ofpants.toml
for the latest information)Check the prerequisites for Python build environment setup for your system.
Docker Compose (v2 required)
(For Linux aarch64/arm64 setups only) Rust to build Pants from its source
Warning
To avoid conflicts with your system Python such as macOS/XCode versions,
our default pants.toml
is configured to search only pyenv
-provided Python versions.
Note
In some cases, locale conflicts between the terminal client and the remote host may cause encoding errors when installing Backend.AI components due to Unicode characters in README files. Please keep correct locale configurations to prevent such errors.
Running the install-dev script
$ git clone https://github.com/lablup/backend.ai bai-dev
$ cd bai-dev
$ ./scripts/install-dev.sh
Note
The script requires sudo
to check and install several system packages
such as build-essential
.
This script will bootstrap Pants and creates the halfstack
containers using docker compose
with fixture population.
At the end of execution, the script will show several command examples about
launching the service daemons such as manager and agent.
You may execute this script multiple times when you encounter prerequisite errors and
resolve them.
Also check out additional options using -h
/ --help
option, such as installing
the CUDA mockup plugin together, etc.
Changed in version 22.09: We have migrated to per-package repositories to a semi-mono repository that contains all Python-based components except plugins. This has changed the installation instruction completely with introduction of Pants.
Note
To install multiple instances/versions of development environments using this script,
just clone the repository in another location and run scripts/install-dev.sh
inside that directory.
It is important to name these working-copy directories differently not to confuse
docker compose
so that it can distinguish the containers for each setup.
Unless you customize all port numbers by the options of scripts/install-dev.sh
,
you should docker compose -f docker-compose.halfstack.current.yml down
and docker compose -f docker-compose.halfstack.current.yml up -d
when switching
between multiple working copies.
Note
By default, the script pulls the docker images for our standard Python kernel and TensorFlow CPU-only kernel. To try out other images, you have to pull them manually afterwards.
Note
Currently there are many limitations on running deep learning images on ARM64 platforms, because users need to rebuild the whole computation library stack, although more supported images will come in the future.
Note
To install the webui in an editable state, try --editable-webui
flag option when running scripts/install-dev.sh
.
Tip
Using the agent’s cgroup-based statistics without the root privilege (Linux-only)
To allow Backend.AI to collect sysfs/cgroup resource usage statistics, the Python executable must have the following Linux capabilities: CAP_SYS_ADMIN
, CAP_SYS_PTRACE
, and CAP_DAC_OVERRIDE
.
$ sudo setcap \
> cap_sys_ptrace,cap_sys_admin,cap_dac_override+eip \
> $(readlink -f $(pyenv which python))
Verifying Installation
Refer the instructions displayed after running scripts/install-dev.sh
.
We recommend to use tmux to open
multiple terminals in a single SSH session.
Your terminal app may provide a tab interface, but when using remote servers,
tmux is more convenient because you don’t have to setup a new SSH connection
whenever adding a new terminal.
Ensure the halfstack containers are running:
$ docker compose -f docker-compose.halfstack.current.yml up -d
Open a terminal for manager and run:
$ ./backend.ai mgr start-server --debug
Open another terminal for agent and run:
$ ./backend.ai ag start-server --debug
Open yet another terminal for client and run:
$ source ./env-local-admin-api.sh # Use the generated local endpoint and credential config.
$ # source ./env-local-user-api.sh # Yo may choose an alternative credential config.
$ ./backend.ai config
$ ./backend.ai run python --rm -c 'print("hello world")'
∙ Session token prefix: fb05c73953
✔ [0] Session fb05c73953 is ready.
hello world
✔ [0] Execution finished. (exit code = 0)
✔ [0] Cleaned up the session.
$ ./backend.ai ps
Resetting the environment
Shutdown all docker containers using docker compose -f docker-compose.halfstack.current.yml down
and delete the entire working copy directory. That’s all.
You may need sudo
to remove the directories mounted as halfstack container volumes
because Docker auto-creates them with the root privilege.
Daily Workflows
Check out Daily Development Workflows for your reference.