Virtual Folders

Virtual folders provide access to shared, persistent, and reused files across different sessions.

You can mount virtual folders when creating new sessions, and use them like a plain directory on the local filesystem. Of course, reads/writes to virtual folder contents may have degraded performance compared to the main scratch directory (usually /home/work in most kernels) as internally it uses a networked file system.

Also, you might share your virtual folders with other users by inviting them and granting them proper permission. Currently, there are three levels of permissions: read-only, read-write, read-write-delete. They are represented by short strings, 'ro', 'rw', 'wd', respectively. The owner of a virtual folder have read-write-delete permission for the folder.

Listing Virtual Folders

Returns the list of virtual folders created by the current keypair.

URI: /folders
Method: GET

Parameters

Parameter	Type	Description
`all`	`bool`	(optional) If this parameter is `True`, it returns all virtual folders, including those that do not belong to the current user. Only available for superadmin (default: `False`).
`group_id`	`UUID \| str`	(optional) If this parameter is set, it returns the virtual folders that belong to the specified group. Have no effect in user-type virtual folders.

Response

HTTP Status Code	Description
200 OK	Success

Fields	Type	Values
(root)	`list[object]`	A list of Virtual Folder List Item Object

Example:

[
   {
      "name": "myfolder",
      "id": "b4b1b16c-d07f-4f1f-b60e-da9449aa60a6",
      "host": "local:volume1",
      "usage_mode": "general",
      "created_at": "2020-11-28 13:30:30.912056+00",
      "is_owner": "true",
      "permission": "rw",
      "user": "dfa9da54-4b28-432f-be29-c0d680c7a412",
      "group": null,
      "creator": "admin@lablup.com",
      "user_email": "admin@lablup.com",
      "group_name": null,
      "ownership_type": "user",
      "unmanaged_path": null,
      "cloneable": "false",
   }
]

Listing Virtual Folder Hosts

Returns the list of available host names where the current keypair can create new virtual folders.

Added in version v4.20190315.

URI: /folders/_/hosts
Method: GET

Parameters

None

Response

HTTP Status Code	Description
200 OK	Success

Fields	Type	Values
`default`	`str`	The default virtual folder host
`allowed`	`list[str]`	The list of available virtual folder hosts

Example:

{
  "default": "seoul:nfs1",
  "allowed": ["seoul:nfs1", "seoul:nfs2", "seoul:cephfs1"]
}

Creating a Virtual Folder

URI: /folders
Method: POST

Creates a virtual folder associated with the current API key.

Parameters

Parameter	Type	Description
`name`	`str`	The human-readable name of the virtual folder
`host`	`str`	(optional) The name of the virtual folder host
`usage_mode`	`str`	(optional) The purpose of the virtual folder. Allowed values are `general`, `model`, and `data` (default: `general`).
`permission`	`str`	(optional) The default share permission of the virtual folder. The owner of the virtual folder always have `wd` permission regardless of this parameter. Allowed values are `ro`, `rw`, and `wd` (default: `rw`).
`group_id`	`UUID \| str`	(optional) If this parameter is set, it creates a group-type virtual folder. If empty, it creates a user-type virtual folder.
`quota`	`int`	(optional) Set the quota of the virtual folder in bytes. Note, however, that the quota is only supported under the xfs filesystems. Other filesystems that do not support per-directory quota will ignore this parameter.

Example:

{
  "name": "My Data",
  "host": "seoul:nfs1"
}

Response

HTTP Status Code	Description
201 Created	The kernel is successfully created.
400 Bad Request	The name is malformed or duplicate with your existing virtual folders.
406 Not acceptable	You have exceeded internal limits of virtual folders. (e.g., the maximum number of folders you can have.)

Fields	Type	Values
`id`	`slug`	The unique folder ID used for later API calls
`name`	`str`	The human-readable name of the created virtual folder
`host`	`str`	The name of the virtual folder host where the new folder is created

Example:

{
  "id": "aef1691db3354020986d6498340df13c",
  "name": "My Data",
  "host": "nfs1",
  "usage_mode": "general",
  "permission": "rw",
  "creator": "admin@lablup.com",
  "ownership_type": "user",
  "user": "dfa9da54-4b28-432f-be29-c0d680c7a412",
  "group": "",
}

Getting Virtual Folder Information

URI: /folders/:name
Method: GET

Retrieves information about a virtual folder. For performance reasons, the returned information may not be real-time; usually they are updated every a few seconds in the server-side.

Parameters

Parameter	Type	Description
`name`	`str`	The human-readable name of the virtual folder

Response

HTTP Status Code	Description
200 OK	The information is successfully returned.
404 Not Found	There is no such folder or you may not have proper permission to access the folder.

Fields	Type	Values
(root)	`object`	Virtual Folder Item Object

Deleting Virtual Folder

URI: /folders/:name
Method: DELETE

This immediately deletes all contents of the given virtual folder and makes the folder unavailable for future mounts.

Danger

If there are running kernels that have mounted the deleted virtual folder, those kernels are likely to break!

Warning

There is NO way to get back the contents once this API is invoked.

Parameters

Parameter	Description
`name`	The human-readable name of the virtual folder

Response

HTTP Status Code	Description
204 No Content	The folder is successfully destroyed.
404 Not Found	There is no such folder or you may not have proper permission to delete the folder.

Rename a Virtual Folder

URI: /folders/:name/rename
Method: POST

Rename a virtual folder associated with the current API key.

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`new_name`	`str`	New virtual folder name

Response

HTTP Status Code	Description
201 Created	The folder is successfully renamed.
404 Not Found	There is no such folder or you may not have proper permission to rename the folder.

Listing Files in Virtual Folder

Returns the list of files in a virtual folder associated with current keypair.

URI: /folders/:name/files
Method: GET

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`path`	`str`	Path inside the virtual folder (default: root)

Response

HTTP Status Code	Description
200 OK	Success.
404 Not Found	There is no such path or you may not have proper permission to access the folder.

Fields	Type	Values
`files`	`list[object]`	List of Virtual Folder File Object

Uploading a File to Virtual Folder

Upload a local file to a virtual folder associated with the current keypair. Internally, the Manager will deligate the upload to a Backend.AI Storage-Proxy service. JSON web token is used for the authentication of the request.

URI: /folders/:name/request-upload
Method: POST

Warning

If a file with the same name already exists in the virtual folder, it will be overwritten without warning.

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`path`	`str`	Path of the local file to upload
`size`	`int`	The total size of the local file to upload

Response

HTTP Status Code	Description
200 OK	Success.

Fields	Type	Values
`token`	`str`	JSON web token for the authentication of the upload session to Storage-Proxy service.
`url`	`str`	Request url for a Storage-Proxy. Client should use this URL to upload the file.

Creating New Directory in Virtual Folder

Create a new directory in the virtual folder associated with current keypair. this API recursively creates parent directories if they does not exist.

URI: /folders/:name/mkdir
Method: POST

Warning

If a directory with the same name already exists in the virtual folder, it may be overwritten without warning.

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder.
`path`	`str`	The relative path of a new folder to create inside the virtual folder
`parents`	`bool`	If `True`, the parent directories will be created if they do not exist.
`exist_ok`	`bool`	If a directory with the same name already exists, overwrite it without an error.

Response

HTTP Status Code	Description
201 Created	Success.
400 Bad Request	There already exists a file, not a directory, with duplicated name.
404 Not Found	There is no such folder or you may not have proper permission to write into folder.

Downloading a File or a Directory from a Virtual Folder

Download a file or a directory from a virtual folder associated with the current keypair. Internally, the Manager will deligate the download to a Backend.AI Storage-Proxy service. JSON web token is used for the authentication of the request.

Added in version v4.20190315.

URI: /folders/:name/request-download
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`path`	`str`	The path to a file or a directory inside the virtual folder to download.
`archive`	`bool`	If this parameter is `True` and `path` is a directory, the directory will be archived into a zip file on the fly (default: `False`).

Response

HTTP Status Code	Description
200 OK	Success.
404 Not Found	File not found or you may not have proper permission to access the folder.

Fields	Type	Values
`token`	`str`	JSON web token for the authentication of the download session to Storage-Proxy service.
`url`	`str`	Request url for a Storage-Proxy. Client should use this URL to download the file.

Deleting Files in Virtual Folder

This deletes files inside a virtual folder.

Warning

There is NO way to get back the files once this API is invoked.

URI: /folders/:name/delete-files
Method: DELETE

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`files`	`list[str]`	File paths inside the virtual folder to delete
`recursive`	`bool`	Recursive option to delete folders if set to True. The default is False.

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	You tried to delete a folder without setting recursive option as True.
404 Not Found	There is no such folder or you may not have proper permission to delete the file in the folder.

Rename a File in Virtual Folder

Rename a file inside a virtual folder.

URI: /folders/:name/rename-file
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`target_path`	`str`	The relative path of target file or directory
`new_name`	`str`	The new name of the file or directory
`is_dir`	`bool`	Flag that indicates the `target_path` is a directory or not

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	You tried to rename a directory without setting is_dir option as True.
404 Not Found	There is no such folder or you may not have proper permission to rename the file in the folder.

Listing Invitations for Virtual Folder

Returns the list of pending invitations that the requested user received. This will display the invitations sent to me by other users.

URI: /folders/invitations/list
Method: GET

Parameters

This API does not need any parameter.

Response

HTTP Status Code	Description
200 OK	Success.

Fields	Type	Values
`invitations`	`list[object]`	A list of Virtual Folder Invitation Object

Creating an Invitation

Invite other users to share a virtual folder with proper permissions. If a user is already invited, then this API does not create a new invitation or update the permission of the existing invitation.

URI: /folders/:name/invite
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`perm`	`str`	The permission to grant to invitee
`emails`	`list[slug]`	A list of user emails to invite

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	No invitee is given.
404 Not Found	There is no invitation.

Fields	Type	Values
`invited_ids`	`list[slug]`	A list of invited user emails

Accepting an Invitation

Accept an invitation and receive permission to a virtual folder as in the invitation.

URI: /folders/invitations/accept
Method: POST

Parameters

Parameter	Type	Description
`inv_id`	`slug`	The unique invitation ID

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	The name of the target virtual folder is duplicate with your existing virtual folders.
404 Not Found	There is no such invitation.

Rejecting an Invitation

Reject an invitation.

URI: /folders/invitations/delete
Method: DELETE

Parameters

Parameter	Type	Description
`inv_id`	`slug`	The unique invitation ID

Response

HTTP Status Code	Description
200 OK	Success.
404 Not Found	There is no such invitation.

Fields	Type	Values
`msg`	`str`	Detail message for the invitation deletion

Listing Sent Invitations

Returns the list of virtual folder invitations the requested user sent. This does not include the invitations those are already accepted or rejected.

URI: /folders/invitations/list-sent
Method: GET

Parameters

This API does not need any parameter.

Response

HTTP Status Code	Description
200 OK	Success.

Fields	Type	Values
`invitations`	`list[object]`	A list of Virtual Folder Invitation Object

Updating an Invitation

Update the permission of an already-sent, but not accepted or rejected, invitation.

URI: /folders/invitations/update/:inv_id
Method: POST

Parameters

Parameter	Type	Description
`:inv_id`	`str`	The unique invitation ID
`perm`	`str`	The permission to grant to invitee

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	No permission is given.
404 Not Found	There is no invitation.

Fields	Type	Values
`msg`	`str`	An update message string

Leave an Shared Virtual Folder

Leave a shared virtual folder.

Cannot leave a group vfolder or a vfolder that the requesting user owns.

URI: /folders/:name/leave
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder

Response

HTTP Status Code	Description
200 OK	Success.
404 Not Found	There is no virtual folder.

Fields	Type	Values
`msg`	`str`	A result message string

Listing Users Share Virtual Folders

Returns the list of users who shares requester’s virtual folders.

URI: /folders/_/shared
Method: GET

Parameters

Parameter	Type	Description
`vfolder_id`	`str`	(Optional) The unique virtual folder ID to list shared users. If not specified, all users who shares any virtual folders the requester created.

Response

HTTP Status Code	Description
200 OK	Success.

Fields	Type	Values
`shared`	`list[object]`	A list of information about shared users.

Example:

[
   {
      "vfolder_id": "aef1691db3354020986d6498340df13c",
      "vfolder_name": "My Data",
      "shared_by": "admin@lablup.com",
      "shared-to": {
         "uuid": "dfa9da54-4b28-432f-be29-c0d680c7a412",
         "email": "user@lablup.com"
      },
      "perm": "ro"
   }
]

Updating the permission of a shared virtual folder

Update the permission of a user for a shared virtual folder.

URI: /folders/_/shared
Method: POST

Parameters

Parameter	Type	Description
`vfolder`	`UUID`	The unique virtual folder ID
`user`	`UUID`	The unique user ID
`perm`	`str`	The permission to update for the `user` on `vfolder`

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	No permission or user is given.
404 Not Found	There is no virtual folder.

Fields	Type	Values
`msg`	`str`	An update message string

Share a Group Virtual Folder to an Individual Users

Share a group virtual folder to users with overriding permission.

This will create vfolder_permission(s) relation directly without creating invitation(s). Only group virtual folders are allowed to be shared directly.

This API can be useful when you want to share a group virtual folder to every group members with read-only permission, but allows some users read-write permission.

NOTE: This API is only available for group virtual folders.

URI: /folders/:name/share
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`permission`	`str`	Overriding permission to share the group virtual folder
`emails`	`list[str]`	A list of user emails to share

Response

HTTP Status Code	Description
201 Created	Success.
400 Bad Request	No permission or email is given.
404 Not Found	There is no virtual folder.

Fields	Type	Values
`shared_emails`	`list[str]`	A list of user emails those are successfully shared the virtual folder

Unshare a Group Virtual Folder from Users

Unshare a group virtual folder from users

NOTE: This API is only available for group virtual folders.

URI: /folders/:name/unshare
Method: DELETE

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`emails`	`list[str]`	A list of user emails to unshare

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	No email is given.
404 Not Found	There is no virtual folder.

Fields	Type	Values
`unshared_emails`	`list[str]`	A list of user emails those are successfully unshared the virtual folder

Clone a Virtual Folder

Clone a virtual folder

URI: /folders/:name/clone
Method: POST

Parameters

Parameter	Type	Description
`:name`	`str`	The human-readable name of the virtual folder
`cloneable`	`bool`	If `True`, cloned virtual folder will be cloneable again.
`target_name`	`str`	The name of the new virtual folder
`target_host`	`str`	The targe host volume of the new virtual folder
`usage_mode`	`str`	(optional) The purpose of the new virtual folder. Allowed values are `general`, `model`, and `data` (default: `general`).
`permission`	`str`	(optional) The default share permission of the new virtual folder. The owner of the virtual folder always have `wd` permission regardless of this parameter. Allowed values are `ro`, `rw`, and `wd` (default: `rw`).

Response

HTTP Status Code	Description
200 OK	Success.
400 Bad Request	No target name, target host, or no permission.
403 Forbidden	The source virtual folder is not permitted to be cloned.
404 Not Found	There is no virtual folder.

Fields	Type	Values
`unshared_emails`	`list[str]`	A list of user emails those are successfully unshared the virtual folder.

Fields	Type	Values
(root)	`list[object]`	Virtual Folder List Item Object

Example:

{
   "name": "my cloned folder",
   "id": "b4b1b16c-d07f-4f1f-b60e-da9449aa60a6",
   "host": "local:volume1",
   "usage_mode": "general",
   "created_at": "2020-11-28 13:30:30.912056+00",
   "is_owner": "true",
   "permission": "rw",
   "user": "dfa9da54-4b28-432f-be29-c0d680c7a412",
   "group": null,
   "creator": "admin@lablup.com",
   "user_email": "admin@lablup.com",
   "group_name": null,
   "ownership_type": "user",
   "unmanaged_path": null,
   "cloneable": "false"
}