{
"cells": [
{
"cell_type": "markdown",
"id": "6d924ae8-e462-4c47-86d9-8560ecedf78f",
"metadata": {},
"source": [
"# Coder management"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb33cb2d-8256-4d65-9442-153a5eb920a6",
"metadata": {},
"outputs": [],
"source": [
"import csv\n",
"import json"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "991fc1f2-61d2-443f-901b-df6acf094287",
"metadata": {},
"outputs": [],
"source": [
"cluster_name='inf110'\n",
"proxyJump='ubuntu@137.194.210.143'"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ae9a0955-2a17-499a-827b-8dfe8be56d65",
"metadata": {},
"outputs": [],
"source": [
"token,=!source openrc && echo $CODER_TOKEN"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "904c792f-2229-44cb-a98b-7714af05e163",
"metadata": {},
"outputs": [],
"source": [
"!https GET \"tp-inf110.r2.enst.fr/api/v2/buildinfo\" | jq '{ coder_version: .version, coder_url: .dashboard_url }'"
]
},
{
"cell_type": "markdown",
"id": "463b7432-7861-44f6-8a42-36f453bf0a24",
"metadata": {},
"source": [
"To retrieve the ID of the default organization, we can list the organizations of the admin user."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3d2d2fe7-c20b-4e7d-a5da-e0c8021cb6f8",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"user=\"Zimmi48\"\n",
"organization_id,=!https GET \"tp-inf110.r2.enst.fr/api/v2/users/{user}/organizations\" \"Coder-Session-Token:{token}\" | jq -r .[0].id"
]
},
{
"cell_type": "markdown",
"id": "e953643e-a910-4b1a-bead-70caf4e2ebdd",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## Create organization\n",
"\n",
"An organization is always created by default, so there is no point in creating another one for now."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eaf47ee5-f83c-4ebd-8828-a5c4c89db924",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"#name=\"INF110\"\n",
"#!https POST \"tp-inf110.r2.enst.fr/api/v2/organizations\" \"Coder-Session-Token:{token}\" \"name={name}\""
]
},
{
"cell_type": "markdown",
"id": "5fe6ded1-6fa1-48b4-b4bd-fc8eee714816",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## Delete organization"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f5dbf307-3352-498a-a175-93ad6ccd1155",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"#name=\"INF110\"\n",
"#!https DELETE \"tp-inf110.r2.enst.fr/api/v2/organizations/{name}\" \"Coder-Session-Token:{token}\""
]
},
{
"cell_type": "markdown",
"id": "19a552b0-8bf8-4cd8-86a1-135b7e2253ed",
"metadata": {},
"source": [
"## Create workspace template\n",
"\n",
"If it doesn't exist yet. Otherwise, retrieve the template and template version IDs."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b4db687e-3e52-448d-b4d7-8c812cdaa2f7",
"metadata": {},
"outputs": [],
"source": [
"template_id,template_version=!https GET \"tp-inf110.r2.enst.fr/api/v2/organizations/{organization_id}/templates\" \"Coder-Session-Token:{token}\" | jq -r '.[0] | .id, .active_version_id'\n",
"if template_id == 'null':\n",
" # Upload template files\n",
" file_hash=!cd inf110-workspace && zip \"template.zip\" \"main.tf\" \"README.md\" && https POST \"tp-inf110.r2.enst.fr/api/v2/files\" \"Coder-Session-Token:{token}\" \"Content-Type:application/zip\" < \"template.zip\" | jq -r '.hash'\n",
" # Create template version\n",
" version_name='INF110-2024'\n",
" template_version=!https POST \"tp-inf110.r2.enst.fr/api/v2/organizations/{organization_id}/templateversions\" \"Coder-Session-Token:{token}\" \"name={version_name}\" \"file_id={file_hash}\" \"storage_method=file\" \"provisioner=terraform\" | jq -r '.id'\n",
" # Create template\n",
" template_name='INF110'\n",
" display_name='TP INF110'\n",
" description=\"Espace de travail pour les TP d'INF110\"\n",
" template_id=!https POST \"tp-inf110.r2.enst.fr/api/v2/organizations/{organization_id}/templates\" \"Coder-Session-Token:{token}\" \"name={template_name}\" \"display_name={display_name}\" \"description={description}\" \"icon=/emojis/1f4d0.png\" \"template_version_id={template_version}\" | jq -r '.id'"
]
},
{
"cell_type": "markdown",
"id": "e20dfefb-0359-421a-92c6-ab403de529dc",
"metadata": {},
"source": [
"The query below can be used to check that the template creation was successful."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ae74b43-8065-4e5f-9976-e00a879d7c99",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"!https GET \"tp-inf110.r2.enst.fr/api/v2/templateversions/{template_version}\" \"Coder-Session-Token:{token}\" | jq '.job.status'"
]
},
{
"cell_type": "markdown",
"id": "5d900272-a3ee-461c-9cdd-df08b17b1d2a",
"metadata": {},
"source": [
"## Create users"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9f05d75f-c300-49ca-9faf-19a1a24dd6d8",
"metadata": {},
"outputs": [],
"source": [
"def create_user(email,username):\n",
" passwords=!pwgen -s 10 -1\n",
" password=passwords[0]\n",
" !echo \"{email},{username},{password}\" >> users.csv && \\\n",
" https POST \"tp-inf110.r2.enst.fr/api/v2/users\" \"Coder-Session-Token:{token}\" \"email={email}\" \"login_type=password\" \"password={password}\" \"username={username}\"\n",
"\n",
"#create_user(\"jean.leneutre@telecom-paris.fr\", \"jleneutre\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "96595327-81a0-4a50-b090-a79d75a3134d",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def create_student_accounts(csv_filename):\n",
" with open(csv_filename, mode='r') as csv_file:\n",
" csv_reader = csv.DictReader(csv_file)\n",
" for student in csv_reader:\n",
" create_user(student['mail'], student['login'])\n",
"\n",
"#create_student_accounts('inf110-2023-B.csv')\n",
"#create_student_accounts('inf110-2023-D.csv')\n",
"#create_student_accounts('MITRO202.csv')"
]
},
{
"cell_type": "markdown",
"id": "9868d906-ca8c-4fda-ad96-ec00bea5fadc",
"metadata": {},
"source": [
"## Delete users\n",
"\n",
"(Requires having deleted their workspaces first.)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "30a56dca-7f8f-4839-94e7-36340f5f199f",
"metadata": {},
"outputs": [],
"source": [
"def delete_user(username):\n",
" !https DELETE \"tp-inf110.r2.enst.fr/api/v2/users/{username}\" \"Coder-Session-Token:{token}\"\n",
"\n",
"#delete_user(\"bbinder\")"
]
},
{
"cell_type": "markdown",
"id": "d6d20c1f-e231-488c-90ef-2daa811e89a2",
"metadata": {},
"source": [
"## Create workspaces"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "80ff8f4e-fa5f-4308-a885-e3d382e2a171",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def create_workspace(username, name='tp'):\n",
" !https POST \"tp-inf110.r2.enst.fr/api/v2/organizations/{organization_id}/members/{username}/workspaces\" \"Coder-Session-Token:{token}\" \"name={name}\" \"template_id={template_id}\"\n",
"\n",
"#create_workspace(\"Zimmi48\")"
]
},
{
"cell_type": "markdown",
"id": "d2cea793-4b57-4718-bd17-7cbda610d5bd",
"metadata": {},
"source": [
"Test auto-scaling by creating many workspaces at once."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e3d06f0c-3183-47e4-a63e-254a6c08612a",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"#for i in range(0,99):\n",
"# create_workspace('Zimmi48', 'autoscaling-test-%d' % i)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "11d58691-691b-41a8-bdbf-51d3b313b64f",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def create_student_workspaces(csv_filename):\n",
" with open(csv_filename, mode='r') as csv_file:\n",
" csv_reader = csv.DictReader(csv_file)\n",
" for student in csv_reader:\n",
" create_workspace(student['login'])\n",
"\n",
"#create_student_workspaces('inf110-2023-B.csv')\n",
"#create_student_workspaces('inf110-2023-D.csv')"
]
},
{
"cell_type": "markdown",
"id": "1b405071-7dd5-4082-80f2-dbd60ea46262",
"metadata": {},
"source": [
"## Start / stop all workspaces from a student group"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9be7a12-c858-4baf-9ac7-53827330798a",
"metadata": {},
"outputs": [],
"source": [
"def get_student_username_list(csv_filename):\n",
" students = []\n",
" with open(csv_filename, mode='r') as csv_file:\n",
" csv_reader = csv.DictReader(csv_file)\n",
" for student in csv_reader:\n",
" students += [student['login']]\n",
" return students\n",
"\n",
"#groupB = get_student_username_list('inf110-2023-B.csv')\n",
"#groupD = get_student_username_list('inf110-2023-D.csv')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1640ad41-d82c-4519-88d3-ef2c2cfc2950",
"metadata": {},
"outputs": [],
"source": [
"len(groupB)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b0b126a-b7e4-43c5-9aab-2c27931ebf91",
"metadata": {},
"outputs": [],
"source": [
"len(groupD)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b495de0f-d085-4db6-a8a4-9a022f345251",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def stop_workspaces_from_group(group):\n",
" workspaces = !https GET \"tp-inf110.r2.enst.fr/api/v2/workspaces\" \"Coder-Session-Token:{token}\" \"limit==100\" \"q==status:running\" -b | jq -cr '.workspaces | .[]'\n",
" workspaces = list(map(json.loads, workspaces))\n",
" for workspace in [workspace['id'] for workspace in workspaces if workspace['owner_name'] in group]:\n",
" !https POST \"tp-inf110.r2.enst.fr/api/v2/workspaces/{workspace}/builds\" \"Coder-Session-Token:{token}\" \"transition=stop\"\n",
"\n",
"stop_workspaces_from_group(['Zimmi48'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13f05a6e-f591-4bcd-9857-63f5a21f1028",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def start_workspaces_from_group(group):\n",
" workspaces = !https GET \"tp-inf110.r2.enst.fr/api/v2/workspaces\" \"Coder-Session-Token:{token}\" \"limit==100\" \"q==status:stopped\" -b | jq -cr '.workspaces | .[]'\n",
" workspaces = list(map(json.loads, workspaces))\n",
" for workspace in [workspace['id'] for workspace in workspaces if workspace['owner_name'] in group]:\n",
" !https POST \"tp-inf110.r2.enst.fr/api/v2/workspaces/{workspace}/builds\" \"Coder-Session-Token:{token}\" \"transition=start\" \"template_version_id={template_version}\"\n",
"\n",
"start_workspaces_from_group(mitro)"
]
},
{
"cell_type": "markdown",
"id": "e8e88940-8c7c-4b3c-8923-aaf0b813f524",
"metadata": {},
"source": [
"## Delete all workspaces and users from a student group"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c22d1b3-923f-45fd-ac04-9f404e5002b6",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def delete_workspaces_from_group(group):\n",
" workspaces = !https GET \"tp-inf110.r2.enst.fr/api/v2/workspaces\" \"Coder-Session-Token:{token}\" \"limit==100\" \"q==status:stopped\" -b | jq -cr '.workspaces | .[]'\n",
" workspaces = list(map(json.loads, workspaces))\n",
" for workspace in [workspace['id'] for workspace in workspaces if workspace['owner_name'] in group]:\n",
" !https POST \"tp-inf110.r2.enst.fr/api/v2/workspaces/{workspace}/builds\" \"Coder-Session-Token:{token}\" \"transition=delete\"\n",
"\n",
"delete_workspaces_from_group(['Zimmi48'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eafaba21-fdf9-46e9-ba8f-bc1f25de96a4",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def delete_students(group):\n",
" for student in group:\n",
" delete_user(student)\n",
"\n",
"delete_students(groupD)"
]
},
{
"cell_type": "markdown",
"id": "12c57f21-23fe-4752-ad28-b05bd33ff08f",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## Stop all unhealthy workspaces"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73ce34f9-b0ad-4434-b448-6e2fdcfed6bd",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def stop_unhealthy_workspaces():\n",
" workspaces = !https GET \"tp-inf110.r2.enst.fr/api/v2/workspaces\" \"Coder-Session-Token:{token}\" \"limit==100\" \"q==status:running\" -b | jq -cr '.workspaces | .[]'\n",
" workspaces = list(map(json.loads, workspaces))\n",
" for workspace in [workspace for workspace in workspaces if not workspace['health']['healthy']]:\n",
" !https POST \"tp-inf110.r2.enst.fr/api/v2/workspaces/{workspace['id']}/builds\" \"Coder-Session-Token:{token}\" \"transition=stop\"\n",
"\n",
"stop_unhealthy_workspaces()"
]
},
{
"cell_type": "markdown",
"id": "acd9cb0e-9596-4091-ae35-82a34ff9f0b4",
"metadata": {},
"source": [
"## Check health of running workspaces"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6c4beb11-717a-4d89-b9a4-e9b033b3e6a4",
"metadata": {},
"outputs": [],
"source": [
"servers = {}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9bcdc633-0b28-46e4-aa54-97dc20f7d091",
"metadata": {},
"outputs": [],
"source": [
"healthy_workspaces = []\n",
"unhealthy_workspaces = []\n",
"\n",
"workspaces = !https GET \"tp-inf110.r2.enst.fr/api/v2/workspaces\" \"Coder-Session-Token:{token}\" \"limit==100\" \"q==status:running\" -b | jq -cr '.workspaces | .[]'\n",
"workspaces = list(map(json.loads, workspaces))\n",
"for workspace in workspaces:\n",
" if workspace['health']['healthy']:\n",
" healthy_workspaces += [workspace['owner_name'].lower()]\n",
" else:\n",
" unhealthy_workspaces += [workspace['owner_name'].lower()]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ffb92e49-aa8b-4371-bade-a06104020e32",
"metadata": {},
"outputs": [],
"source": [
"print(\"Healthy workspaces: %d\" % len(healthy_workspaces))\n",
"print(\"Unhealthy workspaces: %d\" % len(unhealthy_workspaces))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "08d53bc7-fd70-4a7f-97d3-be5e737cdbaa",
"metadata": {},
"outputs": [],
"source": [
"volumes=!source openrc && openstack volume list --long -f json | jq -c '.[] | { id: .Name, name: .Properties.\"csi.storage.k8s.io/pvc/name\", status: .Status, location: .Name, attached_to: .\"Attached to\" } | select(.name != null) | select(.name | startswith(\"coder-pvc-\") and contains(\"-tp-inf110\")) | { name: .id, username: .name | split(\"-\") | .[2], status: .status, server_id: .attached_to[0].server_id, location: (\"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/\" + .location + \"/globalmount\") }'\n",
"volumes = list(map(json.loads, volumes))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c183542f-6abf-4fbe-8b64-386da87a2c95",
"metadata": {},
"outputs": [],
"source": [
"[volume for volume in volumes if volume['server_id'] is not None and not volume['username'] in healthy_workspaces]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db003235-11d9-4386-b8fe-944955e5e257",
"metadata": {},
"outputs": [],
"source": [
"for server in servers:\n",
" servers[server]['users'] = []\n",
"for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" if not volume['server_id'] in servers:\n",
" ip,=!source openrc && openstack server show \"{volume['server_id']}\" -f json | jq -r '.addresses.\"{cluster_name}\"[0]'\n",
" servers[volume['server_id']] = { 'ip': ip, 'users': [volume['username']] }\n",
" else:\n",
" servers[volume['server_id']]['users'] += [volume['username']]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8cee7a31-4e22-4ff4-9b06-5c6b014ddea8",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"for server in servers:\n",
" print(\"Server: \" + servers[server]['ip'])\n",
" for user in servers[server]['users']:\n",
" print('User: ' + user, end='')\n",
" if user in healthy_workspaces:\n",
" print(' (healthy)')\n",
" elif user in unhealthy_workspaces:\n",
" print(' (unhealthy)')\n",
" else:\n",
" print(' (unknown)')\n",
" print()\n",
" print()"
]
},
{
"cell_type": "markdown",
"id": "79725809-695a-4ab4-9e58-79bb747b8b05",
"metadata": {},
"source": [
"## Clean workspaces"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1a29e39d-2abb-4b2a-b663-3455f956b46d",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def remove_lost_found():\n",
" for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" machine_ip = servers[volume['server_id']]['ip']\n",
" lost_found = volume['location'] + \"/lost+found\"\n",
" command = \"set -xe; \"\n",
" command += \"if [ -d \" + lost_found + \" ]; then rmdir \" + lost_found + \"; fi; \"\n",
" !ssh -oStrictHostKeyChecking=no -J core@{public_ip} core@{machine_ip} sudo sh -c \"\\\"{command}\\\"\"\n",
"\n",
"remove_lost_found()"
]
},
{
"cell_type": "markdown",
"id": "75661986-3bb2-42b0-abe6-1a5f18ea9615",
"metadata": {},
"source": [
"## Copy / update files in workspace"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13d7c970-4863-4fac-bb58-722bb9c27f34",
"metadata": {},
"outputs": [],
"source": [
"past_versions = {\n",
" 'tp1.ipynb': [\n",
" 'e6f11284e2ba7d4a01350a1f2c0ab208',\n",
" 'c208201a6aaf72f102fbd2cf46b62e5b',\n",
" ],\n",
" 'tp2.mv': [\n",
" 'b3b2a3e7a8dde6ed8848300158d89e5d',\n",
" '60a26b6a7d07579f63f6054c24b87ea2',\n",
" '190845731e26037dcbc7e3a74d239b1b',\n",
" 'd010ae9ab0188b4cc77a3d5a45df2f1a',\n",
" ],\n",
" 'tp3.mv': [\n",
" '91621d47a84e460e567dc51dfe1fcf45',\n",
" ],\n",
" 'tp4.mv': [\n",
" 'a819a3d31373750cfcb8473a27cd5c3f',\n",
" ],\n",
" 'tp5.mv': [],\n",
" 'settings.json': [\n",
" 'aca81e70023acea605dd44e829d04339', # Autogenerated by coq-lsp\n",
" '7378b13908523bfde3692518f61939cf',\n",
" '2c2178abd6661d10ae492bf6fdc0454b',\n",
" '350ac51d8fea466958d9986c96c84842',\n",
" 'ded5f984f385aa61ef30e01fef43f3e4',\n",
" ],\n",
" \"_CoqProject\": [],\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "876c1a46-9dd4-4f4c-8a4e-185dc361d5a1",
"metadata": {},
"outputs": [],
"source": [
"current_version = {\n",
" 'tp1.ipynb': '009c42561f1853af966f95e9dcee9c24',\n",
" 'tp2.mv': '78139922e6b6a531e7e70acc98c1019c',\n",
" 'tp3.mv': '0a0253e4825146da6f08eff073b10444',\n",
" 'tp4.mv': '203ce16afa53d60cfd8390fb3fb7d898',\n",
" 'tp5.mv': '3ec8626aa544cb075af1f78719a1fb48',\n",
" 'settings.json': '3ddc1036917034cec13503b74d3b051e',\n",
" '_CoqProject': 'b1d60da8dd98106e00006d99428d1403',\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a8f0c5e5-f995-43bd-9cb8-08099e6ea8fa",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def copy_update_file(filename):\n",
" file_hash, = !md5sum {\"../INF110/\" + filename} | cut -f1 -d' '\n",
" if file_hash != current_version[filename]:\n",
" print(\"Hash of \" + filename + \" has changed. Please update past_versions and current_version. New hash is \" + file_hash)\n",
" return\n",
" \n",
" for server in servers:\n",
" !scp -o StrictHostKeyChecking=no -J {proxyJump} {\"../INF110/\" + filename} core@{servers[server]['ip']}:/tmp\n",
" \n",
" for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" machine_ip = servers[volume['server_id']]['ip']\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo cp -nv /tmp/{filename} {volume['location']}\n",
" file_hash, = !ssh -J {proxyJump} core@{machine_ip} sudo md5sum {volume['location'] + \"/\" + filename} | cut -f1 -d' '\n",
" if file_hash in past_versions[filename]:\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo cp -v /tmp/{filename} {volume['location']}\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo chmod go+rw {volume['location'] + \"/\" + filename}\n",
"\n",
"copy_update_file(\"tp1.ipynb\")\n",
"copy_update_file(\"_CoqProject\")\n",
"copy_update_file(\"tp2.mv\")\n",
"copy_update_file(\"tp3.mv\")\n",
"copy_update_file(\"tp4.mv\")\n",
"copy_update_file(\"tp5.mv\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13608dc8-78f6-41d3-9cc3-f96c603a9437",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def copy_read_only_file(filename):\n",
" for server in servers:\n",
" !scp -J {proxyJump} {\"../INF110/\" + filename} core@{servers[server]['ip']}:/tmp\n",
" \n",
" for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" machine_ip = servers[volume['server_id']]['ip']\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo cp -v /tmp/{filename} {volume['location']}\n",
"\n",
"copy_read_only_file('README.md')\n",
"copy_read_only_file(\"tp1-mysterious-tm.png\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4cc929db-08fe-4b7b-9293-d86b7124f7cf",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"def copy_vscode_settings():\n",
" file_hash, = !md5sum {\"../INF110/.vscode/settings.json\"} | cut -f1 -d' '\n",
" if file_hash != current_version['settings.json']:\n",
" print(\"Hash of settings.json has changed. Please update past_versions and current_version. New hash is \" + file_hash)\n",
" return\n",
"\n",
" for server in servers:\n",
" !scp -J {proxyJump} -o StrictHostKeyChecking=no \"../INF110/.vscode/settings.json\" core@{servers[server]['ip']}:/tmp\n",
" \n",
" for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" machine_ip = servers[volume['server_id']]['ip']\n",
" vscode = volume['location'] + \"/.vscode\"\n",
" command = \"set -xe; \"\n",
" command += \"if [ ! -d \" + vscode + \" ]; then mkdir \" + vscode + \"; fi; \"\n",
" command += \"cp -nv /tmp/settings.json \" + vscode + \"/settings.json; \"\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo sh -c \"\\\"{command}\\\"\"\n",
" file_hash, = !ssh -J {proxyJump} core@{machine_ip} sudo md5sum {vscode + \"/settings.json\"} | cut -f1 -d' '\n",
" if file_hash in past_versions['settings.json']:\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo cp -v /tmp/settings.json {vscode}\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo chmod go+rw {vscode + \"/settings.json\"}\n",
"\n",
"copy_vscode_settings()"
]
},
{
"cell_type": "markdown",
"id": "55bd39c3-794e-43f2-8bf0-342361d61b64",
"metadata": {},
"source": [
"## Save snapshots of modified files"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db436f12-abb7-4b48-9e66-2f6ada19bdff",
"metadata": {},
"outputs": [],
"source": [
"def snapshot(filename, snapshot_name):\n",
" !mkdir -p {snapshot_name}\n",
" for volume in volumes:\n",
" if volume['server_id'] is not None:\n",
" machine_ip = servers[volume['server_id']]['ip']\n",
" file_hash, = !ssh -o StrictHostKeyChecking=no -J {proxyJump} core@{machine_ip} sudo md5sum {volume['location'] + \"/\" + filename} | cut -f1 -d' '\n",
" if not file_hash == current_version[filename] and not file_hash in past_versions[filename]:\n",
" snapshot_filename = \"/tmp/\" + volume['username'] + \"-\" + snapshot_name + \"-\" + filename\n",
" !ssh -J {proxyJump} core@{machine_ip} sudo cp {volume['location'] + \"/\" + filename} {snapshot_filename}\n",
" !scp -J {proxyJump} core@{machine_ip}:{snapshot_filename} {snapshot_name + \"/\" + volume['username'] + \"_\" + filename}\n",
"\n",
"#snapshot('tp1.ipynb', '2023-12-04-post-tp')\n",
"#snapshot('tp2.mv', '2023-12-13-post-tp')\n",
"#snapshot('tp3.mv', '2023-12-13-post-tp')\n",
"#snapshot('tp4.mv', '2024-01-12-post-tp')\n",
"#snapshot('tp5.mv', '2024-01-22-post-tp')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "35af9e82-e79c-4ad9-a843-3ebaab9f4d62",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}