Update README.md

3d-generation-pipeline/README.md

@@ -1,39 +1,20 @@
-### TODO
+# Testing the pipeline
-* pidev krabamise heli, kui kahe käega korraga krabada: "(Un)requesting ownership of Cube"
-* FMOD ChannelControl errorid
-* keyboard numbrid ei tööta pärast minigame'i completemist?
-* speech-to-text:
-	* vahepeal lakkab töötamast lih??? tundub et siis kui pikalt tühjust salvestab? ei, vahepeal läheb kohe alguses ka
-	* sööta talle nulle kui on disabled, sest ikka kuidagi tuvastab eelnevat rääkimist
-	* uurida miks buildis Whisper halvemini töötab
-* npc character:
-	* klaas on näha temast eespool
-* shape detection:
-	* peab mängijale kuidagi selgitama, kuidas scale'ida prinditud objekte
-	* prinditud objekti scale'imisele min ja max size limiit
-	* 3d printerile soft particle'id, et ei oleks teravaid ääri seina sisse minnes
-* archery range:
-	* archery targetite rework, et buildis ka töötaks
-	* võiks jääda kordama viimast instruktsiooni, kui mängija ei progressi edasi
-	* UFO möödalendamise animation/cutscene alguses kui ütleb "delta attacked by ufos" + panna juba alguses kapsli sisse ufo pöörlema
-* cafe:
-	* võiks saada hinnata saadud tellimust
-	* "order again" nupp
 
-Can't/Won't Do:
+Install required dependency libraries using
-* glTF loading: vahetada ära shader Universal render pipeline Lit, mitte panna buildi kaasa glTf oma - **ei saa, objekt on siis ilma tekstuurita, lihtsalt hall**
+```
+pip install -r requirements.txt
+```
 
+Make a copy of `.env.example`, rename it as `.env`, and insert the necessary config values.
 
-### Notes
+## Python notebooks
-* Functionalities that were implemented using major help from AI:
-	* animating NPC mouth movement based on voice amplitude
-	* adjusting spawned gltf objects' scale
-	* porting InvokeAI client and TRELLIS client to Unity scripts
-	* Getting Whisper stream to work with FMOD instead of Unity default audio
-* TRELLIS: added functionality to specify texture baking optimisation total steps as an argument (`texture_opt_total_steps`), to replace the hardcoded 2500. But this is not tracked in Git (because modified this https://github.com/IgorAherne/trellis-stable-projectorz/releases/tag/latest)
-* Custom Shader Variant Collection to include glTF-pbrMetallicRoughness shader in build
 
-Katse tähelepanekud:
+* [`local_image_generation.ipynb`](notebooks/local_image_generation.ipynb)
-* peab väga proper english accenti tegema
+* [`local_model_generation.ipynb`](notebooks/local_model_generation.ipynb)
-* mõne inimese puhul kuuleb "cat" asemel "cut"
+* [`subprocess_test.ipynb`](notebooks/subprocess_test.ipynb)
-* kui helitugevus oli liiga suur siis cafe waiter kuulis kohviku taustaheli taldriku klirinat kui "piano music"
+
+## Python scripts
+
+* [`generate_image_local.py`](generate_image_local.py)
+* [`generate_model_local.py`](generate_model_local.py)
+* [`start_pipeline.py`](start_pipeline.py)

3d-generation-pipeline/line_rms.txt

@@ -1,49 +0,0 @@
-3.170447962475009263e-05
-2.923115389421582222e-04
-2.510738931596279144e-02
-1.111792679876089096e-02
-6.692767888307571411e-02
-1.006313711404800415e-01
-7.780694961547851562e-02
-5.686730891466140747e-02
-5.614304915070533752e-02
-4.554714635014533997e-02
-4.514135792851448059e-02
-5.479728057980537415e-02
-4.272024706006050110e-02
-3.989587724208831787e-02
-4.298635944724082947e-02
-4.074911773204803467e-02
-2.244980260729789734e-02
-1.105279754847288132e-02
-1.347438804805278778e-02
-1.654553040862083435e-02
-1.846965588629245758e-02
-2.045047841966152191e-02
-1.407719496637582779e-02
-6.578906439244747162e-03
-1.353173051029443741e-02
-1.625132374465465546e-02
-5.863697826862335205e-02
-1.110599413514137268e-01
-9.950184077024459839e-02
-1.184522062540054321e-01
-1.000181213021278381e-01
-6.772108376026153564e-02
-7.621638476848602295e-02
-3.018615581095218658e-02
-9.624224901199340820e-02
-1.259753555059432983e-01
-1.276500672101974487e-01
-1.206035763025283813e-01
-1.011835709214210510e-01
-6.155343726277351379e-02
-3.734333068132400513e-02
-2.485111355781555176e-02
-2.122259326279163361e-02
-1.139380130916833878e-02
-7.472451310604810715e-03
-5.807624198496341705e-03
-1.960268709808588028e-03
-8.761089411564171314e-04
-3.071058890782296658e-04

3d-generation-pipeline/notebooks/local_model_generation.ipynb

@@ -14,7 +14,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 4,
    "id": "77b23cd8",
    "metadata": {},
    "outputs": [],
@@ -22,29 +22,16 @@
     "# API endpoint\n",
     "BASE_URL = \"http://127.0.0.1:7960\"\n",
     "\n",
-    "def generate_no_preview(image_base64: str):\n",
+    "def generate_no_preview(data: dict):\n",
     "    \"\"\"Generate 3D model from a single base64-encoded image without previews.\n",
     "    \n",
     "    Args:\n",
     "        image_base64: Base64 string of the image (without 'data:image/...' prefix)\n",
     "    \"\"\"\n",
     "    try:\n",
-    "        # Set generation parameters\n",
-    "        params = {\n",
-    "            'image_base64': image_base64,\n",
-    "            'seed': 42,\n",
-    "            'ss_guidance_strength': 7.5,\n",
-    "            'ss_sampling_steps': 30,\n",
-    "            'slat_guidance_strength': 7.5,\n",
-    "            'slat_sampling_steps': 30,\n",
-    "            'mesh_simplify_ratio': 0.95,\n",
-    "            'texture_size': 1024,\n",
-    "            'output_format': 'glb'\n",
-    "        }\n",
-    "        \n",
     "        # Start generation\n",
     "        print(\"Starting generation...\")\n",
-    "        response = requests.post(f\"{BASE_URL}/generate_no_preview\", data=params)\n",
+    "        response = requests.post(f\"{BASE_URL}/generate_no_preview\", data)\n",
     "        print(\"Response status:\", response.status_code)\n",
     "        response.raise_for_status()\n",
     "        \n",
@@ -75,7 +62,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 11,
    "id": "eb122295",
    "metadata": {},
    "outputs": [],
@@ -85,7 +72,19 @@
     "        image_data = image_file.read()\n",
     "\n",
     "    base64_encoded = base64.b64encode(image_data).decode('utf-8')\n",
-    "    model = generate_no_preview(base64_encoded)\n",
+    "    # Set generation parameters\n",
+    "    data = {\n",
+    "        'image_base64': base64_encoded,\n",
+    "        'seed': 42,\n",
+    "        'ss_guidance_strength': 8,\n",
+    "        'ss_sampling_steps': 10,\n",
+    "        'slat_guidance_strength': 6,\n",
+    "        'slat_sampling_steps': 10,\n",
+    "        'mesh_simplify_ratio': 0.9958,\n",
+    "        'texture_size': 1024,\n",
+    "        'output_format': 'glb'\n",
+    "    }\n",
+    "    model = generate_no_preview(data)\n",
     "    \n",
     "    with open(output_path, 'wb') as f:\n",
     "        f.write(model)\n",
@@ -94,7 +93,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 12,
    "id": "2ce7dfdf",
    "metadata": {},
    "outputs": [
@@ -107,14 +106,14 @@
       "Progress: 100%\n",
       "Downloading model...\n",
       "Model downloaded.\n",
-      "Model saved to test_resources/style_test_3_model.glb\n"
+      "Model saved to test_resources/mesh_simplify_ratio_0.998.glb\n"
      ]
     }
    ],
    "source": [
     "\n",
-    "image_path = 'test_resources/style_test_3.jpg'\n",
+    "image_path = 'test_resources/512x512.png'\n",
-    "output_path = \"test_resources/style_test_3_model.glb\"\n",
+    "output_path = \"test_resources/mesh_simplify_ratio_0.998.glb\"\n",
     "\n",
     "generate_model(image_path, output_path)"
    ]
@@ -130,7 +129,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": ".venv",
+   "display_name": ".venv (3.10.11)",
    "language": "python",
    "name": "python3"
   },

Assets/_PROJECT/Components/Bow/Scripts/ArcheryRange.cs

@@ -87,8 +87,6 @@ public class ArcheryRange : NetworkBehaviour
         if (exitedPlayer == _scoredPlayer) keyboardManager.DeActivate(); // If the player ´refuses to enter their name.
     }
 
-
-
     public override void OnStopServer()
     {
         base.OnStopServer();
@@ -133,18 +131,18 @@ public class ArcheryRange : NetworkBehaviour
 
     private ArcheryTarget SpawnTarget(Vector3 randomPos)
     {
-        GameObject targetObject;
+        // Spawn default UFO
-        if (modelDisplay.Model == null)
+        GameObject targetObject = Instantiate(targetPrefab, randomPos, targetStartPosition.rotation, null);
+        if (modelDisplay.Model != null)
         {
-            // spawn default UFO
+            // Spawn generated object directly as default UFO's child
-            targetObject = Instantiate(targetPrefab, randomPos, Quaternion.identity, null);
+            GameObject childObject = Instantiate(modelDisplay.Model);
-        } else
+            childObject.transform.SetParent(targetObject.transform, false);
-        {
+            childObject.transform.localScale *= targetSizeIncrease;
-            // spawn generated model
+
-            targetObject = Instantiate(modelDisplay.Model, randomPos, Quaternion.identity, null);
+            // Disable UFO's default body
-            InitializeArcheryTargetObject(targetObject);
+            targetObject.transform.Find("Body").gameObject.SetActive(false);
         }
-        targetObject.transform.rotation = targetStartPosition.rotation;
 
         ArcheryTarget target = targetObject.GetComponent<ArcheryTarget>();
         target.endPosition = targetEndPosition.position;
@@ -153,22 +151,6 @@ public class ArcheryRange : NetworkBehaviour
         return target;
     }
 
-    private void InitializeArcheryTargetObject(GameObject targetObject)
-    {
-        ArcheryTarget archeryTarget = targetObject.AddComponent<ArcheryTarget>();
-        archeryTarget.pointsText = archeryTargetPointsText;
-
-        Rigidbody rigidbody = targetObject.AddComponent<Rigidbody>();
-        rigidbody.useGravity = false;
-        rigidbody.isKinematic = true;
-
-        targetObject.AddComponent<NetworkObject>();
-        targetObject.AddComponent<NetworkTransform>();
-
-        targetObject.transform.localScale *= targetSizeIncrease;
-    }
-
-
     public void ResetRange()
     {
         if (!IsServer) return;

Assets/_PROJECT/Prefabs/XR/Bow/TargetUFO.prefab

@@ -9,9 +9,10 @@ GameObject:
   serializedVersion: 6
   m_Component:
   - component: {fileID: 3337587390139541606}
-  - component: {fileID: 627565954663756710}
+  - component: {fileID: 1081661767482416249}
+  - component: {fileID: 3779719438397915148}
   m_Layer: 0
-  m_Name: SportModel
+  m_Name: Body
   m_TagString: Untagged
   m_Icon: {fileID: 0}
   m_NavMeshLayer: 0
@@ -26,7 +27,7 @@ Transform:
   m_GameObject: {fileID: 3508152469845169189}
   m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
   m_LocalPosition: {x: 0, y: 0, z: 0}
-  m_LocalScale: {x: 0.24, y: 0.2, z: 0.2}
+  m_LocalScale: {x: 1, y: 1, z: 1}
   m_ConstrainProportionsScale: 0
   m_Children:
   - {fileID: 4011537191296717654}
@@ -35,47 +36,50 @@ Transform:
   m_Father: {fileID: 7527508320363503049}
   m_RootOrder: 0
   m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
---- !u!114 &627565954663756710
+--- !u!136 &1081661767482416249
-MonoBehaviour:
+CapsuleCollider:
   m_ObjectHideFlags: 0
   m_CorrespondingSourceObject: {fileID: 0}
   m_PrefabInstance: {fileID: 0}
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 3508152469845169189}
+  m_Material: {fileID: 0}
+  m_IncludeLayers:
+    serializedVersion: 2
+    m_Bits: 0
+  m_ExcludeLayers:
+    serializedVersion: 2
+    m_Bits: 0
+  m_LayerOverridePriority: 0
+  m_IsTrigger: 0
+  m_ProvidesContacts: 0
   m_Enabled: 1
-  m_EditorHideFlags: 0
+  serializedVersion: 2
-  m_Script: {fileID: 11500000, guid: a2836e36774ca1c4bbbee976e17b649c, type: 3}
+  m_Radius: 1.07
-  m_Name: 
+  m_Height: 0.97
-  m_EditorClassIdentifier: 
+  m_Direction: 1
-  _componentIndexCache: 2
+  m_Center: {x: 0.16, y: -0.51, z: 0}
-  _addedNetworkObject: {fileID: 2689384198849609103}
+--- !u!65 &3779719438397915148
-  _networkObjectCache: {fileID: 2689384198849609103}
+BoxCollider:
-  _synchronizeParent: 0
+  m_ObjectHideFlags: 0
-  _packing:
+  m_CorrespondingSourceObject: {fileID: 0}
-    Position: 1
+  m_PrefabInstance: {fileID: 0}
-    Rotation: 1
+  m_PrefabAsset: {fileID: 0}
-    Scale: 0
+  m_GameObject: {fileID: 3508152469845169189}
-  _interpolation: 2
+  m_Material: {fileID: 0}
-  _extrapolation: 2
+  m_IncludeLayers:
-  _enableTeleport: 0
+    serializedVersion: 2
-  _teleportThreshold: 1
+    m_Bits: 0
-  _clientAuthoritative: 1
+  m_ExcludeLayers:
-  _sendToOwner: 1
+    serializedVersion: 2
-  _synchronizePosition: 1
+    m_Bits: 0
-  _positionSnapping:
+  m_LayerOverridePriority: 0
-    X: 0
+  m_IsTrigger: 0
-    Y: 0
+  m_ProvidesContacts: 0
-    Z: 0
+  m_Enabled: 1
-  _synchronizeRotation: 1
+  serializedVersion: 3
-  _rotationSnapping:
+  m_Size: {x: 4.5, y: 0.5, z: 3.77}
-    X: 0
+  m_Center: {x: 0.12, y: -0.24, z: 0.06}
-    Y: 0
-    Z: 0
-  _synchronizeScale: 1
-  _scaleSnapping:
-    X: 0
-    Y: 0
-    Z: 0
 --- !u!1 &6744593628588765187
 GameObject:
   m_ObjectHideFlags: 0
@@ -87,7 +91,6 @@ GameObject:
   - component: {fileID: 1251512755648678878}
   - component: {fileID: 4963625795242203983}
   - component: {fileID: 5457198189491340073}
-  - component: {fileID: 8959643074287331097}
   m_Layer: 0
   m_Name: BezierCurve.007
   m_TagString: Untagged
@@ -102,13 +105,13 @@ Transform:
   m_PrefabInstance: {fileID: 0}
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 6744593628588765187}
-  m_LocalRotation: {x: 0.00000031987892, y: 0, z: -0, w: 1}
+  m_LocalRotation: {x: 0.00000031987892, y: -0, z: -0, w: 1}
-  m_LocalPosition: {x: 0.5, y: -1.2, z: 0.3}
+  m_LocalPosition: {x: 0.120000005, y: -0.24000001, z: 0.06000042}
-  m_LocalScale: {x: 1, y: 1, z: 1}
+  m_LocalScale: {x: 0.24, y: 0.2, z: 0.2}
   m_ConstrainProportionsScale: 0
   m_Children: []
   m_Father: {fileID: 3337587390139541606}
-  m_RootOrder: 2
+  m_RootOrder: -1
   m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
 --- !u!33 &4963625795242203983
 MeshFilter:
@@ -161,47 +164,6 @@ MeshRenderer:
   m_SortingLayer: 0
   m_SortingOrder: 0
   m_AdditionalVertexStreams: {fileID: 0}
---- !u!114 &8959643074287331097
-MonoBehaviour:
-  m_ObjectHideFlags: 0
-  m_CorrespondingSourceObject: {fileID: 0}
-  m_PrefabInstance: {fileID: 0}
-  m_PrefabAsset: {fileID: 0}
-  m_GameObject: {fileID: 6744593628588765187}
-  m_Enabled: 1
-  m_EditorHideFlags: 0
-  m_Script: {fileID: 11500000, guid: a2836e36774ca1c4bbbee976e17b649c, type: 3}
-  m_Name: 
-  m_EditorClassIdentifier: 
-  _componentIndexCache: 5
-  _addedNetworkObject: {fileID: 2689384198849609103}
-  _networkObjectCache: {fileID: 2689384198849609103}
-  _synchronizeParent: 0
-  _packing:
-    Position: 1
-    Rotation: 1
-    Scale: 0
-  _interpolation: 2
-  _extrapolation: 2
-  _enableTeleport: 0
-  _teleportThreshold: 1
-  _clientAuthoritative: 1
-  _sendToOwner: 1
-  _synchronizePosition: 1
-  _positionSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeRotation: 1
-  _rotationSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeScale: 1
-  _scaleSnapping:
-    X: 0
-    Y: 0
-    Z: 0
 --- !u!1 &7680366769166628765
 GameObject:
   m_ObjectHideFlags: 0
@@ -213,7 +175,6 @@ GameObject:
   - component: {fileID: 4011537191296717654}
   - component: {fileID: 1910778184414920112}
   - component: {fileID: 363608006073787400}
-  - component: {fileID: 4439869206153374580}
   m_Layer: 0
   m_Name: BezierCurve.005
   m_TagString: Untagged
@@ -228,13 +189,13 @@ Transform:
   m_PrefabInstance: {fileID: 0}
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 7680366769166628765}
-  m_LocalRotation: {x: 0.00000031987892, y: 0, z: -0, w: 1}
+  m_LocalRotation: {x: 0.00000031987892, y: -0, z: -0, w: 1}
-  m_LocalPosition: {x: 0.5, y: 0.3, z: 0.3}
+  m_LocalPosition: {x: 0.120000005, y: 0.060000062, z: 0.06000042}
-  m_LocalScale: {x: 1, y: 1, z: 1}
+  m_LocalScale: {x: 0.24, y: 0.2, z: 0.2}
   m_ConstrainProportionsScale: 0
   m_Children: []
   m_Father: {fileID: 3337587390139541606}
-  m_RootOrder: 0
+  m_RootOrder: -1
   m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
 --- !u!33 &1910778184414920112
 MeshFilter:
@@ -286,47 +247,6 @@ MeshRenderer:
   m_SortingLayer: 0
   m_SortingOrder: 0
   m_AdditionalVertexStreams: {fileID: 0}
---- !u!114 &4439869206153374580
-MonoBehaviour:
-  m_ObjectHideFlags: 0
-  m_CorrespondingSourceObject: {fileID: 0}
-  m_PrefabInstance: {fileID: 0}
-  m_PrefabAsset: {fileID: 0}
-  m_GameObject: {fileID: 7680366769166628765}
-  m_Enabled: 1
-  m_EditorHideFlags: 0
-  m_Script: {fileID: 11500000, guid: a2836e36774ca1c4bbbee976e17b649c, type: 3}
-  m_Name: 
-  m_EditorClassIdentifier: 
-  _componentIndexCache: 3
-  _addedNetworkObject: {fileID: 2689384198849609103}
-  _networkObjectCache: {fileID: 2689384198849609103}
-  _synchronizeParent: 0
-  _packing:
-    Position: 1
-    Rotation: 1
-    Scale: 0
-  _interpolation: 2
-  _extrapolation: 2
-  _enableTeleport: 0
-  _teleportThreshold: 1
-  _clientAuthoritative: 1
-  _sendToOwner: 1
-  _synchronizePosition: 1
-  _positionSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeRotation: 1
-  _rotationSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeScale: 1
-  _scaleSnapping:
-    X: 0
-    Y: 0
-    Z: 0
 --- !u!1 &7983999977322064054
 GameObject:
   m_ObjectHideFlags: 0
@@ -340,8 +260,6 @@ GameObject:
   - component: {fileID: 8954823527229761837}
   - component: {fileID: 2689384198849609103}
   - component: {fileID: 1843883317757509263}
-  - component: {fileID: 7451921017238644471}
-  - component: {fileID: 2131608416831220487}
   m_Layer: 0
   m_Name: TargetUFO
   m_TagString: Untagged
@@ -357,7 +275,7 @@ Transform:
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 7983999977322064054}
   m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
-  m_LocalPosition: {x: 1.398, y: 1.1, z: 8.292}
+  m_LocalPosition: {x: 0, y: 0, z: 0}
   m_LocalScale: {x: 1, y: 1, z: 1}
   m_ConstrainProportionsScale: 0
   m_Children:
@@ -429,10 +347,6 @@ MonoBehaviour:
   _networkBehaviours:
   - {fileID: 4850307641519017041}
   - {fileID: 1843883317757509263}
-  - {fileID: 627565954663756710}
-  - {fileID: 4439869206153374580}
-  - {fileID: 6540522042633253623}
-  - {fileID: 8959643074287331097}
   <ParentNetworkObject>k__BackingField: {fileID: 0}
   <ChildNetworkObjects>k__BackingField: []
   _isNetworked: 1
@@ -487,50 +401,6 @@ MonoBehaviour:
     X: 0
     Y: 0
     Z: 0
---- !u!136 &7451921017238644471
-CapsuleCollider:
-  m_ObjectHideFlags: 0
-  m_CorrespondingSourceObject: {fileID: 0}
-  m_PrefabInstance: {fileID: 0}
-  m_PrefabAsset: {fileID: 0}
-  m_GameObject: {fileID: 7983999977322064054}
-  m_Material: {fileID: 0}
-  m_IncludeLayers:
-    serializedVersion: 2
-    m_Bits: 0
-  m_ExcludeLayers:
-    serializedVersion: 2
-    m_Bits: 0
-  m_LayerOverridePriority: 0
-  m_IsTrigger: 0
-  m_ProvidesContacts: 0
-  m_Enabled: 1
-  serializedVersion: 2
-  m_Radius: 1.07
-  m_Height: 0.97
-  m_Direction: 1
-  m_Center: {x: 0.16, y: -0.51, z: 0}
---- !u!65 &2131608416831220487
-BoxCollider:
-  m_ObjectHideFlags: 0
-  m_CorrespondingSourceObject: {fileID: 0}
-  m_PrefabInstance: {fileID: 0}
-  m_PrefabAsset: {fileID: 0}
-  m_GameObject: {fileID: 7983999977322064054}
-  m_Material: {fileID: 0}
-  m_IncludeLayers:
-    serializedVersion: 2
-    m_Bits: 0
-  m_ExcludeLayers:
-    serializedVersion: 2
-    m_Bits: 0
-  m_LayerOverridePriority: 0
-  m_IsTrigger: 0
-  m_ProvidesContacts: 0
-  m_Enabled: 1
-  serializedVersion: 3
-  m_Size: {x: 4.5, y: 0.5, z: 3.77}
-  m_Center: {x: 0.12, y: -0.24, z: 0.06}
 --- !u!1 &8462201784916155029
 GameObject:
   m_ObjectHideFlags: 0
@@ -542,7 +412,6 @@ GameObject:
   - component: {fileID: 2406214921254263651}
   - component: {fileID: 6767210086105080178}
   - component: {fileID: 3766991064380355318}
-  - component: {fileID: 6540522042633253623}
   m_Layer: 0
   m_Name: BezierCurve.006
   m_TagString: Untagged
@@ -558,12 +427,12 @@ Transform:
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 8462201784916155029}
   m_LocalRotation: {x: -0.00000015993946, y: 0.86602545, z: -0.00000027702328, w: -0.49999997}
-  m_LocalPosition: {x: 0.5, y: -3.4, z: 0.3}
+  m_LocalPosition: {x: 0.121, y: -0.663, z: 0.072}
-  m_LocalScale: {x: 1, y: 1, z: 1}
+  m_LocalScale: {x: 0.22, y: 0.2, z: 0.24}
   m_ConstrainProportionsScale: 0
   m_Children: []
   m_Father: {fileID: 3337587390139541606}
-  m_RootOrder: 1
+  m_RootOrder: -1
   m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
 --- !u!33 &6767210086105080178
 MeshFilter:
@@ -615,44 +484,3 @@ MeshRenderer:
   m_SortingLayer: 0
   m_SortingOrder: 0
   m_AdditionalVertexStreams: {fileID: 0}
---- !u!114 &6540522042633253623
-MonoBehaviour:
-  m_ObjectHideFlags: 0
-  m_CorrespondingSourceObject: {fileID: 0}
-  m_PrefabInstance: {fileID: 0}
-  m_PrefabAsset: {fileID: 0}
-  m_GameObject: {fileID: 8462201784916155029}
-  m_Enabled: 1
-  m_EditorHideFlags: 0
-  m_Script: {fileID: 11500000, guid: a2836e36774ca1c4bbbee976e17b649c, type: 3}
-  m_Name: 
-  m_EditorClassIdentifier: 
-  _componentIndexCache: 4
-  _addedNetworkObject: {fileID: 2689384198849609103}
-  _networkObjectCache: {fileID: 2689384198849609103}
-  _synchronizeParent: 0
-  _packing:
-    Position: 1
-    Rotation: 1
-    Scale: 0
-  _interpolation: 2
-  _extrapolation: 2
-  _enableTeleport: 0
-  _teleportThreshold: 1
-  _clientAuthoritative: 1
-  _sendToOwner: 1
-  _synchronizePosition: 1
-  _positionSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeRotation: 1
-  _rotationSnapping:
-    X: 0
-    Y: 0
-    Z: 0
-  _synchronizeScale: 1
-  _scaleSnapping:
-    X: 0
-    Y: 0
-    Z: 0

README.md

@@ -1,85 +1,97 @@
 ![DeltaVR](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/Readme-Header.png)
 
-DeltaVR is a virtual reality experience set in the [Delta Centre](https://delta.ut.ee/) of the [University of Tartu](https://ut.ee/). The virtual Delta Building includes several interactive scenes that demonstrate the teaching and research actively done in the Delta Centre.
+# DeltaVR: Player-Prompted 3D Model Generation Game Mechanics
 
-The application works on PCVR, Meta Quest 2 and 3, HTC Vive, and regular Windows PC platforms. There is cross-platform multiplayer functionality that enables several users to be in the same virtual environment from both VR and PC platforms.
+This is a fork of the [main DeltaVR project](https://cgvrgit.ulno.net/cgvr/DeltaVR). To learn more about DeltaVR itself, look at the main repository's README.
 
+This project is about investigating the feasibility of player-prompted runtime 3D model generation game mechanics. These are game mechanics that allow the player to generate arbitrary 3D objects using natural language speech. This project includes three mini-games that use this speech-to-3D system as a core game mechanic.
 
-## Features
+## Mini-Games
 
-### Exploration
+### Delta Cafe
+<img src="https://cgvrgit.ulno.net/henrisel/DeltaVR3DModelGeneration/media/branch/master/Doc/screenshots/delta_cafe_screenshot.png" width="800">
 
-DeltaVR features the first two floors of the Delta Educational Building for **exploration and discovery**. There are many diegetic elements representing the studies and research conducted at the Delta Building, such as robotics, the high-performance computing server room, video game development, and student life.
+### Archery Range
+<img src="https://cgvrgit.ulno.net/henrisel/DeltaVR3DModelGeneration/media/branch/master/Doc/screenshots/archery_range_screenshot.png" width="800">
 
-![DeltaVR Exploration](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Explore-Clip.gif)
+### Shape Detection
+<img src="https://cgvrgit.ulno.net/henrisel/DeltaVR3DModelGeneration/media/branch/master/Doc/screenshots/shape_detection_gameobjects.png" width="800">
 
-### UFO Bow Game
+Playthrough videos of each mini-game are available on YouTube: https://www.youtube.com/playlist?list=PLqIwPfWyxyaH-YHn64jW59aFhQ5OKf9CW
 
-At the terrace on the second floor, Delta explorers can defend the building from UFO-s using a bow and **achieve high scores**.
+## Setup Instructions
 
-![DeltaVR UFO Bow Game](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/UFO-Bow-Game-Clip.gif)
+The built version of the game is available here: https://owncloud.ut.ee/owncloud/s/tXBd4XGpbXTi7Hm.
 
-### Bolt Self-Driving Car
+The build and the editor version both rely on a speech-to-3D pipeline to function. This pipeline needs to be set up locally and consists of two additional software applications in addition to the Unity game itself: InvokeAI and TRELLIS.
 
-The courtyard between the Educational and Entrepreneurial buildings of the Delta Centre, the explorers can see the Bolt Self-Driving Car. This car is developed by the [[http://adl.cs.ut.ee/|Autonomous Driving Lab]] of the [[https://cs.ut.ee|Institute of Computer Science]]. If one is brave enough, they can stop the car and catch a ride, simulating both the **feeling of being in a self-driving vehicle** as well as VR motion sickness.
+### InvokeAI
 
-![DeltaVR Bolt Self-Driving Car](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Bolt-Self-Driving-Car-Clip.gif)
+InvokeAI is used to generate images from text prompts. To install InvokeAI, follow the instructions in the official guide: https://invoke.ai/start-here/installation.
 
-### Space Walk
+After installing it, an AI model needs to be set up in InvokeAI. This project has been tested with Stable Diffusion XL, but other models can also be used. Documentation on setting up models: https://invoke.ai/concepts/models.
 
-Where the actual Delta building has a set of skywalks connecting it with the entrepreneurship building, DeltaVR has a set of portals leading to the Space walk experience. In it, one can move in the **vastness of space** and experience **changes in gravity**. A fleet of UFO ships react to one's presence and come to investigate the arrival.
+Finally, to enable communication with InvokeAI across different computers, enable the Server Mode in InvokeAI settings.
 
-![DeltaVR Spacewalk](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Skywalk-Clip.gif)
+### TRELLIS
 
-### Server Room
+TRELLIS is used to generate 3D models based on images. The simplest way to install TRELLIS on Windows is via the one-click installer provided in this fork of the original TRELLIS repo: https://github.com/IgorAherne/trellis-stable-projectorz. The download package includes instructions on how to launch TRELLIS as a local API server. For the default configuration, use the following command:
+```
+.\run.bat --ip 0.0.0.0
+```
 
-On the second floor, one can hear the humming of the servers. Should they investigate, they will find a room of server racks and a large red button. Should they push the button, they will trigger the **fire alarm** and have the server room fill with harmful invisible gas. This propms the player to escape the room. This largerly **auditory experience** is noted to be engaging and immersive. It represents the work of UT HPC in maintaining the servers of the University of Tartu.
+### Unity
 
-![DeltaVR Server Room](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Server-Room-Clip.gif)
+In both editor mode and the built version, after first starting the game, the `config.json` file is created in the game's root directory. This file is used to persist configurable values between play sessions. The following values need to be set correctly for the game to be able to communicate with InvokeAI and TRELLIS:
 
-### Interactive Map
+* `invokeAiUrl` - the URL of InvokeAI (for example `http://192.168.0.53:9090`)
+* `invokeAiModelKey` - the key of the specific model set up in InvokeAI to use. To see the keys of your installed models, send a GET request to `{INVOKEAI_URL}/api/v2/models/?model_type=main` (for example, `http://192.168.0.53:9090/api/v2/models/?model_type=main`)
+* `trellisUrl` - the URL of TRELLIS (for example, `http://192.168.0.53:7960`)
 
-To navigate the two floors of the large Delta Educational Building, explorers have an interactive map. This provides a clear overview of where they currently are and what other interactions are located across the building. Explorers can teleport to a **select interactive experiences**, while others are left for them to discover based on the hints on the map.
+The [3d-generation-pipeline](3d-generation-pipeline/README.md) folder includes Python notebooks and scripts that can be used to test the pipeline working together.
 
-![DeltaVR Interactive Map](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Interactive-Map-Clip.gif)
+## Unresolved bugs and potential improvements for the future
+* continuous grabbing sound when grabbing with both hands at the same time: "(Un)requesting ownership of Cube"
+* FMOD ChannelControl errors
+* speech-to-text: investigate why Whisper still sometimes stops detecting speech
+* glTF loading: investigate how to use the Universal Render Pipeline Lit shader for the glTF-loaded objects.
+* NPC characters:
+	* glass wall is sometimes incorrectly visible in front of the NPC
+* Delta Cafe:
+	* ability for players to rate the orders
+	* "order again" button
+* Archery Range:
+	* NPC should repeat the last instruction when the player is not progressing
+	* reuse the UFO flying animation from Space Walk in the beginning, when NPC talks about looking out the window and the UFO attack + have the default UFO already in the model display capsule
+* Shape Detection:
+	* explain to the player how to scale printed objects
+	* set min and max scale limits on printed objects
+	* set 3d printer soft particles to avoid sharp edges when particles go inside the printer wall
 
-### Whiteboard
-
-In the virtual Computer Graphics and Virtual Reality Study Lab, explorers can use spray paint cans to draw on a whiteboard. Surprisingly, this is one of the **more popular interactive experiences** of DeltaVR.
-
-![DeltaVR Whiteboard](https://cgvrgit.ulno.net/cgvr/DeltaVR/media/branch/master/Doc/clips/Whiteboard-Clip.gif)
 
 ## Credits
 
-**Ranno Samuel Adson**<br/>
+**Samuel Saar**<br/>
-User experience design. Additional interactions. Interaction improvements.
+Creation of the following 3D models used in the mini-games:
-
+* Delta Cafe: pen, notepad, serving tray
-**Toomas Tamm**<br/>
+* Archery Range: capsule display, microphone stand, wires, table, screens
-Project architecture, model optimization, lighting. [Bachelor's Thesis](https://comserv.cs.ut.ee/ati_thesis/datasheet.php?id=71682) ([poster](https://courses.cs.ut.ee/student_projects/download/478.pdf)), [Master's Thesis](https://comserv.cs.ut.ee/ati_thesis/datasheet.php?id=77065&language=en).
+* Shape Detection: computer printer, 3D printer, radio transmitter, shape detector
-
-**Joonas Püks**<br/>
-Multiplayer and cross-play functionality. [Bachelor's Thesis](https://comserv.cs.ut.ee/ati_thesis/datasheet.php?id=74390) ([poster](https://courses.cs.ut.ee/student_projects/download/534.pdf)).
 
 **Timur Nizamov**<br/>
-Technical sound design.
+Sound effects and integrating voicelines into Unity
 
-**Raimond Tunnel**<br/>
+**Madis Vasser**<br/>
-Project management, visual design.
+Voice acting for the Ufo and Professor characters
 
-Developed in the [Computer Graphcis and Virtual Reality Study Lab](https://cgvr.cs.ut.ee/) of the [Institute of Computer Science, University of Tartu](https://cs.ut.ee).
+**Ulrich Norbisrath**<br/>
+Voice acting for the Robert character
 
-### Used Attributions
+**Ranno Samuel Adson**<br/>
+Technical help and tips for DeltaVR
 
-| Description                                         | License                                      | Source                                                                                      | Author           |
+(These credits are specifically for the work of this fork. Credits for the main DeltaVR project are shown in the main project's README.)
-|-----------------------------------------------------|----------------------------------------------|---------------------------------------------------------------------------------------------|------------------|
-| Server rack model                                   | Royalty Free, No AI License                  | [Link](https://www.cgtrader.com/free-3d-models/electronics/computer/simple-server-model)   | anymelok         |
-| Robot movement sound                                | Creative Commons 0                           | [Link](https://freesound.org/people/Brazilio123/sounds/661435/)                            | Brazilio123      |
-| Spacewalk UFO sound                                 | Attribution NonCommercial 4.0                           | [Link](https://freesound.org/people/Speedenza/sounds/209366/)                           | Speedenza     |
-| Keyboard icons                                      | Creative Commons Attribution-NoDerivs 3.0    | [Link](https://icons8.com/)                                                                 | icons8           |
 
------
+Developed in the [Computer Graphics and Virtual Reality Study Lab](https://cgvr.cs.ut.ee/) of the [Institute of Computer Science, University of Tartu](https://cs.ut.ee).
-
-DeltaVR was moved to this repository in 2025. The previous repository is available here: [[https://gitlab.com/UT-CGVR/deltavr]]
 
 -----
 

config.json

@@ -1,7 +1,7 @@
 {
     "isContinuousLocomotion": true,
     "continuousLocomotionSpeed": 3.0,
-    "volumeMaster": 0.5969846844673157,
+    "volumeMaster": 0.4399948716163635,
     "volumeAmbient": 0.1889490932226181,
     "volumeMusic": 0.5,
     "volumeSFX": 0.5,