From 34c693aaa3121f3a952c41c8a5db088ea0913e26 Mon Sep 17 00:00:00 2001 From: henrisel Date: Wed, 13 May 2026 14:14:28 +0300 Subject: [PATCH] update READMEs --- 3d-generation-pipeline/README.md | 50 ++++++------------- 3d-generation-pipeline/line_rms.txt | 49 ------------------ ...est.ipynb => local_model_generation.ipynb} | 0 README.md | 21 ++++++++ 4 files changed, 37 insertions(+), 83 deletions(-) delete mode 100644 3d-generation-pipeline/line_rms.txt rename 3d-generation-pipeline/notebooks/{local_model_generation_test.ipynb => local_model_generation.ipynb} (100%) diff --git a/3d-generation-pipeline/README.md b/3d-generation-pipeline/README.md index da726f70..acc71724 100644 --- a/3d-generation-pipeline/README.md +++ b/3d-generation-pipeline/README.md @@ -1,38 +1,20 @@ -### TODO -* pidev krabamise heli, kui kahe käega korraga krabada: "(Un)requesting ownership of Cube" -* FMOD ChannelControl errorid -* keyboard numbrid ei tööta pärast minigame'i completemist? -* speech-to-text: - * vahepeal lakkab töötamast lih??? tundub et siis kui pikalt tühjust salvestab? ei, vahepeal läheb kohe alguses ka - * sööta talle nulle kui on disabled, sest ikka kuidagi tuvastab eelnevat rääkimist - * uurida miks buildis Whisper halvemini töötab -* npc character: - * klaas on näha temast eespool -* shape detection: - * peab mängijale kuidagi selgitama, kuidas scale'ida prinditud objekte - * prinditud objekti scale'imisele min ja max size limiit - * 3d printerile soft particle'id, et ei oleks teravaid ääri seina sisse minnes -* archery range: - * võiks jääda kordama viimast instruktsiooni, kui mängija ei progressi edasi - * UFO möödalendamise animation/cutscene alguses kui ütleb "delta attacked by ufos" + panna juba alguses kapsli sisse ufo pöörlema -* cafe: - * võiks saada hinnata saadud tellimust - * "order again" nupp +# Testing the pipeline -Can't/Won't Do: -* glTF loading: vahetada ära shader Universal render pipeline Lit, mitte panna buildi kaasa glTf oma - **ei saa, objekt on siis ilma tekstuurita, lihtsalt hall** +Install required dependency libraries using +``` +pip install -r requirements.txt +``` +Make a copy of `.env.example`, rename it as `.env`, and insert the necessary config values. -### Notes -* Functionalities that were implemented using major help from AI: - * animating NPC mouth movement based on voice amplitude - * adjusting spawned gltf objects' scale - * porting InvokeAI client and TRELLIS client to Unity scripts - * Getting Whisper stream to work with FMOD instead of Unity default audio -* TRELLIS: added functionality to specify texture baking optimisation total steps as an argument (`texture_opt_total_steps`), to replace the hardcoded 2500. But this is not tracked in Git (because modified this https://github.com/IgorAherne/trellis-stable-projectorz/releases/tag/latest) -* Custom Shader Variant Collection to include glTF-pbrMetallicRoughness shader in build +## Python notebooks -Katse tähelepanekud: -* peab väga proper english accenti tegema -* mõne inimese puhul kuuleb "cat" asemel "cut" -* kui helitugevus oli liiga suur siis cafe waiter kuulis kohviku taustaheli taldriku klirinat kui "piano music" +* [`local_image_generation.ipynb`](notebooks/local_image_generation.ipynb) +* [`local_model_generation.ipynb`](notebooks/local_model_generation.ipynb) +* [`subprocess_test.ipynb`](notebooks/subprocess_test.ipynb) + +## Python scripts + +* [`generate_image_local.py`](generate_image_local.py) +* [`generate_model_local.py`](generate_model_local.py) +* [`start_pipeline.py`](start_pipeline.py) diff --git a/3d-generation-pipeline/line_rms.txt b/3d-generation-pipeline/line_rms.txt deleted file mode 100644 index e6cddc51..00000000 --- a/3d-generation-pipeline/line_rms.txt +++ /dev/null @@ -1,49 +0,0 @@ -3.170447962475009263e-05 -2.923115389421582222e-04 -2.510738931596279144e-02 -1.111792679876089096e-02 -6.692767888307571411e-02 -1.006313711404800415e-01 -7.780694961547851562e-02 -5.686730891466140747e-02 -5.614304915070533752e-02 -4.554714635014533997e-02 -4.514135792851448059e-02 -5.479728057980537415e-02 -4.272024706006050110e-02 -3.989587724208831787e-02 -4.298635944724082947e-02 -4.074911773204803467e-02 -2.244980260729789734e-02 -1.105279754847288132e-02 -1.347438804805278778e-02 -1.654553040862083435e-02 -1.846965588629245758e-02 -2.045047841966152191e-02 -1.407719496637582779e-02 -6.578906439244747162e-03 -1.353173051029443741e-02 -1.625132374465465546e-02 -5.863697826862335205e-02 -1.110599413514137268e-01 -9.950184077024459839e-02 -1.184522062540054321e-01 -1.000181213021278381e-01 -6.772108376026153564e-02 -7.621638476848602295e-02 -3.018615581095218658e-02 -9.624224901199340820e-02 -1.259753555059432983e-01 -1.276500672101974487e-01 -1.206035763025283813e-01 -1.011835709214210510e-01 -6.155343726277351379e-02 -3.734333068132400513e-02 -2.485111355781555176e-02 -2.122259326279163361e-02 -1.139380130916833878e-02 -7.472451310604810715e-03 -5.807624198496341705e-03 -1.960268709808588028e-03 -8.761089411564171314e-04 -3.071058890782296658e-04 diff --git a/3d-generation-pipeline/notebooks/local_model_generation_test.ipynb b/3d-generation-pipeline/notebooks/local_model_generation.ipynb similarity index 100% rename from 3d-generation-pipeline/notebooks/local_model_generation_test.ipynb rename to 3d-generation-pipeline/notebooks/local_model_generation.ipynb diff --git a/README.md b/README.md index 868ffd70..1ae80237 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,27 @@ In both editor mode and the built version, after first starting the game, the `c * `invokeAiModelKey` - the key of the specific model set up in InvokeAI to use. To see the keys of your installed models, send a GET request to `{INVOKEAI_URL}/api/v2/models/?model_type=main` (for example, `http://192.168.0.53:9090/api/v2/models/?model_type=main`) * `trellisUrl` - the URL of TRELLIS (for example, `http://192.168.0.53:7960`) +The [3d-generation-pipeline](3d-generation-pipeline/README.md) folder includes Python notebooks and scripts that can be used to test the pipeline working together. + +## Unresolved bugs and potential improvements for the future +* continuous grabbing sound when grabbing with both hands at the same time: "(Un)requesting ownership of Cube" +* FMOD ChannelControl errors +* speech-to-text: investigate why Whisper still sometimes stops detecting speech +* glTF loading: investigate how to use the Universal Render Pipeline Lit shader for the glTF-loaded objects. +* NPC characters: + * glass wall is sometimes incorrectly visible in front of the NPC +* Delta Cafe: + * ability for players to rate the orders + * "order again" button +* Archery Range: + * NPC should repeat the last instruction when the player is not progressing + * reuse the UFO flying animation from Space Walk in the beginning, when NPC talks about looking out the window and the UFO attack + have the default UFO already in the model display capsule +* Shape Detection: + * explain to the player how to scale printed objects + * set min and max scale limits on printed objects + * set 3d printer soft particles to avoid sharp edges when particles go inside the printer wall + + ## Credits **Samuel Saar**