Architecture and Technology Stack

1. Frontend (Client-Side)

• Three.js: Used for 3D rendering and creating a visually rich environment where users can explore the virtual space in a browser.
• JavaScript (WebSockets): Ensures real-time synchronization between users and the server, transmitting data on user positions, voice data, and objects.

2. Backend (Server-Side)

• Python (FastAPI): Manages API requests and enables asynchronous data handling, essential for real-time operations.
• Matrix: Powers text communication, providing a secure and scalable real-time messaging platform hosted on qure.space.
• PostgreSQL (JSONB): Stores user data and 3D object information with a flexible JSONB structure that simplifies handling dynamic object parameters.

3. AI and Machine Learning

• OpenAI API: Supports the personal assistant, offering recommendations and guidance within the virtual world.
• Potential for Custom AI Models: Future plans include gesture recognition and voice analysis models to enhance the inclusiveness of the platform.

4. Infrastructure and Hosting

• Docker: Containerizes all components, including the web server, Matrix server, and database, ensuring flexibility and scalability.
• VPS on Docker: The full stack is deployed on a Virtual Private Server, simplifying deployment and allowing for scaling as the load increases.

Key Features and Implementation

1. Open Virtual World

• Created using 3D landscapes, allowing users to freely navigate the space. User positions and object states are synchronized via WebSocket to support continuous interaction.

2. Barrier-Free Communication Environment