3D era is coming, everybody will be able to create and share games, film, virtual worlds, and simulations. But, it may not work the way you want it.
Hi Readers
Due to my work, I’ve spent thousands of hours on 3D creations in previous years. Although I don’t publish games, I understand how 3D creations work, from story writing to prototyping and production in a 3D engine. As we all know, 3D creation is unlike photo or video creation; it requires a high learning cost and professional knowledge and is usually done by a team.
To make 3D creation accessible to everyone, we have to solve the problem of the entire pipeline: generating a story, turning the story into game chapters, creating game assets, building and coding the game logic, map, avatar, NPCs, abilities, and more.
AI. Yes, AI has been considered the best way to solve this problem. We have seen some interesting products on the market to solve different problems along the way (I will list them later). But in my opinion, they are heading in the wrong direction.
Coincidentally, I gained inspiration from the office collaboration work platform – Lark. My recent work has transitioned to Lark, an all-in-one work platform. From group messaging, email, documents, video meetings, and calendars, to team wiki knowledge bases and AI assistants, everything can now be accomplished within a single platform. This serves as a compelling example of the integration of AI into the workplace.
This transition prompted me to ponder: AI has already transformed our learning, work, and content creation. Why does 3D content creation still remain complex and somewhat outdated? Could the AI + platform model provide insights and inspiration for the 3D content industry?
So before discussing the future of the 3D engine, let’s take a look at how AI empowers the workspace.
3rd Generation AI workspace
The first-generation workspace is Microsoft Office, where every file is independent, and team members find it hard to collaborate with each other.
The second generation evolved to Google Docs, where team members can access the same file and edit together, but there are issues with file management and knowledge management.
The third generation includes Notion, Lark Doc, etc. The files are based on an integrated knowledge base with robust version control and AI integration.
- Information find people, not people find information
We’ve been using tools Word, Excel, and PowerPoint for decades. Excel is great for finding data and figuring out problems. But can everyone really use all of Excel’s features? (Think about what the World Excel Championship champion two weeks ago) With AI, we can now easily spot the answer behind the data without needing to be Excel experts. AI + data helps us solve problems by getting information directly, without the need for complicated Excel formulas. That is how AI could change the way of getting information. - Proactive and Real-time Information Retrieval
Every team has its own way of welcoming new members, telling them about the company’s culture, document rules, decision-making steps, and more. But new folks might forget things, and if the team’s documents are messy or outdated, it can be hard to find what they need. With an AI assistant, all the team’s latest documents are in one place. Newcomers can just ask the AI if they have questions, making it easier to get company info and standards. This makes it simpler to find information and lowers the cost of team communication. - Personalization – Tailored Workspaces for Everyone
In Word and Excel, everyone sees the same layout, even though we use different features. A good workspace should be like a platform that fits each person’s needs. AI can make the next-gen recommendation engine – not for content but for functions. It can understand what you’re working on and suggest the support and features you need, saving you from a bunch of steps. For example, if a team member asks me for a user profile analysis of last year’s active product users, when I open my workspace, the user behavior data table and analysis template are already there for me.
Will 3D Engines Follow Suit?
There is no all-in-one 3D-dev suite yet
Before the 2010s, most studios created their own internal engines for game development. Nowadays, almost all games are crafted using third-party engines, except some major AAA games that still opt for in-house development (e.g., EA’s Frostbite, Infinity Ward Engine, etc.).
Today, we call them 3D Creation Engines because they’re not just for games; they’re for any virtual simulation. However, these engines mainly tackle issues during the “production stage.” Game development teams still need other tools for different steps, like Google Docs, Slack, Trello, Photoshop, Blender, 3Ds Max, and more.
Potential AI 3D Creation Tools
We have seen some AI tools are trying to solve the 3D creation along the pipeline. Here are some I have spotted
[Story Crafting] Companies like Hidden Door and Storycraft are pioneering AI-driven story-driven games, offering players constant streams of unique, personalized experiences. For instance, game objectives might change based on a player’s Bartle type, broadening the potential audience for each game.
[Game Art] In the realm of art, AI is making waves. Midjourney, DALL-E, Stable Diffusion, and Runway ML are leading the way in generating game art. However, achieving optimal results may require some practice with prompts and familiarity with each model’s temperament.
[3D Models] While text-to-3D modeling is still in its early stages, interesting products are emerging. Tools like DreamFusion can generate 3D models from text inputs, expanding on Google’s 2021 unveiling of Dream Fields. Point-E works similarly to OpenAI’s image generation tool DALL-E; you can describe something, and it generates a 3D model. Meshy aids in turning text or images into 3D or lets you upload a 3D model for AI to add textures.
[In-game Social] Enhancing in-game social and NPC interactions is possible with tools like , convai, and Replica Studios recently showcased a demo featuring a modified version of the Matrix Awakens game, allowing users to converse with NPCs using their own microphone.
I can’t cover them all, but it’s evident that 3D-related AI generation tools are still in their early stages, and we need more time to optimize both the models and user experience. The idea of using over 10 AI tools to create a game is hard to imagine.
AI 3D-dev Suite Will Look Like This
- All-in-One Game Development Information Flow
Similar to AI workspaces, game development info will be more integrated. From brainstorming ideas to the game story, numbers, game design docs, and later, art guidance. No more scattered files on Slack or Google Docs. AI will bring everything together, preventing info from getting lost in big teams or between departments. - AI Generates Over 50% of Framework Content
Whether it’s RPGs, action games, simulations, survival games – the fundamental logic underlying each game is the same. The variations mostly come from art, style, game pace, and other elements. AI models trained on games can generate 50-70% of the basic modules based on the creator’s needs. This allows creators to focus more on crafting the core creative elements. - Easier Creation, Simplified Steps
When the complex foundation of game development is automated, 3D creators can focus more on creativity than technical skills. It means we won’t need as many artists or animators but will require more creative planners. This change makes the process less reliant on a lot of manual work, making it more about creativity. Anyone can turn their ideas into playable 3D content, just like quickly editing a short video with ‘CapCut.’
However, Unity and Unreal won’t be replaced, just like Photoshop remains the most powerful tool for image creation. Even with AI, they still cater to professional users and may not be suitable for the broader consumer market, as the dynamics in the consumer market operate under entirely different rules.
But they might not be the go-to 3D creation tools. The rise of UGC platforms and their creation tools will make game creation easier for non-developers.
For instance, back in 2000, Maxis released modding toolkits for The Sims even before the game itself. This allowed players to make personalized in-game assets, making the game bigger and more unique. These toolkits and templates made it easier for players to start creating. Similarly, UGC services like Roblox Studio have made game creation more accessible, creating a cycle where more people are encouraged to make games.
Therefore, I believe the next AI 3D creation tools won’t be standalone; they’ll come with a user-generated content (UGC) platform, similar to the relationship between TikTok and CapCut. When the general users develop a demand for 3D content creation, AI 3D-dev suites become meaningful.
Imagine a 3D version of TikTok. With a platform where everyone creates game stories, we’ll need an easy-to-use AI 3D tool that turns ideas into playable 3D games in minutes.
Comments? Any thoughts to share?
If you liked this post, please follow me on Twitter. You can also find more essays here.