Fantastic Pixel Castle, a fully remote, AAA first-party NetEase studio founded in 2023 by industry veteran Greg Street, is working on a highly anticipated fantasy MMORPG called Project Ghost
introduction01
The core members of the studio consist of industry veterans from Riot and Blizzard, who worked on World of Warcraft, Diablo 2, Age of Empires, League of Legends, Call of Duty, Halo and more.
Our engineers embedded themselves within the Service Availability team and leveraged their expertise in back-end development and cloud architecture to make the infrastructure ready for running on production scale.
We designed and introduced a new AWS network and infrastructure architecture including networking, routing and security, fully automated with Terraform for infrastructure-as-code, leveraging both AWS services as well as services running in Kubernetes with Karpenter integration.
We fully containerized all back-end services, as well as the Unreal Engine game server and testing client builds with a unified workflow that can be used both in local development as well as in deployment to the cloud.
Bespoke C++ CLI tools for environment setup and deployment
Customization of game server proxy to work in containers
We provisioned a production-grade observability platform, leveraging state-of-the-art technologies such as VictoriaMetrics and VictoriaLogs, fully integrated into the game and back-end services, providing valuable insight to game designers and developers, greatly improving development speed.
Unreal Engine subsystems for telemetry and log capture
Custom implementations for sending logs in C++ services
Infrastructure-level observability setup in Kubernetes
For building the services and spinning up new environments, we have built several internal APIs and contributed to a custom build system replacing Jenkins, now using the same build pipeline used for local development to standardize all workflows across different types of environments.
Project Ghost is using several backend services, all written from scratch in C++. During the containerization process, we modified the services to support running in containers: both locally as well as deployed in Kubernetes.
We designed a secure AWS architecture, including VPNs, permissions, security boundaries, implemented using Terraform. Our fully automated deployment setup also configures all Kubernetes clusters and other resources, no click-ops at all!
Project Ghost uses custom C++ CLI tools for setting up and managing all environments in order to be infrastrucure-agnostic. We seamlessly integrated Kubernetes support into this setup.
We built several systems to persist observability data from Unreal Engine as well as the backend C++ services.
Our Unreal Engine log collection implementation uses existing engine features and requires no game code changes. We added custom ImGUI debug menus to easily find the relevant logs for developers.
We developed several custom internal services running serverless in AWS. These internal services are used for managing builds, environments and content generation jobs.
We created a setup that allows for easy development of new services, with abstracted database operations as well as automated database migrations, so FPC's team can keep focused on building Project Ghost.
testemonial07
Their deep subject matter expertise unlocked us to use best practices in domains outside of the development areas which we're good at, and their energy and drive meant that everything fell into place quickly. Their engineers embedded on our teams and worked closely with us during development, leaving their completed work in a clean and maintainable state.”
Fantastic Pixel Castle - CTO