Overcoming the on-premise bottleneck: the future of HPC and simulation in the cloud

In the world of digital product development, the shift left simulation approach is a competitive necessity. Simulation should not serve as the final validation at the end of the design process. Instead, perform simulation during the conceptual or requirements phases to enable earlier defect detection and verification. However, this shift comes with a significant side effect: a massive explosion in simulation data volume and the frequency of runs puts enormous strain on your existing on-premise infrastructure.

The on-premise bottleneck: simulation data overflow and finite compute

The surge in data volume comes from increasing use of high-fidelity simulation models in early design stages, plus more frequent runs of design of experiments (DOE) and optimization studies.

• Traditional storage can’t handle this data explosion, leaving engineers stuck in file chasing, copy reconciliation, and versioning chaos - rather than focusing on value-added work. This is called the digital landfill problem. Valuable data becomes inaccessible, turning a potential asset into a storage and management burden.
• The finite resource trap: on-premise HPC clusters are limited. When a high-priority job saturates the cluster, everything else stops. We need to move from asking "can I run this?" to "how many thousand designs can I explore in the next few hours?"
• Distributed user base: engineering teams are global, with members spread across different continents. Centralized systems create problems like high latency, slow data transfers, and collaboration barriers. There is a clear need for secure, high-performance access to the same tools and data for everyone, everywhere.

ESTECO VOLTA digital engineering platform evolution: move to cloud-native

To address the challenges of data volume and compute bottlenecks, we recognized that our own technology had to become as flexible as the design processes it supports. And that's why we undertook this fundamental evolution of our technology stack, re-engineering our VOLTA digital engineering platform from a single monolithic application into a collection of independent containerized services.
This architectural shift allows for horizontal scalability and improved resiliency. Whether you need the portability of a standalone installation or the power of a full-scale cloud deployment, the core technology remains the same — only the framework changes to suit business needs.

The architectural shift

This architectural shift allows for horizontal scalability and improved resiliency. Whether you need the portability of a standalone installation or the power of a full-scale cloud deployment, the core technology remains the same — only the framework changes to suit business needs. By adopting a Kubernetes-based architecture, we’ve unlocked two critical advantages for our users:

• on-demand scalability: we can now scale individual services independently based on load, rather than being forced to scale the entire application;
• enhanced resiliency: the modular nature of microservices means the system is more robust, ensuring that simulation workflows remain stable even under heavy demand.

VOLTA deployment options

While we continue to support our traditional deployment models (VOLTA Portable and VOLTA Standalone) the new cloud-native deployment is a game-changer.
It is important to distinguish between running in the cloud and being cloud-native. While traditional versions can be hosted on cloud infrastructure, only the new Kubernetes-based deployment fully leverages horizontal scalability and automated orchestration.

Tailored cloud deployment: customer’s tenant

Every cloud vendor offers the capability to run a Kubernetes cluster, with Elastic Kubernetes Services (EKS) being the Amazon Web Services (AWS) version of it. VOLTA Kubernetes deployment is completely vendor-agnostic. You can even decide to deploy this on your on-premise, possibly classified environment if you need it, but you can also install and use it on your private cloud environment regardless of the vendor that you are choosing. Deploying your stack within your own cloud environment allows you to:

• maintain full control: your IT team retains complete authority over data, security, and networking;
• get zero data leakage: your proprietary data never leaves your environment;
• handle legacy integration: this approach makes it easier to bridge the gap between modern cloud services and local, complex simulation tools that may not yet be cloud-ready.

Why move simulation and SPDM to the cloud: AWS’ viewpoint

At our recent ESTECO Users' Meeting North America, Sandeep Sovani, Global GTM Head of Engineering Simulation HPC Amazon Web Services, illustrated eight core reasons for shifting computer-aided engineering (CAE) and simulation process and data management (SPDM) to the cloud:

1. massive capacity: pay-per-use access to thousands of cores on demand;
2. pay-per-use: on-premise infrastructure is rigid, cloud is elastic;
3. elasticity: switch between GPUs and CPUs as your workload requires;
4. latest technology: access the newest processors without waiting for a 5-year hardware refresh cycle;
5. global reach: deploy infrastructure near your distributed teams in minutes, ensuring that every engineer – regardless of location – has the same high-performance experience with minimal latency;
6. disaster resilience: by building on redundant, multi-region architectures, the cloud reduces the risk of downtime and data loss to nearly zero;
7. cost efficiency: economies of scale often make cloud cheaper than on-premise when accounting for idle time;
8. sustainability: sharing massive, optimized infrastructure is inherently greener than running individual, under-utilized server rooms.

Cloud is not just the hardware, cloud is the microservices

For decades, engineering software was built as a monolith – one giant, interconnected codebase where if one part failed or needed an update, the whole system had to be taken down. The true power of the cloud is the microservices.
A simulation process and data management (SPDM) platform like VOLTA integrates these microservices into a cohesive system. It moves your organization from having storage (where data dies when an engineer leaves) to having data (where information is organized, searchable, and integrated into the digital thread).
By re-engineering VOLTA into independent, containerized microservices, the system allows you to:

• assemble specialized microservices to build a highly complex, customized HPC environment in a fraction of the time it would take with traditional software;
• integrate with the microservices already living on the AWS cloud – from AI tools to advanced data analytics – because these services speak a universal language (APIs).

By using microservices as building blocks, an HPC CAE system can be deployed and made available to a variety of different users around the world in the exact same way. Whether your team is in a corporate headquarters or working remotely across the globe, the platform delivers the same high-performance experience, regardless of the local hardware they are using.

The three pillars of cloud simulation: front-end, compute and data

There are three fundamental pillars to understand how a modern HPC CAE system operates: front-end, compute, and data. Each pillar leverages specific cloud technologies to solve the traditional issues of local simulation.

1. Front-end

• Users can access the platform from anywhere — a thin client, a laptop, or even a tablet. They have access to hundreds of virtual desktop configurations tailored to their specific needs.
• Using the AWS DCV protocol (specifically built for engineering applications), the system transfers only the pixels to the user's screen. The massive simulation files never leave the secure cloud environment, eliminating the lag associated with downloading large datasets.
• Administrators get granular control over budgets. Users can see exactly how much a specific job costs in real-time.

2. Compute

• We’ve moved away from manual hardware setup, we now build infrastructure with code. This allows to stand up entire compute clusters using automated scripts.
• On-premise, the risk is to be stuck with servers bought years ago. In the cloud, there are hundreds of different compute instances at disposal.
• Whether a solver needs high memory, specific processor types, or fast local storage, it’s possible to match each specific job card to the most optimal server type. This means no longer limitation to a "one size fits all" hardware setup.

3. Data

• While storage was once just a place to park files, it has now become the most critical asset for engineering.
• Technology has moved toward AI and machine learning in simulation, how data is organized matters more than how much of it you have. This pillar is the bridge to smart simulation.

Upcoming requirement for AI and integration with digital thread

Turning storage into an asset

Dumping files into a folder is storage, organizing those files into a searchable, versioned, and context-rich environment is data. In many traditional setups, only the engineer who ran the simulation knows what the files mean. If that engineer leaves, that knowledge is effectively lost. AI and machine learning require well-organized data and clean pipelines. By using an SPDM tool like VOLTA digital engineering platform on the cloud, you move away from the digital landfill and start building a structured library that can actually train future AI models to accelerate design cycles.

The digital thread is the seamless flow of information across a product’s entire lifecycle — from initial CAD designs and PLM systems to ERP, MES, and even CRM. For too long, simulation data has lived in a silo. By centralizing simulation process and data management in a cloud-native environment, VOLTA acts as a bridge. Managed data is what allows simulation results to inform manufacturing or influence business decisions in the ERP. VOLTA ensures that simulation isn't just a final check, but a living part of the entire enterprise pipeline.

Conclusion

The transition to cloud-native simulation is a strategic shift that aligns engineering capabilities with business goals.

Key takeaways:

• Shift left scalable tech: don't let infrastructure be the bottleneck for early-stage simulation.
• Modern simulation also relies on APIs and microservices: cloud-native is key – leverage microservices and APIs for a future-proof architecture.
• Security meets scalability: deploying in a private cloud tenant ensures keeping control of data while gaining the power of the cloud.
• Structured data: this means being AI ready. Move from unorganized file storage to structured data pipelines managed by SPDM tools.