How to Build and Own Your AI Stack: Insights from GMI Cloud’s Virtual Event with Jordan Nanos from HPE

In a recent GMI Cloud virtual event, industry experts shared pivotal insights about building and managing AI infrastructure. The event featured presentations and discussions from Alex Yeh, CEO of GMI Cloud; Jordan Nanos, Machine Learning Architect at Hewlett Packard Enterprise; and Yujing Qian, Software Engineer Manager at GMI Cloud. This article provides a detailed overview and summary of the key points discussed in the webinar.

The entire webinar can be watched on YouTube here:

Webinar Highlights

Vision and Strategy

Alex Yeh began by highlighting GMI Cloud’s vision to enable users to build AI applications effortlessly, similar to how Shopify democratizes e-commerce. “We want to empower anyone to build AI applications with one click,” Alex emphasized. He discussed the company’s goal to be the “AI TSMC,” supporting the entire stack of building AI applications on their cloud platform. This involves providing not only raw compute power but also all necessary tools and resources.

Infrastructure Focus

A key aspect of GMI Cloud’s strategy is controlling their hardware to ensure high availability, reliability, and optimized performance. Alex noted, “We control our nodes, which ensures that our customers always have the highest availability and reliability.”

Data Centers and Global Reach

Currently, GMI Cloud operates multiple data centers across the Asia-Pacific region, with plans to expand to 30 locations globally by the end of next year. These data centers are strategically placed in densely populated regions to minimize latency and optimize performance. Alex highlighted, “We have three data centers up and running and two more coming online by mid-August.”

Technical Capabilities

GMI Cloud provides access to top-tier GPU hardware, including the NVIDIA H100, and an in-house designed software stack that simplifies large-scale deployment. The company also offers a cluster engine layer, this includes multi-tenant Kubernetes for expert-level control and customization for container orchestration, essential for both training and inference workloads.

Service Models

To cater to different customer needs, GMI Cloud offers two major service models: On-Demand and Reserved. The On-Demand model is suitable for startups and researchers with unpredictable workloads, while the Reserved model is ideal for large enterprises with stable, long-term requirements. This flexible and predictable pricing structure ensures that various operational scenarios are efficiently managed.

Opening Presentation

Alex Yeh (CEO of GMI Cloud)

Alex emphasized GMI Cloud’s goal to support the entire stack of building AI applications, drawing on the company’s strong roots in the industry and the extensive experience of its core team from Google and OEM backgrounds in Taiwan. He stressed the importance of strategic data center locations, particularly in the Asia-Pacific region, to minimize latency and optimize performance. “Our goal is to have 30 data centers by the end of next year, providing the broadest serving GPU fleets across Asia and eventually expanding globally,” Alex explained.

Fireside Chat: Key Takeaways

During the fireside chat, Alex Yeh and Jordan Nanos delved into scalability and efficiency challenges, with Alex explaining the importance of infrastructure management. “We aim to provide a robust infrastructure that simplifies the complexity of managing AI systems,” he said. Jordan added, “It’s about ensuring reliability and performance through strategic hardware control.”

The discussion also covered data privacy and security. Jordan elaborated on the importance of securing data at multiple layers and leveraging the open-source community for continuous innovation while maintaining compliance. “The open-source ecosystem is vibrant and essential for AI advancement, but we must ensure data integrity and security,” he emphasized.

Jordan also discussed the challenges of managing AI infrastructure, emphasizing the complexity and costs involved. He highlighted the need for robust operations to ensure high uptime and reliability, saying, “Managing hardware is incredibly expensive and complex. Our goal is to simplify these steps for our customers.” Jordan also addressed security concerns, detailing the three layers of security: data privacy, model security, and application compliance. “Ensuring data privacy at multiple layers, from data ingestion to model deployment, is crucial,” he noted.

Looking to the future, Alex and Jordan discussed the short-term disruptions and long-term innovations expected in the AI industry. Alex mentioned, “The advertisement and commerce sectors will see significant changes through tailored AI solutions.” He also highlighted the potential for AI to revolutionize biotech, material science, and other fields, saying, “AI agents will support various enterprise functions, accelerating innovation in multiple industries.”

Alex emphasized GMI Cloud’s goal to support the entire stack of building AI applications, drawing on the company’s strong roots in the industry and the extensive experience of its core team from Google and OEM backgrounds in Taiwan. He stressed the importance of strategic data center locations, particularly in the Asia-Pacific region, to minimize latency and optimize performance. “Our goal is to have 30 data centers by the end of next year, providing the broadest serving GPU fleets across Asia and eventually expanding globally,” Alex explained.

Demo Session: Deploying Llama 3 on GMI Cloud

Yujing Qian’s demo session provided a practical demonstration of GMI Cloud’s platform capabilities. He showcased how the platform allows for the seamless deployment of Llama 3, highlighting its user-friendly interface and flexible container solutions. “Our platform’s flexibility enables quick setup and efficient AI model deployment,” Yujing demonstrated, emphasizing the robust performance of GMI Cloud’s GPUs in handling extensive AI workloads.

Conclusion

The GMI Cloud virtual event underscored the company’s commitment to empowering AI innovation through robust infrastructure, strategic hardware control, and a user-centric platform. By addressing key challenges in scalability, efficiency, and security, GMI Cloud positions itself as a leader in the AI infrastructure space, ready to support enterprises and developers in building the future of AI. A big thanks to Jordan Nanos and our partners at HPE for joining the event and providing valuable insights into the industry.

Stay tuned for more insights and updates from GMI Cloud. Follow us on LinkedIn and YouTube for the latest developments in the AI industry. Feel free to reach out to our sales team (sales@gmicloud.ai) with any questions or comments.