Migrating From Self-Managed File Servers to Amazon FSx for Windows File Server

Photo by Joshua Woroniecki on Unsplash

Managing an on-premise Windows file server within a VMware infrastructure with external storage in SAN is the norm for many organizations. However, the challenges, risks, and limitations of this approach have become increasingly evident in modern hybrid IT environments. In this article, we will explore the challenges SSENSE faced with self-managed Windows file servers and the migration process to Amazon FSx for Windows File Server, a fully managed SMB-compatible file server solution. We will discuss the benefits of this migration, the execution process, and the lessons learned along the way.

Challenges and Risks of On-Premise File Server Management

At SSENSE, the Windows file server was initially meant for users’ home directories. But as time went by, the file server organically evolved into an essential component of the infrastructure that was hosting critical operational and reporting data for all business units.

With the SSENSE cloud-first approach, new applications and services continued to be launched in AWS and these needed to connect to the file server over the long-distance network. In order to improve the experience for these clients, self-managed Windows file server replicas were created in multiple AWS environments (1 per AWS region used), but this added extra cost and maintenance. This also brought the extra complexity of having to deal with file server replication and sync issues.

As this shared infrastructure became indispensable for the business, its availability and reliability became a top priority. Maintaining a self-managed Windows file server presented numerous challenges and risks that were not easy to solve:

• Scalability Limitations: On-premise file servers often face scalability limitations, making it difficult to accommodate growing data storage needs. Capacity planning, procurement, and installation of additional storage require careful planning and timely execution.
• Maintenance Burdens: Hardware and software maintenance is a continuous challenge, requiring regular updates, repairs, and replacements that can disrupt operations. Because it’s a critical component in the IT infrastructure, file server maintenance had to be scheduled in advance with approvals from business stakeholders.
• Data Security Concerns: Ensuring data security is a constant priority. Robust security measures are essential to protect against data breaches. As a critical data store for the enterprise, administrators need to be on top of managing and monitoring access to data.
• Custom Backup Solutions: Data availability and recoverability in a disaster is a constant concern. Frequently, administrators roll out 3rd party backup solutions to address data loss risks, but a centralized file server with data in the double-digit terabyte scale becomes difficult to reliably backup and restore quickly.

What is Amazon FSx for Windows File Server?

Amazon FSx for Windows File Server provides fully managed SMB shared storage built on Windows Server and delivers a wide range of data access, data management, and administrative capabilities. Its key features include:

• High Availability: It offers a highly available multi-AZ setup and resilient file storage solution, reducing downtime and ensuring data accessibility.
• Scalability: Amazon FSx scales with our needs, eliminating scalability limitations associated with on-premise solutions. In addition, FSx supports data duplication and compression to optimize the storage cost.
• Automated Backups: Automated backups simplify data protection and recovery, reducing administrative overhead and increasing the reliability of the backup and recovery procedures.
• Simplified Management: Amazon FSx removes the hardware and software management and maintenance burden from the customer’s plate.
• Robust Security: Amazon FSx automatically encrypts data at rest using AWS KSM keys. Data in transit is encrypted using SMB Kerberos session keys. Being built on top of the Windows Server, Amazon FSx supports native Windows SMB file and folder access management features.
• Integration with Other AWS Services: Amazon FSx natively integrates with AWS IAM, AWS Directory Service for Microsoft Active Directory, Amazon WorkSpaces, AWS Key Management Service, and AWS CloudTrail to capture audit trails of the activities.

The diagram above shows typical Amazon FSx for Windows File Server deployment in a production environment, with clients accessing the shares from multiple locations.

It’s important to keep in mind is that the total cluster cost is the combination of cluster setup (single-AZ or multi-AZ), provisioned throughput capacity (throughput to the storage in MB/s), provisioned storage capacity (HDD vs SSD storage, the amount of storage), and backup costs. I recommend using the AWS calculator to estimate the monthly price tag of your new FSx file system: https://calculator.aws/#/addService.

The Migration Process

At SSENSE, migrating to Amazon FSx for Windows File Server involved a number of critical steps.

• Preparation: Before migration, ensure that the on-premise file server is well-documented and that user access permissions are clearly defined.
• Note: This is a good opportunity to audit the existing Active Directory principals and to validate the accuracy of NTFS ACLs.
• Creation of FSx File System: Launch Amazon FSx for Windows File Server file system keeping in mind the gotchas (see the “Lessons Learned” section below).
• Data Transfer: Use AWS DataSync for efficient data transfer to Amazon FSx file system (https://docs.aws.amazon.com/fsx/latest/WindowsGuide/migrate-files-to-fsx-datasync.html). This tool helps synchronize data while minimizing data transfer time and risk.
• Note: You might need to run AWS DataSync periodically to sync file servers before the eventual switchover.
• DFS Namespace Configuration: Set up DFS namespaces to ensure a seamless transition for users. This step is crucial for maintaining accessibility.
• Network Drive Changes: Adjust network share configurations or SMB endpoints to point them to DFS namespaces.
• User Training: Educate users on the migration and any changes in file server access in order to minimize disruptions.
• Maintenance Window: Book a maintenance window when no clients are writing to the file server and perform one final data synchronization using AWS DataSync.
• Decommissioning the On-Premise Server: Perform DFS namespace changes in order to ensure that on-premise file server is not pointed anymore. Safely decommission the on-premise server, once the migration is complete and data is verified.
• Testing and Monitoring: Continuously monitor the Amazon FSx environment to ensure optimal performance and data integrity.

The above diagram shows how file systems are accessed by the clients once the migration is complete.

At SSENSE, we fully migrated data from the self-managed file server to the FSx file system within a week in mid-January 2023. By deprecating file servers (both on-premise and duplicate file servers in AWS), we reduced our AWS bill for file servers by approximately 50% and, at the same time, freed up resources in our on-premise VMWare infrastructure. Since the migration, SSENSE hasn’t had a single outage with the file server.

It is worth highlighting the exemplary support SSENSE received from the AWS support and account teams in ensuring the smooth migration. The AWS account team is an invaluable resource when adopting any new AWS service and their guidance is essential in avoiding pitfalls.

Lessons Learned

When working with the Amazon FSx for Windows File Server file system, there are a few things to keep in mind:

• Each Amazon FSx for Windows File Server file system is limited to 64TB in size, but administrators can use Microsoft DFS technology to logically combine multiple clusters.
• Windows Volume Shadow Copy (VSS) is disabled by default in the newly created FSx file system, it’s important to activate VSS for your production systems to enable file and folder versioning.
• Certain cluster configurations (storage type, deployment type, chosen VPC and subnets, Active Directory type & KMS keys) cannot be modified post-creation, therefore make sure these choices are deliberately made with long-term maintainability in mind.
• AWS DataSync is an indispensable tool for various migrations (for initial migration to Amazon FSx as well as any subsequent migrations to the new file systems, such as cross-region / cross-account replication or replication to a new file system with a different storage type).
• If you have on-premise workloads or users that need to connect to FSx, it is recommended to either use high-throughput / low-latency network connectivity to AWS, such as AWS Direct Connect or to provision Amazon FSx File Gateway appliance inside your on-premise VMWare infrastructure to be used as a local cache.
• Monitor your FSx file systems and pay close attention to CloudWatch metrics for throughput or storage capacity. It is possible that the initial cluster configuration was either suboptimal or overly ambitious.

Conclusion

Migrating from a self-managed Windows file server to Amazon FSx for Windows File Server is a strategic move that may offer numerous benefits, including higher reliability, scalability, cost savings, and simplified management. By understanding the challenges, embracing the advantages, and following best practices during migration, organizations can ensure a smooth transition to a more efficient and flexible file server solution with Amazon FSx for Windows File Server.

Editorial reviews by Catherine Heim & Mario Bittencourt

Want to work with us? Click here to see all open positions at SSENSE!