Integration documentation for: Network Drive with Azure Storage Mover

Below are listed DataMapper Integration Documentation for Network Drive with Azure Storage Mover 


Overview 

Description:

The integration of DataMapper with Azure Storage Mover facilitates the seamless migration and scanning of large volumes of files from network drives to Azure Blob Storage. This integration benefits users by providing a robust, secure, and efficient method for transferring data to the cloud for subsequent scanning and risk analysis. 


Purpose:

This integration is useful for organizations looking to migrate and analyze data stored on network drives. It solves the problems of manual data migration, potential data loss, and inefficiency by automating the process and ensuring data integrity and security during the transfer. 


Installation and Setup 

Prerequisites: 

  • An active Azure subscription. 
  • Azure Storage Mover service set up in the Azure portal. 
  • Administrator permissions for setting up the Azure Storage Mover and agents. 
  • DataMapper Network scanner purchased and configured. 

Step-by-Step Guide: 

  1. Azure Storage Mover Setup (done by a Safe Online technical ressource):
  • Log in to the Azure portal. 
  • Navigate to the Azure Storage Mover service. 
  • Create a new Storage Mover project and configure the necessary parameters (e.g., target Blob Storage account, network file share details). 
  • Deploy the Storage Mover agent on the network file server using the provided installer and setup instructions. 

  1. Configuring DataMapper (done by a Safe Online technical ressource): 
  • Access the DataMapper user interface. 
  • Navigate to the data locations settings and select "Network Drive Pro". 
  • Enter the connection string to the Blob Storage in the customer’s private tenant securely. 
  • Configure the scanning rules (e.g., folders to exclude from scanning). 

  1. Agent Deployment (done together with a Safe Online technical ressource): 
  • On the DataMapper UI, request the agent’s path and download rules. 
  • The agent responds with the location where it will upload files. 
  • Start the scan, and DataMapper stores the agent’s name, location, and the user(s) for whom the scan is being conducted. 

  1. Meeting for Setup Network Drive Pro scan: 

•Schedule a meeting with the operations team (Caroline or Sebastian) and Ranjith. 

•During the meeting, ensure the customer has the necessary Azure subscription and administrative permissions. 

•Guide the customer through setting up the Storage Mover and connecting it to DataMapper. 


Requirements to Install the Storage Mover Agents

The following requirements are needed to install the storage mover agents on the local machine/pc that has access to the Network Drive to scan the files from the DataMapper application.


1.  Hardware Requirements

Operating System – Windows 10 or more ( Not Home Edition)


2.  Download the Storage Mover Agent

Download the Storage mover agent in a .zip format from the following link

https://www.microsoft.com/en-us/download/details.aspx?id=104590

3.  Enable Hyper-V or VMware on Windows

  1. Open a PowerShell console as Administrator.
  2. Run the following command:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All

If the command couldn't be found, ensure you're running PowerShell as Administrator.

When the installation has been completed, reboot.

More information on “how to enable Hyper-V on windows” can be found here

https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v

or

https://learn.microsoft.com/en-us/azure/azure-arc/vmware-vsphere/browse-and-enable-vcenter-resources-in-azure


Article from Microsoft:

The Azure Storage Mover service utilizes agents to perform the migration jobs you configure in the service. An agent is a virtual machine-based migration appliance that runs on a virtualization host. Ideally, your virtualization host is located as near as possible to the source storage to be migrated. Storage Mover can support multiple agents.

Because an agent is essentially a migration appliance, you interact with it through an agent-local administrative shell. The shell limits the operations you can perform on this machine, though network configuration and troubleshooting tasks are accessible.

Use of the agent in migrations is managed through Azure. Both Azure PowerShell and CLI are supported, and graphical interaction is available within the Azure portal. The agent is made available as a disk image compatible with either new Windows Hyper-V or VMware virtual machines (VMs).


At present, Windows Hyper-V and VMware are the only supported virtualization environments for your agent VM. Other virtualization environments have not been tested and are not supported.


*Video of process in the above step-by-step guide is coming soon to this page.


Functionality 

Core Features: 

  • Automated data transfer from network drives to Azure Blob Storage. 
  • Secure and efficient handling of large file volumes. 
  • Batch processing and meta-data preservation. 

How it Works: 

  • The Mover agent transfers files from the network drive to a specified Blob Storage location. 
  • DataMapper ScanService is notified when the blobs are available. 
  • Files are processed, analyzed, and results are saved in the DataMapper database. 

Use Cases: 

  • Migrating corporate data from legacy network storage to the cloud. 
  • Conducting risk assessments and compliance checks on large datasets. 
  • Streamlining data management during organizational transitions or mergers. 

Usage Instructions 

Daily Operations: 

  • Monitor the migration progress through the DataMapper interface. 
  • Schedule scans to run during off-peak hours to minimize disruption. 
  • Regularly check the scan results and address any identified risks. 

Best Practices: 

  • Ensure the network drive is accessible and has minimal restrictions for the Mover agent. 
  • Regularly update the agent and integration settings to maintain optimal performance. 
  • Keep a backup of critical data before initiating large migrations. 

Troubleshooting: 

  • Connection Errors: Ensure the connection string and network permissions are correctly configured. 
  • Slow Transfer Rates: Check network bandwidth and adjust the batch size if necessary. 
  • Unsupported Files: Verify that file types and sizes conform to Azure Blob Storage limits. 

Limitations 

Known Limitations: 

  • Complex setup requiring knowledge of Azure and VM configurations. 
  • Only supports network file shares (local drives need to be shared as network drives). 

Compatibility Issues: 

  • Potential conflicts with existing security protocols on the network drive. 

Performance Considerations: 

  • High volume transfers may impact network performance; schedule migrations during low-usage periods. 

Security and Compliance 

Data Security: 

  • Data is encrypted during transfer to Azure Blob Storage. 
  • Access to Blob Storage is controlled via secure connection strings and permissions. 

Compliance: 

  • Ensure compliance with relevant data protection regulations (e.g., GDPR, HIPAA) by configuring appropriate access controls and data handling policies. 

Costs 

Additional Costs: 

  • Azure Storage costs based on data volume and storage duration. 
  • Potential charges for support and custom setup assistance. 

Licensing: 

  • No additional licensing required for Azure Storage Mover, but Azure Blob Storage usage will incur costs. 

Cost Management Tips: 

  • Use reserved capacity options in Azure to lower storage costs. 
  • Regularly delete processed files to minimize storage usage. 

Updates and Maintenance 

Update Process: 

  • Azure Storage Mover updates are managed through the Azure portal. 
  • DataMapper updates are handled via the DataMapper interface. 

Maintenance Requirements: 

  • Periodic checks to ensure agents are operational and properly configured. 
  • Regular review of scan results and data integrity. 

Support: 

  • Technical support is available through the DataMapper and Azure support portals. 

FAQs 

Common Questions: 

  • How do I reset the integration? Follow the setup guide to reconfigure the connection and settings. 
  • How can I monitor the transfer status? Use the monitoring tools provided in the DataMapper and Azure Storage Mover interfaces. 

Further Resources: 


How to get started with documenting paths in Network Drive:

Feel free to copy the view below and add to your preferred sheet application and share with your IT manager and other relevant departments (HR or other admin) for an easy overview to see what drives need to be scanned from the network drive - and who needs to be the responsible for cleaning up*:

*The document is also meant to be shared during one or two meetings with Safe Online for the setup process as we would need the Absolut Path and Path Size


Get file here: Path excel document


This documentation provides a comprehensive guide to integrating DataMapper with Azure Storage Mover, ensuring a seamless and efficient data migration and scanning process. 


Questions? Please do reach out to us on support@safeonline.dk where we'd be happy to answer any questions related to DataMapper.

Still need help? Contact Us Contact Us