Fuusion Use Case: Smart Data Migration to the Cloud
Mary Roberts
April 20, 2021

Smart Data Migration to the Cloud.

Any large and distributed business, whether it be a brick and mortar retail venture, car dealerships, energy production, mining, or manufacturing, your data is distributed widely. The larger the company, the more data is generated, typically over a wider area. Energy and mining companies have to spend a good deal of their resources on production and delivery, each with its own challenges and exploration. This means well sites or potential sites across the country, if not across the continent, or even the globe. Retail organizations develop partnerships to provide new revenue streams, whether it is a new clothing line, a new source of raw materials, or the latest gadget. This means small offices in remote locations or infrastructure to connect to newly acquired organizations.  

Whatever the case is, such companies save money by making the sites as lean as possible. Retail organizations try to leverage the existing people working on a newly acquired brand or create transitional teams to bring the new opportunity under their umbrella. Resource companies sub-contract at exploratory sites rather than sending existing employees to a distant location. This eliminates the need for salaried employees in a downturn and makes it easier to manage shorterterm operations. The downside to this approach is that there is less control with regards to the data generated – subcontracted service operators or newly acquired organizational assets may have their own way of doing things or may simply have different equipment (or a different configuration of their equipment) than the home office they report to. In short, different operators produce different data sets.  

This means that the unstructured (or differently structured at least) data from all of these edge locations needs to be processed to some degree to be used effectively. This can be done to a degree at the edge with Fuusion by deploying Fuusion Edge nodes at each site. Still, in most cases, companies want this data aggregated in one place or at least in regional hubs. Because of the large amounts of data generated daily by multiple distributed sites, small though they may be, this aggregation is increasingly happening in the cloud, often in a data lake. 

Fuusion can help with this process as well, whether edge processing is used or not. Because of the wide distribution of the data endpoints, there is almost guaranteed latency across even the bestconnected locations. Some will have limited connectivity and speeds due to rural limitations. Some will have intermittent connectivity due to satellite networks. That fairtrade coffee farm in Guatemala? Until Starbucks foots the bill, it is unlikely to have as much connectivity as the coffee shop it supplies to. There might be hundreds of smaller operations of this nature that report needed data to a central office. This means that to get the data to our AWS or Azure data lake, something needs to be done to optimize the delivery. 

Enter Ultrafast™. Fuusion’s Ultrafast allows faster data transfer over high latency or limited bandwidth connections than standard TCP. Fuusion also ensures that if transfers are interrupted, such as when a satellite connection goes down for the day, that the transfer resumes at the exact point of failure when connectivity is reestablished automatically. To maximize the available bandwidth, Fuusion Ultrafast can also throttle or open bandwidth on a schedule, ensuring that edge locations do not hog resources during peak hours, for example.  

With Fuusion deployed at edge locations, the automated transfer of raw or semiprocessed data (depending on your needs) can be made simple. Once a flow is defined, it can be saved as a template for deployment across an organization. Github integration allows for version control as well. This means that several iterations of a given flow can be stored. This can be handy if some edge locations require a similar process group but with varying degrees of complexity. 

But at the end of the day, it’s about getting the data to where it needs to be, without having to worry about whether it arrived or whether anything is missing. Fuusion automates the flow of data to the data lake (or any designated repository), data provenance, and the ability to playback a flow ensures the data arrives intact, and Ultrafast ensures it arrives in a timely fashion. Let Buurst help you get your data where it needs to be. 

Subscribe to Buurst Monthly Newsletter 

More from Buurst

Try SoftNAS at No Cost