Cloud-savvy, price-aware decision-makers are asking for more than a tool that makes it easy to integrate their cloud storage and databases into their centralized data warehouses. They also care about the total cost of ownership (TCO) — including Microsoft Azure, Google Cloud Platform (GCP) and Amazon Web Services (AWS) data egress pricing — as well as ensuring the security of potentially sensitive data.

It’s no surprise that moving storage from data centers into a cloud-based storage environment can result in substantial cost savings. Once they’re convinced a storage solution will work, the next big question decision-makers will ask is: “How much are data transfer costs?”

For uploading data to the cloud, the data transfer cost is generally low or zero. Cloud storage providers don’t want to put any barriers between you and your utilization of their storage services. But once the data is in the cloud storage system, you may need to pay to get it back out and that’s where data egress charges and best practices come into play.

What is data egress?

Data egress, in the context of cloud storage, refers to data that is transferred outside the boundaries of a network — i.e., when you export data out of a cloud provider. You may need to do this for disaster recovery purposes, to transfer data to another storage provider, to move data from an application into a data warehouse or for other reasons.

‍

‍

Data egress vs data ingress

Data ingress is the opposite of data egress. It refers to when data is imported into a cloud provider from an external source.

‍

As stated earlier, cloud storage providers want to make it as easy as possible for you to use their services. So data ingress is typically free and fairly straightforward. But data egress comes with charges and issues that businesses must be aware of to transfer their data out of a cloud storage provider properly.

Data egress challenges

When transferring data out of a cloud storage provider, organizations must be aware of data egress costs as well as external and internal threats.

Data egress costs

Data egress costs are fairly low. But when managing and transferring large quantities of data, those costs can add up. Organizations that want to keep expenses low must be aware of these costs and make smart financial decisions regarding their data.

Egress charges typically average around a dime per gigabyte (GB), with exponentially scaled pricing that makes it cheaper to export more data per GB after a cloud-specified terabyte (TB) cliff. For example, a cloud storage provider may charge you one price per GB for exporting the first 10 GB, then lower those prices for the next 20 GB. Let’s take a look at the actual egress costs for AWS, GCP and Azure (these figures were last updated in November 2020).

External threats

Organizations must take care to protect their data from external cyber threats that put the security of their data at risk. In-transit data can be intercepted by hackers using malicious software, social engineering and other data exfiltration techniques.

The global cost for data security breaches is $4.35 million on average. They can also cause reputational harm, expose sensitive data that organizations want to keep private and even open them up to lawsuits for mishandling client information.

Internal threats

Organizations must also watch out for internal threats. Internal threats occur when an employee within an organization obtains access to sensitive data and either intentionally or accidentally leaks it. Whether an internal leak is malicious or not, organizations can still face financial and reputational repercussions. 

Most internal threats aren't malicious — they typically occur because employees aren't aware of data management policies and companies don't take proper steps to inform employees and/or prevent unauthorized people and devices from accessing sensitive data. By putting policies and protections in place, companies can avoid these accidents and make it easier to identify malicious actors in the case of intentional data leakages.

6 data egress management best practices

There are certain best practices that you can follow to optimize data egress and avoid the issues described above.

‍

‍

Let’s take a closer look.

1. Use egress filtering

Egress filtering is the process of monitoring egress traffic for malicious activity. If malicious activity is detected, you can stop the data transfer and protect your sensitive data. Egress filtering also makes it easier to quickly and appropriately respond to improper data egress.

2. Create a data egress management policy

Create a thorough data egress management policy that outlines how employees should handle sensitive data. It should include a list of acceptable services that can be used for handling data and describe how data should and shouldn't be used. 

Be sure that employees are aware of your management policy so you can avoid internal data leakages and protect your company's data. You can also use data classification solutions to assign data with classification tags. These solutions can label data based on data privacy/sensitivity, file type, user permissions and more. They can also apply protection and encryptions that prevent actions that don't comply with your data management policies.

Classification of data also makes it easier for employees to be aware of the sensitivity and privacy of important data so they can avoid unauthorized actions and handle data properly.

3. Protect data with firewalls

A firewall is a network security tool that monitors and protects inbound and outbound network data. Firewalls can be used to alert you to malicious actors and activities, as well as to stop data transfer when threats are detected. 

You'll need to tailor your firewall rules in accordance with your data egress and ingress management policies to ensure that data doesn't come into or leave your network without permission.

4. Control who can access data

Make sure you control who and what has access to your data. That means only relevant, trusted employees should be able to access your most sensitive data. And only approved devices can connect to your network.

By limiting access to your data, you can minimize the number and types of people who can access data (which will make it easier to identify offenders in the case of an improper data leak) and ensure that employees don't accidentally mishandle sensitive data.

5. Create an incident response plan for data breaches

No matter how many precautions you take, data breaches are still likely to occur. How you respond to breaches is just as important as how you guard against them. A good response plan can minimize the harmful impacts of a data breach, saving you time, money and headaches.

Your incident response plan should include things like:

• What actions should you take when a breach is detected?
• Who is responsible for taking what actions when a breach is detected?
• How will you investigate the breach to learn from it and upgrade your security?

By clearly outlining how your team responds to data breaches, you can ensure proper actions are taken as quickly as possible to minimize the harmful effects of improper data egress.

6. Reduce data egress costs with a solution like Fivetran

Use software like Fivetran to reduce and minimize data egress costs. When you set up a new connector in Fivetran, we do a full import of existing data (historical sync). From then on, we only capture changed data (updates, deletes, inserts and schema changes).

That means you can typically expect a small, one-time uptick in your egress costs during the historical sync immediately after setting up a Fivetran connector, depending on how large the data set is. Later, incremental syncs will have even lower egress costs because our connectors use cursors to pull only updated data.

Manage data egress costs and risks

Transferring data from a cloud storage provider can come with costs and risks that data management teams must be aware of. But organizations can confidently navigate data egress by identifying those risks, taking steps to manage them (i.e., protecting data from internal and external threats) and using a tool like Fivetran to reduce egress costs.

Fivetran uses log-based change data capture to immediately detect and replicate changes in real-time (delivering data as frequently as five minutes, depending upon data volume), ensuring your data is always accurately updated and minimizing long-term egress costs.