EC2 Inventory Detail Calculations

Inventory Details for EC2 in Clumio Discover provides insight into utilized sizes, change rates, and growth rates of EC2 instances in AWS accounts, based on EBS snapshots of the attached EBS volumes. All calculations are based on EBS snapshots that meet the requirements explained in the following sections. 

Utilized Size of EC2 Instance

EBS volumes attached to EC2 instances provide durable block-level storage for the instances. AWS offers different options for EBS volume sizes for specifying the storage options instances. For example, a General Purpose SSD (gp2) volume can have a volume size between 1GiB to 16TiB. If you spin up a volume of size 1GiB, then this volume size is called the provisioned size of the EBS volume. The utilized size of the EBS volume is the size of the data that is stored in this provisioned volume. For example, if you write 512MiB of data, then the utilized size of the EBS volume is 512MiB. 

The following sections explain how Clumio calculates the utilized sizes of EC2.

EBS snapshots based calculations

EBS snapshots are a point-in-time copy of the data, and each snapshot contains all of the information that is needed to restore the data to a new EBS volume - which means that we can infer the EBS volume’s utilized size based on its snapshot size. For an EC2 instance, Clumio looks for the most recent snapshot of each attached EBS volume in the last 90 days and this snapshot’s size is treated as the utilized size of the EBS volume.

How Clumio calculates the utilized size of an EC2 instance:

For example, assume that an EC2 instance has two attached EBS volumes that are provisioned for 4 blocks each.

Attached volume 1:

Provisioned blocks on July 10, 2022 Utilized blocks on July 13, 2022Snapshot A taken on July 15, 2022(Seeding)Snapshot B taken on July 16, 2022Snapshot C taken on July 17, 2022Snapshot D taken on July 18, 2022
Block 0A0A0A1A2
Block 1B0B0B1B2
Block 2C0C0C1
Block 3D0D0D1

Attached volume 2:

Provisioned blocks on July 10, 2022 Utilized blocks on July 13, 2022Snapshot A taken on July 15, 2022(Seeding)Snapshot B taken on July 16, 2022Snapshot C taken on July 17, 2022Snapshot D taken on July 18, 2022
Block 0A0A0A1A2A3
Block 1B0B0B1B2B3
Block 2C0C0C1
Block 3
  • For both attached volumes, snapshot A, taken on July 15, 2022, is seeding backups which store the data on the blocks of volumes.
  • For subsequent snapshots taken on the following days, some blocks were overwritten and other blocks were newly written to.
  • AWS snapshots are incremental backups, which means that only the blocks on the volume that have changed after the most recent snapshot are stored. This is indicated in the table by empty cells for the snapshots, where empty cells are referencing the previous snapshot storing the data.
  • If we calculate the utilized size of the EC2 instance on Aug 01, 2022, then Clumio looks for the most recent snapshot in the last 90 days for both volumes- which in this case is snapshot D, taken on July 18, 2022.
  • The size of snapshot D of attached volume 1 includes the bolded letters in the table with data (A2, B2, C1, D1) starting from block 0 through block 3, and the total snapshot size is the sum of these blocks. Since each EBS volume block is 512KiB in size, the snapshot size is 4 blocks x 512KiB = 2MiB. The utilized size of attached volume 1 is 2MiB.
  • The size of snapshot D of attached volume 2 includes the bolded letters in the table with data (A3, B3, C1) starting from block 0 through block 2, and the total snapshot size is the sum of these blocks. Since each EBS volume block is 512KiB in size, the snapshot size is 3 blocks x 512KiB = 1.5MiB. The utilized size of attached volume 2 is 1.5MiB.
  • By adding the utilized sizes of both attached volumes, the utilized size of the instance is 2MiB + 1.5MiB = 3.5MiB

Summarized utilized size of EC2 instances

For example, assume that you have two EC2 instances which have a number of attached volumes. These volumes have a number of snapshots taken on different dates. 

mceclip0.png

mceclip1.png
If today is Aug 01, 2022, Clumio looks back 90 days in time and uses the most recent snapshot’s size as the attached volume’s utilized size. The most recent snapshot is chosen because this snapshot is indicative of the latest utilized size of the volume. To calculate the utilized size of the instance, the sizes of volumes are added up.

InstanceAttached volumesMost Recent Snapshot in 90 DaysSnapshot SizeReason
EC2 instance 1Volume V1

Volume V2

Volume V3
V1: “-”

V2: S23

V3: S33
V1: 0GiB

V2: 2GiB

V3: 2GiB
No snapshot found in the last 90 days for V1
EC2 instance 2Volume V1

Volume V2

Volume V3
V1: S13

V2: “-”

V3: S33
V1: 2GiB

V2: 0GiB

V3: 2GiB
No snapshot found in the last 90 days for V2

 

The summarized utilized size of instances calculated by adding the utilized sizes of individual instances.  

Utilized size of EC2 instance 1: 2 GiB + 2GiB ≅ 4GiB  
  
Utilized size of EC2 instance 2: 2 GiB + 2GiB ≅ 4GiB  
  
Summarized utilized size of EC2 instances: 4 GiB + 4GiB ≅ 8GiB

 

Change vs Growth

Data can be written to blocks storing data or new blocks in an EBS volume. When a block storing data is overwritten with new data, we refer to it as “change”, and when data is written to an entirely new block, we refer to it as “growth”.

Change in EBS volume

The following table shows how change occurs in an EBS volume. For simplicity, consider an EBS volume with 10 blocks with indexes 0 through 9. On Day 1, data was written to 6 blocks starting from block 0 through block 5. On Day 2, block 1’s data B was overwritten with new data B1 and we refer to this as “change”. Similarly, block 3’s data D changed from D to D1 on Day 2. All the changed blocks from Day 1 to Day 3 are highlighted with bolded letters in the cells. 

 

BlocksDay 1Day 2Day 3
Block 0AA1
Block 1BB1B2
Block 2CC1
Block 3DD1D2
Block 4E
Block 5F
Block 6
Block 7
Block 8
Block 9

 

Growth in EBS volume

The following table shows how growth occurs in an EBS volume. For simplicity, consider an EBS volume with 10 blocks with indexes 0 through 9. On Day 1, data is written to 5 blocks starting from block 0 through block 4. On Day 2, data is written to new blocks i.e. block 5 and block 6 and we refer to this as “growth”. Similarly, data is written to block 7 and block 8 on Day 3. The cells with bolded letters indicate growth from Day 1 to Day 3 in this EBS volume.

BlocksDay 1Day 2Day 3
Block 0AAA
Block 1BBB
Block 2CCC
Block 3DDD
Block 4EEE
Block 5FF
Block 6GG
Block 7H
Block 8I
Block 9

 

Why Change and Growth are Importan

In the backup world, change and growth are important to understand as both impact your backup storage and the corresponding costs. If a web application writes 10GiB newly to an EBS volume of size 10GiB, it results in a growth of 10GiB and total size of 20GiB. In this case, you can expect your backup storage to be 20GiB. The 10GiB growth will be held onto forever in the backup. On the other hand, if the web application is changing the existing data stored in the 10GiB EBS volume, then your backup storage can vary widely due to the changes. This is because change causes multiple versions of the same block to be stored in the backups, and these versions are retained for the duration specified by your backup policy. Therefore, change and growth in your EBS volumes are important to understand what is contributing to your backup sizes.

To provide insights on how EC2 instances in your AWS accounts are changing and growing based on the change and growth of attached EBS volumes, Clumio calculates daily and monthly change rates and growth rates that can be used to estimate backup sizes if you are taking daily and monthly backups.

Change Rate of an EC2 Instance

The change rate of an EC2 instance refers to the rate at which the blocks of attached volumes of the instance change over time i.e. existing blocks of data getting overwritten by data for a specific time period. 

Daily change rate of an individual EC2 instance

A daily change rate explains the average rate of change in an EC2 instance on a daily basis. 

How Clumio calculates the average daily change rate of an EC2 instance:

For example, let's look at an EC2 instance that has two attached volumes with snapshots taken on different days.

Figure 1:

mceclip2.png
Assuming that today is Aug 01, 2022, Clumio calculates the average daily change rate of this instance in the following steps.

Step 1:

For the last 3 months i.e. from May 01, 2022 to Aug 01, 2022, identify snapshots that are 20 to 28 hours apart in time for each attached volume. The time difference is required to ensure that the two snapshots are approximately a day apart for calculating the daily change rate. Some snapshots that meet this requirement are:

Attached volume 1:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S1S220hr
2S2S327hr 2m
3S4S525h 53m
4S10S1122h
5S11S1223h 30m

Attached volume 2:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S5S624h
2S6S726h
3S8S924h 30m
4S9S1022h
5S10S1122h

Note

  • Each attached volume should have snapshots that meet Clumio’s requirements and should result in 2 to 5 snapshot pairs for Clumio to calculate the change rate successfully. If not, the change rate of the attached volume is shown as “-” in the UI.
  • Snapshots that are copied to a secondary region are not used in the calculations.
  • If you have multiple snapshots taken on two days, then the latest snapshot of each day will be used to create a snapshot pair to prioritize recency. For example, if you have snapshot A, snapshot B taken in order on Day 1 and snapshot C, snapshot D taken in order on Day 2, then we use snapshot B and snapshot D for calculating the daily change rate.

 Step 2:

An EBS volume can have the same blocks overwritten or new blocks written over time. Snapshots are point-in-time and capture the volume state when they are taken. To account for same blocks being overwritten and new blocks written, Clumio calculates two bounds for each snapshot pair:

1. Same blocks changing: Consider an EBS volume where the same block i.e. block 1 changes every 6 hours. The average change rate for a day would be 25%, implying that 1 out of 4 blocks changed.

BlocksData12am6am12pm6pm
Block 0A0
Block 1B0B1B2B3B4
Block 2C0
Block 3D0

 

2. Different blocks changing:Consider the same EBS volume where a different block changes every 6 hours. The average change rate for a day would be 100%, implying that 4 out of 4 blocks changed.

BlocksData12am6am12pm6pm
Block 0AA1
Block 1BB1
Block 2CC1
Block 3DD1

 

As described above, the change rate of the attached EBS volume in question can vary from 25% to 100% depending on whether the same blocks changed or different blocks changed. To account for this, Clumio calculates the two bounds for each snapshot pair as follows.

  1. For the first snapshot pair S1 and S2, calculate the utilized sizes of S1 and S2 by making ListSnapshotBlocks API calls for each snapshot.
  2. Calculate the delta between S1 and S2 (i.e. the blocks that are different between the two snapshots) by making ListChangedBlocks API calls.
  3. Bound 1: We take the delta between the snapshots and subtract the growth between them for fetching changes alone using the formula:
(delta- (snapS2UtilizedSize - snapS1UtilizedSize))/(snapS1UtilizedSize)

From Figure 1, 

For attached volume 1: Utilized sizes of S1 (Snap1Size) and S2 (Snap2Size) are 2GiB and 2.25GiB respectively.

For attached volume 2: Utilized sizes of S5 (Snap1Size) and S6 (Snap2Size) are 2.6GiB and 2.8GiB respectively.

Assuming that the delta between snapshot pairs is 1GiB, bound 1 will be:

Attached volume 1's Bound 1: (1GiB - (2.25GiB - 2GiB)) / 2GiB ≅ 0.375 or 37.5%   
  
Attached volume 2's Bound 1: (1GiB - (2.8GiB - 2.6GiB)) / 2.6GiB ≅ 0.307 or 30.7% 

     4. Bound 2: We divide the previous results by the number of days between the snapshots.

Attached volume 1's Bound 2: 0.375/ 0.83 ≅ 0.451 or 45.1%  
  
Attached volume 2's Bound 2: 0.307/ 1 ≅ 0.307 or 30.7

      5. Bound 1 and Bound 2 will be displayed as min. change rate and max. change rate for each volume’s snapshot pair depending on their values. The avg. change rate is calculated by averaging min. and max. change rates.

      6. The snapshot pair for both volumes will be displayed as follows.

Attached volume 1:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S1 and S237.5%45.1%41.3%

Attached volume 2:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S5 and S630.7%30.7%30.7%

 

Step 3:

Clumio identifies a number of snapshot pairs for each attached volume and calculates min. change rate, max. change rate and avg. change rate for each pair. Assuming that the delta from the ListChangedBlocks API calls is 1GiB for all snapshot pairs of the EBS volume, the following data is generated.

Attached volume 1:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S1 and S237.5%45.1%41.3%
S2 and S339.2%44.45%41.82%
S4 and S522.07%23.8%22.93%
S10 and S1117.78%19.53%18.65%
S11 and S1214.89%15.35%15.12%

 

The average change rate of attached volume 1 is calculated by taking an average of avg. change rates of all snapshot pairs. 

Attached volume 1's avg. daily change rate: (41.3+41.82+22.93+18.65+15.12)/5 ≅ 27.96% 

Attached volume 2:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S5 and S630.7%30.7%30.7%
S6 and S79.91%10.71%10.31%
S8 and S912.07%12.32%12.19%
S9 and S1016.67%18.31%17.49%
S10 and S1117.78%19.53%18.65%

The average change rate of attached volume 2 is calculated by taking an average of avg. change rates of all snapshot pairs. 

Attached volume 2's avg. daily change rate: (30.7+ 10.31+12.19+17.49+ 18.65)/5 ≅ 17.86% 

Lastly, the average daily change rate of the instance is calculated by averaging the change rates of attached volume 1 and attached volume 2.

Avg. daily change rate: (27.96 + 17.86)/2 ≅ 22.91% 

Growth Rate of an EC2 Instance

The growth rate of an EC2 instance refers to the rate at which the blocks of attached volumes of the instance grow by writing to new blocks over a specific time period. 

Daily growth rate of an individual EC2 instance

A daily growth rate explains the average rate of growth in an EC2 instance on a daily basis. 

How Clumio calculates the average daily growth rate of an EC2 instance

For example, consider an EC2 instance that has two attached volumes with snapshots taken on different days.

Figure 2:

mceclip3.png

Assuming that today is Aug 01, 2022, Clumio calculates the average daily growth rate of this EBS volume in the following steps.

Step 1:

For the last 3 months i.e. from May 01, 2022 to Aug 01, 2022, identify snapshots that are 20 to 28 hours apart in time for each attached volume. The time difference is required to ensure that the two snapshots are approximately a day apart for calculating the daily growth rate. Some snapshots that meet this requirement are:

Attached volume 1:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S1S220hr
2S2S327hr 2m
3S4S525h 53m
4S10S1122h
5S11S1223h 30m

Attached volume 2:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S5S624h
2S6S726h
3S8S924h 30m
4S9S1022h
5S10S1122h

Note

  • Each attached volume should have snapshots that meet Clumio’s requirements and should result in 2 to 5 snapshot pairs for Clumio to calculate the change rate successfully. If not, the change rate of the attached volume is shown as “-” in the UI.
  • Snapshots that are copied to a secondary region are not used in the calculations.
  • If you have multiple snapshots taken on two days, then the latest snapshot of each day will be used to create a snapshot pair, to prioritize recency. For example, if you have snapshot A, snapshot B taken in order on Day 1 and snapshot C, snapshot D taken in order on Day 2, then we use snapshot B and snapshot D for calculating the daily growth rate.

Step 2:

For each snapshot pair of the attached volume, calculate the daily growth by using the following formula.

(snapS2UtilizedSize - snapS1UtilizedSize)/snapS1UtilizedSize/(snapS2TimeInDays-snapS1TimeInDays)

For attached volume 1: Utilized sizes of S1 (Snap1Size) and S2 (Snap2Size) are 2GiB and 2.25GiB respectively and the snapshots are 22 hours apart (0.83 day).

Attached volume 1: (2.25GiB - 2GiB)) / 2GiB / 0.83 ≅ 0.1506 or 15.06%

For attached volume 2: Utilized sizes of S5 (Snap1Size) and S6 (Snap2Size) are 2.6GiB and 2.8GiB respectively and the snapshots are 24 hours apart.

 Attached volume 2:  (2.8GiB - 2.6GiB)) / 2.6GiB/ 1  ≅ 0.0769 or 7.69% 

Attached volume 1:

Snapshot PairDaily Growth Rate
S1 and S215.06%

Attached volume 2:

Snapshot PairDaily Growth Rate
S5 and S67.69%

 

Step 3:

Clumio identifies a number of snapshot pairs for each attached volume and calculates growth rate for each pair that will result in the following table.

Attached volume 1:

Snapshot PairDaily Growth Rate
S1 and S215.06%
S2 and S30%
S4 and S522.24%
S10 and S114.83%
S11 and S126.57%

The average growth rate of attached volume 1 is calculated by taking an average of the daily growth rates of all snapshot pairs. 

Attached volume 1's avg. daily growth  rate: (15.06+0+22.24+4.83+6.57)/5 ≅ 9.74% 

Attached volume 2:

Snapshot PairDaily Growth Rate
S5 and S67.69%
S6 and S723.14%
S8 and S914.7%
S9 and S107.8%
S10 and S114.9%

The average growth rate of attached volume 2 is calculated by taking an average of daily growth rates of all snapshot pairs. 

Attached volume 2's avg. daily growth rate: (7.69+ 23.14+14.7+7.8+ 4.9)/5 ≅ 11.64% 

Lastly, the average daily growth rate of the instance is calculated by averaging the growth rates of the attached volumes.

Avg. daily growth  rate: (9.74 + 11.64)/2 ≅ 10.69% 

 

Aggregated daily change rate and daily growth rate of EC2 instances

In the previous sections we covered how daily change rate and daily growth rate are calculated for an individual EC2 instance. Clumio calculates summarized daily change rates and daily growth rates by using EC2 instances that have non NULL values (NULL values are represented by a “-”) for utilized sizes, change rates and growth rates. 

For example, you have 10 EC2 instances, and Clumio’s calculations for utilized sizes, daily change rates, and daily growth rates of these 10 instances resulted in the following data.

 

EC2 InstanceUtilized Size of EC2 InstanceDaily Change RateDaily Growth Rate
Instance 1100GiB6.5%1.5%
Instance 250GiB4.25%3.5%
Instance 3---
Instance 416GiB5%0%
Instance 5---
Instance 610GiB5%4%
Instance 7---
Instance 8200GiB3.75%6%
Instance 930GiB3.5%1.5%
Instance 1050GiB6.25%3.2%

 

Summarized utilized size of EC2 instances:

All EC2 instances that have the utilized size calculated successfully are added to calculate the summarized utilized size.

Utilized size: 100+50+16+10+200+30+50 ≅ 456GiB

Summarized daily change rate of EC2 instances:

This is calculated by a weighted average of the utilized sizes of EC2 instances in order to consider an instance’s contribution to the summarized daily change rate based on its utilized size.

A: (100GiB (6.5%)+ 50GiB (4.25%) + 16GiB (5%) + 10GiB(5%) + 200GiB(3.75%) + 30GiB(3.5%) + 50GiB(6.25%))
B: 100GiB + 50GiB + 16GiB + 10GiB + 200GiB + 30GiB + 50GiB ≅ 456GiB
Avg. daily change rate of EC2 instances: A/B ≅ 2160/456 = 4.73%

Summarized daily growth rate of EC2 instances:

This is calculated by a weighted average of the utilized sizes of EC2 instances in order to consider an instance’s contribution to the summarized daily growth rate based on its utilized size.

A: (100GiB (1.5%)+ 50GiB (3.5%) + 16GiB (0%) + 10GiB(4%) + 200GiB(6%) + 30GiB(1.5%) + 50GiB(3.2%))
B: 100GiB + 50GiB + 16GiB + 10GiB + 200GiB + 30GiB + 50GiB ≅ 456GiB
Avg. daily growth rate of EC2 instances: A/B = 1770/4565 ≅ 3.88%  
  

 

Change Rate of an EC2 Instance

The change rate of an EC2 instance refers to the rate at which the blocks of attached volumes of the instance change over time, i.e., existing blocks of data getting overwritten by data for a specific time period. 

Monthly change rate of an individual EC2 instance

A daily change rate explains the average rate of change in an EC2 instance over a month. 

How Clumio calculates the average monthly change rate of an EC2 instance:

For example, consider an EC2 instance that has two attached volumes with snapshots taken on different days.

Figure 1:

mceclip4.png
Assuming that today is Aug 01, 2022, Clumio calculates the average monthly change rate of this EC2 instance in the following steps.

Step 1:

For the last year, i.e., from Aug 01, 2021 to Aug 01, 2022, identify snapshots that are 20 to 40 days apart in time for each attached volume. The time difference is necessary to ensure that the two snapshots are approximately a month apart for calculating the monthly change rate. Some snapshots that meet this requirement are:

Attached volume 1:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S1S2793
2S4S5553
3S5S6756
4S7S8888
5S8S9613

Attached volume 2:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S3S4859.5
2S4S5769.5
3S6S7805.5
4S7S8912
5S8S9518.5

Note:

  • Each attached volume should have snapshots that meet Clumio’s requirements and should result in 2 to 5 snapshot pairs for Clumio to calculate the change rate successfully. If not, the change rate of the attached volume is shown as “-” in the UI.
  • Snapshots that are copied to a secondary region are not used in the calculations.
  • If you have multiple snapshots taken in two months, then the latest snapshot of each month will be used to create a snapshot pair to prioritize recency. For example, if you have two snapshots in January: snapshot A taken on Jan 10th, snapshot B taken on Jan 28th, and two snapshots in February: snapshot C taken on Feb 15th, snapshot D taken on Feb 25th, then we use snapshot B and snapshot D for calculating the monthly change rate.

Step 2:

As explained in the section of daily change rate, an attached volume can have the same blocks overwritten or new blocks written over time. To account for this, Clumio calculates two bounds for each snapshot pair. 

Assuming that the delta between snapshot pairs is 1GiB, bound 1 will be:

Attached volume 1 Bound 1: (1GiB - (2.5GiB - 2GiB))/2GiB = 0.25 or 25%
Attached volume 2 Bound 1: (1GiB - (6.1GiB - 5.75GiB))/5.75GiB = 0.113 or 11.3%

For bound 2, we divide the previous results by the number of months between the snapshots.

  Attached volume 1's Bound 2: 0.25/ 1.1 ≅ 0.227 or 22.7%
  Attached volume 2's Bound 2: 0.113/ 1.06 ≅ 0.106 or 10.6%

The two bounds of a snapshot pair will be shown as min. change rate and max. change rate. The avg. change rate is calculated by averaging min. and max. change rates.

Attached volume 1:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S1 and S222.7%25%23.85%

Attached volume 2:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S3 and S410.6%11.3%10.95%

Step 3:

Clumio identifies a number of snapshot pairs for each attached volume and calculates min. change rate, max. change rate, and avg. change rate for each pair. Assuming that the delta from the ListChangedBlocks API calls is 1GiB for all snapshot pairs of the EBS volume, the following data is generated.

Attached volume 1:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S1 and S222%25%23.5%
S4 and S523.80%31.31%27.55%
S5 and S629.29%30.76%30.02%
S7 and S819.73%24.28%22%
S8 and S912.32%14.49%13.4%

The average change rate of attached volume 1 is calculated by taking an average of avg. change rates of all snapshot pairs. 

Avg. monthly change rate: (23.5+27.55+30.02+22+13.4)/5 ≅ 23.29% 

Attached volume 2:

Snapshot PairMin. Change RateMax. Change RateAvg. Change Rate
S3 and S410.6%11.3%10.95%
S4 and S513.11%13.9%13.5%
S6 and S711.26%12.5%11.88%
S7 and S87.93%10%8.96%
S8 and S95.48%7.61%6.54%

The average change rate of attached volume 2 is calculated by taking an average of avg. change rates of all snapshot pairs. 

Avg. monthly change rate: (10.95+13.5+11.88+8.96+6.54)/5 ≅  10.37% 

Lastly, the average monthly change rate of the instance is calculated by averaging the change rates of attached volume 1 and attached volume 2.

Avg. monthly change rate: (23.29 + 10.37)/2 ≅ 16.83% 

Growth Rate of an EC2 Instance

The growth rate of an EC2 instance refers to the rate at which the blocks of attached volumes of the instance grow by writing to new blocks over a specific time period. 

Monthly growth rate of an individual EC2 instance

A monthly growth rate explains the average rate of growth in an EC2 instance over a month. 

How Clumio calculates the average monthly growth rate of an EC2 instance

For example, consider an EC2 instance that has two attached volumes with snapshots taken on different days.

Figure 2

mceclip5.png
Assuming that today is Aug 01, 2022, Clumio calculates the average monthly growth rate of this EBS volume in the following steps.

Step 1:

For the last year, i.e., from Aug 01, 2021 to Aug 01, 2022, identify snapshots that are 20 to 40 days apart in time for each attached volume. The time difference is required to ensure that the two snapshots are approximately a month apart for calculating the monthly growth rate. Some snapshots that meet this requirement are:

Attached volume 1:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S1S2793
2S4S5553
3S5S6756
4S7S8888
5S8S9613

Attached volume 2:

Pair No.Snapshot ASnapshot BTime Difference (hours)
1S3S4859.5
2S4S5769.5
3S6S7805.5
4S7S8912
5S8S9518.5

 

Step 2:

For each snapshot pair of the attached volume, calculate the monthly growth by using the following formula.

(snapS2UtilizedSize - snapS1UtilizedSize)/snapS1UtilizedSize/(snapS2TimeInMonths-snapS1TimeInMonths)

For attached volume 1: Utilized sizes of S1 (Snap1Size) and S2 (Snap2Size) are 2GiB and 2.5GiB respectively and the snapshots are 793 hours apart (1.1 months apart).

Attached volume 1: (2.5GiB - 2GiB)) / 2GiB / 1.1 ≅ 0.2272 or 22.72%% 

For attached volume 2: Utilized sizes of S3 (Snap1Size) and S4 (Snap2Size) are 5.75GiB and 6.1GiB respectively and the snapshots are 859.5 hours apart (1.19 months apart).

Attached volume 2:  (6.1GiB - 5.75GiB)) / 5.75GiB/ 1 .19 ≅ 0.0511 or 5.11 % 

The snapshot pair will be displayed as follows.

Attached volume 1:

Snapshot PairMonthly Growth Rate
S1 and S222.72%

Attached volume 2:

Snapshot PairMonthly Growth Rate
S3 and S45.11%

 

Step 3:

Clumio identifies a number of snapshot pairs for each attached volume and calculates growth rate for each pair that will result in the following table.

Attached volume 1:

Snapshot PairMonthly Growth Rate
S1 and S222.72%
S4 and S531.32%
S5 and S67.32%
S7 and S83.48%
S8 and S917.7%

The average growth rate of attached volume 1 is calculated by taking an average of the monthly growth rates of all snapshot pairs. 

Attached volume 1's avg. monthly growth  rate: (22.72+ 31.32+7.32+3.48+17.7)/5 ≅  16.5% 

Attached volume 2:

Snapshot PairMonthly Growth Rate
S3 and S45.11%
S4 and S52.26%
S6 and S70%
S7 and S81.98%
S8 and S99.3%

The average growth rate of attached volume 2 is calculated by taking an average of the monthly growth rates of all snapshot pairs. 

Attached volume 2's avg. monthly growth  rate: (5.11+ 2.26+0+1.98+9.3)/5 ≅  3.73% 

The average monthly growth rate of the instance is calculated by taking an average of growth rates of attached volumes. 

Avg. monthly growth rate: (16.5%+ 3.73)/5 ≅ 10.11% 

Aggregated monthly change rate and monthly growth rate of EC2 instances

In the previous sections, we showed how monthly change rate and monthly growth rate are calculated for an individual EC2 instance. Clumio calculates the summarized monthly change rate and monthly growth rate by using EC2 instances that have non NULL values (NULL values are represented by a “-”) for utilized sizes, change rates, and growth rates. 

For example, you have 10 EC2 instances, and Clumio’s calculations for utilized sizes, monthly change rates, and monthly growth rates of these 10 instances resulted in the following data.

EC2 InstanceUtilized Size of EC2 InstanceMonthly Change RateMonthly Growth Rate
Instance 1100GiB--
Instance 250GiB3.25%2%
Instance 3---
Instance 416GiB5%0%
Instance 5---
Instance 610GiB10.75%3.25%
Instance 7---
Instance 8200GiB3.5%4.2%
Instance 930GiB3.2%2.5%
Instance 1050GiB1.25%5.92%

 

Summarized utilized size of EC2 instances:

All EC2 instances that have the utilized size calculated successfully are added to calculate the summarized utilized size.

Utilized size: 100+50+16+10+200+30+50 ≅ 456GiB

Summarized monthly change rate of EC2 instances:

This is calculated by a weighted average of the utilized sizes of EC2 instances to consider an instance’s contribution to the summarized monthly change rate based on its utilized size.

A: (50GiB (3.25%)+ 16GiB (5%) + 10GiB (10.75%) + 200GiB(3.5%) + 30GiB(3.2%) + 50GiB(1.25%)
B: 50GiB + 16GiB + 10GiB + 200GiB + 30GiB + 50GiB  ≅ 356GiB
Avg. monthly change rate of EC2 instances: A/B = 1208.5/356 ≅ 3.39%

Summarized monthly growth rate of EC2 instances:

This is calculated by a weighted average of the utilized sizes of EC2 instances to consider an instance’s contribution to the summarized monthly growth rate based on its utilized size.

A: (50GiB (2%)+  16GiB (0%) + 10GiB(3.25%) + 200GiB(4.2%) + 30GiB(2.5%) + 50GiB(5.92%))
B: 50GiB + 16GiB + 10GiB + 200GiB + 30GiB + 50GiB  ≅ 356GiB
Avg. monthly growth rate of EC2 instances: A/B = 1343.5/356 ≅ 3.77%