Volume Gateway — A volume gateway provides cloud-backed storage volumes that you can mount as Internet Small Computer System Interface (iSCSI) devices from your on-premises application servers. The gateway supports the following volume configurations:
- Cached volumes — You store your data in Amazon Simple Storage Service (Amazon S3) and retain a copy of frequently accessed data subsets locally. Cached volumes offer a substantial cost savings on primary storage and minimize the need to scale your storage on-premises. You also retain low-latency access to your frequently accessed data.
Steps for this walk through:
1. Beginning provisioning of Cached Volumes Storage Gateway.
2. Download VMWare Esxi Gateway appliance from AWS.
3. Install Gateway appliance locally. Your PC or on-premises server. For this walk through, I will use VMWare Workstation on my PC.
4. Build up and configure Gateway appliance locally.
5. Connect Gateway appliance to AWS Storage Gateway.
6. Create Volume from AWS Storage Gateway Console.
7. Connect your PC to local Gateway appliance through iSCSI connector, initiate the connected volume.
8. Verify everything is working correctly and start uploading files to S3 through Gateway appliance!
Extension usage:
1. Make a snapshot of the volume with data uploaded from AWS Storage Gateway console.
2. Create a EBS volume out from the snapshot.
3. Attach the volume to an EC2 and mount the volume.
4. Verify if the uploaded data is seen correctly!
Basic concept of Cached Volumes Storage Gateway(C.V.S.G) is that, on the gateway appliance, you need to configure two disks, one for cache storage and one for upload buffer. C.V.S.G stores all your data in AWS S3 while keeping a copy of recent accessed data locally on CACHE STORAGE for low latency fast access. Where data is uploaded from and download to UPLOAD BUFFER when your application writes and retrieves data from AWS S3. For more details, refer Cached Volumes Architecture.
Only main/key actions are shown below. Start by creating C.V.S.G.
Options for building your local gateway appliance. For this walk through, I chose VMware ESXi.
Once the OVF template is download, import it to VMware. My case, VMware Workstation player on my local PC.
Once imported, create two disks for the gateway appliance as cached storage and upload buffer. Make sure your gateway appliance has 4 core and at least 16 GB ram.
The recommended size for both disks is150GB but I configured 20 GB for demo.
Repeat the steps twice to create two extra disks. Power on the VM once done.
At the login screen, the account name is sguser and password is sgpassword. Once login, check your network Connectivity and sycn your system time. Copy down your IP address as you will need it later.
Once done with Gateway appliance, contine from AWS Storage Gateway Console and input your Gateway appliance IP you copied earlier.
At this moment, AWS Storage Gateway and Gateway appliance is done setup and connected. Next, we create a storage volume for the data to be async from on-premises to S3.
Set a name and size for your storage volume. For my demo, I created a 10 GB storage volume. Skip CHAP authentication for now, we will do it later.
Finish view of storage volume.
Now back to local PC, we need to configure PC to connect to Gateway appliance through iSCSCI connector. For windows, from search, key in is and the iSCSI connector program will popup.
For English version, please refer to Connecting to Your Volumes to a Windows Client.
For Linux, please refer to Connecting Your Volumes or VTL Devices to a Linux Client.
Follow the steps to setup connector. First discover, at step 3, fill in the Gateway appliance IP.
Now connect, the status of iSCSI connector is connected on the right image.
Next, we need to initiate the disk we connected to through iSCSI so we can start using it.
For disk size smaller then 2TB, choose MBR, bigger than 2TB, choose GPT.
Select the 10 GB storage volume created earlier from AWS Storage Gateway console and initiate it, click NEXT till the end.
Once done, we test it by putting some images to the storage volume.
Now check on C.V.S.G on AWS console and will see that 33.9 MB of data is put into upload buffer.
A while later, data will be transferred to storage volume from upload buffer.
That’s it! We have successfully setup C.V.S.G between on-premises and AWS cloud.
Next, we try out the snapshot function by creating a snapshot of the storage volume.
Quickly check on the snapshot page and see that a snapshot is creating. Once it is done, create a EBS volume out from it from Actions.
Attach the created EBS volume to an EC2.
Lastly, login to the EC2, mount the EBS volume and verify the content!
For better security, CHAP authentication is recommended. Right now the connection is not CHAP authenticated.
Start by getting your local iSCSI initiate’s name. At step 4, set a password, for example, sec2ndpassword.
Now we set up CHAP authentication on AWS console.
For initiator name, fill in the name you just copied from step 2 and sec2ndpassword to Target secret. Set a password to initiator secret, for example, first1stpassword. Once done, you will see that CHAP auth has changed to Yes.
Now back to your iSCSI connector and connect again with CHAP Authentication. When connecting, click on advance(A) show in left image, key in first1stpassword in the TargetSecret(S) in right image.
Once connected, verify that the authentication is on!
And that’s it! Now you are transferring data with higher security!
Reference:
Hands-On Lab with AWS Storage Gateway part 1 - Gateway-Cached
