Guide to backup and restore big tables in DynamoDB completely

To be brief, in Foreks, we are depending on real-time data with no tolerance of high-latencies. The lower the latency, the better results we can provide to our customers/clients. After Amazon launched the Frankfurt region (eu-central-1) we decided to move from Ireland (eu-west-1) to Frankfurt. It’s much lower latency when you ping both regions from Istanbul.

We were looking for a practical and easy solution to the issue of ours but after trying many solutions (like Data Pipeline) and getting tired of their complexity, we tend to lean on some open-source projects to do our task.

After a couple researches, found this little awesome batch tool working with boto: https://github.com/bchew/dynamodump

We gave it a shot and it worked nearly-perfectly.

Read more

OpenVPN-AccessServer on AWS

OpenVPN-AccessServer on AWS

Have you ever need to access your servers in virtual private cloud environment outside from “safe-zones”? Moreover, does your team willing to login on those machines from almost anywhere at the least appropriate times? And all this requires Volvo-Class security? Well, we got you covered.

Since this article focus on AWS, our challenge comprehends Two major concepts.

  • Enabling the content of landing pages via S3 endpoint in addition to IP white listing.
  • Making sure no one routes internet traffic through the cloud network yet without losing connectivity.

Before telling more, let’s assume a basic scenario; you’ve launched a t2.nano instance & started an openvpn server (with docker or not) and you created a security rule which allows your openvpn instance’s Elastic IP address to any machine in the vpc as well as your S3 buckets. Sounds great but not very ideal. The thing is, AWS doesn’t cost a penny for the network traffic inside your VPC. So from going one EC2 to another you have to reach it over internet. That means both cost and speed issues. Same rule applies for your buckets surely. Not even mentioning that you are also providing kind of an open source ZenMate solution to all your users because whatever they download or upload goes through your network and your very own encrypted connection. And that is something no system administrator wants to be responsible for.

So what is the exact solution? Obviously it isn’t to block outgoing traffic for 0.0.0.0/0 and I can assure you, we have tried countless approaches from config files to kernel parameters. Meet the most elegant way now, OpenVPN-AS

Read more

Docker Containers With Gradle Application Plugin

After trying several different approaches, we came up with what we think is the most elegant way of integrating Docker into our build tool Gradle.

Docker and Gradle have been around for a while, and there are many tutorials, blog posts, etc. related to best practices. After trying several different approaches, we came up with what we think is an elegant way of integrating Docker with our build tool Gradle. What follows is a simple and elegant integration of the two technologies.

First, let’s simply start with why we chose and how we can use Gradle’s Application plugin without getting into Docker yet. The Application plugin works hand in hand with Groovy, Scala, and Java plugins to create an executable JVM application. Using the Application plugin itself also implies application of the Distribution plugin. So, when it comes to deciding what plugin to use for making executables, the Application plugin is the most official way of doing things.

Read more