Technical

Amazon S3 operations using signed url's

Amazon S3 is one of the most popular services in AWS suite and has SDK’s available for almost all popular languages and frameworks. One core aspect of our microservice architecture at Shippable is that all services interact with a common API service which serves as the only point of contact for CRUD operations and permission resolution for all objects in the system. One important function of API is to provide credentials to any requesting microservice e.

Read more

Share Comments

Setting permissions on Amazon ECR repositories

Setting up permissions for images on Docker hub is pretty straightforward, which follows a simple Github-like model. Amazon (EC2 Container Registry)ECR is a great service for storing the images but setting correct permissions is slightly complicated. This is especially true when configuring user specific permissions on the images. We’ll create a few users and repos and set up repo permissions. We’ll use aws command line tool for this. Using cli makes it easier to script all the steps and automate the entire process.

Read more

Share Comments

Kubernetes cluster with flannel overlay network

This is the third an final post in the series where we play around with Docker and Kubernetes. The first two posts are available here: – Multi node kubernetes cluster – Docker overlay network using flannel In this tutorial I’ll try to explain how to bring up a multi node kubernetes cluster with an overlay network. This essentially combines what I’ve explained in previous posts. An overlay is necessary to fulfill the networking requirements for a fully functional kubernetes cluster.

Read more

Share Comments

Docker overlay network using Flannel

This is the next blog post in the series where I’ll attempt to build a full multi-node kubernetes cluster from scratch. You can find the previous post HERE where I describe bringing up a two-node cluster without using overlay network. The first thing you need once you start scaling up your containers on different hosts is a consistent networking model, the primary requirement of which is to enable two(or more) containers on different hosts to talk to each other.

Read more

Share Comments

Multi node kubernetes cluster

This guide demonstrates how to build a 2 node kubernetes cluster. Kubernetes comes with a set of scripts to install on different cloud providers and locally on vagrant box that can be used for the cluster setup but the idea here to provide an installation mechanism that is provider and OS agnostic. So, at the end of this guide, we’ll have a script that can be run on any two machines that can communicate with each other.

Read more

Share Comments

e-space cleanup

As one of my new year’s resolution, I wanted to reduce the clutter I have in my life to make sure I only spend time on things I really want. To take this a step further, I have resolved to get rid of anything that is a waste of time. One of the things that have been bothering me since a long time was my website and its escalating maintenance costs both in terms of time and money.

Read more

Share Comments

Secure rabbitmq connections with HAProxy

The problem here is to make a rabbitmq server run behind HAProxy and use ssl for connecting from the internet to HAProxy. So, the proxy serves as the endpoint for ssl connections and from there onward, the connection with rabbitmq are without ssl. Internet <—–ssl—–> HAProxy <—-no-ssl—-> rabbitmq(server and admin) Also, we run rabbitmq here inside a docker container. Step 1 Installing HAProxy: The following steps should install HAProxy 1.5 on your system, assuming it is some version on ubuntu.

Read more

Share Comments

Logstash 101 with Docker

We, at Shippable, are using Docker extensively and ALL of our services(frontend, backend, messaging, etc) are now successfully running inside containers. The latest addition to this is Logstash, which is an awesome tool to store, manage and make sense of tons of logs that are spewed out by a typical application. Its for the same use case we decided to use Logstash which basically collects the data from all our containers to make it easier for us to track down exceptions and some important events.

Read more

Share Comments

Docker containers memory troubleshooting

I’ve been playing around with docker for about a week now to figure out if its the right tool for what we’re planning to do. I’ll briefly describe the setup and dive into the issue. The sandbox environment is setup in a vagrant box(referred to as the ‘host machine’ from now on) running Ubuntu 13.04(updated the default kernel). Memory assigned to the host machine is 512. The base image is available on docker index and is about 2.

Read more

Share Comments

Hosting woes end

So, it was about an year ago I started having problems with my hosting provider. The first one was Znet(http://www.znetlive.com/) . The problems were (biggest) long load times, frequent “maintenance” shutdowns and other accessibility issues. After living with this for some time I decided to move to a different host and this time chose Brainpulse(http://www.brainpulse.com/). The primary reason for choosing these was location preferences. Brainpulse hosting was going good for a while before their screwed up (and completely lost) tech and customer support managed to make sure this site was down for about a week.

Read more

Share Comments