Our team had been seeing instability in many of our Docker environments on ElasticBeanstalk. This usually meant we had to rebuild our environments to get it working again. While researching possible causes, we came across a post about the PID 1 Zombie Reaping problem. I won't be going into detail on why this is a problem as the post covers it pretty thoroughly. Here was our problem: on deploys, zombie processes gets left behind when we kill a container's process to start a new one.
To resolve this issue, we must understand the difference between eval and exec. In eval, the process spawns a child process; in exec, you stay in the same process, which is what we want.
The CMD instruction in the Dockerfile accepts both formats.
CMD command param1 param2
Reference to the CMD instruction can be found here.
An important note is that this is not a complete solution to our problem. If command was a start script, we need to also exec the final command in that script. Otherwise, you will get stuck in the script.
For example, if the start script ended with:
unicorn -c docker/config/unicorn.rb
it would need to be changed to:
exec unicorn -c docker/config/unicorn.rb
Thanks to Tung, Benson and Eddie for explaining this to me.
Docker has become very popular in the last few years. At work, we've been using it for over a year now. As we develop, images for our apps are becoming very large. This is becoming an issue because building, pulling and pushing these images are taking longer and longer. Furthermore, our deployment process has been intermittently failing when doing docker push to Docker Hub; this happens particularly often when upstream internet is slow. Our team tried a handful of different techniques to shrink our image sizes.
This week, I'll go over the different techniques to shrink Docker images. We'll look at the advantages and disadvantages. Depending on your engineering organization and architecture, some will worker better than others.
Our team looked at a number of factors before choosing debian as our base. As ruby is our primary development language, we wanted to base our images on a distro that had strong community support. Debian also provides tooling that our dev ops team is familiar with.
There have been a few shops that are using alpine in production: https://blog.codeship.com/build-minimal-docker-container-ruby-apps/
While alpine isn't something we'll be using, it may be worth revisiting in the future as adoption picks up.
UPDATE: (03/24/2016) Docker just announced that they build their new engine on Alpine Linux. There looks to be some momentum building for Alpine.
As I stated earlier, we are primarily a Ruby on Rails organization. As such, there are plenty of ruby and rails community-supported docker images available to used as base images.
We took a look at some of the ruby and rails images we use. All these images are debian based. Even amongst minor versions, we found large differences in size. In general (though not always the case), the latest minor version will be the smallest size. Since minor versions are backwards compatible, we should be on the latest minor version. It is a good idea to standardize the ruby and rails version we use for our services. Unless there's a specific reason to deviate, we should sit to one version for consistency.
Below is are some of the versions I was able to download and their sizes: