<![CDATA[Stefan Scherer's Blog]]>https://stefanscherer.github.io/Ghost 0.11Sat, 11 Mar 2017 00:14:59 GMT60<![CDATA[How to run encrypted Windows websites with Docker and Træfɪk]]>Nowadays we read it all the time that every website should be encrytped. Adding TLS certificates to your web server sounds like a hard task to do. You have to update your certificates before they get invalid. I don't run public websites on a regular basis, so I - like

]]>
https://stefanscherer.github.io/how-to-run-encrypted-windows-websites-with-docker-and-traefik/ce3e2341-fc8d-4bca-9d3d-c9873d3c6aceFri, 10 Mar 2017 22:21:00 GMTNowadays we read it all the time that every website should be encrytped. Adding TLS certificates to your web server sounds like a hard task to do. You have to update your certificates before they get invalid. I don't run public websites on a regular basis, so I - like many others I guess - have heard of Let's Encrypt, but never really tried it.

But let's learn new things and try it out. I also have promised in the interview in John Willis' Dockercast that I will write a blog post about it. With some modern tools you will see, it's not very complicated to run your Windows website with TLS certificates.

In this blog post I will show you how to run your website in Windows containers with Docker. You can develop your website locally in a container and push it to your server. And another Windows container runs the Træfɪk proxy, that helps us with the TLS certificate as well as with its dynamic configuration to add more than just one website.

Træfɪk is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. It supports several backends like Docker to register and update its configuration for each new started container.

This picture gives you an overview of the architecture:

Traefik architecture

Normally Træfɪk is running inside a container and it is well known in the Linux Docker community. A few weeks ago I have seen that there also are Windows binaries available. Let's see if we can use Træfɪk in a Windows container to provide us encrypted HTTPS traffic to other Windows containers running our IIS website, or other web service.

Step 1: Create a Windows Docker host in Azure

First of all we need a Windows Server 2016 machine with Docker in the cloud. I will use Azure as Microsoft provides a VM template for that. This server will be our webserver later on with an own DNS name and TLS certs running our website.

Go to the Windows Containers quick start guide at docs.microsoft.com and press the "Deploy to Azure" button.

Deploy to Azure

This will bring you to the Azure portal where you can customize the virtual machine. Create a new resource group, choose the location where the server should be running a and public DNS name, as well as the size of the VM.

Customize machine

After you click on "Purchase" the deployment starts which should take only a few minutes.

Azure starts deployment

In the meantime click on the cube symbol on the left. That will show you all resource groups you have.

This Windows + Docker template already creates inbound security rules for HTTPS port 443 as well as the Docker TLS port 2376. So for our purposes we don't need to add more inbound rules.

Step 2: Buy a domain and update DNS records

For Let's Encrypt you need an own domain name to get TLS certificates. For my tests I ordered a domain name at GoDaddy. But after I walked through the steps I realised that Træfɪk also can automatically update your DNS records when you use DNSimple, CloudFlare etc.

But for first time domain name users like me I show you the manual steps. In my case I went to my domain provider and configured the DNS records.

Get the public IP address

Before we can update the DNS record we need the public IP address of the VM. This IP address is also used for the Docker TLS certificates we will create later on.

In the Azure Portal, open the resource group and click on the public IP address.

Resource group

Write down or copy the IP address shown here.

Public IP address

Go back to your domain provider and enter the public IP address in the A record. If you want to run multiple websites within Docker containers, add a CNAME resource record for each sub domain you need. For this tutorial I have added portainer and whoami as additional sub domains.

Update DNS records

After some minutes all the DNS servers should know your domain name with the new IP address of your Windows Server 2016.

Step 3: Secure Docker with TLS

We now log into the Docker host with RDP. You can use the DNS name provided by Azure or use your domain name. But before you connect with RDP, add a shared folder to your RDP session so you can also copy back the Docker TLS client certificates to your local machine. With this you will also be able to control your Windows Docker engine directly from your local computer.

In this example I shared my desktop folder with the Windows VM.

Add folder in RDP client

Now login with the username and password entered at creation time.

Login with RDP

Create Docker TLS certs

To use Docker remotely it is recommended to use client certificates, so nobody without that certs can talk to your Docker engine. The same applies if a Windows container wants to communicate with the Docker engine. Using just the unprotected port 2375 would give every container the possibility to gain access to your Docker host.

Open a PowerShell terminal as an administrator to run a Windows container that can be used to create TLS certificates for your Docker engine. I already have blogged about DockerTLS in more detail so we just use it here as a tool.

Retrieve all local IP addresses to allow the TLS certificate also from the host itself, but as well for other Windows containers to talk to your Docker engine.

$ips = ((Get-NetIPAddress -AddressFamily IPv4).IPAddress) -Join ','

Also create a local folder for the client certificates.

mkdir ~\.docker  

Now run the DockerTLS tool with docker run, just append the public IP address from above to the list of IP_ADDRESSES. Also adjust the SERVER_NAME variable to your domain name.

docker run --rm `  
  -e SERVER_NAME=schererstefan.xyz `
  -e IP_ADDRESSES=$ips,52.XXXXXXX.198 `
  -v "C:\ProgramData\docker:C:\ProgramData\docker" `
  -v "$env:USERPROFILE\.docker:C:\Users\ContainerAdministrator\.docker" `
  stefanscherer/dockertls-windows

Run dockertls

Docker will pull the Windows image from Docker Hub and create the TLS certificates in the correct folders for your Docker engine.

Afterwards you have to restart the Docker engine to use the TLS certificates. The Docker engine now additionally listen on TCP port 2376.

restart-service docker  

Restart docker

Add firewall exception for Docker

This step is needed to make other Windows container talk to the Docker engine at port 2376. But it also has another benefit. With these certs you can use the Docker client on your local machine to communicate with the Windows Docker engine in Azure. But I will start Træfɪk later on from the Docker host itself as we need some volume mount points.

The Windows Server's firewall is active, so we now have to add an exception to allow inbound traffic on port 2376. The network security group for the public IP address already has an inbound rule to the VM. This firewall exception now allows the connection to the Docker engine.

Add firewall exception

From now on you can connect to the Docker engine listing on port 2376 from the internet.

Copy Docker client certs to your local machine

To setup a working communication from your local machine, copy the Docker client certificates from the virtual machine through the RDP session back to your local machine.

Copy Docker TLS certs to client

On your local machine try to connect with the remote Windows Docker engine with TLS encryption and the client certs.

$ DOCKER_CERT_PATH=~/Desktop/.docker DOCKER_TLS_VERIFY=1 docker -H tcp://schererstefan.xyz:2376 version

Docker client from Mac

Now you are able to start and stop containers as you like.

Step 4: Run Træfɪk and other services

Now comes the fun part. We use Docker and Docker Compose to describe which containers we want to run.

Install Docker Compose

To spin up all our containers I use Docker Compose and a docker-compose.yml file that describes all services.

The Windows VM does not come with Docker Compose. So we have to install Docker Compose first. If you are working remotely you can use your local installation of Compose and skip this step.

Invoke-WebRequest "https://github.com/docker/compose/releases/download/1.11.2/docker-compose-Windows-x86_64.exe" `  
  -UseBasicParsing -OutFile $Env:ProgramFiles\docker\docker-compose.exe

If you prefer Chocolatey, use choco install docker-compose instead.

Create data folders on Docker host

You need to persist some data outside of the Docker containers, so we create some data folders. Træfɪk retrieves the TLS certs and these should be persisted outside of the container. Otherwise you run into the Let's Encrypt rate limit of 20 requests per week to obtain new certificates. This happened to me trying different things with Træfɪk and starting and killing the container lots of times.

PS C:\Users\demo> mkdir sample  
PS C:\Users\demo> cd sample  
PS C:\Users\demo\sample> mkdir traefikdata  
PS C:\Users\demo\sample> mkdir portainerdata  

docker-compose.yml

For a first test we define two services, the traefik service and a example web server called whoami. This tutorial should give you just an idea and you can extend the YAML file to your needs. Run an IIS website? Put it into a container image. And another IIS website? Just run a separate container with that other website in it. You see you don't have to mix multiple sites, just leave them alone in single microservice images.

Open up an editor and create the YAML file.

PS C:\Users\demo\sample> notepad docker-compose.yml  
version: '2.1'  
services:  
  traefik:
    image: stefanscherer/traefik-windows
    ports:
      - "8080:8080"
      - "443:443"
    volumes:
      - ./traefikdata:C:/etc/traefik
      - ${USERPROFILE}/.docker:C:/etc/ssl:ro

  whoami:
    image: stefanscherer/whoami-windows
    depends_on:
      - traefik
    labels:
      - "traefik.backend=whoami"
      - "traefik.frontend.entryPoints=https"
      - "traefik.frontend.rule=Host:whoami.schererstefan.xyz"

networks:  
  default:
    external:
      name: nat

I already have built a Træfɪk Windows Docker image that you can use. There might be an official image in the future. If you don't want to use my image, just use this Dockerfile and replace the image: stefanscherer/traefik-windows with build: ., so Docker Compose will build the Træfɪk image for you.

The Dockerfile looks very simple as we directly add the Go binary to the Nanoserver Docker image and define some volumes and labels.

FROM microsoft/nanoserver

ADD https://github.com/containous/traefik/releases/download/v1.2.0-rc2/traefik_windows-amd64 /traefik.exe

VOLUME C:/etc/traefik  
VOLUME C:/etc/ssl

EXPOSE 80  
ENTRYPOINT ["/traefik", "--configfile=C:/etc/traefik/traefik.toml"]

# Metadata
LABEL org.label-schema.vendor="Containous" \  
      org.label-schema.url="https://traefik.io" \
      org.label-schema.name="Traefik" \
      org.label-schema.description="A modern reverse-proxy" \
      org.label-schema.version="v1.2.0-rc2" \
      org.label-schema.docker.schema-version="1.0"

traefik.toml

Træfɪk needs a configuration file where you specify your email address for the Let's Encrypt certificate requests. You will also need the IP address of the container network so that Træfɪk can contact your Docker engine.

$ip=(Get-NetIPAddress -AddressFamily IPv4 `
   | Where-Object -FilterScript { $_.InterfaceAlias -Eq "vEthernet (HNS Internal NIC)" } `
   ).IPAddress
Write-Host $ip  

Now open an editor to create the traefik.toml file.

PS C:\Users\demo\sample> notepad traefikdata\traefik.toml  

Enter that IP address at the endpoint of the [docker] section. Also adjust the domain names

[web]
address = ":8080"

[docker]
domain = "schererstefan.xyz"  
endpoint = "tcp://172.24.128.1:2376"  
watch = true

[docker.tls]
ca = "C:/etc/ssl/ca.pem"  
cert = "C:/etc/ssl/cert.pem"  
key = "C:/etc/ssl/key.pem"

# Sample entrypoint configuration when using ACME
[entryPoints]
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]

[acme]

# Email address used for registration
#
# Required
#
email = "you@yourmailprovider.com"

storage = "c:/etc/traefik/acme.json"  
entryPoint = "https"

[[acme.domains]]
   main = "schererstefan.xyz"
   sans = ["whoami.schererstefan.xyz", "portainer.schererstefan.xyz", "www.schererstefan.xyz"]

Open firewall for all container ports used

Please notice that the Windows firewall is also active for the container network. The whoami service listens on port 8000 in each container. To make Træfɪk connect to the whoami containers you have to add a firewall exception for port 8000.

Docker automatically adds a firewall exception for all ports mapped to the host with ports: in the docker-compose.yml. But for the exposed ports this does not happen automatically.

Spin up Træfɪk and whoami

Now it's time to spin up the two containers.

docker-compose up  

You can see the output of each container and stop them by pressing CTRL+C. If you want to run them detached in the background, use

docker-compose up -d  

So see the output of the services you can use docker-compose logs traefik or docker-compose logs whoami at any time.

Træfɪk now fetches TLS certificates for your domain with the given sub domains. Træfɪk listens for starting and stopping containers.

Test with a browser

Now open a browser on your local machine and try your TLS encrypted website with the subdomain whoami. You should see a text like I'm 3e1f17ecbba3 which is the hostname of the container.

Now let's try Træfɪk load balancing feature by scaling up the whoami service.

docker-compose scale whoami=3  

Now there are three whoami containers running and Træfɪk knows all three of them. Each request to the subdomain will be load balanced to one of these containers. You can SHIFT-reload your page in the browser and see that each request returns another hostname.

Test whoami service with browser

So we have a secured HTTPS connection to our Windows containers.

IIS

The power of Docker is that you can run multiple services on one machine if you have resources left. So let's add another web server, let's choose an IIS server.

Add these lines to the docker-compose.yml.

  www:
    image: microsoft/iis
    expose:
      - 80
    depends_on:
      - traefik
    labels:
      - "traefik.backend=www"
      - "traefik.frontend.entryPoints=https"
      - "traefik.frontend.rule=Host:www.schererstefan.xyz"

Remember to add a firewall exception for port 80 manually. After that spin up the IIS container with

docker-compose up -d www  

And check the new sub domain. You will see the welcome page of IIS.

IIS welcome page

Portainer

Let's add another useful service to monitor your Docker engine. Portainer is a very good UI for that task and it is also available as a Windows Docker image.

Add another few lines to our docker-compose.yml.

  portainer:
    image: portainer/portainer
    command: -H tcp://172.24.128.1:2376 --tlsverify
    volumes:
      - ./portainerdata:C:/data
      - ${USERPROFILE}/.docker:C:/certs
    depends_on:
      - traefik
    labels:
      - "traefik.backend=portainer"
      - "traefik.frontend.entryPoints=https"
      - "traefik.frontend.rule=Host:portainer.schererstefan.xyz"

Portainer also needs the client certs to communicate with the Docker engine. Another volume mount point is used to persist data like your admin login outside the container.

Now run Portainer with

docker-compose up -d portainer  

Then open your browser on your local machine with the subdomain. When you open it the first time Portainer will ask you for an admin password. Enter a password you want to use and then login with it.

Portainer login

Now you have an UI to see all containers running, all Docker images downloaded etc.

Portainer dashboard

Portainer containers

Conclusion

What we have learned is that Træfɪk works pretty good on Windows. It helps us securing our websites with TLS certificates. In combination with Docker Compose you can add or remove new websites on the fly or even scale some services with the built-in load balancer of Træfɪk.

Read more details in the Træfɪk documentation as I can give you only a short intro of its capabilities.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Setup a Windows Docker CI with AppVeyor]]>I love GitHub and all the services around it. It enables you to work from anywhere or any device and still have your complete CI pipeline in your pocket. Every thing is done with a git push. You can add services like Codeship, Travis, Circle and lots of others to

]]>
https://stefanscherer.github.io/setup-windows-docker-ci-appveyor/f979baa4-3b39-4cf6-9d84-a74d2453e9cbFri, 10 Mar 2017 05:54:00 GMTI love GitHub and all the services around it. It enables you to work from anywhere or any device and still have your complete CI pipeline in your pocket. Every thing is done with a git push. You can add services like Codeship, Travis, Circle and lots of others to build and test your code and even the pull requests you get from others.

But I'm on Windows

To build applications for Windows there is a similar cloud based CI service, called AppVeyor.

And it works pretty similar to the other well known services for Linux:

  1. Put a YAML file into your repo with the build, test and deploy steps
  2. Connect your repo to the cloud CI service
  3. From now on a git push will do a lot for you.

Your CI pipeline is set up in a few clicks.

appveyor.yml

Here is an example how such a YAML file looks like for AppVeyor. This is from a small C/C++ project I made long time ago during holiday without Visual Studio at hand. I just created that GitHub repo, added the appveyor.yml and voila - I got a compiled and statically linked Windows binary at GitHub releases.

version: 1.0.{build}  
configuration: Release  
platform: x64  
build:  
  project: myfavoriteproject.sln
  verbosity: minimal
test: off  
artifacts:  
- path: x64/Release/myfavoriteproject.exe
  name: Release
deploy:  
- provider: GitHub
  auth_token:
    secure: xxxxx

The build worker in AppVeyor is fully armed with lots of development tools, so you can build projects for serveral languages like Node.js, .NET, Ruby, Python, Java ...

Docker build

AppVeyor now has released a new build worker with Windows Server 2016 and Docker Enterprise Edition 17.03.0-ee-1 pre-installed. That instantly enables you to build, test and publish Windows Docker images in the same lightweight way.

Docker build with AppVeyor

All you have to do is to select the new build worker by adding image: Visual Studio 2017 to your appveyor.yml. No more work to do to get a fully Windows Docker engine for your build.

The following appveyor.yml gives you an idea how easy an automated Docker build for Windows can be:

version: 1.0.{build}  
image: Visual Studio 2017

environment:  
  DOCKER_USER:
    secure: xxxxxxx
  DOCKER_PASS:
    secure: yyyyyyy
install:  
  - docker version

build_script:  
  - docker build -t me/myfavoriteapp .

test_script:  
  - docker run me/myfavoriteapp

deploy_script:  
  - docker login -u="$env:DOCKER_USER" -p="$env:DOCKER_PASS"
  - docker push me/myfavoriteapp

This is a very simple example. For the tests you can think of some more sophisticated tests like using Pester, Serverspec or Cucumber. For the deploy steps you can decide when to run these, eg. only for a tagged build to push a new release.

Docker Compose

You are not limited to build a single Docker image and run one container. Your build agent is a full Windows Docker host, so you also can install Docker Compose and spin up a multi-container application. The nice thing about AppVeyor is that the builders also have Chocolatey preinstalled. So you only have to add a short single command to your appveyor.yml to download and install Docker Compose.

choco install docker-compose  

Docker Swarm

You also might turn the Docker engine into a single node Docker swarm manager to work with the new commands docker stack deploy. You can create a Docker Swarm with this command

docker swarm init  

Add project to build

Adding AppVeyor to one of your GitHub repos is very simple. Sign in to AppVeyor with your GitHub account and select your project to add.

AppVeyor add project

Now you can also check the pull requests you or others create on GitHub.

GitHub pull request checks green

You can click on the green checkmark to view the console output of the build.

AppVeyor pull request build green

Tell me a secret

To push to the Docker Hub we need to configure some secrets in AppVeyor. After you are logged in to AppVeyor you can select the "Encrypt data" menu item from the drop down menu or use the link https://ci.appveyor.com/tools/encrypt

There you can enter your cleartext secret and it creates the encrypted configuration data you can use in your appveyor.yml.

Appveyor encrypt configuration data

These secret variables don't get injected in pull request builds, so nobody can fork your repo and send you an ls env: pull request to expose that variables in the output.

Immutable builds

One of the biggest advantages over self-hosting a CI pipeline is that you get immutable builds. You just do not have to care about the dirt and dust your build left on the build worker. AppVeyor - like all other cloud based CI systems - just throws away the build worker and you get another empty one for the next build.

AppVeyor immutable build

Even if you build Windows Docker images you don't have to cleanup your Docker host. You can concentrate on your code, the build and your tests, and forget about maintain your CI workers.

Examples

I have some GitHub repos that already use AppVeyor to build Windows Docker images, so you can have a look how my setup works:

Conclusion

AppVeyor is my #1 when it comes to automated Windows builds. With the Docker support built-in it becomes even more interesting.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Is there a Windows Docker image for ...?]]>Do you want to try out Windows containers, but don't want to start too low level? If you are using one of the following programming languages you can benefit of already available official Docker images for Windows.

These Docker images are well maintained and you can just start and put

]]>
https://stefanscherer.github.io/is-there-a-windows-docker-image-for/4b399853-01af-4bf3-a6d6-0c70c21ce821Tue, 21 Feb 2017 23:56:58 GMTDo you want to try out Windows containers, but don't want to start too low level? If you are using one of the following programming languages you can benefit of already available official Docker images for Windows.

These Docker images are well maintained and you can just start and put your application code inside and run your application easily in a Windows container.

Someone else did the hard work how to install the runtime or compiler for language XYZ into Windows Server Core container or even a Nanoserver container.

Prefer NanoServer

So starting to work with NanoServer is really easy with Docker as you only choose the right image for the FROM instruction in your Dockerfile. You can start with windowsservercore images, but I encourage you to test with nanoserver as well. For these languages it is easy to switch and the final Docker images are much smaller.

So let's have a look which languages are already available. The corresponding Docker Hub page normally has a short intro how to use these Docker images.

Go

The Go programming language is available on the Docker Hub as image golang. To get the latest Go 1.8 for either Windows Server Core or NanoServer you choose one of these.

  • FROM golang:windowsservercore
  • FROM golang:nanoserver

Have a look at the tags page if you want another version or if you want to pin a specific version of Golang.

Java

When you hear Java you might immediately think of Oracle Java. But searching for alternatives I found three OpenJDK distros for Windows. One of them recently made it into the official openjdk Docker images. Both Windows Server Core and NanoServer are supported.

  • FROM openjdk:windowsservercore
  • FROM openjdk:nanoserver

If you prefer Oracle Java for private installations, you can build a Docker image with the Dockerfiles provided in the oracle/docker-images repository.

Node.JS

For Node.js there are pull requests awaiting a CI build agent for Windows to make it into the official node images.

In the meantime you can use one of my maintained images, for example the latest Node LTS version for both Windows Server Core and NanoServer:

  • FROM stefanscherer/node-windows:6
  • FROM stefanscherer/node-windows:6-nano

You also can find more tags and versions at the Docker Hub.

Python

The script language Python is available as Windows Server Core Docker image at the official python images. Both major versions of Python are available.

  • FROM python:3-windowsservercore
  • FROM python:2-windowsservercore

I also have a Python Docker image for NanoServer with Python 3.6 to create smaller Docker images.

  • FROM stefanscherer/python-windows:nano

.NET Core

Microsoft provides Linux and Windows Docker images for .NET Core at microsoft/dotnet. For Windows it is NanoServer only, but this is no disadvantage as you should plan for the smaller NanoServer images.

  • FROM microsoft/dotnet:nanoserver

ASP.NET

For ASP.NET there are Windows Server Core Docker images for the major versions 3 and 4 with IIS installed at microsoft/aspnet.

  • FROM microsoft/aspnet:4.6.2-windowsservercore
  • FROM microsoft/aspnet:3.5-windowsservercore

Conclusion

The number of programming languages provided in Windows Docker images is growing. This makes it relatively easy to port Linux applications to Windows or use Docker images to distribute apps for both platforms.

Haven't found an image for your language? Have I missed something? Please let me know, and use the comments below if you have questions how to get started. Thanks for your interest. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Getting started with Docker Swarm-mode on Windows 10]]>Last Friday I noticed a blog post that Overlay Network Driver with Support for Docker Swarm Mode Now Available to Windows Insiders on Windows 10. A long awaited feature to use Docker Swarm on Windows, so it's time to test-drive it.

Well you wonder why this feature is available on

]]>
https://stefanscherer.github.io/docker-swarm-mode-windows10/fcae230c-7580-49dc-a9e0-b57183dc841aMon, 13 Feb 2017 01:31:00 GMTLast Friday I noticed a blog post that Overlay Network Driver with Support for Docker Swarm Mode Now Available to Windows Insiders on Windows 10. A long awaited feature to use Docker Swarm on Windows, so it's time to test-drive it.

Well you wonder why this feature is available on Windows 10 and not Windows Server 2016. Sure it will make more sense in production running a Docker Swarm on multiple servers. The reason is that the Insider preview is the fastest channel to ship new features. Unfortunately there is no equivalent for Windows Server editions.

So if you need it for Windows Server you have to wait a little longer. You can indeed test Swarm-Mode on Windows Server 2016 and Docker 1.13, but only without the Overlay network. To test Swarm-Mode with Overlay network you will need some machines running Windows 10 Insider 15031.

Preparation

In my case I use Vagrant to spin up Windows VM's locally on my notebook. The advantage is that you can describe some test scenarios with a Vagrantfile and share it on GitHub.

I already have played with Docker Swarm-Mode in December and created a Vagrant environment with some Windows Server 2016 VM's. I'll re-use this scenario and just replace the underlying Vagrant box.

So the hardest part is to build a Windows 10 Insider 15031 VM. The latest ISO file with Windows 10 Insider 15025 is a good starting point. You have to switch to the Fast Ring to fetch the latest updates for Insider 15031.

Normally I use Packer with my packer-windows templates available on GitHub to automatically create such Vagrant boxes. In this case I only have a semi-automated template. Download the ISO file, build a VM with the windows_10_insider.json template and update it to Insider 15031 manually. With such a VM, build the final Vagrant box with the windows_10_docker.json Packer template.

What we now have is a Windows 10 Insider 15031 VM with the Containers and Hyper-V features activated, Docker 1.13.1 installed and both Microsoft Docker images downloaded. All the time consuming things should be done in a Packer build to make the final vagrant up a breeze.

In my case I had to add the Vagrant box with

vagrant box add windows_10_docker ./windows_10_insider_15031_docker_vmware.box  

Vagrant 1.9.1 is able to use linked clones for VMware Fusion, VirtualBox and Hyper-V. So you need this big Vagrant box only once on disk. For the Docker Swarm only a clone will be started for each VM to save time and disk space.

Create the Swarm

Now we use the prepared Vagrant environment and adjust it

git clone https://github.com/StefanScherer/docker-windows-box  
cd docker-windows-box/swarm-mode  
vi Vagrantfile  

In the Vagrantfile I had to change only the name of the box after config.vm.box to the newly added Vagrant box. This is like changing the FROM in a Dockerfile.

git diff Vagrantfile

I also adjusted the memory a little bit to spin up more Hyper-V containers.

But now we are ready to create the Docker Swarm with a simple

vagrant up  

This will spin up three Windows 10 VM's and build the Docker Swarm automatically for you. But using linked clones and the well prepared Vagrant basebox it takes only some minutes to have a complete Docker Swarm up and running.

docker node ls

After all three VM's are up and running, go into the first VM and open a PowerShell terminal. With

docker node ls  

you can check if your Docker Swarm is active.

Create a network

Now we create a new overlay network with

docker network create --driver=overlay sample  

You can list all networks with docker network ls as there are already some others.

Create a whoami service

With this new overlay network we start a simple service. I've prepared a Windows version of the whoami service. This is a simple webserver that just responds with its internal container hostname.

docker service create --name=whoami --endpoint-mode dnsrr `  
  --network=sample stefanscherer/whoami-windows:latest

At the moment only DNS round robin is implemented as described in the Microsoft blog post. You cannot use to publish ports externally right now. More to come in the near future.

Run visualizer

To make it more visible what happens in the next steps I recommend to run the Visualizer. On the first VM you can run the Visualizer with this script:

C:\vagrant\scripts\run-visualizer.ps1  

Now open a browser with another helper script:

C:\vagrant\scripts\open-visualizer.ps1  

Now you can scale up the service to spread it over your Docker swarm.

docker service scale whoami=4  

This will bring up the service on all three nodes and one of the nodes is running two instances of the whoami service.

Visualizer

Just play around scaling the service up and down a little bit.

Build and create another service

As I've mentioned above you cannot publish ports and there is no routing mesh at the moment. So the next thing is to create another service that will access the whoami service inside the overlay network. On Linux you probably would use curl to do that. I tried just a simple PowerShell script to do the same.

Two small files are needed to create a Docker image. First the simple script askthem.ps1:

while ($true) {  
  (Invoke-WebRequest -UseBasicParsing http://whoami:8000).Content
  Start-Sleep 1
}

As you can see the PowerShell script will access the webserver with the hostname whoami on port 8000.

Now put this Script into a Docker image with this Dockerfile:

FROM microsoft/nanoserver  
COPY askthem.ps1 askthem.ps1  
CMD ["powershell", "-file", "askthem.ps1"]  

Now build the Docker image with

docker build -t askthem .  

We now can start the second service that consumes the whoami service.

docker service create --name=askthem --network=sample askthem:latest  

You now should see one instance of the newly created askthem service. Let's have a look at the logs. As this Vagrant environment enables the experimental features of Docker we are able to get the logs with this command:

docker service logs askthem  

In my case I had luck and the askthem service got a response from one of the whoami containers that is running on a different Docker node.

Windows 10 Swarm-Mode

I haven't figured out why all the responses are from the same container. Maybe PowerShell or the askthem container itself caches the DNS requests.

But it still proves that overlay networking is working across multiple Windows machines.

More to play with

The Vagrant environment has some more things prepared. You also can spin up Portainer that gives you a nice UI to your Docker swarm. You can have a look at your Nodes, the Docker images, the containers and services running and so on.

I also found out that you can scale services in the Portainer UI by changing the replicas. Running Visualizer and Portainer side-by-side demonstrates that:

Visualizer and Portainer

Conclusion

I think this setup can help you trying out the new Overlay network in Windows 10 Insider, and hopefully in Windows Server 2016 very soon as well.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Dockerizing Ghost and Buster to run a blog on GitHub pages]]>I'm running this blog for nearly three years now. One of my first posts was the description how to setup Ghost for GitHub pages. In the past I've installed lots of tools on my Mac to run Ghost and Buster locally.

I still like this setup hosting only the static

]]>
https://stefanscherer.github.io/dockerizing-ghost-buster/7e4d1fed-d14d-4cad-8f27-c30cc73e00eaSat, 11 Feb 2017 18:46:46 GMTI'm running this blog for nearly three years now. One of my first posts was the description how to setup Ghost for GitHub pages. In the past I've installed lots of tools on my Mac to run Ghost and Buster locally.

I still like this setup hosting only the static files at GitHub without maintaining an online server. But over time you also have to update Ghost, the Node version used and so on. That's why I have revisited my setup to make it easier for me to update Ghost by running all tools in Docker containers.

Requirements

  • Docker for Mac
  • git (is already installed)
  • docker-compose (already installed with D4M)

You can find my setup and all files in my GitHub repo StefanScherer/ghost-buster-docker.

As I'm upgrading from my local Ghost installation to this dockerized version I already have some content, the static files and my GitHub pages repo. Please refer to my old blog post how to create your repo. The following commands should give you an idea how to setup the two folders content and static.

git clone https://github.com/YOURNAME/ghost-buster-docker  
cd ghost-buster-docker  
mkdir content  
git clone https://github.com/YOURNAME/YOURNAME.github.io static  

docker-compose.yml

To simplify running Ghost and Buster I've created a docker-compose.yml with all the published ports and volume mount points.

There are three services

  • ghost
  • buster
  • preview
version: '2.1'

services:  
  ghost:
    image: ghost:0.11.4
    volumes:
      - ./content:/var/lib/ghost
    ports:
      - 2368:2368

  buster:
    image: stefanscherer/buster
    command: /buster.sh
    volumes:
      - ./static:/static
      - ./buster.sh:/buster.sh

  preview:
    image: nginx
    volumes:
      - ./static:/usr/share/nginx/html:ro
    ports:
      - 2369:80

Edit content with Ghost

To create new blog post or edit existing posts you spin up the ghost container with

docker-compose up -d ghost  

and then open up your browser at https://stefanscherer.github.io/ghost to login and edit content. As you can see the folder content is mapped into the ghost container to persist your Ghost blog data and images on your host machine.

Generate static files

To generate the static HTML pages we use the second service with Buster installed. This is no real service, so we do not "up" but "run" it with

docker-compose run buster  

Now you have updated files in the static folder. You may edit the local script buster.sh to fix some links that Buster broke in the past in my pages.

Preview static files

From time to time it is useful to check the generated static HTML files before pushing them to GitHub pages. The third service is useful to run a webserver with the created static pages.

docker-compose up -d preview  

Open your browser at http://localhost:2369 and check if everything looks good. In my setup I've added Disqus and first wanted to try out the results of modifying the post.hbs file of the theme.

Deploy static files

If you are happy with the new static files it's time to push them. I've added a small script deploy.sh to do the final steps on the host as only git is used here. As I'm using GitHub with SSH and a passphrase I don't want to put that into a container. Have a look at the shell script and you will see that it's only a git add && git commit && git push script.

./deploy.sh

Conclusion

I think this setup will help me in the future to update Ghost more easily.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Winspector - a tool to inspect your and other's Windows images]]>In my previous blog post I showed you how to get Windows Updates into your container images. But how do you know if your underlying Docker image you use in the FROM line of your Dockerfile also uses the correct version of the Windows base image?

Is there a way

]]>
https://stefanscherer.github.io/winspector/5326246a-c5c3-4a6f-b062-d9330e9d651aSun, 08 Jan 2017 14:00:00 GMTIn my previous blog post I showed you how to get Windows Updates into your container images. But how do you know if your underlying Docker image you use in the FROM line of your Dockerfile also uses the correct version of the Windows base image?

Is there a way to look into container images without downloading them?

There are several services like imagelayers.io, microbadger, shields.io and others which provide badges and online views for existing Docker images at Docker Hub. Unfortunately not all support Windows images at the moment.

Enter winspector

I found an inspector tool written in Python that might be useful for that task. I've enhanced it and created a tool called winspector which is available as Docker image stefanscherer/winspector for Windows and Linux. With this tool you can inspect any Windows Docker images on the Docker Hub.

Winspector will show you

  • The creation date of the image and the Docker version and Windows version used at build time.
  • The number of layers down to the Windows base image
  • Which Windows base image the given image depends on. So you know whether a random Windows image uses the up to date Windows base image or not.
  • The size of each layer. This is useful to when you try to optimize your image size.
  • The "application size" without the Windows base layers. So you get an idea how small your Windows application image really is and what other users have to download provided that they already have the base image.
  • The history of the image. It tries to reconstruct the Dockerfile commands that have been used to build the image.

Run it from Windows

If you have Docker running with Windows containers, use this command to run the tool with any given image name and an optional tag.

docker run --rm stefanscherer/winspector microsoft/iis  

run from windows

At the moment the Docker image depends on the windowsservercore base image. I'll try to move it to nanoserver to reduce download size for Windows 10 users.

Run it from Mac / Linux

If you have a Linux Docker engine running, just use the exact same command as on Windows. The Docker image stefanscherer/winspector is a multiarch Docker image and Docker will pull the correct OS specific image for you automatically.

docker run --rm stefanscherer/winspector microsoft/iis  

run from mac

Inspecting some images

Now let's try winspector and inspect a random Docker image. We could start with the Windows base image itself.

$ docker run --rm stefanscherer/winspector microsoft/windowsservercore

Even for this image it can show you some details:

Image name: microsoft/windowsservercore  
Tag: latest  
Number of layers: 2  
Sizes of layers:  
  sha256:3889bb8d808bbae6fa5a33e07... - 4069985900 byte
  sha256:3430754e4d171ead00cf67667... - 913145061 byte
Total size (including Windows base layers): 4983130961 byte  
Application size (w/o Windows base layers): 0 byte  
Windows base image used:  
  microsoft/windowsservercore:10.0.14393.447 full
  microsoft/windowsservercore:10.0.14393.693 update

As you can see the latest windowsservercore image has two layers. The sizes shown here are the download sizes of the compressed layers. The smaller one is the layer that will be replaced by a newer update layer with the next release.

How big is the winspector image?

Now let's have a look at the winspector Windows image to see what winspector can retrieve for you.

$ docker run --rm stefanscherer/winspector stefanscherer/winspector:windows-1.4.3

The (shortened) output looks like this:

Image name: stefanscherer/winspector  
Tag: windows-1.4.3  
Number of layers: 14  
Schema version: 1  
Architecture: amd64  
Created: 2017-01-15 21:35:22 with Docker 1.13.0-rc7 on windows 10.0.14393.693  
Sizes of layers:  
  ...

Total size (including Windows base layers): 360497565 byte  
Application size (w/o Windows base layers): 27188879 byte  
Windows base image used:  
  microsoft/nanoserver:10.0.14393.447 full
  microsoft/nanoserver:10.0.14393.693 update
History:  
  ...

So the winspector Windows image is about 27 MByte and it uses the latest nanoserver base image.

Inspecting Linux images

And winspector is not restricted to Windows images, you can inspect Linux images as well.

If you run

$ docker run --rm stefanscherer/winspector stefanscherer/winspector:linux-1.4.3

then winspector will show you

Image name: stefanscherer/winspector  
Tag: linux-1.4.3  
Number of layers: 8  
Schema version: 1  
Architecture: amd64  
Created: 2017-01-15 21:34:21 with Docker 1.12.3 on linux  
Sizes of layers:  
  ...
Total size (including Windows base layers): 32708231 byte  
Application size (w/o Windows base layers): 32708231 byte  
Windows base image used:  
  It does not seem to be a Windows image
History:  
  ...

As you can see the Linux image is about 32 MByte.

So once you have downloaded the latest Windows base images like windowsservercore or nanoserver the download experience is the same for both platforms.

Conclusion

With winspector you can check any Windows container image on the Docker Hub which version of Windows it uses.

You can also see how big each image layer is and learn how to optimize commands in your Dockerfile to create smaller Windows images.

The tool is open source on GitHub at github.com/StefanScherer/winspector. It is community driven, so feel free to send me feedback in form of issues or pull requests.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Keep your Windows Containers up to date]]>Last year in October Microsoft has released Windows Server 2016 and with it the official support for Windows Containers. If you have tried Windows Containers already and built some Windows Container images you may wonder how to implement an update strategy.

How can I install Windows Updates in my container

]]>
https://stefanscherer.github.io/keep-your-windows-containers-up-to-date/9d05e734-fddc-452e-a760-13dd16defde7Sun, 08 Jan 2017 09:23:21 GMTLast year in October Microsoft has released Windows Server 2016 and with it the official support for Windows Containers. If you have tried Windows Containers already and built some Windows Container images you may wonder how to implement an update strategy.

How can I install Windows Updates in my container image?

Working with containers is not the same as working with real servers or VM's you support for months or years. A container image is a static snapshot of the filesystem (and Windows registry and so on) at a given time.

You won't enter a running container and run the Windows Update there. But how should we do it then?

Container images have layers

First have a look how a container image looks like. It is not just a snapshot. A container image consist of multiple layers. When you look at your Dockerfile you normally use a line like FROM microsoft/windowsservercore.

Your container image then uses the Windows base image that contains a layer with all the files needed to run Windows containers.

If you have some higher level application you may use other prebuilt container images like FROM microsoft/iis or FROM microsoft/aspnet. These images also re-use the FROM microsoft/windowsservercore as base image.

Windows app image layers

On top of that you build your own application image with your code and content needed to run the application in a self contained Windows container.

Behind the scenes your application image now uses several layers that will be downloaded from the Docker Hub or any other container registry. Same layers can be re-used for different other images. If you build multiple ASP.NET appliations as Docker images they will re-use the same layers below.

But now back to our first question: How to apply Windows Updates in a container image?

The Windows base images

Let's have a closer look at the Windows base images. Microsoft provides two base images: windowsservercore and nanoserver. Both base images are updated on a regular basis to roll out all security fixes and bug fixes. You might know that the base image for windowsservercore is about 4 to 5 GByte to download.

So do we have to download the whole base image each time for each update?

If we look closer how the base images are built we see that they contain two layers: One big base layer that will be used for a longer period of time. And there is a smaller update layer that contains only the patched and updated files for the new release.

Windows Server Core updates

So updating to a newer Windows base image version isn't painful as only the update layer must be pulled from the Docker Hub.

But in the long term it does not make sense to stick forever to the old base layer. Security scanners will mark them as vulnerable and also all the images that are built from them. And the update layer will increase in size for each new release. So from time to time there is a "breaking" change that replaces the base layer and a new base layer will be used for upcoming releases. We have seen that with the latest release in December.

Windows Server Core major update

From time to time you will have to download the big new base layer which is about 4 GByte for windowsservercore (and only about 240 MByte for nanoserver, so try to use nanoserver whereever you can) when you want to use the latest Windows image release.

Keep or update?

Should I avoid updating the Windows image to revision 576 to keep my downloads small? No!

My recommendation is to update all your Windows container images and rebuild them with the newest Windows image. You have to download that bigger base layer also only once and all your container images will re-use it.

Perhaps your application code also has some updates you want to ship. It's a good time to ship it on top of the newest Windows base image. So I recommend to run

docker pull microsoft/windowsservercore  
docker pull microsoft/nanoserver  

before you build new Windows container images to have the latest OS base image with all security fixes and bug fixes in it.

If you want to keep track which version of the Windows image you use, you can use the tags provided for each release.

Instead of using only the latest version in your Dockerfile

FROM microsoft/windowsservercore  

you can append the tag

FROM microsoft/windowsservercore:10.0.14393.576  

But I still recommend to update the tag after a new Windows image has been published.

You can find the tags for windowsservercore and nanoserver on the Docker Hub.

What about the framework images?

Typically you build your application on top of some kind of framework like ASP.NET, IIS or a runtime language like Node.js, Python and so on. You should have a look at the update cycles of these framework images. The maintainers have to rebuild the framework images after a new release of the Windows base image came out.

If you see some of your framework images lag behind, encourage the maintainer to update the Windows base image and to rebuild the framework image.

With such updated framework images - they hopefully come with a new version tag - you can rebuild your application.

TL/DR

So your part to get Windows Updates into your Windows Container images is to choose the newer image in your Dockerfile and rebuild your application image with it.

If you haven't used version tags of the image below, do a docker pull ... of that image to get sure to have the updated one before you rebuild.

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[How to protect a Windows 2016 Docker engine with TLS]]>Today I have started a Windows Server 2016 VM with Container support in Azure. This is pretty easy as there is a prebuilt VM with the Docker base images. But I want a secured connection from my laptop to the Windows Docker engine running in Azure.

There is a tutorial

]]>
https://stefanscherer.github.io/protecting-a-windows-2016-docker-engine-with-tls/3c86eb6b-cf2a-4376-97dd-d2ec54bc5d67Sun, 23 Oct 2016 22:35:19 GMTToday I have started a Windows Server 2016 VM with Container support in Azure. This is pretty easy as there is a prebuilt VM with the Docker base images. But I want a secured connection from my laptop to the Windows Docker engine running in Azure.

There is a tutorial Protect the Docker daemon socket at the website of Docker which uses the openssl tool to create all the certificates etc. But how should we do this on Windows?

Just containerize what's there

I have seen the DockerTLS script in a GitHub repo from Microsoft. But this script installs OpenSSL on my machine which I don't want to.

My first thought was, let's put this script + OpenSSL into a Docker image and run it in a Windows container.

So this Dockerfile was my first attempt to just use Chocolatey to install OpenSSL, download the PowerShell script from the Microsoft GitHub repo. Done. The script can run in a safe environment and I don't have to install software on my Docker host.

DockerTLS

But there is still work to do on the host to configure the Docker engine which I wanted to automate a little more. So it would be great to have a tool that can

  • generate all TLS certs
  • create or update the Docker daemon.json file
  • Put the client certs into my home directory

But still we need a program or script with OpenSSL to do that. I thought this tool should be deployed in a Docker image and shared on the Docker Hub. And here it is:

docker run dockertls

dockertls

The script generate-certs.ps1 creates the TLS certs and copies them to the folders that would be used on the Docker host. The script would directly work on a Docker host if you have OpenSSL/LibreSSL installed.

The dockertls Docker image is created with this Dockerfile. It installs LibreSSL from OpenBSD (thanks to Michael Friis for that optimization) and copies the PowerShell script inside the image.

You can find the full source code of the dockertls image in my dockerfiles-windows GitHub repo if you want to build the Docker image yourself.

Otherwise you can just the dockertls Docker image from the Docker Hub.

Dry run

As you don't trust me or my Docker image you can do a dry run with some temporary folders where the container can copy files into without destroying your Docker host.

Just create two folders:

mkdir server  
mkdir client\.docker  

Now run the Windows container with the environment variables SERVER_NAME and IP_ADDRESSES as well as two volume mounts to write the certs back to the host:

docker run --rm `  
  -e SERVER_NAME=$(hostname) `
  -e IP_ADDRESSES=127.0.0.1,192.168.254.123 `
  -v "$(pwd)\server:C:\ProgramData\docker" `
  -v "$(pwd)\client\.docker:C:\Users\ContainerAdministrator\.docker" `
  stefanscherer/dockertls-windows

Afterwards check the folders:

dir server\certs.d  
dir server\config  
dir client\.docker  

You will see that there are three pem files for the server, the daemon.json file as well as three pem files for the client.

Of course you could manually copy the files and try them out. But this Docker image can do this for you as well.

Full run

You may have to create the .docker folder in your home directory.

mkdir $env:USERPROFILE\.docker  

Now run the container with the correct paths on the host so it can copy all certs and configs to the right place. The script can read an existing daemon.json and update it to keep all other configuration untouched.

docker run --rm `  
  -e SERVER_NAME=$(hostname) `
  -e IP_ADDRESSES=127.0.0.1,192.168.254.123 `
  -v "C:\ProgramData\docker:C:\ProgramData\docker" `
  -v "$env:USERPROFILE\.docker:C:\Users\ContainerAdministrator\.docker" `
  stefanscherer/dockertls-windows

Now you have to restart the Docker service in an administrator Shell with

restart-service docker  

One last step is needed on your host. You have to open the port 2376 in your firewall so you can access the machine from the outside. But then you're done on your host.

You can recreate the TLS certs with the same command and just restart the Docker service afterwards.

Test TLS connection

Now test the connection to the TLS secured Docker service with

docker --tlsverify `  
  --tlscacert=$env:USERPROFILE\.docker\ca.pem `
  --tlscert=$env:USERPROFILE\.docker\cert.pem `
  --tlskey=$env:USERPROFILE\.docker\key.pem `
  -H=tcp://127.0.0.1:2376 version

Or just set some environment variables

$env:DOCKER_HOST="tcp://127.0.0.1:2376"
$env:DOCKER_TLS_VERIFY="1"
docker version  

Azure

In an Azure VM you should use your DNS name for the VM in the SERVER_NAME environment variable and your public and local IP addresses of that machine.

docker-run

You have to open the firewall port 2376 on your Windows Docker host.

For Azure you also have to add a incoming rule for port 2376 in your network security group.

Then you have to securely transfer the three client pem files from your Azure VM to your laptop.

I've done that on my old Windows 10 machine which is only a 32bit machine, but I still can work with the Windows 2016 Docker engine running in Azure.

docker-version

As always, please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Run Linux and Windows Containers on Windows 10]]>At DockerCon 2016 in Seattle Docker announced the public beta of Docker for Windows. With this you can work with Docker running Linux containers in a very easy way on Windows 10 Pro with Hyper-V installed. In the meantime there is a stable version and a beta channel to retrieve

]]>
https://stefanscherer.github.io/run-linux-and-windows-containers-on-windows-10/e2bf3024-b5bb-4393-890b-dd5b65895e78Sat, 24 Sep 2016 12:55:29 GMTAt DockerCon 2016 in Seattle Docker announced the public beta of Docker for Windows. With this you can work with Docker running Linux containers in a very easy way on Windows 10 Pro with Hyper-V installed. In the meantime there is a stable version and a beta channel to retrieve newer versions.

And Microsoft has added the Containers feature in the Windows 10 Anniversary Update. With some installation steps you are able to run Windows Hyper-V Containers on your Windows 10 machine.

But there is a little bit of confusion which sort of containers can be started with each of the two installations. And you can't run both Docker Engines side-by-side without some adjustments.

This is because each of the installations use the same default named pipe //./pipe/docker_engine causing one of the engines to fail to start.

Beta 26 to rule them all

Beginning with the Docker for Windows Beta 26 there is a very easy approach to solve this confusion. You only have to install Docker for Windows with the MSI installer. There is a new menu item in the Docker tray icon to switch between Linux and Windows containers.

switching

As you can see in the video you don't have to change environment variables or use the -H option of the Docker client to talk to the other Docker engine.

So if you download Docker for Windows beta or switch to the beta channel in your installation you can try this out yourself.

The installer will activate the Containers feature if you haven't done that yet. A reboot is required for this to add this feature.

From now on you can easily switch with the menu item in the tray icon.

There also is a command line tool to switch the engine. In a PowerShell windows you can type

& 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon

and it switches from Linux to Windows or vice versa. Take care and type the option as shown here as the option is case sensitive.

Proxy for the rescue

But how does the switching work without the need to use another named pipe or socket from the Docker client?

The answer is that there is running a Proxy process com.docker.proxy.exe which listens on the default named pipe //./pipe/docker_engine.

If you switch from Linux to Windows the Windows Docker engine dockerd.exe will be started for you which is listening on another named pipe //./pipe/docker_engine_windows and a new started Proxy process redirects to this.

Under the hood

I have installed the Sysinternals Process Monitor tool to learn what happens while switching from Linux to Windows containers. With the Process Tree function you can see a timeline with green bars when each process has started or exited.

The following screenshot shows the processes before and after the switch. I have switched about in the middle of the green bar.

The current com.docker.proxy.exe (above dockerd.exe in the list) that talked to the MobyLinuxVM exits as the dark green bar highlights that.

The dockerd.exe Windows Docker engine is started, as well as a new com.docker.proxy.exe (below dockerd.exe) which talks to the Windows Docker engine.

So just after the switch you still can use the docker.exe Client or your Docker integration in your favorite editor or IDE without any environment changes.

Running both container worlds in parallel

The proxy process just switches the connection to the Docker engine. After such a switch both the Linux and Windows Docker engine are running.

Run a Linux web server

To try this out we first switch back to the Linux containers. Now we run the default nginx web server on port 80

docker run -p 80:80 -d nginx  

then switch to the Windows containers with

& 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon

docker-run-nginx

Now let's run some Windows containers. But first we try if the Linux container is still running and reachable with

start http://localhost  

With the start command you open Edge with the welcome page of the nginx running in a Linux container

nginx

Yes, the Linux container is still running.

Build a Windows web server

On Windows 10 you only can run Nanoserver containers. There is no IIS docker image for Nanoserver. Ignite update: You can run Nanoserver AND windowsservercore containers on Windows 10.

But to demo how simple nanoserver containers could be I'll keep the following sample as it is. So we create our own small Node.js web server. First we write the simple web server app

notepad app.js  

Enter this code as the mini web server in the file app.js and save the file.

var http = require('http');  
var port = 81;

function handleRequest(req, res) {  
  res.end('Hello from Windows container, path = ' + req.url);
}

var server = http.createServer(handleRequest);

server.listen(port);  

Now we build a Windows Docker image with that application. We open another editor to create the Dockerfile with this command

notepad Dockerfile.  

Enter this as the Dockerfile. As you can see only the FROM line is different from a typical Linux Dockerfile. This one uses a Windows base image from the Docker Hub.

FROM stefanscherer/node-windows:6.7.0-nano

COPY app.js app.js

CMD [ "node", "app.js" ]  

Save the file and build the Docker image with the usual command

docker build -t webserver .  

Run the Windows web server as a Docker container with

docker run -p 81:81 -d webserver  

docker-run-webserver

At the moment you can't connect directly with 127.0.0.1 to the container. But it is possible to use the IP address of the container. We need the ID or name of the container, so list the containers running with

docker ps  

Then open the browser with the container's IP address:

start http://$(docker inspect -f "{{ .NetworkSettings.Networks.nat.IPAddress }}" grave_thompson):81  

docker-inspect

Additionally the port forwarding from the host to the container allows you to contact the web server on port 81 from another machine.

curl-to-windows-10

And yes, the Windows container is also handling requests.

Conclusion

The new Docker for Windows beta combines the two container worlds and simplifies building Docker images for both Linux and Windows, making a Windows 10 machine a good development platform for both.

And with a little awareness when to switch to the right Docker engine, both Linux and Windows containers can run side-by-side.

Please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Adding Hyper-V support to 2016 TP5 Docker VM]]>Back in June I have attended the DockerCon in Seattle. Beside lots of new features in Docker 1.12 we heard about Windows Server and Docker and upcoming features in the Windows Docker engine.

Another highlight for me after the conference was a visit at the Microsoft Campus in Redmond

]]>
https://stefanscherer.github.io/adding-hyper-v-support-to-2016-tp5-docker-vm/b8c0b0b0-ea77-4a52-982d-3d754c76a1deThu, 04 Aug 2016 19:59:37 GMTBack in June I have attended the DockerCon in Seattle. Beside lots of new features in Docker 1.12 we heard about Windows Server and Docker and upcoming features in the Windows Docker engine.

Another highlight for me after the conference was a visit at the Microsoft Campus in Redmond to meet the Windows Container team around Taylor Brown. After a meeting and having lunch we talked about making my Packer template for a Windows Server 2016 TP5 Docker VM work with Hyper-V. At that time my packer template supported only VirtualBox and VMware with a blog post describing how to build it.

So Patrick Lang from Microsoft and I started to have a look at the pull request mitchellh/packer#2576 by Taliesin Sisson that adds a Hyper-V builder to Packer. After a couple of days (already back to Germany working in different time zones) we improved the template through GitHub and finally got it working.

packer build, vagrant up

If you haven't heard about Packer and Vagrant let me explain it with the following diagram. If you want to create a VM from an ISO file you normally click through your hypervisor UI and then follow the installation steps inside the VM.

packer build, vagrant up

With Packer you can automate that step building a VM from an ISO file, put all steps into a Packer template and then just share the template so others can just run

packer build template.json  

In our case the output is a Vagrant box. That is a compressed VM ready to be used with the next tool - Vagrant. It takes a Vagrant box, creates a copy of it to turn it on so you can work again and again with the same predefined VM that was built by Packer. You want to turn your VM on? Just type

vagrant up  

You want to stop the VM after work? Just type

vagrant halt  

You want to try something out and want to undo all that to start over with the clean state. Just destroy it and start it again.

vagrant destroy  
vagrant up  

There are much more commands and even snapshots can be used. The advantage is that you don't have to know all the buttons in your hypervisor. Both Packer and Vagrant are available for Windows, Mac and Linux and also support multiple hypervisors and even cloud providers.

So you only have to learn one or both of these tools and you're done if you have to work with VM's.

Adding Hyper-V builder

The Packer template for a VM has one or more builder sections. The Hyper-V section looks like this and contains the typical steps

  • Adding files for a virtual floppy for the first boot
  • Defining disk size, memory and CPU's
  • How to login into the VM
    {
      "vm_name":"WindowsServer2016TP5Docker",
      "type": "hyperv-iso",
      "disk_size": 41440,
      "boot_wait": "0s",
      "headless": false,
      "guest_additions_mode":"disable",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "iso_checksum": "{{user `iso_checksum`}}",
      "floppy_files": [
        "./answer_files/2016/Autounattend.xml",
        "./floppy/WindowsPowershell.lnk",
        "./floppy/PinTo10.exe",
        "./scripts/disable-winrm.ps1",
        "./scripts/docker/enable-winrm.ps1",
        "./scripts/microsoft-updates.bat",
        "./scripts/win-updates.ps1"
      ],
      "communicator":"winrm",
      "winrm_username": "vagrant",
      "winrm_password": "vagrant",
      "winrm_timeout" : "4h",
      "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
      "ram_size_mb": 2048,
      "cpu": 2,
      "switch_name":"{{user `hyperv_switchname`}}",
      "enable_secure_boot":true
    },

Packer can also download ISO files from a download link to make automation very easy.

The installation of a Windows Server 2016 VM can be automated with an Autounattend.xml file. This file contains information to setup the Windows VM until the WinRM service is up and running and Packer can login from the host machine to run further provision scripts to setup the VM with additional installations.

In case of the Windows Server 2016 TP5 Docker VM we additionally install Docker 1.12 and pull the Windows base OS docker images into the VM.

All these steps defined in the Packer template build a good Vagrant box to have Docker preinstalled with the base docker image as it takes some time to download it the first time.

So after a vagrant destroy you still have the Windows OS docker images installed and can work with a clean installation again. Only from time to time when there is a new OS docker image version you have to rebuild your Vagrant box with Packer.

Build the Hyper-V Vagrant box

To build the Vagrant box locally on a Windows 10 machine you only need the Hyper-V feature activated and you need a special version of packer.exe (notice: with choco install packer you only get the upstream packer where the hyperv builder is not integrated yet). The packer.exe with hyperv builder can be downloaded at https://dl.bintray.com/taliesins/Packer/.

Clone my packer template from GitHub and build it with these commands:

git clone https://github.com/StefanScherer/packer-windows  
cd packer-windows  
packer build --only=hyperv-iso windows_2016_docker.json  

This will take some time downloading and caching the ISO file, booting, installing the software and pulling the first Docker images.

Share Vagrant boxes with Atlas

Another advantage of Vagrant is that you can share Vagrant base boxes through Atlas, a service by HashiCorp. So only one has to run Packer and build the Vagrant box and provide it for other team members or the community.

packer atlas vagrant

Others can create a Vagrantfile with the box name of one of the provided Vagrant boxes. That name will be used at the first vagrant up to download the correct Vagrant box for the hypervisor to be used.

Even Microsoft has its first Vagrant box at Atlas which can be used with VirtualBox only at the moment. But it is only a matter of time that more Hyper-V based Vagrant boxes will show up in Atlas, also boxes for other hypervisors.

If you don't have a Vagrantfile you even can create a simple one to start a new test environment with two commands and a suitable Vagrant box from Atlas.

vagrant init Microsoft/EdgeOnWindows10  
vagrant up --provider virtualbox  

Vagrant itself can log into the VM through WinRM and run further provision scripts to setup a good development or test environment. It is just a decision what to install in a Vagrant box with Packer and what to install with Vagrant afterwards. You decide which flexibility you want or if you prefer a faster vagrant up experience with a full provisioned Vagrant box that was built with a longer running Packer build once.

docker-windows-box

If you are looking for a test environment for Windows Docker containers you might have a look at my docker-windows-box GitHub repo that installs Git and some additional Docker tools to get started working on some Windows Dockerfiles.

docker windows box

Conclusion

I'm happy that there is a Hyper-V builder for Packer that really works. Vagrant already has a Hyper-V provider built in so you can have the same experience running and working with VM's as others have with VMware or VirtualBox.

With a such a TP5 Vagrant box it is very easy to get in touch with Windows Docker Containers, regardless if you are working on Windows 10 with Hyper-V or from your Mac or Linux machine with another hypervisor.

Packer multiprovider

Please leave a comment if you have questions or improvements or want to share your thoughts. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[How to build a Node.js Nano Server Docker base image]]>Beginning with Windows 10 Insider 14342 you can enable the new Containers feature in Windows. This allows you to run Windows Containers directly in Windows 10 as Hyper-V containers. At the moment only NanoServer containers are supported. So it's time to get used to NanoServer and how to create some

]]>
https://stefanscherer.github.io/how-to-build-nodejs-nanoserver-image/a41f8283-bde8-4e84-ad0f-79de76a1f022Sun, 29 May 2016 18:00:53 GMTBeginning with Windows 10 Insider 14342 you can enable the new Containers feature in Windows. This allows you to run Windows Containers directly in Windows 10 as Hyper-V containers. At the moment only NanoServer containers are supported. So it's time to get used to NanoServer and how to create some base images.

In this blog post I'll show how to build small base images to deploy Node.js applications as NanoServer Docker images that you can run in Windows 10 or in Windows Server 2016 TP5.

What is Nano Server?

Each Docker image must use of of the two OS images: windowsservercore or nanoserver.

A Windows Server Core image is highly compatible with previous Windows Server Core installations. You don't have a GUI, but you can install almost everything into it. But this compatibility comes with a prize, the size of the OS image is about 9.3 GByte as it contains about a whole server.

A Nano Server image is highly optimized and almost everything was taken out of it to be able to deploy more of these containers in cloud servers. The size is about 817 MByte. This makes installing Docker on Windows 10 much faster than with the windowsservercore OS image.

So if you get asked which to choose you probably would answer to take the smaller one.

The challenge: MSI packages

But as soon as you try to write a Dockerfile to install some software into a Docker image you will notice that the minimized OS comes with a new challenge. You can't install MSI packages in Nano Server.

If you look at nodejs.org there are only MSI packages for Windows if you also want to have npm installed.

So how could we build a Node.js Docker image based on Nano Server? I've tried different ways, eg. also installing tools like lessmsi while building the Nano Server image only to find out that lessmsi is only a 32 bit application. Another restriction in Nano Server: You only can run 64 bit processes in Nano Server.

Another approach would be installing Node.js on your host machine and copying the files into the Docker image. But I don't want to install more tools than needed on a Docker host.

Therefore I show you a way to use only Docker commands and a Windows Server 2016 TP5 machine to build both Windows Server Core image as well as a Nano Server image with Node.js + npm installed.

Step 1 - Install MSI into Windows Server Core image

A much easier way is to start with a Windows Server Core image. You can download and install the Node.js MSI package with this Dockerfile. This is very similar to the Linux version and downloads, verifies the download, installs the package and then removes the download again.

Now open an editor with

notepad Dockerfile.  

and enter this Dockerfile:

FROM windowsservercore

ENV NPM_CONFIG_LOGLEVEL info  
ENV NODE_VERSION 4.4.5  
ENV NODE_SHA256 7b2409605c871a40d60c187bd24f6f6ddf10590df060b7d905ef46b3b3aa7f81

RUN powershell -Command \  
    wget -Uri https://nodejs.org/dist/v%NODE_VERSION%/node-v%NODE_VERSION%-x64.msi -OutFile node.msi -UseBasicParsing ; \
    if ((Get-FileHash node.msi -Algorithm sha256).Hash -ne $env:NODE_SHA256) {exit 1} ; \
    Start-Process -FilePath msiexec -ArgumentList /q, /i, node.msi -Wait ; \
    Remove-Item -Path node.msi

CMD [ "node.exe" ]  

You can build the Node.js Docker image with this command

docker build -t node:4.4.5 .  

After that you have Node.js + npm installed in this Docker image.

Step 2 - Distill Node.js folder

Now we want to distill the Node.js folder from the Docker image. To do that we have to run a Docker container and then we can copy the folder to a temporary folder on the host machine.

docker run --name=node-temp node:4.4.5 node --version  
docker cp "node-temp:c:\Program Files\nodejs" nodejs  
docker rm -vf node-temp  

Step 3 - COPY deploy to Nano Server image

With this extracted folder we build the Nano Server image. The following Dockerfile copies the contents of that temporary folder into the Windows directory which is already in PATH. You may want to put the files in another directory, but then you also have to add it to the PATH environment.

Create a sub folder for the Nano Dockerfile

mkdir nano  
notepad nano\Dockerfile.  

and create this Dockerfile:

FROM nanoserver

COPY nodejs /windows/system32

CMD [ "node.exe" ]  

Run this command to build the Nano Server image

docker build -t node:4.4.5-nano nano  

Now we have two Docker images, one for Windows Server Core and one for Nano Server.

Here is a small diagram of the three steps we just did:

I've uploaded both Docker images to the Docker Hub and found out that the Windows Server Core image is about 55 MByte and the Nano Server image is only 9 MByte.

Extracting all the layers of the first Docker image I found out that installing an MSI package also saves a copy of the package in the MSI cache. And running commands also leaves some changes in local databases and the registry which is also stored as difference in the layer.

So one advice to build small Windows Docker images is to avoid installing MSI packages and prefer ZIP files or even using a COPY deployment into an image. Of course MSI packages are often more convenient to install but result in bigger image sizes.

Build app with ONBUILD

Another easy way to dockerize your Node.js application is the ONBUILD feature in the Dockerfile. At least for simple examples it is very convenient to use a prepared Docker images with that feature.

So let's build another Dockerfile just like the official node:onbuild image which does these steps to install your application and all the dependencies:

  1. copy package.json
  2. run npm install
  3. copy rest of sources

So we create another folder for that Dockerfile

mkdir nano\onbuild  
notepad nano\onbuild\Dockerfile.  

and the following content:

FROM node:4.4.5-nano

RUN mkdir \app  
WORKDIR /app

ONBUILD COPY package.json package.json  
ONBUILD RUN npm install  
ONBUILD COPY . .

CMD [ "npm.cmd", "start" ]  

Now build the Nano Server image with the ONBUILD feature:

docker build --isolation=hyperv -t node:4.4.5-nano-onbuild nano/onbuild  

I have tested this with a small Node.js web server that uses Express and some other dependencies.

To build a dockerized Node.js application running in a Nano Server container you only have to go to your Node.js source code and add a single line Dockerfile

FROM nano:4.4.5-nano-onbuild  

and build your application Docker image with

docker build --isolation=hyperv -t mynodeapp:nano .  

Optimizations

Investigating the layers of such an application showed some other temporary folders which aren't needed inside the Docker image.

  1. A npm-cache folder
  2. Many files in temp folder, also from npm

So we can optimize that ONBUILD Dockerfile a little bit to remove these temp folders while building your application Docker image. There is a command npm cache clean, but this didn't work for me, so I have changed that to some rd commands. This is the final ONBUILD Dockerfile:

FROM node:4.4.5-nano

RUN mkdir \app  
WORKDIR /app

ONBUILD COPY package.json package.json  
ONBUILD RUN npm install & rd /s /q %APPDATA%\npm-cache & for /d %G in ("%TEMP%\npm-*") do rd /s /q "%~G"  
ONBUILD COPY . .

CMD [ "npm.cmd", "start" ]  

With that optimized Docker image deploying a simple Express web server the final application Docker image went down from 24 MByte to 15 MByte. In comparison with the unoptimized Windows Server Core image the same application is 82 MByte on the Docker Hub.

Conclusion

If you don't want to build these Node.js Docker images manually you can find them on the Docker Hub with links to the Dockerfiles in the GitHub repo.

With such a Node.js Nano Server base image on the Docker Hub you can start developing on your Windows 10 machine. Now dockerize your Node.js apps into Nano Server containers and share them with others via the Docker Hub.

The Windows Server 2016 is only needed to install MSI packages and distill the software into Nano Server images.

If you find this blog post useful, just share it with your friends and colleages. Please leave a comment if you have questions or improvements. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Setup a local Windows 2016 TP5 Docker VM]]>It is great to see that more and more people are starting to test Windows Docker containers. The latest Windows Server 2016 Technical Preview 5 is a good starting point to see the current state of the Docker Engine running on Windows and get in touch with Windows Containers.

Very

]]>
https://stefanscherer.github.io/setup-local-windows-2016-tp5-docker-vm/9903a0ba-2f77-4f45-950d-cef82a573854Sun, 01 May 2016 08:43:00 GMTIt is great to see that more and more people are starting to test Windows Docker containers. The latest Windows Server 2016 Technical Preview 5 is a good starting point to see the current state of the Docker Engine running on Windows and get in touch with Windows Containers.

Very soon there will be a Microsoft Azure template to get started very easily. Another simple way will be creating a Docker Machine in Azure, once a pull request is merged.

Tutorials

But for now it takes some effort and time to set up such a local VM. There are good tutorials that guide you through all the steps needed.

Packer + Vagrant = Automation

If you don't want to do all these whole setup manually and wait in front of your computer to enter the next step you can use Packer and Vagrant.

Packer takes the ISO file and bakes a base box VM to be used with Vagrant. With Vagrant you can spin up one ore more such VM's and even form a Windows Docker Swarm.

The Packer template to create a Windows 2016 TP5 VM including the Docker Engine is tested with VirtualBox 5.0.20 and VMware Fusion 8.1 and should also work with VMware Workstation if you are working with a Windows PC.

Run Packer

To build the Vagrant base box with Packer 0.10.0 you just clone the GitHub repo.

git clone https://github.com/StefanScherer/packer-windows  
cd packer-windows  

Now build the Vagrant base box for VMware

packer build --only=vmware-iso windows_2016_docker.json  

or for VirtualBox.

packer build --only=virtualbox-iso windows_2016_docker.json  

This takes about an hour, so you can leave your computer for a while.

packer-build

After that a box file should be created in the current directory. Now add this box to Vagrant with

vagrant box add windows_2016_tp5_docker windows_2016_docker_vmware.box  

If you have both hypervisors, you also can create both base boxes and add them. You can list all your base boxes with

$ vagrant box list
windows_2016_tp5_docker (virtualbox, 0)  
windows_2016_tp5_docker (vmware_desktop, 0)  

Run Vagrant

Now you can create test and dev scenarios with this new base box. There is another GitHub repo for that. We just clone it with

git clone https://github.com/StefanScherer/docker-windows-box  
cd docker-windows-box  

With the current Vagrant 1.8.1 it is easy to spin up a VM to have Docker running on Windows 2016 TP5.

vagrant up  

Vagrant powers up a VM and install further Docker tools like Machine and Compose. Also Git will be installed to get in touch with some Windows Dockerfiles available on GitHub.

vagrant-up

You can open a PowerShell window and run for example

docker version  
docker images  

docker-version

Congratulations! You can start working with a brand new Docker Engine running on Windows 2016 TP5.

Have fun!

What Packer does for you

If you want to understand what Packer does building such a VM in an automated way here is a list of provision scripts that Packer runs.

Install features

In the script enable-winrm.ps1, just before the WinRM port will be opened to let Packer login and do further provisioning some Windows features like Containers and Hyper-V (only VMware) will be enabled.

Install Docker

The next script install-docker.ps1 installs the Docker service and client and the windowsservercore base Docker image. If Hyper-V is enabled, also the nanoserver base Docker images will be installed.

Patch windowsservercore image

As the TP5 and relating files and images are very fresh and it is still a technical preview, there might be some obstacles here and there.

At the moment we need this script to speed up the windowsservercore Docker image a little bit. The script patch-boot-time-for-containers.ps1 just fixes that for you.

Enable insecure Docker port 2375

For a local test environment we also just open the insecure Docker port 2375 with the script enable-docker-insecure.ps1.
You can remote control your Windows Docker engine from your host machine that is running the VM. Try it out, especially if you are normally working with Linux or Mac.

Once there is a Docker Machine driver for local Windows VM's is available I would prefer that and have the secure TLS connection.

Add docker group

The new Windows Docker engine listens to a Windows named pipe. This is very similar to the Unix socket on Linux.

A normal user does not have access to it, so you have to open an administrator shell to work with the Docker engine.

The script add-docker-group.ps1
adds the option -G docker to the Docker engine to give all members of the Windows group docker access to that named pipe.

The script also adds the user vagrant to that group. So in the final Vagrant box you just open a normal PowerShell window and can use the Docker engine.

Remove key.json

The final script remove-docker-key-json.ps1
removes the key.json file from the initial installation. This file will be created on the first start of the Docker engine in each Vagrant VM and creates different ID's for each Docker engine.

This is important if you want to build a Windows Docker Swarm. Each Docker engine needs a different ID.

Conclusion

As there might be updates in the Docker base images and the Docker engine itself it is now very easy to reproduce the base VM with Packer and Vagrant without all the manual steps again.

If you find this blog post useful, just share it with your friends and colleages. Please leave a comment if you have questions or improvements. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[First impressions of Docker on Windows 2016 TP5]]>Yesterday Microsoft announced the availabilty of the Windows 2016 Technical Preview 5. Sure I had to get in touch with that new toy as soon as possible to see what has changed since TP4 that was released in November 2015.

As I haven't found an Azure template (update: today I

]]>
https://stefanscherer.github.io/first-impressions-of-windows-2016-tp5/dfed9567-e981-4940-9ec5-a3e390afb2b5Fri, 29 Apr 2016 00:11:03 GMTYesterday Microsoft announced the availabilty of the Windows 2016 Technical Preview 5. Sure I had to get in touch with that new toy as soon as possible to see what has changed since TP4 that was released in November 2015.

As I haven't found an Azure template (update: today I found Windows Server 2016 Technical Preview 5), I prepared my packer templates in a tp5 branch that still need some changes to automatically build and install Docker and the Docker base images.

packer build

Even without a 100% solution this was a good first starting point build a first Vagrant box to start with and to do the remaining steps from Container Host Deployment - Windows Server manually.

I'm really addicted to the automation workflow

  1. packer build
  2. vagrant box add
  3. vagrant up
  4. vagrant destroy -f

because you won't do it only once to build and have such a test environment. But setting it up is the hard way.

If you are impatient to wait for a complete packer template you can follow Arun Gupta's steps to build a TP5 VM in VirtualBox.

Now let's see what's new in the Technical Preview 5.

Way faster - no more sleep

Building some Docker images with one of the Windows Dockerfiles I tested with TP4 are now much faster and as well you do not need to do the workaround of sleeping some seconds in each RUN command. Just remove this and forget about it.

IP adresses can be inspected

One of the obstacles porting the Docker Birthday app to TP4 Windows containers was the missing ability to fetch the IP adress of each container.

Running docker inspect now shows the IP address of the container.

container ip address

I will have to test whether the links in a docker-compose.yml file will work as well to set up a simple multi container app using simple host names.

But that's a good improvement of the networking in Windows containers.

npipe is your unix socket

Another interesting thing is that the Docker engine now listens to a Windows named pipe, just like a Unix socket on Linux.

This leads to a very unixish behavior that you have to be "sudo" to run docker commands:

Running the docker client without adminstratotion rights you can't connect to it.

There is a way to allow specific users to use this named pipe just like on Linux adding users to the docker group, but I haven't found out the group name for this short test.

Kitematic meets Windows (again)

At this evening I also played with Docker for Windows Beta in a parallel VM and installed Kitematic there. As the Beta uses Kitematic without VirtualBox I thought this should also work well with TP5. So I took the ZIP file from the Beta download and tweaked my TP5 installation a little bit to make Kitematic talk to the Window Docker Engine.

My very pragmatic way to make this work was to add the local TCP port 2375 to the Docker Engine by editing the start script

notepad C:\ProgramData\docker\runDockerDaemon.cmd  

and change the dockerd command to dockerd -H npipe:// -H 127.0.0.1:2375 and then run

restart-service docker  

The downloaded version of Kitematic for the Beta also searches for docker.local, so I added the loopback IP address to the host table with

notepad C:\Windows\system32\drivers\etc\hosts  

kitematic-on-tp5

Now Kitematic starts up without creating a local VM and shows the local Windows containers on the left side.

There's a lot more to explore in TP5 as you now can pull and push Docker Images from and to the Docker Hub.

So stay tuned of upcoming blog posts. With the TP5 the community can follow the progress of Docker on Windows much better and can check the latest improvements and pull requests of the Windows Docker Engine.

If you find this blog post useful, just share it with your friends and colleages. Please leave a comment if you have questions or improvements. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Build Docker Swarm binary for Windows the "Docker way"]]>In my last blog post I showed how to run a Windows Docker Swarm. That tutorial just used a prebuilt Swarm Docker image that was loaded from local disk.

Some people asked me how I have built that container image. Assuming you have a Windows Server 2016 TP4 machine you

]]>
https://stefanscherer.github.io/build-docker-swarm-for-windows-the-docker-way/5d5c817e-283f-487c-a9df-23e65d0b8bfdSat, 05 Mar 2016 20:58:38 GMTIn my last blog post I showed how to run a Windows Docker Swarm. That tutorial just used a prebuilt Swarm Docker image that was loaded from local disk.

Some people asked me how I have built that container image. Assuming you have a Windows Server 2016 TP4 machine you nowadays need nothing more than the right Dockerfiles to do this.

So let's start writing a Dockerfile to build Docker Swarm inside a Docker container. For the next steps open a PowerShell window and then open an editor.

notepad Dockerfile.builder  

Use a base image

Swarm is written in Go, so we look for a useful base image. And there already is one useful base image in the Docker Hub. So we begin the Dockerfile with the following line.

FROM microsoft/golang  

Install Git

Additionally to Go we also need Git installed inside the Container image. So we use the following lines that will download and install Git for Windows and then remove the installer again. As a further step we add Git to the PATH variable.

ENV GIT_VERSION 2.7.2

RUN powershell -Command \  
    Sleep 2 ; \
    wget https://github.com/git-for-windows/git/releases/download/v%GIT_VERSION%.windows.1/Git-%GIT_VERSION%-64-bit.exe -outfile gitinstaller.exe ; \
    Start-Process .\gitinstaller.exe -ArgumentList '/VERYSILENT /SUPPRESSMSGBOXES /CLOSEAPPLICATIONS /DIR=c:\git' -Wait ; \
    rm .\gitinstaller.exe

RUN setx PATH %PATH%;C:\git\cmd;C:\git\bin;C:\git\usr\bin  

Install and build Godep and Swarm

The Swarm project uses Godep to manage the dependencies. We also need this tool as well as the Swarm sources. So let's install both and run the godep tool to build the swarm.exe.

ENV SWARM_VERSION v1.1.3  
ENV GOPATH C:/go

WORKDIR /go

RUN powershell -Command \  
    Sleep 2 ; \
    go get github.com/tools/godep ; \
    mkdir src\github.com\docker ; \
    cd src\github.com\docker ; \
    git clone https://github.com/docker/swarm ; \
    cd swarm ; \
    git checkout %SWARM_VERSION% ; \
    Start-Process -FilePath godep.exe -ArgumentList go, install, . -Wait

Now save the Dockerfile.builder file and close the editor.

Back in the PowerShell build the Docker image with the name swarm-builder and the given Dockerfile with the following command.

docker build -t swarm-builder -f Dockerfile.builder .  

This now creates a Docker image with multiple layers, for each ENV and RUN command in the Dockerfile. You can see the actions on your screen and finally the command finishes and you can list your images with

docker images  

You should see a image with the name swarm-builder:

REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE  
swarm-builder       latest              04e557dd075a        31 minutes ago      1.219 GB  

The size of the Docker image is not good for deployment as we have all the development tools and source codes inside it.

Test and extract the swarm.exe

We only need the swarm.exe from this image as it is a static binary. So we just run the Docker image and have a very little test of the binary just created by showing its version.

After that we copy the swarm.exe from the container back to the host.

docker run --name swarm-builder swarm-builder swarm --version  
docker cp swarm-builder:/go/bin/swarm.exe ./swarm.exe  

Build a small Swarm image

The final Docker image for deployment can be produced with a much simpler Dockerfile. As the time of writing the Technical Preview 4 needs a small workaround to access the network inside a container. We have to add a small helper script for now.

nodepad run.ps1  

The small PowerShell script just sleeps a little and then calls swarm.exe with the arguments given.

Sleep 2  
\swarm.exe $args

Now it is time to write the Dockerfile. So open an editor with the following command. Notice the dot '.' as this helps Notepad create a Dockerfile without an extension.

notepad Dockerfile.  

The Dockerfile now uses the windowsservercore base image and copies the helper script and the swarm binary into the image and sets the entrypoint.

FROM windowsservercore

COPY ./swarm.exe /swarm.exe  
COPY ./run.ps1 /run.ps1

ENV SWARM_HOST :2375

ENTRYPOINT ["powershell.exe", "-File", "/run.ps1"]  
CMD ["--help"]  

We now build the final Docker image with

docker build -t swarm .  

Just check the size of the Docker images again with

docker images  

docker images

If you build a new version of the Swarm image later it is useful to tag each Docker image with its version. For this tutorial we have built Swarm 1.1.3, so we tag it with

docker tag swarm:latest swarm:1.1.3  

You now can run a Swarm container eg. with

docker run -it swarm:1.1.3 --version  

and you will see the version of the swarm.exe.

Automate all the things

Both the Dockerfiles and a build script can be found in my dockerfiles-windows GitHub repo. Even to download the sources you do not need more than a PowerShell window. Just download it as a ZIP and extract it:

wget -outfile src.zip https://github.com/StefanScherer/dockerfiles-windows/archive/master.zip  
Expand-Archive .\dockerfiles-windows.zip  

The steps to build both the development Docker image as well the final image is added to a small build script which can be run with this commands

cd src\dockerfiles-windows-master\swarm  
.\build.bat

TL/DR

Even on Windows you now can put development environments into Docker images and build small images for deployment with a two step approach. No special setup is needed on your host machine, just Docker should be reachable.

Everything is reproducable on every team members machines. And a CI build pipeline can take advantage of this dockerized setup as well.

I hope you get a feeling what you can do with your own software project and start dockerizing as well.

If you find this blog post useful, just share it with your friends and colleages. Please leave a comment if you have questions or improvements. You can follow me on Twitter @stefscherer.

]]>
<![CDATA[Run a local Windows Docker Swarm]]>At the DockerCon EU 2015 Arnaud Porterie showed us a Docker Swarm between a Linux and a Windows Docker Engine. In this blog post I explain how to set up a Windows Docker Swarm. I use this setup for demonstration purposes, so the VM's are running locally on my laptop.

]]>
https://stefanscherer.github.io/build-your-local-windows-docker-swarm/87bd3787-f4b9-4d55-b765-47f6f601cb81Tue, 01 Mar 2016 17:31:00 GMTAt the DockerCon EU 2015 Arnaud Porterie showed us a Docker Swarm between a Linux and a Windows Docker Engine. In this blog post I explain how to set up a Windows Docker Swarm. I use this setup for demonstration purposes, so the VM's are running locally on my laptop.

At the time of writing there is no docker-machine driver to create the Windows Docker VM's. And my laptop at work is still running with Windows 7, so I have chosen VirtualBox as a local hypervisor instead of Hyper-V.

Network diagram

As you can see in the diagram there also is a private Docker registry. I just want to show you that saving and restoring Windows Docker images is just as easy as on Linux.

A laptop with 16GB memory is capable of running this setup. With less memory, just run only two of the Windows VM's and reduce the memory setting.

Setup your host

If you like to follow this tutorial you need some tools on your host machine. Pleas keep in mind that VirtualBox does not work with Hyper-V on the same machine. My preferred way to setup a Windows machine is by install the software with Chocolatey. So these are the steps needed for the host tools:

iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))  
choco install -y virtualbox  
choco install -y vagrant  
choco install -y packer  
choco install -y docker  

Build the Vagrant box

Before spinning up the swarm you have to build the Windows Server 2016 VM with Docker installed. As I'm a big fan of automation I do not click through the installers, but use Packer and Vagrant to build and run local VM's.

Before running Packer you should register to the technical preview evaluation and accept the license for the Windows Server ISO file.

After that just clone my Packer templates and build the VirtualBox VM with

git clone https://github.com/StefanScherer/packer-windows  
cd packer-windows  
packer build --only=virtualbox-iso windows_2016_docker.json  

Now add the Vagrant box to the local list of baseboxes with

vagrant box add windows_2016_docker windows_2016_docker_virtualbox.box  

Build the Docker swarm

The next step is also very easy. We clone the repo with the Vagrantfile for the Docker swarm and spin up all boxes.

git clone https://github.com/StefanScherer/docker-windows-box  
cd docker-windows-box  
cd swarm-demo  
vagrant up --provider virtualbox  

One after another the VM's are booted and provisioned with the shell scripts provided in the scripts folder.

registry

The Linux box registry is used for the Docker Swarm manager and a Docker registry.

  1. The swarm manager is using a token in config/swarm-token. Edit this as you like.
  2. The registry is using the registry-v2 folder on your host to store the Docker images.

sw-win-01 ...

The Windows Server 2016 TP4 machines that spin up a Swarm container to join the Docker Swarm. The Docker Engines have a connection to the insecure registry running at registry:5000.

Up and running

After a while all VM's are up and running and you can see the status with vagrant status

vagrant status

The Docker Swarm is built and all Windows VM's have joined the swarm. You can check this by connecting with the Docker Client to the Swarm Manager.

docker -H tcp://192.168.38.100:3375 info  

docker info

Run containers in the swarm

For a very simple demo we now run three containers that are spread over all three Windows Docker Swarm agents.

docker -H tcp://192.168.38.100:3375 run -d windowsservercore powershell -Command sleep 60  

docker run

As you can see the containers are running on all different Docker nodes sw-win-01 ... sw-win-03.

With the local registry you can push and pull Windows images from it and safe your work. The registry files are stored at your host so it is safe to destroy all VM's if you do not use them.

I hope you have enjoyed this tutorial. In a future blog post I probably show how to build a Windows Docker Swarm in Azure or in Hyper-V.

If you find this blog post useful, just share it with your friends and colleages. Please leave a comment if you have questions or improvements. You can follow me on Twitter @stefscherer

]]>