Backing up data with Ansible in Linux and Windows (Part 1)

In this post I will explain how to connect to a windows/linux machine using Ansible  to make backup copies of your data. In my company, Genera Games, I managed all the processes with a Python project called Professor-X. Each module of the project is a member of the X-Men, which performs a specific task. In this post I am going to talk about Wolverine, who is in charge of managing the backups in my office.

The first step is to create a Python 3 virtual environment with the following command  virtualenv wolverine -p python3 . Once installed you must install the following dependencies:

Now we can try to make a small playbook that connects to the windows you have chosen and do an ipconfig. The playbook pb-wintest-data.yml would have the following content:

In hosts we put the name of the group of machines to which we are going to access, then we will use this name to list the ip of the same ones. That same name is then used in the group_vars directory to define variables that affect that group of machines. The inventory file will contain the following information:

In the group_vars folder you should have an all.yml file that will be variables common to all the playbooks and a wintest.yml file whose content should be:

I still have to figure out how to configure Active Directory to support ssl and validate the certificate. For now we will make it “insecure”. Now try to run the playbook with ansible-playbook -i ../inventory pb-wintest-data.yml -vvvv. I put “-vvvv” because I want all the possible information to come out of the errors that are not going to come out until it works, usually we do not put anything to avoid long logs. The output that should give you is the following:

To fix this error we have to following several steps:

  • Enable Windows Remote Management ports (ports 5985 and 5986) in the firewall of the target machine. For that we enter the advanced configuration of the Windows Firewall.
  • Run the script that ansible provides to configure a windows host for remote administration ConfigureRemotingForAnsible.ps1
  • In our case, as we have not yet configured any certificates or secure traffic, we must also execute the command:  winrm set winrm/config/Service '@{AllowUnencrypted = "true"}'  To check that it worked, we printed the winrm configuration with the command winrm get winrm/config

If everything has worked, we should see the result of executing the command on the remote machine ipconfig /all.

Once the connection is established, we will do what a priori seems simple: we connect to the remote machine, compress what we want to keep in a zip file and rsync to a machine that we have chosen to save our backups. But in windows this can be a bit messy, to begin with if we try to do a rsync of the pull we’re going to get an error in powershell. To get rsync, we must install cygwin and to get it installed we must install Chocolatey. Chocolatey is a package manager for windows and has a role in Ansible (win_chocolatey) that allows easy installation so we are going to start adding more tasks to the playbook.

The second task will be used a lot to obtain information about what is installed and what has been installed. Now we have to install cygwin and cyg-get in order to install additional packages in cygwin.

Once this is done we install the openssh and the rsync to be able to perform the rsync by ssh. We must also create a symbolic link, from the home path of the user of the computer to the cygwin user, to be able to do the rsync.

With this we are ready to make an rsync to another machine by ssh. We need to install a file compression software that is easy to use by command line, I have chosen the 7zip. We install it equally with chocolatey and add it to the path.

To compress I will use a ps1 script that I found on github. It can be used to make incremental backups or full backup. That’s my case, for which you have to delete a file that stores where we left the incremental backup, name backupconfig.ini

Now we only need to do the rsync itself, so we have to do two previous steps: first we create the ssh keys for the user of that machine, the public key must be hosted on the machine to which we are going to do the rsync as authorized_keys and we need to add the fingerprint of the destination machine to the known_hosts of the source machine, for this we use the following task.

To create the ssh key we use the command  ssh-keygen -t rsa -b 4096  the contents of the file id_rsa.pub will go to the file .ssh / authorized_keys folder of the destination machine. With this we can perform the rsync without asking for a password. Finally we perform the rsync of the 7z files that contain the backup folder and delete them once finished

And here’s how to make backups with Ansible on Windows. In part 2 we will see how to do it in Linux that, is much simpler 🙂

References

  • http://docs.ansible.com/ansible/intro_windows.html
  • https://github.com/traschke/7z-backup/blob/master/backup.ps1
  • https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1
  • http://www.virtualtothecore.com/en/configuring-windows-machines-for-ansible/
  • http://www.techrepublic.com/blog/the-enterprise-cloud/set-the-powershell-execution-policy-via-group-policy/
  • https://www.404techsupport.com/2012/10/use-group-policy-to-allow-ping-and-remote-management-on-windows-7/
  • https://wyssmann.com/my-first-steps-with-ansible/
  • http://nokitel.im/index.php/2016/11/09/how-to-manage-windows-server-2016-with-ansible/
  • https://superuser.com/questions/221136/bypass-a-licence-agreement-when-mounting-a-dmg-on-the-command-line
  • https://github.com/diyan/pywinrm
  • https://github.com/spencergibb/ansible-osx
  • https://github.com/ultimateboy/ansible.osx
  • https://github.com/lafarer/ansible-role-osx-defaults
  • https://github.com/geerlingguy/ansible-role-nodejs/issues/30
  • https://github.com/mas-cli/mas/issues/70
  • https://blog.vandenbrand.org/2016/01/04/how-to-automate-your-mac-os-x-setup-with-ansible/
  • https://github.com/ansible/ansible/issues/16478
  • http://www.grouppolicy.biz/2014/05/enable-winrm-via-group-policy/
  • http://oriolrius.cat/blog/2015/01/29/ansible-and-windows-playbooks/
  • https://github.com/Homebrew/brew/blob/master/docs/How-to-Create-and-Maintain-a-Tap.md
  • https://blog.vandenbrand.org/2016/01/04/how-to-automate-your-mac-os-x-setup-with-ansible/
  • http://docs.ansible.com/ansible/playbooks_best_practices.html
  • http://answers.unity3d.com/questions/1280198/unable-to-list-target-platforms-cant-build-into-an.html
  • https://social.msdn.microsoft.com/Forums/windowsserver/en-US/fb154aea-33ee-4182-a345-66f88a6769bc/allowunecrypted-winrm-property?forum=windowssecurity
  • https://www.groovypost.com/howto/join-a-windows-10-client-domain/

 

Install Python 3 coexisting with Python 2 in Linux

In this post I’m going to explain how to install Python 3 coexisting with Python 2 in Linux (Kubuntu 13.10 in my case). First step is install some dependencies.

After that, download from the website and uncompress the latest version of Python 3 in a folder of your choice.

Inside the uncompressed folder, we’re going to compile and install Python 3 with the following commands:

To make sure about the correct installation execute the following command  /opt/python3.3/bin/python3 and you must get something like that:

Also you should have brand new commands like python3  and python3.3. For the Python 3 modules management we’re going to install pip, but for Python 3 :-). Happily there is a debian package for that, use the following command  sudo apt-get install python3-pip to install it.

Finally we’re going to create a virtual environment of Python 3. There is two ways to do that. The first one, the hard way is specifing the python path that you want to use.

The second one, needs a previous installation of the virutalenv module using pip3  sudo pip3 install virtualenv . After you have installed it, to create Python 2 virtual environment you’ll must use virtualenv-2.7 and to create Python 3 virtual environment virtualenv-3.3. Enter the following command to get your Python 3 virtual environment.

Once you’ve created and set your Python 3 virtual environment, you’ll be able to install new modules using the pip command (inside our Python 3 virtual environment pip command is the same as pip-3.3). Let’s test it by installing the django module with  pip install django. After that, run the Python 3 console and write down the following instructions:

I hope it helps you!

Setup Android development environment in Debian 7 (Wheezy)

In this post I’m going to describe the steps for setup the Android development environment in Debian 7 (Wheezy).

Firstly we are going to change the apt repository list, to do this we’ll edit the source.list file

We comment the line/s of the installations cd/s, in my case:

And in the end of each debian repostory urls we are going to add “non-free contrib”, for example:

After that, we need to install the build-essential package that we are going to need for compile some stuff  apt-get install build-essential.

The next step, only going to be needed if as me, you are using a 64-bit Debian distribution. For the proper functioning of the Android SDK we need to install the ia32-libs package. To do this, the Debian’s package manager need to know that we want to install packages for another architecture using the dpkg command.

The fourth line is optional, in my case both packages was been requested.

Now we just need to download the last eclipse (http://www.eclipse.org/downloads/), the Android SDK (http://developer.android.com/sdk/index.html#download), uncompress these files, install the ADT plugin for ecplise from the update site (https://dl-ssl.google.com/android/eclipse/) and finally configure the ADT plugin setting where your SDK is.

I hope it helps!!