Last night I had the pleasure of presenting at Dot Net Notts (@dotnetnotts), a relatively new User Group in Nottingham.  Having been born in Nottingham, it was a really nice experience to go back visit some family and then present at this vibrant and welcoming new user group.

Last night I was talking about Telemetry and Logging experiences on a Cloud Platform (namely Microsoft Azure) and was trying to impart the benefit of some experiences running a large multi product platform.  Hopefully the talk was well received and everyone enjoyed it.  The talk was a different type to the majority I’ve given before and I’ve been wanting to deliver it for a while.  The content of the talk discussed mistakes that are easy to make, processes that can help, and some tips along the way for mitigating mistakes and providing a supportable approach to telemetry.  Here are some of the tips, conclusions and resources from the talk last night:

Logging Tips:

• Instrument for insight into application
• Capture inter-service application activity and latency
• Ensure level can be altered at run-time
• Abstract logging – gives agility to change framework
• ALWAYS ENABLE LOGGING

Logging Level guides from a talk by Scott Guthrie at NDC 2013

NDC 2013 – Scott Guthrie: Building Real World Cloud Apps With Windows Azure PT2

Conclusions

• Involve key stakeholders early in the design phase from a product and platform perspective
• May have to tell them what they need
• Consider telemetry needs at early stage of development
• Consider SLA want to provide and how to prove it
• Choose frameworks and services carefully
• Iterate repeatedly – requirements evolve during lifetime of a product/platform
• Telemetry is important – just as important as new product features

Resources

• Microsoft Azure CAT Team (https://twitter.com/MSAzureCAT) Guidance - http://bit.ly/azurecat
• Azure Telemetry Blog Series - http://bit.ly/AzureTelemetry
• Microsoft Azure Blog - http://azure.microsoft.com/blog
• Microsoft ALM Blog - http://bit.ly/vsalm
• The Fix It Sample Application (Building Real-World Cloud Apps with Windows Azure) - http://bit.ly/FixItApp and eBook - http://bit.ly/FixIteBook and videos - http://bit.ly/ScottGuNDC2013
• The Easy Way Of Collecting Security Logs From Your Windows Azure Roles - http://bit.ly/AzureSecurityLogs
• ELK Stack - http://bit.ly/ELKStack

Recently I’ve been working with Elasticsearch as I look to implement a logging solution using Logstash.  Elasticsearch is a popular, flexible open source real-time search and analytics engine and I’ve been looking at making use of the engine to perform full text searches and analytics against the mountains of log data we have on our platform.

As part of this work I’ve been setting up an Elasticsearch cluster of three Windows Server 2012 R2 Datacenter edition virtual machines in Windows Azure.  We’ve chosen Windows based virtual machines as the team who will support it have more experience and knowledge of Windows OS than Linux.  To get up an running I’ve used the following blog post from Thomas Ardal, which has a further link to info on installing Elasticsearch on the VMS – Running ElasticSearch in a cluster on Azure.

One of the elements of configuring the cluster concerns the discovery of nodes in the cluster.  By default Elasticsearch makes use of multicast to discover the nodes in the cluster, however multicast doesn’t appear to work fully in Azure at present, therefore Thomas’ blog post details making changes to the elasticsearch.yml configuration file to disable multicast and add in the node ip addresses manually. 

However on Friday (31st Jan 2014) Elasticsearch announced an Azure Cloud Plugin for Elasticsearch and the goal for this plugin is to enable automatic unicast discovery of Windows Azure Virtual Machines in the Cloud Service.  The article details how to set the plugin up in Linux Virtual Machines, but as I’ve already mentioned we are using Windows Server 2012 R2 for all of our virtual machines in the cluster.  Therefore I set off today to configure the plugin for running Elasticsearch on Windows.

1. Create Windows Azure VMs as detailed in Thomas’ post, installing Elasticsearch on each Virtual Machine you wish to use as a node.
2. Create a self-signed certificate for working with the Azure REST API using makecert (You will need the Windows SDK for this in order to make use of makecert).
3. Upload the .cer file to the Management Certificates (Settings->Management Certificates) store for your Azure subscription in the Azure Portal. 
4. Using Certificate Manager (certmgr.msc) export the certificate with the public key (in .pfx format) and give the certificate a password of suitable complexity.  Copy the pfx file to a folder on your virtual machines, e.g. C:\MyCertificates
5. Install the Azure-Cloud plugin on each of your Elasticsearch Virtual Machines by running the following command from a command point at your elasticsearch installation directory
   bin/plugin -install elasticsearch/elasticsearch-cloud-azure/1.0.0.alpha1
   
6. Next edit the Elasticsearch configuration file, Elasticsearch.yml, which is found in the config folder of your Elasticsearch installation on each virtual machine in your cluster.  At the bottom add the following configuration and update with the relevant settings:

   ################################# AZURE PLUGIN ###############################
   cloud:
     azure:
      keystore: <path to certificate i.e. c:/MyCertificates/mycert.pfx>
      password: <Password for certificate file>
      subscription_id: <your subscription id>
      service_name: <name of cloud service>
    
   discovery:
      type: azure

   for example:

   ################################# AZURE PLUGIN ###############################
   cloud:
     azure:
      keystore: c:/MyCertificates/mycert.pfx
      password: verystrongpassword
      subscription_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      service_name: myescluster
    
   discovery:
      type: azure

   Thanks to David Pilato and Thomas Conté for their help in getting the configuration correctly formatted and populated.

7. Start Elasticsearch on each server

That’s it all configured, if you’re using plugins to monitor your Elasticsearch cluster such as the Head plugin you should now see all of your nodes in your cluster listed, for example:

WARNING: As a side note, it’s VERY easy to break the syntax in yml files when not familiar with them, I’m now using Notepad++ to work with them and I’ve also been using a VS Extension – YamlDotNetEditor - when editing the yml file so I can reduce the risk of making mistakes.  The extension formats and lays out the file in a readable format with syntax highlighting (although this is currently not working in VS2013).

Are you interested in getting started with Windows Azure and have are free on Saturday April 27th 2013?  Then why not join me in Bradford at the Global Windows Azure Bootcamp being hosted by Black Marble?  I’ll be joining Microsoft Integration MVP, Robert Hogg, Windows Azure MVP, Steve Spencer and Microsoft Alliance Manager, Linda Hogg to deliver a one day deep dive class which will help you get up to speed with developing for Windows Azure.  This is a great opportunity to get real hands on experience with developing on Windows Azure from professionals who are actively developing solutions on the Windows Azure platform.  Windows Azure is a fantastic platform to develop on and we hope to see many of you at the event getting up to speed with development.

To register for the Bradford event please go to - http://www.blackmarble.co.uk/Register.aspx?Event=Global Windows Azure Boot Camp&Date=27-Apr-2013 and make sure to check out the list of pre-requisites for the bootcamp - http://globalwindowsazure.azurewebsites.net/?page_id=171 – please arrived prepared as we hate for you to miss out on the good stuff and spend the day getting your laptop ready .

If you’re not able to get to Bradford but would still like to take part in the Global Windows Azure Bootcamp be sure to check out all the locations worldwide where other events are running -http://globalwindowsazure.azurewebsites.net/?page_id=151

On Wednesday we celebrated the third birthday of NEBytes, a user group in the North East of England for Developers and IT Pros which I am a co-founder of.  We were delighted to partner up with the SharePoint User Group for the event and welcomed Dr. Bill Ayers to speak on Agile SharePoint Development.

In addition we’d planned to have a speaker visit to talk about Windows Azure IaaS at the event but unfortunately had problems sourcing a speaker due to a number of issues.  I therefore stepped in and reviewed some content from Microsoft and presented it at our event.  This was an interesting way to prepare for a presentation and I hope those who attended found the material and demos interesting and easy to follow.

I am currently spending a lot of time on the PaaS (Platform as a Service) side of Windows Azure and it was interesting for me to delve into the IaaS (Infrastructure as a Service) elements of Windows Azure.  There is quite a comprehensive offering and whilst I haven’t spent a huge amount of time in Amazon Web Services, apart from using an EC2 instance for Load Testing, I feel it’s growing fast and new features are coming on board frequently.

A few people asked for the slides after the presentation and here they are http://www.andrewwestgarth.co.uk/presentations/WindowsAzureVirtualMachines.pptx