Azure Automation

As indicated in my last blog post this is the first of two posts describing my experiences and learning in connection with setting up a Minecraft server.

In this one we will look at Azure Automation and create a small scheduled runbook or job that will ensure the server is closed down for the night (to save on the pocket money).

If you don’t have an Azure subscription, you can get a free trial.

I will not go into details of how to actually set up the Minecraft server as Jon Buckley has already created this excellent instruction video. If you know Azure and don’t want to see the whole video, the steps are the following:

  • Create a new VM using the Windows Server gallery image.
  • Create a new endpoint opening up port 25565.
  • Open up the Windows firewall on the VM to allow traffic to this port.
  • Download Minecraft.

To leverage Azure Automation, you’ll need to activate the preview feature. This can be done from the Preview Features page.

It may take a few minutes for the feature to be activated. Once available you will see a new menu item in the left navigation bar.

image

Select this and click the Create button at the bottom of the page to create a new Automation account.

I have create one called strobaek. Note, that the Automation feature is currently only available in the region East US.

image

Azure Automation authenticates to Microsoft Azure subscriptions using certificate-based authentication. You can create a new management certificate in a number of ways. I usually open up a Visual Studio command-prompt and issue the following command

makecert -sky exchange -r -n CN=KarstenCert -pe -a sha1 -len 2048 -ss My "KarstenCert.cer"

This will insert a new certificate in the Personal certificate store

image

Export this certificate twice, both without the private key (as DER encoded binary X.509 .CER) and with the private key (as .pfx file).

You should end up with something like this:

image

Now that we have a management certificate we need to upload it to the Azure Management Portal.

Log in – if you are not already – and select Settings in the left navigation bar; it is the last menu item.

Select Management Certificates from the top menu and click Upload at the bottom of the screen. Browse for you CER-file and select OK.

Make a note of the Subscription and Subscription ID as we will need these later (I have blanked out some of my subscription ID in the figure below)

image

OK, that was one part of the deal using the .CER file. Now for the second part using the .PFX file you also created. For your Azure Automation account to be able to authenticate to your Azure subscription, you’ll need to upload the certificate .PFX file. You’ll create what is known as an Asset in the Azure Automation account. This way it can be consistently leveraged across multiple runbooks.

Click on the Automation item in the left navigation bar and enter the Azure Automation account you created earlier. Click on the Assets tab and then Add Setting at the bottom. When prompted, select Add Credential.

image

On the Define Credential page, select Certificate in the Credential Type list and enter a name

image

Click the Next button and browse to the .PFX file to upload the certificate. Enter the password used while exporting the certificate and press OK.

image

Your new asset has now been created

image

Next step is to create a connection asset. Doing so allows you to easily relate your Azure subscription name, subscription ID and management certificate together as a centralized definition for use in all of your runbooks.

Again click Add Setting, but this time select Add Connection

image

On the Configure connection page, select Azure as the Connection Type and enter a Name that matches your Azure subscription name recorded earlier.

image

Click Next

Enter the name of the management certificate asset previously uploaded/created and enter your Azure subscription ID (which you should also have recorded previously).

We are now ready to create the actual runbook.

There is a few lines of code that are used to connect a runbook to your Azure subscription using the management certificate asset and connection asses that were previously defined. To promote easy maintenance of runbooks, it is recommended to centralize this code into one runbook, called e.g. Connect-Azure, that other runbooks can reference.

The Azure Automation team has made this approach super-easy by providing us with a standard runbook template on the Azure Automation Script Center.

Go to the script center and download the Connect-Azure runbook template.

On the details page of your Azure Automation account, click the Runbooks tab.

At the bottom of the page click the Import button. Browse to the Connect-Azure.ps1 file just downloaded and click OK to import the template.

image

On the Runbooks tab click on Connect-Azure to drill into the details of the runbook.

Then click the Author tab and click the Publish button at the bottom of the page to publish the runbook. Until this is done the runbook is in “draft” mode and can be edited, but not used.

When prompted select Yes to confirm that you really want to publish the runbook.

We now have the fundamentals for creating our own runbook.

Click New | App Services | Automation | Runbook | Quick Create

image

Enter a name, e.g. Stop-VMs and a description, e.g. ‘Stop all VMs at night’. Select your automation account from the drop down and verify the subscription is correct. Then click Create.

Note that runbook automation scripts are defined using PowerShell workflows. As such, the recommended practice is to name runbooks using a PowerShell verb-noun cmdlet naming convention.

On the runbook page you should see the new runbook after creation is done.

image

Drill into the detailed property pages of the runbook.

Click the Author tab and then the Draft tab to begin editing the PowerShell code for the new runbook.

The first thing to do is leverage the Connect-Azure runbook to connect to your Azure subscription. Inside the Workflow code block enter the following:

workflow Stop-VMs
{
    # Specify Azure Subscription Name
    $subName = '[Enter your Azure subscription name]'
    
    # Connect to Azure Subscription
    Connect-Azure -AzureConnectionName $subName
        
    Select-AzureSubscription -SubscriptionName $subName
}
Remember to replace the value for $subName with the correct value (which you recorded earlier).

Now that we are connected to the subscription we can enter the code to actually stop and deallocate the VMs.

    $vmList = ('App1','App2','App3','DC')
    $svcName = 'mycloudservice'
    
    foreach($vm in $vmList)
    {
        $anon = Get-AzureVM -ServiceName $svcName -Name $vm
        Write-Output $anon.Name $anon.InstanceStatus
        
        if ($anon.InstanceStatus -eq 'ReadyRole')
        {
            Stop-AzureVM -ServiceName $svcName -Name $anon.Name -Force
        }
    }

Update the two variables $vmList and $svcName with the name of the virtual machines you wish to stop and the name of the cloud service they live in.

The whole script is shown below for your convenience.

workflow Stop-MyVMs
{
    # Specify Azure Subscription Name
    $subName = '[Enter your Azure subscription name]'
    
    # Connect to Azure Subscription
    Connect-Azure -AzureConnectionName $subName
        
    Select-AzureSubscription -SubscriptionName $subName

    $vmList = ('App1','App2','App3','DC')
    $svcName = 'mycloudservice'
    
    foreach($vm in $vmList)
    {
        $anon = Get-AzureVM -ServiceName $svcName -Name $vm
        Write-Output $anon.Name $anon.InstanceStatus
        
        if ($anon.InstanceStatus -eq 'ReadyRole')
        {
            Stop-AzureVM -ServiceName $svcName -Name $anon.Name -Force
        }
    }
}

Click the Save button at the bottom of the page.

Once the runbook is saved you can test it to confirm that it runs successfully.

Click the Test buttton next to the Save button. NOTE: When you test the runbook it is actually executed against your subscription, hence if you test the new Stop-VMs runbook, your virtual machines will be stopped.

When the runbook is tested and confirmed that it executes successfully, it can be published.

Click the Publish button on the bottom toolbar (and confirm when prompted) and then click the Published tab to confirm that is has been published successfully.

image

The final step is to create a schedule and attach it to the runbook. This is to make sure the Minecraft server is automatically stopped and deallocated when not being used (read: when the kids are supposed to sleep). To execute a runbook on a scheduled basis, we can link the runbook to a recurring schedule.

Next to the Author tab you can see the Schedule tab. Click this.

image

Click Link to a New Schedule and give the schedule a name

image

and click next.

On the Configure Schedule page set type as Daily and a start time, e.g. 21:00. Note, the time is not adjusted for daylight saving. However, the time entered seems to be based on the time on the work station creating the schedule. If I enter 21:00 the runbook is executed at 21:00 CET which is my local daylight saving adjusted time.

image

Click OK and you are done!

The next post will look at how to use the Azure Management Libraries from a small .NET library to start and stop our virtual machine.

Posted in Azure | Leave a comment

Azure Automation, Azure Management Libraries and Minecraft

For a long time the “household trolls” have been bugging me about setting up their own Minecraft server. I finally got around to do this the other day, using Microsoft Azure for the hosting. All is working great, but it left me with two challenges, namely making sure it is turned off then not used, so nothing is charged for it and making it possible for the kids to actually start it up, without annoying me or giving them access to my whole Azure subscription.

The following couple of blog posts will address how we dealt with these two items.

The first will look at setting up and using Azure Automation to create a small runbook to turn off the VM each night, should it have been left on.

The second will show how I created a small utility running in the taskbar, giving the status of the VM and allowing the user to turn a give VM on and off and just this VM. To do this we will use the Azure Management Libraries.

Stay tuned 🙂

Posted in Azure | Leave a comment

Cleaning up an Azure Deployment

You most likely know this scenarios: you have deployed a nice environment into Windows Azure. Just a few AD/DC, a few front-end servers, a couple of application servers and of course two or three SQL Servers.

Now you don’t need the VMs any more and you want to remove everything. And you spend the rest of the afternoon clicking around in the Management Portal, first to remove all the VMs. They wait around until the Disks are no long registered as being attached to the VM and then delete the Disks and the underlying VHD-files.

Would it not be nice, if this could be done in a single command (or maybe two if you just wanted to remove the VMs)?

Look no further. I have created two small PowerShell scripts that will do exactly this: Remove VMs and remove Disks and underlying VHDs.

Let us first remove the VMs.

$remove = $false

if(-not $remove)
{
    Write-Host "Are you sure you want to do this?"
    Write-Host "Change bool to true"
    return
}

$serviceName = <Service Name>

$azureVMs = Get-AzureVM –Serviceame $serviceName | select Name

foreach($azureVM in $azureVMs)
{
    Remove-AzureVM -ServiceName $serviceName -Name $azureVM
}

I like to have a safe guard at the top of my scripts. Set the constant to the name of your Cloud Service (the $serviceName variable).

To remove all Disks and underlying VHD-files for VMs having belonged to a given Cloud Service run the following:

$remove = $false

if(-not $remove)
{
    Write-Host "Are you sure you want to do this?"
    Write-Host "Change bool to true"
    return
}

$serviceName = <Service Name>
$azureDisks = Get-AzureDisk | select DiskName, AttachedTo

foreach($azureDisk in $azureDisks)
{
    if($azureDisk.AttachedTo.HostedServiceName -eq $serviceName)
    {
        Remove-AzureDisk -DiskName $azureDisk -DeleteVHD
    }
}

If you have played around with the scripts for the automated Share Point deployment found at GitHub I have created a few scripts that from the created configuration files will Export all settings, remove the VMs and (re)deploy them. More about this in a later post.

Posted in Azure | Leave a comment

Installing SQL Server 2012 in a Mirror Setup

In my series on creating a SharePoint farm in Windows Azure we last time created the virtual machines for the two front-end servers, the two application servers and the three servers to be used by SQL Server.

In this sixth post we will look at how to enable SQL Server for high availability by enabling them in a mirror setup.

First of all, this is not really directly related to Windows Azure. The steps taken are the same as you would, should you enable a mirror on-premises, but as it is not something you do every day – at least I don’t – I thought it might be of interest.

Due to several factors, the structure on the underlying storage in Windows Azure being one of them, you cannot run a SQL Server Cluster in Windows Azure, so if you require redundancy and fast failover, you need something else, like a mirror.

There are two modes of database mirroring – synchronous and asynchronous. With synchronous mirroring, transactions cannot commit on the principal until all transaction log records have been successfully copied to the mirror (but not necessarily replayed yet). This guarantees that if a failure occurs on the principal and the principal am mirror are synchronized, committed transactions are present in the mirror when it comes online – in other words, it is possible to achieve zero data loss.

Synchronous mirroring can be configured to provide automatic failover, through the use of a third SQL Server instance called the witness server (usually hosted on another physically separate server). The sole purpose of the witness is to agree (or not) with the mirror that the principal cannot be contacted. If the witness and mirror agree, that mirror can initiate failover automatically. If synchronous mirroring is configured with a witness, the operating mode is known as high-availability mode and povides a hot standby solution. When no witness is defined, the operating mode is known as high-safety mode, which provides a warm standby solution.

With asynchronous mirroring there is no such guarantee, because transactions can commit on the principal without having to wait for database mirroring to copy all the transaction’s log records. This configuration can offer higher performance because transactions do not have to wait, and it is often used when the principal and mirror servers are separated by large distances (that is, implying a large network latency and possible lower network bandwidth). Consequently, the operating mode is also known as high-performance mode and provides a warm standby solution.

If a failure occurs on the principal, a mirroring failover occurs, either manually (in the high-performance and high-safety modes) or automatically (only in the high-availability mode). The mirror database is brought online after all the transaction log records have been replayed (that is, after recovery has completed). The mirror becomes the new principal and the applications can reconnect to it. The amount of downtime required depends on how long it takes for the failure to be detected and how much transaction log needs to be replayed before the mirror database can be brought online.

In the previous post on the subject we created three VMs and attached an extra disk. As was the case for the two domain controllers, you need to log on to the SQL Servers and attach the disk.

Before we begin I must make a comment about the screen shots. If some of the text is missing it is because I have removed it due to confidentiality related issues. I apologize.

First task is to install .NET 3.51. If this is not done the installation of SQL Server might hang. We do this by using Add Feature.

Open the Server Manager and select Manage and then Add Roles and Features.

image

The Add Roles and Features Wizard will be displayed

image

Click Next.

In the Select installation type dialog ensure the correct option is selected.

image

Click Next.

Select the server in the Select destination server dialog. Then click Next.

image

Select the feature (.NET Framework 3.5 Features)

image

 

image

Click Next.

You get a change to confirm the selections.

image

If you are satisfied click Install.

The installation will now begin. You can either Close the dialog right away or you can wait until the installation has completed.

image

We are now ready with the actual installation of SQL Server.

Download and attach media (here SQL Server 2012 SP1 Enterprise Edition)

image

Run the Setup.exe file.

Select Installation in the menu to the left.

image

On the installation page, select the New SQL Server stand-alone installation.

image

This will install the setup support files. Once that is done, click OK.

image

Accept the suggested product key or enter the correct one.

image

Click Next.

In the License Terms dialog accept the terms and click Next.

image

Ensure that all is green in the Setup Support Rules dialog. If the Windows Firewall rule is yellow it is most likely because port 1433 is not open. It will have no influence on the installation, but may be an issue later on.

Click Next.

image

In the dialog for Setup Role select the top option (SQL Server Feature Installation)

image

Click Next.

On the dialog for the Feature Selection select the required features. In my case I did not require Analysis Services nor Reporting Services, but I did select to install the management tools.

image

Click Next.

The setup process will now determine if any process will be blocked. If all is green click Next.

image

Accept the default setting for the instance configuration.

imageAcc

Click Next.

You should have enough space for the installation of the actual bits.

image

Click Next.

If you have installed SQL Server 2008 and R2 you will notice that the default values for the Server Configuration has changed.

You can keep the default settings, but if you plan to use this server in a mirror setup – which is the subject of this blog post – I will recommend that you use a domain account. It will make setting up the security during the mirror configuration so much easier. The reason being that the local account on Server 1 does not know anything about the local account on Server 2.

image

If you just keep the default values you can always change them later using the SQL Server Configuration Manager.

Click Next.

In the Database Engine Configuration dialog on the Server Configuration tab keep the default value for the Authentication Mode.

image

Select the Data Directories tab.

Change the Data root directory to the additional disk we attached. Again, if this had been a production setup, you would spread your directories over a lot more drives.

image

Select the FILESTREAM tab.

You want to select both the Enable FILESTREAM for Transact-SQL access and the Enable FILESTREAM for file I/O access options. You don’t need to enable the last one.

image

Click Next.

image

In the Error Reporting option, click Next

Setup will run some additional checks. If all is green in the Installation Configuration Rules click Next.

image

Click Install.

image

The installation will begin and it is time to go and get a cup of coffee.

If all goes as expected, you should have a lot of green markers and you can close the dialog and exit the setup.

image

One down and two more to go. Repeat the above process for the other two SQL Servers. Once all are installed we will have the primary, mirror partner and witness servers and we are ready to enable the mirror.

However, before actually do this, we need to install SharePoint. The reason is that the mirror is enabled by backing up and restoring databases, hence we need something “in” SQL so to speak. As I am not a SharePoint person, I will refrain from trying to describe the process.

Therefore:

And we have a working SharePoint installed in to the primary SQL Server (in this case SP-SQL1).

First step is to ensure that all SQL logins are present on the primary and mirror server.

Then we must ensure that all databases are running with recovery mode set to full. RDP into SP-SQL1 (the primary) and open op SQL Server Management Studio.

Ensure there are no data connections to the SQL Server (you may want to close down the SP site).

Execute the following T-SQL to set recovery mode:

USE master;
GO
ALTER DATABASE AdminContent SET RECOVERY FULL;

We then first backup the database

USE master;
GO

BACKUP DATABASE AdminContent
TO DISK = ‘F:\BackUp\AdminContent.bak’
WITH FORMAT
GO

and afterwards the log:

USE master;
GO

BACKUP LOG AdminContent
TO DISK = ‘F:\BackUp\AdminContent_log.bak’

GO

Copy the files to the mirror partner (SP-SQL2). Ensure they are placed in the same location, e.g. F:\BackUp in the above example. It is not a requirement, but the syntax of the T-SQL is slightly different if the location is different.

Connect to the mirror partner (SP-SQL2) from the open Management Studio or RDP into the server and open SSMS from here.

First we restore the database

USE master;
GO

RESTORE DATABASE AdminContent
FROM DISK = ‘F:\BackUp\AdminContent.bak’
WITH NORECOVERY
GO

and then the log

USE master;
GO

RESTORE LOG AdminContent
FROM DISK = ‘F:\BackUp\AdminContent_log.bak’
WITH FILE=1, NORECOVERY
GO

We are now ready to enable mirroring. In SSMS right click on one of the databases and select Tasks and then Mirrror….

image

In the Database Properties dialog, click Configure Security.

image

The first step in the configuration wizard is to decide whether or not a witness server should be used. As want automatic failover, I select the Yes option and click Next.

image

In the dialog to choose what servers to configure, ensure that all three are selected.

image

Click Next.

During the configuration you will have to connect to each SQL Server.

As I was working from SP-SQL1 and this is going to by my Principal server I am already logged in and can just click Next.

image

Next select the Mirror server. This is going to be SP-SQL2. Click Connect and enter your credentials. When done click Next.

image

Repeat the steps for the Witness server.

image

Click Next.

You now have to set up the Service Accounts information. If you your SQL Servers are running under a domain account this is going to be easy. If not you will afterwards have to enable the local accounts on each server. Doable, but a lot more hassle.

Enter the required information for each server.

image

Click Next.

You have made it to the end of the wizard and can review the information.

image

Click Danish. Sorry Finish. Bad joke.

If all goes well you should see something like the figure below. Click Close to close the dialog.

image

When you close the dialog you can either start the mirror right away or you can do so later.

I just hit the Start Mirroring button.

image

If the mirror process is able to start you will return to the initial dialog and the status should be Synchronizing.

image

Looking in SSMS at the databases you can also see that mirroring has been set up and is active.

image

This was a really long post for which I apologize.

There is a lot more to SQL Server mirroring than the above, but I hope it will serve as an introduction and maybe enable people not working with SQL on a daily basis to get up and running more quickly.

Posted in Azure | 20 Comments

Default size of OS-image in Windows Azure VMs

In my recent series of blog posts (first can be found here) on creating a SharePoint farm in Windows Azure, I showed how to extend the OS disk from the initial 30 GB.

With the general availability of virtual machines in Windows Azure the default size has been increased to 127 GB, so the trick is less of importance now.

Posted in Azure | Leave a comment

Rolling out Images for SharePoint farm in Windows Azure

In this fifth post in the series on creating a SharePoint farm in Windows Azure we will look at the main script used to create the VMs.

For those who might have missed the previous posts they are:

The script will automatically create and domain join the remaining 7 virtual machines required by our design: 2 web front-end servers, 2 application servers and 3 SQL Servers (one principal, one mirror and one witness).

An upcoming post will talk about the SQL Server installation, but just a few comments at this point. As described in the initial post, the SQL Servers are installed in a high-safety mode, that is the database mirroring session operates synchronously and uses a witness as well as the principal server and mirror server. For better performance mirroring can be enabled in high-performance mode where the database mirroring session operates asynchronously and uses only the principal server and mirror server. Note however, that the only form of role switching is forces service (with possible data loss).

To ensure we target the correct account we first set the active subscription:

# your imported subscription name
$subscriptionName = “MySubscription”
$storageAccount = “mystorageaccount”
Select-AzureSubscription $subscriptionName
Set-AzureSubscription $subscriptionName -CurrentStorageAccount $storageAccount

Next we set the Cloud Service Parameters. This is the “public” container holding all the VMs. It is also what allows up to load balance the two front-end servers as they will share the same VIP. Remember, that the service name must be unique, so SP-Service is most likely already taken.

# Cloud Service Parameters
$serviceName = “SP-Service”
$serviceLabel = “SP-Service”
$serviceDesc = “Cloud Service for SharePoint Farm”

Some more configuration options about base images and the virtual network.

# Image create in post: Creating a Base Image for use in Windows Azure
$spimage = ‘spbase100gbws2008r2’
$sqlimage = ‘base100gbsysprep’
$vnetname = ‘SP-VNET’
$subnetNameWFE = ‘SP-WFESubnet’
$subnetNameApp = ‘SP-AppSubnet’
$subnetNameSql = ‘SP-SqlSubnet’
$ag = ‘SP-AG’
$primaryDNS = ‘10.1.1.4’

As shown in the first post we will place the three layers (front-end, application and database) in three different availability sets.

# Availability Sets
$avset1 = ‘avset1’
$avset2 = ‘avset2’
$avset2 = ‘avset3’

The domain settings from when we configured the domain

# Domain Settings
$domain = ‘lab’
$joindom = ‘lab.azure’
$domuser = ‘administrator’
$dompwd = ‘P@ssw0rd’
$advmou = ‘OU=AzureVMs,DC=lab,DC=azure’

The location of the VHD-files

# MediaLocation
$mediaLocation =
http://mystorageaccount.blob.core.windows.net/vhds/

 

Next we set the configuration for the different VMs. Please note, that I have just set the size to Small and Medium.

Also note, that I have defined a prope port and path for the two front-end servers. This is what the Load Balancer (LB) uses to check if traffic should be forwarded to the servers.

It will also be obvious, that I have only create/attached one extra disk to the SQL Servers. In a production setup you should not place data, log and temporary files on the same disk.

# Create SP WFE1
$size = “Small”
$vmStorageLocation = $mediaLocation + “sp-wfe1.vhd”
$spwfe1 = New-AzureVMConfig -Name ‘sp-wfe1’ -AvailabilitySetName $avset1 -ImageName $spimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Add-AzureEndpoint -Name ‘http’ -LBSetName ‘lbhttp’ -LocalPort 80 -PublicPort 80 -Protocol tcp -ProbeProtocol http -ProbePort 80 -ProbePath ‘/healthcheck/iisstart.htm’ |
Set-AzureSubnet $subnetNameWFE

# Create SP WFE2
$size = “Small”
$vmStorageLocation = $mediaLocation + “sp-wfe2.vhd”
$spwfe2 = New-AzureVMConfig -Name ‘sp-wfe2’ -AvailabilitySetName $avset1 -ImageName $spimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Add-AzureEndpoint -Name ‘http’ -LBSetName ‘lbhttp’ -LocalPort 80 -PublicPort 80 -Protocol tcp -ProbeProtocol http -ProbePort 80 -ProbePath ‘/healthcheck/iisstart.htm’ |
Set-AzureSubnet $subnetNameWFE

# Create SP App1
$size = “Small”
$vmStorageLocation = $mediaLocation + “sp-app1.vhd”
$spapp1 = New-AzureVMConfig -Name ‘sp-app1’ -AvailabilitySetName $avset2 -ImageName $spimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Set-AzureSubnet $subnetNameApp

# Create SP App2
$size = “Small”
$vmStorageLocation = $mediaLocation + “sp-app2.vhd”
$spapp2 = New-AzureVMConfig -Name ‘sp-app2’ -AvailabilitySetName $avset2 -ImageName $spimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Set-AzureSubnet $subnetNameApp

# Create SQL Server1
$size = “Medium”
$vmStorageLocation = $mediaLocation + “sp-sql1.vhd”
$spsql1 = New-AzureVMConfig -Name ‘sp-sql1’ -AvailabilitySetName $avset3 -ImageName $sqlimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 100 -DiskLabel ‘datalog’ -LUN 0 |
Set-AzureSubnet $subnetNameSql

# Create SQL Server 2
$size = “Medium”
$vmStorageLocation = $mediaLocation + “sp-sql2.vhd”
$spsql2 = New-AzureVMConfig -Name ‘sp-sql2’ -AvailabilitySetName $avset3 -ImageName $sqlimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 100 -DiskLabel ‘datalog’ -LUN 0 |
Set-AzureSubnet $subnetNameSql

# Create SQL Server 3 (Witness)
$size = “Medium”
$vmStorageLocation = $mediaLocation + “sp-sql3.vhd”
$spsql3 = New-AzureVMConfig -Name ‘sp-sql3’ -ImageName $sqlimage -InstanceSize $size -MediaLocation $vmStorageLocation |
Add-AzureProvisioningConfig -WindowsDomain -Password $dompwd -Domain $domain -DomainUserName $domuser -DomainPassword $dompwd -MachineObjectOU $advmou -JoinDomain $joindom |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 100 -DiskLabel ‘datalog’ -LUN 0 |
Set-AzureSubnet $subnetNameSql

$dns1 = New-AzureDns -Name ‘dns1’ -IPAddress $primaryDNS

Last thing is to call New-AzureVM to actually create the VMs.

New-AzureVM -ServiceName $serviceName -ServiceLabel $serviceLabel `
-ServiceDescription $serviceDesc `
-AffinityGroup $ag -VNetName $vnetname -DnsSettings $dns1 `
-VMs $spwfe1,$spwfe2,$spapp1,$spapp2,$spsql1,$spsql2,$spsql3

 

Now go grab a cup of coffee and wait for your VMs to be provisioned, domain joined and started.

When done you should see something like the following in the PowerShell windows:

image

Looking in the portal:

image

In the next post we will look at how to set up the SQL servers in a mirror. Not really an Azure subject, but still something you what to do to ensure redundancy.

Posted in Azure | Leave a comment

Creating a DC/AD for use in Windows Azure

In the fourth post on the experiences gained during the creation of a SharePoint farm in Windows Azure will look at establishing the domain controllers and active directory.

The previous three were:

Most of the details in this post is really not Azure-specific. We are going to deploy a couple of VMs, but we have already seen how to do that in the last blog post. We are then going to promote these VMs to domain controllers and configure a new forest. If you know anything about infrastructure, you properly know more about this than I do. If not read on.

Open a Windows Azure PowerShell prompt. We first set the subscription name and storage account.

# your imported subscription name
$subscriptionName = “MySubscription”
$storageAccount = “mystorageaccount”

Select-AzureSubscription $subscriptionName
Set-AzureSubscription $subscriptionName -CurrentStorageAccount $storageAccount

We then set the image name, size of VM, location to store the VDH-file, what subnet to deploy the VMs into and finally the (local) admin password.

# Domain Controller Paramaters
$imageName = ‘a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201301.01-en.us-30GB.vhd’
$size = “Small”
$mediaLocation = “
http://mystorageaccount.blob.core.windows.net/vhds/”
$subnet = ‘SP-ADSubnet’
$password = ‘P@ssw0rd’

The command to get a list of available images from the gallery is:

Get-AzureVMImage | Select ImageName

We are going to install the domain controllers into their own could service. It could just as well be the one containing the rest of the servers for the SharePoint farm, but I personally prefer to have them separately. Remember that the service name must be unique.

# Cloud Service Paramaters
$serviceName = “DC-Service”
$serviceLabel = “DC-Service”
$serviceDesc = “Cloud Service for DC for SharePoint Farm”
$vnetname = ‘SP-VNET’
$ag = ‘SP-AG’

The VNET and affinity group are the ones created during the creation of the VNET.

The configuration for the first domain controller. Notice that we add an extra disk to both VM. This if for the (AD) global catalog.

# Create VM Configuration (DC1)
$vmName = ‘sp-dc1’
$vmStorageLocation = $mediaLocation + “sp-dc1.vhd”
$dc1 = New-AzureVMConfig -Name $vmName ‘
-InstanceSize $size ‘
-ImageName $imageName ‘
-MediaLocation $vmStorageLocation |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 20 -DiskLabel ‘data’ -LUN 0

Add-AzureProvisioningConfig -Windows -Password $password -VM $dc1
Set-AzureSubnet -SubnetNames $subnet -VM $dc1

Configuration for the second domain controller:

# Create VM Configuration (DC2)
$vmName = ‘sp-dc2’
$vmStorageLocation = $mediaLocation + “sp-dc2.vhd”
$dc2 = New-AzureVMConfig -Name $vmName ‘
-InstanceSize $size ‘
-ImageName $imageName ‘
-MediaLocation $vmStorageLocation |
Add-AzureDataDisk -CreateNew -DiskSizeInGB 20 -DiskLabel ‘data’ -LUN 0

Add-AzureProvisioningConfig -Windows -Password $password -VM $dc2
Set-AzureSubnet -SubnetNames $subnet -VM $dc2

And finally we execute the New-AzureVM command.

# Create the DCs
New-AzureVM -ServiceName $serviceName -ServiceLabel $serviceLabel ‘
-ServiceDescription $serviceDesc -AffinityGroup $ag -VNetName $vnetname -VMs $dc1, $dc2

If all goes as expected we will have two new VMs. Looking under virtual machines in the Management Portal should give us something like the following:

 

image

 

Under Disks you can see that the extra data disk has been deployed as well as the OS disk.

 

image

 

And if you look under Cloud Services

 

image

 

The next step is to attach and format the data disk.

Open a remote desktop connection in to SP-DC1.

Go to Disk Management

image

The Initialize Disk dialog will pop up

Ensure the Disk is selected and press OK.

image

Right click on the unallocated disk (most likely Disk 2) and select New Simple Volume

image

Click Next on the welcome screen

image

Accept the default values and press Next.

image

Assign a drive letter and click Next.

image

Format the partition and click Next.

image

On the final screen review the settings and click Finish.

image

Once done you will have a nicely formatted disk ready to be put to use.

Viewed from the Disk Manager

image

And the file explorer

image

You need to preform the same steps for the second domain controller, so open a remote desktop connection into SP-DC2 and repeat the above steps.

The next thing to do is promote the server to a domain controller. The procedure for doing this has changed slightly going from Windows Server 2008 to Windows Server 2012.

A good guild with additional references can be found here.

The first thing to do is install the Role Active Directory Domain Services.

Open the Server Manager and select Add roles and features.

image

Click Next in the “Before you begin” dialog. You may want to check the “Skip this page” checkbox.

image

On the Select installation type ensure the first option is selected.

image

Select the local server as the destination server

image

In the dialog for selecting roles, select the Active Directory Domain Services

image

As soon as you make the selection the following dialog will pop up asking you add the required roles and features.

Click Add Features to accept.

image

Ensure the Active Directory Domain Services is selected and click Next.

image

In the select features dialog just click Next.

image

Click Next in the Active Directory Domain Services dialog.

image

Confirm the different selections and click Install.

image

The installation will now commence and you can follow the process.

image

As stated in the dialog you can close the wizard.

Once the installation process is complete you will be notified in the Server Manager.

Click the Promote this server link to promote the server to a domain controller.

image

This will start the AD DC Configuration Wizard.

Select Add a new forest and enter the Root domain name. Once the name is entered the Next button can be pressed.

image

Set the different Domain Controller Options and enter the DSRM password. Then click Next.

image

Click Next in the DNS Options dialog.

image

In the Additional Options dialog enter the NetBIOS domain name and click Next.

image

In the Paths dialog you have to select the location of the AD DS database, the log files and the SYSVOL. I have placed them on the extra disk we instantiated above. Click Next afterwards.

image

You can review the options and selections you have made in the dialog before actually starting the process. When satisfied, click Next.

image

Before the system will promote the server to a domain controller it will perform a number of prerequisites checks. If all looks good, press Install.

image

The server will reboot once the installation process is finalized. When it is up again you can log in with your AD credentials.

How that you have a running domain controller you can add the second one to the forest to ensure redundancy.

The first initial steps are the same: attach and format disk and install the Role Active Directory Domain Services. Once this is done promote the server to DC.

When you get to the Deployment Configuration you should not add a new forest, but add a domain controller to an existing domain.

Enter the domain name you specified during the configuration of the primary domain controller and click Next.

image

Select options and enter credentials; then press Next.

image

Click Next in the DNS Options dialog.

image

In the Additional Options dialog, select to replicate from Any domain controller and click Next.

image

As was the case with the primary domain controller we place the database, the log files and the SYSVOL on the extra disk. Click Next after this has been set.

image

Review the configuration and click Next to perform the prerequisites check.

image

If all is green click Install to begin installation.

image.

When done we now have two domain controllers and an active directory ready to be configured.

image

Again this turned out to quite a long post. As stated in the beginning most of the steps are really not Windows Azure specific.

This stresses a very important point, namely that running a virtual machine in Windows Azure is just as easy as running a virtual machine on-premises or at a remote branch office.

We have now in a number of posts worked our way toward the main script or workload: creating the remaining 7 virtual machines that together with our two domain controllers will make up our SharePoint farm/environment.

The next post will focus on the PowerShell script to do this. It will turn out to be very similar to the one used above.

Posted in Azure | Leave a comment

Creating a Base Image for use in Windows Azure

This is the third blog post on the subject of creating and deploying a SharePoint farm in Windows Azure. The previous two are:

    In this post we will create an image that can be used when creating virtual machines. The advantage of this is, that it gives you the ability to “pack” the image/disk, both with software (installed or just the binaries) and – in our case – with an extended OS-disk.
    I assume that you have followed the initial steps and created the VNET, the storage account and the affinity group.
    We will use one of the images supplied in the gallery. To get the available images, submit the following command from a Windows Azure PowerShell prompt:

Get-AzureVMImage | Select ImageName

This will give you something similar to:

image

The name of the VM, Cloud Service and VHD-disk in the script below is not important as we will delete them later.

# Your Storage account
$storageAccount = “mystorageaccount”

# Your subscription name
$subscriptionName = “MySubscription“

Select-AzureSubscription $subscriptionName
Set-AzureSubscription $subscriptionName -CurrentStorageAccount $storageAccount

# VM Paramaters
$imageName = “a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201301.01-en.us-30GB.vhd”
$size = “Small”
$mediaLocation = “
http://mystorageaccount.blob.core.windows.net/vhds/
$subnet = “SP-AppSubnet”
$password =
“P@ssw0rd” # This will be the password for the VM

$vmName = “sp-base”
$vmStorageLocation = $mediaLocation + “sp-base.vhd”
$bi = New-AzureVMConfig -Name $vmName ‘
-InstanceSize $size -ImageName $imageName ‘
-MediaLocation $vmStorageLocation
Add-AzureProvisioningConfig -Windows -Password $password -VM $bi
Set-AzureSubnet -SubnetNames $subnet -VM $base

# Cloud Service Paramaters
$serviceName = “BaseServiceImage”
$serviceLabel = “BaseServiceImage”
$serviceDesc = “Base Image. Will be deleted”
$ag = “SP-AG”

# Create the Base VM
New-AzureVM -ServiceName $serviceName ‘
-ServiceLabel $serviceLabel  ‘
-ServiceDescription $serviceDesc `
-AffinityGroup $ag -VMs $bi

We now have a plain VM running Windows Server 2012.

The following will be displayed in the PowerShell windows

image

If you log on to the management portal you will see the new VM under virtual machines. I have removed the name of the subscription due to certain confidentiality considerations, hence the black bar.

image

image

Next step is to extend the disk.

First we need to delete the VM just created. Ensure it is selected and then click Delete from the bottom menu.

image

and confirm

image

The system will begin to remove the VM.

image

and after a short while

image

Next step is to delete the disk.

Select Disk from the top menu

image

If the delete process is not completed you will see that the VM is still having a lease on the disk

image

Wait until the lease has expired

image

Delete the disk, but retail the VDH-file. This is very important as this is the one we wish to extend.

image

If you look in your storage account under the container vhds you can see, that the VHD-file is still there

image

Maarten Balliauw has created a small utility that will change the header information on the VHD-file allowing us to extend it. The maximum size is 127 GB; we will make it 100 GB.

Download the WindowsAzureDiskResizer tool from GitHub.

The syntax is the following:

image

You can get the accountname and accountkey from the storage page in the management portal.

As stated previously I set the size to 100

image

Before the tool was executed the VHD-file looked like this. Notice the size.

image

After running the tool the picture is the following:

image

Next step is to create an image using the VDH-file.

Select Images from the top menu under Virtual Machines.

image

and then click Create in the bottom menu

image

Enter a name and browse to the sp-base.vhd file

image

You need to check the “I have run Sysprep” even though that is not the case.

When the process is complete you can see the new image is available

image

We now want to create a new VM based on this new image, extend the disk from the Disk Manager, copy any software onto the disk or install it and when capture that VM.

I will show how this is done using the management portal, but you could amend the script above and just set the image name to

$imageName = “spbaseimage”

Select Virtual Machine Instances from the top menu

image

and then New from the bottom one.

image

You want to create a new VM from the gallery

image

Using your new image (the other two images are some I have created previously)

 

image

Give the VM a name, password and select the size

image

Give the VM a public facing DNS name and place it in the VNET. As we are actually not going to use this VM for anything other than begin the “template” for further work, it is more to ensure it is placed in our storage account.

 

image

Click OK to kick off the creating process.

Once the VM has been provisioned you want to open a Remote Desktop session (RDP) and log into it.

Enter the Disk Management

You can see the “original” 30 GB and the new unallocated 70 GB

image

Right click on C and select Extend Volume.

image

This will open up the Extend Volume Wizard.

image

Click Next.

Select the disk and the maximum size

image

and click Next again.

image

Click Finish to complete the wizard.

You now have a single OS volume of 100 GB.

image

The final step is to run Sysprep.

This is usually found in C:\Windows\System32\sysprep

Remember to set the Shutdown Option to Shutdown.

image

Press OK.

You can now Capture the VM. Select it and click Capture from the bottom menu. The VM has to be stopped before you can select the menu item.

image

Give the image a name and select the Sysprep option.

 

image

This turned out to be quite a long post.

In the next one in the series we will create the virtual machines for the two domain controllers and look at how to promote them to DCs.

Posted in Azure | Leave a comment

Creating a Virtual Network in Windows Azure

As announced in my last post I will in a number of blog posts walk through the steps taken to create a SharePoint farm in Windows Azure.

This post will explain how to setup a Virtual Network (VNET) in Windows Azure. It can be done in a number of ways the two obvious ones being either through the portal or using PowerShell. I will concentrate on the later.

The following will assume you have a Windows Azure subscription. If not sign up for on.

It will also assume that you have downloaded and install the PowerShell cmdlets for Windows Azure. If not get them here.

You have to be very careful when creating or changing the VNET settings. There is currently no validation so if you mess up, you will potentially take the whole farm off line. I usually download the existing setting, make the required modifications and then apply the settings again.

In the script shown below, we will just apply the settings contained in an XML-file, so we assume the download-amend has taken place.

Let us first look at the VNET configuration. The table below shown the configuration as described in the introduction post.

VNET Configuration
<NetworkConfiguration
xmlns=”http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” >
<VirtualNetworkConfiguration>
<Dns>
<DnsServers>
<!– List as many DNS servers as you need. –>
<DnsServer name=”SP-DNS” IPAddress=”10.1.1.4″/>
</DnsServers>
</Dns>
<LocalNetworkSites />
<VirtualNetworkSites>
<VirtualNetworkSite name=”SP-VNET” AffinityGroup=”SP-AG”>
<AddressSpace>
<AddressPrefix>10.1.0.0/16</AddressPrefix>
</AddressSpace>
<Subnets>
<Subnet name=”SP-ADSubnet”>
<AddressPrefix>10.1.1.0/24</AddressPrefix>
</Subnet>
<Subnet name=”SP-AppSubnet”>
<AddressPrefix>10.1.2.0/24</AddressPrefix>
</Subnet>
<Subnet name=”SP-WFESubnet”>
<AddressPrefix>10.1.3.0/24</AddressPrefix>
</Subnet>
<Subnet name=”SP-SqlSubnet”>
<AddressPrefix>10.1.4.0/24</AddressPrefix>
</Subnet>
</Subnets>
<DnsServersRef>
<!– This is needed to reference the DNS servers listed above –>
<DnsServerRef name=”SP-DNS” />
</DnsServersRef>
</VirtualNetworkSite>
</VirtualNetworkSites>
</VirtualNetworkConfiguration>
</NetworkConfiguration>

As you can see the address scope of the VNET is defined as well as the four subnets. We have also defined a DNS server. The reason we can do this and be sure of the IP-address even before we have deployed the actual DC-servers is due to the way Windows Azure allocate IP-addresses. Azure will reserve the first three in a subnet for internal use, so the first available one will always be .4. Because we have put the (two) domain controllers into their own subnet we know it will always have the IP-address 10.1.1.4.

The first thing you want to do is download the publishing settings for your subscription. Open an Internet Explorer browser and go to https://windows.azure.com/download/publishprofile.aspx.

Open an elevated Windows Azure PowerShell prompt.

If you have not executed any PowerShell before you may have to set the execution policy to RemoteSigned. This is done with the following command:

Set-ExecutionPolicy RemoteSigned

You import the publishing settings using the command:

Import-AzurePublishSettingsFile ‘[YOUR-PUBLISH-SETTINGS-PATH]’

You need the name of your subscription and it can be obtained with the following command:

Get-AzureSubscription | select SubscriptionName

You also need to decide in what data center your solution should be hosted. A list of the available locations can be found executing the command:

Get-AzureLocation | select displayname

We are now ready to create the VNET.

A few things to note. We first create a so called affinity group. Affinity groups are a way to physically group Windows Azure services together at the same data center to increase performance.

The name of the variable AGName must corresponded with the the attribute AffinityGroup in the XLM-file containing the VNET configuration.

The name of the storage account must be globally unique. You will get an error if you try to select one that is not available.

Now execute the following:

# Affinity Group parameters
$AGLocation = “West Europe”
$AGDesc = “Azure Affinity Group”
$AGName = “SP-AG”
$AGLabel = “SP-AG”

# Create a new affinity Group
New-AzureAffinityGroup -Location $AGLocation -Description $AGDesc `
-Name $AGName -Label $AGLabel

$storageAccount = “[Storage Account Name]”
$label = “SharePoint Storage Account”

# Create storage account
New-AzureStorageAccount -StorageAccountName $storageAccount `
-Label $label -AffinityGroup $AGName

# Your subscription name
$subscriptionName = “[Name of your subscription]”

Select-AzureSubscription $subscriptionName
Set-AzureSubscription $subscriptionName -CurrentStorageAccount $storageAccount

# Clear current settings
Remove-AzureVNetConfig -ErrorAction SilentlyContinue

# Apply new network
$configPath = (Split-Path -Path $MyInvocation.MyCommand.Definition -Parent) `
+ “\SharePointFarmVNET.xml”

Set-AzureVNetConfig -ConfigurationPath $configPath

You can check the result by the following command:

# Get-AzureVNetConfig | Select -ExpandProperty XMLConfiguration

Or you can log into the management portal. If all went well you should see something along the lines of:

image

 

image

 

In the next post we will create the base image(s) and extend the OS-drive from the default 30 GB to 100 GB.

Posted in Azure | Leave a comment

Creating a SharePoint Farm in Windows Azure

I have recently been involved in a (pilot) project moving a public facing SharePoint farm to Windows Azure. This will be the first of a series of blog posts “from the trenches” about the different steps taken to achieve this goal.

First a few words on the architecture. For the pilot it was decided that a separate Active Directory forest would be created in Azure and as no other on-premise resources was required we avoided the task of setting up a VPN. Time permitting I may describe these tasks in a future blog post.

A few words on availability sets and Windows Azure. To ensure the availability of the application you would use multiple Virtual Machines (VMs). By using multiple VMs, you can make sure that your application is available during local network failures, local disk hardware failures, and any planned downtime that the platform may required.

You manage the availability of your application that uses multiple VMs by adding the machines to an availability set. Availability sets are directly related to fault domains and update domains. A fault domain in Windows Azure is defined by avoiding single points of failure, like the network switch or power unit of a rack of servers. In fact, a fault domain is closely equivalent to a rack of physical servers. When multiple virtual machines are connected together in a cloud service, an availability set can be used to ensure that the machines are located in different fault domains. Also, by placing the VMs into the same availability set, you ensure that the fabric controller will never shut all of them down at the same time, e.g. for maintenance of the OS.

In all we defined four availability sets, one for the domain controllers and one for each of the tiers in the applications: The frontend servers, the application servers and the database servers.

To ensure redundancy and automatic failover the SQL Servers are established in a mirror setup with a witness server.

All VMs are placed in a Virtual Network (VNET), with each tier in its own subnet; the domain controllers also got their own. In the current release of Windows Azure all VM can by default see each other and it is not easily done to change this, but this is being worked on for future releases, hence the sub-netting.

The address scope for the VNET is 10.1.0.0/16.

The table below gives the VNET configuration:

Name Description Address Scope
VNET Main VNET 10.1.0.0/16
ADSubnet AD/DNS subnet 10.1.1.0/24
AppSubnet Application server subnet 10.1.2.0/24
WFESubnet Frontend server subnet 10.1.3.0/24
SQLSubnet SQL Server subnet 10.1.4.0/24

The figure below shows the topology of the VMs in Windows Azure.

The internal load balancer in Windows Azure will distribute load between the front-end servers.

image

 

The following table shows the configuration of the VMs.

Name Description OS Disks
DC1 Domain Controller Windows Server 2012 1 x 30 GB (OS)
1 x 20 GB
DC2 Domain Controller Windows Server 2012 1 x 30 GB (OS)
1 x 20 GB
WFE1 Frontend Server Windows Server 2008 R2 1 x 100 GB
WFE2 Frontend Server Windows Server 2008 R2 1 x 100 GB
App1 Application Server Windows Server 2008 R2 1 x 100 GB
App2 Application Server Windows Server 2008 R2 1 x 100 GB
SQL1 Primary Database Server Windows Server 2012 2 x 100 GB
SQL2 Secondary Database Server Windows Server 2012 2 x 100 GB
SQL3 Witness Database Server Windows Server 2012 2 x 100 GB

Due to business requirements SharePoint 2010 is used. The SQL Server is 2012 Enterprise Edition.

The size of the OS drive on the images supplied by the gallery in Windows Azure is only 30 GB. To ensure we had enough space “to play around” I created two base images (one for Window Server 2008 R2 and one for Windows Server 2012) each having 100 GB on the OS-drive. As I did not want to upload 100 GB I used a small trick and utility to change the header information of the VHD-file, hence expanding the drive from the original 30 GB to 100 GB. More about this in one of the future posts. All the required bits were placed on the base images. This way we only had to download and distribute them once.

The two domain controllers were created from the images supplied by the gallery.

In the next post I will look at creating the VNET.

Posted in Azure | Leave a comment