About ten years ago, a group of engineers from different storage companies got together to create a powerful management abstraction that would enable tools for centralized storage environments like storage area networks (SAN) to become less vendor-specific but still allow any vendor to fully surface the capabilities of their storage hardware. One of the original goals was to build a model that relied on industry standards, largely from the DMTF, W3C and IETF. This included using a common model and schema (the Common Information Model or CIM). Unfortunately a model and schema only get you so far; they don’t give you something that can be implemented. You need a transport and a language to encapsulate the model. At the time, the logical choice for encoding the data was XML, specifically CIM-XML (also standardized in the DMTF) and the logical transport was HTTP. Collectively these technologies are known as Web-Based Enterprise Management, or WBEM.

[There are other choices for transport now available, such as WS-Management, a SOAP protocol that is also on the way to becoming an international standard, and the Windows-only WMI COM implementation of WBEM. I’ll speak more about those technologies at a later time, as they are also key to Microsoft’s standards-based management approach.]

The direction had been set in the Storage Network Industry Association, but the work moved into a private consortium known as PDP for reasons that I won’t go into here. A few years later, the group handed the work back to SNIA as a draft called Bluefin, which later became the Storage Management Initiative Specification, or SMI-S 1.0.

So what is SMI-S? Well, it’s basically a cookbook that explains how to use the model elements to do useful work, like discover and provision storage. Actually, it has grown into multiple cookbooks now, divided into topics like Common Profiles, Block, Filesystems, Fabric, etc. The SNIA has active working groups updating the specification every year or two to keep up with the latest technologies. The current (recently) published version is 1.6.0, with some updates expected by the end of the year.

A closer look

Let’s look at the different components that make up SMI-S:

Providers

An SMI-S provider is a software component produced by or for a particular storage product or family of products. It implements an HTTP server, it can parse XML requests, and it knows how to control those specific devices which usually have their own unique management interfaces. The provider can be embedded in the firmware or operating system of the device itself, or it can exist as a standalone application running on a general purpose computer operating system. The latter model is called a “Proxy” provider, and it is the most common way to implement SMI-S providers at this time.

Profiles

Profiles are a way of describing what components of the schema will be used to describe a particular management domain. For example, there is an Array profile. If a vendor claims support for the Array profile, they also agree to implement the classes defined in that profile, the specific properties and methods of those classes deemed necessary or optional to do useful work, and they also have to implement other related component profiles or subprofiles.

Profiles and subprofiles also tell you what the array can do. If the array can perform snapshotting or cloning of storage volumes, the provider can advertise that capability by claiming support for Replication Services. Application developers then know that they can use the CreateElementReplica method to create a new copy of the storage volume – an incredibly powerful capability used heavily in virtual machine management.

Profiles were created by SNIA and also by the DMTF. In some cases, SNIA specializes the profiles for use with SMI-S.

Classes

A class is a set of properties and methods that pertain to a specific object, or to an aspect of an object. Classes typically inherit properties from base or superclasses. An example class is a Disk Drive (in the CIM model, CIM_DiskDrive). Some properties of the disk drive might be what kind of drive it is (SSD, hard disk, etc.), how fast it spins, and what it’s doing right now.

A vendor can introduce new properties and methods by creating their own version of a class, inheriting from the base class and then adding their uniqueness to their derived class. In that way, the model allows extensibility while still preserving the common functionality that any application can use without understanding the specific value-added capabilities of the device it is trying to manage.

Most of the classes used by SMI-S are standardized in the DMTF CIM schema. There are over 1500 classes already defined, and new classes and additions to existing classes can happen frequently as the schema gets updated about three times a year.

The provider creates an instance of a class when it represents the actual objects. In our disk drive example, there will be an instance of the CIM_DiskDrive (or vendor-derived version of that class) for each physical drive in the storage array. An enumeration operation will retrieve all the instances, or a specific instance can be retrieved by itself.

Associations

It isn’t really enough to just count up all the objects of a particular class. Objects of one class might be related to another class and this is represented by associations. So a disk drive would be associated directly or indirectly to the object that represents the storage array. That way you can start at the top and find the related objects, or you can go from the bottom up to the top.

What Microsoft is doing with SMI-S

Microsoft has become an active participant in the SNIA technical working groups and also in the SMI-Lab plugfests. Over the past few years we have been testing hardware vendors’ providers with our client interfaces, first with System Center Virtual Machine Manager 2012, and later with Windows Server 2012. One goal was to make sure the most important scenarios for these products will work well with our interpretation of the specifications. And another objective was to allow for extensibility so that anything exposed through SMI-S could be leveraged on Windows without having to redo the considerable plumbing for interacting with a remote WBEM client. I’ll present more about this “passthrough” capability in a future blog.

What it means for Windows customers

As part of our commitment to management standards, Microsoft developed support for SMI-S and now includes this support with VMM 2012 and Windows Server 2012. Working in conjunction with the new storage management interfaces in Windows Server (see the references below), SMI-S enables command line access and easy scripting of many common storage tasks using Windows PowerShell and can also be used with the newly enhanced File and Storage Services role in Server Manager.

SMI-S support is implemented as an optional feature in Windows Server called Windows Standards-Based Storage Management. Keep in mind that it is designed for consolidated storage environments, and although most providers support simultaneous access from multiple clients, it is a best practice to also centralize the management of those devices. It would be unnecessary (and unwise) for the majority of servers to all be managing the same resources individually.

What it means for Hardware vendors

Because SMI-S is an industry standard built on top of other standards and widely implemented in other environments (such as VMware® and AIX®), vendors no longer need to produce proprietary providers for one particular operating system (or release). They can expose all of their product’s goodies using one provider as extensibility is inherent in the object-based approach of CIM. As client applications learn to take advantage of the advanced features, they can do so without resorting to yet another provider-model. CIM also has a good backward and forward compatibility story.

Next up…

On Windows Server 2012, once an SMI-S Provider is configured for use with the service, the storage will be manageable using the new Windows Storage Management API, and the Storage module in Powershell.

In the next post, I will explain how to get started using SMI-S with Windows Server 2012. Some assembly is required, but you should be up and running in a few minutes.

References

To see more about Microsoft’s commitment to standards and storage management, refer to Jeffrey Snover’s blog on this topic.

System Center 2012 Virtual Machine Manager also uses SMI-S providers. See http://technet.microsoft.com/library/gg610600.aspx for more information about storage automation and VMM.

Refer to the following section on technet for more information about the Storage cmdlets http://technet.microsoft.com/en-us/library/hh848705.aspx or the Storage Management team blog for the latest information at http://blogs.msdn.com/san

Learn more about the Storage Management Initiative at http://snia.org/forums/smi.

SMI-S is based on many industry and worldwide (ISO) standards:

CIM, WBEM and WS-Management are the products of the Distributed Management Task Force. For more information, see the DMTF website.

The Service Location Protocol, Version 2 (SLPv2), and Transport Layer Security (TLS, including Secure Sockets Layer or SSL) are protocols from the Internet Engineering Task Force (IETF).

The current specification for Hypertext Transport Protocol (HTTP) is a joint effort by the IETF and the World Wide Web Consortium (W3C).

In my last blog, I discussed the industry standard Storage Management Initiative (SMI-S) in general terms. Microsoft started supporting SMI-S for storage provisioning in System Center 2012 Virtual Machine Manager, released earlier this year. With the upcoming Windows Server 2012, SMI-S support will be available to all of our server customers. Coupled with the new Storage Management API (SMAPI), which consists of new WMI interfaces and cmdlets, it is possible to manage SAN or direct attached storage in a vendor-independent fashion, and also in a system-independent fashion if you have more than Windows in your datacenter. The new File and Storage Services canvas in Server Manager can take advantage of SMI-S providers, giving you a GUI for managing basic array functionality right out of the box.

SMI-S is based on the DMTF CIM Model and Schema, and as such it can support very complex environments. The SMAPI simplifies that by hiding a lot of the details and relationships that go into the model. Many vendors took part in the formation of SMI-S. Each vendor has to create one or more SMI-S providers for their products and customers need to obtain the providers from their storage vendors. But first, let’s talk about getting SMI-S going on Windows Server 2012.

Adding SMI-S support to Windows Server 2012

The Windows Standards-Based Storage Management Service is the optional feature for Windows Server only, which communicates with SMI-S providers (in SMI-S speak, it is a “client”). I will refer to it below as the “Storage Service”. It is not installed by default, so you will need to add the feature by using either Server Manager or a PowerShell cmdlet.

The diagram below shows the full architecture of the new Storage Management infrastructure in Windows. This blog focuses mostly on the lower left area highlighted by the red box.

Figure 1 Architecture of Storage Management on Windows Server 2012

The Storage Service also providers other functionality such as extensive (and secure) caching of management objects surfaced by one or more SMI-S providers, handling of dynamic events through SMI-S indications, managing asynchronous tasks and secure credential management. [Windows also supports a different hardware provider model, known as Storage Management Provider (or SMP). This blog only addresses the SMI-S support, and the Storage Service translates between SMI-S and SMP for you.]

You should think carefully about where you want to install this service – in a datacenter you would centralize the management of your resources as much as practicable, and you don’t want to have too many points of device management all competing to control the storage devices. This can result in conflicting changes and possibly even data loss or corruption if too many users can manage the same arrays. Multiple levels of permissions can help mitigate this possibility. SMI-S providers also typically manage more than one hardware device so you don’t need to install one provider per array.

Using Server Manager

Server Manager has been redesigned from the ground up for Windows Server 2012. It is now a tool that can manage across many server instances. It is one way that you can install the Windows Standards-Based Storage Management Service.

Server Manager typically opens when you log in to the Windows Server as an administrator. All actions described below must be performed with administrator privilege (Windows may prompt to elevate privilege if necessary).

Figure 2 Server Manager Dashboard

Pick the server you wish to install on to begin the feature installation (here I chose the Local Server but you could install on any other server being managed). Click on Manage and select Add Roles and Features:

Figure 3 Add Roles and Features to the Local Server

Continue with the Wizard, selecting Next for the next three screens

Figure 4 Select Role-based or feature-based installation

Figure 5 Server Selection

Figure 6 Server Roles

Select Windows Standards-Based Storage Management and confirm the installation:

Figure 7 Server Features

Figure 8 Confirm

Click Install and wait for the Wizard to complete. You could close this window but it shouldn’t take long.

Figure 9 Feature installed

Using PowerShell to enable the storage service

Another (easier) way to install the service is using the Windows PowerShell Add-WindowsFeature cmdlet. Open an Administrative PowerShell prompt and enter

Add-WindowsFeature WindowsStorageManagementService

This will install and enable the service and add three new cmdlets to the system, but the service doesn’t know about any SMI-S providers yet. That will happen when you register providers.

Note: at the time this blog was first posted, cmdlet help was not yet online for the three SMI-S specific cmdlets (Register-SmisProvider, Unregister-SmisProvider and Search-SmisProvider) so you may see an error message when you attempt to Update-Help. The Search-SmisProvider cmdlet will be covered in a later topic; for now I will assume you know the name or IP address of the SMI-S providers you will be using.

SMI-S providers

In general, you will download these from your storage vendor. Each vendor has their own mechanism for distributing, licensing, installing and configuring providers, so it’s difficult to give you generic rules.

The steps are as follows:

1. Download the SMI-S provider from your storage array vendor.
2. Install the provider on a Windows or Linux server. It should not be installed on a server running any other SMI-S provider. It can be installed on the same system as the Storage Service but you should install SMI-S servers in as few places as necessary.
3. Add firewall rules if you installed the provider on a Windows Server.
4. Change or add account credentials for the provider, as appropriate.
5. Make any changes to the provider’s properties (if necessary) and restart it.
6. Add arrays to manage.

Registering Providers

In order to use an SMI-S provider, it must be registered. The register process will do the following:

1)      Save the provider information for reuse whenever the service or system restarts

2)      Save the credentials for the provider (securely!)

3)      Allow adding certificates to the proper store if you are using the recommended SSL (https) communication protocol

4)      Perform a basic discovery (level 0)

5)      Subscribe for indications – this will be the subject of a later post

To register a provider, use the Register-SmisProvider cmdlet:

Register-SmisProvider –ConnectionUri https://<name or IP address of provider>:<port>

A prompt for provider credentials will appear (you can also script this using a PSCredential). Although the storage service supports HTTP, you are encouraged to use HTTPS with the register cmdlet. This will ensure that the provider’s SSL certificate is properly configured on the machine running the storage service and will give the highest level of security for communications. It is also important for SMI-S indications, which are only delivered using HTTPS.

It’s worth noting that permissions for the provider are restricted to the user account you are currently logged in with on the machine running the Storage Service. You can give other users permission by specifying –AdditionalUserswith the Register-SmisProvidercmdlet.

At this point, if the register succeeds, only basic discovery information has been collected (provider and array details, also known as Level 0).

Get-StorageProvider

(Observe the Names returned; Storage Spaces will always be shown but that does not use SMI-S so I won’t discuss it here.)

Get-StorageSubSystem

Storage pools and volumes will not be discovered yet. To do a deeper discovery, execute this cmdlet:

Update-StorageProviderCache –Name <name from above> -DiscoveryLevel Level2

I want to reiterate that the Storage Service has an extensive, multi-level cache. Once objects are discovered, they can be operated upon efficiently. But beware: deep discoveries can take a lot of time, especially with high-end hardware which supports tens of thousands of objects! Each level is cumulative so a Level3 discovery also does levels 0-2. I plan to talk more about limiting the scope of the discovery in a more tree-structured approach in a later post.

To remove the provider and credentials from your system, use the Unregister-SmisProvider cmdlet.

Some other cmdlets

For any cmdlet, you can type -? to get the complete syntax and information about parameter sets or use any of the other PowerShell help features. You can also pipe the output to control formatting or for use as input to another cmdlet.

Some advanced functionality may be limited by the array hardware, the SMI-S provider, and of course, the features you licensed from the storage vendor. The cmdlets below are just a sample of the complete set available through the Storage Management API on Windows Server 2012.

Pools and Virtual Disks

Masking Operations

Unmasking allows virtual disks to be seen by specific systems and their HBAs or iSCSI initiators. This is the key to sharing arrays with different hosts and allows large-scale storage to be used in a multi-computer environment. Masking is the reverse, hiding a virtual disk. Different arrays have different rules such as allowing virtual disks to participate in a limited number of masking sets, or allowing you control over which target ports can be specified in a set.

References

To see more about our commitment to standards and storage management, refer to http://blogs.technet.com/b/server-cloud/archive/2011/10/14/windows-server-8-standards-based-storage-management.aspx.

The Windows Server 2012 PowerShell Storage cmdlets are used with SMI-S providers as well as native SMP providers.

The previous blog in this series gives an overview of SMI-S. This blog gives you some information about using the EMC SMI-S provider with the storage service.

Learn more about the SNIA’s Storage Management Initiative at http://snia.org/forums/smi.

All providers have slightly different installation procedures and characteristics. This information should help you get started using the EMC provider which can be downloaded from Powerlink (you will need an account). Make sure you download the latest version (currently 4.4). This information applies for use with either the Windows Server 2012 Windows Standards-Based Storage Management Service or System Center Virtual Machine Manager 2012 and the provider supports EMC Symmetrix VMAX, Clariion CX4 and VNX arrays. Please consult the EMC documentation for the appropriate array firmware levels and also for what platform the provider can be installed on.

Note that an SMI-S provider is assumed to be running somewhere other than the Windows Standards-Based Storage Management Service (I’ll call it Storage Service for short). Providers are either standalone server applications (known as proxy providers) or embedded in the array firmware. For EMC, the provider being discussed is a proxy provider so it needs to be installed on a running system with a supported version of Windows or Linux installed. Since vendors have not yet certified their SMI-S Providers to run on Windows Server 2012, this blog will discuss getting the EMC Provider running on Win2k8R2.

Download the EMC provider

The EMC SMI-S provider is a part of the “Solutions Enabler with SMI” package which you can download from Powerlink (requires registration); search for “SMI-S Provider” once you log in. There are provider versions available for Windows and Linux, and either can be used. Make sure you select the latest 32-bit or 64-bit version, and Windows or Linux version, as appropriate.

Install the provider

Installation is straightforward, just run the installer you downloaded as an administrative user. Use all the defaults and make sure you only select the “Array provider” as Windows does not use the Host providers and installing it may create conflicts with other software. We assume throughout this document that the provider is running on a different system than the Storage Service. It may be possible to install it on the same system once vendors support the Windows Server 2012 platform. Be aware that installing multiple providers may not be supported or may require additional configuration and non-standard port numbers.

Add firewall rules

If the provider runs on a Windows Server, you will need to configure the firewall to allow SMI-S and SLP traffic. Please do not turn off the firewall. The general rules below can be made stricter by eliminating HTTP support (port 5988) and by specifying the specific CIM Server application (ECOM) for ports 5988-5989, and SLP server (SLPD.exe) for port 427 as the process for the rules. You can also limit which systems can manage through the provdier by limiting the firewall to passing only traffic from those IP addresses. I am assuming the firewall is in its default state (blocks incoming/allows outgoing traffic).

These command lines must be run from an administrative account and will work for Windows Server 2008 R2. You can also use the firewall control panel, or the equivalent PowerShell cmdlet if the provider was installed on a Windows Server 2012 system.

Change the default password and add additional users, if required

The EMC provider security can be configured through a webpage; open https://localhost:5989/ecomconfig. This is what you will see first:

At this point you have several options, but first a word about “self-signed” certificates. All SMI-S providers create or copy self-signed certificates to the system when they are first installed which means the certificate is not issued by a trusted party such as VeriSign. These certificates can be used “as-is” if that is consistent with your company’s security policies AND you trust the host where you installed the provider. You have the option to use more formally signed certificates, that is, certificates that “chain” to a trusted Certificate Authority or a CA. A full discussion of this can be found on the web. If you stay with the self-signed certificate, your options right now are to a) “Continue to this website” or b) change to using the fully-qualified domain name (FQDN) of the server instead of localhost, and add the certificate to the local certificate store which tells IE that you trusted this site. This only affects the use of the configuration page below; the storage service will prompt you for action when you register the SMI-S provider the first time.

Login with the default account (admin) and password (see the EMC documentation) and proceed to change the password, add an additional user or make any other changes to the security. Note the user name and password since you will need this when you register the provider for use with the Storage Service.

While you are here, there is one more change that we will need to make. Click on the Dynamic Settings link from the ECOM Administration Page and locate the setting for SSLClientAuthentication. Select None, check the Persist box, then click on Apply – this avoids a potential problem with SSL negotiations without lowering the security level. You will not need to restart the ECOM service if you modify parameters on this page.

Provider configuration changes for VMM

We need to adjust some settings for the EMC provider in order for it to work best with System Center 2012 Virtual Machine Manager. Navigate to C:\Program Files\EMC\ECIM\ECOM\conf and open the file Security_settings.xml with Notepad or another text editor.

 Change

<ECOMSetting Name="ExternalConnectionLimit" Type="uint32" Value="100"/>

to

<ECOMSetting Name="ExternalConnectionLimit" Type="uint32" Value="600"/>

 Change

<ECOMSetting Name="ExternalConnectionLimitPerHost" Type="uint32" Value="100"/>

to

<ECOMSetting Name="ExternalConnectionLimitPerHost" Type="uint32" Value="600"/>

Save the file and restart the provider. You can use the Services control panel, or from a command prompt:

net stop ecom

net start ecom

I also modify my PATH environment variable to include the EMC command line utilities:

set PATH=%PATH%;”C:\Program Files\EMC\SYMCLI\bin;C:\Program Files\EMC\ECIM\ECOM\bin”

(Or use the Advanced System Settings property page so this takes effect every time you open a command prompt.)

Adding arrays to manage

Depending on which EMC arrays you have, the process for managing them with SMI-S will be slightly different. The Symmetrix product line requires a direct, in-band connection using either Fibre Channel or iSCSI. This also requires creating “gatekeeper” LUNs on the array and unmasking them to the system where the provider is running and for Fibre Channel, configuring the zoning as well. Clariion and VNX can be managed either in-band or out-of-band using an Ethernet connection.

The EMC Provider Release Notes contains full information for adding arrays including zoning when you use Fibre Channel for inband management. See their Post-Installation Tasks for more information.

Indications

Indications are asynchronous events that come from the provider, informing a “listener” such as the storage service of events that may be of interest. I will discuss indication support in a later blog post. At that time, I will also discuss anything on the provider that you need to change or be aware of to support indications.

Quick update: The content hasn't changed but I noticed that the formatting on the PS script got messed up, let's see if this is better. Also attached the script as a file.

Indications are a mechanism used in the Common Information Model (CIM) to provide events from a CIM server to a client application. The storage service can use these events to make sure its cached information is up-to-date with the provider and the arrays it manages.

When the storage service is the single management point, the chances are pretty good that the cache will be reasonably current. But if multiple management points exist, such as more than one SMI-S client, more than one SMI-S provider managing a single array, or out-of-band mechanisms like vendor tools, the state of a managed array can change and the storage service’s view of the discovered objects will become out-of-sync. Since discovery is a time consuming operation, it’s best to not require refreshing the cache (Update-StorageProviderCache cmdlet) more often than necessary.

Some examples of state that might change:

• Volumes can be created or deleted or their size might change
• Volumes can be unmasked to servers or masked from them
• New storage pools can be created or deleted, they can be expanded or they can run out of free space
• The FriendlyName of an object can be changed Various objects might change status, from healthy to unhealthy

Indications can help keep the cache updated, but some assembly is required. Out of the box, the storage service will subscribe to indications and it will listen for the indications, but more steps are necessary to set up the security so that the system can receive them. I will try to make this as painless as possible by providing a PowerShell script to do most of the heavy lifting.

Keep in mind that if you don’t follow these steps, the storage service will still attempt to subscribe to indications and the provider will not be able to deliver them. At best, this produces lots of messages in the provider’s log files; at worst, the provider may not accept indications from other clients and we have even seen providers fail completely.

Configuration for Indications

The storage service implements an HTTPS listener using TCP port 5990, in accordance with DMTF requirements. There is no support for HTTP delivery of indications. The instructions below apply to Windows Server 2012 systems and require PowerShell.

At this time, you must use a certificate with the Common Name set to “msstrgsvc”. You could use a signed certificate if you have the ability to create one. Otherwise, use a self-signed certificate as demonstrated below. Also note that I have ExportPolicy set to 1 in the script; this is for debugging purposes and is not required.

I avoided wrap around lines by using the PowerShell continuation character ‘`’, but if you encounter syntax errors, make sure double quotes are just normal double quote characters and not typographic ones. Same for dashes.

Step 1:  Copy the following script to a file called configsmis.ps1:

# Install the Storage Service feature - safe if it is already installed

echo "Installing the feature - safe it is already installed"

Add-WindowsFeature WindowsStorageManagementService

# Create a firewall rule to allow incoming traffic on port 5990

echo "Adding a firewall rule for indications"

New-NetFirewallRule -DisplayName "CIMXML Indications" -Direction Inbound `

–LocalPort 5990 -Protocol TCP –Enabled True –Action Allow `

 -Description "CIM-XML Indications come in on HTTPS"

#Generate a self-signed certificate – the Common Name must be msstrgsvc

echo "creating a self-signed certificate"

$name = new-object -com "X509Enrollment.CX500DistinguishedName.1"

$name.Encode("CN=msstrgsvc", 0)

$key = new-object -com "X509Enrollment.CX509PrivateKey.1"

$key.ProviderName = "Microsoft RSA SChannel Cryptographic Provider"

$key.KeySpec = 1

$key.Length = 2048

$key.ExportPolicy = 1

$key.SecurityDescriptor = "D:PAI(A;;0xd01f01ff;;;SY)(A;;0xd01f01ff;;;BA)(A;;0x80120089;;;NS)"

$key.MachineContext = 1

$key.Create()

# create a certificate that is suitable for all purposes

$serverauthoid = new-object -com "X509Enrollment.CObjectId.1"

$serverauthoid.InitializeFromValue("1.3.6.1.4.1.311.10.12.1")

$ekuoids = new-object -com "X509Enrollment.CObjectIds.1"

$ekuoids.add($serverauthoid)

$ekuext = new-object -com "X509Enrollment.CX509ExtensionEnhancedKeyUsage.1"

$ekuext.InitializeEncode($ekuoids)

# this script will create certificate that is good for 1000 days – you can make it

# longer if you choose. After that time, the storage service will stop accepting

# indications and may stop handling mutual auth

$cert = new-object -com "X509Enrollment.CX509CertificateRequestCertificate.1"

$cert.InitializeFromPrivateKey(2, $key, "")

$cert.Subject = $name

$cert.Issuer = $cert.Subject

$cert.NotBefore = get-date

$cert.NotAfter = $cert.NotBefore.AddDays(1000)

$cert.X509Extensions.Add($ekuext)

$cert.Encode()

$enrollment = new-object -com "X509Enrollment.CX509Enrollment.1"

$enrollment.InitializeFromRequest($cert)

$certdata = $enrollment.CreateRequest(0)

$enrollment.InstallResponse(6, $certdata, 0, "")

# Now find the cert we just created (it will be in ‘MY’) and hold on to the thumbprint

cd cert:

$thumbprint=(dir -recurse | where {$_.Subject -match"CN=msstrgsvc*"} | `

  Select-Object -Last 1).thumbprint

$thumbprint

# clear out old certificates for this port and add our new one for all IPv4 and IPv6

# addresses. Deletes might fail if there were no old certificates.

# Note that I tried to use Server 2012 PowerShell cmdlets but I couldn’t exactly match

# all the requirements with the existing ones, so some of the commands rely on netsh

Set-Alias netsh c:\Windows\System32\netsh.exe

echo "You can ignore delete failures if this is a first time conifugration"

netsh http delete sslcert ipport=0.0.0.0:5990

netsh http delete sslcert ipport=[::]:5990

netsh http add sslcert ipport=0.0.0.0:5990 certhash=$($thumbprint) `

  appid="{468e21d1-a4cb-4134-8d9f-800c5ff2086f}"

netsh http add sslcert ipport=[::]:5990 certhash=$($thumbprint) `

  appid="{468e21d1-a4cb-4134-8d9f-800c5ff2086f}"

# Apply an ACL to the port so that NETWORK SERVICE can bind to it.

# Delete might fail if the ACL was never applied before.

netsh http delete urlacl url=https://*:5990/

netsh http add urlacl url=https://*:5990/ user="NT AUTHORITY\NETWORK SERVICE"

# restart the storage service so it can bind to the port properly

# you will need to perform Update-StorageProviderCache if level was > 0

echo "Restarting the service. Remember to run Update-StorageProviderCache since this resets the cache."

Restart-Service MSStrgSvc

Step 2: From a PowerShell Administrative Command prompt, execute these commands:

PS C:\Users\Administrator> $policy = Get-ExecutionPolicy

PS C:\Users\Administrator> Set-ExecutionPolicy Bypass

PS C:\Users\Administrator> .\configsmis.ps1 (assuming this is the correct directory)

PS C:\Users\Administrator> Set-ExecutionPolicy $policy

Test indications to port 5990

Before sending indication to port 5990, you may want to verify that HTTPS listener is working properly. Open https://localhost:5990 on the storage server machine in IE and it will show “MSP Storage Project” (choose to ignore server certificate error).

Note: Hit cancel if prompted for a certificate.

Test that the listener can be reached by pointing your web browser to the storage service machine and port 5990 (e.g., https://<storage-servername>:5990. You should see a certificate error which you can ignore for this test – this is sufficient to show that you have reached the indication listener. 

One-way authorization

The storage service only supports HTTPS connections for indications. When a CIM server attempts to connect to the storage service to deliver an indication, it will challenge the storage service and request a certificate. If the provider validates the server certificate, that certificate will need to be placed in the trusted store for the CIM server. This will vary depending on vendor implementation and is not described here.

 Very often the CIM server does little or no validation of the certificate and no other action is required.

Mutual Authentication

 Not currently supported. 

Exporting the Storage Service certificate

The above script will place the msstgsvc certificate in the trusted store. Using the certificates snap-in, it is possible to export this cert for use by the provider if it does certificate validation. Make sure to export in a format the provider can understand, typically .DER or .P7B, and don’t include the private key if you are asked. 

Registry values that affect Indications

 All registry values are located in

 HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Storage Management

 The defaults enable indications with certificate checking so no changes should be needed.

Microsoft added the ability to manage storage using Storage Management Initiative (SMI-S) providers to Windows Server 2012. Sometimes things don't go quite the way you plan and some debugging is needed to figure out what is going on.

So in order to provide better ability for storage vendors and customers to debug problems encountered with SMI-S providers, the Windows Standards-Based Storage Management Service (I’ll refer to it as Storage Service) offers a tracing facility as well as various other logging options.

CIMXML Logging

If you followed my earlier blogs, you would have learned that SMI-S is an implementation of the Common Information Model (CIM) encoded in XML format (CIM-XML) and transported using the Hypertext Transport Protocol (HTTP).

It is possible to save the requests and responses in a log file so storage vendors can see what requests are issued and what responses were captured by the Storage Service. Most SMI-S providers have similar capabilities and the logs on either side can be compared if problems are encountered.

Normally, this form of logging is not enabled but it can be turned on fairly easily when necessary. It will have some (small) impact on performance, and the log file can get quite large. Note that no security information is recorded in this log so user names and passwords are never captured.

There are a few registry values that control CIMXML logging, all under the key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog

TIP: Although the default file name ends in .txt, I usually change the name to have a .xml extension. When the file is opened in a text editor like Notepad++, the editor applies color coding and allows collapsing and expanding sections of the log, as if it was XML (it’s pretty close, just the timestamps aren't).

The easiest way to enable CIMXML logging is using PowerShell. Open an administrative PowerShell window and use these cmdlets (no line break in the Set-ItemProperty cmdlet):

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogLevel" -Value "4" -Force

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogFileName" -Value "C:\Temp\cimxml.xml" -Force

You may need to restart the service for the change to take effect.

Restart-Service MsStrgSvc

After restarting the storage service, you will have to do a rediscovery operation (Update-StorageProviderCache) to the appropriate level. To disable:

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogLevel" -Value "0" -Force

Restart-Service MsStrgSvc

Note: Logging is only available for SMI-S providers communicating through CIMXML, which is the most common transport. The Storage Service can also communicate with providers implemented under the Windows Management Instrumentation (WMI) framework. This logging function is not available for SMI-S WMI providers.

Windows Event Log

All provisioning activities are recorded in the StorageManagementService Event Log channel (which is enabled by default). Errors are also recorded here. The following sample shows a typical activity that was recorded when I registered a provider using HTTPS. You can see that the user name is logged so auditing of actions carried out through the Storage Service is possible.

Tracing

The Storage Service uses Event Tracing for Windows (ETW) which allows narrowing down issues without using a debugger or source code. In many code paths, the tracing capability has the ability to record unexpected error and warning conditions or informational messages in an Event Tracing Log (.etl) file. Tracing to a file can be enabled by using the logman command line.

Real-time tracing is possible using the TraceView utility (part of the Windows Driver Kit) and the appropriate symbol files. I mention this here so you can see that we have built in a lot of diagnostic capabilities into the service. Typically this information is only valuable to developers. However the public symbols do not contain the tracing information, so this is mostly useful if you need to open a support case.

To enable tracing, use the following command (from a command prompt, not PowerShell):

logman create trace "ETWTrace" -ow -o c:\ETWTrace.etl -p {18C2F19C-F79D-408F-837B-F0B23F20A0F7} 0x3f 0x5 -nb 16 16 -bs 1024 -mode Circular -f bincirc -max 4096 -ets

To stop tracing

logman stop "ETWTrace" –ets

This post is a part of the nine-part “What’s New in Windows Server & System Center 2012 R2” series that is featured on Brad Anderson’s In the Cloud blog.  Today’s blog post covers storage management through SMI-s and how it applies to the larger topic of “Transform the Datacenter.”  To read that post and see the other technologies discussed, read today’s post: “What’s New in 2012 R2: IaaS Innovations.”

Although my day job is no longer working on SMI-S, I did want to let folks know that a lot of work went into the Standards-Based Storage Management Service (aka Storage Service) in the upcoming Windows Server release. Here are the highlights:

Discovery - Discovery is what happens when you register (Register-SmisProvider) or update (Update-StorageProviderCache): the storage service tries to find out as much information as you wanted (there are four levels of discovery) and that information resides in the service’s cache so you don’t need to constantly go out to the provider to get the data. In Windows Server 2012, the way the information was discovered was through “walking the model”, which is to say, starting with an object and then following associations to get additional information. Unfortunately, this can take a long time on all but the smallest storage configurations.

For Windows Server 2012 R2, the mechanisms have been changed. Instead of model-walking, the service will do enumerations of objects and then figure out (in memory) how they inter-relate. This turns out to decrease discovery times up to 90%! We worked with vendors to make sure providers will work well with this change, and for some of those vendors, you will need to get updated providers.

Updates (through Indications) - I recently blogged about the indication support for SMI-S. The internals of this are much improved in Windows Server 2012 R2 and more provider changes will be caught through indications so that rediscovery won’t be needed as often. The information in the older blog still applies, except that the firewall rule will already be in place (the provided script will still run unchanged).

Secure connections (using HTTPS) - In my first posting, I advised against using Mutual Authentication with the storage service. For Windows Server 2012 R2, this has been improved. This applies to indications as well as normal SMI-S traffic. Follow the Indications blog for configuration information (you need to have the certificates in place). Not all providers will work well with mutual auth. I will post more about security in the near future.

Resiliency Settings - When creating pools and volumes using the Storage Management cmdlets, you could easily specify various parameters that were just never going to work. SMI-S is too generic here. This has been simplified - stick with vendor defined settings and you should be fine. For Windows Server 2012 R2, we’ll give you an error if you try to override any of the parameters.

Pull Operations - One of the efficiency improvements is to change how enumerations are done by using a newer mechanism called “pull” operations. (You can read more about this here.) This allows chunking of the data coming back from the provider, and generally will lower the memory required on the provider side for large configurations. This works now with EMC providers; others will update in the future. To enable pull operations, you will need to modify the registry value

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Storage Management\PullOperationCount

Set it to something like 100 which tells the provider to send 100 instances of a particular class at a time. Through PowerShell, this would look like:

To turn it off, just set the value back to 0. (Leaving it enabled for a provider that does not support pull operations will only cause a small performance degradation so you might want to leave this set if you have more than one provider and either supports pull operations.)

Registering a provider - You can now register a provider that does not have any arrays to manage (yet). An Update-StorageProviderCache cmdlet will find any new devices at a later time. More significantly, if for some reason the provider lost contact with an array, this would result in errors that could be difficult to recover from.

Node/Port address confusion- In Windows Server 2012, we had these defined backwards from what is mandated by standards or used by things like the iSCSI initiator. This has been corrected but may require provider updates because we found some bugs in existing implementations.

Snapshot/Volume deletion - Under some conditions, it would not be possible to delete a volume or a snapshot. For example, we sometimes thought a volume was exposed to a host when it wasn’t, or a snapshot was in the wrong state for deletion. This has been improved.

Snapshot target pool - Specifying a -TargetStoragePoolName for snapshots is now supported, that is, if the provider/array allows it. However, be careful when you have more than one array with pools of the same name (which might be common).

Masking operations. There have been cases where masking/unmasking operations can take a long time to complete, particularly when SC VMM issues multiple requests in parallel; these could result in timeouts. Also with the current SMI-S model, masking operations performed by Windows might require multiple method calls to providers. Windows Server 2012 R2 now uses jobs for such operations instead of making all masking operations synchronous.

To see all of the posts in this series, check out the What’s New in Windows Server & System Center 2012 R2 archive.

Context

Windows Server 2012 R2 ships with a rich set of standards-compliant storage management functionality. This functionality was originally introduced in Windows Server 2012, and you should reference Jeff’s excellent blog that introduced related concepts.

iSCSI Target Server on its part shipped its SMI-S provider as part of the OS distribution for the first time in Windows Server 2012 R2. For more details on its design and features, see my previous blog post iSCSI Target Server in Windows Server 2012 R2.

I will briefly summarize here how the iSCSI Target Server SMI-S provider fits into Storage management ecosystem, please refer to Jeff’s blog post for the related deep-dive discussion. Storage cmdlets are logically part of the Storage Management API (SM-API) layer, and the ‘Windows Standards-based Storage Management Service’ plumbs the Storage cmdlet management interactions through to the new iSCSI Target Server SMI-S provider. This blog post is all about how you can use the new SMI-S provider hands-on, using the SM-API Storage cmdlets. Note that iSCSI Target Sever can alternatively be managed through its native iSCSI-specific cmdlets, and related WMI provider APIs. While the native approach allows you to comprehensively manage all iSCSI-specific configuration aspects, the Storage cmdlet approach helps you normalize on a standard set of cmdlets across different technologies (e.g. Storage Spaces, 3^rd party storage arrays).

Before we jump into the discussion of Storage cmdlet-based management, you should keep two critical caveats in mind:

• Jeff’s caution about having multiple management clients potentially tripping up each other; to quote –

“You should think carefully about where you want to install this service – in a datacenter you would centralize the management of your resources as much as practicable, and you don’t want to have too many points of device management all competing to control the storage devices. This can result in conflicting changes and possibly even data loss or corruption if too many users can manage the same arrays.”

• You must decide how you want to manage a Windows-based iSCSI Target Server starting from the time the feature is installed on Windows Server. To begin with, you are free to choose from either WMI/PowerShell, or SM-API/SMI-S. But once you start using one management approach, you must stick with that approach until the iSCSI Target Server is decommissioned. Switching between the management approaches could leave the iSCSI Target Server in an inconsistent and unmanageable state, and potentially could even cause data loss.

For most users, the compelling reason for managing an iSCSI Target Server through SMI-S is usually one of the following:

1. They have existing scripts that already are written to the Windows Storage cmdlets and the Windows SMI-S cmdlets, and want to use the same scripts to manage Windows Server-based iSCSI Target Server, or,
2. They use 3^rd party storage management products that consume the Storage Management API (SM-API) and they plan to manage Windows Server-based iSCSI Target Server using that same 3^rd party software, or,
3. (Perhaps most likely) They use SCVMM to manage iSCSI Target Server-based storage through SMI-S. This can be accomplished using SCVMM-specific cmdlets and UI, and is covered in detail elsewhere. So it will not be the focus of this blog post, we will focus only on non-SCVMM management approach in this blog post.

Relating Terminology

Let us do a quick review of SM-API/SMI-S concepts so you can intuitively relate them to native Windows terminology as we move into hands-on discussion:

Register SMI-S Provider on a Management Client

To register an SMI-S provider, you need to know the provider’s URI – it is the machine name for a standalone target and the cluster resource group name in the case of a clustered iSCSI Target Server – and credentials for a user account that is in that target server’s local administrators security group. In the following example, the SMI-S provider can be accessed on the machine “fsf-7809-09” and the user account with administrative privileges is “contoso\user1”. Get-Credential cmdlet prompts for the password at run time (you can reference Get-Credential page for other more scripting-friendly, albeit less secure, options to accomplish the same).

Discover the Storage Objects

After registering the provider, you can now update storage provider cache to get an inventory of all the manageable storage objects through this SMI-S provider:

You can then list the storage subsystem and related details. Note that although the following screen shots show items related to Storage Spaces, they are unrelated to iSCSI Target Server SMI-S provider. iSCSI Target Server SMI-S provider items are highlighted in green.

You can inspect the available storage pools and filter by the friendly name, as shown in the example. Notice that the hosting volumes, which you already know iSCSI Target Server reports as storage pools, carry friendly names that include respective drive letters on the target server. Also notice the Primordial storage pool, which effectively represents entire capacity available on the iSCSI Target Server. However, keep in mind that you can create SCSI Logical Units only out of what SMI-S calls “concrete storage pools”, i.e. only from pools which have the ‘IsPrimordial’ attribute set to false in the following screen shot.

Create Storage Objects

The first operation we show carves out a new logical disk out of an existing concrete storage pool. You can use one of two cmdlets to create a new virtual disk: New-VirtualDisk and New-StorageSubsystemVirtualDisk. Technically, iSCSI Target Server SMI-S provider works fine with either cmdlet and we will show examples for both, although you probably want to use the New-VirtualDisk so you can intentionally select the storage pool to provision the storage volume from. New-StorageSubsystemVirtualDisk in contrast auto-selects the storage pool to provision the capacity from.

Or you can provision the possible maximum-sized virtual disk by using “-UseMaximumSize” parameter as shown:

If you prefer to use the New-StorageSubsystemVirtualDisk cmdlet, you need to specify the storage subsystem parameter, and in the example below, you can see it auto-selected a storage pool in the selected subsystem - the “iSCSITarget: FSF-7809-09: C:” pool.

With the New-VirtualDiskSnapshot cmdlet, you can take a snapshot of a virtual disk.

To create masking set for a storage subsystem, you can use New-MaskingSet. You must include at least one initiator access in the new masking set. You can also map one or multiple virtual disks to the new masking set. An iSCSI initiator is identified by its iqn name and the virtual disks through their names. The script below creates a new masking set and adds one initiator and two virtual disks. And then we will query the new masking set to confirm the details.

Modify and Remove the Storage Objects

With the Resize-VirtualDisk cmdlet, you can expand an existing virtual disk as shown in the following example. Note however that you will still need to extend the partition and the volume to be able to make the additional capacity usable.

You can also modify the masking set that you’ve just created, by adding additional virtual disks or additional initiators to it. Look at the following examples. Note however that you do not really want to share a virtual disk with a file system across multiple initiators unless the initiators (hosts) are clustered, else the setup will inevitably cause data corruption sooner or later!

You can of course also remove the masking set and virtual disks that you just created, as we illustrate in the following examples. Further, note also the order of operations – you have to remove a virtual disk first from masking sets (or remove the masking sets), and then delete the virtual disk.

Finally, when you no longer need to manage the iSCSI Target Server from this management client, you can unregister the SMI-S provider as shown in the following example.

I want to acknowledge my colleague Juan Tian who had helped me with all the preceding Windows PowerShell examples.

Finally, I sincerely hope that you now have the tools you need to work with an iSCSI Target Server SMI-S provider. Give it a try and let me know how it’s working for you!

Context

iSCSI Target Server shipped its SMI-S provider as part of the OS distribution for the first time in Windows Server 2012 R2. For more details on its design and features, see my previous blog post iSCSI Target Server in Windows Server 2012 R2.

System Center 2012 SP1 Virtual Machine Manager (VMM) and later versions manage the storage presented from an iSCSI Target Server for provisioning block storage to Hyper-V hosts. VMM configuration guidance for managing the iSCSI Target Server running on Windows Server 2012 is available in this TechNet page. This guidance is still accurate for Windows Server 2012 R2 with the two following exceptions.

1. iSCSI Target Server SMI-S provider is now included as part of the OS distribution, so you no longer need to install it from VMM media. In fact, only the SMI-S provider included in Windows Server 2012 R2 distribution is the compatible supported provider. Further, when you install the iSCSI Target Server feature, the right SMI-S provider is transparently installed.
2. SAN-based Rapid Provisioning scenario of VMM requires one additional step to work with iSCSI Target Server

The rest of this blog post is all about #2.

VMM SAN-based Rapid Provisioning

VMM SAN-based rapid provisioning, as the name suggests, helps an administrator rapidly provision new Hyper-V virtual machines. The key to this fast provisioning is copying the VHD files for the new virtual machine in the most efficient possible manner. In this case, VMM relies on iSCSI Target Server snapshot functionality to accomplish this. Specifically, iSCSI Target Server SMI-S provider exposes this snapshot functionality for usage by SM-API storage management framework, which VMM then uses to create iSCSI Virtual Disk snapshots. As a brief aside, check out my previous blog post for examples on how the same iSCSI SMI-S snapshot functionality can be used by a storage administrator directly via SM-API Storage cmdlets, outside of VMM.

Let’s focus back on VMM though, especially on the snapshot-related VMM rapid provisioning work flow and what each of these steps mean to the iSCSI Target Server:

1. Administrator creates, customizes, and generalizes (syspreps) the desired VM OS image on a storage volume, hosted on iSCSI Target Server storage
   • iSCSI Target Server perspective: It simply exposes a VHDX-based Virtual Disk as a SCSI disk to the connecting initiator. All the creation, customization and sysprep actions are simply I/Os on that SCSI Logical Unit (LU).
2. Administrator mounts that SCSI LU on the VMM Library Server, let’s call it Disk-G for Golden, hosting the storage volume. Administrator also makes sure to mask the LU from any other initiators.
   • iSCSI Target Server perspective: Disk-G is persisted as a VHDX format file on the hosting volume, but the initiator (Library Server) does not know or care about this server-side implementation detail
3. Administrator creates a VM template and associates the generalized SAN copy-capable OS image VHD files to this template. This process thus makes the template a SAN copy-capable VM template.
   • iSCSI Target Server perspective: This action is transparent to the iSCSI Target Server, it does not participate unless there are specific related I/Os to Disk-G
4. From this point on, VMM can rapidly provision each new VM by creating a snapshot of Disk-G (say Disk-S1, Disk-S2 etc.) and assigning it to the appropriate Hyper-V host that will host the new VM guest being instantiated.
   • iSCSI Target Server perspective: For each disk snapshot taken via SMI-S, iSCSI Target Server creates a Diff VHDX file to store its content, so effectively:
     • Disk-G parent VHDX file
     • Disk-S1 Diff VHDX (Disk-G is parent)
     • Disk-S2 Diff VHDX (Disk-G is parent)

For a more detailed discussion of SAN-based VMM rapid provisioning concepts, see this TechNet Library article.

The entire scenario of course works flawlessly both in Windows Server 2012 and Windows Server 2012 R2. However in Windows Server 2012 R2, it turns out the storage administrator needs to take one additional step between Steps #3 and #4 – let’s call it “Step 3.5” – in the preceding list. Let’s then discuss what exactly changed in Windows Server 2012 R2 and what the additional step is.

iSCSI Target Server SMI-S Snapshots in Windows Server 2012 R2

On each successful SMI-S snapshot request, iSCSI Target Server creates a Diff VHDX-based iSCSI Virtual Disk. In Windows Server 2012 R2, iSCSI Target Server realizes this through native Hyper-V APIs. In contrast, iSCSI Target Server used to have its own implementation of creating Diff VHD files back in Windows Server 2012 – see my discussion of redesigned persistence layer in Windows Server 2012 R2 in one of my earlier blog posts for more detail. The new Hyper-V APIs enforce that the parent VHDX file must not be open in read & write mode while the new Diff VHDX is being created. This is to ensure that the parent VHDX can no longer be written to, once the diff VHDX is created. Thus while creating Disk-S1/S2 iSCSI Target Server SMI-S snapshots in the example discussion, Disk-G cannot stay mounted for read/write by the Library Server. Disk-G must be unmounted and re-mounted as read-only disk first – otherwise, creation of snapshots Disk-S1 and Disk-S2 will fail.

Now you might be wondering why this wasn’t an issue in Windows Server 2012-based iSCSI Target Server. iSCSI Target Server’s private implementation of Diff VHD creation in Windows Server 2012 did not enforce the read-only requirement on the parent VHD file, but the VMM Library server (initiator) always ensures that no more writes are performed on Disk-G once the sysprep process is complete. So the overall solution worked just fine in Windows Server 2012. With Windows Server 2012 R2 though, in addition to the same initiator behavior, iSCSI Target Server is effectively adding an additional layer of safety on the target (server) side to ensure writes are simply not possible at all on Disk-G. This is additional goodness.

I have briefly alluded to the nature of the additional process step required for rapid provisioning, but here’s the complete list of actions within that additional step:

“Step 3.5”:

• Save the volume mount point (and the drive letter if applicable) and offline the iSCSI LU on the Library Server.
• Unmap the LU from its current masking set (Remove-VirtualDiskFromMaskingSet). This ensures that the LU under the previous read/write access permissions can no longer be accessed by any initiator.
• Re-add the same SCSI LU (Add-VirtualDiskToMaskingSet) back to the same masking set, albeit this time as read-only through the “-DeviceAccesses ReadOnly” PS parameter. This sets the disk access to Read-Only.
  • Note: Only one Library Server should have the volume mounted off that SCSI LU. Even if the Library Server is configured as a highly-available failover cluster, only one of the cluster nodes should have mounted the disk at one time.
• Online the SCSI LU on the Library Server and restore its previous mount point (and drive letter, if applicable)

Here is the good news. My colleague Juan Tian has written a sample Windows PowerShell script which takes a single VMM template name parameter and performs all the above actions in the “Step 3.5” in one sweep. The script should work without any changes if you run it with VMM administration credentials. Feel free to check it out at the bottom of the blog post, customize if necessary for your deployment, and be sure to run it as “Step 3.5” in the VMM rapid deployment work flow that I summarized above.

Finally, let me wrap up this blog post with a quick version compatibility reference:

Hope this blog post provided you all the required details you need to move to production with your Windows Server 2012 R2-based new iSCSI Target Server and VMM. Give it a try and let me know how it’s working for you!

>>>>>>>>>SetLibraryServerLUToReadOnly.ps1 Windows PowerShell script>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

# Description:

# Script to set a VMM Library Server access of a SCSI Logical Unit (LU) to read-only access.

# Library Server and the disk (SCSI LU) are identified based on the VM template name parameter

# Script always offlines the disk, removes it from the masking set, and re-adds the disk back in as read-only.

# Script finally re-mounts the newly-read-only disk on the Library Server.

# Must be run with the VMM administration credentials

#

param([string] $VMTemplate = "")

if(!$VMTemplate)

{

$VMTemplate = Read-Host "Enter the name of your Template: "

}

Write-Host "Get Template $VMTemplate"

$libShare=(Get-SCLibraryShare -ID ((Get-SCVMTemplate -name $VMTemplate | Get-SCVirtualHardDisk).libraryshareid))

$share=$libShare.path

if ($share.count -lt 1)

{

Write-Host "Cannot find libraryshare!"

exit 1

}

Write-Host "Get libraryshare $share"

$path=(Get-SCVMTemplate -name $VMTemplate| Get-SCVirtualHardDisk).directory

if ($path.count -lt 1)

{

Write-Host "Cannot find SCVirtualHardDisk!"

exit 1

}

Write-Host "Get virtualdisk directory $path"

$path2=$path.replace($share, "")

$key="*"+$path2+"*"

Write-Host "Get Key $key"

$lib=($libShare.LibraryServer).FQDN

if ($lib.count -lt 1)

{

Write-Host "Cannot find libraryserver!"

exit 1

}

Write-Host "Get libraryserver $lib"

$partition = Invoke-Command -computername $lib -scriptblock {get-partition } | where-object {$_.accesspaths -like $key}

if (!$partition)

{

Write-Host "Cannot find disk partition!"

exit 1

}

$disk = $partition | Invoke-Command -computername $lib -scriptblock {get-disk}

if (!$disk)

{

Write-Host "Cannot find disk!"

exit 1

}

#offline disk

Write-Host "Offline disk ..."

$disk | Invoke-Command -computername $lib -scriptblock {set-disk -isoffline $true}

Write-Host "Offline disk completed!"

Write-Host "Looking for disk.uniqueid - $disk.uniqueid"

$vdisk = Get-VirtualDisk | where-object {$_.uniqueid -match $disk.uniqueid}

if (!$vdisk)

{

Write-Host "Cannot find virtual disk!"

exit 1

}

$ms = $vdisk | get-maskingset

if (!$ms)

{

Write-Host "Cannot find maskingset!"

exit 1

}

#remove virtual disk from masking set

Write-Host "Call Remove-VirtualDiskFromMaskingSet ..."

Remove-VirtualDiskFromMaskingSet -maskingsetuniqueid $ms.uniqueid -virtualdisknames $vdisk.Name

Write-Host "Call Remove-VirtualDiskFromMaskingSet completed!"

#add virtual disk back in masking set

Write-Host "Call Add-VirtualDiskToMaskingSet ..."

Add-VirtualDiskToMaskingSet -maskingsetuniqueid $ms.uniqueid -virtualdisknames $vdisk.Name -deviceaccesses ReadOnly

Write-Host "Call Add-VirtualDiskToMaskingSet completed!"

#online disk

Write-Host "Online disk ..."

$disk | Invoke-Command -computername $lib -scriptblock {set-disk -isoffline $false}

Write-Host "Online disk completed!”

About ten years ago, a group of engineers from different storage companies got together to create a powerful management abstraction that would enable tools for centralized storage environments like storage area networks (SAN) to become less vendor-specific but still allow any vendor to fully surface the capabilities of their storage hardware. One of the original goals was to build a model that relied on industry standards, largely from the DMTF, W3C and IETF. This included using a common model and schema (the Common Information Model or CIM). Unfortunately a model and schema only get you so far; they don’t give you something that can be implemented. You need a transport and a language to encapsulate the model. At the time, the logical choice for encoding the data was XML, specifically CIM-XML (also standardized in the DMTF) and the logical transport was HTTP. Collectively these technologies are known as Web-Based Enterprise Management, or WBEM.

[There are other choices for transport now available, such as WS-Management, a SOAP protocol that is also on the way to becoming an international standard, and the Windows-only WMI COM implementation of WBEM. I’ll speak more about those technologies at a later time, as they are also key to Microsoft’s standards-based management approach.]

The direction had been set in the Storage Network Industry Association, but the work moved into a private consortium known as PDP for reasons that I won’t go into here. A few years later, the group handed the work back to SNIA as a draft called Bluefin, which later became the Storage Management Initiative Specification, or SMI-S 1.0.

So what is SMI-S? Well, it’s basically a cookbook that explains how to use the model elements to do useful work, like discover and provision storage. Actually, it has grown into multiple cookbooks now, divided into topics like Common Profiles, Block, Filesystems, Fabric, etc. The SNIA has active working groups updating the specification every year or two to keep up with the latest technologies. The current (recently) published version is 1.6.0, with some updates expected by the end of the year.

A closer look

Let’s look at the different components that make up SMI-S:

Providers

An SMI-S provider is a software component produced by or for a particular storage product or family of products. It implements an HTTP server, it can parse XML requests, and it knows how to control those specific devices which usually have their own unique management interfaces. The provider can be embedded in the firmware or operating system of the device itself, or it can exist as a standalone application running on a general purpose computer operating system. The latter model is called a “Proxy” provider, and it is the most common way to implement SMI-S providers at this time.

Profiles

Profiles are a way of describing what components of the schema will be used to describe a particular management domain. For example, there is an Array profile. If a vendor claims support for the Array profile, they also agree to implement the classes defined in that profile, the specific properties and methods of those classes deemed necessary or optional to do useful work, and they also have to implement other related component profiles or subprofiles.

Profiles and subprofiles also tell you what the array can do. If the array can perform snapshotting or cloning of storage volumes, the provider can advertise that capability by claiming support for Replication Services. Application developers then know that they can use the CreateElementReplica method to create a new copy of the storage volume – an incredibly powerful capability used heavily in virtual machine management.

Profiles were created by SNIA and also by the DMTF. In some cases, SNIA specializes the profiles for use with SMI-S.

Classes

A class is a set of properties and methods that pertain to a specific object, or to an aspect of an object. Classes typically inherit properties from base or superclasses. An example class is a Disk Drive (in the CIM model, CIM_DiskDrive). Some properties of the disk drive might be what kind of drive it is (SSD, hard disk, etc.), how fast it spins, and what it’s doing right now.

A vendor can introduce new properties and methods by creating their own version of a class, inheriting from the base class and then adding their uniqueness to their derived class. In that way, the model allows extensibility while still preserving the common functionality that any application can use without understanding the specific value-added capabilities of the device it is trying to manage.

Most of the classes used by SMI-S are standardized in the DMTF CIM schema. There are over 1500 classes already defined, and new classes and additions to existing classes can happen frequently as the schema gets updated about three times a year.

The provider creates an instance of a class when it represents the actual objects. In our disk drive example, there will be an instance of the CIM_DiskDrive (or vendor-derived version of that class) for each physical drive in the storage array. An enumeration operation will retrieve all the instances, or a specific instance can be retrieved by itself.

Associations

It isn’t really enough to just count up all the objects of a particular class. Objects of one class might be related to another class and this is represented by associations. So a disk drive would be associated directly or indirectly to the object that represents the storage array. That way you can start at the top and find the related objects, or you can go from the bottom up to the top.

What Microsoft is doing with SMI-S

Microsoft has become an active participant in the SNIA technical working groups and also in the SMI-Lab plugfests. Over the past few years we have been testing hardware vendors’ providers with our client interfaces, first with System Center Virtual Machine Manager 2012, and later with Windows Server 2012. One goal was to make sure the most important scenarios for these products will work well with our interpretation of the specifications. And another objective was to allow for extensibility so that anything exposed through SMI-S could be leveraged on Windows without having to redo the considerable plumbing for interacting with a remote WBEM client. I’ll present more about this “passthrough” capability in a future blog.

What it means for Windows customers

As part of our commitment to management standards, Microsoft developed support for SMI-S and now includes this support with VMM 2012 and Windows Server 2012. Working in conjunction with the new storage management interfaces in Windows Server (see the references below), SMI-S enables command line access and easy scripting of many common storage tasks using Windows PowerShell and can also be used with the newly enhanced File and Storage Services role in Server Manager.

SMI-S support is implemented as an optional feature in Windows Server called Windows Standards-Based Storage Management. Keep in mind that it is designed for consolidated storage environments, and although most providers support simultaneous access from multiple clients, it is a best practice to also centralize the management of those devices. It would be unnecessary (and unwise) for the majority of servers to all be managing the same resources individually.

What it means for Hardware vendors

Because SMI-S is an industry standard built on top of other standards and widely implemented in other environments (such as VMware® and AIX®), vendors no longer need to produce proprietary providers for one particular operating system (or release). They can expose all of their product’s goodies using one provider as extensibility is inherent in the object-based approach of CIM. As client applications learn to take advantage of the advanced features, they can do so without resorting to yet another provider-model. CIM also has a good backward and forward compatibility story.

Next up…

On Windows Server 2012, once an SMI-S Provider is configured for use with the service, the storage will be manageable using the new Windows Storage Management API, and the Storage module in Powershell.

In the next post, I will explain how to get started using SMI-S with Windows Server 2012. Some assembly is required, but you should be up and running in a few minutes.

References

To see more about Microsoft’s commitment to standards and storage management, refer to Jeffrey Snover’s blog on this topic.

System Center 2012 Virtual Machine Manager also uses SMI-S providers. See http://technet.microsoft.com/library/gg610600.aspx for more information about storage automation and VMM.

Refer to the following section on technet for more information about the Storage cmdlets http://technet.microsoft.com/en-us/library/hh848705.aspx or the Storage Management team blog for the latest information at http://blogs.msdn.com/san

Learn more about the Storage Management Initiative at http://snia.org/forums/smi.

SMI-S is based on many industry and worldwide (ISO) standards:

CIM, WBEM and WS-Management are the products of the Distributed Management Task Force. For more information, see the DMTF website.

The Service Location Protocol, Version 2 (SLPv2), and Transport Layer Security (TLS, including Secure Sockets Layer or SSL) are protocols from the Internet Engineering Task Force (IETF).

The current specification for Hypertext Transport Protocol (HTTP) is a joint effort by the IETF and the World Wide Web Consortium (W3C).

In my last blog, I discussed the industry standard Storage Management Initiative (SMI-S) in general terms. Microsoft started supporting SMI-S for storage provisioning in System Center 2012 Virtual Machine Manager, released earlier this year. With the upcoming Windows Server 2012, SMI-S support will be available to all of our server customers. Coupled with the new Storage Management API (SMAPI), which consists of new WMI interfaces and cmdlets, it is possible to manage SAN or direct attached storage in a vendor-independent fashion, and also in a system-independent fashion if you have more than Windows in your datacenter. The new File and Storage Services canvas in Server Manager can take advantage of SMI-S providers, giving you a GUI for managing basic array functionality right out of the box.

SMI-S is based on the DMTF CIM Model and Schema, and as such it can support very complex environments. The SMAPI simplifies that by hiding a lot of the details and relationships that go into the model. Many vendors took part in the formation of SMI-S. Each vendor has to create one or more SMI-S providers for their products and customers need to obtain the providers from their storage vendors. But first, let’s talk about getting SMI-S going on Windows Server 2012.

Adding SMI-S support to Windows Server 2012

The Windows Standards-Based Storage Management Service is the optional feature for Windows Server only, which communicates with SMI-S providers (in SMI-S speak, it is a “client”). I will refer to it below as the “Storage Service”. It is not installed by default, so you will need to add the feature by using either Server Manager or a PowerShell cmdlet.

The diagram below shows the full architecture of the new Storage Management infrastructure in Windows. This blog focuses mostly on the lower left area highlighted by the red box.

Figure 1 Architecture of Storage Management on Windows Server 2012

The Storage Service also providers other functionality such as extensive (and secure) caching of management objects surfaced by one or more SMI-S providers, handling of dynamic events through SMI-S indications, managing asynchronous tasks and secure credential management. [Windows also supports a different hardware provider model, known as Storage Management Provider (or SMP). This blog only addresses the SMI-S support, and the Storage Service translates between SMI-S and SMP for you.]

You should think carefully about where you want to install this service – in a datacenter you would centralize the management of your resources as much as practicable, and you don’t want to have too many points of device management all competing to control the storage devices. This can result in conflicting changes and possibly even data loss or corruption if too many users can manage the same arrays. Multiple levels of permissions can help mitigate this possibility. SMI-S providers also typically manage more than one hardware device so you don’t need to install one provider per array.

Using Server Manager

Server Manager has been redesigned from the ground up for Windows Server 2012. It is now a tool that can manage across many server instances. It is one way that you can install the Windows Standards-Based Storage Management Service.

Server Manager typically opens when you log in to the Windows Server as an administrator. All actions described below must be performed with administrator privilege (Windows may prompt to elevate privilege if necessary).

Figure 2 Server Manager Dashboard

Pick the server you wish to install on to begin the feature installation (here I chose the Local Server but you could install on any other server being managed). Click on Manage and select Add Roles and Features:

Figure 3 Add Roles and Features to the Local Server

Continue with the Wizard, selecting Next for the next three screens

Figure 4 Select Role-based or feature-based installation

Figure 5 Server Selection

Figure 6 Server Roles

Select Windows Standards-Based Storage Management and confirm the installation:

Figure 7 Server Features

Figure 8 Confirm

Click Install and wait for the Wizard to complete. You could close this window but it shouldn’t take long.

Figure 9 Feature installed

Using PowerShell to enable the storage service

Another (easier) way to install the service is using the Windows PowerShell Add-WindowsFeature cmdlet. Open an Administrative PowerShell prompt and enter

Add-WindowsFeature WindowsStorageManagementService

This will install and enable the service and add three new cmdlets to the system, but the service doesn’t know about any SMI-S providers yet. That will happen when you register providers.

Note: at the time this blog was first posted, cmdlet help was not yet online for the three SMI-S specific cmdlets (Register-SmisProvider, Unregister-SmisProvider and Search-SmisProvider) so you may see an error message when you attempt to Update-Help. The Search-SmisProvider cmdlet will be covered in a later topic; for now I will assume you know the name or IP address of the SMI-S providers you will be using.

SMI-S providers

In general, you will download these from your storage vendor. Each vendor has their own mechanism for distributing, licensing, installing and configuring providers, so it’s difficult to give you generic rules.

The steps are as follows:

1. Download the SMI-S provider from your storage array vendor.
2. Install the provider on a Windows or Linux server. It should not be installed on a server running any other SMI-S provider. It can be installed on the same system as the Storage Service but you should install SMI-S servers in as few places as necessary.
3. Add firewall rules if you installed the provider on a Windows Server.
4. Change or add account credentials for the provider, as appropriate.
5. Make any changes to the provider’s properties (if necessary) and restart it.
6. Add arrays to manage.

Registering Providers

In order to use an SMI-S provider, it must be registered. The register process will do the following:

1)      Save the provider information for reuse whenever the service or system restarts

2)      Save the credentials for the provider (securely!)

3)      Allow adding certificates to the proper store if you are using the recommended SSL (https) communication protocol

4)      Perform a basic discovery (level 0)

5)      Subscribe for indications – this will be the subject of a later post

To register a provider, use the Register-SmisProvider cmdlet:

Register-SmisProvider –ConnectionUri https://<name or IP address of provider>:<port>

A prompt for provider credentials will appear (you can also script this using a PSCredential). Although the storage service supports HTTP, you are encouraged to use HTTPS with the register cmdlet. This will ensure that the provider’s SSL certificate is properly configured on the machine running the storage service and will give the highest level of security for communications. It is also important for SMI-S indications, which are only delivered using HTTPS.

It’s worth noting that permissions for the provider are restricted to the user account you are currently logged in with on the machine running the Storage Service. You can give other users permission by specifying –AdditionalUserswith the Register-SmisProvidercmdlet.

At this point, if the register succeeds, only basic discovery information has been collected (provider and array details, also known as Level 0).

Get-StorageProvider

(Observe the Names returned; Storage Spaces will always be shown but that does not use SMI-S so I won’t discuss it here.)

Get-StorageSubSystem

Storage pools and volumes will not be discovered yet. To do a deeper discovery, execute this cmdlet:

Update-StorageProviderCache –Name <name from above> -DiscoveryLevel Level2

I want to reiterate that the Storage Service has an extensive, multi-level cache. Once objects are discovered, they can be operated upon efficiently. But beware: deep discoveries can take a lot of time, especially with high-end hardware which supports tens of thousands of objects! Each level is cumulative so a Level3 discovery also does levels 0-2. I plan to talk more about limiting the scope of the discovery in a more tree-structured approach in a later post.

To remove the provider and credentials from your system, use the Unregister-SmisProvider cmdlet.

Some other cmdlets

For any cmdlet, you can type -? to get the complete syntax and information about parameter sets or use any of the other PowerShell help features. You can also pipe the output to control formatting or for use as input to another cmdlet.

Some advanced functionality may be limited by the array hardware, the SMI-S provider, and of course, the features you licensed from the storage vendor. The cmdlets below are just a sample of the complete set available through the Storage Management API on Windows Server 2012.

Pools and Virtual Disks

Masking Operations

Unmasking allows virtual disks to be seen by specific systems and their HBAs or iSCSI initiators. This is the key to sharing arrays with different hosts and allows large-scale storage to be used in a multi-computer environment. Masking is the reverse, hiding a virtual disk. Different arrays have different rules such as allowing virtual disks to participate in a limited number of masking sets, or allowing you control over which target ports can be specified in a set.

References

To see more about our commitment to standards and storage management, refer to http://blogs.technet.com/b/server-cloud/archive/2011/10/14/windows-server-8-standards-based-storage-management.aspx.

The Windows Server 2012 PowerShell Storage cmdlets are used with SMI-S providers as well as native SMP providers.

The previous blog in this series gives an overview of SMI-S. This blog gives you some information about using the EMC SMI-S provider with the storage service.

Learn more about the SNIA’s Storage Management Initiative at http://snia.org/forums/smi.

All providers have slightly different installation procedures and characteristics. This information should help you get started using the EMC provider which can be downloaded from Powerlink (you will need an account). Make sure you download the latest version (currently 4.4). This information applies for use with either the Windows Server 2012 Windows Standards-Based Storage Management Service or System Center Virtual Machine Manager 2012 and the provider supports EMC Symmetrix VMAX, Clariion CX4 and VNX arrays. Please consult the EMC documentation for the appropriate array firmware levels and also for what platform the provider can be installed on.

Note that an SMI-S provider is assumed to be running somewhere other than the Windows Standards-Based Storage Management Service (I’ll call it Storage Service for short). Providers are either standalone server applications (known as proxy providers) or embedded in the array firmware. For EMC, the provider being discussed is a proxy provider so it needs to be installed on a running system with a supported version of Windows or Linux installed. Since vendors have not yet certified their SMI-S Providers to run on Windows Server 2012, this blog will discuss getting the EMC Provider running on Win2k8R2.

Download the EMC provider

The EMC SMI-S provider is a part of the “Solutions Enabler with SMI” package which you can download from Powerlink (requires registration); search for “SMI-S Provider” once you log in. There are provider versions available for Windows and Linux, and either can be used. Make sure you select the latest 32-bit or 64-bit version, and Windows or Linux version, as appropriate.

Install the provider

Installation is straightforward, just run the installer you downloaded as an administrative user. Use all the defaults and make sure you only select the “Array provider” as Windows does not use the Host providers and installing it may create conflicts with other software. We assume throughout this document that the provider is running on a different system than the Storage Service. It may be possible to install it on the same system once vendors support the Windows Server 2012 platform. Be aware that installing multiple providers may not be supported or may require additional configuration and non-standard port numbers.

Add firewall rules

If the provider runs on a Windows Server, you will need to configure the firewall to allow SMI-S and SLP traffic. Please do not turn off the firewall. The general rules below can be made stricter by eliminating HTTP support (port 5988) and by specifying the specific CIM Server application (ECOM) for ports 5988-5989, and SLP server (SLPD.exe) for port 427 as the process for the rules. You can also limit which systems can manage through the provdier by limiting the firewall to passing only traffic from those IP addresses. I am assuming the firewall is in its default state (blocks incoming/allows outgoing traffic).

These command lines must be run from an administrative account and will work for Windows Server 2008 R2. You can also use the firewall control panel, or the equivalent PowerShell cmdlet if the provider was installed on a Windows Server 2012 system.

Change the default password and add additional users, if required

The EMC provider security can be configured through a webpage; open https://localhost:5989/ecomconfig. This is what you will see first:

At this point you have several options, but first a word about “self-signed” certificates. All SMI-S providers create or copy self-signed certificates to the system when they are first installed which means the certificate is not issued by a trusted party such as VeriSign. These certificates can be used “as-is” if that is consistent with your company’s security policies AND you trust the host where you installed the provider. You have the option to use more formally signed certificates, that is, certificates that “chain” to a trusted Certificate Authority or a CA. A full discussion of this can be found on the web. If you stay with the self-signed certificate, your options right now are to a) “Continue to this website” or b) change to using the fully-qualified domain name (FQDN) of the server instead of localhost, and add the certificate to the local certificate store which tells IE that you trusted this site. This only affects the use of the configuration page below; the storage service will prompt you for action when you register the SMI-S provider the first time.

Login with the default account (admin) and password (see the EMC documentation) and proceed to change the password, add an additional user or make any other changes to the security. Note the user name and password since you will need this when you register the provider for use with the Storage Service.

While you are here, there is one more change that we will need to make. Click on the Dynamic Settings link from the ECOM Administration Page and locate the setting for SSLClientAuthentication. Select None, check the Persist box, then click on Apply – this avoids a potential problem with SSL negotiations without lowering the security level. You will not need to restart the ECOM service if you modify parameters on this page.

Provider configuration changes for VMM

We need to adjust some settings for the EMC provider in order for it to work best with System Center 2012 Virtual Machine Manager. Navigate to C:\Program Files\EMC\ECIM\ECOM\conf and open the file Security_settings.xml with Notepad or another text editor.

 Change

<ECOMSetting Name="ExternalConnectionLimit" Type="uint32" Value="100"/>

to

<ECOMSetting Name="ExternalConnectionLimit" Type="uint32" Value="600"/>

 Change

<ECOMSetting Name="ExternalConnectionLimitPerHost" Type="uint32" Value="100"/>

to

<ECOMSetting Name="ExternalConnectionLimitPerHost" Type="uint32" Value="600"/>

Save the file and restart the provider. You can use the Services control panel, or from a command prompt:

net stop ecom

net start ecom

I also modify my PATH environment variable to include the EMC command line utilities:

set PATH=%PATH%;”C:\Program Files\EMC\SYMCLI\bin;C:\Program Files\EMC\ECIM\ECOM\bin”

(Or use the Advanced System Settings property page so this takes effect every time you open a command prompt.)

Adding arrays to manage

Depending on which EMC arrays you have, the process for managing them with SMI-S will be slightly different. The Symmetrix product line requires a direct, in-band connection using either Fibre Channel or iSCSI. This also requires creating “gatekeeper” LUNs on the array and unmasking them to the system where the provider is running and for Fibre Channel, configuring the zoning as well. Clariion and VNX can be managed either in-band or out-of-band using an Ethernet connection.

The EMC Provider Release Notes contains full information for adding arrays including zoning when you use Fibre Channel for inband management. See their Post-Installation Tasks for more information.

Indications

Indications are asynchronous events that come from the provider, informing a “listener” such as the storage service of events that may be of interest. I will discuss indication support in a later blog post. At that time, I will also discuss anything on the provider that you need to change or be aware of to support indications.

Quick update: The content hasn't changed but I noticed that the formatting on the PS script got messed up, let's see if this is better. Also attached the script as a file.

Indications are a mechanism used in the Common Information Model (CIM) to provide events from a CIM server to a client application. The storage service can use these events to make sure its cached information is up-to-date with the provider and the arrays it manages.

When the storage service is the single management point, the chances are pretty good that the cache will be reasonably current. But if multiple management points exist, such as more than one SMI-S client, more than one SMI-S provider managing a single array, or out-of-band mechanisms like vendor tools, the state of a managed array can change and the storage service’s view of the discovered objects will become out-of-sync. Since discovery is a time consuming operation, it’s best to not require refreshing the cache (Update-StorageProviderCache cmdlet) more often than necessary.

Some examples of state that might change:

• Volumes can be created or deleted or their size might change
• Volumes can be unmasked to servers or masked from them
• New storage pools can be created or deleted, they can be expanded or they can run out of free space
• The FriendlyName of an object can be changed Various objects might change status, from healthy to unhealthy

Indications can help keep the cache updated, but some assembly is required. Out of the box, the storage service will subscribe to indications and it will listen for the indications, but more steps are necessary to set up the security so that the system can receive them. I will try to make this as painless as possible by providing a PowerShell script to do most of the heavy lifting.

Keep in mind that if you don’t follow these steps, the storage service will still attempt to subscribe to indications and the provider will not be able to deliver them. At best, this produces lots of messages in the provider’s log files; at worst, the provider may not accept indications from other clients and we have even seen providers fail completely.

Configuration for Indications

The storage service implements an HTTPS listener using TCP port 5990, in accordance with DMTF requirements. There is no support for HTTP delivery of indications. The instructions below apply to Windows Server 2012 systems and require PowerShell.

At this time, you must use a certificate with the Common Name set to “msstrgsvc”. You could use a signed certificate if you have the ability to create one. Otherwise, use a self-signed certificate as demonstrated below. Also note that I have ExportPolicy set to 1 in the script; this is for debugging purposes and is not required.

I avoided wrap around lines by using the PowerShell continuation character ‘`’, but if you encounter syntax errors, make sure double quotes are just normal double quote characters and not typographic ones. Same for dashes.

Step 1:  Copy the following script to a file called configsmis.ps1:

# Install the Storage Service feature - safe if it is already installed

echo "Installing the feature - safe it is already installed"

Add-WindowsFeature WindowsStorageManagementService

# Create a firewall rule to allow incoming traffic on port 5990

echo "Adding a firewall rule for indications"

New-NetFirewallRule -DisplayName "CIMXML Indications" -Direction Inbound `

–LocalPort 5990 -Protocol TCP –Enabled True –Action Allow `

 -Description "CIM-XML Indications come in on HTTPS"

#Generate a self-signed certificate – the Common Name must be msstrgsvc

echo "creating a self-signed certificate"

$name = new-object -com "X509Enrollment.CX500DistinguishedName.1"

$name.Encode("CN=msstrgsvc", 0)

$key = new-object -com "X509Enrollment.CX509PrivateKey.1"

$key.ProviderName = "Microsoft RSA SChannel Cryptographic Provider"

$key.KeySpec = 1

$key.Length = 2048

$key.ExportPolicy = 1

$key.SecurityDescriptor = "D:PAI(A;;0xd01f01ff;;;SY)(A;;0xd01f01ff;;;BA)(A;;0x80120089;;;NS)"

$key.MachineContext = 1

$key.Create()

# create a certificate that is suitable for all purposes

$serverauthoid = new-object -com "X509Enrollment.CObjectId.1"

$serverauthoid.InitializeFromValue("1.3.6.1.4.1.311.10.12.1")

$ekuoids = new-object -com "X509Enrollment.CObjectIds.1"

$ekuoids.add($serverauthoid)

$ekuext = new-object -com "X509Enrollment.CX509ExtensionEnhancedKeyUsage.1"

$ekuext.InitializeEncode($ekuoids)

# this script will create certificate that is good for 1000 days – you can make it

# longer if you choose. After that time, the storage service will stop accepting

# indications and may stop handling mutual auth

$cert = new-object -com "X509Enrollment.CX509CertificateRequestCertificate.1"

$cert.InitializeFromPrivateKey(2, $key, "")

$cert.Subject = $name

$cert.Issuer = $cert.Subject

$cert.NotBefore = get-date

$cert.NotAfter = $cert.NotBefore.AddDays(1000)

$cert.X509Extensions.Add($ekuext)

$cert.Encode()

$enrollment = new-object -com "X509Enrollment.CX509Enrollment.1"

$enrollment.InitializeFromRequest($cert)

$certdata = $enrollment.CreateRequest(0)

$enrollment.InstallResponse(6, $certdata, 0, "")

# Now find the cert we just created (it will be in ‘MY’) and hold on to the thumbprint

cd cert:

$thumbprint=(dir -recurse | where {$_.Subject -match"CN=msstrgsvc*"} | `

  Select-Object -Last 1).thumbprint

$thumbprint

# clear out old certificates for this port and add our new one for all IPv4 and IPv6

# addresses. Deletes might fail if there were no old certificates.

# Note that I tried to use Server 2012 PowerShell cmdlets but I couldn’t exactly match

# all the requirements with the existing ones, so some of the commands rely on netsh

Set-Alias netsh c:\Windows\System32\netsh.exe

echo "You can ignore delete failures if this is a first time conifugration"

netsh http delete sslcert ipport=0.0.0.0:5990

netsh http delete sslcert ipport=[::]:5990

netsh http add sslcert ipport=0.0.0.0:5990 certhash=$($thumbprint) `

  appid="{468e21d1-a4cb-4134-8d9f-800c5ff2086f}"

netsh http add sslcert ipport=[::]:5990 certhash=$($thumbprint) `

  appid="{468e21d1-a4cb-4134-8d9f-800c5ff2086f}"

# Apply an ACL to the port so that NETWORK SERVICE can bind to it.

# Delete might fail if the ACL was never applied before.

netsh http delete urlacl url=https://*:5990/

netsh http add urlacl url=https://*:5990/ user="NT AUTHORITY\NETWORK SERVICE"

# restart the storage service so it can bind to the port properly

# you will need to perform Update-StorageProviderCache if level was > 0

echo "Restarting the service. Remember to run Update-StorageProviderCache since this resets the cache."

Restart-Service MSStrgSvc

Step 2: From a PowerShell Administrative Command prompt, execute these commands:

PS C:\Users\Administrator> $policy = Get-ExecutionPolicy

PS C:\Users\Administrator> Set-ExecutionPolicy Bypass

PS C:\Users\Administrator> .\configsmis.ps1 (assuming this is the correct directory)

PS C:\Users\Administrator> Set-ExecutionPolicy $policy

Test indications to port 5990

Before sending indication to port 5990, you may want to verify that HTTPS listener is working properly. Open https://localhost:5990 on the storage server machine in IE and it will show “MSP Storage Project” (choose to ignore server certificate error).

Note: Hit cancel if prompted for a certificate.

Test that the listener can be reached by pointing your web browser to the storage service machine and port 5990 (e.g., https://<storage-servername>:5990. You should see a certificate error which you can ignore for this test – this is sufficient to show that you have reached the indication listener. 

One-way authorization

The storage service only supports HTTPS connections for indications. When a CIM server attempts to connect to the storage service to deliver an indication, it will challenge the storage service and request a certificate. If the provider validates the server certificate, that certificate will need to be placed in the trusted store for the CIM server. This will vary depending on vendor implementation and is not described here.

 Very often the CIM server does little or no validation of the certificate and no other action is required.

Mutual Authentication

 Not currently supported. 

Exporting the Storage Service certificate

The above script will place the msstgsvc certificate in the trusted store. Using the certificates snap-in, it is possible to export this cert for use by the provider if it does certificate validation. Make sure to export in a format the provider can understand, typically .DER or .P7B, and don’t include the private key if you are asked. 

Registry values that affect Indications

 All registry values are located in

 HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Storage Management

 The defaults enable indications with certificate checking so no changes should be needed.

Microsoft added the ability to manage storage using Storage Management Initiative (SMI-S) providers to Windows Server 2012. Sometimes things don't go quite the way you plan and some debugging is needed to figure out what is going on.

So in order to provide better ability for storage vendors and customers to debug problems encountered with SMI-S providers, the Windows Standards-Based Storage Management Service (I’ll refer to it as Storage Service) offers a tracing facility as well as various other logging options.

CIMXML Logging

If you followed my earlier blogs, you would have learned that SMI-S is an implementation of the Common Information Model (CIM) encoded in XML format (CIM-XML) and transported using the Hypertext Transport Protocol (HTTP).

It is possible to save the requests and responses in a log file so storage vendors can see what requests are issued and what responses were captured by the Storage Service. Most SMI-S providers have similar capabilities and the logs on either side can be compared if problems are encountered.

Normally, this form of logging is not enabled but it can be turned on fairly easily when necessary. It will have some (small) impact on performance, and the log file can get quite large. Note that no security information is recorded in this log so user names and passwords are never captured.

There are a few registry values that control CIMXML logging, all under the key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog

TIP: Although the default file name ends in .txt, I usually change the name to have a .xml extension. When the file is opened in a text editor like Notepad++, the editor applies color coding and allows collapsing and expanding sections of the log, as if it was XML (it’s pretty close, just the timestamps aren't).

The easiest way to enable CIMXML logging is using PowerShell. Open an administrative PowerShell window and use these cmdlets (no line break in the Set-ItemProperty cmdlet):

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogLevel" -Value "4" -Force

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogFileName" -Value "C:\Temp\cimxml.xml" -Force

You may need to restart the service for the change to take effect.

Restart-Service MsStrgSvc

After restarting the storage service, you will have to do a rediscovery operation (Update-StorageProviderCache) to the appropriate level. To disable:

Set-ItemProperty -Path "HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog" -Name "LogLevel" -Value "0" -Force

Restart-Service MsStrgSvc

Note: Logging is only available for SMI-S providers communicating through CIMXML, which is the most common transport. The Storage Service can also communicate with providers implemented under the Windows Management Instrumentation (WMI) framework. This logging function is not available for SMI-S WMI providers.

Windows Event Log

All provisioning activities are recorded in the StorageManagementService Event Log channel (which is enabled by default). Errors are also recorded here. The following sample shows a typical activity that was recorded when I registered a provider using HTTPS. You can see that the user name is logged so auditing of actions carried out through the Storage Service is possible.

Tracing

The Storage Service uses Event Tracing for Windows (ETW) which allows narrowing down issues without using a debugger or source code. In many code paths, the tracing capability has the ability to record unexpected error and warning conditions or informational messages in an Event Tracing Log (.etl) file. Tracing to a file can be enabled by using the logman command line.

Real-time tracing is possible using the TraceView utility (part of the Windows Driver Kit) and the appropriate symbol files. I mention this here so you can see that we have built in a lot of diagnostic capabilities into the service. Typically this information is only valuable to developers. However the public symbols do not contain the tracing information, so this is mostly useful if you need to open a support case.

To enable tracing, use the following command (from a command prompt, not PowerShell):

logman create trace "ETWTrace" -ow -o c:\ETWTrace.etl -p {18C2F19C-F79D-408F-837B-F0B23F20A0F7} 0x3f 0x5 -nb 16 16 -bs 1024 -mode Circular -f bincirc -max 4096 -ets

To stop tracing

logman stop "ETWTrace" –ets

This post is a part of the nine-part “What’s New in Windows Server & System Center 2012 R2” series that is featured on Brad Anderson’s In the Cloud blog.  Today’s blog post covers storage management through SMI-s and how it applies to the larger topic of “Transform the Datacenter.”  To read that post and see the other technologies discussed, read today’s post: “What’s New in 2012 R2: IaaS Innovations.”

Although my day job is no longer working on SMI-S, I did want to let folks know that a lot of work went into the Standards-Based Storage Management Service (aka Storage Service) in the upcoming Windows Server release. Here are the highlights:

Discovery - Discovery is what happens when you register (Register-SmisProvider) or update (Update-StorageProviderCache): the storage service tries to find out as much information as you wanted (there are four levels of discovery) and that information resides in the service’s cache so you don’t need to constantly go out to the provider to get the data. In Windows Server 2012, the way the information was discovered was through “walking the model”, which is to say, starting with an object and then following associations to get additional information. Unfortunately, this can take a long time on all but the smallest storage configurations.

For Windows Server 2012 R2, the mechanisms have been changed. Instead of model-walking, the service will do enumerations of objects and then figure out (in memory) how they inter-relate. This turns out to decrease discovery times up to 90%! We worked with vendors to make sure providers will work well with this change, and for some of those vendors, you will need to get updated providers.

Updates (through Indications) - I recently blogged about the indication support for SMI-S. The internals of this are much improved in Windows Server 2012 R2 and more provider changes will be caught through indications so that rediscovery won’t be needed as often. The information in the older blog still applies, except that the firewall rule will already be in place (the provided script will still run unchanged).

Secure connections (using HTTPS) - In my first posting, I advised against using Mutual Authentication with the storage service. For Windows Server 2012 R2, this has been improved. This applies to indications as well as normal SMI-S traffic. Follow the Indications blog for configuration information (you need to have the certificates in place). Not all providers will work well with mutual auth. I will post more about security in the near future.

Resiliency Settings - When creating pools and volumes using the Storage Management cmdlets, you could easily specify various parameters that were just never going to work. SMI-S is too generic here. This has been simplified - stick with vendor defined settings and you should be fine. For Windows Server 2012 R2, we’ll give you an error if you try to override any of the parameters.

Pull Operations - One of the efficiency improvements is to change how enumerations are done by using a newer mechanism called “pull” operations. (You can read more about this here.) This allows chunking of the data coming back from the provider, and generally will lower the memory required on the provider side for large configurations. This works now with EMC providers; others will update in the future. To enable pull operations, you will need to modify the registry value

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Storage Management\PullOperationCount

Set it to something like 100 which tells the provider to send 100 instances of a particular class at a time. Through PowerShell, this would look like:

To turn it off, just set the value back to 0. (Leaving it enabled for a provider that does not support pull operations will only cause a small performance degradation so you might want to leave this set if you have more than one provider and either supports pull operations.)

Registering a provider - You can now register a provider that does not have any arrays to manage (yet). An Update-StorageProviderCache cmdlet will find any new devices at a later time. More significantly, if for some reason the provider lost contact with an array, this would result in errors that could be difficult to recover from.

Node/Port address confusion- In Windows Server 2012, we had these defined backwards from what is mandated by standards or used by things like the iSCSI initiator. This has been corrected but may require provider updates because we found some bugs in existing implementations.

Snapshot/Volume deletion - Under some conditions, it would not be possible to delete a volume or a snapshot. For example, we sometimes thought a volume was exposed to a host when it wasn’t, or a snapshot was in the wrong state for deletion. This has been improved.

Snapshot target pool - Specifying a -TargetStoragePoolName for snapshots is now supported, that is, if the provider/array allows it. However, be careful when you have more than one array with pools of the same name (which might be common).

Masking operations. There have been cases where masking/unmasking operations can take a long time to complete, particularly when SC VMM issues multiple requests in parallel; these could result in timeouts. Also with the current SMI-S model, masking operations performed by Windows might require multiple method calls to providers. Windows Server 2012 R2 now uses jobs for such operations instead of making all masking operations synchronous.

To see all of the posts in this series, check out the What’s New in Windows Server & System Center 2012 R2 archive.

Context

Windows Server 2012 R2 ships with a rich set of standards-compliant storage management functionality. This functionality was originally introduced in Windows Server 2012, and you should reference Jeff’s excellent blog that introduced related concepts.

iSCSI Target Server on its part shipped its SMI-S provider as part of the OS distribution for the first time in Windows Server 2012 R2. For more details on its design and features, see my previous blog post iSCSI Target Server in Windows Server 2012 R2.

I will briefly summarize here how the iSCSI Target Server SMI-S provider fits into Storage management ecosystem, please refer to Jeff’s blog post for the related deep-dive discussion. Storage cmdlets are logically part of the Storage Management API (SM-API) layer, and the ‘Windows Standards-based Storage Management Service’ plumbs the Storage cmdlet management interactions through to the new iSCSI Target Server SMI-S provider. This blog post is all about how you can use the new SMI-S provider hands-on, using the SM-API Storage cmdlets. Note that iSCSI Target Sever can alternatively be managed through its native iSCSI-specific cmdlets, and related WMI provider APIs. While the native approach allows you to comprehensively manage all iSCSI-specific configuration aspects, the Storage cmdlet approach helps you normalize on a standard set of cmdlets across different technologies (e.g. Storage Spaces, 3^rd party storage arrays).

Before we jump into the discussion of Storage cmdlet-based management, you should keep two critical caveats in mind:

• Jeff’s caution about having multiple management clients potentially tripping up each other; to quote –

“You should think carefully about where you want to install this service – in a datacenter you would centralize the management of your resources as much as practicable, and you don’t want to have too many points of device management all competing to control the storage devices. This can result in conflicting changes and possibly even data loss or corruption if too many users can manage the same arrays.”

• You must decide how you want to manage a Windows-based iSCSI Target Server starting from the time the feature is installed on Windows Server. To begin with, you are free to choose from either WMI/PowerShell, or SM-API/SMI-S. But once you start using one management approach, you must stick with that approach until the iSCSI Target Server is decommissioned. Switching between the management approaches could leave the iSCSI Target Server in an inconsistent and unmanageable state, and potentially could even cause data loss.

For most users, the compelling reason for managing an iSCSI Target Server through SMI-S is usually one of the following:

1. They have existing scripts that already are written to the Windows Storage cmdlets and the Windows SMI-S cmdlets, and want to use the same scripts to manage Windows Server-based iSCSI Target Server, or,
2. They use 3^rd party storage management products that consume the Storage Management API (SM-API) and they plan to manage Windows Server-based iSCSI Target Server using that same 3^rd party software, or,
3. (Perhaps most likely) They use SCVMM to manage iSCSI Target Server-based storage through SMI-S. This can be accomplished using SCVMM-specific cmdlets and UI, and is covered in detail elsewhere. So it will not be the focus of this blog post, we will focus only on non-SCVMM management approach in this blog post.

Relating Terminology

Let us do a quick review of SM-API/SMI-S concepts so you can intuitively relate them to native Windows terminology as we move into hands-on discussion:

Register SMI-S Provider on a Management Client

To register an SMI-S provider, you need to know the provider’s URI – it is the machine name for a standalone target and the cluster resource group name in the case of a clustered iSCSI Target Server – and credentials for a user account that is in that target server’s local administrators security group. In the following example, the SMI-S provider can be accessed on the machine “fsf-7809-09” and the user account with administrative privileges is “contoso\user1”. Get-Credential cmdlet prompts for the password at run time (you can reference Get-Credential page for other more scripting-friendly, albeit less secure, options to accomplish the same).

Discover the Storage Objects

After registering the provider, you can now update storage provider cache to get an inventory of all the manageable storage objects through this SMI-S provider:

You can then list the storage subsystem and related details. Note that although the following screen shots show items related to Storage Spaces, they are unrelated to iSCSI Target Server SMI-S provider. iSCSI Target Server SMI-S provider items are highlighted in green.

You can inspect the available storage pools and filter by the friendly name, as shown in the example. Notice that the hosting volumes, which you already know iSCSI Target Server reports as storage pools, carry friendly names that include respective drive letters on the target server. Also notice the Primordial storage pool, which effectively represents entire capacity available on the iSCSI Target Server. However, keep in mind that you can create SCSI Logical Units only out of what SMI-S calls “concrete storage pools”, i.e. only from pools which have the ‘IsPrimordial’ attribute set to false in the following screen shot.

Create Storage Objects

The first operation we show carves out a new logical disk out of an existing concrete storage pool. You can use one of two cmdlets to create a new virtual disk: New-VirtualDisk and New-StorageSubsystemVirtualDisk. Technically, iSCSI Target Server SMI-S provider works fine with either cmdlet and we will show examples for both, although you probably want to use the New-VirtualDisk so you can intentionally select the storage pool to provision the storage volume from. New-StorageSubsystemVirtualDisk in contrast auto-selects the storage pool to provision the capacity from.

Or you can provision the possible maximum-sized virtual disk by using “-UseMaximumSize” parameter as shown:

If you prefer to use the New-StorageSubsystemVirtualDisk cmdlet, you need to specify the storage subsystem parameter, and in the example below, you can see it auto-selected a storage pool in the selected subsystem - the “iSCSITarget: FSF-7809-09: C:” pool.

With the New-VirtualDiskSnapshot cmdlet, you can take a snapshot of a virtual disk.

To create masking set for a storage subsystem, you can use New-MaskingSet. You must include at least one initiator access in the new masking set. You can also map one or multiple virtual disks to the new masking set. An iSCSI initiator is identified by its iqn name and the virtual disks through their names. The script below creates a new masking set and adds one initiator and two virtual disks. And then we will query the new masking set to confirm the details.

Modify and Remove the Storage Objects

With the Resize-VirtualDisk cmdlet, you can expand an existing virtual disk as shown in the following example. Note however that you will still need to extend the partition and the volume to be able to make the additional capacity usable.

You can also modify the masking set that you’ve just created, by adding additional virtual disks or additional initiators to it. Look at the following examples. Note however that you do not really want to share a virtual disk with a file system across multiple initiators unless the initiators (hosts) are clustered, else the setup will inevitably cause data corruption sooner or later!

You can of course also remove the masking set and virtual disks that you just created, as we illustrate in the following examples. Further, note also the order of operations – you have to remove a virtual disk first from masking sets (or remove the masking sets), and then delete the virtual disk.

Finally, when you no longer need to manage the iSCSI Target Server from this management client, you can unregister the SMI-S provider as shown in the following example.

I want to acknowledge my colleague Juan Tian who had helped me with all the preceding Windows PowerShell examples.

Finally, I sincerely hope that you now have the tools you need to work with an iSCSI Target Server SMI-S provider. Give it a try and let me know how it’s working for you!

Context

iSCSI Target Server shipped its SMI-S provider as part of the OS distribution for the first time in Windows Server 2012 R2. For more details on its design and features, see my previous blog post iSCSI Target Server in Windows Server 2012 R2.

System Center 2012 SP1 Virtual Machine Manager (VMM) and later versions manage the storage presented from an iSCSI Target Server for provisioning block storage to Hyper-V hosts. VMM configuration guidance for managing the iSCSI Target Server running on Windows Server 2012 is available in this TechNet page. This guidance is still accurate for Windows Server 2012 R2 with the two following exceptions.

1. iSCSI Target Server SMI-S provider is now included as part of the OS distribution, so you no longer need to install it from VMM media. In fact, only the SMI-S provider included in Windows Server 2012 R2 distribution is the compatible supported provider. Further, when you install the iSCSI Target Server feature, the right SMI-S provider is transparently installed.
2. SAN-based Rapid Provisioning scenario of VMM requires one additional step to work with iSCSI Target Server

The rest of this blog post is all about #2.

VMM SAN-based Rapid Provisioning

VMM SAN-based rapid provisioning, as the name suggests, helps an administrator rapidly provision new Hyper-V virtual machines. The key to this fast provisioning is copying the VHD files for the new virtual machine in the most efficient possible manner. In this case, VMM relies on iSCSI Target Server snapshot functionality to accomplish this. Specifically, iSCSI Target Server SMI-S provider exposes this snapshot functionality for usage by SM-API storage management framework, which VMM then uses to create iSCSI Virtual Disk snapshots. As a brief aside, check out my previous blog post for examples on how the same iSCSI SMI-S snapshot functionality can be used by a storage administrator directly via SM-API Storage cmdlets, outside of VMM.

Let’s focus back on VMM though, especially on the snapshot-related VMM rapid provisioning work flow and what each of these steps mean to the iSCSI Target Server:

1. Administrator creates, customizes, and generalizes (syspreps) the desired VM OS image on a storage volume, hosted on iSCSI Target Server storage
   • iSCSI Target Server perspective: It simply exposes a VHDX-based Virtual Disk as a SCSI disk to the connecting initiator. All the creation, customization and sysprep actions are simply I/Os on that SCSI Logical Unit (LU).
2. Administrator mounts that SCSI LU on the VMM Library Server, let’s call it Disk-G for Golden, hosting the storage volume. Administrator also makes sure to mask the LU from any other initiators.
   • iSCSI Target Server perspective: Disk-G is persisted as a VHDX format file on the hosting volume, but the initiator (Library Server) does not know or care about this server-side implementation detail
3. Administrator creates a VM template and associates the generalized SAN copy-capable OS image VHD files to this template. This process thus makes the template a SAN copy-capable VM template.
   • iSCSI Target Server perspective: This action is transparent to the iSCSI Target Server, it does not participate unless there are specific related I/Os to Disk-G
4. From this point on, VMM can rapidly provision each new VM by creating a snapshot of Disk-G (say Disk-S1, Disk-S2 etc.) and assigning it to the appropriate Hyper-V host that will host the new VM guest being instantiated.
   • iSCSI Target Server perspective: For each disk snapshot taken via SMI-S, iSCSI Target Server creates a Diff VHDX file to store its content, so effectively:
     • Disk-G parent VHDX file
     • Disk-S1 Diff VHDX (Disk-G is parent)
     • Disk-S2 Diff VHDX (Disk-G is parent)

For a more detailed discussion of SAN-based VMM rapid provisioning concepts, see this TechNet Library article.

The entire scenario of course works flawlessly both in Windows Server 2012 and Windows Server 2012 R2. However in Windows Server 2012 R2, it turns out the storage administrator needs to take one additional step between Steps #3 and #4 – let’s call it “Step 3.5” – in the preceding list. Let’s then discuss what exactly changed in Windows Server 2012 R2 and what the additional step is.

iSCSI Target Server SMI-S Snapshots in Windows Server 2012 R2

On each successful SMI-S snapshot request, iSCSI Target Server creates a Diff VHDX-based iSCSI Virtual Disk. In Windows Server 2012 R2, iSCSI Target Server realizes this through native Hyper-V APIs. In contrast, iSCSI Target Server used to have its own implementation of creating Diff VHD files back in Windows Server 2012 – see my discussion of redesigned persistence layer in Windows Server 2012 R2 in one of my earlier blog posts for more detail. The new Hyper-V APIs enforce that the parent VHDX file must not be open in read & write mode while the new Diff VHDX is being created. This is to ensure that the parent VHDX can no longer be written to, once the diff VHDX is created. Thus while creating Disk-S1/S2 iSCSI Target Server SMI-S snapshots in the example discussion, Disk-G cannot stay mounted for read/write by the Library Server. Disk-G must be unmounted and re-mounted as read-only disk first – otherwise, creation of snapshots Disk-S1 and Disk-S2 will fail.

Now you might be wondering why this wasn’t an issue in Windows Server 2012-based iSCSI Target Server. iSCSI Target Server’s private implementation of Diff VHD creation in Windows Server 2012 did not enforce the read-only requirement on the parent VHD file, but the VMM Library server (initiator) always ensures that no more writes are performed on Disk-G once the sysprep process is complete. So the overall solution worked just fine in Windows Server 2012. With Windows Server 2012 R2 though, in addition to the same initiator behavior, iSCSI Target Server is effectively adding an additional layer of safety on the target (server) side to ensure writes are simply not possible at all on Disk-G. This is additional goodness.

I have briefly alluded to the nature of the additional process step required for rapid provisioning, but here’s the complete list of actions within that additional step:

“Step 3.5”:

• Save the volume mount point (and the drive letter if applicable) and offline the iSCSI LU on the Library Server.
• Unmap the LU from its current masking set (Remove-VirtualDiskFromMaskingSet). This ensures that the LU under the previous read/write access permissions can no longer be accessed by any initiator.
• Re-add the same SCSI LU (Add-VirtualDiskToMaskingSet) back to the same masking set, albeit this time as read-only through the “-DeviceAccesses ReadOnly” PS parameter. This sets the disk access to Read-Only.
  • Note: Only one Library Server should have the volume mounted off that SCSI LU. Even if the Library Server is configured as a highly-available failover cluster, only one of the cluster nodes should have mounted the disk at one time.
• Online the SCSI LU on the Library Server and restore its previous mount point (and drive letter, if applicable)

Here is the good news. My colleague Juan Tian has written a sample Windows PowerShell script which takes a single VMM template name parameter and performs all the above actions in the “Step 3.5” in one sweep. The script should work without any changes if you run it with VMM administration credentials. Feel free to check it out at the bottom of the blog post, customize if necessary for your deployment, and be sure to run it as “Step 3.5” in the VMM rapid deployment work flow that I summarized above.

Finally, let me wrap up this blog post with a quick version compatibility reference:

Hope this blog post provided you all the required details you need to move to production with your Windows Server 2012 R2-based new iSCSI Target Server and VMM. Give it a try and let me know how it’s working for you!

>>>>>>>>>SetLibraryServerLUToReadOnly.ps1 Windows PowerShell script>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

# Description:

# Script to set a VMM Library Server access of a SCSI Logical Unit (LU) to read-only access.

# Library Server and the disk (SCSI LU) are identified based on the VM template name parameter

# Script always offlines the disk, removes it from the masking set, and re-adds the disk back in as read-only.

# Script finally re-mounts the newly-read-only disk on the Library Server.

# Must be run with the VMM administration credentials

#

param([string] $VMTemplate = "")

if(!$VMTemplate)

{

$VMTemplate = Read-Host "Enter the name of your Template: "

}

Write-Host "Get Template $VMTemplate"

$libShare=(Get-SCLibraryShare -ID ((Get-SCVMTemplate -name $VMTemplate | Get-SCVirtualHardDisk).libraryshareid))

$share=$libShare.path

if ($share.count -lt 1)

{

Write-Host "Cannot find libraryshare!"

exit 1

}

Write-Host "Get libraryshare $share"

$path=(Get-SCVMTemplate -name $VMTemplate| Get-SCVirtualHardDisk).directory

if ($path.count -lt 1)

{

Write-Host "Cannot find SCVirtualHardDisk!"

exit 1

}

Write-Host "Get virtualdisk directory $path"

$path2=$path.replace($share, "")

$key="*"+$path2+"*"

Write-Host "Get Key $key"

$lib=($libShare.LibraryServer).FQDN

if ($lib.count -lt 1)

{

Write-Host "Cannot find libraryserver!"

exit 1

}

Write-Host "Get libraryserver $lib"

$partition = Invoke-Command -computername $lib -scriptblock {get-partition } | where-object {$_.accesspaths -like $key}

if (!$partition)

{

Write-Host "Cannot find disk partition!"

exit 1

}

$disk = $partition | Invoke-Command -computername $lib -scriptblock {get-disk}

if (!$disk)

{

Write-Host "Cannot find disk!"

exit 1

}

#offline disk

Write-Host "Offline disk ..."

$disk | Invoke-Command -computername $lib -scriptblock {set-disk -isoffline $true}

Write-Host "Offline disk completed!"

Write-Host "Looking for disk.uniqueid - $disk.uniqueid"

$vdisk = Get-VirtualDisk | where-object {$_.uniqueid -match $disk.uniqueid}

if (!$vdisk)

{

Write-Host "Cannot find virtual disk!"

exit 1

}

$ms = $vdisk | get-maskingset

if (!$ms)

{

Write-Host "Cannot find maskingset!"

exit 1

}

#remove virtual disk from masking set

Write-Host "Call Remove-VirtualDiskFromMaskingSet ..."

Remove-VirtualDiskFromMaskingSet -maskingsetuniqueid $ms.uniqueid -virtualdisknames $vdisk.Name

Write-Host "Call Remove-VirtualDiskFromMaskingSet completed!"

#add virtual disk back in masking set

Write-Host "Call Add-VirtualDiskToMaskingSet ..."

Add-VirtualDiskToMaskingSet -maskingsetuniqueid $ms.uniqueid -virtualdisknames $vdisk.Name -deviceaccesses ReadOnly

Write-Host "Call Add-VirtualDiskToMaskingSet completed!"

#online disk

Write-Host "Online disk ..."

$disk | Invoke-Command -computername $lib -scriptblock {set-disk -isoffline $false}

Write-Host "Online disk completed!”

Hi all!

We just released the Data Deduplication documentation for Windows Server 2016 over on TechNet! The new documentation includes a more detailed explanation of how Dedup works, crisper guidance on how to evaluate workloads for deduplication, and information on the available Dedup settings, with context for why you would want to change them.

Check it out:

As always, questions, concerns, or feedback are very welcome! Please feel free to comment at the bottom of this post, or reach out to us directly at dedupfeedback@microsoft.com.

Hi all,

I’m Jeff Patterson, Program Manager for Work Folders and Offline Files.

Windows 10, version 1607 will be available to Enterprise customers soon so I wanted to cover support for Windows Information Protection (a.k.a. Enterprise Data Protection) when using Work Folders or Offline Files.

Windows Information Protection Overview

Windows Information Protection (WIP) is a new security feature introduced in Windows 10, version 1607 to protect against data leaks.

Benefits of WIP

• Separation between personal and corporate data, without requiring employees to switch environments or apps
• Additional data protection for existing line-of-business apps without a need to update the apps
• Ability to wipe corporate data from devices while leaving personal data alone
• Use of audit reports for tracking issues and remedial actions
• Integration with your existing management system (Microsoft Intune, System Center Configuration Manager 2016, or your current mobile device management (MDM) system) to configure, deploy, and manage WIP for your company

For additional information on Windows Information Protection, please reference our TechNet documentation.

Work Folders support for Windows Information Protection

Work Folders was updated in Windows 10 to support Windows Information Protection.

If a WIP policy is applied to a Windows 10 device, all user data stored in the Work Folders directory will be encrypted using the same key and Enterprise ID that is used by Windows Information Protection.

Note: The user data is only encrypted on the Windows 10 device. When the user data is synced to the Work Folders server, it’s not encrypted on the server. To encrypt the user data on the Work Folders server, you need to use RMS encryption.

Offline Files and Windows Information Protection

Offline Files (a.k.a. Client Side Caching) is an older file sync solution and was not updated to support Windows Information Protection. This means any user data stored on a network share that’s cached locally on the Windows 10 device using Offline Files is not protected by Windows Information Protection.

If you’re currently using Offline Files, our recommendation is to migrate to a modern file sync solution such as Work Folders or OneDrive for Business which supports Windows Information Protection.

If you decide to use Offline Files with Windows Information Protection, you need to be aware of the following issue if you try to open cached files while working offline:

Can’t open files offline when you use Offline Files and Windows Information Protection
https://support.microsoft.com/en-us/kb/3187045

Conclusion

Offline Files does not support Windows Information Protection, you should use a modern file sync solution such as Work Folders or OneDrive for Business that supports WIP.

Hello, Claus here again.

One of the most important aspects when creating a volume is to choose the resiliency settings. The purpose of resiliency is to provide resiliency in case of failures, such as failed drive or a server failure. It also enables data availability when performing maintenance, such as server hardware replacement or operating system updates. Storage Spaces Direct supports two resiliency types; mirror and parity.

Mirror resiliency

Mirror resiliency is relatively simple. Storage Spaces Direct generates multiple block copies of the same data. By default, it generates 3 copies. Each copy is stored on a drive in different servers, providing resiliency to both drive and server failures. The diagram shows 3 data copies (A, A’ and A’’) laid out across a cluster with 4 servers.

Figure 1 3-copy mirror across 4 servers

Assuming there is a failure on the drive in server 2 where A’ is written. A’ is regenerated from reading A or A’’ and writing a new copy of A’ on another drive in server 2 or any drive in server 3. A’ cannot be written to drives in server 1 or server 4 since it is not allowed to have two copies of the same data in the same server.

If the admin puts a server in maintenance mode, the corresponding drives also enters maintenance mode. While maintenance mode suspends IO to the drives, the administrator can still perform drive maintenance tasks, such as updating drive firmware. Data copies stored on the server in maintenance mode will not be updated since IOs are suspended. Once the administrator takes the server out of maintenance mode, the data copies on the server will be updated using data copies from other servers. Storage Spaces Direct tracks which data copies are changed while the server is in maintenance mode, to minimize data resynchronization.

Mirror resiliency is relatively simple, which means it has great performance and does not have a lot of CPU overhead. The downside to mirror resiliency is that it is relatively inefficient, with 33.3% storage efficiency when storing 3 full copies of all data.

Parity resiliency

Parity resiliency is much more storage efficient compared to mirror resiliency. Parity resiliency uses parity symbols across a larger set of data symbols to drive up storage efficiency. Each symbol is stored on a drive in different servers, providing resiliency to both drive and server failures. Storage Spaces Direct requires at least 4 servers to enable parity resiliency. The diagram shows two data symbols (X₁ and X₂) and two parity symbols (P₁ and P₂) laid out across a cluster with 4 servers.

Figure 2 Parity resiliency across 4 servers

Assuming there is a failure on the drive in server 2 where X₂ is written. X₂ is regenerated from reading the other symbols (X₁, P₁ and P₂), recalculate the value of X₂ and write X₂ on another drive in server 2. X₂ cannot be written to drives in others servers, since it is not allowed to have two symbols in the same symbol set in the same server.

Parity resiliency works similar to mirror resiliency when a server is in maintenance mode.

Parity resiliency has better storage efficiency than mirror resiliency. With 4 servers the storage efficiency is 50%, and it can be as high as 80% with 16 servers. The downside of parity resiliency is twofold:

• Performing data reconstruction involves all of the surviving symbols. All symbols are read, which is extra storage IO, Lost symbols are recalculated, which incurs expensive CPU cycles and written back to disk.
• Overwriting existing data involves all symbols. All data symbols are read, data is updated, parity is recalculated, and all symbols are written. This is also known as Read-Modify-Write and incurs significant storage IO and CPU cycles.

Local Reconstruction Codes

Storage Spaces Direct uses Reed-Solomon error correction (aka erasure coding) for parity calculation in smaller deployments for the best possible efficiency and resiliency to two simultaneous failures. A cluster with four servers has 50% storage efficiency and resiliency to two failures. With larger clusters storage efficiency is increased as there can be more data symbols without increasing the number of parity symbols. On the flip side, data reconstruction becomes increasingly inefficient as the total number of symbols (data symbols + parity symbols) increases, as all surviving symbols will have to be read in order to calculate and regenerate the missing symbol(s). To address this, Microsoft Research invented Local Reconstruction Codes, which is being used in Microsoft Azure and Storage Spaces Direct.

Local Reconstruction Codes (LRC) optimizes data reconstruction for the most common failure scenario, which is a single drive failure. It does so by grouping the data symbols and calculate a single (local) parity symbol across the group using simple XOR. It then calculates a global parity across all the symbols. The diagram below shows LRC in a cluster with 12 servers.

Figure 3 LRC in a cluster with 12 servers

In the above example we have 11 symbols, 8 data symbols represented by X₁, X₂, X_3, X_4, Y_1, Y_2, Y₃ and Y₄, 2 local parity symbols represented by P_X and P_Y, and finally one global parity symbol represented by Q. This particular layout is also sometimes described as (8,2,1) representing 8 data symbols, 2 groups and 1 global parity.

Inside each group the parity symbol is calculated as simple XOR across the data symbols in the group. XOR is not a very computational intensive operation and thus requires few CPU cycles. Q is calculated using the data symbols and local parity symbols across all the groups. In this particular configuration, the storage efficiency is 8/11 or ~72%, as there are 8 data symbols out of 11 total symbols.

As mentioned above, in storage systems a single failure is more common than multiple failures and LRC is more efficient and incurs less storage IO when reconstructing data in the single device failure scenario and even some multi-failure scenarios.

Using the example from figure 3 above:

What happens if there is one failure, e.g. the disk that stores X₂ fails? In that case X₂ is reconstructed by reading X_1, X_3, X_4, and P_X (four reads), perform XOR operation (simple), and write X₂ (one write) on a different disk in server 2. Notice that none of the Y symbols or the global parity Q are read or involved in the reconstruction.

What happens if there are two simultaneous failures, e.g. the disk that stores X₁ fails and the disk that stores Y₂ also fails. In this case, because the failures occurred in two different groups, X₁ is reconstructed by reading X_2, X_3, X₄ and P_X (four reads), perform XOR operation, and write X₁ (one write) on a different disk in server 1. Similarly, Y₂ is reconstructed by reading Y_1, Y_3, Y₄ and P_Y (four reads), perform XOR operation, and write Y₂ (one write) to a different disk in server 5. A total of eight reads and two writes. Notice that only simple XOR was involved in data reconstruction thus reducing the pressure on the CPU.

What happens if there are two failures in the same group, e.g. the disks that stores X₁ and X₂ have both failed. In this case X₁ is reconstructed by reading X_3, X₄ P_X, Y₁, Y_2, Y_3, Y₄ and Q (8 reads), perform erasure code computation and write X₁ to a different disk in server 1. It is not necessary to read P_Y, since it can be calculated it from knowing Y_1, Y_2, Y₄ and Y₄. Once X₁ is reconstructed, X₂ can be reconstructed using the same mechanism described for one failure above, except no additional reads are needed.

Notice how, in the example above, one server does not have symbols? This configuration allows reconstruction of symbols even in the case where a server has malfunctioned and is permanently retired, after which the cluster effective will have only 11 servers until a replacement server is added to the cluster.

The number of data symbols in a group depends on the cluster size and the drive types being used. Solid state drives perform better, so the number of data symbols in a group can be larger. The below table, outlines the default erasure coding scheme (RS or LRC) and the resulting efficiency for hybrid and all-flash storage configuration in various cluster sizes.

Accelerating parity volumes

In Storage Spaces Direct it is possible to create a hybrid volume. A hybrid volume is essentially a volume where some of the volume uses mirror resiliency and some of the volume uses parity resiliency.

Figure 4 Hybrid Volume

The purpose of mixing mirror and parity in the volume is to provide a balance between storage performance and storage efficiency. Hybrid volumes require the use of the ReFS on-disk file system as it is aware of the volume layout:

• ReFS always writes data to the mirror portion of the volume, taking advantage of the write performance of mirror
• ReFS rotates data into the parity portion of the volume when needed, taking advantage of the efficiency of parity
• Parity is only calculated when rotating data into the parity portion
• ReFS writes updates to data stored in the parity portion by placing new data in the mirror portion and invalidating the old stored in to parity portion – again to take advantage of the write performance of mirror

ReFS starts rotating data into the parity portion at 60% utilization of the mirror portion and gradually becomes more aggressive in rotating data as utilization increases. It is highly desirable to:

• Size the mirror portion to twice the size of the active working set (hot data) to avoid excessive data rotation
• Size the overall volume to always have 20% free space to avoid excessive fragmentation due to data rotation

Conclusion

I hope this blog post helps provide more insight into how mirror and parity resiliency works in Storage Spaces Direct, how data is laid out across servers, and how data is reconstructed in various failure cases.

We also discussed how Local Reconstruction Codes (LRC) increases the efficiency of data reconstruction in both reduced storage IO churn and CPU cycles, and overall helps reach a healthy system quicker.

And finally we discussed how hybrid volumes provide a balance between the performance of mirror and the efficiency of parity.

Let me know what you think.

Until next time

Claus

Hi folks, Ned here again. I have a very quick survey for you if you’re a Windows Server customer. It is anonymous, only has one mandatory question, and one secondary optional question – shouldn’t take you more than 30 seconds and will help us understand our customer base better.

Why do you deploy R2 versions so much more than non-R2?

Thanks in advance.

• Ned “actual monkey” Pyle