Convert vmdk disk from Auto-growth to Preallocated

I made the unfortunate decision not to preallocate my 100GB C: on my SharePoint development machine. Since it was set to auto grow, only after a few months I had no less than 9 vmdk files, and the drive had ballooned out to over 200GB. With SSD space precious I had to try and compact this VM. Using the "Compact" tool within the VM settings didn't seem to make much of a difference at all. What I needed to do, so I thought, was "shrink" the vmdks. After reading a lot of very out of date help/forums from VMWare, it looked like I needed to fire up the vmware-vdiskmanager.exe and shrink the vmdk files via that. All good so far. Instructions lead me to believe I had to mount the disks first and prepare them. Since I am running Workstation 8, I think these instructions were quite out-of-dated. If you are interested you can mount/map a vmdk files to your host and browse it's contents (without the VM started!) - pretty cool! But this was not going to help my issue. Turns out I could skip the "mount and prepare" steps and lunch straight into the "shrink".

I located the vmware-vdiskmanager.exe at C:\Program Files (x86)\VMware\VMware Workstation and ran the command (-k is for shrinking a vmdk):

vmware-vdiskmanager.exe -k d:\MyVM\MyVeryLargeDisk.vmdk

Shrinking began, and I thought I was in the home straight now. On the first of the 9 vmdk files, the shrinkage was extremely poor (your mileage may vary). It only shrink about 2%. I tried this on the remaining vmdk files and the result was the same. Shrinking had not solved my problem, and in my case the overall VM reduced from 240BG to 235GB. I have no idea why shrinkage was so ineffective, especially as I knew I had a good 30% of free space within the VMs themselves. Plan B.

Since my D drive was preallocated at 50GB, there was 1 nice file of size 52,428,800kBs (50 x 1024 x1024). I need to convert my ballooned C drive into a prealloacted drive. I tried the follow command:


vmware-vdiskmanager.exe -r d:\MyVM\MyVeryLargeDisk.vmdk -t 2 d:\MyVM\newsmallerdisk.vmdk

The console app started ...
..."Creating disk 'd:\MyVM\newsmallerdisk.vmdk'" ...

And eventually (after about 3 hours on a SATA drive)
  Convert: 100% done.
  Virtual disk conversion successful.

Last steps were to :

1. go into VMware Workstation
2. open the VM (not power on)
3. remove the non-preallocated hard drive
4. add the new pre-allocated hard drive (newsmallerdisk.vmdk)
5. Save and Start.

Done. I could then backup and delete those remaining old 9 vmdk files and happy days. Moved everything back to my modest SSD.

PS - some of those options like "-t 2" ... related to the disk type:

  Disk types:

      0                   : single growable virtual disk

      1                   : growable virtual disk split in 2GB files

      2                   : preallocated virtual disk

      3                   : preallocated virtual disk split in 2GB files

      4                   : preallocated ESX-type virtual disk

      5                   : compressed disk optimized for streaming

      6                   : thin provisioned virtual disk - ESX 3.x and above

SharePoint Dev Exam - 70-573 - Study Links

This excel spreadsheet contains links to useful articles that may help you whilst studying for the Microsoft SharePoint 2010, Application Development Exam.

Most of this is based off the list Becky Bertram put together so a big thanks to her.

I used this list to also track my progress, kind of like a timetable.

Hope it helps. You can download the spreadsheet here.

Site Provisioning Techniques

Whilst researching provisioning techniques, these are some notes I recorded.

Feature stapling

• Extend the existing site definition.
• DISADV: Only activated on provisioning time – so won’t affect already existing sites.
• DISADV: Can’t provide NEW site definitions with feature stapling
• DISADV: Feature stapling is activated before module or list element is activated in ONET.xml – can cause issues
• DISADV: Doesn’t work if provisioning Site Templates or Web Templates

Site Templates

• Creates a solution package (WSP)
• ADV: Can be moved between any environment bring with it all the libraries, lists, and optionally content
• ADV: Can provision sites using alternate xml definitions (using WebTemplate element and CustomSchema attribute for ListInstance element)
• ADV/DISADV: Can be imported into Visual Studio and tweaked, but there are an enormous amount of files.
• DISADV: Not supported when publishing features are enabled.

Web Templates

• Uses the WebTemplate element in Elements.xml, together with onet.xml site definition file
• DISADV: Cannot use feature stapling ... BUT since we can modify onet.xml directly and deploy a new version anytime, there’s no need to feature staple.
• DISADV: Variations won’t work with Web Templates BUT workarounds do exist.
• DISADV: No VS2010 template, but once a copy of onet.xml (from /site templates/*) is made then just add elements.xml. Not to hard.
• ADV: Supports both FARM Sandboxed Solutions. (site collection scoped for sandboxed solution)
• ADV: Feature based
• ADV: Supports Cloud deployment
• ADV: Can utilise the Publishing Feature in web templates (unlike site templates)
• ADV: Can easily change onet.xml and redploy it to affect all new sites. (won’t affect old ones since there is no runtime reference to them)
• ADV: Upgrades to SP2015?
• ADV: Hierarchies can be easily created using Feature Receivers.

Site Definitions

• Consist of WebTemp*.xml (how the definition is visible) and ONET.xml (contains the actual definitions for site provisioning)
• Site created off site definitions store identity in: SPWeb.Template, SPWeb.Configuration, and SPWeb.TempalteId
• ADV: Deploy via Farm Solutions, files located on the file system, but can’t do sandboxed solution (so that could be a disadvantage)
• ADV: Site Collection features are not activated in sub sites (unlike in SP2007)
• ADV: Can feature staple
• ADV: Visual Studio 2010 has a site definition template.
• ADV: Variations will only work with Site Definitions (not web templates, site templates etc)
• ADV: Hierarchies can be easily created using XML formats.
• DISADV: Not allowed to change the ONET.xml file after it has been used to provision a site. Never ever. Not supported.
• DISADV: No support for Cloud solutions
• DISADV: A lot of messy XML to deal with.
• DISADV: Potentially upgrade issues to deal with. They won’t upgrade automatically as OOB site defs will, so extra costs involved. MS will advise not to use Site Definitions going forward. Our MOSS RAP highlighted our current site defs needed to be remediated. (A bit over kill maybe)

Provisioning Providers

• Replace the onet.xml file with Code, and only use WebTemp*.xml.
• Use an OOB site definition as a starting point
• Inherit from Microsoft.SharePoint.SPWebProvisioningProvider
• Uses SPWeb.ApplyTemaplate()
• ADV: Supports using PortalProvisioningProvider (not foundation) which creates portal site templates (hierarchy files)
• DISADV: Not support for Cloud solutions
• DISADV: Indications from Microsoft are that it won’t be supported going forward and will soon be deprecated.
• DISADV: Can cause an infinite loop when using CallBack.

SharePoint 2010 Code Review

Having a consistant approach to code reviews in any SharePoint development team can go along way to ensuring the solutions produced are of a high quality (or atleast a higher quality than would have been possible without the review). Not only do we get the benefits in terms of more robust, elegant solutions, it is also a very good knowledge sharing exercise for the reviewer.

Here are a few quick thoughts on what makes a good SharePoint code review (ignoring the standard C# questions, like comments, coding standards, error handing):

• Have SharePoint objects such as SPSite and SPWeb been Disposed of appropriately?
• Has the Developer run SPDisposeCheck on the solution and what were the findings?
• Have Get methods been used where appropriate instead of using the index from a collection? ie oList.Items["abc"], as opposed to oList.GetItems(oQuery).
• Does the developer make good use of CAML queries to avoid long running enumerations of SharePoint collections?
• Has the developer overwritten throttling and if so, why?
• When Running as Elevated Privledges has the AllowUnsafeUpdate been set back to False?
• Is everything packaged in one or more features where possible?
• Has the developer modified any Out of the Box SharePoint files? If so, calmly walk him off the premises.
• When developing web parts, has the developer used RenderContents as opposed to CreateChildControls? If so, why?
• ...what else?

Creating a SharePoint Virtual Machine Template

I thought I would take the time to detail my experiences with creating a SharePoint Virtual Machine template. This post elaborates on what is explained in the VMWare white paper here. Reading this whitepaper was a bit of a eurika moment for me, but in hindsight a fairly simple concept. Essentially this all boils down to two things:

1. After installing SharePoint, ensure the check box Run the SharePoint Products and Technologies Configuration Wizard now is un-checked. 
2. When creating an instance of the VM, run SysPrep. 

By doing these two things, you are ensuring you're not writing machine specific information to the database and secondly the machine is generating new system IDs.

Once the base was completed, I found we could intantiate clones of this base in a few hours (max). That includes copying the VM (say 100 gig), running sysprep, and running psconfig. Pretty sweet.

Building the VM template.

My Development VM consisted of the following (all installed as local administrator):

• Windows Server 2008 R2 (64bit) 
• SQL Server 2008 R2 
• SharePoint Server (Enterprise) 2010 (and language packs where necessary) 
• Visual Studio Ultimate 2010 
• Office Professional Plus 2010 (x86) 
• Visio Professional 2010 
• SharePoint Designer 2010 
• SharePoint WorkSpace 2010 
• ULS Viewer 
• SharePoint Manager 2010 
• CKS Codeplex Visual Studio tools (v1.2)
• PowerGUI 
• ...obviously everything is service packed (except SharePoint). 

Once all the applications are installed, do an ipconfig /release and power off the VM.
Save a backup copy of the 'base'. Always a good idea.

Creating an instance from the Base VM

1. Make a copy the SharePoint VM template from the base. 
2. Rename the .vmx file and VM folder to the server name you will use. 
3. Whilst the VM is off, edit it's properties and rename the Virtual Machine Name to the intended VM name before boot up. 
4. When prompted, click "I copied it". 
5. Log in as the Administrator.
6. To run SysPrep, click to Start > Run, type sysprep and press OK. 
7. Select Enter System Out-of-Box Experience (OOBE). Uncheck Generalize. As 'Shutdown Options' select Reboot. 
8. After rebooting you’ll have to enter some data, for example, Country or region, Time and currency and Keyboard input. Also, accept EULA and reboot. 
9. Change the local administrators password when prompted. 
10. Right Click computer and click properties. Change the name of the server to the name that was changed in step 2. Remain in workgroups here. Do not join the domain yet. The system will ask you to reboot. 
11. After the reboot, log back in as Administrator. 
12. You’ll now attempt to join the domain. It might be a good idea to 'ping' the domain controller to see if the connection works. If it doesn’t you’ll probably have to change the VM’s network adapter from Bridged to NAT (that way you are using the host’s access to the domain). You will need domain admin access here to join the domain. 
13. Optional: once the computer has joined the domain, don’t power it back on until you have moved the object to the appropriate OU. Some OUs in organisation get a lot more settings pumped down to them via the GPO. We have a developement OU where only the minimal stuff gets downloaded.
14. Re-start the VM. 
15. Log back in as local Administrator 
16. Add your domain accounts as local administrators on the VM. 
17. Also, you will need to give this account access to SQL as well. I tend to use the same domain account as the farm account and service accounts for SharePoint (obviously not advised for production but this is for dev only) 
18. Log out and log in as this domain account. 

At this point you can now run the SharePoint 2010 Product Configuration Wizard (psconfig). An alternative approach here is to use the CodePlex project AutoSPInstaller (highly recommended). This will streamline your VM creation even better becuase it will create all the necessary service applications, and web applications, site collections, and farm solutions you like.

One thing to note: Every Microsoft product installed in the VM will have to be re-activated from a licensing perspective. This includes the OS. Don't bother activating the products in the base (and it pays to use your enterprise licenses not your MSDN license that everyone you share this base with will use. I learnt the hard way on that one)

Hope this helps a SharePoint development team build quicker VMs.