Github – Merge conflict and how to solve it

I recently contributed to one of Microsoft’s DSC resource modules via Github web interface (click on the file > edit > paste revised code > create pull request) as I’ve done countless times before. This time however I ran into a merge conflict that couldn’t be solved through the web site so I had to do it from the command line.

I contributed to the dev branch of the following repository and created a Pull Request:

  • Repository: https://github.com/PowerShell/xPSDesiredStateConfiguration
  • Branch: dev
  • Pull Request

There are countless guides on how to solve github merge conflicts but the ones I had a look at all presumed things that were missing in my case (see here or here).

So I clone my fork of the official xPSDesiredStateConfiguration repository and my default branch is dev:

PS D:\dsc> git clone https://github.com/megamorf/xPSDesiredStateConfiguration.git
Cloning into 'xPSDesiredStateConfiguration'...
remote: Counting objects: 713, done.
Receiving objects:  96% (685/713)   0 (delta 0), pack-reused 713 eceiving objects:  95% (678/713)
Receiving objects: 100% (713/713), 261.82 KiB | 0 bytes/s, done.
Resolving deltas: 100% (398/398), done.
Checking connectivity... done.
PS D:\dsc> cd .\xPSDesiredStateConfiguration\
PS D:\dsc\xPSDesiredStateConfiguration> git branch
* dev

Fetching updates from the upstream repo is required to update our fork but it fails because there is no upstream repository configured:

PS D:\dsc\xPSDesiredStateConfiguration> git fetch upstream
fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I add Microsoft’s repo as remote upstream location (as you can see it’s listed in the git remote output) and update my fork’s dev branch with all the changes from the official repo’s dev branch:

PS D:\dsc\xPSDesiredStateConfiguration> git remote add upstream https://github.com/PowerShell/xPSDesiredStateConfiguration.git
PS D:\dsc\xPSDesiredStateConfiguration> git remote -v
origin https://github.com/megamorf/xPSDesiredStateConfiguration.git (fetch)
origin https://github.com/megamorf/xPSDesiredStateConfiguration.git (push)
upstream https://github.com/PowerShell/xPSDesiredStateConfiguration.git (fetch)
upstream https://github.com/PowerShell/xPSDesiredStateConfiguration.git (push)
PS D:\dsc\xPSDesiredStateConfiguration> git fetch upstream
remote: Counting objects: 38, done.
remote: Total 38 (delta 18), reused 18 (delta 18), pack-reused 20
Unpacking objects: 100% (38/38), done.
From https://github.com/PowerShell/xPSDesiredStateConfiguration
 * [new branch] TravisEz13-patch-1 -> upstream/TravisEz13-patch-1
 * [new branch] dev -> upstream/dev
 * [new branch] master -> upstream/master
 * [new tag] 3.8.0.0-PSGallery -> 3.8.0.0-PSGallery
PS D:\dsc\xPSDesiredStateConfiguration> git merge upstream/dev
Updating 171b484..e1cc5eb
Fast-forward
 .../PublishModulesAndMofsToPullServer.psm1 | 167 +++++++++++++++++++++
 DSCPullServerSetup/README.md | Bin 0 -> 2482 bytes
 .../MSFT_xDSCWebService.Schema.mof | 1 -
 .../MSFT_xDSCWebService/MSFT_xDSCWebService.psm1 | 146 +++++++-----------
 .../PullServerSetupTests.ps1 | 135 +++++++++++++++++
 Examples/Sample_xDscWebService.ps1 | 14 +-
 Examples/Sample_xDscWebServiceRegistration.ps1 | 14 +-
 Examples/Sample_xDscWebServiceRemoval.ps1 | 7 -
 README.md | 15 +-
 Tests/Integration/MSFT_xDSCWebService.xxx.ps1 | 36 ++---
 appveyor.yml | 4 +-
 xPSDesiredStateConfiguration.psd1 | 5 +-
 12 files changed, 384 insertions(+), 160 deletions(-)
 create mode 100644 DSCPullServerSetup/PublishModulesAndMofsToPullServer.psm1
 create mode 100644 DSCPullServerSetup/README.md
 create mode 100644 Examples/PullServerDeploymentVerificationTest/PullServerSetupTests.ps1
PS D:\dsc\xPSDesiredStateConfiguration> ise .\DSCPullServerSetup\PublishModulesAndMofsToPullServer.psm1

But wait, something is wrong here – my changes aren’t visible in the ISE. Instead, the file seems to be the one from Microsoft’s dev branch. So where did my changes go?

It turns out that editing a file from a remote repository through the Github web interface creates an entirely new branch in your forked repository called patch-#. What you have to do is merge the upstream repo’s changes with that branch and not the dev branch of your fork. Then fix remaining merge conflicts manually, add the affected files and commit with a descriptive message before you push it to your fork.

PS D:\dsc\xPSDesiredStateConfiguration> git checkout patch-1
Branch patch-1 set up to track remote branch patch-1 from origin.
Switched to a new branch 'patch-1'
PS D:\dsc\xPSDesiredStateConfiguration> git merge upstream/dev
Auto-merging DSCPullServerSetup/PublishModulesAndMofsToPullServer.psm1
CONFLICT (content): Merge conflict in DSCPullServerSetup/PublishModulesAndMofsToPullServer.psm1
Automatic merge failed; fix conflicts and then commit the result.
PS D:\dsc\xPSDesiredStateConfiguration> ise DSCPullServerSetup/PublishModulesAndMofsToPullServer.psm1
PS D:\dsc\xPSDesiredStateConfiguration> git add DSCPullServerSetup/PublishModulesAndMofsToPullServer.psm1
PS D:\dsc\xPSDesiredStateConfiguration> git commit -m "Merged patch-1 fixed conflict."
[patch-1 c77509d] Merged patch-1 fixed conflict.
PS D:\dsc\xPSDesiredStateConfiguration> git branch
 dev
* patch-1
PS D:\dsc\xPSDesiredStateConfiguration> git status
On branch patch-1
Your branch is ahead of 'origin/patch-1' by 10 commits.
 (use "git push" to publish your local commits)
nothing to commit, working directory clean
PS D:\dsc\xPSDesiredStateConfiguration> git push

Github will tell you in your Pull Request whether all merge conflicts have been solved.

Hope this helps 🙂

Attention: DSC V2 Pull Server on 2012 R2 Core supported again

For a while now a DSC V2 Pull Server hasn’t been functioning properly on Server 2012 R2 Core. The underlying cause is that the xPSDesiredStateConfiguration resource module has been configuring the DSC Service with an OLEDB provider (Jet) which is actually deprecated starting with 2016 and doesn’t work with Server 2012 R2 Core.

The error messages you encounter if this bug has hit you look like this (see pull server event log):

Registration of Dsc Agent with AgentID = f7fb8bbc-f48a-435b-8952-e4a534f4d1b0 failed. The underlying error is : The ‘Microsoft.Jet.OLEDB.4.0’ provider is not registered on the local machine.

Since module version 3.8.0.0 the db provider has been changed to ESENT again which removes the incompatibility and makes the Pull Server work on 2012 R2 Core again.

This means that all V2 Pull Servers that were set up with the xDscWebService resource on Server 2012 R2 prior to 3.8.0.0 are configured with OLEDB so consider redeploying the Server entirely.

The file C:\inetpub\PSDSCPullServer\web.config should contain this section…:

<appSettings>
    <add key="MaxConcurrentRequests" value="10000" />
    <add key="MaxRequestsPerTimeslot" value="10000" />
    <add key="TimeslotSize" value="1" />
    <add key="dbprovider" value="ESENT" />
    <add key="dbconnectionstr" value="C:\Program Files\WindowsPowerShell\DscService\Devices.edb" />
    <add key="ConfigurationPath" value="C:\Program Files\WindowsPowerShell\DscService\Configuration" />
    <add key="ModulePath" value="C:\Program Files\WindowsPowerShell\DscService\Modules" />
    <add key="RegistrationKeyPath" value="C:\Program Files\WindowsPowerShell\DscService" />
  </appSettings>

…and upon registration of a Pull server client via LCM ReportServerWeb resource the edb gets created (indicators for proper configuration in red):

PS H:\> ls "C:\Program Files\WindowsPowerShell\DscService"


    Direcory: C:\Program Files\WindowsPowerShell\DscService


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d---s-       17.03.2016     16:49                Configuration
d---s-       17.03.2016     16:49                Modules
-a----       21.03.2016     18:33        1056768 Devices.edb
-a----       13.01.2016     04:59        1310720 Devices.mdb
-a----       21.03.2016     18:33           8192 edb.chk
-a----       21.03.2016     18:33         131072 edb.log
-a----       21.03.2016     18:33         131072 edb00001.log
-a----       21.03.2016     18:33         131072 edbres00001.jrs
-a----       21.03.2016     18:33         131072 edbres00002.jrs
-a----       21.03.2016     18:33         131072 edbtmp.log
-a----       21.03.2016     17:26             39 RegistrationKeys.txt
             21.03.2016     18:33        1056768 tmp.edb

Just a heads-up: Don’t use the sample scripts from the xPSDesiredStateConfiguration module as they are outdated (time of this writing – 22.03.16). Please have a look for the official DSC documentation at https://msdn.microsoft.com/en-us/powershell/dsc/pullserver for Pull Server configuration examples.

 

 

 

 

Quick Tip: Bulk remove selected vSphere snapshots

If you want to manually select the snapshots you want to delete in vSphere through PowerCLI the only viable option is piping your snapshots to the Out-GridView Cmdlet. Unfortunately it uses the default display properties of the snaps

$snaps = Get-VM | Get-Snapshot
$snaps | Out-GridView -PassThru 

VMwareSnapshotsOutGridViewDefaultProperties

As you can see the properties that are shown by default don’t help at all. So what do we do? Right, we select the properties we want to see before piping to Out-GridView:
VMwareSnapshotsOutGridViewSelectedProperties

So far so good, but what happens when we pipe the selected snapshots to Remove-Snapshot? It fails. Select-Object creates new objects and they have a different type than the snapshot objects:
VMwareSnapshotsObjectType

The best workaround is to use the Select-FromGridView function from the technet gallery which separates the displayed object’s properties from the objects passed down the pipeline:

$delete = $snaps | Select-FromGridView -By vm,created,name,description,sizegb
$delete | Remove-Snapshot -RunAsync -Confirm:$false -Verbose

Hope this helps 🙂

Quick tip: Pre-compile assemblies to improve PS startup time

Just a quick reminder for myself.

$FrameworkDir=[Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory()
$NGENPath = Join-Path $FrameworkDir 'ngen.exe'

[AppDomain]::CurrentDomain.GetAssemblies() |
  Select-Object -ExpandProperty Location |
  ForEach-Object { & $NGENPath """$_""" } 

Powershell loads and JIT compiles tons, and tons, and tons of stuff. This takes a lot of excessive CPU cycles and disk I/O.

ngen.exe (Native Image Generator) can be used to precompile the assemblies that Powershell loads during startup.

Source:
http://serverfault.com/questions/761301/calling-powershell-exe-is-extremely-slow

Using self-signed certificates for encryption

Update: Changed Hash Algorithm from SHA1 to SHA256 and pointed out that the certificate template needs to be properly encoded.

We needed a way to securely store credentials for a number of tasks in our Citrix environment so I suggested using an encryption certificate for that.

It’s quite simple:

[Version]
Signature = "$Windows NT$"

[Strings]
szOID_ENHANCED_KEY_USAGE = "2.5.29.37"
szOID_DOCUMENT_ENCRYPTION = "1.3.6.1.4.1.311.80.1"

[NewRequest]
Subject = "cn=CitrixPowershellEncryption"
MachineKeySet = false
KeyLength = 2048
KeySpec = AT_KEYEXCHANGE
HashAlgorithm = SHA256
Exportable = true
RequestType = Cert
KeyUsage = "CERT_KEY_ENCIPHERMENT_KEY_USAGE | CERT_DATA_ENCIPHERMENT_KEY_USAGE"
ValidityPeriod = "Years"
ValidityPeriodUnits = "1000"

[Extensions]
%szOID_ENHANCED_KEY_USAGE% = "{text}%szOID_DOCUMENT_ENCRYPTION%"

Save this certificate template as ASCII-encoded CitrixEncryptionTemplate.inf (it contains both the data and key encipherment key usage attributes to make it compliant with WMF5) and use it to create a certificate like this:

certreq.exe -new C:\Users\megamorf\Desktop\CitrixEncryptionTemplate.inf C:\Users\megamorf\Desktop\CitrixEncryption.cer

This will do two things:

  1. Generate the public key C:\Users\megamorf\Desktop\CitrixEncryption.cer
  2. Store the full certificate in Cert:\CurrentUser\My

I exported the full certificate with private key and all extensions as pfx and used a long and complex passphrase to protect it.

Import the pfx for the account where you need it and use it like this:

$Cert = Get-ChildItem Cert:\CurrentUser\My | Where-Object {$_.Subject -like "CN=CitrixPowershellEncryption"}

# Encryption:
$msg = "my super secret password"
$EncodedPwd = [system.text.encoding]::UTF8.GetBytes($msg)
$EncryptedBytes = $Cert.PublicKey.Key.Encrypt($EncodedPwd, $true)
$EncryptedPwd = [System.Convert]::ToBase64String($EncryptedBytes)
$EncryptedPwd

lwH5A9ltjlHyFmjJ84A+XSQC+ZQE6Yta/WaYnW1zPwf7tZNeNwbDx4KiiHMGardRO/q+Ixb9DDRT/kWckx8sLzp5MTOfiN5phsT8Nh+hh8GIcLKr0GrSTAhI
5kmvOgrYQAWFVBiS+WXS7v52g6VBNR8D41KgZxMChH5syE8pzNftGMfulDLjaf/pVNloPgfMvRy1yvf49lPJF5sfjEvu4heunJc4RhNZiHIx65SdWKRzOOpq
Jgi+3Qd1/6Qtpzj0ln5qEJEpjx/rRO9VdtTpnpEjYs9sJf0sY9PTHJU2I452jpLyy7RmCclzybz7TBMncZivbcexqpmqkAQ31VVyhg==

# Decryption:
$EncryptedBytes = [System.Convert]::FromBase64String($EncryptedPwd)
$DecryptedBytes = $Cert.PrivateKey.Decrypt($EncryptedBytes, $true)
$DecryptedPwd = [system.text.encoding]::UTF8.GetString($DecryptedBytes)
$DecryptedPwd

my super secret password

Using PowerShell v5

# Encryption:
# version 1 - public key is stored in the filesystem
Protect-CmsMessage -Content $msg -To .\CitrixEncryption.cer -OutFile .\EncryptedMessage.cms

# version 2 - public key from certificate store is used
Protect-CmsMessage -Content $msg -To "CN=CitrixPowershellEncryption" -OutFile .\EncryptedMessage.cms

# Decryption:
Unprotect-CmsMessage -Path .\EncryptedMessage.cms

my super secret password

Sources:

 

WMF5 finally re-released

Microsoft have finally released the long-awaited fixed version of WMF5. I personally believe that it’s intolerable to let one’s users wait two months just for this simple fix.

Today, we have republished the Windows Management Framework (WMF) 5.0 RTM packages for Windows Server 2012 R2, Windows Server 2012, Windows 2008 R2 SP1, Windows 8.1, and Windows 7 SP1 to Microsoft Download Center. These packages replace previously released WMF 5.0 RTM packages and WMF 5.0 Production Preview packages.

Important Note: For systems running Windows 7 SP1 or Windows Server 2008 R2 SP1, you’ll need WMF 4.0 and .NET Framework 4.5 or higher installed to run WMF 5.0.

We want to extend our sincere thanks to the PowerShell Community for your continued patience and support while we were addressing thePSModulePath issue found in the previously released WMF 5.0 RTM packages.

To learn about the new features and improvements in WMF 5.0 RTM, please refer to (and contribute to) the WMF 5.0 release notes.

Please continue to provide feedback via our new UserVoice site.

Changes in the republished packages
  • The KB numbers of these packages (KB3134758, KB3134759, and KB3134760) are different than previously released WMF 5.0 RTM packages (KB3094174, KB3094175, and KB3094176).
  • These packages have fixes only for the PSModulePath issue compared to the previously released WMF 5.0 RTM packages.
Uninstall previously released WMF 5.0 RTM packages

You must uninstall previously released WMF 5.0 RTM (KB3094174, KB3094175, and KB3094176) packages.

Source: https://blogs.msdn.microsoft.com/powershell/2016/02/24/windows-management-framework-wmf-5-0-rtm-packages-has-been-republished/

Pester – Validate home directory share permissions

In our company a user’s home directory is created through a script. Unfortunately, the script had been somewhat broken for at least a few months. Home directories created during that time have wrong permissions and need to be fixed.

With hundreds of home directories this needs to be scripted – so I thought why not use Pester for that.

Describing Tests ACL of [\\homedrive\home$\jsp\]
   Context jsp
    [+] ACL should contain only 3 ACEs 66ms
    [+] ACL should contain [mydomain\fileradmins] 14ms
    [+] ACL should contain [mydomain\Domain-Admins] 13ms
    [+] ACL should contain [mydomain\jsp] 14ms
Describing Tests ACL of [\\homedrive\home$\jsp\MyScans]
   Context jsp
    [+] [\\homedrive\home$\jsp\MyScans] should exist 83ms
    [+] ACL should contain only 4 ACEs 9ms
    [+] ACL should contain [mydomain\printerserviceaccount] 15ms
Tests completed in 219ms
Passed: 7 Failed: 0 Skipped: 0 Pending: 0





Describing Tests ACL of [\\homedrive\home$\pfisterer\]
   Context pfisterer
    [-] ACL should contain only 3 ACEs 65ms
      Expected: {3}
      But was:  {4}
      at line: 10 in \\homedrive\home$\megamorf\gitlab\HomedrivePermissions\HomedrivePermissions.Tests.ps1
      10:             $Acl.Access.Count | Should Be 3
    [-] ACL should contain [mydomain\fileradmins] 14ms
      Expected: {FullControl}
      But was:  {}
      at line: 18 in \\homedrive\home$\megamorf\gitlab\HomedrivePermissions\HomedrivePermissions.Tests.ps1
      18:             $ACE.FileSystemRights  | Should Be 'FullControl'
    [-] ACL should contain [mydomain\Domain-Admins] 13ms
      Expected: {FullControl}
      But was:  {}
      at line: 30 in \\homedrive\home$\megamorf\gitlab\HomedrivePermissions\HomedrivePermissions.Tests.ps1
      30:             $ACE.FileSystemRights  | Should Be 'FullControl'
    [-] ACL should contain [mydomain\pfisterer] 14ms
      Expected: {Modify, Synchronize}
      But was:  {}
      at line: 42 in \\homedrive\home$\megamorf\gitlab\HomedrivePermissions\HomedrivePermissions.Tests.ps1
      42:             $ACE.FileSystemRights  | Should Be 'Modify, Synchronize'
Describing Tests ACL of [\\homedrive\home$\pfisterer\MyScans]
   Context pfisterer
    [+] [\\homedrive\home$\pfisterer\MyScans] should exist 110ms
    [-] ACL should contain only 4 ACEs 10ms
      Expected: {4}
      But was:  {5}
      at line: 66 in \\homedrive\home$\megamorf\gitlab\HomedrivePermissions\HomedrivePermissions.Tests.ps1
      66:             $Acl.Access.Count | Should Be 4
    [+] ACL should contain [mydomain\printerserviceaccount] 16ms
Tests completed in 244ms
Passed: 2 Failed: 5 Skipped: 0 Pending: 0

By default our home directories have Access Control Entries (ACEs) for the domain admins and storage admins groups with full access and the respective user account with modify permissions. For our printers’ “scan to home directory”-feature we have to ensure that a folder called MyScans exists in the user’s home directory. That folder needs an additional ACE of the printing service account with write permissions.