Continuous Deployment execute database scripts with TFS

In another post, we’ve learned how we can use a Build definition and Release Management in a CI/CD pipeline. Please go through the articles if you need more information on these topics.

In this post, I’ll explain how we can use Release Definition to execute database scripts as part of the continuous deployment strategy. We will add a task to execute PowerShell script to achieve this.

Create Release definition

The task to be added is PowerShell under Utility. This task should be added before the Application files are deployed in the CD pipeline.

Create a Release definition for your Release pipeline as shown below, the path for the PowerShell script should be a shared path and the database Server is accessible.

Add the below code snippet to the PowerShell script. The script should be modified as per the requirement.

$localScriptRoot = "C:\Scripts"
$Server = "dbserver"
$scripts = Get-ChildItem $localScriptRoot | Where-Object {$_.Extension -eq ".sql"} | Sort-Object -Property Name
$qt = 0
foreach ($s in $scripts)
    {
        Write-Host "Running Script : " $s.Name -BackgroundColor DarkGreen -ForegroundColor White
        $script = $s.FullName	
        Invoke-Sqlcmd -ServerInstance $Server -InputFile $script -username "TFSBuildUser" -password "****" -querytimeout $qt
    }
Write-Verbose "scripts executed successfully" -verbose

The scripts to be executed are present at the location C:\Scripts with extension “.sql”. The password can be also passed as an argument to the PowerShell script and added to the Release Definition secret variable.

Use the Triggers tab to link to the required Build definition of your Project so that the Release definition is integrated with the Build pipeline. The linking can also be done using the Artifacts tab.

You can setup multiple environments using this strategy like Dev, POC and Production.

Move files based on Date Time using PowerShell

This post shows how we can use PowerShell to move files from one folder to another say while Archiving the files.

$SrcFolder = "C:\Files"
$DestFolder = "C:\Backup\2017"
$count = 0

 if(-not (Test-Path $DestFolder)){
        New-Item -Path $DestFolder -ItemType Directory
 }
 else{
    Write-Host $DestFolder "Path exists"
 }

 Get-ChildItem -Path $SrcFolder | Where-Object {$_.LastWriteTime -lt (Get-Date).AddMonths(-13)} | ForEach-Object {
    
    Write-Host "Moving: " $SrcFolder"\"$_ $_.LastWriteTime to $DestFolder
    Move-Item -Path $SrcFolder"\"$_ -Destination $DestFolder
    $count = $count + 1
 }

 Write-Host "Moved" $count "items."

The above code takes each item found in the Source Folder and checks LastWriteTime say for 13 months earlier, and moves the file to the Destination folder.

The command used to Move the files is Move-Item which takes -Path and -Destination as parameters.

We have the following tasks that requires to be done on a Windows Server 2012 R2 for this example using PowerShell:

  1. Archive Files from a Source Folder to Destination Folder (could be a Shared folder on the Network) based on time-stamp, only for files older than 2 years in this example.
  2. It takes 2 parameters, Year parameter is mandatory. Destination Folder is optional. Both Source and Destination are however hard-coded into the script.
  3. The script creates a folder with Year as folder-name at the Destination and Archive all the files for that year in the created folder.
  4. A log file is created in the same path on which the script is running under the Logs folder. All Archiving activity including any errors faced for any files are logged.
  5. The progress of the Archiving Files is also shown in the Console.
  6. Errors are handled like no files or folder found for the Year Parameter.
  7. The user running the script has to be an Administrator on the Machine. The Script has to be running in the Admin Mode.
param(
      [Parameter(Mandatory=$True)]
      [int]$Year,      
      [Parameter()]
      [String]$DestinationFolder
      )


Add-Type -AssemblyName System.Windows.Forms

$SourceFolder = "C:\TestSource"
$Date=((Get-Date).AddYears(-2).Year)

if ($Year -gt $Date) 
     {
     [System.Windows.Forms.MessageBox]::Show("Only Process the documents before Year $Date")
     break
     }

if($DestinationFolder -eq "")
  {
  $DestinationFolder = "\\TestDestinationFolder"
  } 

If (-NOT ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator"))
{
[System.Windows.Forms.MessageBox]::Show("Please run powershell application as administator")
break
}

$i=0
$DirectoryChain=(Get-ChildItem -Path $SourceFolder -Recurse)
$DirCount=$DirectoryChain.Count
$time=Get-Date
$DirectoryChain |
ForEach-Object {

$FileYear = (Get-Date($_.LastWriteTime)).Year

      if($FileYear -eq $Year){

        ########Creating Log File############
        if(!(Test-Path -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt"))
           { 
           Write-Host "Starting Data Archiving Process for $Year" -BackgroundColor Green        
           New-Item -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt" -ItemType File -Value "Archival Process Started at $(Get-Date)
==================================================================================================================================
"          
            if(!(Test-Path -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt")){[System.Windows.Forms.MessageBox]::Show("Log File could not created and exiting script")
               break}
            }
        
         try {

           if(Test-Path -Path "$DestinationFolder\$FileYear")
               {  
               Move-Item -LiteralPath $_.FullName -Destination "$DestinationFolder\$FileYear" -ErrorAction Ignore
               "Moved '$_' to $DestinationFolder\$FileYear successfully at $(Get-Date)" | add-content ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt"
               $i++
               }
            else
               {
               New-Item -Path "$DestinationFolder\$FileYear" -ItemType Directory -ErrorAction Ignore
               Move-Item -LiteralPath $_.FullName -Destination "$DestinationFolder\$FileYear"
               "Moved '$_' to $DestinationFolder\$FileYear successfully at $(Get-Date)" | add-content ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt"
               $i++
               }

           $Remaining=$DirCount-$i
           [int]$PercentageStat=(($i/$DirCount)*100)
           
           Write-Progress -Activity "Archival Document Process" -Status "Archiving Document $_  $i" -PercentComplete $PercentageStat
           #Start-Sleep -Seconds 2


            }
         Catch
            {
             "Error in moving object $_.FullName" | add-content ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt"
            }
          }

}


######Checking for existence of log file and appending comment in log file"########
if(Test-Path -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt")
{"===================================================================================================================================
Archival Processs Finished at $(Get-date)" | add-content ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt"

Write-Host "Data Archiving Process Finished, Total files/folders Archived $i" -BackgroundColor Green

Write-Host "Archival Processs Finished at $(Get-date), Please check the log file Archive $(($time).ToString("MM-dd-yyyy HH-MM")).Txt on Location $(Get-Item -Path ".\")" -BackgroundColor Green -ForegroundColor Blue

If(!(Test-Path -Path ".\Logs"))
     {
      New-Item -Path .\ -ItemType Directory -Name "Logs"
      Move-Item -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt" -Destination ".\Logs"
      Write-Host "Log File .\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt has been moved to $(Get-Item -Path ".\Logs")" -BackgroundColor Green -ForegroundColor White 
     }
else{
      Move-Item -Path ".\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt" -Destination ".\Logs"
      Write-Host "Log File .\Archive $(($time).ToString("MM-dd-yyyy hh-mm tt")).Txt has been moved to $(Get-Item -Path ".\Logs")" -BackgroundColor Green -ForegroundColor White 
     }
}

else
{
Write-Host "No files or folder found for $Year" -BackgroundColor Green -ForegroundColor White
}

The script can be run in PowerShell Console Admin mode as below:

./Archive_files.ps1 -Year 2016

Optional Parameter $Destination Folder can be ignored as it is hard-coded in the script. You can pass it if the Destination folder is different.

Modify config files using PowerShell

Suppose you need to hide the password in your clear text connection strings at a particular back-up path or any other path in config files. This can be achieved using PowerShell with the following code.

$configFiles = Get-ChildItem "C:\Path\Backup" *.config -rec
foreach ($file in $configFiles)
{
    (Get-Content $file.PSPath) |
    Foreach-Object { $_ -replace "pwd=(\w+);", "pwd=****;" } |
    Set-Content $file.PSPath
}

Write-Verbose "Password changed!"

The above script will replace the string “pwd=(\w+);” with “pwd=****;” in the *.config files matching the regular expression.

I’m using the following parameters for the Get-ChildItem command:

-Path “C:\Path\Backup” 

-Include *.config for including config files

-rec for Recursion to get items in all child containers

Enable disable Windows Service and Scheduled Tasks with PowerShell

Often we may need to enable/disable Windows Services and Scheduled tasks on a Windows Server.

Below is the snippet to Enable Windows Service and Scheduled task with PowerShell:

$tasknames = "Test Task1","Test task2" $servicename = "Test Service1","Test Service2" foreach($task in $tasknames) {  ##disabling tasks on server  $taskstatus = Get-ScheduledTask -TaskName $task  if($taskstatus.State -eq "Disabled")   {    Enable-ScheduledTask -TaskName $task -Verbose   }   elseif($taskstatus.State -eq "Ready")   {   Write-Host "Task: $task already running"   }      } ##### Service checks and execution  foreach($service in $servicename)  {  $servicestatus = Get-Service $service  if($servicestatus.Status -eq "Stopped")   {    Start-Service $service  -Verbose    }    elseif($servicestatus.Status -eq "Started")    {    Write-host "Service: $service already Started"    }   } 

The below snippet shows how to disable the Scheduled tasks and Windows Service:

$tasknames = "Test Task1","Test task2" $servicename = "Test Service1","Test Service2" foreach($task in $tasknames) {  ##disabling tasks on server  $taskstatus = Get-ScheduledTask -TaskName $task  if($taskstatus.State -eq "Ready")   {    Disable-ScheduledTask -TaskName $task -Verbose   }   elseif($taskstatus.State -eq "Disabled")   {   Write-Host "Task: $task already disabled"   }      } ##### Service checks and execution  foreach($service in $servicename)  {  $servicestatus = Get-Service $service  if($servicestatus.Status -eq "Running")   {    Stop-Service $service -Force -Verbose    }    elseif($servicestatus.Status -eq "Stopped")    {    Write-host "Service: $service already stopped"    }   }  

You can save the above scripts in .ps1 format for PowerShell and call them e.g. using a .bat file.

Show system tray or balloon tip notification with PowerShell

We had a requirement where we needed to notify the user when a certain number of days were past, the user should be shown a System notification in the tray area. There are a few solutions you can work on to achieve this:

  1. Write a Windows forms (formless application) in VB.Net or C#. Run the application with Windows Service at scheduled intervals.
  2. Use vbscript with Task scheduler. Run at system startup and schedule.
  3. Write PowerShell script with Task scheduler. Run at system startup and schedule.

We’ll explore the 3rd option to use PowerShell script. I found a wonderful article on mcpmag where Mr. Boe Prox explains “Creating a Balloon Tip Notification Using PowerShell”. This article gave me the starting point of solving the problem by using this script. Do check it out!

Using the PowerShell ISE, in the editor, write the following function that’ll get the system uptime.

Function Get-Uptime {
$os = Get-WmiObject win32_operatingsystem -ErrorAction SilentlyContinue
$uptime = (Get-Date) – $os.ConvertToDateTime($os.LastBootUpTime)
#Write-Output (“Last boot: ” + $os.ConvertToDateTime($os.LastBootUpTime) )
#Write-Output (“Uptime : ” + $uptime.Days + ” Days ” + $uptime.Hours + ” Hours ” + $uptime.Minutes + ” Minutes” )
return $uptime.Days
}

The following method as taken from the article shared above, helps generate the balloon tip by including the System.Windows.Forms assembly into our PowerShell session before we can make use of the NotifyIcon class.

Function Invoke-BalloonTip {
[CmdletBinding()]
Param (
[Parameter(Mandatory=$True,HelpMessage=”The message text to display. Keep it short and simple.”)]
[string]$Message,

[Parameter(HelpMessage=”The message title”)]
[string]$Title=”Attention $env:username”,

[Parameter(HelpMessage=”The message type: Info,Error,Warning,None”)]
[System.Windows.Forms.ToolTipIcon]$MessageType=”Info”,

[Parameter(HelpMessage=”The path to a file to use its icon in the system tray”)]
[string]$SysTrayIconPath=’C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe’,

[Parameter(HelpMessage=”The number of milliseconds to display the message.”)]
[int]$Duration=5000
)

Add-Type -AssemblyName System.Windows.Forms

If (-NOT $global:balloon) {
$global:balloon = New-Object System.Windows.Forms.NotifyIcon

#Mouse double click on icon to dispose
[void](Register-ObjectEvent -InputObject $balloon -EventName MouseDoubleClick -SourceIdentifier IconClicked -Action {
#Perform cleanup actions on balloon tip
Write-Verbose ‘Disposing of balloon’
$global:balloon.dispose()
Unregister-Event -SourceIdentifier IconClicked
Remove-Job -Name IconClicked
Remove-Variable -Name balloon -Scope Global
})
}

#Need an icon for the tray
$path = Get-Process -id $pid | Select-Object -ExpandProperty Path

#Extract the icon from the file
$balloon.Icon = [System.Drawing.Icon]::ExtractAssociatedIcon($SysTrayIconPath)

#Can only use certain TipIcons: [System.Windows.Forms.ToolTipIcon] | Get-Member -Static -Type Property
$balloon.BalloonTipIcon = [System.Windows.Forms.ToolTipIcon]$MessageType
$balloon.BalloonTipText = $Message
$balloon.BalloonTipTitle = $Title
$balloon.Visible = $true

#Display the tip and specify in milliseconds on how long balloon will stay visible
$balloon.ShowBalloonTip($Duration)

Write-Verbose “Ending function”

}

Get the system uptime using the Get-UpTime function written above, and call the Invoke-BalloonTip to generate the System tray notification.

$lastrebootdays = Get-Uptime
#echo $lastrebootdays

if ($lastrebootdays -eq 0) {
$restartstr = ‘You last restarted your system ‘ + $lastrebootdays + ‘ days back.’
$userstr = ‘Attention ‘ + $env:username + ‘!’
Invoke-BalloonTip -Message $restartstr -Title $userstr -MessageType Info

$msgBoxInput = [System.Windows.Forms.MessageBox]::Show(‘Your last reboot happened ‘ + $lastrebootdays + ‘ days back. ‘ + ‘Would you like to restart your system?’,’Restart System!’,’YesNoCancel’,’Error’)

switch ($msgBoxInput) {
‘Yes’ {
[System.Windows.Forms.MessageBox]::Show(‘Restarted’)
}

‘No’ {
[System.Windows.Forms.MessageBox]::Show(‘Not Restarted’)
}

}
}

The above logic just shows a message box if Yes/No is selected by the user.