Tuesday, December 4, 2018

Background Intelligent Transfer Service - BITS

BITS or Background Intelligent Transfer Service has been available for many years with Windows Servers, it is generally considered only as a background service for Web Servers, Application, etc., though it can be used by System Administrator, IT Administrators or DBAs too. As a Database Professional I find it very useful thus I would like to share a demo of this service. The purpose of BITS is simple, transfer files between source and destination with fault tolerance capabilities.
Making it easy to use, BITS cmdlets are available with PowerShell too. The purpose of BITS as written on Microsoft Documentation gives more insight on its capabilities:
Background Intelligent Transfer Service (BITS) is used by programmers and system administrators to download files from or upload files to HTTP web servers and SMB file shares. BITS will take the cost of the transfer into consideration, as well as the network usage so that the user's foreground work has as little impact as possible. BITS also handles network interruptions, pausing and automatically resuming transfers, even after a reboot. BITS includes PowerShell cmdlets for creating and managing transfers as well as the BitsAdmin command-line utility.
Here is quick guide on using BITS with PowerShell:

Importing Module in PowerShell
  • This will be a good practice to run PowerShell with elevated rights (Right Click and Run as administrator) or run in same security context each time, otherwise there may be inconsistencies as some of the resources may not be available.
  • Once Powershell Opens, execute Import-Module BitsTransfer
  • You should not get any message or prompt if Import was successful
  • This demonstration has been tested with PowerShell Version 4 and 5. You may verify the version by executing $psversiontable.psversion from PS prompt.
Pre-Requisites before starting transfer
  • Obviously you would need to know the source and destination locations and file to be transfer. Also make sure both locations are accessible from PS prompt.
  • BITS provides two types of transfer modes, Synchronous and Asynchronous.
    • In Asynchronous mode BITS job will be registered and pointer will be returned to prompt, job will continue in background by service.
    • In Synchronous mode, BITS job will be registered and pointer will not be returned to prompt unless transfer completes.
  • The only major limitation with BITS is that it can only be run if user’s session is active, i.e. BITS commands cannot be scheduled by schedulers such as SQL Agent, Task Scheduler, etc., with only exception that user can disconnect the session without logging off from the Windows Session and the process will continue. If this restriction is removed then this will become my favorite file transfer utility.
  • To start\initiate transfer: start-BitsTransfer –Source [sourcepath] –Destination [DestinationLocation] –Asynchronous [If not specified then it becomes Synchronous]
  • To pause transfer: suspend-BitsTransfer 
  • To resume transfer: resume-BitsTransfer
  • To modify existing transfer: set-BitsTransfer
  • To get existing transfer jobs: get-BitsTransfer
  • To stop\terminate transfer: remove-BitsTransfer
  • To complete the transfer job: complete-BitsTransfer
  • To demonstrate, I will transfer a 800 GB file using BITS as shown in the image below and will use some of the commonly used parameters as well as will get more useful details out of the job.

  • The source and destination are “C:\demo\source\SSMS-Setup-ENU.exe” and “\\swarns\demo\destination\” respectively

Starting Transfer in Asynchronous mode
  • import-module bitstransfer
  • start-bitsTransfer -Source "C:\demo\source\SSMS-Setup-ENU.exe" –Destination “\\swarns\demo\destination” –Asynchronous
  • The image above shows the job has been registered and current job status as Connecting

Suspending the job
  • I have intentionally suspended this job so that I can read properties of this job before copy finishes, to do that the cmdlet is suspend-bitstransfer
  • To suspend you need to provide jobid as a parameter in two different ways, first is using PIPE and other is using variable. I am going to use PIPE and first need jobid
    • get-bitstransfer | select jobid
    • get-BitsTransfer -jobid 1c6eaea9-efb5-4e1c-bae9-64b79b5d8433 | suspend-BitsTransfer

Reading BITS properties in easily readable format
  • I have intentionally suspended this job so that I can read properties of this job before copy finishes, to do that the cmdlet is suspend-bitstransfer
  • get-BitsTransfer -jobid 1c6eaea9-efb5-4e1c-bae9-64b79b5d8433 | select *
  • The image above shows basic output of command and this can be formatted to get more usable
  • I have used the following scriptIf you want to test the following script, please write the following scriptlet in single line, I have intentionally divided the scriptlet at each Pipe to make it readable
    • Get-BitsTransfer -jobid  1c6eaea9-efb5-4e1c-bae9-64b79b5d8433 
    • | SELECT *,  @{name='Xferred_percent';expr={$_.BytesTransferred/$_.BytesTotal*100}}, @{name='Minutes'; expr={((Get-Date)-$_.creationtime).TotalMinutes}}
    • |SELECT @{name='BytesPerMinute';expr={$_.BytesTransferred/$_.Minutes}},  @{name='EtaInMinutes';expr={$_.Minutes/$_.BytesTransferred*($_.BytesTotal-$_.BytesTransferred)}}, *
    • | SELECT @{name='CompletionEstimate';expr={(Get-Date).AddMinutes($_.EtaInMinutes)}}, Xferred_percent, Minutes, BytesPerMinute, JobState | Format-Table
  • The script above may sound little bit complex, but let me simplify it below:
    • A PIPE  “|” is used to pass the value to next cmdlet or expression, to define an expression, a hashtable “@{}” is used with name and expr tags. Calculations can be performed in expr tag. Next PIPE can access the expr by refering to name defined in hashtable.
Last Step
  • To resume the suspended job, the cmdlet will be get-BitsTransfer -jobid 1c6eaea9-efb5-4e1c-bae9-64b79b5d8433  | resume-BitsTransfer –Asynchronous #Optional
  • Once transfer completes, if you started job with Asynchronous file transfer, this would require you to manually complete the job by issuing get-BitsTransfer -jobid 1c6eaea9-efb5-4e1c-bae9-64b79b5d8433  | remove-BitsTransfer

This service is robust, runs very well, I never got into any issues after transferring bunch of files. I have used robocopy too for many transfers, but could not read transfer statistics asynchronously. At the same time the inability to use BITS if user did not login, reduces its usage for many. I wish if there was no such restriction. Thanks for reading this article, please leave your comments if you find it useful.

No comments:

Post a Comment

Using Azure Storage for SQL Backups

you may have taken backups on local storage, file servers, NAS, etc. Now a days it is becoming crucial you have backup copies available in...

Contact Me


Email *

Message *