23 July 2015

Extract links froma web site and download multiple files with Poweshell Script

Few weeks ago, I've posted about how to download multiple files from web using powershell. In this post I'm going to show you a script that retrieves the links from a given web site and downloads the files to a given directory on your local machine.

#place to store downloaded files
$filepath = Read-Host "Please enter the location to store the files.(eg. C:\Downloads\)"
$webSite = Read-Host "Please enter the site url that contains the links: (eg. https://www.example.com/photos)"
#gathering link details from source web
$links = ((Invoke-WebRequest Uri $webSite).Links)
foreach($link in $links) {
#link to files
$item = $webSite + $link.href
Write-Host Url: $item
# innertext contains the file name
Write-Host Title:  $link.innerText
#download path on local machine
$path = $filepath + $link.innerText
Write-Host $path
#downloading files
Start-BitsTransfer $item  $path


08 July 2015

IIS error when installing SharePoint 2013 on Windows Server 2012 R2 : The tool was unable to install Application Server Role, Web Server (IIS) Role

This error has cost me five or six hours. Then I tried everything from applying patches, trying to install IIS manually, etc. Once I figured it out, I installed SharePoint Server and went with my life happily.
It happened to one of my colleague and I couldn't remember what fix I did a month ago. Later, I figured it out and solved. I'm documenting it here so that if anyone come across the same issue, they can solve it.

This error occurred because the pre-request installer is trying to execute ServerManagerCmd.exe from system 32 folder and failing. (I found out this from the pre-request installer logs). So I get in to the system32 folder to see what this fuss about. Voila! There was no ServerManagerCmd.exe in the folder, instead there was a ServerManager.exe. Looks like the executable file was renamed in WS 2012 R2.

So, I made duplicate copy of ServerManager.exe by copy and pasting in the same folder(system32), then renamed the copy as  ServerManagerCmd.exe. Ran the pre-request installer again and installed successfully.


Issue: The tool was unable to install Application Server Role, Web Server (IIS)

                Step 1: Navigate to system32 folder (C:\Windows\system32)
                Step 2: Copy ServerManager.exe and paste in the same location. (system32 folder)
                Step 3: Rename ServerManager-Copy.exe to ServerManagerCmd.exe
                Step 4: Run the pre-request installer

By the way, SharePoint 2013 with SP1 works fine without any workarounds. So, If you have Windows Server 2012 R2, it is recommended to install SharePoint 2013 with SP1.

24 April 2015

Download Multiple files from web using powershell

Today I've come across a task that requires 1000 of files to be downloaded from a web directory. Now I have to click the link, then when save as option shows up click again to save. That would require 2000+ mouse clicks. No way. So I took powershell, and made a script to do it for me.

First I saved the Url and file name is a csv file. Csv file looks like
Url, Title
http://example.com/example/SomeFile.jpg, SomeFile.jpg
http://example.com/example/AnotherFile.jpg, AnotherFile.jpg

Now how I get this? I just right click the page, go to source then copied all to my favorite text editor Notepad++. After that did some Find & Replace to remove the html elements. (replace

  • with "" ).

  • Anyway, here is the script

    #csv file to compare

    $csv = IMPORT-CSV C:\R\picts.csv
    $filepath = "C:\Users\janbalagan\Downloads\Picts\Collection\"
    foreach($item in $csv)
    $path = $filepath + $item.Title
    Write-Host "Path: " $path
     Write-Host $item.Title " being downloaded: "
    Invoke-WebRequest $item.Url -OutFile $path
    Write-Host "Download Completed" -BackgroundColor DarkRed

    In case if you are working with powershell 2.0 which comes with windows 7 and previous versions, you must use 
    (New-Object Net.WebClient).DownloadFile($item.Url, $path) instead of
    Invoke-WebRequest $item.Url -OutFile $path

    19 April 2015

    Compare and Update Excel spread sheet using powershell

    Windows Power-shell is the most useful tool to automate lot of our work. It comes installed with Windows OS. Last week I've got some task where I have to compare some data from a csv file to an excel spread sheet and update the relevant data. Excel sheet contained thousands of records which has to be compared with a csv file contained hundreds of records. I have got more than 20 sets of files to compare and update.
    As a programmer, I hate this kind of mundane tasks. Those are ultimately boring. So, I came up with a power-shell script to do the job for me. This script can be modified to compare and update between two or more excel spread sheets as well. Feel free to modify and use it.

    # excel file path
    $excel_file_path = "C:\R\All Sites for migration 4-16.xlsx";
    #csv file to compare
    $csv = IMPORT-CSV C:\R\16_4_2015_B.csv
    # date completed to update excel column
    $dateNow = "17/4"
    # migration status to update excel column
    $status = "TEST STATUS"
    # Instantiate the COM object
    $Excel = New-Object -ComObject Excel.Application
    # Opens the excel workbook
    $ExcelWorkBook = $Excel.Workbooks.Open($excel_file_path);
    # prepare the work sheet. Here 1 means the first worksheet
    # you can use the worksheet name as well. eg: Sheet 1
    $ExcelWorkSheet = $Excel.WorkSheets.item(1)
    Write-host "Reading Excel File" -NoNewline
    # row number. I set this as 2 because I'm goint to read from second row
    # as first row contains the heading
    $intRow = 2;
    # starting the loop
    #reading from cell. First parametrer $intRow is the row being read
    # second parameter 3 gets the third column from the spread sheet
    $readValue = $ExcelWorkSheet.Cells.Item($intRow,3).value2
    # traversing through csv file contents to compare with the selected row
    foreach($item in $csv)
     # if third column value matches with csv file Url
     if($item.Url -eq $readValue)
      # writing the match on screen
      Write-Host "Match:" $readValue "DB:" $item.DBSize -foregroundcolor "Magenta"
         # updating data to excel file columns 5,6 and 9
        $ExcelWorkSheet.Cells.Item($intRow,5) = $dateNow
        $ExcelWorkSheet.Cells.Item($intRow,6) = $status
        $ExcelWorkSheet.Cells.Item($intRow,9) = $item.DBSize
    # increment row to the next row
    $intRow ++
    }While ($ExcelWorkSheet.Cells.Item($intRow,1).value2 -ne $null) # loop untill there is no value in the first column
    # Saving the file
    # Closing Work book
    # Closing Excel
    # Releasing the com object
    Stop-Process -Name EXCEL -Force