24 April 2015

Download Multiple files from web using powershell

Today I've come across a task that requires 1000 of files to be downloaded from a web directory. Now I have to click the link, then when save as option shows up click again to save. That would require 2000+ mouse clicks. No way. So I took powershell, and made a script to do it for me.

First I saved the Url and file name is a csv file. Csv file looks like
Url, Title
http://example.com/example/SomeFile.jpg, SomeFile.jpg
http://example.com/example/AnotherFile.jpg, AnotherFile.jpg

Now how I get this? I just right click the page, go to source then copied all to my favorite text editor Notepad++. After that did some Find & Replace to remove the html elements. (replace


  • with "" ).


  • Anyway, here is the script

     
    #csv file to compare

    $csv = IMPORT-CSV C:\R\picts.csv
    $filepath = "C:\Users\janbalagan\Downloads\Picts\Collection\"
    foreach($item in $csv)
    {
    $path = $filepath + $item.Title
    Write-Host "Path: " $path
     Write-Host $item.Title " being downloaded: "
    Invoke-WebRequest $item.Url -OutFile $path
    }
    Write-Host "Download Completed" -BackgroundColor DarkRed


    In case if you are working with powershell 2.0 which comes with windows 7 and previous versions, you must use 
    (New-Object Net.WebClient).DownloadFile($item.Url, $path) instead of
    Invoke-WebRequest $item.Url -OutFile $path