What wget is

Configurare noua (How To)

Situatie

wget is a command-line tool to download files from the web using HTTP, HTTPS, and FTP. It works non-interactively (can run in background), supports recursion, resuming, and rate-limiting.

Solutie

Basic usage

  • Download a single file:
    wget https://example.com/file.zip
  • Save with a different filename:
    wget -O myfile.zip https://example.com/file.zip
  • Download in background (log to file):
    wget -b https://example.com/largefile.iso
  • Resume an interrupted download:
    wget -c https://example.com/largefile.iso

Downloading multiple files

  • From a list in a file (one URL per line):
    wget -i urls.txt

Recursive download (mirror a site or directory)

  • Basic mirror (downloads site, follows links, converts links for local viewing):
    wget --mirror -p --convert-links -P localdir https://example.com/
    • –mirror = -r -N -l inf –no-remove-listing
    • -p downloads all files necessary to display pages (images, CSS)
    • –convert-links adjusts links for local browsing
    • -P localdir saves into localdir

Authentication and headers

  • HTTP basic auth:
    wget --user=username --password=secret https://example.com/protected/file
  • Send a custom header (e.g., API key):
    wget --header="Authorization: Bearer TOKEN" https://api.example.com/data.json

Limit speed, retries, and timeout

  • Limit download rate:
    wget --limit-rate=200k https://example.com/file.iso
  • Set number of retries (default 20) and timeout:
    wget --tries=10 --timeout=30 https://example.com/file

FTP, proxies, and passive mode

  • Use FTP URL:
    wget ftp://ftp.example.com/pub/file.tar.gz
  • Force passive FTP:
    wget --ftp-user=user --ftp-password=pass --passive-ftp ftp://ftp.example.com/file

Common useful flags (summary)

  • -O : write output to file
  • -c : continue/resume
  • -b : background
  • -q : quiet
  • -nv : non-verbose
  • -r : recursive
  • -l : recursion depth
  • -np : no parent (don’t ascend to parent directories)
  • –limit-rate=RATE : throttle speed
  • –header=”Header: value” : custom header

Examples:

  • Download a webpage and its resources for offline viewing:
wget –mirror -p –convert-links -P site_copy https://example.com
  • Download 10 files listed in urls.txt concurrently (using xargs for parallelism):
    cat urls.txt | xargs -n1 -P10 wget -q

Tips:

  • Check wget version: wget --version
  • Use -q for scripting (quiet), and check exit status ($?) for success
  • wget is simpler for straightforward downloads and mirroring.

Tip solutie

Permanent

Voteaza

(7 din 13 persoane apreciaza acest articol)

Despre Autor

Leave A Comment?