Monday 16 November 2020

Introduction to Linux Networking

 

What is the difference between curl and wget?
curl vs Wget
How to download files in Linux from command line with dynamic url

to download a file when it is pointed by a dynamic url, all you need to to is to use single quotes for the url; -O = specify the output filename

$ wget 'http://some.site.com/download?id=234&status=download' -O output_filename


What to do if ping does not help? 

Options are:

  • wget
  • curl
  • traceroute
    • installation: $ sudo apt install traceroute
  • lft
    • "layer 4 traceroute"
    • $ lft -S 192.168.0.20


$ ping example.com
$ traceroute example.com
$ curl -v example.com
$ wget example.com


How to find what DNS servers are used on the local machine?

$ cat /etc/resolv.conf 
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "systemd-resolve --status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0
search whatever.example.com

To find out which DNS servers are used by each network adapter:

$ systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 137 (enxa44cc8e41d0f)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 136 (vethed1f504)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 38 (br-53b4f1b3fbda)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 5 (br-3c8c9487a095)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 4 (docker0)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 3 (wlp2s0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: x.y.z.q
                      x.y.z.v
          DNS Domain: ~.
                      whatever.example.com

Link 2 (enp0s31f6)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
lines 44-81/81 (END)

Another way:

$ nmcli dev show | grep 'IP4.DNS'
IP4.DNS[1]:                             x.y.z.q
IP4.DNS[2]:                             x.y.z.v

DNS on Ubuntu 18.04

How to block some domain?

Add entry e.g.

0.0.0.0 domain.to.be.blocked

to /etc/hosts and then flush DNS cache:

How To Flush Linux / UNIX DNS Cache - nixCraft

How to direct network traffic use Proxy server?

Use environment variables

export http_proxy=10.21.32.70:8080
export https_proxy=10.21.32.70:8080

Domain names can be used instead of IP addresses.

How To Use Proxy Server To Access Internet at Shell Prompt With http_proxy Variable - nixCraft

How to test proxy?

curl -v -x 11.22.33.44:8080 -L 'https://www.example.com/examples/1'

       -x, --proxy [protocol://]host[:port]
              Use the specified proxy.

              The  proxy string can be specified with a protocol:// prefix. No
              protocol specified or http:// will be treated as HTTP proxy. Use
              socks4://, socks4a://, socks5:// or socks5h:// to request a spe‐
              cific SOCKS version to be used.  (The protocol support was added
              in curl 7.21.7)

              HTTPS  proxy  support  via https:// protocol prefix was added in
              7.52.0 for OpenSSL, GnuTLS and NSS.

To let cURL follow redirects (3xx statuses) add -L:

       -L, --location
              (HTTP)  If  the server reports that the requested page has moved
              to a different location (indicated with a Location: header and a
              3XX  response code), this option will make curl redo the request
              on the new place.

To see HTTP status code, use verbose flag:
     -v, --verbose

How to get public IP address of the computer?


$ dig TXT +short o-o.myaddr.l.google.com @ns1.google.com | awk -F'"' '{ print $2}'

3rd party GeoIP service (its web API) can be accessed via curl:

$ curl -m 30 -X GET "https://api.ipify.org?format=json"
{"ip":"197.44.76.233"}

The output of this command (an IP address) can be assigned to a variable

AGENT_IP=$(curl -s -m 30 -X GET "https://api.ipify.org?format=json")
echo "This build agent '%teamcity.agent.name%' has public IP: $AGENT_IP"



No comments: