How to install Scrapy on Ubuntu

$ which python3
/usr/bin/python3

$ python3 --version
Python 3.6.9

$ virtualenv --python=python3.6 venv
Running virtualenv with interpreter /usr/bin/python3.6
Already using interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /home/bojan/dev/github/scrapy-demo/venv/bin/python3.6
Also creating executable in /home/bojan/dev/github/scrapy-demo/venv/bin/python
Installing setuptools, pip, wheel...
done.

$ source venv/bin/activate

(venv) $ pip install scrapy
Collecting scrapy
  Downloading Scrapy-2.1.0-py2.py3-none-any.whl (239 kB)
Collecting service-identity>=16.0.0
  Downloading service_identity-18.1.0-py2.py3-none-any.whl (11 kB)
Collecting cryptography>=2.0
  Downloading cryptography-2.9.2-cp35-abi3-manylinux2010_x86_64.whl (2.7 MB)
Collecting cssselect>=0.9.1
  Downloading cssselect-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting w3lib>=1.17.0
  Downloading w3lib-1.21.0-py2.py3-none-any.whl (20 kB)
Collecting pyOpenSSL>=16.2.0
  Downloading pyOpenSSL-19.1.0-py2.py3-none-any.whl (53 kB)
Collecting zope.interface>=4.1.3
  Downloading zope.interface-5.1.0-cp36-cp36m-manylinux2010_x86_64.whl (234 kB)
Collecting protego>=0.1.15
  Downloading Protego-0.1.16.tar.gz (3.2 MB)
Collecting parsel>=1.5.0
  Downloading parsel-1.5.2-py2.py3-none-any.whl (12 kB)
Collecting queuelib>=1.4.2
  Downloading queuelib-1.5.0-py2.py3-none-any.whl (13 kB)
Collecting lxml>=3.5.0
  Downloading lxml-4.5.0-cp36-cp36m-manylinux1_x86_64.whl (5.8 MB)
Collecting PyDispatcher>=2.0.5
  Downloading PyDispatcher-2.0.5.tar.gz (34 kB)
Collecting Twisted>=17.9.0
  Downloading Twisted-20.3.0-cp36-cp36m-manylinux1_x86_64.whl (3.1 MB)
Collecting pyasn1-modules
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting pyasn1
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=16.0.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting six>=1.4.1
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting cffi!=1.11.3,>=1.8
  Downloading cffi-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (399 kB)
Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (from zope.interface>=4.1.3->scrapy) (46.1.3)
Collecting incremental>=16.10.1
  Downloading incremental-17.5.0-py2.py3-none-any.whl (16 kB)
Collecting Automat>=0.3.0
  Downloading Automat-20.2.0-py2.py3-none-any.whl (31 kB)
Collecting hyperlink>=17.1.1
  Downloading hyperlink-19.0.0-py2.py3-none-any.whl (38 kB)
Collecting PyHamcrest!=1.10.0,>=1.9.0
  Downloading PyHamcrest-2.0.2-py3-none-any.whl (52 kB)
Collecting constantly>=15.1
  Downloading constantly-15.1.0-py2.py3-none-any.whl (7.9 kB)
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting idna>=2.5
  Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Building wheels for collected packages: protego, PyDispatcher
  Building wheel for protego (setup.py) ... done
  Created wheel for protego: filename=Protego-0.1.16-py3-none-any.whl size=7765 sha256=7696d4fa63732f3509349e932b23800b7dacc6034fc150eaa3f4448fa401c6aa
  Stored in directory: /home/bojan/.cache/pip/wheels/b2/74/25/517a0ec6186297704db56664268e72686f5cfa8ab398582f33
  Building wheel for PyDispatcher (setup.py) ... done
  Created wheel for PyDispatcher: filename=PyDispatcher-2.0.5-py3-none-any.whl size=11515 sha256=17b011eb905d7eccda1eaafb51833fbe7a1b8406fb4a7764e7216ef19fafd698
  Stored in directory: /home/bojan/.cache/pip/wheels/28/db/61/691c759da06ba9b86da079bdd17cb3e01828d49d5c152cb3af
Successfully built protego PyDispatcher
Installing collected packages: six, pycparser, cffi, cryptography, pyasn1, pyasn1-modules, attrs, service-identity, cssselect, w3lib, pyOpenSSL, zope.interface, protego, lxml, parsel, queuelib, PyDispatcher, incremental, Automat, idna, hyperlink, PyHamcrest, constantly, Twisted, scrapy
Successfully installed Automat-20.2.0 PyDispatcher-2.0.5 PyHamcrest-2.0.2 Twisted-20.3.0 attrs-19.3.0 cffi-1.14.0 constantly-15.1.0 cryptography-2.9.2 cssselect-1.1.0 hyperlink-19.0.0 idna-2.9 incremental-17.5.0 lxml-4.5.0 parsel-1.5.2 protego-0.1.16 pyOpenSSL-19.1.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 queuelib-1.5.0 scrapy-2.1.0 service-identity-18.1.0 six-1.14.0 w3lib-1.21.0 zope.interface-5.1.0

(venv) $ pip list --local
Package          Version
---------------- -------
attrs            19.3.0 
Automat          20.2.0 
cffi             1.14.0 
constantly       15.1.0 
cryptography     2.9.2  
cssselect        1.1.0  
hyperlink        19.0.0 
idna             2.9    
incremental      17.5.0 
lxml             4.5.0  
parsel           1.5.2  
pip              20.0.2 
Protego          0.1.16 
pyasn1           0.4.8  
pyasn1-modules   0.2.8  
pycparser        2.20   
PyDispatcher     2.0.5  
PyHamcrest       2.0.2  
pyOpenSSL        19.1.0 
queuelib         1.5.0  
Scrapy           2.1.0  
service-identity 18.1.0 
setuptools       46.1.3 
six              1.14.0 
Twisted          20.3.0 
w3lib            1.21.0 
wheel            0.34.2 
zope.interface   5.1.0  

(venv) $ pip freeze 
attrs==19.3.0
Automat==20.2.0
cffi==1.14.0
constantly==15.1.0
cryptography==2.9.2
cssselect==1.1.0
hyperlink==19.0.0
idna==2.9
incremental==17.5.0
lxml==4.5.0
parsel==1.5.2
Protego==0.1.16
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
PyDispatcher==2.0.5
PyHamcrest==2.0.2
pyOpenSSL==19.1.0
queuelib==1.5.0
Scrapy==2.1.0
service-identity==18.1.0
six==1.14.0
Twisted==20.3.0
w3lib==1.21.0
zope.interface==5.1.0


Verification:

(venv) $ scrapy
Scrapy 2.1.0 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

  [ more ]      More commands available when run from project directory

Use "scrapy <command> -h" to see more info about a command

(venv) $ scrapy bench
2020-04-25 23:57:01 [scrapy.utils.log] INFO: Scrapy 2.1.0 started (bot: scrapybot)
2020-04-25 23:57:01 [scrapy.utils.log] INFO: Versions: lxml 4.5.0.0, libxml2 2.9.10, cssselect 1.1.0, parsel 1.5.2, w3lib 1.21.0, Twisted 20.3.0, Python 3.6.9 (default, Apr 18 2020, 01:56:04) - [GCC 8.4.0], pyOpenSSL 19.1.0 (OpenSSL 1.1.1g  21 Apr 2020), cryptography 2.9.2, Platform Linux-4.15.0-96-generic-x86_64-with-Ubuntu-18.04-bionic
2020-04-25 23:57:01 [scrapy.crawler] INFO: Overridden settings:
{'CLOSESPIDER_TIMEOUT': 10, 'LOGSTATS_INTERVAL': 1, 'LOG_LEVEL': 'INFO'}
2020-04-25 23:57:01 [scrapy.extensions.telnet] INFO: Telnet Password: 6642b95b9fd73c04
2020-04-25 23:57:01 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.logstats.LogStats']
2020-04-25 23:57:01 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2020-04-25 23:57:01 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2020-04-25 23:57:01 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2020-04-25 23:57:01 [scrapy.core.engine] INFO: Spider opened
2020-04-25 23:57:01 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:01 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2020-04-25 23:57:03 [scrapy.extensions.logstats] INFO: Crawled 93 pages (at 5580 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:03 [scrapy.extensions.logstats] INFO: Crawled 157 pages (at 3840 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:04 [scrapy.extensions.logstats] INFO: Crawled 230 pages (at 4380 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:06 [scrapy.extensions.logstats] INFO: Crawled 301 pages (at 4260 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:07 [scrapy.extensions.logstats] INFO: Crawled 365 pages (at 3840 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:08 [scrapy.extensions.logstats] INFO: Crawled 422 pages (at 3420 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:09 [scrapy.extensions.logstats] INFO: Crawled 485 pages (at 3780 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:10 [scrapy.extensions.logstats] INFO: Crawled 541 pages (at 3360 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:11 [scrapy.extensions.logstats] INFO: Crawled 597 pages (at 3360 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:12 [scrapy.core.engine] INFO: Closing spider (closespider_timeout)
2020-04-25 23:57:12 [scrapy.extensions.logstats] INFO: Crawled 646 pages (at 2940 pages/min), scraped 0 items (at 0 items/min)
2020-04-25 23:57:12 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 301439,
 'downloader/request_count': 661,
 'downloader/request_method_count/GET': 661,
 'downloader/response_bytes': 2092067,
 'downloader/response_count': 661,
 'downloader/response_status_count/200': 661,
 'elapsed_time_seconds': 10.596581,
 'finish_reason': 'closespider_timeout',
 'finish_time': datetime.datetime(2020, 4, 25, 22, 57, 12, 551514),
 'log_count/INFO': 20,
 'memusage/max': 55128064,
 'memusage/startup': 55128064,
 'request_depth_max': 22,
 'response_received_count': 661,
 'scheduler/dequeued': 661,
 'scheduler/dequeued/memory': 661,
 'scheduler/enqueued': 13220,
 'scheduler/enqueued/memory': 13220,
 'start_time': datetime.datetime(2020, 4, 25, 22, 57, 1, 954933)}

2020-04-25 23:57:12 [scrapy.core.engine] INFO: Spider closed (closespider_timeout)

How to install Jupyter Notebook on Ubuntu 18.04

Just wanted to share my experience with installing Jupyter Notebook on my Ubuntu 18.04 box.

$ cd my-project

$ virtualenv venv
Using base prefix '/usr'
New python executable in /home/bojan/dev/my-project/venv/bin/python3
Also creating executable in /home/bojan/dev/my-project/venv/bin/python
Installing setuptools, pip, wheel...
done.

$ ls
README.md  venv

$ source venv/bin/activate

(venv) $ ls
README.md  venv

(venv) $ which pip
/home/bojan/dev/my-project/venv/bin/pip

(venv) $ which pip3
/home/bojan/dev/my-project/venv/bin/pip3

(venv) $ pip --version
pip 20.0.2 from /home/bojan/dev/my-project/venv/lib/python3.6/site-packages/pip (python 3.6)

(venv) $ pip3 --version
pip 20.0.2 from /home/bojan/dev/my-project/venv/lib/python3.6/site-packages/pip (python 3.6)

(venv) $ pip install jupyter
Collecting jupyter
  Using cached jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)
Collecting jupyter-console
  Downloading jupyter_console-6.1.0-py2.py3-none-any.whl (21 kB)
Collecting ipywidgets
  Using cached ipywidgets-7.5.1-py2.py3-none-any.whl (121 kB)
Collecting ipykernelbojan@bobox:~/dev/my-project models
  Downloading ipykernel-5.2.1-py3-none-any.whl (118 kB)
Collecting notebook
  Downloading notebook-6.0.3-py3-none-any.whl (9.7 MB)
Collecting nbconvert
  Using cached nbconvert-5.6.1-py2.py3-none-any.whl (455 kB)
Collecting qtconsole
  Downloading qtconsole-4.7.3-py2.py3-none-any.whl (117 kB)
Collecting ipython
  Downloading ipython-7.13.0-py3-none-any.whl (780 kB)
Collecting pygments
  Downloading Pygments-2.6.1-py3-none-any.whl (914 kB)
Collecting jupyter-client
  Downloading jupyter_client-6.1.3-py3-none-any.whl (106 kB)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Downloading prompt_toolkit-3.0.5-py3-none-any.whl (351 kB)
Collecting traitlets>=4.3.1
  Using cached traitlets-4.3.3-py2.py3-none-any.whl (75 kB)
Collecting nbformat>=4.2.0
  Downloading nbformat-5.0.6-py3-none-any.whl (170 kB)
Collecting widgetsnbextension~=3.5.0
  Using cached widgetsnbextension-3.5.1-py2.py3-none-any.whl (2.2 MB)
Collecting tornado>=4.2
  Downloading tornado-6.0.4.tar.gz (496 kB)
Processing /home/bojan/.cache/pip/wheels/1c/54/34/fd47cd9b308826cc4292b54449c1899a30251ef3b506bc91ea/prometheus_client-0.7.1-cp36-none-any.whl
Collecting pyzmq>=17
  Downloading pyzmq-19.0.0-cp36-cp36m-manylinux1_x86_64.whl (1.1 MB)
Collecting jinja2
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting Send2Trash
  Using cached Send2Trash-1.5.0-py3-none-any.whl (12 kB)
Collecting jupyter-core>=4.6.1
  Downloading jupyter_core-4.6.3-py2.py3-none-any.whl (83 kB)
Collecting ipython-genutils
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting terminado>=0.8.1
  Using cached terminado-0.8.3-py2.py3-none-any.whl (33 kB)
Collecting testpath
  Using cached testpath-0.4.4-py2.py3-none-any.whl (163 kB)
Collecting mistune<2,>=0.8.1
  Using cached mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting bleach
  Downloading bleach-3.1.4-py2.py3-none-any.whl (151 kB)
Collecting defusedxml
  Using cached defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Collecting entrypoints>=0.2.2
  Using cached entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Processing /home/bojan/.cache/pip/wheels/39/01/56/f1b08a6275acc59e846fa4c1e1b65dbc1919f20157d9e66c20/pandocfilters-1.4.2-cp36-none-any.whl
Collecting qtpy
  Downloading QtPy-1.9.0-py2.py3-none-any.whl (54 kB)
Requirement already satisfied: setuptools>=18.5 in ./venv/lib/python3.6/site-packages (from ipython->jupyter-console->jupyter) (46.1.3)
Collecting pexpect; sys_platform != "win32"
  Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting decorator
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Processing /home/bojan/.cache/pip/wheels/98/b0/dd/29e28ff615af3dda4c67cab719dd51357597eabff926976b45/backcall-0.1.0-cp36-none-any.whl
Collecting jedi>=0.10
  Downloading jedi-0.17.0-py2.py3-none-any.whl (1.1 MB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting python-dateutil>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting wcwidth
  Downloading wcwidth-0.1.9-py2.py3-none-any.whl (19 kB)
Collecting six
  Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting jsonschema!=2.5.0,>=2.4
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting MarkupSafe>=0.23
  Using cached MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl (27 kB)
Collecting ptyprocess; os_name != "nt"
  Using cached ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting parso>=0.7.0
  Downloading parso-0.7.0-py2.py3-none-any.whl (100 kB)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.16.0.tar.gz (108 kB)
Collecting importlib-metadata; python_version < "3.8"
  Downloading importlib_metadata-1.6.0-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting zipp>=0.5
  Downloading zipp-3.1.0-py3-none-any.whl (4.9 kB)
Building wheels for collected packages: tornado, pyrsistent
  Building wheel for tornado (setup.py) ... done
  Created wheel for tornado: filename=tornado-6.0.4-cp36-cp36m-linux_x86_64.whl size=427634 sha256=2889594a465ff46f8fd99501b165ee4f674855322f0fe1a1df215836d56e20a7
  Stored in directory: /home/bojan/.cache/pip/wheels/37/a7/db/2d592e44029ef817f3ef63ea991db34191cebaef087a96f505
  Building wheel for pyrsistent (setup.py) ... done
  Created wheel for pyrsistent: filename=pyrsistent-0.16.0-cp36-cp36m-linux_x86_64.whl size=97736 sha256=06c739603fe2717a15b5bb65e9e0d6cae6ca8de2f1d3ee4988966b6711ca9b89
  Stored in directory: /home/bojan/.cache/pip/wheels/d1/8a/1c/32ab9017418a2c64e4fbaf503c08648bed2f8eb311b869a464
Successfully built tornado pyrsistent
Installing collected packages: tornado, six, python-dateutil, pyzmq, decorator, ipython-genutils, traitlets, jupyter-core, jupyter-client, ptyprocess, pexpect, backcall, parso, jedi, wcwidth, prompt-toolkit, pygments, pickleshare, ipython, ipykernel, jupyter-console, pyrsistent, zipp, importlib-metadata, attrs, jsonschema, nbformat, prometheus-client, MarkupSafe, jinja2, testpath, mistune, webencodings, bleach, defusedxml, entrypoints, pandocfilters, nbconvert, Send2Trash, terminado, notebook, widgetsnbextension, ipywidgets, qtpy, qtconsole, jupyter
Successfully installed MarkupSafe-1.1.1 Send2Trash-1.5.0 attrs-19.3.0 backcall-0.1.0 bleach-3.1.4 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 importlib-metadata-1.6.0 ipykernel-5.2.1 ipython-7.13.0 ipython-genutils-0.2.0 ipywidgets-7.5.1 jedi-0.17.0 jinja2-2.11.2 jsonschema-3.2.0 jupyter-1.0.0 jupyter-client-6.1.3 jupyter-console-6.1.0 jupyter-core-4.6.3 mistune-0.8.4 nbconvert-5.6.1 nbformat-5.0.6 notebook-6.0.3 pandocfilters-1.4.2 parso-0.7.0 pexpect-4.8.0 pickleshare-0.7.5 prometheus-client-0.7.1 prompt-toolkit-3.0.5 ptyprocess-0.6.0 pygments-2.6.1 pyrsistent-0.16.0 python-dateutil-2.8.1 pyzmq-19.0.0 qtconsole-4.7.3 qtpy-1.9.0 six-1.14.0 terminado-0.8.3 testpath-0.4.4 tornado-6.0.4 traitlets-4.3.3 wcwidth-0.1.9 webencodings-0.5.1 widgetsnbextension-3.5.1 zipp-3.1.0

Verification

(venv) $ jupyter notebook

[I 01:18:31.669 NotebookApp] Serving notebooks from local directory: /home/bojan/dev/my-project

[I 01:18:31.669 NotebookApp] The Jupyter Notebook is running at:

[I 01:18:31.670 NotebookApp] http://localhost:8888/?token=c9c50e5b8874194d2aaf815a79e079dacdac8eace7691c60

[I 01:18:31.670 NotebookApp]  or http://127.0.0.1:8888/?token=c9c50e5b8874194d2aaf815a79e079dacdac8eace7691c60

[I 01:18:31.670 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

[C 01:18:31.674 NotebookApp] 

    

    To access the notebook, open this file in a browser:

        file:///home/bojan/.local/share/jupyter/runtime/nbserver-29213-open.html

    Or copy and paste one of these URLs:

        http://localhost:8888/?token=c9c50e5b8874144d2aaf815a79e079dacdac8eace7691c60

     or http://127.0.0.1:8888/?token=c9c50e5b8874494d2aaf815a79e079dacdac8eace7691c60

Opening in existing browser session.

References:

How To Set Up Jupyter Notebook with Python 3 on Ubuntu 18.04 | DigitalOcean

How to upgrade VirtualBox on Ubuntu 18.04

In one of my previous posts I demonstrated how to install VirtualBox and I installed its version 6.0. But in the mean time a new version (6.1) has been released and I want to upgrade it.

Upgrading to the newer minor version

I was upgrading version 6 from 6.0 to 6.1:

Old version: 6.0.4

New version: 6.1.6

I downloaded the most recent deb package but installation failed:

~/Downloads$ sudo dpkg -i virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb 
Selecting previously unselected package virtualbox-6.1.
dpkg: regarding virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb containing virtualbox-6.1:
 virtualbox-6.1 conflicts with virtualbox
  virtualbox-6.0 provides virtualbox and is present and installed.

dpkg: error processing archive virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb (--install):
 conflicting packages - not installing virtualbox-6.1
Errors were encountered while processing:
 virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb

I checked the name of my previous installation:

~/Downloads$ dpkg -l | grep virtualbox
ii  virtualbox-6.0                                              6.0.4-128413~Ubuntu~bionic                       amd64        Oracle VM VirtualBox

...and used it to uninstall existing version:

~/Downloads$ sudo dpkg -r virtualbox-6.0 
(Reading database ... 290543 files and directories currently installed.)
Removing virtualbox-6.0 (6.0.4-128413~Ubuntu~bionic) ...
Processing triggers for shared-mime-info (1.9-2) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...

After this, the new installation was successful:

~/Downloads$ sudo dpkg -i virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb 
(Reading database ... 289716 files and directories currently installed.)
Preparing to unpack virtualbox-6.1_6.1.6-137129_Ubuntu_bionic_amd64.deb ...
Unpacking virtualbox-6.1 (6.1.6-137129~Ubuntu~bionic) ...
Setting up virtualbox-6.1 (6.1.6-137129~Ubuntu~bionic) ...
addgroup: The group `vboxusers' already exists as a system group. Exiting.
Processing triggers for systemd (237-3ubuntu10.39) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for shared-mime-info (1.9-2) ...

Let's check the state of packages:

$ dpkg -l | grep virtualbox

rc  virtualbox-6.0                                              6.0.4-128413~Ubuntu~bionic                       amd64        Oracle VM VirtualBox

ii  virtualbox-6.1                                              6.1.6-137129~Ubuntu~bionic                       amd64        Oracle VM VirtualBox

From command line - What do the various dpkg flags like 'ii' 'rc' mean? - Ask Ubuntu:

First letter -> desired package state ("selection state"):

• i ... install
• r ... remove/deinstall
• p ... purge (remove including config files)

Second letter -> current package state:

• i ... installed
• c ... config-files (only the config files are installed)

ii = It should be installed and it is installed

rc = It's removed/uninstalled but it's configuration files are still there

Upgrading to the newer patch version

I was upgrading version 6.1 from 6.1.6 to 6.1.16:

Old version: 6.1.6

New version: 6.1.16

Upgrading to the newer patch version (3rd number in the version) does not require uninstalling the previous version:

$ dpkg -l | grep virtualbox

rc  virtualbox-6.0                                              6.0.4-128413~Ubuntu~bionic            amd64        Oracle VM VirtualBox

ii  virtualbox-6.1                                              6.1.6-137129~Ubuntu~bionic            amd64        Oracle VM VirtualBox

$ sudo dpkg -i virtualbox-6.1_6.1.16-140961_Ubuntu_bionic_amd64.deb 

(Reading database ... 342850 files and directories currently installed.)

Preparing to unpack virtualbox-6.1_6.1.16-140961_Ubuntu_bionic_amd64.deb ...

Unpacking virtualbox-6.1 (6.1.16-140961~Ubuntu~bionic) over (6.1.6-137129~Ubuntu~bionic) ...

Setting up virtualbox-6.1 (6.1.16-140961~Ubuntu~bionic) ...

addgroup: The group `vboxusers' already exists as a system group. Exiting.

Processing triggers for systemd (245.4-4ubuntu3.3) ...

Processing triggers for ureadahead (0.100.0-21) ...

Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...

Processing triggers for desktop-file-utils (0.24-1ubuntu3) ...

Processing triggers for mime-support (3.64ubuntu1) ...

Processing triggers for hicolor-icon-theme (0.17-2) ...

Processing triggers for shared-mime-info (1.15-1) ...

$ dpkg -l | grep virtualbox

rc  virtualbox-6.0                                              6.0.4-128413~Ubuntu~bionic            amd64        Oracle VM VirtualBox

ii  virtualbox-6.1                                              6.1.16-140961~Ubuntu~bionic           amd64        Oracle VM VirtualBox

Go language: When shall a method be a function?

I came across a method which was basically a helper method doing some data filtering. Its input data are struct's data members and the output is some collection used by another method. It didn't look right to me that this method does not use or modify receiver's state but is still bound to it. As it only takes some data from it and spits out a result, with no side effects, I decided to turn it into a function with parameters.

When deciding if Go method should actually be a function ask yourself: does it change or depends on the state of its receiver? If it doesn't, it should be a function. One immediate benefit is that it's easier to set up its unit tests.

Reddit user krocos summarized the rules in post When we should use a method receiver instead of a function argument and vice versa? : golang:

There are some rules already to use a receiver: 

• when you implement an interface 
• when the function mutate some state (encapsulated in the receiver) 
• when your function querying some state to work (i.e.: might calling the same method with the same arguments produce a different result) 
• when you want to chain method calls 

In all other cases it's necessary to use an argument.

From my old C++ days I remember the advice from Scott Meyers (from his Effective C++ book and also this article How Non-Member Functions Improve Encapsulation | Dr Dobb's):

• classes should contain no more member functions than are absolutely necessary
• non-friend non-member functions should be preferred to member functions

This can generally be applied to Go language as well. Ideally, interface members should be methods, they are struct's public API and should be set in stone. All other routines should be made functions if possible.

Further reading:

Go Best Practices: Should you use a method or a function?
When we should use a method receiver instead of a function argument and vice versa? : golang
When to use methods? : golang
How Non-Member Functions Improve Encapsulation | Dr Dobb's
Non-Member Functions in OOP - CodeProject